Tracking Axonal Transports in Time-Lapse Images Obtained from a Microfluidic Culture Platform

In this paper, a procedure is described for tracking moving object trajectories from image sequences acquired from a microfluidic culture platform. Since particles move along the axons, curve structures need to be detected first from the input image sequence. A kymograph analysis technique is applied to detect axon structures from the consolidated image of the input sequence. Horizontally and vertically oriented axons are then detected by applying the process twice to the original and the 90degree rotated image. Multiple kymographs are generated along the detected axons by projecting image intensity variation through the time-axis. The trajectory detection process is then applied to each kymograph image. To obtain the particle motion information from the entire image sequence, an integration process is applied to each horizontal and vertical kymograph data set. The proposed technique has been applied to image sequences in the present application area. It is demonstrated that practical results can be obtained using time-lapse image sequence data. Keywords—Axonal transports; kymograph; trajectory detection; image sequence analysis; motion parameter extraction


I. INTRODUCTION
Time-lapse video of cellular motion in microfluidic culture platform consists of microscopic images acquired at fixed time intervals [1], [2], [3]. In fluorescence microscopic images, it is important to trace cellular motions appearing as a collection of irregular movements of small particles. While most organelles occupy stationary positions through time, some particles exhibit motion. The purpose of the analysis of mitochondrial transports is to identify motion trajectories and to detect the amounts of motions and speeds of particles. Tracking mitochondria in neural time-lapse video is a basic processing step in diverse biological research [4], [5].
For tracing various intracellular objects in cell imaging, multi-target tracking methods [6], [7] have been exploited previously. However, in the present application domain, tracking-based methods have not been successful for tracing mitochondria for several reasons. First, while all objects are moving in most other cell image sequences, there are many stationary objects in our neural image sequences. Over half of the particles are stationary, and some target objects move at low speeds. Frequent merging and splitting of stationary and moving particles, and sudden starting and stopping of moving targets make it hard to trace individual objects. Second, in the typical input image, the size of a moving target is very small. A target occupies only a few pixels in images. In addition, most particles appear as small dots with similar shapes and brightness. Thus, it is hard to identify individual particle based on shape and brightness information only.
An important characteristic of moving targets is that they are moving only through axons. Thus, once axons are traced first in images, the motion of the target object can be traced by locating curvilinear trajectories. While input video is a collection of images, a kymograph is constructed by combining temporal variation of image intensities on a selected axon. A kymograph is a time-space plot illustrating the intensity changes along an axon as a function of time. It is much easier to trace the curve on a 2D kymograph than finding and tracking small dots on 3D image sequences. Thus, many previous research works on mitochondria tracking have utilized kymographs for analysis. Techniques using image correlation [8] and Hough transform [9] were proposed for the analysis of axonal transports.
An automated kymograph analysis was proposed for tracking secretory granules [10]. As kymograph analysis obtains wider acceptance, automated analysis techniques have been proposed recently [11], [12], [13], [14]. However, since the performance of automated techniques usually depends on the characteristics of input images, the application of these automated techniques to other application domains have been somewhat limited. Recently neural net-based machine learning techniques have been applied to biomedical application domains as well [15], [16], [17], [18]. U-Net architecture has been successful in this application area [15]. An internet-based kymograph analysis tool [19] has been proposed using U-Net architecture.
In this paper, an integrated procedure is described for tracking moving objects trajectories from image sequences acquired from a microfluidic culture platform. The proposed approach is based on a kymograph analysis. Since the particles move along the axons in this application area, axon structures need to be detected first from the input image sequence. This process has been typically performed using curve trace techniques [20], [10]. In our approach, we apply kymograph analysis technique to detect axon structures. Kymograph is an image on 2D time-space domain. While the input is a 2D image defined on ( , ) plane, by regarding the vertical direction as the time axis, vertically oriented axons can be detected by a kymograph analysis process.
Similarly, horizontally oriented axons can be detected by applying the kymograph analyzer after rotating the image by 90 degree. Once axons are detected from input image sequences, multiple kymographs are generated along the detected axons. www.ijacsa.thesai.org Using multiple kymographs generated from the image sequence, a trajectory detection process is applied to kymograph data set. Finally, an integration process is applied to each horizontal and vertical kymograph data set to obtain the particle motion information from the image sequence. We have applied the proposed technique to image sequences in our application area. Experimental results will be presented using time-lapse image sequence data.

II. KYMOGRAPH ANALYSIS
In biomedical image applications, tracking particles on complex trajectories has been one of the basic processing tasks [4], [6], [7]. In practical applications, low image quality usually makes it impractical to track and analyze particle movements directly in images. While some methodologies have been proposed to detect particle movements directly on images, applications to other domains have been limited.
In time-lapse image sequences obtained from microfluidic culture platforms, particle motions usually arise on axons only, which remain as stationary curves in images. It is often unnecessary to track particles on 3D space. A kymograph is a 2D image depicting the temporal variations of image intensities along an axon curve. Since it is much easier to trace motions of target objects in kymographs than in video frames, kymographs have been utilized as intermediate target images for object tracking. A number of methods have been proposed for enabling kymograph analysis, and some methods have provided software packages for public access [12], [13], [14]. Most of previous methods have somewhat limited applicability, depending on the application domains and program usability.
As deep learning techniques have become successful in image recognition and segmentation areas, machine learning approaches have been adopted to biomedical applications. Currently, U-Net architecture has been the most successful for biomedical image analysis [15]. A U-Net based architecture, KymoButler [19] has been proposed for kymograph analysis. This architecture has a public-accessible implementation, providing the analysis results from a kymograph image supplied through internet.

A. Kymograph Detection
In microfluidic image sequences, object particles appear as scattered dots on each image frame. Experimental microfluidic image sequences consist of 100 images, taken at fixed time intervals. The purpose of the analysis is to identify moving trajectories and detect lengths and speeds of such motions.  Fig. 1-A depicts the first image of a 100-frame image sequence. At each image, small dark dots are object particles, and the analysis is performed to locate trajectories of moving particles. It can be seen that axon structures are not revealed on the first image. Since a moving particle occupies different positions at different frames, when the image sequence is summed through time, trajectories of moving particles tend to become continuous curves. A kymograph is constructed by projecting the image sequence to either ( , ) or ( , ) plane. In order to derive axons, a consolidated image is composed using the entire image sequence, by taking the minimum value in all image frames at each pixel. Here, since moving mitochondria appear on different locations, axons tend to become apparent. Fig. 1-B shows the consolidated image for Fig. 1-A, and it can be seen that each motion trajectory appears as a continuous curve.
Since there are many moving particles on each image sequence, a kymograph is generated separately for each axon curve. From the consolidated image in Fig. 1, it can be seen that several continuous axons have become apparent. After an axon detection algorithm is applied to the consolidated image, eleven continuous axons are detected, as shown in Fig. 1-C as the red or blue curves. A kymograph analysis tool [19] has been applied for the axon detection. Like other kymograph analysis tools, this tool tries to detect a continuous curve along the vertical (time) axis. We apply the tool twice, once to the original image, and once to the 90° rotated image. In Fig. 1-C, the red and the blue curves represent the axons detected by the two trials applied to the original and the rotated images.
In general, there may be many moving particles on each axon. In order to detect all moving particles on the image sequence, each axon needs to be examined separately.  Fig. 1-D was generated by composing image intensities along the specified axon. Each horizontal scanline on the kymograph depicts the image intensity on the axon at each time step. Since the orientations of the red and the blue triangle axons are near horizontal or vertical, Fig. 1-D and E have been obtained by projecting the image sequence to the horizontal and vertical directions. On kymograph, a vertical, near-straight line represents the image of a stationary particle, and a slanted curve shows a trajectory of a moving particle. To detect the direction and the speed of moving particle, it is necessary to track slanted curves. Two figures in Fig. 1-E show the trajectories obtained from images in Fig. 1-D, detected by the kymograph analysis. After applying the kymograph analysis, each trajectory is classified into stationary and moving curves, as described in Section III-C.

B. Analysis Procedure
The analysis procedure is depicted in Fig. 2. After the consolidated image is constructed from the image sequence, continuous axons are detected using the kymograph analysis process. Axons are then classified into horizontal and vertical groups to decide the direction of the kymograph projection. Depending on the axon orientation, kymograph projection is performed either on ( , ) or ( , ) plane. The number of detected kymographs varies on each axon image. Each kymograph is then analyzed separately, and consolidated trajectories are composed by combining trajectories from all kymographs.
A kymograph image is analyzed using a kymograph analysis tool [19], which can be accessed by supplying each kymograph image through the internet. The result of the analysis is given by a set of points ( , , , ), = 1, … , , = 1, … , , where K and denote the number of trajectories and the number of points on the k-th trajectory, respectively.

C. Detection of Moving Trajectories
As can be seen in Fig. 1-E, detected trajectories consist of stationary and moving curves. Since the purpose of the kymograph analysis is to extract motion information of moving particles, stationary trajectories are not examined and they need to be removed in a preliminary stage. First, the motion deviation of a trajectory is defined as the difference between the maximum and the minimum horizontal positions, i.e. ∆ or ∆ . The local speed at each location can be approximated as ≈ ∆ ∆ , where ∆ denote the difference of positions between neighboring points. Both the motion deviation and the local speed can be computed easily using the trajectory values.
Using the motion deviation and the local speed, the differences of moving and stationary trajectories can be defined as follows.

 Stationary trajectories have small motion deviations:
|∆ | < or |∆ | < , where is a small value (such as 5 pixels)  The local speed of a point on stationary trajectories is small: | | < , where is a small speed value.

D. Integration of Multiple Trajectories
Each kymograph depicts particle motions on a single axon. To find the motion trajectories on the entire image sequence, the results from all kymographs need to be integrated. For instance, there have been 11 axons on Fig. 1-C. Since there are kymographs projected into horizontal and vertical directions, the trajectory integration is performed twice through the horizontal and the vertical directions. The integration of the detected trajectories along two orientations is performed as follows.
 Perform trajectory analysis for each kymograph.
 Integrate trajectories from kymographs along the horizontal and the vertical directions separately.
From the integrated trajectory information, it is straightforward to derive the motion information including the number of moving particles at each time, the speed of each particle, the variation of speeds, etc.

IV. EXPERIMENTAL RESULTS
In this research, experiments have been performed using a set of real image sequences, acquired from a microfluidic culture platform using a confocal microscopy. Each experimental video consists of one-hundred 256x256 images, acquired at fixed time intervals. The purpose of the analysis is to detect the trajectories of moving particles. From the trajectories, motion parameters can be computed including the number of moving particles, the length of motion, the widths of motions, and so on.
Each video was analyzed through the procedure depicted in Fig. 2. A consolidated image was composed from the video to reveal the mitochondria trajectories. Since axons have structures similar to kymographs, we have applied the kymograph analysis software available through the internet, to detect axon structures. An example of detected axon structures is illustrated in Fig. 1-C. Since several axons are usually present in a single consolidated image, it is necessary to generate a separate kymograph for each axon. Moving trajectories are detected from each kymograph. The whole motion information is obtained by combining the motion information obtained from the kymograph analysis applied to each separate axon.

A. Accuracy Evaluation of the Trajectory Detection
In order to measure the accuracy of the kymograph analysis technique in this application area, a set of test images were analyzed manually, and ground truth trajectory data were prepared. The performance of the trajectory detection was quantified using the ground truth particle position data prepared manually by tracing the trajectory lines on each kymograph. The detected particle locations were compared with the ground truth positions. A detected point was classified as a true positive match, if there is a corresponding point on the ground data. Since a small amount of deviation may arise in point location, a point on the same time position located within 3-pixel horizontal distance was classified as a matched correspondence. A detected point with no ground correspondence is classified as a false match.
Ground truth data were prepared for four representative kymographs obtained from four different image sequences. Since the complexity of the input image sequences tend to be similar on different image sequences, similar levels of the analysis performance have been observed on other sequence images.
Ground truth and detected trajectories are shown in Fig. 3. Since only motion information is utilized in this research, only moving trajectories are denoted in the ground truth images. expressed as time-position pair. Notice that each trajectory curve looks like a collection of discrete points rather than a continuous curve, since the trajectory consists of a separate point representing each time-position location. The detected points can be seen at the bottom row of Fig. 3.
The accuracy of the trajectory detection is measured using two parameters, Recall and Precision, defined as follows. Recall and Precision for the test kymographs are shown on Table I. Since Recall is approximately above 85%, it can be seen that most moving particles are detected correctly.
The results of the kymograph analysis are a sequence of time-position data pair. In Fig. 3, the stationary and the moving trajectories are denoted using different colors. This classification was carried out using the motion detection rules described in Section III-C, that motion trajectories have narrow widths in stationary curves. It can be seen that Precision is lower than Recall. The reason for this phenomenon can be observed by comparing ground truths and detection results in Fig. 3, where it can be seen that some segments are mixtures of stationary and moving parts, while only moving parts are marked in the ground truth data. Since the purpose of the analysis is to detect the moving parts, the value of Recall is more important than that of Precision.

B. Integration of the Detected Kymograph Trajectories
To find the motion trajectories from the entire image sequence, the results from all kymographs detected from multiple axons are integrated. There have been 11 axons on Fig. 1-C. Fig. 4 and 5 depict the detected trajectories from the horizontally (red) and the vertically oriented (blue) axons in Fig. 1-C. Here, moving and stationary trajectories are denoted using the red and the green colors, respectively. It can be seen that moving trajectories have been detected correctly. From the detected trajectories, motion parameters including the amount of movement and the speed can be computed.

V. CONCLUSION
An integrated procedure has been described for tracking moving object trajectories from image sequences acquired from a microfluidic culture platform. The proposed approach is based on the kymograph analysis. Since particles move along the axons in this application, the axon structures need to be detected first from the input image sequence. We apply a kymograph analysis technique to detect axon structures from the consolidated image of the input image sequence. While the input is a 2D planar image, by regarding the vertical direction as the time axis, vertically oriented axons can be detected by a kymograph analysis process. Similarly, horizontally oriented axons can be detected by applying the kymograph analyzer after rotating the image by 90 degree. Once axons are found from input image sequences, multiple kymographs are generated along the detected axons.
A trajectory detection process is then applied to each kymograph data set. To obtain the particle motion information from the entire image sequence, the integration process is applied to each horizontal and vertical kymograph data set. The proposed technique is applied to image sequences in our application area. It has been demonstrated that practical results can be obtained using time-lapse image sequence data, where the detection accuracy is comparable to other kymographic analysis.