A Markerless-based Gait Analysis and Visualization Approach for ASD Children

This study proposed a new method in gait acquisition and analysis for autistic children based on the markerless technique versus the gold standard marker-based technique. Here, the gait acquisition stage is conducted using a depth camera with a customizable skeleton tracking function that is the Microsoft Kinect sensor for recording the walking gait trials of the 23 children with autism spectrum disorder (ASD) and 30 typically healthy developing (TD) children. Next, the Kinect depth sensor outputs information is translated into kinematic gait features. Further, analysis and evaluation are done specifically the kinematic angles of the hip, knee, and ankle in analyzing and visualizing the pattern of the plots versus the kinematic plots acquired from the marker-based that is the Vicon motion system gait technique. In addition, these kinematic angles are also validated using the statistical method namely the Analysis of Variance (ANOVA). Results showed that the ρ-values are insignificant for all angles upon computing both the intragroup and inter-group normalization. Hence, these findings have proven that the proposed markerless-based gait technique is indeed apt to be used as a new alternative markerless method for gait analysis of ASD children. Keywords—Autism spectrum disorder (ASD); kinematic; marker-based; markerless-based; gait analysis


I. INTRODUCTION
Autism Spectrum Disorder (ASD) is a development disorder that can be characterized by several difficulties such as in the learning process, communication, as well as social skills [1]. Note that the deficiency can be seen in early childhood, for instance as early as two years old [2] & [3]. Recently, there has been a growth in the literature regarding unbalanced and sensory disturbances, especially during walking for the ASD children [4], [5] & [6]. Due to the sensory conflicts, researchers face challenges in performing gait assessments to further provide suitable care and treatment [7]. On top of that, ASD children are found to face more challenges in receiving both the treatment and care as compared to other children with developmental delays, especially the requirement for these children to be examined in an unfamiliar testing environment [7], [8] & [9]. Here, gait analysis involving ASD children is indeed vital. This is because reliable gait information will help determine the level of deficit specificity as the evaluation tools [5].

II. RELATED WORK
There are two categories of gait techniques namely the marker-based technique and markerless-based technique. The marker-based technique includes an optoelectronic system with synchronization between the camera(s) and the system. This system works by tracking the movement through an optical sensor (infrared camera) by identifying the position of the markers attached to the body or object followed by postprocessing via specialized software [10]. This technique also offered an accurate three-dimensional (3D) model and is normally performed inside the laboratory [10] & [11]. Moreover, the position of the markers must properly adhere according to the system's template to produce accurate gait data and analysis [12]. For example, Hasan et al. [13] has placed 35 retro-reflective markers on the ASD and typically healthy developing (TD) subject's body to analyze the gait pattern from two groups whilst, Eggleston et al. [14] used 19 retroreflective markers to perform analysis of gait symmetry specifically at the lower body during the over-ground walking for the ASD children.
On the other hand, researchers attempt to overcome the limitation from the marker-based techniques with marker-free or markerless gait techniques and thus giving greater freedom of movement to the users [15]. This technique offers a low-cost system as compared to the marker-based technique [16]. The analysis for marker-less technique is based on the movement analysis specifically the subject's silhouette in contrast to the background [17]. Thus, markerless requires post-processing analysis to enhance the appearance of the silhouette or to translate the movements into a three-dimensional (3D) coordinate point that can be mapped by the computer [17] & [18]. For example, Vilensky et al. [19] used two high-speed cameras, with one camera was placed perpendicularly with the walking track while the other camera was facing the track to measure gait disturbance between the normal and autistic children at the hip, knee, and ankle joints. Meanwhile, Zakaria et al. [20] performed ASD gait classification based on the calculated distance between joints captured by the depth camera.
In another study, Al-Jubouri et al. [21] classify full-body movement of the ASD and normal children based on the fourstage approaches which are augmentation, feature extraction, dimension reduction, and rough set. Based on previous 426 | P a g e www.ijacsa.thesai.org researches, it was found that there are not many studies conducted related to the gait features using the markerlessbased technique as reported in [22] & [23]. On top of that, there is a need to validate and evaluate the markerless-based features to provide an accurate analysis and evaluation method [24] that further can be used as the evaluation tools for ASD [5]. Earlier, the validity of the Kinect sensor was compared with the Vicon system as reported by Abiddin et al. [25]. The validity has been measured for the assessment of postural control based on several different types of movements. Here, selected body points from Kinect were imposed with the marker trajectories from Vicon to measure the outcome of anatomical landmark displacements or changes in angle relative to the ground [26]. Based on the Pearson correlation, the r-values obtained were greater than 0.90 for the majority of measurements as reported in [26]. In a different study, the intra-class (ICC) and Pearson analysis (r-value) of the vertical displacement of the knee marker for Parkinson Disease (PD) patients obtained using the Vicon system and Kinect sensor were compared and it was found that both techniques achieved ICC and r greater than 0.9 for walking on the same spot [27]. On the other hand, Cocchi et al. [28] have estimated the sagittal joints kinematics of children with cerebral palsy as well. In this study, the validity of the marker-less kinematic features was verified by comparing the markerless gait features with the marker-based gait features and results showed that the differences were heavily affected by the presence of the offset attained at the mean values of the joint kinematics. However, upon removing the angular offsets, the Pearson correlation specifically the r-values obtained were between 0.8 and 1, which indicated a good result [28]. To the extent of our knowledge and based on findings from previous studies, there is no formal study that has been done related to evaluation and validation for ASD using the markerless gait feature. Therefore, this study aims to evaluate and validate the kinematic gait features of both ASD and TD groups using a viable method like the Kinect sensor and the gold standard Vicon gait system. The use of the markerless-based gait technique can be of benefit to ASD children since most of these children often confront physical challenges such as lack of focus and anxiety during the laboratory gait trials acquisition and experiment.

III. METHODOLOGY
This section elaborated in detail the method used and the overall process methodology in this study. The experiment was conducted in the Human Motion and Gait Analysis (HMGA) Premier Laboratory of Universiti Teknologi MARA (UiTM), Shah Alam, Selangor. In this study, data acquisition consists of 23 ASD and 30 TD children. Prior to this study, parents were given an information sheet that needs to be completed in the Consent Form based on recommendation and approval by UiTM Shah Alam Ethics Committee.
The mean and standard deviation (SD) of the subjects is tabulated as in Table I For the laboratory layout, the HMGA Laboratory has an 8.3-meter wooden walkway with two force plates embedded at the center of the walkway as depicted in Fig. 1. Two digital video cameras which are installed on tripods were placed perpendicularly to one another for recording the walking trials from the side and front views as the subject walk along the walkway. In addition, all the infrared (IR) cameras of the Vicon System were wall-mounted approximately 2.5 meters above the floor level to ensure that the capability of the volume area to be captured at the center of the lab based on the length, width, and height of 4 meters, 3 meters, and 2 meters respectively. Meanwhile, the Kinect sensor is placed on a stand located 0.5 meters in height from the floor and is facing the walking direction, with a working range between 1.2 and 3.5 meters view from the camera at 57° horizontal wide and a vertical view of 43°. For this laboratory setup, the walking direction started at the point "A" and finished at the point marked as "B".

A. Data Acquisition
For the data acquisition stage, a depth camera namely the Microsoft Kinect camera is used as a motion-sensing device for the markerless-based gait technique. The camera is equipped with a depth sensor, Red-Green-Blue (RGB) camera, microphones, and a tilt motor to provide full-body motion sensing for 20 primary body landmarks or coordinates. In addition, this device works at a frequency of 30 Hz. The IR depth sensor gives the Kinect its depth measuring capabilities. The emitter emits infrared light beams and the depth sensor reads the IR beams reflected to the sensor. The reflected beams are converted into depth information measuring the distance between the body and the sensor within its capture volume with a resolution of 640×480 to acquire the skeletal data. On the other hand, Vicon Motion System is used as a motionsensing for the marker-based gait technique and also as the gold standard to establish the accuracy of the Kinect. Refer to Fig. 1, the Vicon system is equipped with eight optical cameras labeled as C1 -C8 (MX T-Series camera), wall-mounted, and operated at 100Hz. This Vicon system is also integrated with two force plates, a host computer is installed with the Vicon Nexus 1.8.5 software, and two digital video cameras namely DV1 and DV2.

B. Procedure
For the Vicon system, the subject's preparation was done before the experiment started. A total of 35 retro-reflective markers were attached to the subject's body based on the plugin-gait full-body marker set that further produced the skeleton output or stick-figure model. Here, subjects need to perform a T-pose stand with the complete sets of markers so that the system can recognized and able to create the model based on the markers as in Fig. 2(a) [29]. On the other hand, a skeleton function was used to create the model from the Kinect device as shown in Fig. 2(b) [30].
Further, each subject performed walking trials while being concurrently monitored and captured using both the Kinect sensor and Vicon system. Refer to Fig. 2(c), for the walking task, subjects were required to walk freely barefooted with their comfortable normal speed to ensure their walking behavior is natural, without any assistant or walking aid. The walking task was carried out and repeated several times until all the ten successful trials per subject were completed. A successful trial is defined once the subject can successfully walk from point A to point B as in Fig. 1, without stopping or pausing during the walking trial or assisted by the caretakers. During the experimental, a walking trial is captured and saved simultaneously by both Kinect and Vicon systems.

C. Data Analysis
Data from the systems were screened and timesynchronized visually before data extraction. At this stage, a single gait cycle for the same walking trial session was acquired to represent each subject's walking pattern from both systems. For the data acquired via the Kinect sensor, the angles were derived using the cosine rule. As for the Vicon system, the output angles of the gait were extracted. Further, the kinematic angles from both systems were computed and analyzed. In order to analyze, evaluate and validate the walking gait among these subjects, the interpolation technique is used to standardize the number of frames that is fixed at 30 frames for all trials and further is expressed as the percentage of the gait cycle. Additionally, two types of normalization techniques are implemented before the evaluation of the gait data. Firstly, the intra-group normalization, x intra was calculated to eliminate the influence of the subject's gait patterns which are different for each person and are expressed as in Equation (1) [31]. On the other hand, the inter-group normalization, x inter was computed in minimizing the effect of gait patterns among the group as given in Equation (2) Further, the kinematic plots for the hip, knee, and ankle angles from both Kinect and Vicon systems are visualized based on the plot pattern. The F-statistic was used to assess the quality of variances by computing the ratio of variation between the sample means and variation within the sample means. Then, the Analysis of Variance (ANOVA) test set as the significant value, ρ is less than 0.05 (ρ < 0.05) with the null hypothesis as the mean values of the kinematic plots are similar for both systems was used to test any mean differences in all angles. Here, the mean differences indicated that the plot is not similar in terms of its form or magnitude.

IV. EXPERIMENTAL RESULTS AND DISCUSSION
This section presents the results of the evaluation and validation of the gait data from both the gait techniques utilized in comparing the kinematic plots of the markerless-based gait technique that is from the Kinect sensor and the marker-based gait technique which is from the Vicon system. Fig. 3 showed the kinematic plots of the hip, knee, and ankle angles for the original, intra-group, and inter-group normalization gait data for both the ASD and TD children. The green color line represents the markerless method specifically the Kinect sensor while the magenta color line is for the Vicon system that is the marker-based technique. Further, the ANOVA results for all the kinematic plots of Fig. 3 are tabulated in Table II. Firstly, the plots based on the original gait data similar shapes are visualized for the knee angles of the walking gait generated from both techniques in the ASD as well as the TD group. As for the plots for the hip angle, the plots are also similar however the amplitude values between the two gait techniques differ for both groups. The same goes for the ankle angle plots which are not similar using both techniques for the ASD and TD groups. Based on the original gait data, the differences in the knee angles and ankle angles of the kinematic plots are supported by the statistical results computed as tabulated in Table II. For the ASD group, the ANOVA value for F is 577.55 with ρ equals to 0.00 for the hip angles whilst the TD group with F=375.09. As for the ankle angles, the computed F values are 84.450 and 81.70 respectively for both the ASD and TD groups. These values resembled significant differences for the knee and ankle angles using the markerless and marker-based techniques for both the ASD and TD groups and matched results of the visualized plots. This showed that the hip and ankle angles owned large group means relative to the within-group variability as compared to the knee angle. Next, for the knee angles with F=1.443 and ρ is equal to 0.23 for the ASD group and with F=1.26 and ρ=0.23 which is larger than 0.05 resembled that there is no significant difference between both techniques for the ASD and TD groups. These values supported and proved that the visualization of both plots also showed similar patterns. Recall that this study is to prove that there are no significant differences amongst the kinematic features with the ρ-values must be greater than 0.05 since similar plots showed that the proposed markerless-based technique is similar to the gold standard which is the marker-based gait technique utilized.
Furthermore, the intra-group and inter-group normalization were computed to analyze and eliminate data redundancy thus maintains the same information that adheres to a common standard approach for datasets. All three angles of the hip, knee, and ankle of the kinematic plots are as plotted in Fig. 3 for both intra-group and inter-group normalization. As explained earlier, for the hip angle the magnitude gaps differ, however upon normalization, the differences between the magnitudes are significantly reduced especially towards the end of the gait cycle in both the ASD and TD groups. Next, the plots revealed that the knee flexion during the swing phase (60% to 100% of the gait cycle) has improved. However, there is a difference in flexion and extension that existed during initial contact (0% of the gait cycle) between the ASD and TD groups for the knee angle using the proposed markerless-based approach. As for the ankle angle plots, the normalization process failed to enhance or improve the plots. The only improvement based on the visualization of these plots is the similar shapes of the ankle dorsiflexion and ankle plantarflexion during the stance phase that is between 0% to 60% of the gait cycle and towards the end of the gait cycle. The ANOVA results for the intra-group and inter-group normalization showed that the mean of the hip, knee, and ankle angles for both gait techniques is similar for both groups, which means that there are no significant differences in the mean for the respective angles. The F-statistic revealed a small ratio for the hip, knee and ankle angles with F=0.00 respectively. Once again, the small F-value indicates that the group means are similar with minimal variability. Also, the ρvalue that evaluated the mean differences showed that there were no significant mean differences for the hip, knee, and ankle angles since the ρ-value was greater than 0.05.  In conclusion, an evaluation and validation of gait data from the Vicon system versus the proposed markerless-based model are analyzed. Kinematic features namely the hip, knee, and ankle angles are evaluated based on visualization of the angles plots as further verified based on ANOVA. From the plots, the hip, and ankle angles showed differences between the two gait techniques for both ASD and TD groups. Further, the ANOVA test was performed to measure the significance of these angles between the marker-based and the proposed markerless approach.
The ANOVA results showed that there are significant differences for both the hip and ankle angles and these findings are in accordance based on the visualization of these plots. Further, the intra-group and inter-group normalizations are performed to minimize the data redundancy as well as maintaining similar information of the kinematic plots for both the marker-based and the proposed markerless technique. Upon normalization, the kinematic plots for the hip, knee, and ankle angles of both techniques showed similar shapes. With the ρ equals 1, hence these confirmed that there are no significant differences between these plots of gait features upon normalization. This showed that the proposed markerless-based gait technique is indeed suitable and potentially can be used as gait analysis for ASD children. The proposed markerless method is easy to set up, non-intrusive and portable as well. The next stage of work will include the use of the proposed markerless-based technique for Parkinson's disease (PD) or cerebral palsy (CP). This new proposed markerless gait approach could further assist in developing more suitable gait analysis intervention programs for autistic children and offer great research opportunities related to pathological gait.