A Deep Neural Network Study of the ABIDE Repository on Autism Spectrum Classification

The objective of this study is to implement deep neural network (DNN) models to classify autism spectrum disorder (ASD) patients and typically developing (TD) participants. The experimental design utilizes functional connectivity features extracted from resting-state functional magnetic resonance imaging (rs-fMRI) originating in the multisite repository Autism Brain Imaging Data Exchange (ABIDE) over a significant set of training samples. Our methodology and results have two main parts. First, we build DNN models using the TensorFlow framework in python to classify ASD from TD. Here we acquired an accuracy of 75.27%. This is significantly higher than any known accuracy (71.98%) using the same data. We also obtained a recall of 74% and a precision of 78.37%. In summary, and based on our literature review, this study demonstrated that our DNN (128-64) model achieves the highest accuracy, recall, and precision on the ABIDE dataset to date. Second, using the same ABIDE data, we implemented an identical experimental design with four distinct hidden layer configuration DNN models each preprocessed using four different industry accepted strategies. These results aided in identifying the preprocessing technique with the highest accuracy, recall, and precision: the Configurable Pipeline for the Analysis of Connectomes (CPAC). Keywords—DNN; ASD; rs-fMRI; ABIDE; CPAC


I. INTRODUCTION
The complexity of the human brain is staggering. It is made up of hundreds of billions of neurons with trillions of connections making the brain neural function very complicated. Before the development of neuroimaging methods, the only way to understand the workings of brain neural function was to examine individual brains that had been damaged by a stroke, infection, or injury. Early discoveries about the localization of the neural function in the brain were made through these initial studies. However, due to many practical difficulties, the research on neurocognition of the brain has been limited [1].
To better understand the relationship between neural function and brain processes researchers began looking for a way to image its function. With the rapid growth of functional magnetic resonance imaging (fMRI), modern cognitive neuroscientists now have an imaging tool which overcomes the limitations of earlier neurocognition studies.
The initial development of the fMRI technique was driven by researchers interested in the brain's response to external mental stimuli. As a result, most initial research has focused on responses to external task-evoked activity. In 1995, Biswal et al. [2] found that correlations in resting state activity can also provide meaningful insight into the neural function even without external events or mental stimuli. Since then, the subsequent studies on brain function using resting-state fMRI (rs-fMRI) data has exploded. For example, recent studies have shown that rs-fMRI has become an essential technique to analyze the brain's spontaneous activity and intrinsic functional connectivity [3].
Autism spectrum disorder (ASD) is a brain disorder which is characterized by the impaired development of social interactions and communication skills. Recent epidemiological studies have shown that the prevalence of ASD has increased dramatically over the past few decades. Early diagnosis of ASD is essential in increasing the probability of providing early intervention. In turn, early intervention could provide a suitable treatment plan and aid in the later rehabilitation of ASD patients [4].
Although genetic and environmental factors are suspected, the exact etiology of ASD remains unknown. Generally, ASD patients are diagnosed using symptom-based clinical criteria requiring a significant amount of behavioral assessments. Current practice guidelines include structured observations of the child's behavior; extensive parental interviews; testing of cognition, speech and language, hearing, vision, and motor function; a physical examination; the collection of medical and family histories, etc. [5]. Numerous studies suggest that social and communicative impairments are the core symptoms of ASD. However, the neuropathology of these symptoms is still unestablished. Further research in this area can provide useful information to better understand neuronal pathology in patients with ASD.
With the rise of neuroimaging researchers are now using fMRI data to analyze ASD. Many studies suggest that the social and communicative impairments are associated with functioning and connectivity of cortical networks [6][7][8]. In neuroimaging, researchers often use multi-voxel pattern analysis (MVPA) to investigate how a pattern of brain activity is related to different cognitive states [9][10][11]. For the fMRI data analysis, machine learning classifiers are promising methods to perform MVPA. A growing number of studies has shown that machine learning classifiers can be used to extract useful information from neuroimaging data [12]. www.ijacsa.thesai.org A significant number of these machine learning studies use traditional algorithms for classification such as support vector machines (SVMs), decision tree, naïve Bayes, and others. However, recent research in deep learning methods shows that in the case of high-dimensional datasets such as fMRI data, deep learning models are much more efficient than traditional machine learning methods [13][14][15]. Recently, deep neural networks (DNN) have been successfully applied to both voxel-based classification [16] and functional connectivitybased classification [17,18]. Nonetheless, many challenges still lie in the application of DNN to fMRI data classification. For example, an important prerequisite for deep learning is to provide a significant number of training samples [19]. In most fMRI data analysis, the number of training samples is limited to several hundred. To address this challenge, the Autism Brain Imaging Data Exchange (ABIDE) initiative has aggregated functional and structural brain imaging data collected from laboratories around the world. From this valuable resource we have downloaded over one thousand rs-fMRI samples for our study.
The goal of this paper is twofold: First, build deep-learning models in TensorFlow to classify ASD patients and typically developing (TD) participants using the rs-fMRI data from a large multisite data repository ABIDE. Second, identify the most promising preprocessing pipeline in ABIDE for these models.

A. The ABIDE Dataset
The ABIDE (Autism Brain Imaging Data Exchange) repository consists of 1112 datasets, including 539 individuals with ASD and 573 typically developing (TD) controls. These 1112 datasets are composed of structural and resting state fMRI data along with the corresponding phenotypic information. In accordance with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) guidelines, over 1000 functional connection group projects, and the instrument neutral distributed interface (INDI) control protocol, all data sets have been completely anonymized (i.e., do not contain protected health information). More details about the dataset are available at: http://fcon_1000.projects.nitrc.org/indi/abide/.
From these 1112 subjects, 1035 subjects are screened as qualified candidates for our study since these subjects have complete phenotypic information. In these 1035 subjects, there are 505 ASD and 530 TD subjects, of which 157 females and 878 males. The summary information of the screened 1035 subjects is displayed in Table I. Table I contains the summary phenotypical information of the ASD and TD such as gender, age, and lab site name.

B. Preprocessing
Datasets from ABIDE were preprocessed by using four different preprocessing pipelines: Connectome Computation System (CCS), Configurable Pipeline for the Analysis of Connectomes (CPAC), Data Processing Assistant for Resting-State fMRI (DPARSF), and the Neuroimaging Analysis Kit (NIAK). Table II provides an overview of the different preprocessing steps for the above four pipelines.

C. Region of Interests
A common method for analyzing fMRI data involves extracting signals from a specified region of interests (ROIs). These ROIs can be used to examine activity within a set of voxels that are functionally coherent [20]. From preprocessed blood oxygenation level dependent (BOLD) images for each subject, the mean time-series were extracted from a ROI based atlas. In our previous studies [21], we drew our ROIs from seven different brain atlases: Automated Anatomical Labeling (AAL), Eickhoff-Zilles (EZ), Harvard-Oxford (HO), Talaraich and Tournoux (TT), Dosenbach 160, Craddock 200 (CC200), and Craddock 400 (CC400). By applying traditional classifiers such as SVM, logistic, and Ridge regression (used to quantify the overfitting of data through measuring the magnitude of coefficients), the classification results show that CC400 is the most promising atlas since it achieved the highest accuracy, recall, and precision. Therefore, in this study all the ROIs are extracted from the CC400 atlas.
For the CC400 atlas, functional parcellation was accomplished using a two-state spatially constrained functional procedure applied to preprocessed resting-state data. Labels were generated for each of the resulting ROIs from their overlap with AAL, EZ, HO, and TT atlases using the cluster naming script distributed with the pyClusterROI toolbox. More details about the atlas are available at: http://preprocessed-connectomes-project.org/abide/.

D. Feature Extraction from rs-fMRI data
Resting-state fMRI research focuses on measuring the correlation between spontaneous activation patterns of brain regions. During a resting-state experiment, subjects are instructed to relax and think of nothing while the spontaneous brain activity level was measured throughout the experiment. In 1995, Biswal et al. revealed a groundbreaking insight in their studies that the left and right hemispheric motor cortex shown a high correlation between their fMRI BOLD timeseries at the resting-state [22]. Subsequent studies confirmed this groundbreaking result, which shows not only the high level functional connectivity between the left and right hemispheric motor networks but also between regions of other known functional networks (e.g., the primary visual network and auditory network) [23][24][25].All these studies demonstrate that the resting-state brain network shows highly correlated, spontaneous activity between these regions. Our brain is a network consisting of different regions, each having independent tasks and functions, which are intimately interconnected functionally and structurally. Functional communication between brain regions plays a key role in cognitive processes making the functional connectivity of the human brain very important. Recent advances in functional neuroimaging have provided new tools to measure and explore functional interactions between brain regions thus accelerating research of the brain's functional connectivity. In the past decades, an increasing number of neuroimaging studies has begun to investigate functional connectivity by measuring the co-activation level of rs-fMRI time-series between anatomically separated brain regions [26]. These studies bridge the gap in exploring how functional connectivity relates to human behavior and how they may be altered by neurological disease [27][28][29].
In this study, functional connectivity extracted from the BOLD time-series signal means was used to classify ASD and TD subjects. From these time series we can calculate the full connectivity matrices using Pearson correlation of pairwise brain regions. Each connectivity feature (or entry) in the connectivity matrix is a Pearson correlation coefficient. The Pearson correlation coefficient is an indicator of the correlation between two brain regions. The coefficients range from -1 to 1. A coefficient value close to -1 indicates that the brain region is inversely correlated; a coefficient value close to 1 indicates that the brain region is highly correlated. Additionally, the Pearson correlation connectivity matrix is symmetric (i.e., the corresponding upper triangular and lower triangular entry values agree). Therefore, only the upper triangle values in the correlation matrix are used as features. Furthermore, the main diagonal of the connectivity matrix was also removed since these entries represent an area of selfcorrelation. Finally, we flattened the strictly upper triangle values to a vector of features. To calculate ROI-based functional connectivity we used the CC400 brain atlas, consisting of 400 functional regions of interest, which resulted in 77028 pairwise independent connectivity features for each subject. This set of 77028 features for each subject was used as our input layer in Section II-F.

E. Classification
In the past decades, a growing number of studies have shown that machine learning (ML) classifiers can be used for the analysis of fMRI data. For the classification of ASD and TD, most studies applied the supervised learning method of support vector machines (SVM). Generally, the studies with a small number of subjects from a single data site could achieve a high classification accuracy up to 97% [30]. However, the classification accuracy drops significantly when larger number of subjects from a multisite are studied. For example, only 60% accuracy was obtained with 964 subjects from 16 separate international sites in [31].
In our previous study [21], we applied the classical machine learning classifiers such as SVM, logistic regression, and Ridge regression to classify ASD and TD. To obtain a better classification accuracy, we used a grid search method to find the optimal parameters for each classifier. By using the optimal parameters, the best classification accuracy we achieved is 71.98% utilizing a Ridge classifier. Based on our literature review, 71.98% was the highest classifications to date involving 1035 subjects from 17 international sites. In addition to the overall classification accuracy, we also obtained satisfactory results for recall and precision.
Most known machine learning studies use traditional algorithms for classification: SVMs, decision tree, logistic regression, and so on. Recently, many deep neural networks (DNNs) have been effectively applied to identify ASD using fMRI. Research on deep learning methods shows that for high-dimensional data sets, such as fMRI data are more efficient than traditional machine learning models. In this study we apply DNN to classify ASD patients and TD participants using functional connectivity features and (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 4, 2020 4 | P a g e www.ijacsa.thesai.org improve the current highest classification accuracy achieved in [4,21]. Our experimental design is summarized in Fig. 1.

F. Deep Neural Networks
Deep neural networks have been successfully applied to both voxel-based classification and functional connectivitybased classification. Using the ABIDE repository to obtain sufficiently many samples for our DNN study, we applied a multilayer perceptron (MLP) with four different configurations, all of which are listed in Table III. We also attempted other configurations with more than two hidden layers (e.g., three and four hidden layers) but the experimental results decreased due to the lack of training samples. Additional layers also increased the experimental cycle due to limitations in our current hardware. Nonetheless, the experimental results were still better than the known non-DNN methods. This suggests the potential in applying further layering to our DNNs when we gain access to larger datasets and/or experimental platforms with more robust hardware in future studies.
The MLP with 77028-1024-512-2 is illustrated in Fig. 2. The MLP accepts an input space of 77028 features (our pairwise independent connectivity features for each subject derived in Section II-D) and an output space of 2 numbers. Between the input and output layers, the network has two hidden layers with 1024 and 512 units, respectively.
From Fig. 2, we can see that the MLP contains a significant number of weights. The objective of supervised training is to adjust the weights to output the expected classes and minimize the prediction error. The loss function in (1) is used to measure the prediction error that the model produces. The output layer contains two output units where each unit represents the probability of an input comes from an ASD or a TD subject (e.g., the output probability of being ASD is 89%, and of being TD is 11%). The output probability is obtained by applying a softmax function.  This study involves a binary classification task. Thus, the loss function we used here is binary cross entropy. All training processes for this study are implemented under the deep learning framework TensorFlow together with the optimization algorithm AdamOptimizer. In order to reduce the overfitting, we also add a 2 regularization term in the loss function. In (1), is the total number of samples ( = 1025), is the total number of labels ( = 2), is the total number of hidden layers of the DNN ( = 2), is the total number of the units in hidden layer 1, and +1 is the total number of the units in hidden layer 2. (1)

III. EXPERIMENTAL RESULTS
We implemented DNN with four different hidden layer configurations in the TensorFlow framework and we successfully classified ASD from TD using 1035 subjects from the ABIDE repository. To identify the most promising preprocessing pipeline in ABIDE, we applied the same experimental methodology for all four datasets which were preprocessed by four different pipelines (see Table II in Section II-C). The five-cross validation of accuracy, recall, and precision results for all four pipelines are reported in Tables IV, V, VI and VII. Comparing these tables, we identified the most promising preprocessing pipeline is CPAC since it had the best classification results. In Table IV (CPAC), we can see that the DNN with two hidden layers of 128 and 64 units achieve the highest accuracy (75.27%) and highest recall (74%). The DNN with two hidden layers of 256 and 128 units achieve the highest precision (78.37%). Fig. 3 summarizes the best five-fold cross validation accuracy, recall, and precision for the four different preprocessing pipelines. Overall, CPAC obtained the highest accuracy, recall and precision where NIAK obtained the lowest performance.
To evaluate the performance of our DNN models, the best results obtained from the DNN with two hidden layers of 128 and 64 units under CPAC pipeline in Table IV are compared with other studies. First, we compared the accuracy, recall, and precision with other traditional machine learning classifiers Ridge, Logistic Regression, linear SVM, and RBF SVM in [21]. We also compared our DNN results with another deep learning research in [4]. Based on these comparisons, the results in Table VIII show that our DNN (128-64) model achieves the highest classifications to date using the same data from the multisite ABIDE repository.

IV. CONCLUDING REMARKS AND FUTURE WORK
In this study, we implemented deep neural networks (DNNs) with four different hidden layer configuration models to classify Autism Spectrum Disorder (ASD) and typically developing (TD) subjects using the functional connectivity features extracted from resting-state functional magnetic resonance imaging (rs-fMRI) data. Our analysis utilized 1035 samples from the Autism Brain Imaging Data Exchange (ABIDE) multisite repository. To identify the most promising preprocessing pipeline, we implemented DNNs with four different hidden layer configurations using the four different pipeline datasets from the ABIDE repository. Our results indicate that the dataset preprocessed by using CPAC (Configurable Pipeline for the Analysis of Connectomes) pipeline achieves the highest accuracy, recall and precision. Based on our literature studies, the results in Table VIII show that our DNN (128-64) model achieves the highest accuracy, recall, and precision to date using the same ABIDE data.
Compared to single-site datasets, classifications across multi-sites must accommodate variance in subjects, scanning protocols, differences in equipment, and other sources. Generally, such differences affect overall classification performance. Our results demonstrate that DNN models can be used as promising classifiers for large multi-site datasets despite these variations.
Even though DNN models can be used to classify ASD and TD with statistically significant accuracy from restingstate fMRI features, Plitt et al. have pointed out that the classification of ASD and TD using resting-state fMRI data does not provide a biomarker metric [32]. Therefore, there is a gap in the classification performance between the brain-based classifiers and symptom or behavior-based classifiers. This disparity in performance makes diagnosis of ASD an extremely difficult classification problem since it relies on a wide range of symptom expression profiles.
Based on our current research and results further studies on the ABIDE dataset and other multi-site repositories are warranted using DNN in conjunction with CPAC to identify the biomarker ROIs of the ASD group. We also observed in [33] that the use of graph theoretical analysis and machine learning is effective at identifying ASD using a multi-site dataset, being more robust than previous machine learning methods. Motivated by this work, we expect a combination of topological analysis and DNN using the right preprocessing pipeline (CPAC) would be a promising direction to identify the biomarker ROIs for the ASD group. In this research our plan involves the recently developed field of topological data analysis and its Mapper methodology. These strategies will be deployed to ascertain global and local connectivity for ROIs, identify biomarkers, contribute to the development of a biomarker metric, and close the performance gap between brain-based and behavior-based ASD classifiers.