Multi-Modal Data Fusion for Classification of Autism Spectrum Disorder Using Phenotypic and Neuroimaging Data

  • Sumaira Kausar Center of Excellence in Artificial Intelligence (COE-AI), Department of Computer Science, Bahria University, Islamabad, Pakistan
  • Adnan Younas Center of Excellence in Artificial Intelligence (COE-AI), Department of Computer Science, Bahria University, Islamabad, Pakistan
  • Muhammad Yousuf Kamal Center of Excellence in Artificial Intelligence (COE-AI), Department of Computer Science, Bahria University, Islamabad, Pakistan
  • Samabia Tehsin Center of Excellence in Artificial Intelligence (COE-AI), Department of Computer Science, Bahria University, Islamabad, Pakistan
Keywords: Asperger, Autism Spectrum Disorder (ASD), diagnosis, feature fusion, machine learning, psychiatry


Abstract Views: 0

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that causes disrupted social behaviors and interactions of individuals. Hence, it can adversely affect the social functioning of individuals. Each autistic individual is said to have a sort of unique behavioral pattern. ASD has three major sub-categories, namely autism, Asperger, and pervasive developmental disorder, not otherwise specified. The term spectrum indicates that ASD possesses a large variety of symptoms of severity. Practitioners need to have a vast experience and expertise for the accurate analysis of the symptoms of ASD. These symptoms need to be acquired from a range of modalities. An accurate diagnosis requires the analysis of brain scan and phenotypic data. These aspects present a multifold challenge for computer-aided ASD diagnosis. Most of the existing computer aided ASD diagnosis systems are capable of diagnosing only whether an individual is affected with ASD or not. A detailed categorization into the subcategories of ASD in such diagnosis is missing. Another aspect that is missing in the existing techniques is that symptoms are observed from a single modality. This can adversely affect the accuracy of diagnosis, since different modalities focus on different aspects of symptoms. These challenges and gaps provided the motivation to present a method that covers the variety exhibited in ASD, while considering the dire need of acquiring symptoms from a variety of data sources. The proposed method showed rather encouraging results. Moreover, the achieved results are evident of the efficacy of the proposed method.


Download data is not yet available.


H. Hodges, C. Fealko, and N. Soares, “Autism spectrum disorder: Definition, epidemiology, causes, and clinical evaluation,” Transla. Pediat., vol. 9, no. Suppl 1, pp. S55–S65, Feb. 2020, doi: 21037/tp.2019.09.09

A. Mannion and G. Leader, “Attention-deficit/hyperactivity disorder (AD/HD) in autism spectrum disorder,” Res. Autism Spec. Disord., vol. 8, no. 4, pp. 432–439, Apr. 2014, doi: 2013.12.021

N. Yirmiya and T. Charman, “The prodrome of autism: early behavioral and biological signs, regression, peri- and post-natal development and genetics,” J Child Psychol. Psych., vol. 51, no. 4, pp. 432–458, Jan. 2010, doi: .2010.02214.x

Md. M. Rahman, O. L. Usman, R. C. Muniyandi, S. Sahran, S. Mohamed, and R. A. Razak, “A Review of machine learning methods of feature selection and classification for autism spectrum disorder,” Brain Sci., vol. 10, no. 12, Art. no. 949, Dec. 2020, doi:

R. P. Goin-Kochel et al., “Beliefs about causes of autism and vaccine hesitancy among parents of children with autism spectrum disorder,” Vaccine, vol. 38, no. 40, pp. 6327–6333, Sep. 2020, doi:

C. M. Brewton et al., “Parental beliefs about causes of autism spectrum disorder: An investigation of a research measure using principal component analysis,” Res Autism Spect. Disord., vol. 87, Art, no. 101825, Sep. 2021, doi: 10.1016/j.rasd.2021.101825

A. Z. Guo, “Automated autism detection based on characterizing observable patterns from photos,” IEEE Trans. Affect. Comput., 2020, doi: /taffc.2020.3035088

J. N. Constantino and T. Charman, “Diagnosis of autism spectrum disorder: reconciling the syndrome, its diverse origins, and variation in expression,” Lancet Neurol., vol. 15, no. 3, pp. 279–291, Mar. 2016, doi: (15)00151-9

F. Thabtah and D. Peebles, “A new machine learning model based on induction of rules for autism detection,” Health Inform. J., Art. no. 146045821882471, Jan. 2019, doi:

W. Guthrie, L. B. Swineford, C. Nottke, and A. M. Wetherby, “Early diagnosis of autism spectrum disorder: stability and change in clinical diagnosis and symptom presentation,” J. Child Psychol. Psych., vol. 54, no. 5, pp. 582–590, Oct. 2012, doi: 1111/jcpp.12008

S. Woolfenden, V. Sarkozy, G. Ridley, and K. Williams, “A systematic review of the diagnostic stability of Autism Spectrum Disorder,” Res. Autism Spect. Disord., vol. 6, no. 1, pp. 345–354, Jan. 2012, doi: /10.1016/j.rasd.2011.06.008

T. Iidaka, “Resting state functional magnetic resonance imaging and neural network classified autism and control,” Cortex, vol. 63, pp. 55–67, Feb. 2015, doi: 1016/j.cortex.2014.08.011

C. P. Chen et al., “Diagnostic classification of intrinsic functional connectivity highlights somatosensory, default mode, and visual regions in autism,” NeuroImage, vol. 8, pp. 238–245, 2015, doi: /10.1016/j.nicl.2015.04.002

K. Khosla, K. Jamison, A. Kuceyeski, and M. R. Sabuncu, “3D convolutional neural networks for classification of functional connectomes,” in Proc. Deep Learn. Med. Image Anal. Multimodal Learn. Clinic. Deci. Granada, Spain, Sep. 20, 2018, pp. 137–145.

X. Li, N. C. Dvornek, J. Zhuang, P. Ventola, and J. S. Duncan, “Brain biomarker interpretation in ASD using deep learning and fMRI,” In Int. Conf. Med. Image Comput. Comput-assist. Interven., pp. 206-214. Granada, Spain, Sept. 16-20, 2018, pp. 206–214.

A. S. Heinsfeld, A. R. Frano, R. C. Craddock, A. Buchweitz, and F. Meneguzzia, “Identification of autism spectrum disorder using deep learning and the ABIDE dataset,” NeuroImage, vol. 17, pp. 16–23, Jan. 2018, doi:

X. Yang, P. T. Schrader, and N. Zhang, “A deep neural network study of the abide repository on autism spectrum classification,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 4, 2020, doi:

W. Yin, S. Mostafa, and F. Wu, “Diagnosis of autism spectrum disorder based on functional brain networks with deep learning,” J. Comput. Biol., Oct. 2020, doi:

D. Arya et al., “Fusing structural and functional MRIs using graph convolutional networks for autism classification,” in Med. Imag. Deep Lear., 2020, pp. 44–61.

H. Sewani and R. Kashef, “An Autoencoder-Based deep learning classifier for efficient diagnosis of autism,” Children, vol. 7, no. 10, Art. no. 182, Oct. 2020, doi:

M. Pominova, E. Kondrateva, M. Sharaev, A. Bernstein, E. Burnaev, “Fader networks for domain adaptation on fmri: abide-ii study,” in 13th Int. Conf. Mach. Vision, Rome, Italy, Nov. 2–6, 2021, pp. 570–577, doi:

H. Lu, S. Liu, H. Wei, and J. Tu, “Multi-kernel fuzzy clustering based on auto-encoder for fMRI functional network,” Expert Syst. Appl., vol. 159, Art. no. 113513, Nov. 2020, doi:

F. Huang et al., “Self-weighted adaptive structure learning for ASD diagnosis via multi-template multi-center representation,” Med. Image Anal., vol. 63, Art. no. 101662, Jul. 2020, doi: 1016/

M. Rakić, M. Cabezas, K. Kushibar, A. Oliver, and X. Lladó, “Improving the detection of autism spectrum disorder by combining structural and functional MRI information,” NeuroImage, vol. 25, Art. no. 102181, 2020, doi:

A. Di Martino et al., “Enhancing studies of the connectome in autism using the autism brain imaging data exchange II,” Sci. Data, vol. 4, no. 1, Art. no. 170010, Mar. 2017, doi: