Next Article in Journal
A Versatile Machine Vision Algorithm for Real-Time Counting Manually Assembled Pieces
Next Article in Special Issue
Classification Models for Skin Tumor Detection Using Texture Analysis in Medical Images
Previous Article in Journal
A Review on Computer Vision-Based Methods for Human Action Recognition
Previous Article in Special Issue
Explainable Machine Learning Framework for Image Classification Problems: Case Study on Glioma Cancer Prediction
Article

Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder

1
Science, Math, and Computer Science Magnet Program, Montgomery Blair High School, Silver Spring, MD 20901, USA
2
Institute for Advanced Computer Studies, University of Maryland College Park, College Park, MD 20742, USA
*
Author to whom correspondence should be addressed.
Received: 5 May 2020 / Revised: 4 June 2020 / Accepted: 5 June 2020 / Published: 10 June 2020
(This article belongs to the Special Issue Deep Learning in Medical Image Analysis)
Recent medical imaging technologies, specifically functional magnetic resonance imaging (fMRI), have advanced the diagnosis of neurological and neurodevelopmental disorders by allowing scientists and physicians to observe the activity within and between different regions of the brain. Deep learning methods have frequently been implemented to analyze images produced by such technologies and perform disease classification tasks; however, current state-of-the-art approaches do not take advantage of all the information offered by fMRI scans. In this paper, we propose a deep multimodal model that learns a joint representation from two types of connectomic data offered by fMRI scans. Incorporating two functional imaging modalities in an automated end-to-end autism diagnosis system will offer a more comprehensive picture of the neural activity, and thus allow for more accurate diagnoses. Our multimodal training strategy achieves a classification accuracy of 74% and a recall of 95%, as well as an F1 score of 0.805, and its overall performance is superior to using only one type of functional data. View Full-Text
Keywords: deep learning; multimodal learning; convolutional neural networks; autism; fMRI deep learning; multimodal learning; convolutional neural networks; autism; fMRI
Show Figures

Figure 1

MDPI and ACS Style

Tang, M.; Kumar, P.; Chen, H.; Shrivastava, A. Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder. J. Imaging 2020, 6, 47. https://0-doi-org.brum.beds.ac.uk/10.3390/jimaging6060047

AMA Style

Tang M, Kumar P, Chen H, Shrivastava A. Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder. Journal of Imaging. 2020; 6(6):47. https://0-doi-org.brum.beds.ac.uk/10.3390/jimaging6060047

Chicago/Turabian Style

Tang, Michelle; Kumar, Pulkit; Chen, Hao; Shrivastava, Abhinav. 2020. "Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder" J. Imaging 6, no. 6: 47. https://0-doi-org.brum.beds.ac.uk/10.3390/jimaging6060047

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop