Developing a multi-modal dataset for deep learning-based neural networks in autism spectrum disorder diagnosis

• Main Authors: Rashid, Muhammad Mahbubur, Alam, Mohammad Shafiul, Ali, Mohammad Yeakub, Yvette, Susiapan
• Format: Proceeding Paper
• Language: English; English
• Published: AIP publishing 2024
• Subjects: TS200 Metal manufactures. Metalworking
• Online Access: http://irep.iium.edu.my/115388/7/115388_Developing%20a%20multi-modal%20dataset.pdf; http://irep.iium.edu.my/115388/8/115388_Developing%20a%20multi-modal%20dataset_Scopus.pdf; http://irep.iium.edu.my/115388/; https://pubs.aip.org/aip/acp/article-abstract/3161/1/020123/3310612/Developing-a-multi-modal-dataset-for-deep-learning?redirectedFrom=fulltext; https://doi.org/10.1063/5.0229867
• Institution: Universiti Islam Antarabangsa Malaysia

This study introduces a novel multi-modal dataset that has been meticulously curated to significantly improve diagnostic capabilities for Autism Spectrum Disorder (ASD). The dataset is intended to exploit the inherent diagnostic potential of facial images by emphasizing their systematic analysis. It is comprised of video excerpts from the Self- Stimulatory Behavior Dataset (SSBD) demonstrating behaviors such as hand flapping, head banging, and spinning, as well as 50 videos from ASD therapists and specialized educational institutions. In order to establish a comparative baseline, 100 videos from traditional educational contexts were collected for the Normal control group. Face Recognition (FR) pipeline with MTCNN is used for facial recognition, followed by precision-driven cropping, alignment, and scaling stages, resulting in RGB (2D) facial representations. Moreover, the incorporation of 3DDFA-V2 facilitates the transformation of 2D RGB images into a comprehensive 3D dataset by means of depth maps. The dataset contains 173 individuals aged 1 to 11 years in the Normal control group and 123 individuals, including 93 males and 30 females, in the ASD group. The data was divided into training and testing sets, with 1068 training samples and 100 testing samples, respectively. Open access to this dataset for researchers, medical professionals, and technologists will foster collaborative efforts in the evelopment of automated ASD diagnostic methods, leveraging multi-modal data and deep learning to improve diagnostic precision and efficiency