Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures
Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, a...
Saved in:
Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2024
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/174949 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-174949 |
---|---|
record_format |
dspace |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Medicine, Health and Life Sciences Machine learning Major depression |
spellingShingle |
Medicine, Health and Life Sciences Machine learning Major depression Belov, Vladimir Erwin-Grabner, Tracy Aghajani, Moji Aleman, Andre Amod, Alyssa R. Basgoze, Zeynep Benedetti, Francesco Besteher, Bianca Bülow, Robin Ching, Christopher R. K. Connolly, Colm G. Cullen, Kathryn Davey, Christopher G. Dima, Danai Dols, Annemiek Evans, Jennifer W. Fu, Cynthia H. Y. Gonul, Ali Saffet Gotlib, Ian H. Grabe, Hans J. Groenewold, Nynke Hamilton, J. Paul Harrison, Ben J. Ho, Tiffany C. Mwangi, Benson Jaworska, Natalia Jahanshad, Neda Klimes-Dougan, Bonnie Koopowitz, Sheri-Michelle Lancaster, Thomas Li, Meng Linden, David E. J. MacMaster, Frank P. Mehler, David M. A. Melloni, Elisa Mueller, Bryon A. Ojha, Amar Oudega, Mardien L. Penninx, Brenda W. J. H. Poletti, Sara Pomarol-Clotet, Edith Portella, Maria J. Pozzi, Elena Reneman, Liesbeth Sacchet, Matthew D. Sämann, Philipp G. Schrantee, Anouk Sim, Kang Soares, Jair C. Stein, Dan J. Thomopoulos, Sophia I. Uyar-Demir, Aslihan van der Wee, Nic J. A. van der Werff, Steven J. A. Völzke, Henry Whittle, Sarah Wittfeld, Katharina Wright, Margaret J. Wu, Mon-Ju Yang, Tony T. Zarate, Carlos Veltman, Dick J. Schmaal, Lianne Thompson, Paul M. Goya-Maldonado, Roberto Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures |
description |
Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, and/or overfitting. Here, we overcome these limitations with the largest multi-site sample size to date (N = 5365) to provide a generalizable ML classification benchmark of major depressive disorder (MDD) using shallow linear and non-linear models. Leveraging brain measures from standardized ENIGMA analysis pipelines in FreeSurfer, we were able to classify MDD versus healthy controls (HC) with a balanced accuracy of around 62%. But after harmonizing the data, e.g., using ComBat, the balanced accuracy dropped to approximately 52%. Accuracy results close to random chance levels were also observed in stratified groups according to age of onset, antidepressant use, number of episodes and sex. Future studies incorporating higher dimensional brain imaging/phenotype features, and/or using more advanced machine and deep learning methods may yield more encouraging prospects. |
author2 |
Lee Kong Chian School of Medicine (LKCMedicine) |
author_facet |
Lee Kong Chian School of Medicine (LKCMedicine) Belov, Vladimir Erwin-Grabner, Tracy Aghajani, Moji Aleman, Andre Amod, Alyssa R. Basgoze, Zeynep Benedetti, Francesco Besteher, Bianca Bülow, Robin Ching, Christopher R. K. Connolly, Colm G. Cullen, Kathryn Davey, Christopher G. Dima, Danai Dols, Annemiek Evans, Jennifer W. Fu, Cynthia H. Y. Gonul, Ali Saffet Gotlib, Ian H. Grabe, Hans J. Groenewold, Nynke Hamilton, J. Paul Harrison, Ben J. Ho, Tiffany C. Mwangi, Benson Jaworska, Natalia Jahanshad, Neda Klimes-Dougan, Bonnie Koopowitz, Sheri-Michelle Lancaster, Thomas Li, Meng Linden, David E. J. MacMaster, Frank P. Mehler, David M. A. Melloni, Elisa Mueller, Bryon A. Ojha, Amar Oudega, Mardien L. Penninx, Brenda W. J. H. Poletti, Sara Pomarol-Clotet, Edith Portella, Maria J. Pozzi, Elena Reneman, Liesbeth Sacchet, Matthew D. Sämann, Philipp G. Schrantee, Anouk Sim, Kang Soares, Jair C. Stein, Dan J. Thomopoulos, Sophia I. Uyar-Demir, Aslihan van der Wee, Nic J. A. van der Werff, Steven J. A. Völzke, Henry Whittle, Sarah Wittfeld, Katharina Wright, Margaret J. Wu, Mon-Ju Yang, Tony T. Zarate, Carlos Veltman, Dick J. Schmaal, Lianne Thompson, Paul M. Goya-Maldonado, Roberto |
format |
Article |
author |
Belov, Vladimir Erwin-Grabner, Tracy Aghajani, Moji Aleman, Andre Amod, Alyssa R. Basgoze, Zeynep Benedetti, Francesco Besteher, Bianca Bülow, Robin Ching, Christopher R. K. Connolly, Colm G. Cullen, Kathryn Davey, Christopher G. Dima, Danai Dols, Annemiek Evans, Jennifer W. Fu, Cynthia H. Y. Gonul, Ali Saffet Gotlib, Ian H. Grabe, Hans J. Groenewold, Nynke Hamilton, J. Paul Harrison, Ben J. Ho, Tiffany C. Mwangi, Benson Jaworska, Natalia Jahanshad, Neda Klimes-Dougan, Bonnie Koopowitz, Sheri-Michelle Lancaster, Thomas Li, Meng Linden, David E. J. MacMaster, Frank P. Mehler, David M. A. Melloni, Elisa Mueller, Bryon A. Ojha, Amar Oudega, Mardien L. Penninx, Brenda W. J. H. Poletti, Sara Pomarol-Clotet, Edith Portella, Maria J. Pozzi, Elena Reneman, Liesbeth Sacchet, Matthew D. Sämann, Philipp G. Schrantee, Anouk Sim, Kang Soares, Jair C. Stein, Dan J. Thomopoulos, Sophia I. Uyar-Demir, Aslihan van der Wee, Nic J. A. van der Werff, Steven J. A. Völzke, Henry Whittle, Sarah Wittfeld, Katharina Wright, Margaret J. Wu, Mon-Ju Yang, Tony T. Zarate, Carlos Veltman, Dick J. Schmaal, Lianne Thompson, Paul M. Goya-Maldonado, Roberto |
author_sort |
Belov, Vladimir |
title |
Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures |
title_short |
Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures |
title_full |
Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures |
title_fullStr |
Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures |
title_full_unstemmed |
Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures |
title_sort |
multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures |
publishDate |
2024 |
url |
https://hdl.handle.net/10356/174949 |
_version_ |
1814047334019891200 |
spelling |
sg-ntu-dr.10356-1749492024-04-21T15:41:10Z Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures Belov, Vladimir Erwin-Grabner, Tracy Aghajani, Moji Aleman, Andre Amod, Alyssa R. Basgoze, Zeynep Benedetti, Francesco Besteher, Bianca Bülow, Robin Ching, Christopher R. K. Connolly, Colm G. Cullen, Kathryn Davey, Christopher G. Dima, Danai Dols, Annemiek Evans, Jennifer W. Fu, Cynthia H. Y. Gonul, Ali Saffet Gotlib, Ian H. Grabe, Hans J. Groenewold, Nynke Hamilton, J. Paul Harrison, Ben J. Ho, Tiffany C. Mwangi, Benson Jaworska, Natalia Jahanshad, Neda Klimes-Dougan, Bonnie Koopowitz, Sheri-Michelle Lancaster, Thomas Li, Meng Linden, David E. J. MacMaster, Frank P. Mehler, David M. A. Melloni, Elisa Mueller, Bryon A. Ojha, Amar Oudega, Mardien L. Penninx, Brenda W. J. H. Poletti, Sara Pomarol-Clotet, Edith Portella, Maria J. Pozzi, Elena Reneman, Liesbeth Sacchet, Matthew D. Sämann, Philipp G. Schrantee, Anouk Sim, Kang Soares, Jair C. Stein, Dan J. Thomopoulos, Sophia I. Uyar-Demir, Aslihan van der Wee, Nic J. A. van der Werff, Steven J. A. Völzke, Henry Whittle, Sarah Wittfeld, Katharina Wright, Margaret J. Wu, Mon-Ju Yang, Tony T. Zarate, Carlos Veltman, Dick J. Schmaal, Lianne Thompson, Paul M. Goya-Maldonado, Roberto Lee Kong Chian School of Medicine (LKCMedicine) Yong Loo Lin School of Medicine, NUS Institute of Mental Health Medicine, Health and Life Sciences Machine learning Major depression Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, and/or overfitting. Here, we overcome these limitations with the largest multi-site sample size to date (N = 5365) to provide a generalizable ML classification benchmark of major depressive disorder (MDD) using shallow linear and non-linear models. Leveraging brain measures from standardized ENIGMA analysis pipelines in FreeSurfer, we were able to classify MDD versus healthy controls (HC) with a balanced accuracy of around 62%. But after harmonizing the data, e.g., using ComBat, the balanced accuracy dropped to approximately 52%. Accuracy results close to random chance levels were also observed in stratified groups according to age of onset, antidepressant use, number of episodes and sex. Future studies incorporating higher dimensional brain imaging/phenotype features, and/or using more advanced machine and deep learning methods may yield more encouraging prospects. Published version ENIGMA MDD: This work was supported by NIH grants U54 EB020403 (PMT) and R01MH116147 (PMT) and R01 MH117601 (NJ & LS). AMC: supported by ERA-NET PRIOMEDCHILD FP 6 (EU) grant 11.32050.26. AFFDIS: this study was funded by the University Medical Center Göttingen (UMG Startförderung) and VB and RGM are supported by German Federal Ministry of Education and Research (Bundesministerium fuer Bildung und Forschung, BMBF: 01 ZX 1507, “PreNeSt—e:Med”). Barcelona-SantPau: MJP is funded by the Ministerio de Ciencia e Innovación of the Spanish Government and by the Instituto de Salud Carlos III through a ‘Miguel Servet’ research contract (CP16–0020); National Research Plan (Plan Estatal de I + D + I 2016–2019); and co-financed by the European Regional Development Fund (ERDF). CARDIFF supported by the Medical Research Council (grant G 1100629) and the National Center for Mental Health (NCMH), funded by Health Research Wales (HS/14/20). CSAN: This work was supported by grants from Johnson & Johnson Innovation (S.E.), the Swedish Medical Research Council (S.E.: 2017–00875, M.H.: 2013–07434, 2019–01138), the ALF Grants, Region Östergötland M.H., J.P.H.), National Institutes of Health (R.D.: R01 CA193522 and R01 NS073939), MD Anderson Cancer Support Grant (R.D.: P30CA016672) Calgary: supported by Canadian Institutes for Health Research, Branch Out Neurological Foundation. FPM is supported by Alberta Children's Hospital Foundation and Canadian Institutes for Health Research. DCHS: supported by the Medical Research Council of South Africa. ETPB: Funding for this work was provided by the Intramural Research Program at the National Institute of Mental Health, National Institutes of Health (IRP-NIMH-NIH; ZIA-MH002857). Episca (Leiden): EPISCA was supported by GGZ Rivierduinen and the LUMC. FIDMAG: This work was supported by the Generalitat de Catalunya (2014 SGR 1573) and Instituto de Salud Carlos III (CPII16/00018) and (PI14/01151 and PI14/01148). Gron: This study was supported by the Gratama Foundation, the Netherlands (2012/35 to NG). Houst: supported in part by NIMH grant R01 085667 and the Dunn Research Foundation. LOND This paper represents independent research (BRCDECC, London) part-funded by the NIHR Maudsley Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. MODECT: This study was supported by the Department of Psychiatry of GGZ inGeest and Amsterdam UMC, location VUmc. MPIP: The MPIP Sample comprises patients included in the Recurrent Unipolar Depression (RUD) Case-Control study at the clinic of the Max Planck Institute of Psychiatry, Munich, Germany. We wish to acknowledge Rosa Schirmer, Elke Schreiter, Reinhold Borschke, and Ines Eidner for MR image acquisition and data preparation, and Benno Pütz, and Bertram Müller-Myhsok for distributed computing support and the MARS and RUD Study teams for clinical phenotyping. We thank Dorothee P. Auer for initiation of the RUD study. Melbourne: funded by National Health and Medical Research Council of Australia (NHMRC) Project Grants 1064643 (Principal Investigator BJH) and 1024570 (Principal Investigator CGD). Minnesota the study was funded by the National Institute of Mental Health (K23MH090421; Dr. Cullen) and Biotechnology Research Center (P41 RR008079; Center for Magnetic Resonance Research), the National Alliance for Research on Schizophrenia and Depression, the University of Minnesota Graduate School, and the Minnesota Medical Foundation. This work was carried out in part using computing resources at the University of Minnesota Supercomputing Institute. Moral dilemma: study was supported by the Brain and Behavior Research Foundation and by the National Health and Medical Research Council ID 1125504 to SLW. NESDA: The infrastructure for the NESDA study ( www.nesda.nl ) is funded through the Geestkracht program of the Netherlands Organisation for Health Research and Development (Zon-Mw, grant number 10–000–1002) and is supported by participating universities (VU University Medical Center, GGZ inGeest, Arkin, Leiden University Medical Center, GGZ Rivierduinen, University Medical Center Groningen) and mental health care organizations, see www.nesda.nl . QTIM: The QTIM data set was supported by the Australian National Health and Medical Research Council (Project Grants No. 496682 and 1009064) and US National Institute of Child Health and Human Development (RO1HD050735). UCSF: This work was supported by the Brain and Behavior Research Foundation (formerly NARSAD) to TTY; the National Institute of Mental Health (R01MH085734 to TTY; K01MH117442 to TCH) and by the American Foundation for Suicide Prevention (PDF-1-064-13) to TCH. SHIP: The Study of Health in Pomerania (SHIP) is part of the Community Medicine Research net (CMR) ( http://www.medizin.uni-greifswald.de/icm ) of the University Medicine Greifswald, which is supported by the German Federal State of Mecklenburg—West Pomerania. MRI scans in SHIP and SHIP-TREND have been supported by a joint grant from Siemens Healthineers, Erlangen, Germany and the Federal State of Mecklenburg-West Pomerania. This study was further supported by the EU-JPND Funding for BRIDGET (FKZ:01ED1615). SanRaffaele (Milano): Italian Ministry of Health, Grant/Award Number: RF‐2011‐02349921 and RF-2018-12367489 Italian Ministry of Education, University and Research (Miur). Number: PRIN -201779W93T. Singapore: The study was supported by grant NHG SIG/15012. KS was supported by National Healthcare Group, Singapore (SIG/15012) for the project.SoCAT: Socat studies supported by Ege University Research Fund (17-TIP-039; 15-TIP-002; 13-TIP-054) and the Scientific and Technological Research Council of Turkey (109S134, 217S228). StanfFAA and StanfT1wAggr: This work was supported by NIH grant R37 MH101495. TIGER: Support for the TIGER study includes the Klingenstein Third Generation Foundation the National Institute of Mental Health K01MH117442 the Stanford Maternal Child Health Research Institute and the Stanford Center for Cognitive and Neurobiological Imaging TCH receives partial support from the Ray and Dagmar Dolby Family Fund. We acknowledge support by the Open Access Publication Funds of the Göttingen University. Open Access funding enabled and organized by Projekt DEAL. 2024-04-17T02:32:50Z 2024-04-17T02:32:50Z 2024 Journal Article Belov, V., Erwin-Grabner, T., Aghajani, M., Aleman, A., Amod, A. R., Basgoze, Z., Benedetti, F., Besteher, B., Bülow, R., Ching, C. R. K., Connolly, C. G., Cullen, K., Davey, C. G., Dima, D., Dols, A., Evans, J. W., Fu, C. H. Y., Gonul, A. S., Gotlib, I. H., ...Goya-Maldonado, R. (2024). Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures. Scientific Reports, 14(1), 1084-. https://dx.doi.org/10.1038/s41598-023-47934-8 2045-2322 https://hdl.handle.net/10356/174949 10.1038/s41598-023-47934-8 38212349 2-s2.0-85182306914 1 14 1084 en NHG SIG/15012 SIG/15012 Scientific Reports © The Author(s) 2024. Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. application/pdf |