Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls.

doi:10.1016/J.NEUROIMAGE.2016.02.079

Home
/
Papers
/
Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls.

Journal Article•DOI•

Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls.

Mohammad R. Arbabshirani¹, Sergey M. Plis¹, Jing Sui², Vince D. Calhoun³•Institutions (3)

The Mind Research Network¹, Chinese Academy of Sciences², University of New Mexico³

15 Jan 2017-NeuroImage (Neuroimage)-Vol. 145, pp 137-165

TL;DR: There is extensive evidence showing the great potential of neuroimaging data for single subject prediction of various disorders, however, the main bottleneck of this exciting field is still the limited sample size, which could be potentially addressed by modern data sharing models such as the ones discussed in this paper.

read less

About: This article is published in NeuroImage.The article was published on 2017-01-15 and is currently open access. It has received 699 citations till now. The article focuses on the topics: Neuroimaging.

...read moreread less

Citations

PDF

Open Access

More filters

Diagnostic and Statistical Manual of Mental Disorders(DSM)-5による分類と診断 (特集周産期メンタルヘルス : 妊婦の不安とどう立ち向かうか)

[...]

栄高橋

01 Jul 2014

917 citations

Journal Article•DOI•

Machine learning algorithm validation with a limited sample size

[...]

Andrius Vabalas¹, Emma Gowen¹, Ellen Poliakoff¹, Alexander J. Casson¹•Institutions (1)

University of Manchester¹

07 Nov 2019-PLOS ONE

TL;DR: The authors' simulations show that K-fold Cross-Validation (CV) produces strongly biased performance estimates with small sample sizes, and the bias is still evident with sample size of 1000, while Nested CV and train/test split approaches produce robust and unbiased performance estimates regardless of sample size.

...read moreread less

Abstract: Advances in neuroimaging, genomic, motion tracking, eye-tracking and many other technology-based data collection methods have led to a torrent of high dimensional datasets, which commonly have a small number of samples because of the intrinsic high cost of data collection involving human participants. High dimensional data with a small number of samples is of critical importance for identifying biomarkers and conducting feasibility and pilot work, however it can lead to biased machine learning (ML) performance estimates. Our review of studies which have applied ML to predict autistic from non-autistic individuals showed that small sample size is associated with higher reported classification accuracy. Thus, we have investigated whether this bias could be caused by the use of validation methods which do not sufficiently control overfitting. Our simulations show that K-fold Cross-Validation (CV) produces strongly biased performance estimates with small sample sizes, and the bias is still evident with sample size of 1000. Nested CV and train/test split approaches produce robust and unbiased performance estimates regardless of sample size. We also show that feature selection if performed on pooled training and testing data is contributing to bias considerably more than parameter tuning. In addition, the contribution to bias by data dimensionality, hyper-parameter space and number of CV folds was explored, and validation methods were compared with discriminable data. The results suggest how to design robust testing methodologies when working with small datasets and how to interpret the results of other studies based on what validation method was used.

...read moreread less

622 citations

Cites background or methods or result from "Single subject prediction of brain ..."

...[3] survey, a clear pattern emerged with higher prediction accuracy reported in the studies with small sample sizes....
[...]
...[3] conducted a survey of neuroimaging studies which used supervised ML methods to classify healthy individuals and individuals with brain disorders....
[...]
...Arbabshirani et al. [3] and Varoquaux [4] reviewed studies, which applied ML methods for prediction of various brain disorders and found that studies with smaller sample sizes tended to report higher classification performance....
[...]
...Most of the studies in our own and other surveys [3, 4] used feature selection....
[...]
...In their survey Arbabshirani et al. [3] noted that most of the neuroimaging studies consisted of two parts....
[...]

Journal Article•DOI•

Identification of autism spectrum disorder using deep learning and the ABIDE dataset.

[...]

Anibal Sólon Heinsfeld¹, Alexandre Rosa Franco², R. Cameron Craddock³, Augusto Buchweitz², Felipe Meneguzzi², Felipe Meneguzzi¹ - Show less +2 more•Institutions (3)

Pontifícia Universidade Católica do Rio Grande do Sul¹, Allen Institute for Brain Science², Nathan Kline Institute for Psychiatric Research³

01 Jan 2018-NeuroImage: Clinical

TL;DR: The results improved the state-of-the-art by achieving 70% accuracy in identification of ASD versus control patients in the dataset, and identified the areas of the brain that contributed most to differentiating ASD from typically developing controls as per the deep learning model.

...read moreread less

583 citations

Journal Article•DOI•

A review on neuroimaging-based classification studies and associated feature extraction methods for Alzheimer's disease and its prodromal stages

[...]

Saima Rathore¹, Mohamad Habes¹, Muhammad Aksam Iftikhar², Amanda Shacklett¹, Christos Davatzikos¹ - Show less +1 more•Institutions (2)

University of Pennsylvania¹, COMSATS Institute of Information Technology²

15 Jul 2017-NeuroImage

TL;DR: The studies reviewed indicate that the classification frameworks formulated on the basis of these features show promise for individualized diagnosis and prediction of clinical progression, and a detailed account of AD classification challenges is provided.

...read moreread less

460 citations

Cites background from "Single subject prediction of brain ..."

...Recent review papers (Arbabshirani et al., 2017; Falahati et al., 2014) reported studies on MRI- and multimodality-based classification of AD and MCI, limiting AD classification to MRI or its combination with other modalities only....
[...]

Journal Article•DOI•

Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications

[...]

Sandra Vieira¹, Walter H. L. Pinaya², Andrea Mechelli¹•Institutions (2)

King's College London¹, Universidade Federal do ABC²

01 Mar 2017-Neuroscience & Biobehavioral Reviews

TL;DR: The underlying concepts of DL are introduced and studies that have used this approach to classify brain‐based disorders are reviewed, indicating that DL could be a powerful tool in the current search for biomarkers of psychiatric and neurologic disease.

...read moreread less

455 citations

Cites background or methods from "Single subject prediction of brain ..."

...In such cases, a high accuracy can be misleading, as it may reflect an overestimation of the algorithm’s performance (Arbabshirani et al., 2016)....
[...]
...73 Over the past decade, several ML methods have been applied to neuroimaging data from psychiatric and neurological patients with varying degrees of success (Arbabshirani et al., 2016; Wolfers et al., 2015)....
[...]
...Over the past decade, several ML methods have been applied to neuroimaging data from psychiatric and neurological patients with varying degrees of success (Arbabshirani et al., 2016; Wolfers et al., 2015)....
[...]
...In such cases, a high accuracy can be misleading, as it may reflect an overestimation of the algorithm‟s performance (Arbabshirani et al., 2016)....
[...]
...number of voxels) for a subject is much larger than the total number of subjects, resulting in high-dimensional data (Arbabshirani et al., 2016)....
[...]

1
2
3
4
…
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140

Collapse

References

PDF

Open Access

More filters

Journal Article•DOI•

Regularization and variable selection via the elastic net

[...]

Hui Zou¹, Trevor Hastie¹•Institutions (1)

Stanford University¹

01 Apr 2005-Journal of The Royal Statistical Society Series B-statistical Methodology

TL;DR: It is shown that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation, and an algorithm called LARS‐EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lamba.

...read moreread less

Abstract: Summary. We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together.The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the

...read moreread less

16,538 citations

"Single subject prediction of brain ..." refers methods in this paper

...Examples of regularization algorithms used in embedded feature selection methods are LASSO, elastic net and ridge regression (Hastie et al., 2004; Ng, 2004; Park and Hastie, 2007; Zou and Hastie, 2005)....
[...]

Journal Article•

Diagnostic and Statistical Manual of Mental Disorders (DSM-5)

[...]

Goran Arbanas

01 Jan 2015-Alcoholism and psychiatry research

TL;DR: Diagnostic and statistical manual of mental disorders (DSM-5) was translated by psychiatrists and psychologists, mainly from the University psychiatric hospital Vrapce and published by the Naklada Slap publisher.

...read moreread less

Abstract: Title: Diagnostic and statistical manual of mental disorders (DSM-5) Author: American Psychiatric Association Editors of Croatian Edition: Vlado Jukic, Goran Arbanas ISBN: 978-953-191-787-2 Publisher: Naklada Slap, Jastrebarsko, Croatia Number of pages: 936Diagnostic and statistical manual of mental disorders is a national classification, but since its third edition it became a worldwide used manual. [1] It has been published by the American Psychiatric Association and two years ago the fifth edition was released. [2] Croatian was among the first languages this book was translated to. [3] DSM-5 was translated by psychiatrists and psychologists, mainly from the University psychiatric hospital Vrapce and published by the Naklada Slap publisher.DSM has always been more publicly debated than the other main classification - the International Classification of Diseases (ICD). [4] The same happened with this fifth edition. Even before it was released, numerous individuals, organizations, groups and associations were publicly speaking about the classification, new diagnostic entities and changing criteria. [5]Although there is a tendency of authors of both DSM and ICD to synchronize these two classifications and to make them more harmonized with each new edition, there are several differences among them. While ICD covers all the diseases, disorders and reasons for making a contact with the health system, DSM covers "only" mental disorders. Other disorders (medical conditions, as they are named in DSM-5) are not included, except in situations when they lead to a development of a mental disorder. The other main difference is that DSM is more operational zed, and gives criteria for each of the disorders, listing how many criteria have to be met to make a diagnosis of a particular disorder, and what excluding criteria are.Due to the fact that it is used all around the globe and since it has become the most used psychiatric manual, it is sometimes said that DSM is a "psychiatric Bible". [6]Some critics of DSM say that it stigmatizes people and that in each edition it includes more diagnostic entities. It is true that in each edition of DSM there are more disorders listed, but this is due to the fact that medicine is a developing area and new insights are made every year, so some disorders are separated into different subtypes or subgroups and different new diagnoses, giving the impression more behaviour are being pathologized. The intention of the authors was to make more homogenous groups. But, the truth is that, compared with ICD, it is more difficult to get a diagnosis in DSM, than in ICD, with the same clinical presentation. [7] DSM requires functional impairment or distress to pathologize behaviour, while in ICD this criterion is not present in every case.During the process of developing DSM-5 there was an open public discussion. [2] For over a year any person was able to participate in the discussion about future criteria, inclusion or exclusion of diagnostic entities from DSM. More than 21000 letters was sent to the authors. This was the unprecedented way of developing a classification that ICD now tries to follow in preparation of its 11th edition.As a direct consequence of such an open and wide discussion, some new disorders were included (e.g. hoarding disorder), some were excluded even though they were included during the proposal period (e.g. hypersexual disorders), some were heavily debated (e.g. narcissistic personality disorder). [8-10]As previously mentioned, DSM and ICD systems try to harmonize more. There were more non-American authors included in DSM-5 than ever before and some of the experts in the field were in the task force of DSM-5 and ICD-11. [2, 11]What is new in DSM-5, compared to DSM-IV. The organization of the chapters has been changed, so now the flow of the disorders follow life cycle. The book starts with neurodevelopmental disorders, followed by schizophrenia, bipolar and depressive disorders, and closing with neurocognitive disorders. …

...read moreread less

15,478 citations

Journal Article•DOI•

An introduction to variable and feature selection

[...]

Isabelle Guyon, André Elisseeff¹•Institutions (1)

Max Planck Society¹

01 Mar 2003-Journal of Machine Learning Research

TL;DR: The contributions of this special issue cover a wide range of aspects of variable selection: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.

...read moreread less

Abstract: Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. These areas include text processing of internet documents, gene expression array analysis, and combinatorial chemistry. The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data. The contributions of this special issue cover a wide range of aspects of such problems: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.

...read moreread less

14,509 citations

"Single subject prediction of brain ..." refers background or methods in this paper

...(2008) t2:36 sMRI AD GM maps based on VBM 384,065 SVM HC = 137, AD = 108, MCI = 203, Total = 448 63.7–80.3% Adaszewski et al. (2013) t2:37 sMRI AD Gray matter probability maps 2E6 SVM HC = 226, AD = 91, Total = 417 87% Abdulkadir et al....
[...]
...Finally, there are embedded feature selection methods (Guyon and Elisseeff, 2003)....
[...]
...(2008) t2:36 sMRI AD GM maps based on VBM 384,065 SVM HC = 137, AD = 108, MCI = 203, Total = 448 63.7–80.3% Adaszewski et al. (2013) t2:37 sMRI AD Gray matter probability maps 2E6 SVM HC = 226, AD = 91, Total = 417 87% Abdulkadir et al. (2011) t2:38 sMRI and dMRI AD FA and GM volumes 142 SVM NC = 15, AD = 21, Total = 36 94....
[...]
...t2:7 rsfMRI AD Averaged voxel intensities of selected resting-state networks 4 Multivariate ROC HC = 16, AD = 15, Total = 31 100% Wu et al. (2013) t2:8 rsfMRI AD Graph measures based on FC analysis among ROIs 454 SVM HC = 20, AD = 20, Total = 40 100% Khazaee et al. (2015) t2:9 sMRI AD Eigen brains of key slices 10 SVM NC = 98, AD = 28, Total = 126 92.3% Zhang et al. (2015) t2:10 sMRI AD ODVBA of RAVENs maps N/A SVM HC = 50, AD = 50, Total = 100 90% Zhang and Davatzikos (2011) t2:11 sMRI AD Hippocampus shape measures using LDDMM and PCA 20 principal components (3–4 selected by the classifier) Logistic regression HC = 26, DAT = 18, Total = 44 81....
[...]
...t2:7 rsfMRI AD Averaged voxel intensities of selected resting-state networks 4 Multivariate ROC HC = 16, AD = 15, Total = 31 100% Wu et al. (2013) t2:8 rsfMRI AD Graph measures based on FC analysis among ROIs 454 SVM HC = 20, AD = 20, Total = 40 100% Khazaee et al....
[...]

Journal Article•DOI•

The diagnosis of dementia due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer's disease

[...]

Guy M. McKhann¹, Guy M. McKhann², David S. Knopman³, Howard Chertkow⁴, Bradley T. Hyman⁵, Clifford R. Jack³, Claudia H. Kawas⁶, William E. Klunk⁷, Walter J. Koroshetz, Jennifer J. Manly⁸, Richard Mayeux⁸, Richard C. Mohs⁹, John C. Morris¹⁰, Martin N. Rossor¹¹, Philip Scheltens¹², Maria C. Carrillo¹³, Bill Thies¹³, Sandra Weintraub¹⁴, Creighton H. Phelps¹⁵ - Show less +15 more•Institutions (15)

Johns Hopkins University¹, Johns Hopkins University School of Medicine², Mayo Clinic³, McGill University⁴, Harvard University⁵, University of California, Irvine⁶, University of Pittsburgh⁷, Columbia University Medical Center⁸, Eli Lilly and Company⁹, Washington University in St. Louis¹⁰, UCL Institute of Neurology¹¹, VU University Medical Center¹², Alzheimer's Association¹³, Northwestern University¹⁴, National Institutes of Health¹⁵

01 May 2011-Alzheimers & Dementia

TL;DR: The workgroup sought to ensure that the revised criteria would be flexible enough to be used by both general healthcare providers without access to neuropsychological testing, advanced imaging, and cerebrospinal fluid measures, and specialized investigators involved in research or in clinical trial studies who would have these tools available.

...read moreread less

Abstract: The National Institute on Aging and the Alzheimer's Association charged a workgroup with the task of revising the 1984 criteria for Alzheimer's disease (AD) dementia. The workgroup sought to ensure that the revised criteria would be flexible enough to be used by both general healthcare providers without access to neuropsychological testing, advanced imaging, and cerebrospinal fluid measures, and specialized investigators involved in research or in clinical trial studies who would have these tools available. We present criteria for all-cause dementia and for AD dementia. We retained the general framework of probable AD dementia from the 1984 criteria. On the basis of the past 27 years of experience, we made several changes in the clinical criteria for the diagnosis. We also retained the term possible AD dementia, but redefined it in a manner more focused than before. Biomarker evidence was also integrated into the diagnostic formulations for probable and possible AD dementia for use in research settings. The core clinical criteria for AD dementia will continue to be the cornerstone of the diagnosis in clinical practice, but biomarker evidence is expected to enhance the pathophysiological specificity of the diagnosis of AD dementia. Much work lies ahead for validating the biomarker diagnosis of AD dementia.

...read moreread less

13,710 citations

Journal Article•DOI•

Lifetime and 12-Month Prevalence of DSM-III-R Psychiatric Disorders in the United States: Results From the National Comorbidity Survey

[...]

Ronald C. Kessler¹, Katherine A. McGonagle², Shanyang Zhao², Christopher B. Nelson², Michael R. Hughes², Suzann Eshleman², Hans-Ulrich Wittchen², Kenneth S. Kendler² - Show less +4 more•Institutions (2)

University of Michigan¹, Max Planck Society²

01 Jan 1994-Archives of General Psychiatry

TL;DR: The prevalence of psychiatric disorders is greater than previously thought to be the case, and morbidity is more highly concentrated than previously recognized in roughly one sixth of the population who have a history of three or more comorbid disorders.

...read moreread less

Abstract: Background: This study presents estimates of lifetime and 12-month prevalence of 14 DSM-III-R psychiatric disorders from the National Comorbidity Survey, the first survey to administer a structured psychiatric interview to a national probability sample in the United States. Methods: The DSM-III-R psychiatric disorders among persons aged 15 to 54 years in the noninstitutionalized civilian population of the United States were assessed with data collected by lay interviewers using a revised version of the Composite International Diagnostic Interview. Results: Nearly 50% of respondents reported at least one lifetime disorder, and close to 30% reported at least one 12-month disorder. The most common disorders were major depressive episode, alcohol dependence, social phobia, and simple phobia. More than half of all lifetime disorders occurred in the 14% of the population who had a history of three or more comorbid disorders. These highly comorbid people also included the vast majority of people with severe disorders. Less than 40% of those with a lifetime disorder had ever received professional treatment, and less than 20% of those with a recent disorder had been in treatment during the past 12 months. Consistent with previous risk factor research, it was found that women had elevated rates of affective disorders and anxiety disorders, that men had elevated rates of substance use disorders and antisocial personality disorder, and that most disorders declined with age and with higher socioeconomic status. Conclusions: The prevalence of psychiatric disorders is greater than previously thought to be the case. Furthermore, this morbidity is more highly concentrated than previously recognized in roughly one sixth of the population who have a history of three or more comorbid disorders. This suggests that the causes and consequences of high comorbidity should be the focus of research attention. The majority of people with psychiatric disorders fail to obtain professional treatment. Even among people with a lifetime history of three or more comorbid disorders, the proportion who ever obtain specialty sector mental health treatment is less than 50%. These results argue for the importance of more outreach and more research on barriers to professional help-seeking.

...read moreread less

11,648 citations

"Single subject prediction of brain ..." refers background in this paper

...It is estimated that by the year 2020, depression will account for 15% of the disease burden in the world ranking second after heart disease (Kessler et al., 1994)....
[...]