Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

[신간의 별자리x] 우리/미술, 그리고 ‘슬픔의 박물관’

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

The WU-Minn Human Connectome Project: An Overview

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Many fields view stochasticity as a way to gain computational efficiency, while often having to trade off accuracy. In this perspective article, we argue that stochasticity plays a fundamentally different role in machine learning (ML) and is likely a critical ingredient of intelligent systems. As we review the ML literature, we notice that stochasticity features in many ML methods, affording them robustness, generalizability, and calibration. We also note that randomness seems to be prominent in biological intelligence, from the spiking patterns of individual neurons to the complex behavior of animals. We conclude with a discussion of how we believe stochasticity might shape the future of ML.

Intelligence plays dice: Stochasticity is essential for machine learning

Alzheimer’s disease (AD) is a neurodegenerative condition that progresses over decades. Early detection of individuals at high risk of future progression toward AD is likely to be of critical significance for the successful treatment and/or prevention of this devastating disease. In this paper, we present an empirical study to characterize how predictable an individual subjects’ future AD trajectory is, several years in advance, based on rich multi-modal data, and using modern deep learning methods. Crucially, the machine learning strategy we propose can handle different future time horizons and can be trained with heterogeneous data that exhibit missingness and non-uniform follow-up visit times. Our experiments demonstrate that our strategy yields predictions that are more accurate than a model trained on a single time horizon (e.g. 3 years), which is common practice in prior literature. We also provide a comparison between linear and nonlinear models, verifying the well-established insight that the latter can offer a boost in performance. Our results also confirm that predicting future decline for cognitively normal (CN) individuals is more challenging than for individuals with mild cognitive impairment (MCI). Intriguingly, however, we discover that prediction accuracy decreases with increasing time horizon for CN subjects, but the trend is in the opposite direction for MCI subjects. Additionally, we quantify the contribution of different data types in prediction, which yields novel insights into the utility of different biomarkers. We find that molecular biomarkers are not as helpful for CN individuals as they are for MCI individuals, whereas magnetic resonance imaging biomarkers (hippocampus volume, specifically) offer a significant boost in prediction accuracy for CN individuals. Finally, we show how our model’s prediction reveals the evolution of individual-level progression risk over a five-year time horizon. Our code is available at https://github.com/batuhankmkaraman/mlbasedad.

Machine learning based multi-modal prediction of future decline toward Alzheimer’s disease: An empirical study

Multi-modal latent factor exploration of atrophy, cognitive and tau heterogeneity in Alzheimer's disease

The previously established LOUPE (Learning-based Optimization of the Under-sampling Pattern) framework for optimizing the k-space sampling pattern in MRI was extended in three folds: firstly, fully sampled multi-coil k-space data from the scanner, rather than simulated k-space data from magnitude MR images in LOUPE, was retrospectively under-sampled to optimize the under-sampling pattern of in-vivo k-space data; secondly, binary stochastic k-space sampling, rather than approximate stochastic k-space sampling of LOUPE during training, was applied together with a straight-through (ST) estimator to estimate the gradient of the threshold operation in a neural network; thirdly, modified unrolled optimization network, rather than modified U-Net in LOUPE, was used as the reconstruction network in order to reconstruct multi-coil data properly and reduce the dependency on training data. Experimental results show that when dealing with the in-vivo k-space data, unrolled optimization network with binary under-sampling block and ST estimator had better reconstruction performance compared to the ones with either U-Net reconstruction network or approximate sampling pattern optimization network, and once trained, the learned optimal sampling pattern worked better than the hand-crafted variable density sampling pattern when deployed with other conventional reconstruction methods.

Extending LOUPE for K-space Under-sampling Pattern Optimization in Multi-coil MRI

Individuals with Alzheimer’s disease (AD) exhibit significant heterogeneity across clinical symptoms, atrophy patterns, and spatial distribution of Tau deposition. Most previous studies exploring AD heterogeneity have focused on atypical clinical phenotypes, defined subtypes with a single modality, or restricted their analyses to a priori brain regions and cognitive tests. Here, we considered a data-driven Bayesian model to identify latent factors from atrophy patterns and cognitive deficits simultaneously, thus exploiting the rich dimensionality within each modality, and applied these methods to typical late-onset AD. Departing from most previous AD subtyping studies, our model allowed each factor to be expressed to varying degrees within an individual. This approach resulted in three atrophy-cognitive factors. The first factor was associated with atrophy of the medial temporal lobe (MTL), episodic memory deficits and disorientation to time and place (“MTL-Memory”). The second factor was associated with atrophy of the lateral temporal cortex and the default network, as well as language deficits (“Lateral Temporal-Language”). The third factor was associated with diffuse atrophy in parietal and posterior lateral temporal cortex, and visuospatial executive function deficits (“Posterior Cortical-Executive”). While the MTL-Memory and Posterior Cortical-Executive factors have been previously discussed in the literature, the Lateral Temporal-Language factor is novel and emerged only by considering atrophy and cognition jointly. The factors were replicated in an independent ADNI dementia cohort and were present in mild cognitive impaired (MCI) participants, thus capturing changes that occur prior to the onset of clinical dementia. The factors were also longitudinally stable within an individual, suggesting that these factors capture heterogeneity across patients, rather than longitudinal disease progression. Finally, using 18F-AV1451 PET scans of MCI participants, we found that the atrophy-cognitive factor loadings were associated with inter-individual regional variation in Tau uptake. Furthermore, the spatial patterns of Tau uptake were consistent with the patterns of atrophy. Taken together, these results suggest that distinct atrophy-cognitive patterns exist in “typical” late-onset Alzheimer’s disease and are associated with distinct patterns of Tau depositions before clinical dementia emerges.

/pdf/multi-modal-latent-factor-exploration-of-atrophy-cognitive-2n507m7e0m.pdf

Mert R. Sabuncu

Papers

Intelligence plays dice: Stochasticity is essential for machine learning

Machine learning based multi-modal prediction of future decline toward Alzheimer’s disease: An empirical study

Multi-modal latent factor exploration of atrophy, cognitive and tau heterogeneity in Alzheimer's disease

Extending LOUPE for K-space Under-sampling Pattern Optimization in Multi-coil MRI

Multi-modal Latent Factor Exploration of Atrophy, Cognitive and Tau Heterogeneity in Typical Late-Onset Alzheimer\'s Disease