A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.

/pdf/a-survey-on-transfer-learning-1hjmu3otql.pdf

A Survey on Transfer Learning

We develop fast algorithms for estimation of generalized linear models with convex penalties. The models include linear regression, two-class logistic regression, and multinomial regression problems while the penalties include l(1) (the lasso), l(2) (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.

/pdf/regularization-paths-for-generalized-linear-models-via-39r1t606ac.pdf

Regularization Paths for Generalized Linear Models via Coordinate Descent

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

Pd-0571 rapid learning in practice: validation of an eu population-based prediction model in usa trial data for h&n cancer.

Computer aided detection systems for mammography typically use standard classification algorithms from machine learning for detecting lesions. However, these general purpose learning algorithms make implicit assumptions that are commonly violated in CAD problems. We propose a new ensemble algorithm that explicitly accounts for the small fraction of outlier images which tend to produce a large number of false positives. A bootstrapping procedure is used to ensure that the candidates from these outlier images do not skew the statistical properties of the training samples. Experimental studies on the detection of clusters of micro-calcifications indicate that the proposed method significantly outperforms a state-of-the-art general purpose method for designing classifiers (SVM), in terms of FROC curves on a hold out test set.

/pdf/addressing-image-variability-while-learning-classifiers-for-18n4uztx8a.pdf

Addressing image variability while learning classifiers for detecting clusters of micro-calcifications

Medical treatment is automatically surveyed Drugs or other treatments may be monitored post-market This surveillance may be accomplished in two ways: (1) Identify patients that potentially match templates consistent with possible adverse reactions, possibly including adverse reactions not associated with the treatment Potentially, if the match is good enough, a single patient may be sufficient to raise an alert Alternately, multiple patients partially matching a template may cause an alert (2) Identify patient clusters with unusual patterns Multiple patients associated with greater rates of adverse events or event severity not expected with the treatment are identified The data for surveillance is acquired from multiple sources, so may be more comprehensive for early recognition of adverse effects Data gathering and surveillance are computerized, so early, cost effective recognition may be more likely

Computerized Surveillance of Medical Treatment

This workshop report is an overview of the Predictive Models in Personalized Medicine workshop held on Dec. 11, 2010 at 2010 Neural Information Processing Systems (NIPS) Conference in Whistler, Canada. The workshop included 3 keynote talks and 6 oral and 5 poster presentations on peer reviewed submissions. The workshop also featured a panel discussion on the growing trends of the field. The workshop was very well received and attended. Based on the response and multiple requests received, it is planned to organize the workshop again at NIPS 2011.

Balaji Krishnapuram

Papers

Pd-0571 rapid learning in practice: validation of an eu population-based prediction model in usa trial data for h&n cancer.

Addressing image variability while learning classifiers for detecting clusters of micro-calcifications

Computerized Surveillance of Medical Treatment

Predictive Models in Personalized Medicine: Neural Information Processing Systems (NIPS), 2010 workshop report

PO-0886: Privacy-preserving, multi-centric machine learning across hospitals and countries: does it work?