There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining - Concepts and Techniques.

Data Mining Practical Machine Learning Tools and Techniques

The implementation of Machine Learning (ML) algorithms on stand-alone small-scale devices allows the incorporation of new services and advanced functionalities without the need of resorting to remote com- puting systems. Despite having undeniable advantages with respect to conventional general-purpose devices, e.g. in terms of cost/performance ratios, small-scale systems suffer of issues related to their resource-limited nature, like limited battery capacity and processing power. In order to deal with such limitations, we propose to merge local Rademacher Complex- ities and bit-based hypothesis spaces to build thrifty models, which can be effectively implemented on small-scale resource-limited devices. Exper- iments, carried out on a smartphone in a Human Activity Recognition application, show the benefits of the proposed approach in terms of model accuracy and battery duration.

https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2014-27.pdf

Learning with Few Bits on Small-Scale Devices: from Regularization to Energy Efficiency

This paper discusses whether the law is up to regulate machine learning (”ML”) model-based decision-making in the context of the railways. We especially deal with the fairness and accountability of these models when exploited in the context of train traffic management (”TTM”). Railway sector-specific regulation, in their quality as network industry, hereby serves as a pilot. We show that, even where technological solutions are available, the law needs to keep up to support and accurately regulate the use of the technological solutions and we identify stumble points in this regard.

Fairness and Accountability of Machine Learning Models in Railway Market: are Applicable Railway Laws Up to Regulate Them?

In this paper we prove that a randomized algorithm based on the data generating dependent prior and data dependent posterior Boltzmann distributions of Catoni (2007) is Differentially Private (DP) and shows better generalization properties than the Gibbs (randomized) classifier associated to the same distributions. For this purpose, we will develop a tight DP-based generalization bound, which improve over the current state-of-the-art Hoeffding-type bound.

https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2017-53.pdf

Generalization Performances of Randomized Classifiers and Algorithms built on Data Dependent Distributions.

Activity-Based Computing [1] aims to capture the state of the user and its environment
by exploiting heterogeneous sensors in order to provide adaptation to
exogenous computing resources. When these sensors are attached to the subject’s
body, they permit continuous monitoring of numerous physiological signals. This
has appealing use in healthcare applications, e.g. the exploitation of Ambient Intelligence
(AmI) in daily activity monitoring for elderly people. In this paper,
we present a system for human physical Activity Recognition (AR) using smartphone
inertial sensors. As these mobile phones are limited in terms of energy and
computing power, we propose a novel hardware-friendly approach for multiclass
classification. This method adapts the standard Support Vector Machine (SVM)
and exploits fixed-point arithmetic. In addition to the clear computational advantages
of fixed-point arithmetic, it is easy to show the regularization effect of the
number of bits and then the connections with the Statistical Learning Theory. A
comparison with the traditional SVM shows a significant improvement in terms
of computational costs while maintaining similar accuracy, which can contribute
to develop more sustainable systems for AmI.

/pdf/human-activity-recognition-on-smartphones-for-mobile-context-1uofo2fub6.pdf

Human activity recognition on smartphones for mobile context awareness

In this work we propose a novel algorithm for training L1-L2 Support Vector Machine (SVM) classifiers. L1-L2 SVMs allow to combine the effectiveness of L2 models and the feature selection characteristics of L1 solutions. The proposed training approach for L1-L2 SVM requires a minimal effort for its implementation, relying on the exploitation of well-known and widespread tools already developed for conventional L2 SVMs. Moreover, the proposed method is flexible, as it allows to train L1, L1-L2 and L2 SVMs, as well as to fine tune the trade-off between dimensionality reduction and classification accuracy. This scope is of clear importance in applications on resource-limited devices, such as smartphones, like the one we consider to verify the main advantages of the proposed approach: the UCI Human Activity Recognition real-world dataset.

Luca Oneto

Papers

Learning with Few Bits on Small-Scale Devices: from Regularization to Energy Efficiency

Fairness and Accountability of Machine Learning Models in Railway Market: are Applicable Railway Laws Up to Regulate Them?

Generalization Performances of Randomized Classifiers and Algorithms built on Data Dependent Distributions.

Human activity recognition on smartphones for mobile context awareness

A Novel Procedure for Training L1-L2 Support Vector Machine Classifiers