Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining Practical Machine Learning Tools and Techniques

http://www.csun.edu/~andrzej/COMP529-S05/papers/Mesh%20Networks%20Survey.pdf

Wireless mesh networks: a survey

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

A two-stage predictor is proposed. It is capable of predicting ARMA processes accurately with a reduced number of parameters. This is achieved by cascading a classical linear predictor, where the constraint on the predictor order is relaxed, and a pole-zero recursive like structure. Adaptive algorithms and sample results are given to demonstrate the excellent performance of the proposed technique.

A two stage approach for adaptive prediction of ARMA processes

Unsupervised deep learning methods for solving audio restoration problems extensively rely on carefully tailored neural architectures that carry strong inductive biases for defining priors in the time or spectral domain. In this context, lot of recent success has been achieved with sophisticated convolutional network constructions that recover audio signals in the spectral domain. However, in practice, audio priors require careful engineering of the convolutional kernels to be effective at solving ill-posed restoration tasks, while also being easy to train. To this end, in this paper, we propose a new U-Net based prior that does not impact either the network complexity or convergence behavior of existing convolutional architectures, yet leads to significantly improved restoration. In particular, we advocate the use of carefully designed dilation schedules and dense connections in the U-Net architecture to obtain powerful audio priors. Using empirical studies on standard benchmarks and a variety of ill-posed restoration tasks, such as audio denoising, in-painting and source separation, we demonstrate that our proposed approach consistently outperforms widely adopted audio prior architectures.

On the Design of Deep Priors for Unsupervised Audio Restoration

Natural images suffer from defocus blur due to the presence of objects at different depths from the camera. Automatic estimation of spatially-varying sharpness has several applications including depth estimation, image quality assessment, information retrieval, image restoration among others. In this paper, we propose a sharpness metric based on the quotient of high- to low-frequency bands of the log-spectrum of the image gradients. Using the proposed sharpness metric, we obtain a descriptive dense sharpness map. We also propose a simple yet effective method to segment out-of-focus regions using a global threshold which is defined using weak textured regions present in the input image. Results over two publicly available databases show that the proposed method provides competitive performance when compared with state-of-the-art methods.

Spatially-Varying Sharpness Map Estimation Based on the Quotient of Spectral Bands

We address the problem of predicting the generalization gap of deep neural networks under large, natural, and synthetic distribution shifts between source and target domains. This is crucial in understanding how models behave in uncontrollable ‘in-the-wild’ scenarios, but existing techniques fail when target domain becomes very different from the source. Accurately capturing the relationship and distance between the source and target domains is critical for a reliable post-hoc estimation of generalization. In this paper, we propose a novel strategy for directly predicting accuracy on unseen target data with the help of anchoring and pre-text encoding in predictive models. Anchoring has been shown previously to perform effectively in characterizing domain shifts, which we exploit for predicting the generalization gap. Our experiments on the PACS dataset along with synthetic ablations indicate that our approach produces well calibrated accuracy estimates outperforming existing baselines.

Predicting the Generalization Gap in Deep Models using Anchoring

With continuing growth in sensor capabilities and database complexity, the analysis and understanding of large data sets becomes increasingly important. Numerous researchers have recently explored the geometry and topology of high contrast patches in natural images in an attempt to understand the underlying manifold of the data. The results show that the majority of natural image patches are best represented by corners and edges, as one would expect from visual inspection. In this paper, we extend this analysis to log-magnitude SAR images from the MSTAR database. Our results show that the most representative high contrast patches in SAR images lie among the clutter however methods extracting target patches only show results more similar to that obtained for natural imagery. Contrary to their natural image counterparts however, high contrast patches in SAR imagery lack a significant geometric structure.

Andreas Spanias

Papers

A two stage approach for adaptive prediction of ARMA processes

On the Design of Deep Priors for Unsupervised Audio Restoration

Spatially-Varying Sharpness Map Estimation Based on the Quotient of Spectral Bands

Predicting the Generalization Gap in Deep Models using Anchoring

Topology of high-contrast patches in SAR images