We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

计量经济分析 = Econometric analysis

In this paper, wavelet shrinkage techniques that make use of wavelets defined on quincunx grid are explored in comparison with techniques involving wavelets defined on separable grid. In order to better exploit the advantages of the nonseparable transform and to surpass certain disadvantages of quincunx wavelets, the empirical wavelet-domain Wiener filtering that combines separable and quincunx wavelets is additionally investigated.

/pdf/image-denoising-by-combined-quincunx-and-separable-wavelet-1ly53qjpif.pdf

Image denoising by combined Quincunx and separable wavelet-domain wiener filtering

A filter structure formed as a linear combination of stack filters (L-stack-filters), is studied. This type of filters include many useful filter classes, e.g., linear FIR filters and nonlinear threshold Boolean filters, and L-filters. An efficient algorithm for finding optimal filter coefficients under the mean squared error (MSE) criterion is derived. A subclass of the above filters, called FFT-ordered L-filters (FFT-LF), is studied in detail. In this case the bank of filters is formed according to a generalized structure of the FFT flowchart. It is shown that FFT-LFs effectively remove mixed Gaussian and impulsive noise. While possessing good characteristics of performance, FFT-LFs are simple in implementation. In the sense of implementation the most complicated FFT-LFs are the well-known L-filters. We suggest an efficient parallel architecture implementing FFT-LFs as well as a family of discrete orthogonal transforms including discrete Fourier and Walsh transforms. Both linear and nonlinear L-filter-type filters are implemented effectively on the architecture. Comparison with known architectures implementing both linear and nonlinear filters reveals advantages of the proposed architecture. An efficient implementation of L-stack-filters is also proposed.

Efficient nonlinear transform methods for image processing

In different applications, it is often desirable to retrieve useful information from multichannel (color, multispectral, dual or full-polarization) images. On one hand, multichannel images are potentially able to provide a lot of useful information about sensed objects (terrains). On the other hand, the task of its reliable extraction is very complicated. And there are many reasons behind this like inherent noise, lack of a priori information about object features, complexity of scenes, etc. Therefore, numerous different approaches based on various functional principles and mathematical background have been already put forward. In majority of them, image classification and segmentation are common operations that precede estimation of object parameters. However, practically all methods are far away from completeness and/or perfection since they suffer from different drawbacks and application restrictions. Recently we have proposed methods based on learning with local parameter clustering that were rather successfully applied to image locally adaptive filtering and detection of objects with certain properties. This paper is an attempt to extend this approach to image classification, segmentation and object parameter estimation. A particular application of substance quantitative analysis from color images is considered. The proposed approach is shown to solve the aforementioned task quite well and to have a rather high potential for other applications.

Object classification, segmentation and parameter estimation in multichannel images by classifier learning with clustering of local parameters

Lensless phase-retrieval system with phase modulation of free propagation wavefront is proposed. Contrary to the traditional super-resolution phase-retrieval, the method in this paper requires a single observation only and uses advanced SR-SPAR iterative technique. Successful object imaging relies on modulation of the object wavefront with a random phase-mask, which generates enlarged intensity patterns, allowing us to extract more information than it is possible without such a mask. The achieved high-quality super-resolution phase-imaging is demonstrated by simulation-tests produced with the parameters corresponding to the physical prototype of the considered optical system.

Single exposure lensless subpixel phase imaging

The tasks of digital elevation map reconstruction from isogram maps and the arising problems are considered. An iterative algorithm based on discrete cosine transform and histogram filtering with taking into account several nonlinear constraints is proposed. The algorithm efficiency and accuracy analysis for test and real data is carried out. The advantages of the proposed technique and the perspectives of further investigations are shown.

Karen Egiazarian

Papers

Image denoising by combined Quincunx and separable wavelet-domain wiener filtering

Efficient nonlinear transform methods for image processing

Object classification, segmentation and parameter estimation in multichannel images by classifier learning with clustering of local parameters

Single exposure lensless subpixel phase imaging

Digital elevation map reconstruction from isogram map using iterative DCT algorithm with nonlinear contraints