Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

One of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data. We consider the problem of constructing a representation for data lying on a low-dimensional manifold embedded in a high-dimensional space. Drawing on the correspondence between the graph Laplacian, the Laplace Beltrami operator on the manifold, and the connections to the heat equation, we propose a geometrically motivated algorithm for representing the high-dimensional data. The algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality-preserving properties and a natural connection to clustering. Some potential applications and illustrative examples are discussed.

/pdf/laplacian-eigenmaps-for-dimensionality-reduction-and-data-4k0gjfvthz.pdf

Laplacian Eigenmaps for dimensionality reduction and data representation

Presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritage from computer vision, the influence on computer vision, the role of similarity and of interaction, the need for databases, the problem of evaluation, and the role of the semantic gap.

/pdf/content-based-image-retrieval-at-the-end-of-the-early-years-q45iy9ejai.pdf

Content-based image retrieval at the end of the early years

Few prior works study deep learning on point sets. PointNet by Qi et al. is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

In this paper we propose a prototype size selection method for a set of sample graphs. Our first contribution is to show how approximate set coding can be extended from the vector to graph domain. With this framework to hand we show how prototype selection can be posed as optimizing the mutual information between two partitioned sets of sample graphs. We show how the resulting method can be used for prototype graph size selection. In our experiments, we apply our method to a real-world dataset and investigate its performance on prototype size selection tasks.

/pdf/information-theoretic-prototype-selection-for-unattributed-31cbjvf94i.pdf

Information theoretic prototype selection for unattributed graphs

Surface analysis is important for automatic terrain cartography and for airborne navigation. This paper proposes a new approach to shape-from-shading (SFS) in synthetic aperture radar (SAR) images. The SFS problem is embedded in a Bayesian framework. We maximize the surface orientation probability using SAR image statistics, local smoothing and constraints imposed by object discontinuities. We model the statistics of the SAR image distribution as a product between the Rayleigh and Bessel functions. We derive the optimal edge detector for this distribution. The resulting edges are classified as ridges and ravines according to a statistical test. Afterwards, the edges are used as constraints in the estimation of the surface normals. We propose various smoothing algorithms for the vector field of surface normals using robust statistics and surface curvature consistency. The results provided by these algorithms are compared with those given by local averaging.

http://ieeexplore.ieee.org/iel5/6905/18572/00855251.pdf

3-D terrain from synthetic aperture radar images

The wave kernel provides a richer and potentially more expressive means of characterising graphs than the more widely studied wave equation. Unfortunately the wave equation whose solution gives the kernel is less easily solved than the corresponding heat equation. There are two reasons for this. First, the wave equation can not be expressed in terms of the familiar node-based Laplacian, and must instead be expressed in terms of the edge-based Laplacian. Second, the eigenfunctions of the edge-based Laplacian are more complex than those of the node-based Laplacian. This paper presents the solution of a wave equation on a graph. Wave equation provides an interesting alternative to the heat equation defined using the Edge-based Laplacian. This provides the prerequisites for deeper analysis of graphs and their characterisation. For instance it potentially allows the study of non-dispersive solutions or solitons. In this paper we give a complete solution of the wave equation for a Gaussian wave packet. To simulate the equation on a graph, we assume the initial distribution be a Gaussian wave packet on a single edge of the graph. We show the evolution of this Gaussian wave packet with time on some synthetic graphs.

Gaussian Wave Packet on a Graph

We present a method for constructing a generative model for sets of graphs by adopting a minimum description length approach. The method is posed in terms of learning a generative supergraph model from which the new samples can be obtained by an appropriate sampling mechanism. We commence by constructing a probability distribution for the occurrence of nodes and edges over the supergraph. We encode the complexity of the supergraph using the von-Neumann entropy. A variant of EM algorithm is developed to minimize the description length criterion in which the node correspondences between the sample graphs and the supergraph are treated as missing data.The maximization step involves updating both the node correspondence information and the structure of supergraph using graduated assignment. In the experimental part, we demonstrate the practical utility of our proposed algorithm and show that our generative model gives good graph classification results. Besides, we show how to perform graph clustering with Jensen-Shannon kernel and generate new sample graphs.

An information theoretic approach to learning generative graph prototypes

In a recent work we introduced a measure of importance for groups of vertices in a complex network. This centrality for groups is always between 0 and 1 and induces the eigenvector centrality over vertices. Furthermore, its value over any group is the fraction of all network flows intercepted by this group. Here we provide the rigorous mathematical constructions underpinning these results via a semi-commutative extension of a number theoretic sieve. We then established further relations between the eigenvector centrality and the centrality proposed here, showing that the latter is a proper extension of the former to groups of nodes. We finish by comparing the centrality proposed here with the notion of group-centrality introduced by Everett and Borgatti on two real-world networks: the Wolfe’s dataset and the protein-protein interaction network of the yeast Saccharomyces cerevisiae. In this latter case, we demonstrate that the centrality is able to distinguish protein complexes

/pdf/a-centrality-measure-for-cycles-and-subgraphs-ii-yhopcfkv3q.pdf

Richard C. Wilson

Papers

Information theoretic prototype selection for unattributed graphs

3-D terrain from synthetic aperture radar images

Gaussian Wave Packet on a Graph

An information theoretic approach to learning generative graph prototypes

A centrality measure for cycles and subgraphs II.