Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

One of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data. We consider the problem of constructing a representation for data lying on a low-dimensional manifold embedded in a high-dimensional space. Drawing on the correspondence between the graph Laplacian, the Laplace Beltrami operator on the manifold, and the connections to the heat equation, we propose a geometrically motivated algorithm for representing the high-dimensional data. The algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality-preserving properties and a natural connection to clustering. Some potential applications and illustrative examples are discussed.

/pdf/laplacian-eigenmaps-for-dimensionality-reduction-and-data-4k0gjfvthz.pdf

Laplacian Eigenmaps for dimensionality reduction and data representation

Presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritage from computer vision, the influence on computer vision, the role of similarity and of interaction, the need for databases, the problem of evaluation, and the role of the semantic gap.

/pdf/content-based-image-retrieval-at-the-end-of-the-early-years-q45iy9ejai.pdf

Content-based image retrieval at the end of the early years

Few prior works study deep learning on point sets. PointNet by Qi et al. is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.

PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

/pdf/image-retrieval-ideas-influences-and-trends-of-the-new-age-51whw4w7su.pdf

Image retrieval: Ideas, influences, and trends of the new age

Population coding has become an essential paradigm in cognitive neuroscience over the past decade and is increasingly studied within the neural network community. We investigate the use of population vector decoding for local edge orientation estimation from a discrete set of Gabor filters. Vectorial combination of the broadly tuned filter outputs yields a resultant population vector, which gives a precise and robust estimate of the local contour orientation. We present results on the accuracy and robustness of orientation measurement.

Population codes for orientation estimation

Finding an optimal assignment between two sets of objects is a fundamental problem arising in many applications, including the matching of `bag-of-words' representations in natural language processing and computer vision. Solving the assignment problem typically requires cubic time and its pairwise computation is expensive on large datasets. In this paper, we develop an algorithm which can find an optimal assignment in linear time when the cost function between objects is represented by a tree distance. We employ the method to approximate the edit distance between two graphs by matching their vertices in linear time. To this end, we propose two tree distances, the first of which reflects discrete and structural differences between vertices, and the second of which can be used to compare continuous labels. We verify the effectiveness and efficiency of our methods using synthetic and real-world datasets.

Computing Optimal Assignments in Linear Time for Approximate Graph Matching

In prior work, we have shown how to compute global network entropy using a heat bath analogy and Maxwell-Boltzmann statistics. In this work, we show how to project out edge-entropy components so that the detailed distribution of entropy across the edges of a network can be computed. This is particularly useful if the analysis of non-homogeneous networks with a strong community as hub structure is being attempted. To commence, we view the normalized Laplacian matrix as the network Hamiltonian operator which specifies a set of energy states with the Laplacian eigenvalues. The network is assumed to be in thermodynamic equilibrium with a heat bath. According to this heat bath analogy, particles can populate the energy levels according to the classical Maxwell-Boltzmann distribution, and this distribution together with the energy states determines thermodynamic variables of the network such as entropy and average energy. We show how the entropy can be decomposed into components arising from individual edges using the eigenvectors of the normalized Laplacian. Compared to previous work based on the von Neumann entropy, this thermodynamic analysis is more effective in characterizing changes of network structure since it better represents the edge entropy variance associated with edges connecting nodes of large degree. Numerical experiments on real-world datasets are presented to evaluate the qualitative and quantitative differences in performance.

/pdf/network-edge-entropy-from-maxwell-boltzmann-statistics-2imqhmvjob.pdf

Network Edge Entropy from Maxwell-Boltzmann Statistics

In a recent work we introduced a measure of importance for groups of vertices in a complex network. This centrality for groups is always between 0 and 1 and induces the eigenvector centrality over vertices. Furthermore, its value over any group is the fraction of all network flows intercepted by this group. Here we provide the rigorous mathematical constructions underpinning these results via a semi-commutative extension of a number theoretic sieve. We then established further relations between the eigenvector centrality and the centrality proposed here, showing that the latter is a proper extension of the former to groups of nodes. We finish by comparing the centrality proposed here with the notion of group-centrality introduced by Everett and Borgatti on two real-world networks: the Wolfe's dataset and the protein-protein interaction network of the yeast \textit{Saccharomyces cerevisiae}. In this latter case, we demonstrate that the centrality is able to distinguish protein complexes.

/pdf/a-centrality-measure-for-cycles-and-subgraphs-ii-3bg4cr5bxs.pdf

A Centrality Measure for Cycles and Subgraphs II

In this paper we consider how coined quantum walks can be applied to exact graph matching. The matching problem is abstracted using an auxiliary structure that connects pairs of vertices from the graphs to be matched by way of auxiliary vertices. We locate matches using coined quantum walks on this structure. We have tested the algorithm on graphs derived from the NCI molecule database and found it to significantly reduce the space of possible matchings thereby allowing the graphs to be matched directly. We also perform a sensitivity analysis on the algorithm in order to examine its behaviour in the presence of noise.

Richard C. Wilson

Papers

Population codes for orientation estimation

Computing Optimal Assignments in Linear Time for Approximate Graph Matching

Network Edge Entropy from Maxwell-Boltzmann Statistics

A Centrality Measure for Cycles and Subgraphs II

Graph Matching using Interference of Coined Quantum Walks