Enrico M. Malatesta

Bio: Enrico M. Malatesta is an academic researcher from Bocconi University. The author has contributed to research in topics: Travelling salesman problem & Bipartite graph. The author has an hindex of 7, co-authored 22 publications receiving 126 citations. Previous affiliations of Enrico M. Malatesta include University of Milan.

Properties of the Geometry of Solutions and Capacity of Multilayer Neural Networks with Rectified Linear Unit Activations

Abstract: Rectified linear units (ReLUs) have become the main model for the neural units in current deep learning systems. This choice was originally suggested as a way to compensate for the so-called vanishing gradient problem which can undercut stochastic gradient descent learning in networks composed of multiple layers. Here we provide analytical results on the effects of ReLUs on the capacity and on the geometrical landscape of the solution space in two-layer neural networks with either binary or real-valued weights. We study the problem of storing an extensive number of random patterns and find that, quite unexpectedly, the capacity of the network remains finite as the number of neurons in the hidden layer increases, at odds with the case of threshold units in which the capacity diverges. Possibly more important, a large deviation approach allows us to find that the geometrical landscape of the solution space has a peculiar structure: While the majority of solutions are close in distance but still isolated, there exist rare regions of solutions which are much more dense than the similar ones in the case of threshold units. These solutions are robust to perturbations of the weights and can tolerate large perturbations of the inputs. The analytical results are corroborated by numerical findings.

Solution for a bipartite Euclidean traveling-salesman problem in one dimension.

Abstract: The traveling-salesman problem is one of the most studied combinatorial optimization problems, because of the simplicity in its statement and the difficulty in its solution We characterize the optimal cycle for every convex and increasing cost function when the points are thrown independently and with an identical probability distribution in a compact interval We compute the average optimal cost for every number of points when the distance function is the square of the Euclidean distance We also show that the average optimal cost is not a self-averaging quantity by explicitly computing the variance of its distribution in the thermodynamic limit Moreover, we prove that the cost of the optimal cycle is not smaller than twice the cost of the optimal assignment of the same set of points Interestingly, this bound is saturated in the thermodynamic limit

Exact value for the average optimal cost of the bipartite traveling salesman and two-factor problems in two dimensions

Abstract: We show that the average optimal cost for the traveling salesman problem in two dimensions, which is the archetypal problem in combinatorial optimization, in the bipartite case, is simply related to the average optimal cost of the assignment problem with the same Euclidean, increasing, convex weights. In this way we extend a result already known in one dimension where exact solutions are available. The recently determined average optimal cost for the assignment when the cost function is the square of the distance between the points provides therefore an exact prediction $\overline{{E}_{N}}=\frac{1}{\ensuremath{\pi}}\phantom{\rule{0.16em}{0ex}}logN$ for large number of points $2N$. As a by-product of our analysis, also the loop covering problem has the same optimal average cost. We also explain why this result cannot be extended to higher dimensions. We numerically check the exact predictions.

Finite-size corrections in the random assignment problem

Abstract: We analytically derive, in the context of the replica formalism, the first finite-size corrections to the average optimal cost in the random assignment problem for a quite generic distribution law for the costs. We show that, when moving from a power-law distribution to a $\mathrm{\ensuremath{\Gamma}}$ distribution, the leading correction changes both in sign and in its scaling properties. We also examine the behavior of the corrections when approaching a $\ensuremath{\delta}$-function distribution. By using a numerical solution of the saddle-point equations, we provide predictions that are confirmed by numerical simulations.

Selberg integrals in 1D random Euclidean optimization problems

Abstract: We consider a set of Euclidean optimization problems in one dimension, where the cost function associated to the couple of points $x$ and $y$ is the Euclidean distance between them to an arbitrary power $p\ge1$, and the points are chosen at random with flat measure. We derive the exact average cost for the random assignment problem, for any number of points, by using Selberg's integrals. Some variants of these integrals allows to derive also the exact average cost for the bipartite travelling salesman problem.

Exactly Solved Models in Statistical Mechanics

Abstract: R J Baxter 1982 London: Academic xii + 486 pp price £43.60 Over the past few years there has been a growing belief that all the twodimensional lattice statistical models will eventually be solved and that it will be Professor Baxter who solves them. Baxter has inherited the mantle of Onsager who started the process by solving exactly the two-dimensional Ising model in 1944.

Statistical physics of spin glasses and information processing : an introduction

Abstract: 1 Mean-field theory of phase transitions 2 Mean-field theory of spin glasses 3 Replica symmetry breaking 4 Gauge theory of spin glasses 5 Error-correcting codes 6 Image restoration 7 Associative memory 8 Learning in perceptron 9 Optimization problems A Eigenvalues of the Hessian B Parisi equation C Channel coding theorem D Distribution and free energy of K-Sat References Index