We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

다중혈관 관상동맥 환자에서 y-문합을 이용하여 양쪽 내흉동맥만을 사용한 우회술의 조기 성적

Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

/pdf/distributed-optimization-and-statistical-learning-via-the-8s9gtddw0d.pdf

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

The optimal design of photonic band gaps for two-dimensional square lattices is considered. We use the level set method to represent the interface between two materials with two different dielectric constants. The interface is moved by a generalized gradient ascent method. The biggest gap of GaAs in air that we found is 0.4418 for TM (transverse magnetic field) and 0.2104 for TE (transverse electric field).

/pdf/maximizing-band-gaps-in-two-dimensional-photonic-crystals-by-16i75rkq40.pdf

Maximizing band gaps in two-dimensional photonic crystals by using level set methods

Real images usually have sparse approximations under some tight frame systems derived from framelets, an oversampled discrete (window) cosine, or a Fourier transform. In this paper, we propose a method for image deblurring in tight frame domains. It is reduced to finding a sparse solution of a system of linear equations whose coefficient matrix is rectangular. Then, a modified version of the linearized Bregman iteration proposed and analyzed in [J.-F. Cai, S. Osher, and Z. Shen, Math. Comp., to appear, UCLA CAM Report (08-52), 2008; J.-F. Cai, S. Osher, and Z. Shen, Math. Comp., to appear, UCLA CAM Report (08-06), 2008; S. Osher et al., UCLA CAM Report (08-37), 2008; W. Yin et al., SIAM J. Imaging Sci., 1 (2008), pp. 143-168] can be applied. Numerical examples show that the method is very simple to implement, robust to noise, and effective for image deblurring.

/pdf/linearized-bregman-iterations-for-frame-based-image-24v4hkwje5.pdf

Linearized Bregman Iterations for Frame-Based Image Deblurring

This article describes a general formalism for obtaining spatially localized ("sparse") solutions to a class of problems in mathematical physics, which can be recast as variational optimization problems, such as the important case of Schrodinger's equation in quantum mechanics. Sparsity is achieved by adding an regularization term to the variational principle, which is shown to yield solutions with compact support ("compressed modes"). Linear combinations of these modes approximate the eigenvalue spectrum and eigenfunctions in a systematically improvable manner, and the localization properties of compressed modes make them an attractive choice for use with efficient numerical algorithms that scale linearly with the problem size.

/pdf/compressed-modes-for-variational-problems-in-mathematics-and-1c8un7sdtk.pdf

Compressed modes for variational problems in mathematics and physics.

One of the key steps in compressed sensing is to solve the basis pursuit problem min u∈ℝ n{∥u∥: Au = f }. Bregman iteration was very successfully used to solve this problem in [40]. Also, a simple and fast iterative algorithm based on linearized Bregman iteration was proposed in [40], which is described in detail with numerical simulations in [35]. A convergence analysis of the smoothed version of this algorithm was given in [11]. The purpose of this paper is to prove that the linearized Bregman iteration proposed in [40] for the basis pursuit problem indeed converges.

/pdf/convergence-of-the-linearized-bregman-iteration-for-l1-norm-2vbs55c3bm.pdf

CONVERGENCE OF THE LINEARIZED BREGMAN ITERATION FOR ℓ1-NORM MINIMIZATION

In this paper, we propose a new variational model for image denoising and decomposition, witch combines the total variation minimization model of Rudin, Osher and Fatemi from image restoration, with spaces of oscillatory functions, following recent ideas introduced by Meyer. The spaces introduced here are appropriate to model oscillatory patterns of zero mean, such as noise or texture. Numerical results of image denoising, image decomposition and texture discrimination are presented, showing that the new models decompose better a given image, possible noisy, into cartoon and oscillatory pattern of zero mean, than the standard ones. The present paper develops further the models previously introduced by the authors in Vese and Osher (Modeling textures with total variation minimization and oscillating patterns in image processing, UCLA CAM Report 02-19, May 2002, to appear in Journal of Scientific Computing, 2003). Other recent and related image decomposition models are also discussed.

/pdf/image-denoising-and-decomposition-with-total-variation-3mj4p3g9oq.pdf

Stanley Osher

Papers

Maximizing band gaps in two-dimensional photonic crystals by using level set methods

Linearized Bregman Iterations for Frame-Based Image Deblurring

Compressed modes for variational problems in mathematics and physics.

CONVERGENCE OF THE LINEARIZED BREGMAN ITERATION FOR ℓ1-NORM MINIMIZATION

Image Denoising and Decomposition with Total Variation Minimization and Oscillatory Functions