We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

다중혈관 관상동맥 환자에서 y-문합을 이용하여 양쪽 내흉동맥만을 사용한 우회술의 조기 성적

Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

/pdf/distributed-optimization-and-statistical-learning-via-the-8s9gtddw0d.pdf

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

Geometric optics makes its impact both in mathematics and real world applications related to ray tracing, migration, and tomography. Of special importance in these problems are the wavefronts, or points of constant traveltime away from sources, in the medium. Previously in [Osher, Cheng, Kang, Shim, and Tsai (2002)], we initiated a level set approach for the construction of wavefronts in isotropic media that handled the two major algorithmic issues involved with this problem: resolution and multivalued solutions. This approach was quite general and we were able to construct wavefronts in the presence of refraction, reflection, higher dimensions, and, in [Qian, Cheng, and Osher (2003)], anisotropy as well. However, the technique proposed for handling reflections of waves off objects, an important phenomenon involved in all applications of geometric optics, was inefficient and unwieldy to the point of being unusable, especially in the presence of multiple reflections. We introduce here an alternative approach based on the foundation presented in [Osher, Cheng, Kang, Shim, and Tsai (2002)]. This reworking allows the level set method to be considered for realistic applications involving reflecting surfaces in geometric optics.

Reflection in a Level Set Framework for Geometric Optics

An Eulerian Approach for Vortex Motion Using a Level Set Regularization Procedure

A boundary value problem for the Lavrent’ev–Bitsadze differential equation of mixed type is divided into an elliptic problem involving an unusual boundary condition on the parabolic line, and a hyperbolic problem. The elliptic problem is then converted into an equivalent variational form and shown to be well posed under certain conditions which always include the characteristic boundary case. The finite element method is applied using triangular elements and a few singular functions near the intersection of the parabolic line and the elliptic boundary. This method is shown to be second order accurate and easy to implement. Once the elliptic problem is solved, thus giving u and its derivatives along the parabolic line, the straightforward Cauchy problem is solved in the hyperbolic region. Numerical verification of these results is presented.

A Finite Element Method for a Boundary Value Problem of Mixed Type

We propose a novel method for resolution enhancement for volumetric images based on a variational-based reconstruction approach. The reconstruction problem is posed using a deconvolution model that seeks to minimize the total variation norm of the image. Additionally, we propose a new edge-preserving operator that emphasizes and even enhances edges during the up-sampling and decimation of the image. The edge enhanced reconstruction is shown to yield significant improvement in resolution, especially preserving important edges containing anatomical information. This method is demonstrated as an enhancement tool for low-resolution, anisotropic, 3D brain MRI images, as well as a pre-processing step to improve skull-stripping segmentation of brain images.

MRI resolution enhancement using total variation regularization

We introduce a new iterative regularization procedure for inverse problems based on the use of Bregman distances, with particular focus on problems arising in image processing. We are motivated by the problem of restoring noisy and blurry images via variational methods, specifically by using the BV seminorm. Although our procedure applies in quite general situations it was obtained by geometric considerations (first discussed in [23]) associated with the Rudin-OsherFatemi procedure developed in [29] for image restoration. We obtain rigorous convergence results, and effective stopping criteria for the general procedure. The numerical results for denoising appear to be state-of-the-art and preliminary results for deblurring/denoising are very encouraging.

Stanley Osher

Papers

Reflection in a Level Set Framework for Geometric Optics

An Eulerian Approach for Vortex Motion Using a Level Set Regularization Procedure

A Finite Element Method for a Boundary Value Problem of Mixed Type

MRI resolution enhancement using total variation regularization

Using geometry and iterated refinement for inverse problems (1): total variation based image restoration