We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

다중혈관 관상동맥 환자에서 y-문합을 이용하여 양쪽 내흉동맥만을 사용한 우회술의 조기 성적

Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

/pdf/distributed-optimization-and-statistical-learning-via-the-8s9gtddw0d.pdf

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

In this paper, we incorporate the global convex segmentation method and the split Bregman technique into the region-scalable fitting energy model. The new proposed method based on the region-scalable model can draw upon intensity information in local regions at a controllable scale, so that it can segment images with intensity inhomogeneity. Furthermore, with the application of the global convex segmentation method and the split Bregman technique, the method is very robust and efficient. By using a non-negative edge detector function to the proposed method, the algorithm can detect the boundaries more easily and achieve results that are very similar to those obtained through the classical geodesic active contour model. Experimental results for synthetic and real images have shown the robustness and efficiency of our method and also demonstrated the desirable advantages of the proposed method.

/pdf/split-bregman-method-for-minimization-of-region-scalable-4nwdj8gcxm.pdf

Split Bregman method for minimization of region-scalable fitting energy for image segmentation

Anomaly detection is a hot topic in hyperspectral signal processing. The key point of hyperspectral anomaly detection is the modeling of the background. In this paper, we propose a novel anomaly detection method via global and local joint modeling of background. Based on the observation that the local three-dimensional patch belonging to the background in hyperspectral image (HSI) usually lies in a low dimensional manifold, we propose to reconstruct the background part of a HSI from its subsample by scalable low dimensional manifold modeling (SLDMM). Thus, the background of HSI can be well characterized in both global and local aspects. Taking into consideration that the SLDMM reconstructs the background part at a low sampling ratio, we propose a multiple random sampling reconstruction strategy to further improve the detection accuracies and robustness. The final background is generated by the mean of backgrounds reconstructed from the multiple random sampling and the anomalies are contained in the residual between the observed HSI and the mean background. Experimental results on three real datasets demonstrate that the proposed anomaly detection method outperforms other state-of-the-art hyperspectral anomaly detection methods.

Hyperspectral Anomaly Detection via Global and Local Joint Modeling of Background

The authors of [Proc. Natl. Acad. Sci. USA, 110 (2013), pp. 6634--6639] proposed sparse Fourier domain approximation of solutions to multiscale PDE problems by soft thresholding. We show here that the method enjoys a number of desirable numerical and analytic properties, including convergence for linear PDEs and a modified equation resulting from the sparse approximation. We also extend the method to solve elliptic equations and introduce sparse approximation of differential operators in the Fourier domain. The effectiveness of the method is demonstrated on homogenization examples, where its complexity is dependent only on the sparsity of the problem and constant in many cases.

/pdf/on-the-compressive-spectral-method-3o2wvggrv3.pdf

On the Compressive Spectral Method

In this paper, we present a novel framework for constructing large deformation log-unbiased image registration models that generate theoretically and intuitively correct deformation maps. Such registration models do not rely on regridding and are inherently topology preserving. We apply information theory to quantify the magnitude of deformations and examine the statistical distributions of Jacobian maps in the logarithmic space. To demonstrate the power of the proposed framework, we generalize the well known viscous fluid registration model to compute log-unbiased deformations. We tested the proposed method using a pair of binary corpus callosum images, a pair of two-dimensional serial MRI images, and a set of three-dimensional serial MRI brain images. We compared our results to those computed using the viscous fluid registration method, and demonstrated that the proposed method is advantageous when recovering voxel-wise maps of local tissue change.

/pdf/topology-preserving-log-unbiased-nonlinear-image-czhblsxr4y.pdf

Topology Preserving Log-Unbiased Nonlinear Image Registration: Theory and Implementation

A class was devised of fast wavelet based algorithms for linear evolution equations whose coefficients are time independent. The method draws on the work of Beylkin, Coifman, and Rokhlin which they applied to general Calderon-Zygmund type integral operators. A modification of their idea is applied to linear hyperbolic and parabolic equations, with spatially varying coefficients. A significant speedup over standard methods is obtained when applied to hyperbolic equations in one space dimension and parabolic equations in multidimensions.

/pdf/fast-wavelet-based-algorithms-for-linear-evolution-equations-b1giunsl7b.pdf

Stanley Osher

Papers

Split Bregman method for minimization of region-scalable fitting energy for image segmentation

Hyperspectral Anomaly Detection via Global and Local Joint Modeling of Background

On the Compressive Spectral Method

Topology Preserving Log-Unbiased Nonlinear Image Registration: Theory and Implementation

Fast wavelet based algorithms for linear evolution equations