Stanley Osher

Bio: Stanley Osher is an academic researcher from University of California, Los Angeles. The author has contributed to research in topics: Level set method & Hyperbolic partial differential equation. The author has an hindex of 114, co-authored 510 publications receiving 104028 citations. Previous affiliations of Stanley Osher include University of Minnesota & University of Innsbruck.

Essentially non-oscillatory shock capturing methods applied to turbulence amplification in shock wave calculations

Abstract: ENO (essentially non-oscillatory) schemes can provide uniformly high order accuracy right up to discontinuities while keeping sharp, essentially non-oscillatory shock transitions. Recently, an efficient implementation of ENO schemes was obtained based on fluxes and TVD Runge-Kutta time discretizations. The resulting code is very simple to program for multi-dimensions. ENO schemes are especially suitable for computing problems with both discontinuities and fine structures in smooth regions, such as shock interaction with turbulence, for which results for 1-D and 2-D Euler equations are presented. Much better resolution is observed by using third order ENO schemes than by using second order TVD schemes for such problems.

FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention.

Abstract: We propose FMMformers, a class of efficient and flexible transformers inspired by the celebrated fast multipole method (FMM) for accelerating interacting particle simulation. FMM decomposes particle-particle interaction into near-field and far-field components and then performs direct and coarse-grained computation, respectively. Similarly, FMMformers decompose the attention into near-field and far-field attention, modeling the near-field attention by a banded matrix and the far-field attention by a low-rank matrix. Computing the attention matrix for FMMformers requires linear complexity in computational time and memory footprint with respect to the sequence length. In contrast, standard transformers suffer from quadratic complexity. We analyze and validate the advantage of FMMformers over the standard transformer on the Long Range Arena and language modeling benchmarks. FMMformers can even outperform the standard transformer in terms of accuracy by a significant margin. For instance, FMMformers achieve an average classification accuracy of $60.74\%$ over the five Long Range Arena tasks, which is significantly better than the standard transformer's average accuracy of $58.70\%$.

Motion Involving Mean Curvature

Abstract: In the last chapter we discussed the motion of an interface in an externally generated velocity field V↦(x↦, t). In this chapter we discuss interface motion for a self-generated velocity field (x↦ that depends directly on the level set function φ. As an example, we consider motion by mean curvature where the interface moves in the normal direction with a velocity proportional to its curvature; i.e., V↦ = -bκN↦, where b > 0 is a constant and κ is the curvature. When b > 0, the interface moves in the direction of concavity, so that circles (in two dimensions) shrink to a single point and disappear. When b < 0, the interface moves in the direction of convexity, so that circles grow instead of shrink. This growing-circle effect leads to the growth of small perturbations in the front including those due to round-off errors. Because b < 0 allows small erroneous perturbations to incorrectly grow into 0(1) features, the b < 0 case is ill-posed, and we do not consider it here. Figure 4.1 shows the motion of a wound spiral in a curvature-driven flow. The high-curvature ends of the spiral move significantly faster than the relatively low curvature elongated body section. Figure 4.2 shows the evolution of a star-shaped interface in a curvature-driven flow. The tips of the star move inward, while the gaps in between the tips move outward.

High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems

Abstract: We design and compute first-order implicit-in-time variational schemes with high-order spatial discretization for initial value gradient flows in generalized optimal transport metric spaces. We first review some examples of gradient flows in generalized optimal transport spaces from the Onsager principle. We then use a one-step time relaxation optimization problem for time-implicit schemes, namely generalized Jordan-Kinderlehrer-Otto schemes. Their minimizing systems satisfy implicit-in-time schemes for initial value gradient flows with first-order time accuracy. We adopt the first-order optimization scheme ALG2 (Augmented Lagrangian method) and high-order finite element methods in spatial discretization to compute the one-step optimization problem. This allows us to derive the implicit-in-time update of initial value gradient flows iteratively. We remark that the iteration in ALG2 has a simple-to-implement point-wise update based on optimal transport and Onsager's activation functions. The proposed method is unconditionally stable for convex cases. Numerical examples are presented to demonstrate the effectiveness of the methods in two-dimensional PDEs, including Wasserstein gradient flows, Fisher--Kolmogorov-Petrovskii-Piskunov equation, and two and four species reversible reaction-diffusion systems.

A fast multiphase level set algorithm for solving the Chan‐Vese model

Abstract: In this work, we specifically solve the C-V active contour model by multiphase level set methods. We first develop a fast algorithm based on calculating the variational energy of the C-V model without the length term. We check whether the energy decreases or not when we move a point to another segmented region. Then we draw a connection between this algorithm and the topological derivative, a concept emerged from the shape optimization field. Furthermore, to include the length term of the C-V model, a preprocessing step is taken by using nonlinear diffusion. Numerical experiments have demonstrated the efficiency and the robustness of our algorithm. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)

Going deeper with convolutions

Abstract: We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

I and i

Abstract: There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality. Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

Fast parallel algorithms for short-range molecular dynamics

Abstract: Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.