There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Soil Chemical Analysis

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

The paradigm of differentiable programming has considerably enhanced the scope of machine learning via the judicious use of gradient-based optimization. However, standard differentiable programming methods (such as autodiff) typically require that the models be differentiable, limiting their applicability. We introduce a new, principled approach to extend gradient-based optimization to piecewise smooth models, such as k-histograms, splines, and segmentation maps. We derive an accurate form to the weak Jacobian of such functions, and show that it exhibits a block-sparse structure that can be computed implicitly and efficiently. We show that using the redesigned Jacobian leads to improved performance in applications such as denoising with piecewise polynomial regression models, data-free generative model training, and image segmentation.

/pdf/differentiable-programming-for-piecewise-polynomial-4jeci1yspr.pdf

Differentiable Programming for Piecewise Polynomial Functions

We consider the distributed training of large-scale neural networks that serve as PDE solvers producing full field outputs. We specifically consider neural solvers for the generalized 3D Poisson equation over megavoxel domains. A scalable framework is presented that integrates two distinct advances. First, we accelerate training a large model via a method analogous to the multigrid technique used in numerical linear algebra. Here, the network is trained using a hierarchy of increasing resolution inputs in sequence, analogous to the 'V', 'W', 'F', and 'Half-V' cycles used in multigrid approaches. In conjunction with the multi-grid approach, we implement a distributed deep learning framework which significantly reduces the time to solve. We show the scalability of this approach on both GPU (Azure VMs on Cloud) and CPU clusters (PSC Bridges2). This approach is deployed to train a generalized 3D Poisson solver that scales well to predict output full-field solutions up to the resolution of 512x512x512 for a high dimensional family of inputs.

Distributed Multigrid Neural Solvers on Megavoxel Domains

The alarmingly degrading state of transportation infrastr uctures combined with their key societal and economic importa nce calls for automatic condition assessment methods to facili t te smart management of maintenance and repairs. In particular , scalable data-driven approaches is of great interest, beca use it can deal with large volume of streaming data without requiri ng models that can be inaccurate and computationally expensiv to run. Properly designed, a data-driven methodology could en able fast and automatic evaluation of infrastructures, dis covery of causal dependencies among various sub-system dynamic re sponses, and inference and decision making with uncertaint ies and lack of labeled data. In this work, a spatiotemporal pattern network (STPN) strategy built on symbolic dynamic filte ring (SDF) is applied to explore spatiotemporal behaviors in bri dge network. Data from strain gauges installed on two bridges ar e simulated by finite element method, and the causality among strain data in spatial and temporal resolutions is analyzed . Case studies are conducted for truck identification and damage de tection from simulation data. Results show significant capa bilities of the proposed approach in: (i) capturing spatiotemp oral features to discover causality between bridges (geographi cally close), (ii) robustness to noise in data for feature extract ion, and (iii) detecting and localizing damage via the comparison of behaviors within the bridge network.

Damage Detection of Bridge Network With Spatiotemporal Pattern Network

In distributed machine learning, where agents collaboratively learn from diverse private data sets, there is a fundamental tension between consensus and optimality. In this paper, we build on recent algorithmic progresses in distributed deep learning to explore various consensus-optimality trade-offs over a fixed communication topology. First, we propose the incremental consensus-based distributed stochastic gradient descent (i-CDSGD) algorithm, which involves multiple consensus steps (where each agent communicates information with its neighbors) within each SGD iteration. Second, we propose the generalized consensus-based distributed SGD (g-CDSGD) algorithm that enables us to navigate the full spectrum from complete consensus (all agents agree) to complete disagreement (each agent converges to individual model parameters). We analytically establish convergence of the proposed algorithms for strongly convex and nonconvex objective functions; we also analyze the momentum variants of the algorithms for the strongly convex case. We support our algorithms via numerical experiments, and demonstrate significant improvements over existing methods for collaborative deep learning.

/pdf/on-consensus-optimality-trade-offs-in-collaborative-deep-2p8vh0w47k.pdf

On Consensus-Optimality Trade-offs in Collaborative Deep Learning

This paper develops a distributed algorithm for decision/awareness propagation in mobile-agent networks. A time-dependent proximity network topology is adopted to represent a mobile-agent scenario. The agent-interaction policy formulated here is inspired from the recently developed language-measure-theory. Analytical results related to convergence of statist ical moments of agent states are derived and then validated by numerical simulation. The results show that a single (user-defined) param eter in the agent interaction policy can be identified to control the tradeoff between Propagation Radius(i.e., how far a decision spreads from its source) and Localization Gradient (i.e., the extent to which the spatial variations may affect localization of the source) as well as the temporal convergence properties.

/pdf/distributed-decision-propagation-in-mobile-agent-proximity-5youpwps62.pdf

Soumik Sarkar

Papers

Differentiable Programming for Piecewise Polynomial Functions

Distributed Multigrid Neural Solvers on Megavoxel Domains

Damage Detection of Bridge Network With Spatiotemporal Pattern Network

On Consensus-Optimality Trade-offs in Collaborative Deep Learning

Distributed Decision Propagation in Mobile-agent Proximity Networks F