There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Three parallel algorithms for classical molecular dynamics are presented. The first assigns each processor a fixed subset of atoms; the second assigns each a fixed subset of inter-atomic forces to compute; the third assigns each a fixed spatial region. The algorithms are suitable for molecular dynamics models which can be difficult to parallelize efficiently—those with short-range forces where the neighbors of each atom change rapidly. They can be implemented on any distributed-memory parallel machine which allows for message-passing of data between independently executing processors. The algorithms are tested on a standard Lennard-Jones benchmark problem for system sizes ranging from 500 to 100,000,000 atoms on several parallel supercomputers--the nCUBE 2, Intel iPSC/860 and Paragon, and Cray T3D. Comparing the results to the fastest reported vectorized Cray Y-MP and C90 algorithm shows that the current generation of parallel machines is competitive with conventional vector supercomputers even for small problems. For large problems, the spatial algorithm achieves parallel efficiencies of 90% and a 1840-node Intel Paragon performs up to 165 faster than a single Cray C9O processor. Trade-offs between the three algorithms and guidelines for adapting them to more complex molecular dynamics simulations are also discussed.

Fast parallel algorithms for short-range molecular dynamics

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

The organization of behavior: A neuropsychological theory

Driven by the visions of Internet of Things and 5G communications, recent years have seen a paradigm shift in mobile computing, from the centralized mobile cloud computing toward mobile edge computing (MEC). The main feature of MEC is to push mobile computing, network control and storage to the network edges (e.g., base stations and access points) so as to enable computation-intensive and latency-critical applications at the resource-limited mobile devices. MEC promises dramatic reduction in latency and mobile energy consumption, tackling the key challenges for materializing 5G vision. The promised gains of MEC have motivated extensive efforts in both academia and industry on developing the technology. A main thrust of MEC research is to seamlessly merge the two disciplines of wireless communications and mobile computing, resulting in a wide-range of new designs ranging from techniques for computation offloading to network architectures. This paper provides a comprehensive survey of the state-of-the-art MEC research with a focus on joint radio-and-computational resource management. We also discuss a set of issues, challenges, and future research directions for MEC research, including MEC system deployment, cache-enabled MEC, mobility management for MEC, green MEC, as well as privacy-aware MEC. Advancements in these directions will facilitate the transformation of MEC from theory to practice. Finally, we introduce recent standardization efforts on MEC as well as some typical MEC application scenarios.

A Survey on Mobile Edge Computing: The Communication Perspective

This work proposes the first strategy to make distributed training of neural networks resilient to computing errors, a problem that has remained unsolved despite being first posed in 1956 by von Neumann. He also speculated that the efficiency and reliability of the human brain is obtained by allowing for low power but error-prone components with redundancy for error-resilience. It is surprising that this problem remains open, even as massive artificial neural networks are being trained on increasingly low-cost and unreliable processing units. Our coding-theory-inspired strategy, "CodeNet," solves this problem by addressing three challenges in the science of reliable computing: (i) Providing the first strategy for error-resilient neural network training by encoding each layer separately; (ii) Keeping the overheads of coding (encoding/error-detection/decoding) low by obviating the need to re-encode the updated parameter matrices after each iteration from scratch. (iii) Providing a completely decentralized implementation with no central node (which is a single point of failure), allowing all primary computational steps to be error-prone. We theoretically demonstrate that CodeNet has higher error tolerance than replication, which we leverage to speed up computation time. Simultaneously, CodeNet requires lower redundancy than replication, and equal computational and communication costs in scaling sense. We first demonstrate the benefits of CodeNet in reducing expected computation time over replication when accounting for checkpointing. Our experiments show that CodeNet achieves the best accuracy-runtime tradeoff compared to both replication and uncoded strategies. CodeNet is a significant step towards biologically plausible neural network training, that could hold the key to orders of magnitude efficiency improvements.

CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors.

We consider the problem of information embedding where the encoder modifies a white Gaussian host signal in a power-constrained manner to encode a message, and the decoder recovers both the embedded message and the modified host signal. This partially extends the recent work of Sumszyk and Steinberg to the continuous-alphabet Gaussian setting. Through a control-theoretic lens, we observe that the problem is a minimalist example of what is called the triple role of control actions. We show that a dirty-paper-coding strategy achieves the optimal rate for perfect recovery of the modified host and the message for any message rate. For imperfect recovery of the modified host, by deriving bounds on the minimum mean-square error (MMSE) in recovering the modified host signal, we show that Dirty-Paper Coding-based strategies are guaranteed to attain within a uniform constant factor of 16 of the optimal weighted sum of power required in host signal modification and the MMSE in the modified host signal reconstruction for all weights and all message rates. When specialized to the zero-rate case, our results provide the tightest known lower bounds on the asymptotic costs for the vector version of a famous open problem in decentralized control: the Witsenhausen counterexample. Numerically, this tighter bound helps us characterize the asymptotically optimal costs for the vector Witsenhausen problem to within a factor of 1.3 for all problem parameters, improving on the earlier best known bound of 2.

Information Embedding and the Triple Role of Control

In order to minimize the transmit power in a communication system, it is well known that it is optimal to operate close to the channel capacity. Recently derived fundamental limits show that this strategy is not optimal when the goal is to minimize total power consumption, i.e., transmit as well as processing power, because of high decoding complexity close to capacity. This paper investigates how close LDPC codes with corresponding message-passing decoders get to the fundamental limits. We focus on some one-bit decoding algorithms, namely Gallager-A and Gallager-B, to decode LDPC codes. We compute the minimum total (transmit and decoding) power consumed using these two decoding algorithms for two decoding models: first where only the computational nodes consume all the power (the “Node Model”), and next where wires consume all the power (the “Wire Model”). For each model and each decoding algorithm, we compare this minimum total power with the (transmit) power required for uncoded transmission and with fundamental lower bounds on power consumption for these models. In some cases, we observe that the transmit power needs to increase unboundedly in order for the total power to be asymptotically (as P e → 0) smaller than that for uncoded transmission. We also observe that the power consumed in the Wire Model asymptotically dominates the power consumed in the Node Model, which suggests the importance of characterizing the wiring complexity of decoding, and not just the number of operations.

How far are LDPC codes from fundamental limits on total power consumption

Standard human EEG systems based on spatial Nyquist estimates suggest that 20-30 mm electrode spacing suffices to capture neural signals on the scalp, but recent studies posit that increasing sensor density can provide higher resolution neural information. Here, we compared "super-Nyquist" density EEG ("SND") with Nyquist density ("ND") arrays for assessing the spatiotemporal aspects of early visual processing. EEG was measured from 128 electrodes arranged over occipitotemporal brain regions (14 mm spacing) while participants viewed flickering checkerboard stimuli. Analyses compared SND with ND-equivalent subsets of the same electrodes. Frequency-tagged stimuli were classified more accurately with SND than ND arrays in both the time and the frequency domains. Representational similarity analysis revealed that a computational model of V1 correlated more highly with the SND than the ND array. Overall, SND EEG captured more neural information from visual cortex, arguing for increased development of this approach in basic and translational neuroscience.

/pdf/very-high-density-eeg-elucidates-spatiotemporal-aspects-of-md4ut36q1u.pdf

Very high density EEG elucidates spatiotemporal aspects of early visual processing

Performing deep-brain stimulation (DBS) noninva-sively can have a drastic impact on neuroscience and clinical care. A recent work suggests that such noninvasive DBS is feasible using “Temporal Interference” (TI) stimulation, where two interfering high-frequency sinusoidal currents produce electrical stimulation only when the amplitudes of the sinusoids are approximately equal. While electrical stimulation using TI has been studied for decades, the understanding of its fundamental neurobiological mechanisms is still poor. In order to establish a better understanding of TI stimulation, a theoretical and computational approach is needed. In this work, we adopt such an approach using a computational model that is a sea of single neurons inside a spherical head. We demonstrate that the classical Hodgkin-Huxley squid neurons and some mammalian cells do exhibit TI stimulation in this model. We also observe that some mammalian neurons do not exhibit TI stimulation, suggesting that deeper studies are needed to understand if TI stimulation works with all neuron-types, or if it is network mechanisms (instead of single neuron dynamics) that enable TI stimulation.

Pulkit Grover

Papers

CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors.

Information Embedding and the Triple Role of Control

How far are LDPC codes from fundamental limits on total power consumption

Very high density EEG elucidates spatiotemporal aspects of early visual processing

Do single neuron models exhibit temporal interference stimulation