Showing papers by "Pulkit Grover published in 2019"

“Short-Dot”: Computing Large Linear Transforms Distributedly Using Coded Short Dot Products

Abstract: We consider the problem of computing a matrix-vector product $Ax$ using a set of $P$ parallel or distributed processing nodes prone to “straggling,” ie , unpredictable delays Every processing node can access only a fraction $({s}/{N})$ of the $N$ -length vector $x$ , and all processing nodes compute an equal number of dot products We propose a novel error correcting code-that we call “Short-Dot”–that introduces redundant, shorter dot products such that only a subset of the nodes’ outputs are sufficient to compute $Ax$ To address the problem of straggling in computing matrix-vector products, prior work uses replication or erasure coding to encode parts of the matrix $A$ , but the length of the dot products computed at each processing node is still $N$ The key novelty in our work is that instead of computing the long dot products as required in the original matrix-vector product, we construct a larger number of redundant and short dot products that only require a fraction of $x$ to be accessed during the computation Short-Dot is thus useful in a communication-constrained scenario as it allows for only a fraction of $x$ to be accessed by each processing node Further, we show that in the particular regime where the number of available processing nodes is greater than the total number of dot products, Short-Dot has lower expected computation time under straggling under an exponential model compared to existing strategies, eg replication, in a scaling sense We also derive fundamental limits on the trade-off between the length of the dot products and the recovery threshold, ie, the required number of processing nodes, showing that Short-Dot is near-optimal

Coded Elastic Computing

Abstract: Cloud providers have recently introduced new offerings whereby spare computing resources are accessible at discounts compared to on-demand computing. Exploiting such opportunity is challenging inasmuch as such resources are accessed with low-priority and therefore can elastically leave (through preemption) and join the computation at any time. In this paper, we design a new technique called coded elastic computing enabling distributed computations over elastic resources. The proposed technique allows machines to leave the computation without sacrificing the algorithm-level performance, and, at the same time, flexibly reduce the workload at existing machines when new ones join the computation. Leveraging coded redundancy, our approach is able to achieve similar computational cost as the original (uncoded) method when all machines are present; the cost gracefully increases when machines are preempted and reduces when machines join. The performance of the proposed technique is evaluated on matrix-vector multiplication and linear regression tasks, and shows improvements over existing techniques.

Cortical Hyper-Excitability in Migraine in Response to Chromatic Patterns.

Abstract: Objective Individuals with migraine exhibit heightened sensitivity to visual input that continues beyond their migraine episodes. However, the contribution of color to visual sensitivity, and how it relates to neural activity, has largely been unexplored in these individuals. Background Previously, it has been shown that, in non-migraine individuals, patterns with greater chromaticity separation evoked greater cortical activity, regardless of hue, even when colors were isoluminant. Therefore, to investigate whether individuals with migraine experienced increased visual sensitivity, we compared the behavioral and neural responses to chromatic patterns of increasing separation in migraine and non-migraine individuals. Methods Seventeen individuals with migraine (12 with aura) and 18 headache-free controls viewed pairs of colored horizontal grating patterns that varied in chromaticity separation. Color pairs were either blue-green, red-green, or red-blue. Participants rated the discomfort of the gratings and electroencephalogram was recorded simultaneously. Results Both groups showed increased discomfort ratings and larger N1/N2 event-related potentials (ERPs) with greater chromaticity separation, which is consistent with increased cortical excitability. However, individuals with migraine rated gratings as being disproportionately uncomfortable and exhibited greater effects of chromaticity separation in ERP amplitude across occipital and parietal electrodes. Ratings of discomfort and ERPs were smaller in response to the blue-green color pairs than the red-green and red-blue gratings, but this was to an equivalent degree across the 2 groups. Conclusions Together, these findings indicate that greater chromaticity separation increases neural excitation, and that this effect is heightened in migraine, consistent with the theory that hyper-excitability of the visual system is a key signature of migraine.

CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors.

Abstract: This work proposes the first strategy to make distributed training of neural networks resilient to computing errors, a problem that has remained unsolved despite being first posed in 1956 by von Neumann. He also speculated that the efficiency and reliability of the human brain is obtained by allowing for low power but error-prone components with redundancy for error-resilience. It is surprising that this problem remains open, even as massive artificial neural networks are being trained on increasingly low-cost and unreliable processing units. Our coding-theory-inspired strategy, "CodeNet," solves this problem by addressing three challenges in the science of reliable computing: (i) Providing the first strategy for error-resilient neural network training by encoding each layer separately; (ii) Keeping the overheads of coding (encoding/error-detection/decoding) low by obviating the need to re-encode the updated parameter matrices after each iteration from scratch. (iii) Providing a completely decentralized implementation with no central node (which is a single point of failure), allowing all primary computational steps to be error-prone. We theoretically demonstrate that CodeNet has higher error tolerance than replication, which we leverage to speed up computation time. Simultaneously, CodeNet requires lower redundancy than replication, and equal computational and communication costs in scaling sense. We first demonstrate the benefits of CodeNet in reducing expected computation time over replication when accounting for checkpointing. Our experiments show that CodeNet achieves the best accuracy-runtime tradeoff compared to both replication and uncoded strategies. CodeNet is a significant step towards biologically plausible neural network training, that could hold the key to orders of magnitude efficiency improvements.

An Algorithm for Automated, Noninvasive Detection of Cortical Spreading Depolarizations Based on EEG Simulations

Abstract: Objective: We present a novel signal processing algorithm for automated, noninvasive detection of cortical spreading depolarizations (CSDs) using electroencephalography (EEG) signals and validate the algorithm on simulated EEG signals. CSDs are waves of neurochemical changes that suppress the neuronal activity as they propagate across the brain's cortical surface. CSDs are believed to mediate secondary brain damage after brain trauma and cerebrovascular diseases like stroke. We address the following two key challenges in detecting CSDs from EEG signals: i) attenuation and loss of high spatial resolution information; and ii) cortical folds, which complicate tracking CSD waves. Methods: Our algorithm detects and tracks “wavefronts” of a CSD wave, and stitch together data across space and time to make a detection. To test our algorithm, we provide different models of CSD waves, including different widths of CSD suppressions and different patterns, and use them to simulate scalp EEG signals using head models of four subjects. Results and conclusion: Our results suggest that low-density EEG grids (40 electrodes) can detect CSD widths of 1.1 cm on average, while higher density EEG grids (340 electrodes) can detect CSD patterns as thin as 0.43 cm (less than minimum widths reported in prior works), among which single-gyrus CSDs are the hardest to detect because of their small suppression area. Significance: The proposed algorithm is a first step toward noninvasive, automated detection of CSDs, which can help in reducing secondary brain damages.

How should we define Information Flow in Neural Circuits

Abstract: We develop a theoretical framework for defining information flow in neural circuits, within the context of "eventrelated" experimental paradigms in neuroscience. Here, a neural circuit is modeled as a directed graph, with "clocked" nodes that send transmissions to each other along the edges of the graph at discrete points in time. We are interested in a definition that captures the flow of "stimulus"-related information, and which guarantees a continuous information path between appropriately defined inputs and outputs in the directed graph. Prior measures, including those based on Granger Causality and Directed Information, fail to provide clear assumptions and guarantees about when they correctly reflect stimulus-related information flow, due to the absence of a theoretical foundation with a mathematical definition. We take a methodical approach— iterating through candidate definitions and counterexamples— to arrive at a definition for information flow that is based on conditional mutual information, and which satisfies desirable properties, including the existence of information paths.

Information Flow in Computational Systems

Abstract: We develop a theoretical framework for defining and identifying flows of information in computational systems. Here, a computational system is assumed to be a directed graph, with "clocked" nodes that send transmissions to each other along the edges of the graph at discrete points in time. We are interested in a definition that captures the dynamic flow of information about a specific message, and which guarantees an unbroken "information path" between appropriately defined inputs and outputs in the directed graph. Prior measures, including those based on Granger Causality and Directed Information, fail to provide clear assumptions and guarantees about when they correctly reflect information flow about a message. We take a systematic approach---iterating through candidate definitions and counterexamples---to arrive at a definition for information flow that is based on conditional mutual information, and which satisfies desirable properties, including the existence of information paths. Finally, we describe how information flow might be detected in a noiseless setting, and provide an algorithm to identify information paths on the time-unrolled graph of a computational system.

Silence Localization

Abstract: We present a novel algorithm for non-invasive localization of regions of "silence" in the brain. "Silences" are sources in the brain without any electrical activity due to the ischemic conditions or lesions in stroke, traumatic brain injuries (TBIs) and hemorrhages. We aim to turn the widely used electroencephalography (EEG) systems into silence detectors. Our algorithm, in a nutshell, estimates the silence region by determining the contribution of each source (dipole) in the recorded signals and detects the sources with a reduced contribution as silences in the brain. In simulations on real-head MRI models, our algorithm detects different sizes of silent regions, ranging from the smallest ones (single-source silence) to the large ones, and it reduces bias by a factor of ~4 over classical source localization algorithms such as multiple signal classification approach (MUSIC) and minimum norm estimation (MNE), appropriately modified for recovering silences.

Systematic Matrix Multiplication Codes

Abstract: The problem of computing distributed matrix multiplication reliably has been of immense interest for several decades. Recently, it was shown that Polynomial codes achieve the theoretically minimum recovery bandwidth. However, existing constructions for Polynomial codes are nonsystematic, which can impose substantial overhead in distributed computing. In this paper, we propose two different systematic code constructions that achieve the same recovery bandwidth as Polynomial codes. First uses a random coding argument, and the second is polynomial-based, but uses bivariate instead of originally used univariate polynomials. We show that the proposed constructions are communication optimal with high probability.

Energy-Adaptive Error Correcting for Dynamic and Heterogeneous Networks

Abstract: In an era of ever-increasing dynamicity and heterogeneity of wireless networks, energy is fast becoming the most constrained resource. First, we review recent studies that suggest that using one single error-correcting code (ECC) designed to meet the worst case requirement is inefficient in terms of energy consumption when there are many heterogeneous nodes in the network. These works extend the classical Shannon theory and incorporate circuit energy and signal transmit energy to optimize total energy/power consumption of today’s communication systems. Then, we survey recent work on designing adaptive ECCs to operate energy efficiently even in the presence of extremely large heterogeneity in requirements and conditions. Two constructions of energy-adaptive codes are summarized: energy-adaptive low-density parity-check (LDPC) codes and energy-adaptive polar codes. These constructions have shown theoretically and empirically that having adaptivity in code design can save substantial energy, especially when the network has very diverse communication scenarios. Finally, we suggest a few possible applications where energy-adaptive codes can be employed and outline interesting future directions and challenges.

Cell-type Selective Stimulation of Neurons Based on Single Neuron Models

Abstract: Selectively stimulating neuron types within the brain can enable new treatment possibilities for neurological disorders and feedback in brain-machine interfaces. Prior computational work has shown that by choosing a sinusoidal signal with appropriate amplitude and frequency, one can obtain one-directional stimulation, i.e., one can stimulate a mammalian inhibitory neuron without stimulating an excitatory neuron. However, bidirectional selectivity is not achievable using just sinusoidal inputs. In this work, which is also computational, we design novel current waveforms to achieve this bidirectional selectivity. To do so, we explicitly exploit the non-linearity of neuronal membrane potential in response to stimulating currents. These current waveforms are able to stimulate either of the two neuron-types without stimulating the other. Further, we are also able to design a waveform which stimulates both neurons. Moreover, we can ensure a relatively high firing rate (~100 Hz) when a neuron-type is targeted for stimulation.

Feedback-based Electrode Rehydration for High Quality, Long Term, Noninvasive Biopotential Measurements and Current Delivery

Abstract: Diagnoses and treatments of neurological disorders rely heavily on non-invasive measurement of bio-potentials, or non-invasive injection of currents into the body, by placing electrodes on the skin. A major impedance in using these techniques for continuous, long-term monitoring is the deterioration of electrode performance over time. Long term operation of electrodes requires that they should continue to have low electrode-skin impedance, even in presence of hair, for extended periods of time. There are currently no electrode systems that satisfy all these requirements. Our work proposes and demonstrates a proof-of-concept, feedback-based electrode impedance monitoring and control system. We use a sponge electrode, housed in a custom-designed casing that is connected to a low-speed peristaltic pump. The pump releases a small amount of saline solution when the electrode-skin impedance crosses a threshold. Our system succeeds in keeping the impedance below a target impedance for at least 20 hours on bench-top experiments. On human participant experiments, our results from 90 minutes of continuous measurement show that our saline electrodes outperform conventional clinical wet electrodes in the presence of hair.

Robust Molecular Dynamics Simulations Using Coded FFT Algorithm

Abstract: As error/failure rates in supercomputers are projected to grow, computationally intensive scientific applications that lever-age large-scale parallelization will suffer from the increased error rate. In this work, we apply "coded computing" to protein folding simulations in an error-prone environment. We implemented the fast Fourier Poisson method for solving electrostatic equations at each time step of the simulation, and we utilize coded FFT algorithm to protect the compute-intensive FFT algorithm from soft errors. Through experiments on Amazon AWS, we showed that coded protein folding can be implemented with less than 10% overhead in total simulation time, and also showed that coded computing approach is faster than classical checkpointing method when the error rate is high.