Ahmed T. Elthakeb

Bio: Ahmed T. Elthakeb is an academic researcher from University of California, San Diego. The author has contributed to research in topics: Quantization (signal processing) & Artificial neural network. The author has an hindex of 6, co-authored 16 publications receiving 219 citations. Previous affiliations of Ahmed T. Elthakeb include American University in Cairo.

High Density Individually Addressable Nanowire Arrays Record Intracellular Activity from Primary Rodent and Human Stem Cell Derived Neurons

Abstract: We report a new hybrid integration scheme that offers for the first time a nanowire-on-lead approach, which enables independent electrical addressability, is scalable, and has superior spatial resolution in vertical nanowire arrays. The fabrication of these nanowire arrays is demonstrated to be scalable down to submicrometer site-to-site spacing and can be combined with standard integrated circuit fabrication technologies. We utilize these arrays to perform electrophysiological recordings from mouse and rat primary neurons and human induced pluripotent stem cell (hiPSC)-derived neurons, which revealed high signal-to-noise ratios and sensitivity to subthreshold postsynaptic potentials (PSPs). We measured electrical activity from rodent neurons from 8 days in vitro (DIV) to 14 DIV and from hiPSC-derived neurons at 6 weeks in vitro post culture with signal amplitudes up to 99 mV. Overall, our platform paves the way for longitudinal electrophysiological experiments on synaptic activity in human iPSC based dis...

ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks

Abstract: Deep Neural Networks (DNNs) typically require massive amount of computation resource in inference tasks for computer vision applications. Quantization can significantly reduce DNN computation and storage by decreasing the bitwidth of network encodings. Recent research affirms that carefully selecting the quantization levels for each layer can preserve the accuracy while pushing the bitwidth below eight bits. However, without arduous manual effort, this deep quantization can lead to significant accuracy loss, leaving it in a position of questionable utility. As such, deep quantization opens a large hyper-parameter space (bitwidth of the layers), the exploration of which is a major challenge. We propose a systematic approach to tackle this problem, by automating the process of discovering the quantization levels through an end-to-end deep reinforcement learning framework (ReLeQ). We adapt policy optimization methods to the problem of quantization, and focus on finding the best design decisions in choosing the state and action spaces, network architecture and training framework, as well as the tuning of various hyperparamters. We show how ReLeQ can balance speed and quality, and provide an asymmetric general solution for quantization of a large variety of deep networks (AlexNet, CIFAR-10, LeNet, MobileNet-V1, ResNet-20, SVHN, and VGG-11) that virtually preserves the accuracy (=< 0.3% loss) while minimizing the computation and storage cost. With these DNNs, ReLeQ enables conventional hardware to achieve 2.2x speedup over 8-bit execution. Similarly, a custom DNN accelerator achieves 2.0x speedup and energy reduction compared to 8-bit runs. These encouraging results mark ReLeQ as the initial step towards automating the deep quantization of neural networks.

Scaling Effects on the Electrochemical Performance of poly(3,4-ethylenedioxythiophene (PEDOT), Au, and Pt for Electrocorticography Recording

Abstract: Reduced contact size would permit higher resolution cortical recordings, but the effects of diameter on crucial recording and stimulation properties are poorly understood. Here, the first systematic study of scaling effects on the electrochemical properties of metallic Pt and Au and organic poly(3,4-ethylenedioxythiophene):polystyrene sulfonate (PEDOT:PSS) electrodes is presented. PEDOT:PSS exhibits better faradaic charge transfer and capacitive charge coupling than metal electrodes, and these characteristics lead to improved electrochemical performance and reduced noise at smaller electrode diameters. PEDOT:PSS coating reduces the impedances of metallic electrodes by up to 18x for diameters <200 µm, but has no effect for millimeter scale contacts due to the dominance of series resistances. Therefore, the performance gains are especially significant at smaller diameters and lower frequencies essential for recording cognitive and pathological activities. Additionally, the overall reduced noise of the PEDOT:PSS electrodes enables a lower noise floor for recording action potentials. These results permit quantitative optimization of contact material and diameter for different electrocorticography applications.

ReLeQ : A Reinforcement Learning Approach for Automatic Deep Quantization of Neural Networks

Abstract: Deep Quantization (below eight bits) can significantly reduce the DNN computation and storage by decreasing the bitwidth of network encodings. However, without arduous manual effort, this deep quantization can lead to significant accuracy loss, leaving it in a position of questionable utility. We propose a systematic approach to tackle this problem, by automating the process of discovering the bitwidths through an end-to-end deep reinforcement learning framework (ReLeQ). This framework utilizes the sample efficiency of proximal policy optimization to explore the exponentially large space of possible assignment of the bitwidths to the layers. We show how ReLeQ can balance speed and quality, and provide a heterogeneous bitwidth assignment for quantization of a large variety of deep networks with minimal accuracy loss ($\leq$ ≤ 0.3% loss) while minimizing the computation and storage costs. With these DNNs, ReLeQ enables conventional hardware and custom DNN accelerator to achieve $~2.2\times$ 2 . 2 × speedup over 8-bit execution.

Not All Features Are Equal: Discovering Essential Features for Preserving Prediction Privacy

Abstract: When receiving machine learning services from the cloud, the provider does not need to receive all features; in fact, only a subset of the features are necessary for the target prediction task. Discerning this subset is the key problem of this work. We formulate this problem as a gradient-based perturbation maximization method that discovers this subset in the input feature space with respect to the functionality of the prediction model used by the provider. After identifying the subset, our framework, Cloak, suppresses the rest of the features using utility-preserving constant values that are discovered through a separate gradient-based optimization process. We show that Cloak does not necessarily require collaboration from the service provider beyond its normal service, and can be applied in scenarios where we only have black-box access to the service provider’s model. We theoretically guarantee that Cloak’s optimizations reduce the upper bound of the Mutual Information (MI) between the data and the sifted representations that are sent out. Experimental results show that Cloak reduces the mutual information between the input and the sifted representations by 85.01% with only negligible reduction in utility (1.42%). In addition, we show that Cloak greatly diminishes adversaries’ ability to learn and infer non-conducive features.

The Monte Carlo method for semiconductor device simulation

Abstract: There can be little doubt that the Monte Carlo method for semiconductor device simulation has enormous power as a research tool. It represents a detailed physical model of the semiconductor material(s), and provides a high degree of insight into the microscopic transport processes. However, if the authority ascribed to Monte Carlo models of devices at 1/spl mu/m feature size is to be maintained for devices below O.1/spl mu/m, modelling of the fundamental physics must be further improved. And if the Monte Carlo method is to be successful as a semiconductor device design tool, the device model must be made more realistic. Success in the industrial sector depends on this, but also on achieving fast run-times optimisation - where the scope and need for ingenuity is now greatest.

Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks

Abstract: Hardware-friendly network quantization (e.g., binary/uniform quantization) can efficiently accelerate the inference and meanwhile reduce memory consumption of the deep neural networks, which is crucial for model deployment on resource-limited devices like mobile phones. However, due to the discreteness of low-bit quantization, existing quantization methods often face the unstable training process and severe performance degradation. To address this problem, in this paper we propose Differentiable Soft Quantization (DSQ) to bridge the gap between the full-precision and low-bit networks. DSQ can automatically evolve during training to gradually approximate the standard quantization. Owing to its differentiable property, DSQ can help pursue the accurate gradients in backward propagation, and reduce the quantization loss in forward process with an appropriate clipping range. Extensive experiments over several popular network structures show that training low-bit neural networks with DSQ can consistently outperform state-of-the-art quantization methods. Besides, our first efficient implementation for deploying 2 to 4-bit DSQ on devices with ARM architecture achieves up to 1.7× speed up, compared with the open-source 8-bit high-performance inference framework NCNN [31].

A nanoelectrode array for obtaining intracellular recordings from thousands of connected neurons.

Abstract: Current electrophysiological or optical techniques cannot reliably perform simultaneous intracellular recordings from more than a few tens of neurons. Here we report a nanoelectrode array that can simultaneously obtain intracellular recordings from thousands of connected mammalian neurons in vitro. The array consists of 4,096 platinum-black electrodes with nanoscale roughness fabricated on top of a silicon chip that monolithically integrates 4,096 microscale amplifiers, configurable into pseudocurrent-clamp mode (for concurrent current injection and voltage recording) or into pseudovoltage-clamp mode (for concurrent voltage application and current recording). We used the array in pseudovoltage-clamp mode to measure the effects of drugs on ion-channel currents. In pseudocurrent-clamp mode, the array intracellularly recorded action potentials and postsynaptic potentials from thousands of neurons. In addition, we mapped over 300 excitatory and inhibitory synaptic connections from more than 1,700 neurons that were intracellularly recorded for 19 min. This high-throughput intracellular-recording technology could benefit functional connectome mapping, electrophysiological screening and other functional interrogations of neuronal networks. An electronic interface with 4,096 electrodes can intracellularly record postsynaptic potentials and action potentials from thousands of connected mammalian neurons in vitro.

Recent Advances in Materials, Devices, and Systems for Neural Interfaces.

Abstract: Technologies capable of establishing intimate, long-lived optical/electrical interfaces to neural systems will play critical roles in neuroscience research and in the development of nonpharmacological treatments for neurological disorders. The development of high-density interfaces to 3D populations of neurons across entire tissue systems in living animals, including human subjects, represents a grand challenge for the field, where advanced biocompatible materials and engineered structures for electrodes and light emitters will be essential. This review summarizes recent progress in these directions, with an emphasis on the most promising demonstrated concepts, materials, devices, and systems. The article begins with an overview of electrode materials with enhanced electrical and/or mechanical performance, in forms ranging from planar films, to micro/nanostructured surfaces, to 3D porous frameworks and soft composites. Subsequent sections highlight integration with active materials and components for multiplexed addressing, local amplification, wireless data transmission, and power harvesting, with multimodal operation in soft, shape-conformal systems. These advances establish the foundations for scalable architectures in optical/electrical neural interfaces of the future, where a blurring of the lines between biotic and abiotic systems will catalyze profound progress in neuroscience research and in human health/well-being.

Bioinspired bio-voltage memristors

Abstract: Memristive devices are promising candidates to emulate biological computing. However, the typical switching voltages (0.2-2 V) in previously described devices are much higher than the amplitude in biological counterparts. Here we demonstrate a type of diffusive memristor, fabricated from the protein nanowires harvested from the bacterium Geobacter sulfurreducens, that functions at the biological voltages of 40-100 mV. Memristive function at biological voltages is possible because the protein nanowires catalyze metallization. Artificial neurons built from these memristors not only function at biological action potentials (e.g., 100 mV, 1 ms) but also exhibit temporal integration close to that in biological neurons. The potential of using the memristor to directly process biosensing signals is also demonstrated. Designing energy efficient systems capable to directly process signals at biological voltages remains a challenge. Here, the authors propose a bio-compatible memristor device based on protein-nanowire dielectric, harvested from the bacterium Geobactor sulfurreducens, working at biological voltages.