Showing papers by "IBM published in 2015"

GraRep: Learning Graph Representations with Global Structural Information

Abstract: In this paper, we present {GraRep}, a novel model for learning vertex representations of weighted graphs. This model learns low dimensional vectors to represent vertices appearing in a graph and, unlike existing work, integrates global structural information of the graph into the learning process. We also formally analyze the connections between our work and several previous research efforts, including the DeepWalk model of Perozzi et al. as well as the skip-gram model with negative sampling of Mikolov et al. We conduct experiments on a language network, a social network as well as a citation network and show that our learned global representations can be effectively used as features in tasks such as clustering, classification and visualization. Empirical results demonstrate that our representation significantly outperforms other state-of-the-art methods in such tasks.

Deep Learning with Limited Numerical Precision

Abstract: Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network's behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.

Deep Learning with Limited Numerical Precision

Abstract: Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of lowprecision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network's behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.

TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip

Abstract: The new era of cognitive computing brings forth the grand challenge of developing systems capable of processing massive amounts of noisy multisensory data. This type of intelligent computing poses a set of constraints, including real-time operation, low-power consumption and scalability, which require a radical departure from conventional system design. Brain-inspired architectures offer tremendous promise in this area. To this end, we developed TrueNorth, a 65 mW real-time neurosynaptic processor that implements a non-von Neumann, low-power, highly-parallel, scalable, and defect-tolerant architecture. With 4096 neurosynaptic cores, the TrueNorth chip contains 1 million digital neurons and 256 million synapses tightly interconnected by an event-driven routing infrastructure. The fully digital 5.4 billion transistor implementation leverages existing CMOS scaling trends, while ensuring one-to-one correspondence between hardware and software. With such aggressive design metrics and the TrueNorth architecture breaking path with prevailing architectures, it is clear that conventional computer-aided design (CAD) tools could not be used for the design. As a result, we developed a novel design methodology that includes mixed asynchronous–synchronous circuits and a complete tool flow for building an event-driven, low-power neurosynaptic chip. The TrueNorth chip is fully configurable in terms of connectivity and neural parameters to allow custom configurations for a wide range of cognitive and sensory perception applications. To reduce the system’s communication energy, we have adapted existing application-agnostic very large-scale integration CAD placement tools for mapping logical neural networks to the physical neurosynaptic core locations on the TrueNorth chips. With that, we have successfully demonstrated the use of TrueNorth-based systems in multiple applications, including visual object recognition, with higher performance and orders of magnitude lower power consumption than the same algorithms run on von Neumann architectures. The TrueNorth chip and its tool flow serve as building blocks for future cognitive systems, and give designers an opportunity to develop novel brain-inspired architectures and systems based on the knowledge obtained from this paper.

An updated performance comparison of virtual machines and Linux containers

Abstract: Cloud computing makes extensive use of virtual machines because they permit workloads to be isolated from one another and for the resource usage to be somewhat controlled. In this paper, we explore the performance of traditional virtual machine (VM) deployments, and contrast them with the use of Linux containers. We use KVM as a representative hypervisor and Docker as a container manager. Our results show that containers result in equal or better performance than VMs in almost all cases. Both VMs and containers require tuning to support I/Ointensive applications. We also discuss the implications of our performance results for future cloud architectures.

The Quest for Scalable Blockchain Fabric: Proof-of-Work vs. BFT Replication

Abstract: Bitcoin cryptocurrency demonstrated the utility of global consensus across thousands of nodes, changing the world of digital transactions forever. In the early days of Bitcoin, the performance of its probabilistic proof-of-work (PoW) based consensus fabric, also known as blockchain, was not a major issue. Bitcoin became a success story, despite its consensus latencies on the order of an hour and the theoretical peak throughput of only up to 7 transactions per second.

Supervised Discrete Hashing

Abstract: Recently, learning based hashing techniques have attracted broad research interests because they can support efficient storage and retrieval for high-dimensional data such as images, videos, documents, etc. However, a major difficulty of learning to hash lies in handling the discrete constraints imposed on the pursued hash codes, which typically makes hash optimizations very challenging (NP-hard in general). In this work, we propose a new supervised hashing framework, where the learning objective is to generate the optimal binary hash codes for linear classification. By introducing an auxiliary variable, we reformulate the objective such that it can be solved substantially efficiently by employing a regularization algorithm. One of the key steps in this algorithm is to solve a regularization sub-problem associated with the NP-hard binary optimization. We show that the sub-problem admits an analytical solution via cyclic coordinate descent. As such, a high-quality discrete solution can eventually be obtained in an efficient computing manner, therefore enabling to tackle massive datasets. We evaluate the proposed approach, dubbed Supervised Discrete Hashing (SDH), on four large image datasets and demonstrate its superiority to the state-of-the-art hashing methods in large-scale image retrieval.

Supervised Discrete Hashing

Abstract: Recently, learning based hashing techniques have attracted broad research interests because they can support efficient storage and retrieval for high-dimensional data such as images, videos, documents, etc. However, a major difficulty of learning to hash lies in handling the discrete constraints imposed on the pursued hash codes, which typically makes hash optimizations very challenging (NP-hard in general). In this work, we propose a new supervised hashing framework, where the learning objective is to generate the optimal binary hash codes for linear classification. By introducing an auxiliary variable, we reformulate the objective such that it can be solved substantially efficiently by employing a regularization algorithm. One of the key steps in this algorithm is to solve a regularization sub-problem associated with the NP-hard binary optimization. We show that the sub-problem admits an analytical solution via cyclic coordinate descent. As such, a high-quality discrete solution can eventually be obtained in an efficient computing manner, therefore enabling to tackle massive datasets. We evaluate the proposed approach, dubbed Supervised Discrete Hashing (SDH), on four large image datasets and demonstrate its superiority to the state-of-the-art hashing methods in large-scale image retrieval.

Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element

Abstract: Using two phase-change memory devices per synapse, a three-layer perceptron network with 164 885 synapses is trained on a subset (5000 examples) of the MNIST database of handwritten digits using a backpropagation variant suitable for nonvolatile memory (NVM) + selector crossbar arrays, obtaining a training (generalization) accuracy of 82.2% (82.9%). Using a neural network simulator matched to the experimental demonstrator, extensive tolerancing is performed with respect to NVM variability, yield, and the stochasticity, linearity, and asymmetry of the NVM-conductance response. We show that a bidirectional NVM with a symmetric, linear conductance response of high dynamic range is capable of delivering the same high classification accuracies on this problem as a conventional, software-based implementation of this same network.

A new spin on magnetic memories

Abstract: Solid-state memory devices with all-electrical read and write operations might lead to faster, cheaper information storage.

Solvating additives drive solution-mediated electrochemistry and enhance toroid growth in non-aqueous Li–O2 batteries

Abstract: Given their high theoretical specific energy, lithium-oxygen batteries have received enormous attention as possible alternatives to current state-of-the-art rechargeable Li-ion batteries. However, the maximum discharge capacity in non-aqueous lithium-oxygen batteries is limited to a small fraction of its theoretical value due to the build-up of insulating lithium peroxide (Li₂O₂), the battery's primary discharge product. The discharge capacity can be increased if Li₂O₂ forms as large toroidal particles rather than as a thin conformal layer. Here, we show that trace amounts of electrolyte additives, such as H₂O, enhance the formation of Li₂O₂ toroids and result in significant improvements in capacity. Our experimental observations and a growth model show that the solvating properties of the additives prompt a solution-based mechanism that is responsible for the growth of Li₂O₂ toroids. We present a general formalism describing an additive's tendency to trigger the solution process, providing a rational design route for electrolytes that afford larger lithium-oxygen battery capacities.

Memory on the racetrack

Abstract: Racetrack memory stores digital data in the magnetic domain walls of nanowires. This technology promises to yield information storage devices with high reliability, performance and capacity.

Domain-wall velocities of up to 750 m s −1 driven by exchange-coupling torque in synthetic antiferromagnets

Abstract: Racetrack memories made from synthetic antiferromagnetic structures with almost zero net magnetization allow for fast current-driven motion of domain walls.

Heterogeneous Network Embedding via Deep Architectures

Abstract: Data embedding is used in many machine learning applications to create low-dimensional feature representations, which preserves the structure of data points in their original space. In this paper, we examine the scenario of a heterogeneous network with nodes and content of various types. Such networks are notoriously difficult to mine because of the bewildering combination of heterogeneous contents and structures. The creation of a multidimensional embedding of such data opens the door to the use of a wide variety of off-the-shelf mining techniques for multidimensional data. Despite the importance of this problem, limited efforts have been made on embedding a network of scalable, dynamic and heterogeneous data. In such cases, both the content and linkage structure provide important cues for creating a unified feature representation of the underlying network. In this paper, we design a deep embedding algorithm for networked data. A highly nonlinear multi-layered embedding function is used to capture the complex interactions between the heterogeneous data in a network. Our goal is to create a multi-resolution deep embedding function, that reflects both the local and global network structures, and makes the resulting embedding useful for a variety of data mining tasks. In particular, we demonstrate that the rich content and linkage information in a heterogeneous network can be captured by such an approach, so that similarities among cross-modal data can be measured directly in a common embedding space. Once this goal has been achieved, a wide variety of data mining problems can be solved by applying off-the-shelf algorithms designed for handling vector representations. Our experiments on real-world network datasets show the effectiveness and scalability of the proposed algorithm as compared to the state-of-the-art embedding methods.

Demonstration of a quantum error detection code using a square lattice of four superconducting qubits

Abstract: The physical realization of a quantum computer requires built-in error-correcting codes that compensate the disruption of quantum information arising from noise. Here, the authors demonstrate a quantum error detection scheme for arbitrary single-qubit errors on a four superconducting qubit lattice.

Prediction of Weyl semimetal in orthorhombic MoTe2

Abstract: We investigate the orthorhombic phase $({T}_{d})$ of the layered transition-metal dichalcogenide ${\mathrm{MoTe}}_{2}$ as a Weyl semimetal candidate. ${\mathrm{MoTe}}_{2}$ exhibits four pairs of Weyl points lying slightly above $(\ensuremath{\sim}6\phantom{\rule{0.16em}{0ex}}\mathrm{meV})$ the Fermi energy in the bulk band structure. Different from its cousin ${\mathrm{WTe}}_{2}$, which was recently predicted to be a type-II Weyl semimetal, the spacing between each pair of Weyl points is found to be as large as 4% of the reciprocal lattice in ${\mathrm{MoTe}}_{2}$ (six times larger than that of ${\mathrm{WTe}}_{2}$). When projected onto the surface, the Weyl points are connected by Fermi arcs, which can be easily accessed by angle-resolved photoemission spectroscopy due to the large Weyl point separation. In addition, we show that the correlation effect or strain can drive ${\mathrm{MoTe}}_{2}$ from a type-II to a type-I Weyl semimetal.

Unraveling the Molecular Structures of Asphaltenes by Atomic Force Microscopy

Abstract: Petroleum is one of the most precious and complex molecular mixtures existing. Because of its chemical complexity, the solid component of crude oil, the asphaltenes, poses an exceptional challenge for structure analysis, with tremendous economic relevance. Here, we combine atomic-resolution imaging using atomic force microscopy and molecular orbital imaging using scanning tunnelling microscopy to study more than 100 asphaltene molecules. The complexity and range of asphaltene polycyclic aromatic hydrocarbons are established in detail. Identifying molecular structures provides a foundation to understand all aspects of petroleum science from colloidal structure and interfacial interactions to petroleum thermodynamics, enabling a first-principles approach to optimize resource utilization. Particularly, the findings contribute to a long-standing debate about asphaltene molecular architecture. Our technique constitutes a paradigm shift for the analysis of complex molecular mixtures, with possible applications ...

Role of transparency of platinum–ferromagnet interfaces in determining the intrinsic magnitude of the spin Hall effect

Abstract: The spin Hall effect induces spin currents in nonmagnetic layers, which can control the magnetization of neighbouring ferromagnets. The transparency of the interface is shown to strongly influence the efficiency of such manipulation.

Classifying Relations by Ranking with Convolutional Neural Networks

Abstract: Relation classification is an important semantic processing task for which state-ofthe-art systems still rely on costly handcrafted features. In this work we tackle the relation classification task using a convolutional neural network that performs classification by ranking (CR-CNN). We propose a new pairwise ranking loss function that makes it easy to reduce the impact of artificial classes. We perform experiments using the the SemEval-2010 Task 8 dataset, which is designed for the task of classifying the relationship between two nominals marked in a sentence. Using CRCNN, we outperform the state-of-the-art for this dataset and achieve a F1 of 84.1 without using any costly handcrafted features. Additionally, our experimental results show that: (1) our approach is more effective than CNN followed by a softmax classifier; (2) omitting the representation of the artificial class Other improves both precision and recall; and (3) using only word embeddings as input features is enough to achieve state-of-the-art results if we consider only the text between the two target nominals.

Automated analysis of free speech predicts psychosis onset in high-risk youths

Abstract: BACKGROUND/OBJECTIVES: Psychiatry lacks the objective clinical tests routinely used in other specializations Novel computerized methods to characterize complex behaviors such as speech could be used to identify and predict psychiatric illness in individuals AIMS: In this proof-of-principle study, our aim was to test automated speech analyses combined with Machine Learning to predict later psychosis onset in youths at clinical high-risk (CHR) for psychosis METHODS: Thirty-four CHR youths (11 females) had baseline interviews and were assessed quarterly for up to 25 years; five transitioned to psychosis Using automated analysis, transcripts of interviews were evaluated for semantic and syntactic features predicting later psychosis onset Speech features were fed into a convex hull classification algorithm with leave-one-subject-out cross-validation to assess their predictive value for psychosis outcome The canonical correlation between the speech features and prodromal symptom ratings was computed RESULTS: Derived speech features included a Latent Semantic Analysis measure of semantic coherence and two syntactic markers of speech complexity: maximum phrase length and use of determiners (eg, which) These speech features predicted later psychosis development with 100% accuracy, outperforming classification from clinical interviews Speech features were significantly correlated with prodromal symptoms CONCLUSIONS: Findings support the utility of automated speech analysis to measure subtle, clinically relevant mental state changes in emergent psychosis Recent developments in computer science, including natural language processing, could provide the foundation for future development of objective clinical tests for psychiatry

Automated Volumetry and Regional Thickness Analysis of Hippocampal Subfields and Medial Temporal Cortical Structures in Mild Cognitive Impairment

Abstract: We evaluate a fully automatic technique for labeling hippocampal subfields and cortical subregions in the medial temporal lobe in in vivo 3 Tesla MRI. The method performs segmentation on a T2-weighted MRI scan with 0.4 × 0.4 × 2.0 mm(3) resolution, partial brain coverage, and oblique orientation. Hippocampal subfields, entorhinal cortex, and perirhinal cortex are labeled using a pipeline that combines multi-atlas label fusion and learning-based error correction. In contrast to earlier work on automatic subfield segmentation in T2-weighted MRI [Yushkevich et al., 2010], our approach requires no manual initialization, labels hippocampal subfields over a greater anterior-posterior extent, and labels the perirhinal cortex, which is further subdivided into Brodmann areas 35 and 36. The accuracy of the automatic segmentation relative to manual segmentation is measured using cross-validation in 29 subjects from a study of amnestic mild cognitive impairment (aMCI) and is highest for the dentate gyrus (Dice coefficient is 0.823), CA1 (0.803), perirhinal cortex (0.797), and entorhinal cortex (0.786) labels. A larger cohort of 83 subjects is used to examine the effects of aMCI in the hippocampal region using both subfield volume and regional subfield thickness maps. Most significant differences between aMCI and healthy aging are observed bilaterally in the CA1 subfield and in the left Brodmann area 35. Thickness analysis results are consistent with volumetry, but provide additional regional specificity and suggest nonuniformity in the effects of aMCI on hippocampal subfields and MTL cortical subregions.

Opportunities and challenges in liquid cell electron microscopy

Abstract: BACKGROUND Transmission electron microscopy offers structural and compositional information with atomic resolution, but its use is restricted to thin, solid samples. Liquid samples, particularly those involving water, have been challenging because of the need to form a thin liquid layer that is stable within the microscope vacuum. Liquid cell electron microscopy is a developing technique that allows us to apply the powerful capabilities of the electron microscope to the imaging and analysis of liquid specimens. We can examine liquid-based processes in materials science and physics that are traditionally inaccessible to electron microscopy, and image biological structures at high resolution without the need for freezing or drying. The changes that occur inside batteries during operation, the attachment of atoms during the self-assembly of nanocrystals, and the structures of biological materials in liquid water are examples in which a microscopic view is providing unique insights. ADVANCES The difficulty of imaging water and other liquids was recognized from the earliest times in the development of transmission electron microscopy. Achieving a practical solution, however, required the use of modern microfabrication techniques to build liquid cells with thin but strong windows. Usually made of silicon nitride on a silicon support, these liquid cells perform two jobs: They separate the liquid from the microscope vacuum while also confining it into a layer that is thin enough for imaging with transmitted electrons. Additional functionality such as liquid flow, electrodes, or heating can be incorporated in the liquid cell. The first experiments to make use of modern liquid cells provided information on electrochemical deposition, nanomaterials synthesis, diffusion in liquids, and the structure of biological assemblies. Materials and processes now under study include corrosion, biomolecular structure, bubble dynamics, radiation effects, and biomineralization. New window materials such as graphene can improve resolution, and elemental analysis is possible by measuring energy loss or x-ray signals. Advances in electron optics and detectors, and the correlation of liquid cell microscopy data with probes such as fluorescence, have increased the range of information available from the sample. Because the equipment is not too expensive and works in existing electron microscopes, liquid cell microscopy programs have developed around the world. OUTLOOK Liquid cell electron microscopy is well positioned to explore new frontiers in electrochemistry and catalysis, nanomaterial growth, fluid physics, diffusion, radiation physics, geological and environmental processes involving clays and aerosols, complex biomaterials and polymers, and biological functions in aqueous environments. Continuing improvements in equipment and technique will allow materials and processes to be studied under different stimuli—for example, in extreme temperatures, during gas/liquid mixing, or in magnetic or electric fields. Correlative approaches that combine liquid cell electron microscopy with light microscope or synchrotron data promise a deeper study of chemical, electrochemical, and photochemical reactions; analytical electron microscopy will provide details of composition and chemical bonding in water; high-speed and aberration-corrected imaging extend the scales of the phenomena that can be examined. As liquid cell microscopy becomes more capable and quantitative, it promises the potential to extend into new areas, adopt advanced imaging modes such as holography, and perhaps even solve grand challenge problems such as the structure of the electrochemical double layer or molecular movements during biological processes.

NetVM: High Performance and Flexible Networking Using Virtualization on Commodity Platforms

Abstract: NetVM brings virtualization to the Network by enabling high bandwidth network functions to operate at near line speed, while taking advantage of the flexibility and customization of low cost commodity servers. NetVM allows customizable data plane processing capabilities such as firewalls, proxies, and routers to be embedded within virtual machines, complementing the control plane capabilities of Software Defined Networking. NetVM makes it easy to dynamically scale, deploy, and reprogram network functions. This provides far greater flexibility than existing purpose-built, sometimes proprietary hardware, while still allowing complex policies and full packet inspection to determine subsequent processing. It does so with dramatically higher throughput than existing software router platforms. NetVM is built on top of the KVM platform and Intel DPDK library. We detail many of the challenges we have solved such as adding support for high-speed inter-VM communication through shared huge pages and enhancing the CPU scheduler to prevent overheads caused by inter-core communication and context switching. NetVM allows true zero-copy delivery of data to VMs both for packet processing and messaging among VMs within a trust boundary. Our evaluation shows how NetVM can compose complex network functionality from multiple pipelined VMs and still obtain throughputs up to 10 Gbps, an improvement of more than 250% compared to existing techniques that use SR-IOV for virtualized networking.

Data augmentation for deep neural network acoustic modeling

Abstract: This paper investigates data augmentation for deep neural network acoustic modeling based on label-preserving transformations to deal with data sparsity. Two data augmentation approaches, vocal tract length perturbation (VTLP) and stochastic feature mapping (SFM), are investigated for both deep neural networks (DNNs) and convolutional neural networks (CNNs). The approaches are focused on increasing speaker and speech variations of the limited training data such that the acoustic models trained with the augmented data are more robust to such variations. In addition, a two-stage data augmentation scheme based on a stacked architecture is proposed to combine VTLP and SFM as complementary approaches. Experiments are conducted on Assamese and Haitian Creole, two development languages of the IARPA Babel program, and improved performance on automatic speech recognition (ASR) and keyword search (KWS) is reported.

Robust helical edge transport in gated InAs/GaSb bilayers.

Abstract: We have engineered electron-hole bilayers of inverted $\mathrm{InAs}/\mathrm{GaSb}$ quantum wells, using dilute silicon impurity doping to suppress residual bulk conductance. We have observed robust helical edge states with wide conductance plateaus precisely quantized to $2{e}^{2}/h$ in mesoscopic Hall samples. On the other hand, in larger samples the edge conductance is found to be inversely proportional to the edge length. These characteristics persist in a wide temperature range and show essentially no temperature dependence. The quantized plateaus persist to a 12 T applied in-plane field; the conductance increases from $2{e}^{2}/h$ in strong perpendicular fields manifesting chiral edge transport. Our study presents a compelling case for exotic properties of a one-dimensional helical liquid on the edge of $\mathrm{InAs}/\mathrm{GaSb}$ bilayers.

SPHINX: Detecting Security Attacks in Software-Defined Networks.

Abstract: Software-defined networks (SDNs) allow greater control over network entities by centralizing the control plane, but place great burden on the administrator to manually ensure security and correct functioning of the entire network. We list several attacks on SDN controllers that violate network topology and data plane forwarding, and can be mounted by compromised network entities, such as end hosts and soft switches. We further demonstrate their feasibility on four popular SDN controllers. We propose SPHINX to detect both known and potentially unknown attacks on network topology and data plane forwarding originating within an SDN. SPHINX leverages the novel abstraction of flow graphs, which closely approximate the actual network operations, to enable incremental validation of all network updates and constraints. SPHINX dynamically learns new network behavior and raises alerts when it detects suspicious changes to existing network control plane behavior. Our evaluation shows that SPHINX is capable of detecting attacks in SDNs in realtime with low performance overheads, and requires no changes to the controllers for deployment.