Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360× and the energy consumption by ~895×, across the evaluated machine learning benchmarks.

PRIME: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory

IEEE transactions on computer-aided design of integrated circuits and systems : a publication of the IEEE Circuits and Systems Society

Large capacity memory systems are constructed using stacked memory integrated circuits or chips. The stacked memory chips are constructed in such a way that eliminates problems such as signal integrity while still meeting current and future memory standards.

Methods and apparatus of stacking DRAMs

Historically, server designers have opted for simple memory systems by picking one of a few commoditized DDR memory products. We are already witnessing a major upheaval in the off-chip memory hierarchy, with the introduction of many new memory products—buffer-on-board, LRDIMM, HMC, HBM, and NVMs, to name a few. Given the plethora of choices, it is expected that different vendors will adopt different strategies for their high-capacity memory systems, often deviating from DDR standards and/or integrating new functionality within memory systems. These strategies will likely differ in their choice of interconnect and topology, with a significant fraction of memory energy being dissipated in I/O and data movement. To make the case for memory interconnect specialization, this paper makes three contributions.First, we design a tool that carefully models I/O power in the memory system, explores the design space, and gives the user the ability to define new types of memory interconnects/topologies. The tool is validated against SPICE models, and is integrated into version 7 of the popular CACTI package. Our analysis with the tool shows that several design parameters have a significant impact on I/O power.We then use the tool to help craft novel specialized memory system channels. We introduce a new relay-on-board chip that partitions a DDR channel into multiple cascaded channels. We show that this simple change to the channel topology can improve performance by 22% for DDR DRAM and lower cost by up to 65% for DDR DRAM. This new architecture does not require any changes to DIMMs, and it efficiently supports hybrid DRAM/NVM systems.Finally, as an example of a more disruptive architecture, we design a custom DIMM and parallel bus that moves away from the DDR3/DDR4 standards. To reduce energy and improve performance, the baseline data channel is split into three narrow parallel channels and the on-DIMM interconnects are operated at a lower frequency. In addition, this allows us to design a two-tier error protection strategy that reduces data transfers on the interconnect. This architecture yields a performance improvement of 18% and a memory power reduction of 23%.The cascaded channel and narrow channel architectures serve as case studies for the new tool and show the potential for benefit from re-organizing basic memory interconnects.

https://dl.acm.org/doi/pdf/10.1145/3085572

CACTI 7: New Tools for Interconnect Exploration in Innovative Off-Chip Memories

Systems, among other embodiments, include topologies (data and/or control/address information) between an integrated circuit buffer device (that may be coupled to a master, such as a memory controller) and a plurality of integrated circuit memory devices. For example, data may be provided between the plurality of integrated circuit memory devices and the integrated circuit buffer device using separate segmented (or point-to-point link) signal paths in response to control/address information provided from the integrated circuit buffer device to the plurality of integrated circuit buffer devices using a single fly-by (or bus) signal path. An integrated circuit buffer device enables configurable effective memory organization of the plurality of integrated circuit memory devices. The memory organization represented by the integrated circuit buffer device to a memory controller may be different than the actual memory organization behind or coupled to the integrated circuit buffer device. The buffer device segments and merges the data transferred between the memory controller that expects a particular memory organization and actual memory organization.

Memory system topologies including a buffer device and an integrated circuit memory device

A main die and a stacked die are included in the same component package. A transmission gate (370) is implemented on the main die, and can be enabled to receive leakage current in a connection (318) between the main die and the stacked die, and to conduct the leakage current to a bonding pad (344) that is accessible external to the package. Thus, the connectivity between the main die and the stacked die can be tested after the dies are packaged. The transmission gate is disabled during high-speed testing and normal operation. The package can also include a multiplexer (364) that is enabled during high-speed testing to input and output test signals at the package level. A direction signal is used to indicate whether test signals are being input to or output from the main die.

Systems and methods for testing packaged dies

A signal interface circuit has a signal path for communicatively coupling host circuitry to peripheral circuitry of multiple peripherals. Communication signals in the signal path are of a peripheral signal level. The signal path has electronic components adapted for use in communicating signals between the host circuitry and the peripheral circuitry. The electronic components in the signal path have reliability limits less than the peripheral signal level. The configuration of the electronic components in the signal path allow communication of signals at the peripheral signal level.

High signal level compliant input/output circuits

We describe CACTI-IO, an extension to CACTI [4] that includes power, area and timing models for the IO and PHY of the off-chip memory interface for various server and mobile configurations. CACTI-IO enables design space exploration of the off-chip IO along with the DRAM and cache parameters. We describe the models added and three case studies that use CACTI-IO to study the tradeoffs between memory capacity, bandwidth and power. The case studies show that CACTI-IO helps (i) provide IO power numbers that can be fed into a system simulator for accurate power calculations, (ii) optimize off-chip configurations including the bus width, number of ranks, memory data width and off-chip bus frequency, especially for novel buffer-based topologies, and (iii) enable architects to quickly explore new interconnect technologies, including 3-D interconnect. We find that buffers on board and 3-D technologies offer an attractive design space involving power, bandwidth and capacity when appropriate interconnect parameters are deployed.

/pdf/cacti-io-cacti-with-off-chip-power-area-timing-models-3u1sitw0o7.pdf

CACTI-IO: CACTI with off-chip power-area-timing models

In this paper, we describe CACTI-IO, an extension to CACTI that includes power, area, and timing models for the IO and PHY of the OFF-chip memory interface for various server and mobile configurations. CACTI-IO enables design space exploration of the OFF-chip IO along with the dynamic random access memory and cache parameters. We describe the models added and four case studies that use CACTI-IO to study the tradeoffs between memory capacity, bandwidth (BW), and power. The case studies show that CACTI-IO helps to: 1) provide IO power numbers that can be fed into a system simulator for accurate power calculations; 2) optimize OFF-chip configurations including the bus width, number of ranks, memory data width, and OFF-chip bus frequency, especially for novel buffer-based topologies; and 3) enable architects to quickly explore new interconnect technologies, including 3-D interconnect. We find that buffers on board and 3-D technologies offer an attractive design space involving power, BW, and capacity when appropriate interconnect parameters are deployed.

Vaishnav Srinivas

Papers

CACTI 7: New Tools for Interconnect Exploration in Innovative Off-Chip Memories

Systems and methods for testing packaged dies

High signal level compliant input/output circuits

CACTI-IO: CACTI with off-chip power-area-timing models

CACTI-IO: CACTI With OFF-Chip Power-Area-Timing Models