scispace - formally typeset
Search or ask a question
Proceedings Article•DOI•

A case for small row buffers in non-volatile main memories

Justin Meza1, Jing Li2, Onur Mutlu1•
Carnegie Mellon University1, IBM2
30 Sep 2012-Vol. 2012, pp 484-485
TL;DR: It is found that on a multi-core system, reducing the row buffer size can greatly reduce main memory dynamic energy compared to a DRAM baseline with large row sizes, without greatly affecting endurance, and for some NVM technologies, leads to improved performance.
Abstract: DRAM-based main memories have read operations that destroy the read data, and as a result, must buffer large amounts of data on each array access to keep chip costs low. Unfortunately, system-level trends such as increased memory contention in multi-core architectures and data mapping schemes that improve memory parallelism lead to only a small amount of the buffered data to be accessed. This makes buffering large amounts of data on every memory array access energy-inefficient; yet organizing DRAM chips to buffer small amounts of data is costly, as others have shown [11]. Emerging non-volatile memories (NVMs) such as PCM, STT-RAM, and RRAM, however, do not have destructive read operations, opening up opportunities for employing small row buffers without incurring additional area penalty and/or design complexity. In this work, we discuss and evaluate architectural changes to enable small row buffers at a low cost in NVMs. We find that on a multi-core system, reducing the row buffer size can greatly reduce main memory dynamic energy compared to a DRAM baseline with large row sizes, without greatly affecting endurance, and for some NVM technologies, leads to improved performance.

Summary (2 min read)

Jump to: [Introduction] – [II. MOTIVATION] – [III. A SMALL ROW BUFFER NVM ARCHITECTURE] and [IV. RESULTS]

Introduction

  • Abstract—DRAM-based main memories have read operations that destroy the read data, and as a result, must buffer large amounts of data on each array access to keep chip costs low.
  • Emerging non-volatile memories (NVMs) such as PCM, STT-RAM, and RRAM, however, do not have destructive read operations, opening up opportunities for employing small row buffers without incurring additional area penalty and/or design complexity.
  • Over time, this charge leaks, causing the stored data to be lost.
  • As a result, the performance benefit of large row buffers may decrease in multi-core systems.

II. MOTIVATION

  • Emerging NVM technologies have several promising attributes compared to existing memory technologies such as SRAM (used in on-chip caches), DRAM, and Flash.
  • NVMs provide cost advantages compared to SRAM and DRAM, and latency advantages compared to Flash.
  • Typical DRAM chip micro-architectures (JEDEC-standard DDRtype SDRAM) are divided into banks that consist of rows and columns .
  • Comparing the 1- and 8-core row-interleaved data, the authors see that while row interleaving does enable more row buffer locality, its benefits diminish as memory system contention increases with more cores: row buffer hit rate is less than 50% for row interleaving even with large, 1KB rows.

III. A SMALL ROW BUFFER NVM ARCHITECTURE

  • Figure 1(b) shows the organization of their NVM architecture.
  • Compared to a traditional DRAM organization, the physical placement of the row buffer and the column multiplexer (part of the I/O gating circuitry in DRAM designs) are swapped in the data path (shown in gray).
  • This rearrangement makes better use of resources by sharing a smaller number of sense amplifiers (the devices which store bits in the row buffer) among multiple bitlines.
  • Note that this is not possible in DRAM (without reducing the row size) because a sense amplifier for each bit in the row is required in DRAM to restore the charge of the cell after it is read.
  • Unlike DRAM, however, their organization requires decoding both the row address and the column address during a RAS command, so that only a subset of the row containing the bits of interest will be selected, sensed, and stored in the row buffer.

IV. RESULTS

  • The authors modify their memory simulator timings according to those in Table I for PCM and STT-RAM.
  • The authors evaluate 31 multiprogrammed workloads composed of SPEC, TPC, and STREAM benchmarks.
  • Note that this reduction is achieved despite worse underlying technology parameters 2For more details, please refer to their accompanying tech report [5].
  • For a given memory technology, reducing the row buffer size does not greatly affect system performance due to the already low row buffer locality present on their multi-core system .
  • NVM cells have a limited lifetime in terms of the number of times they can be written to before their ability to store data fails, also known as Durability.

Did you find this useful? Give us your feedback

Figures (3)

Content maybe subject to copyright    Report

A Case for Small Row Buffers in Non-Volatile Main Memories
Justin Meza
∗
Jing Li
†
Onur Mutlu
∗
∗
Carnegie Mellon University
†
IBM T.J. Watson Research Center
{meza,onur}@cmu.edu jli@us.ibm.com
Abstract—DRAM-based main memories have read operations that
destroy the read data, and as a result, must buffer large amounts of data
on each array access to keep chip costs low. Unfortunately, system-level
trends such as increased memory contention in multi-core architectures
and data mapping schemes that improve memory parallelism lead to only
a small amount of the buffered data to be accessed. This makes buffering
large amounts of data on every memory array access energy-inefficient;
yet organizing DRAM chips to buffer small amounts of data is costly, as
others have shown [11].
Emerging non-volatile memories (NVMs) such as PCM, STT-RAM,
and RRAM, however, do not have destructive read operations, opening
up opportunities for employing small row buffers without incurring
additional area penalty and/or design complexity. In this work, we
discuss and evaluate architectural changes to enable small row buffers
at a low cost in NVMs. We find that on a multi-core system, reducing
the row buffer size can greatly reduce main memory dynamic energy
compared to a DRAM baseline with large row sizes, without greatly
affecting endurance, and for some NVM technologies, leads to improved
performance.
I. INTRODUCTION
Modern main memory is composed of dynamic random-access
memory (DRAM). A DRAM cell stores data as charge on a capacitor.
Over time, this charge leaks, causing the stored data to be lost. To
prevent this, data stored in DRAM must be periodically read out and
rewritten, a process called refreshing. In addition, reading data stored
in a DRAM cell destroys its state, requiring data to be later restored,
leading to increased cell access time and energy. For this reason,
DRAM devices require buffering data which are read. To keep costs
low, the buffering circuitry in DRAM devices is amortized among
large rows of cells, in peripheral storage called the row buffer, at
least one per bank [2]. Refreshing data and buffering large amounts of
data wastes energy in DRAM devices, causing main memory power
to constitute a significant fraction of the total system power.
Data fetched into the row buffer, however, can be accessed at much
lower latencies and less energy than accessing the DRAM memory
array. Therefore, large row buffer sizes can improve performance
and efficiency if many accesses can be served in the same row.
Unfortunately, there are several reasons why such row buffer locality
can be low in systems: (1) some applications inherently do not have
significant locality within rows (e.g., random access applications),
(2) as more cores are placed on chip, applications running on those
cores interfere with each other in the row buffers, leading to reduced
locality, especially if the memory scheduling policy is unaware of
applications’ interference in the row buffers [7], as also observed
by others [10, 11], and (3) interleaving techniques that improve
parallelism in the memory system (e.g., cache block interleaving)
tend to reduce row buffer locality because they stripe consecutive
cache blocks across different banks. As a result, the performance
benefit of large row buffers may decrease in multi-core systems.
New non-volatile memory (NVM) technologies, such as phase-
change memory (PCM), spin-transfer torque RAM (STT-RAM), and
resistive RAM (RRAM), on the other hand, provide non-destructive
reads and do not require refreshing and restoring their data after
sensing. This is because NVMs do not store their data as charge, and
thus their data persists after being read. This not only eliminates the
refresh problem of DRAM devices but also opens up opportunities for
employing smaller row buffers in NVMs without incurring additional
area penalty and/or design complexity.
II. MOTIVATION
Emerging NVM technologies have several promising attributes
compared to existing memory technologies such as SRAM (used in
on-chip caches), DRAM, and Flash. For example, NVMs provide cost
advantages compared to SRAM and DRAM, and latency advantages
compared to Flash. Importantly, these NVMs feature non-destructive
read operations, which DRAM does not have (i.e., data sensing does
not destroy the contents of cells).
Typical DRAM chip micro-architectures (JEDEC-standard DDR-
type SDRAM) are divided into banks that consist of rows (wordlines)
and columns (bitlines). Due to physical pin limitations, all the
information required to service a memory request must be supplied
over multiple commands. The Row Address Strobe (RAS) command
sends the row and bank address to select one of the banks and a row
within that bank. Then, an entire row (usually 1 to 2KB per chip)
is read out into latch-based sense amplifiers which comprise the row
buffer [2]. The Column Address Strobe (CAS) command then selects
a subset (i.e., column) of data from the row buffer (8B in a DDR3
×8 device [6]). Thus, a DRAM access first fetches many kilobytes
of data into the row buffer (RAS) and, in the worst case, uses only
a tiny portion of it (CAS). If multiple columns of the row buffer are
needed, multiple consecutive CAS commands can be issued, which
amortizes the cost of fetching the large row into the row buffer.
To illustrate how much buffered data is actually used in real
applications, Figure 1a shows average row buffer locality (row hit
rate) when employing various row buffer sizes on several system con-
figurations using the FR-FCFS scheduling policy [8].
1
In particular,
we show 1- and 8-core systems employing two different schemes
for mapping data in main memory: (1) row interleaving, which
places consecutive memory addresses in the same row, and (2) block
interleaving, which stripes data in consecutive memory addresses
(usually cache blocks) across different banks. Row interleaving helps
exploit row buffer locality by enabling data with spatial locality
to reside in the same row buffer, while block interleaving aims
to improve memory parallelism by enabling concurrent access of
memory channels/banks for consecutive memory addresses.
Comparing the 1- and 8-core row-interleaved data, we see that
while row interleaving does enable more row buffer locality, its
benefits diminish as memory system contention increases with more
cores: row buffer hit rate is less than 50% for row interleaving even
with large, 1KB rows. Block interleaving reduces row buffer locality
over row interleaving, to less than 10% in the 8-core case. While
it is clear that row locality is lower on multi-core systems, what
is less obvious is how row buffer size affects system-level tradeoffs,
such as energy-efficiency, performance, and durability, in NVM main
memories. This work evaluates these tradeoffs.
8B 16B 32B 64B 128B 256B 512B
1KB
Per DRAM-Chip Row Buffer Size
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Row Buffer Locality
1-core, row interleaved
8-core, row interleaved
1-core, cache block interleaved
8-core, cache block interleaved
(a)
Bank N
Bank 2
Bank 1
Bank 0
Memory Array
Row Buffer
Column Decoder
and Multiplexer
Row Decoder
...
...
...
(b)
Fig. 1: Row size affects row locality (a); our NVM architecture (b).
1
Application-aware memory request scheduling policies (e.g., [1, 3, 7])
provide better performance, but they can reduce row buffer locality.
1
8B 16B 32B 64B 128B 256B 512B
1KB
Baseline
Per DRAM-Chip Row Buffer Size
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
Normalized Memory Energy
STT-RAM
DRAM
PCM
(a) Memory energy with block interleaving.
8B 16B 32B 64B 128B 256B 512B
1KB
Per DRAM-Chip Row Buffer Size
0
1
2
3
4
5
Weighted Speedup
STT-RAM
DRAM
PCM
(b) Performance with block interleaving.
8B 16B 32B 64B 128B 256B 512B
1KB
Per DRAM-Chip Row Buffer Size
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Normalized Writes
No cache
32MB cache
(c) Writes with and without a 32MB cache.
Fig. 2: Multi-core results for energy (normalized to DRAM with 1KB rows), performance, and number of writes (normalized to 1KB rows).
III. A SMALL ROW BUFFER NVM ARCHITECTURE
Figure 1(b) shows the organization of our NVM architecture. Com-
pared to a traditional DRAM organization, the physical placement of
the row buffer and the column multiplexer (part of the I/O gating
circuitry in DRAM designs) are swapped in the data path (shown in
gray). This rearrangement makes better use of resources by sharing a
smaller number of sense amplifiers (the devices which store bits in the
row buffer) among multiple bitlines. Note that this is not possible in
DRAM (without reducing the row size) because a sense amplifier for
each bit in the row is required in DRAM to restore the charge of the
cell after it is read. Unlike DRAM, however, our organization requires
decoding both the row address and the column address during a RAS
command, so that only a subset of the row containing the bits of
interest will be selected, sensed, and stored in the row buffer. During
a CAS command, the data bits from the row buffer corresponding to
the desired column are further selected by the I/O gating circuitry
and sent to a prefetch buffer.
2
While related prior work [4] employed multiple, narrow rows
in a PCM main memory for reducing array reads and writes, it
focused on (1) a traditional DRAM data path design, (2) an iso-
area reorganization, requiring more area overhead than our technique
which employs smaller row buffers, and (3) assumed a standard
DRAM protocol for device access.
IV. RESULTS
We developed a cycle-accurate DDR3 memory simulator which we
use as part of an in-house x86 multi-core simulator, whose front-end
is based on Pin. We modify our memory simulator timings according
to those in Table I for PCM and STT-RAM. We show results for an
8-core system with different memories and row buffer sizes, though
reducing row buffer size in DRAM incurs significant area overhead
and chip cost, as discussed in [10, 11], which we do not evaluate.
We evaluate 31 multiprogrammed workloads composed of SPEC,
TPC, and STREAM benchmarks. We will focus on a DRAM chip
micro-architecture with 1KB row buffers and block interleaving as
our baseline (our findings are similar for row interleaving [5]).
Technology Energy (Read/Write) Latency (Read/Write)
PCM 2×/100× 5×/10×
STT-RAM 0.5×/1× 1×/1×
TABLE I: NVM array parameters, relative to DRAM.
Energy (Figure 2a): In all cases, reducing the row buffer size can
significantly reduce memory energy consumption, though there are
diminishing marginal returns. The diminishing marginal returns are
because, as the row buffer size decreases, memory energy becomes
dominated by the energy required to transfer data between the row
buffer and I/O pads during read and write operations.
A modest row buffer size of 64B per chip leads to 47%/67%
less main memory energy for PCM/STT-RAM, compared to an all-
DRAM main memory with large rows (1KB per chip). Note that this
reduction is achieved despite worse underlying technology parameters
2
For more details, please refer to our accompanying tech report [5].
than DRAM (cf. Table I) because the energy saved by reducing the
row buffer size more than makes up for the higher average memory
array access energy. Hence, an NVM main memory with smaller
row buffers can significantly reduce memory energy consumption
compared to a DRAM baseline with large row buffers.
Performance (Figure 2b): We evaluate the performance of our
system using the weighted speedup metric [9] (higher is better). For
a given memory technology, reducing the row buffer size does not
greatly affect system performance due to the already low row buffer
locality present on our multi-core system (cf. Figure 1a). Interestingly,
with similar technology-dependent timing parameters as DRAM, an
STT-RAM main memory can achieve better performance because our
new organization enables a more efficient access protocol (detailed
in [5]) which eliminates the precharge delay incurred on row buffer
misses, and relaxes the t
RRD
and t
FAW
timing parameters to enable
more banks to be accessed simultaneously.
Durability (Figure 2c): NVM cells have a limited lifetime in terms
of the number of times they can be written to before their ability
to store data fails. We examine the effects of different row buffer
sizes on device durability with and without a small 32MB e-DRAM
cache to a PCM main memory. We find that with or without a cache,
decreasing the row buffer size has only a small effect on the number
of NVM writes performed due to the low row buffer locality present
in the system. In contrast, the addition of a reasonably-sized e-DRAM
cache has a large impact on the reduction of writes, decreasing the
number of writes by 39% to 47% across the various row buffer sizes.
V. CONCLUSIONS
We showed that on a multi-core system, reducing the row buffer
size can greatly reduce main memory dynamic energy compared
to a DRAM baseline with large rows, without greatly affecting
performance and durability. Our future work includes exploring
architectural techniques which effectively leverage small row buffer
sizes for improved performance and energy-efficiency.
REFERENCES
[1] Y. Kim et al. ATLAS: A scalable and high-performance scheduling
algorithm for multiple memory controllers. HPCA ’10.
[2] Y. Kim et al. A case for exploiting subarray-level parallelism (SALP)
in DRAM. ISCA ’12.
[3] Y. Kim et al. Thread cluster memory scheduling: Exploiting differences
in memory access behavior. MICRO ’10.
[4] B. C. Lee, E. Ipek, O. Mutlu, and D. Burger. Architecting phase change
memory as a scalable DRAM alternative. ISCA ’09.
[5] J. Meza, J. Li, and O. Mutlu. A case for small row buffers in non-volatile
main memories. http://safari.ece.cmu.edu/tr/tr-2012-002.pdf.
[6] Micron. 1Gb: ×4, ×8, ×16 DDR3 SDRAM data sheet. http://download.
micron.com/pdf/datasheets/dram/ddr/1GbDDRx4x8x16.pdf.
[7] S. P. Muralidhara et al. Reducing memory interference in multicore
systems via application-aware memory channel partitioning. MICRO’11.
[8] S. Rixner, W. J. Dally, U. J. Kapasi, P. Mattson, and J. D. Owens.
Memory access scheduling. ISCA ’00.
[9] A. Snavely et al. Symbiotic jobscheduling for a simultaneous multi-
threading processor. ASPLOS ’10.
[10] K. Sudan, N. Chatterjee, D. Nellans, et al. Micro-pages: increasing dram
efficiency with locality-aware data placement. ASPLOS ’10.
[11] A. N. Udipi, N. Muralimanohar, N. Chatterjee, et al. Rethinking DRAM
design and organization for energy-constrained multi-cores. ISCA ’10.
2
Citations
More filters
Proceedings Article•DOI•
Low power multi-level-cell resistive memory design with incomplete data mapping

[...]

Niu Dimin1, Qiaosha Zou1, Cong Xu1, Yuan Xie1•
Pennsylvania State University1
07 Nov 2013
TL;DR: In order to improve endurance of MLC RAM, which is much smaller than single-level cell (SLC) ReRM due to the complex programming method, the Dynamic Data ReMapping (DDRM) is proposed to selectively regulate memory blocks from IDM state back to complete data mapping (CDM) state.
Abstract: Phase change memory (PCM) has been widely studied as a potential DRAM alternative. The multi-level cell (MLC) can further increase the memory density and reduce the fabrication cost by storing multiple bits in a single cell. Nevertheless, large write power, high write latency, as well as reliability issue resulted from the resistance drift, bring in challenges for MLC PCM based memory design. In contrast, the emerging Resistive Random Access Memory (ReRAM), which has similar MLC property as PCM, demonstrates better performance and energy efficiency compared to PCM. In addition, due to the physical switching behaviors of ReRAM cell, the resistance drift phenomenon does not exist. In this paper, we propose a low power MLC ReRAM design. We first study the programming method of MLC ReRAM and identify that programming latency and energy are highly dependent on the data pattern written to the cell. Based on this observation, we propose incomplete data mapping (IDM), which maps an eight-level-cell into six states to prevent the time/energy consuming data patterns from appearing in the cell. Furthermore, in order to improve endurance of MLC RAM, which is much smaller than single-level cell (SLC) ReRM due to the complex programming method, we propose Dynamic Data ReMapping (DDRM) to selectively regulate memory blocks from IDM state back to complete data mapping (CDM) state. We demonstrate that the proposed design can work effectively with existing error-correction schemes but requires much smaller space overhead. Experimental results show that, IDM can reduce the energy performance by at most 15% with negligible performance overhead. By combining the DDRM with existing error-correction scheme, DDRM can improve the memory lifetime by 2.75× compared with conventional memory architectures.

44 citations

Cites background from "A case for small row buffers in non..."

  • ...Since the row buffer size of NVMs is much smaller than that of DRAM [6], [38], the overhead of the circuity is negligible, which will be discussed in Section V....

    [...]

Journal Article•DOI•
In-Memory Data Rearrangement for Irregular, Data-Intensive Computing

[...]

Scott Lloyd1, Maya Gokhale1•
Lawrence Livermore National Laboratory1
11 Aug 2015-IEEE Computer
TL;DR: An emulation on a field-programmable gate array shows how the data rearrangement engine could improve speedup, memory bandwidth, and energy consumption on three representative benchmarks.
Abstract: The data rearrangement engine (DRE) performs in-memory data restructuring to accelerate irregular, data-intensive applications. An emulation on a field-programmable gate array shows how the DRE could improve speedup, memory bandwidth, and energy consumption on three representative benchmarks.

41 citations

Cites background from "A case for small row buffers in non..."

  • ...While reducing access size introduces perhaps unacceptable complexity into DRAM, it has been suggested that such an organization is better suited to future persistent memories [12]....

    [...]

Posted Content•
FIGARO: Improving System Performance via Fine-Grained In-DRAM Data Relocation and Caching

[...]

Yaohua Wang1, Lois Orosa2, Xiangjun Peng3, Yang Guo1, Saugata Ghose4, Minesh Patel2, Jeremie S. Kim2, Juan Gómez Luna2, Mohammad Sadrosadati, Nika Mansouri Ghiasi2, Onur Mutlu2 •
National University of Defense Technology1, ETH Zurich2, The Chinese University of Hong Kong3, University of Illinois at Urbana–Champaign4
17 Sep 2020-arXiv: Hardware Architecture
TL;DR: A new substrate, FIGARO, is proposed that uses existing shared global buffers among subarrays within a DRAM bank to provide support for in-DRAM data relocation across subar-rays at the granularity of a single cache block, and it is shown that FIGCache outperforms state-of-the-art in- DRAM caching techniques, and that its performance gains are robust across many system and mechanism parameters.
Abstract: DRAM Main memory is a performance bottleneck for many applications due to the high access latency. In-DRAM caches work to mitigate this latency by augmenting regular-latency DRAM with small-but-fast regions of DRAM that serve as a cache for the data held in the regular-latency region of DRAM. While an effective in-DRAM cache can allow a large fraction of memory requests to be served from a fast DRAM region, the latency savings are often hindered by inefficient mechanisms for relocating copies of data into and out of the fast regions. Existing in-DRAM caches have two sources of inefficiency: (1) the data relocation granularity is an entire multi-kilobyte row of DRAM; and (2) because the relocation latency increases with the physical distance between the slow and fast regions, multiple fast regions are physically interleaved among slow regions to reduce the relocation latency, resulting in increased hardware area and manufacturing complexity. We propose a new substrate, FIGARO, that uses existing shared global buffers among subarrays within a DRAM bank to provide support for in-DRAM data relocation across subarrays at the granularity of a single cache block. FIGARO has a distance-independent latency within a DRAM bank, and avoids complex modifications to DRAM. Using FIGARO, we design a fine-grained in-DRAM cache called FIGCache. The key idea of FIGCache is to cache only small, frequently-accessed portions of different DRAM rows in a designated region of DRAM. By caching only the parts of each row that are expected to be accessed in the near future, we can pack more of the frequently-accessed data into FIGCache, and can benefit from additional row hits in DRAM. Our evaluations show that FIGCache improves the average performance of a system using DDR4 DRAM by 16.3% and reduces average DRAM energy consumption by 7.8% for 8-core workloads, over a conventional system without in-DRAM caching.

37 citations

Cites background from "A case for small row buffers in non..."

  • ...Individual applications in multiprogrammed workloads are likely to interfere with each other, thus exacerbating bank conflicts [33, 35, 40, 70, 71, 84, 99, 102, 104, 110, 140, 141, 143, 144, 147, 155], which FIGCache can help to alleviate....

    [...]

  • ...However, it is difficult for an application to access the entire contents of a row when the row is opened, as (1) the application may not have high spatial locality [3, 4, 34, 145], and (2) interference among multiple programs running on a multicore processor limits data reuse from an open row [33, 35, 40, 70, 71, 84, 99, 102, 104, 110, 140, 141, 143, 144, 147, 155]....

    [...]

  • ...The interference among concurrently running programs in a multicore system further hurts the row buffer locality [33, 35, 40, 70, 71, 84, 99, 102, 104, 110, 140, 141, 143, 144, 147, 155]....

    [...]

Proceedings Article•DOI•
HpMC: An Energy-aware Management System of Multi-level Memory Architectures

[...]

Chun-Yi Su1, David G. Roberts2, Edgar A. León3, Kirk W. Cameron1, Bronis R. de Supinski3, Gabriel H. Loh2, Dimitrios S. Nikolopoulos4 •
Virginia Tech1, Advanced Micro Devices2, Lawrence Livermore National Laboratory3, Queen's University Belfast4
05 Oct 2015
TL;DR: This work proposes HpMC, a new memory controller design that combines the best aspects of existing management policies to improve performance and energy, and creates HMsim, an infrastructure that enables n-level, heterogeneous memory studies by leveraging existing memory simulators.
Abstract: DRAM technology faces density and power challenges to increase capacity because of limitations of physical cell design. To overcome these limitations, system designers are exploring alternative solutions that combine DRAM and emerging NVRAM technologies. Previous work on heterogeneous memories focuses, mainly, on two system designs: PCache, a hierarchical, inclusive memory system, and HRank, a flat, non-inclusive memory system. We demonstrate that neither of these designs can universally achieve high performance and energy efficiency across a suite of HPC workloads. In this work, we investigate the impact of a number of multi-level memory designs on the performance, power, and energy consumption of applications. To achieve this goal and overcome the limited number of available tools to study heterogeneous memories, we created HMsim, an infrastructure that enables n-level, heterogeneous memory studies by leveraging existing memory simulators. We, then, propose HpMC, a new memory controller design that combines the best aspects of existing management policies to improve performance and energy. Our energy-aware memory management system dynamically switches between PCache and HRank based on the temporal locality of applications. Our results show that HpMC reduces energy consumption from 13% to 45% compared to PCache and HRank, while providing the same bandwidth and higher capacity than a conventional DRAM system.

31 citations

Proceedings Article•DOI•
FIGARO: Improving System Performance via Fine-Grained In-DRAM Data Relocation and Caching

[...]

Yaohua Wang1, Lois Orosa2, Xiangjun Peng3, Yang Guo1, Saugata Ghose4, Minesh Patel2, Jeremie S. Kim2, Juan Gómez Luna2, Mohammad Sadrosadati, Nika Mansouri Ghiasi2, Onur Mutlu2 •
National University of Defense Technology1, ETH Zurich2, The Chinese University of Hong Kong3, University of Illinois at Urbana–Champaign4
01 Oct 2020
TL;DR: In this paper, a fine-grained in-DRAM cache called FIGCache is proposed to cache only frequently-accessed portions of different DRAM rows in a designated region of DRAM.
Abstract: Main memory, composed of DRAM, is a performance bottleneck for many applications, due to the high DRAM access latency. In-DRAM caches work to mitigate this latency by augmenting regular-latency DRAM with small-but-fast regions of DRAM that serve as a cache for the data held in the regular-latency (i.e., slow) region of DRAM. While an effective in-DRAM cache can allow a large fraction of memory requests to be served from a fast DRAM region, the latency savings are often hindered by inefficient mechanisms for migrating (i.e., relocating) copies of data into and out of the fast regions. Existing in-DRAM caches have two sources of inefficiency: (1) their data relocation granularity is an entire multi-kilobyte row of DRAM, even though much of the row may never be accessed due to poor data locality; and (2) because the relocation latency increases with the physical distance between the slow and fast regions, multiple fast regions are physically interleaved among slow regions to reduce the relocation latency, resulting in increased hardware area and manufacturing complexityWe propose a new substrate, FIGARO, that uses existing shared global buffers among subarrays within a DRAM bank to provide support for in-DRAM data relocation across subar-rays at the granularity of a single cache block. FIGARO has a distance-independent latency within a DRAM bank, and avoids complex modifications to DRAM (such as the interleaving of fast and slow regions). Using FIGARO, we design a fine-grained in-DRAM cache called FIGCache. The key idea of FIGCache is to cache only small, frequently-accessed portions of different DRAM rows in a designated region of DRAM. By caching only the parts of each row that are expected to be accessed in the near future, we can pack more of the frequently-accessed data into FIGCache, and can benefit from additional row hits in DRAM (i.e., accesses to an already-open row, which have a lower latency than accesses to an unopened row). FIGCache provides benefits for systems with both heterogeneous DRAM banks (i.e., banks with fast regions and slow regions) and conventional homogeneous DRAM banks (i.e., banks with only slow regions)Our evaluations across a wide variety of applications show that FIGCache improves the average performance of a system using DDR4 DRAM by 16.3% and reduces average DRAM energy consumption by 7.8% for 8-core workloads, over a conventional system without in-DRAM caching. We show that FIGCache outperforms state-of-the-art in-DRAM caching techniques, and that its performance gains are robust across many system and mechanism parameters.

30 citations

References
More filters
Proceedings Article•DOI•
Architecting phase change memory as a scalable dram alternative

[...]

Benjamin C. Lee1, Engin Ipek1, Onur Mutlu2, Doug Burger1•
Microsoft1, Carnegie Mellon University2
20 Jun 2009
TL;DR: This work proposes, crafted from a fundamental understanding of PCM technology parameters, area-neutral architectural enhancements that address these limitations and make PCM competitive with DRAM.
Abstract: Memory scaling is in jeopardy as charge storage and sensing mechanisms become less reliable for prevalent memory technologies, such as DRAM. In contrast, phase change memory (PCM) storage relies on scalable current and thermal mechanisms. To exploit PCM's scalability as a DRAM alternative, PCM must be architected to address relatively long latencies, high energy writes, and finite endurance.We propose, crafted from a fundamental understanding of PCM technology parameters, area-neutral architectural enhancements that address these limitations and make PCM competitive with DRAM. A baseline PCM system is 1.6x slower and requires 2.2x more energy than a DRAM system. Buffer reorganizations reduce this delay and energy gap to 1.2x and 1.0x, using narrow rows to mitigate write energy and multiple rows to improve locality and write coalescing. Partial writes enhance memory endurance, providing 5.6 years of lifetime. Process scaling will further reduce PCM energy costs and improve endurance.

1,568 citations

Proceedings Article•DOI•
Memory access scheduling

[...]

Scott Rixner1, William J. Dally2, Ujval J. Kapasi2, Peter Mattson2, John D. Owens2 •
Massachusetts Institute of Technology1, Stanford University2
01 May 2000
TL;DR: This paper introduces memory access scheduling, a technique that improves the performance of a memory system by reordering memory references to exploit locality within the 3-D memory structure.
Abstract: The bandwidth and latency of a memory system are strongly dependent on the manner in which accesses interact with the “3-D” structure of banks, rows, and columns characteristic of contemporary DRAM chips. There is nearly an order of magnitude difference in bandwidth between successive references to different columns within a row and different rows within a bank. This paper introduces memory access scheduling, a technique that improves the performance of a memory system by reordering memory references to exploit locality within the 3-D memory structure. Conservative reordering, in which the first ready reference in a sequence is performed, improves bandwidth by 40% for traces from five media benchmarks. Aggressive reordering, in which operations are scheduled to optimize memory bandwidth, improves bandwidth by 93% for the same set of applications. Memory access scheduling is particularly important for media processors where it enables the processor to make the most efficient use of scarce memory bandwidth.

1,009 citations

Journal Article•DOI•
Symbiotic jobscheduling for a simultaneous multithreaded processor

[...]

Allan Snavely1, Dean M. Tullsen1•
University of California, San Diego1
12 Nov 2000
TL;DR: It is demonstrated that performance on a hardware multithreaded processor is sensitive to the set of jobs that are coscheduled by the operating system jobscheduler, and that a small sample of the possible schedules is sufficient to identify a good schedule quickly.
Abstract: Simultaneous Multithreading machines fetch and execute instructions from multiple instruction streams to increase system utilization and speedup the execution of jobs. When there are more jobs in the system than there is hardware to support simultaneous execution, the operating system scheduler must choose the set of jobs to coscheduleThis paper demonstrates that performance on a hardware multithreaded processor is sensitive to the set of jobs that are coscheduled by the operating system jobscheduler. Thus, the full benefits of SMT hardware can only be achieved if the scheduler is aware of thread interactions. Here, a mechanism is presented that allows the scheduler to significantly raise the performance of SMT architectures. This is done without any advance knowledge of a workload's characteristics, using sampling to identify jobs which run well together.We demonstrate an SMT jobscheduler called SOS. SOS combines an overhead-free sample phase which collects information about various possible schedules, and a symbiosis phase which uses that information to predict which schedule will provide the best performance. We show that a small sample of the possible schedules is sufficient to identify a good schedule quickly. On a system with random job arrivals and departures, response time is improved as much as 17% over a schedule which does not incorporate symbiosis.

619 citations

Proceedings Article•DOI•
ATLAS: A scalable and high-performance scheduling algorithm for multiple memory controllers

[...]

Yoongu Kim1, Dongsu Han1, Onur Mutlu1, Mor Harchol-Balter1•
Carnegie Mellon University1
01 Apr 2010
TL;DR: It is shown that the implementation of least-attained-service thread prioritization reduces the time the cores spend stalling and significantly improves system throughput, and ATLAS's performance benefit increases as the number of cores increases.
Abstract: Modern chip multiprocessor (CMP) systems employ multiple memory controllers to control access to main memory. The scheduling algorithm employed by these memory controllers has a significant effect on system throughput, so choosing an efficient scheduling algorithm is important. The scheduling algorithm also needs to be scalable — as the number of cores increases, the number of memory controllers shared by the cores should also increase to provide sufficient bandwidth to feed the cores. Unfortunately, previous memory scheduling algorithms are inefficient with respect to system throughput and/or are designed for a single memory controller and do not scale well to multiple memory controllers, requiring significant finegrained coordination among controllers. This paper proposes ATLAS (Adaptive per-Thread Least-Attained-Service memory scheduling), a fundamentally new memory scheduling technique that improves system throughput without requiring significant coordination among memory controllers. The key idea is to periodically order threads based on the service they have attained from the memory controllers so far, and prioritize those threads that have attained the least service over others in each period. The idea of favoring threads with least-attained-service is borrowed from the queueing theory literature, where, in the context of a single-server queue it is known that least-attained-service optimally schedules jobs, assuming a Pareto (or any decreasing hazard rate) workload distribution. After verifying that our workloads have this characteristic, we show that our implementation of least-attained-service thread prioritization reduces the time the cores spend stalling and significantly improves system throughput. Furthermore, since the periods over which we accumulate the attained service are long, the controllers coordinate very infrequently to form the ordering of threads, thereby making ATLAS scalable to many controllers. We evaluate ATLAS on a wide variety of multiprogrammed SPEC 2006 workloads and systems with 4–32 cores and 1–16 memory controllers, and compare its performance to five previously proposed scheduling algorithms. Averaged over 32 workloads on a 24-core system with 4 controllers, ATLAS improves instruction throughput by 10.8%, and system throughput by 8.4%, compared to PAR-BS, the best previous CMP memory scheduling algorithm. ATLAS's performance benefit increases as the number of cores increases.

439 citations

Proceedings Article•DOI•
Thread Cluster Memory Scheduling: Exploiting Differences in Memory Access Behavior

[...]

Yoongu Kim1, Michael K. Papamichael1, Onur Mutlu1, Mor Harchol-Balter1•
Carnegie Mellon University1
04 Dec 2010
TL;DR: This paper presents a new memory scheduling algorithm that addresses system throughput and fairness separately with the goal of achieving the best of both, and evaluates TCM on a wide variety of multiprogrammed workloads and compares its performance to four previously proposed scheduling algorithms, finding that TCM achieves both the best system throughputand fairness.
Abstract: In a modern chip-multiprocessor system, memory is a shared resource among multiple concurrently executing threads. The memory scheduling algorithm should resolve memory contention by arbitrating memory access in such a way that competing threads progress at a relatively fast and even pace, resulting in high system throughput and fairness. Previously proposed memory scheduling algorithms are predominantly optimized for only one of these objectives: no scheduling algorithm provides the best system throughput and best fairness at the same time. This paper presents a new memory scheduling algorithm that addresses system throughput and fairness separately with the goal of achieving the best of both. The main idea is to divide threads into two separate clusters and employ different memory request scheduling policies in each cluster. Our proposal, Thread Cluster Memory scheduling (TCM), dynamically groups threads with similar memory access behavior into either the latency-sensitive (memory-non-intensive) or the bandwidth-sensitive (memory-intensive) cluster. TCM introduces three major ideas for prioritization: 1) we prioritize the latency-sensitive cluster over the bandwidth-sensitive cluster to improve system throughput, 2) we introduce a ``niceness'' metric that captures a thread's propensity to interfere with other threads, 3) we use niceness to periodically shuffle the priority order of the threads in the bandwidth-sensitive cluster to provide fair access to each thread in a way that reduces inter-thread interference. On the one hand, prioritizing memory-non-intensive threads significantly improves system throughput without degrading fairness, because such ``light'' threads only use a small fraction of the total available memory bandwidth. On the other hand, shuffling the priority order of memory-intensive threads improves fairness because it ensures no thread is disproportionately slowed down or starved. We evaluate TCM on a wide variety of multiprogrammed workloads and compare its performance to four previously proposed scheduling algorithms, finding that TCM achieves both the best system throughput and fairness. Averaged over 96 workloads on a 24-core system with 4 memory channels, TCM improves system throughput and reduces maximum slowdown by 4.6%/38.6% compared to ATLAS (previous work providing the best system throughput) and 7.6%/4.6% compared to PAR-BS (previous work providing the best fairness).

375 citations

Related Papers (5)
Scalable high performance main memory system using phase-change memory technology

[...]

20 Jun 2009
Moinuddin K. Qureshi, Vijayalakshmi Srinivasan, Jude A. Rivers
Architecting phase change memory as a scalable dram alternative

[...]

20 Jun 2009
Benjamin C. Lee, Engin Ipek, Onur Mutlu, Doug Burger 
Evaluating STT-RAM as an energy-efficient main memory alternative

[...]

21 Apr 2013
Emre Kultursay, Mahmut Kandemir, Anand Sivasubramaniam, Onur Mutlu 
Parallelism-Aware Batch Scheduling: Enhancing both Performance and Fairness of Shared DRAM Systems

[...]

01 Jun 2008
Onur Mutlu, Thomas Moscibroda
ATLAS: A scalable and high-performance scheduling algorithm for multiple memory controllers

[...]

01 Apr 2010
Yoongu Kim, Dongsu Han, Onur Mutlu, Mor Harchol-Balter 
Frequently Asked Questions (2)
Q1. What have the authors contributed in "A case for small row buffers in non-volatile main memories" ?

In this work, the authors discuss and evaluate architectural changes to enable small row buffers at a low cost in NVMs. The authors find that on a multi-core system, reducing the row buffer size can greatly reduce main memory dynamic energy compared to a DRAM baseline with large row sizes, without greatly affecting endurance, and for some NVM technologies, leads to improved performance. 

Their future work includes exploring architectural techniques which effectively leverage small row buffer sizes for improved performance and energy-efficiency.