We expect that many-core microprocessors will push performance per chip from the 10 gigaflop to the 10 teraflop range in the coming decade. To support this increased performance, memory and inter-core bandwidths will also have to scale by orders of magnitude. Pin limitations, the energy cost of electrical signaling, and the non-scalability of chip-length global wires are significant bandwidth impediments. Recent developments in silicon nanophotonic technology have the potential to meet these off- and on-stack bandwidth requirements at acceptable power levels. Corona is a 3D many-core architecture that uses nanophotonic communication for both inter-core communication and off-stack communication to memory or I/O devices. Its peak floating-point performance is 10 teraflops. Dense wavelength division multiplexed optically connected memory modules provide 10 terabyte per second memory bandwidth. A photonic crossbar fully interconnects its 256 low-power multithreaded cores at 20 terabyte per second bandwidth. We have simulated a 1024 thread Corona system running synthetic benchmarks and scaled versions of the SPLASH-2 benchmark suite. We believe that in comparison with an electrically-connected many-core alternative that uses the same on-stack interconnect power, Corona can provide 2 to 6 times more performance on many memory intensive workloads, while simultaneously reducing power.

/pdf/corona-system-implications-of-emerging-nanophotonic-1p6apqmj51.pdf

Corona: System Implications of Emerging Nanophotonic Technology

This paper introduces the Graphite open-source distributed parallel multicore simulator infrastructure. Graphite is designed from the ground up for exploration of future multi-core processors containing dozens, hundreds, or even thousands of cores. It provides high performance for fast design space exploration and software development. Several techniques are used to achieve this including: direct execution, seamless multicore and multi-machine distribution, and lax synchronization. Graphite is capable of accelerating simulations by distributing them across multiple commodity Linux machines. When using multiple machines, it provides the illusion of a single process with a single, shared address space, allowing it to run off-the-shelf pthread applications with no source code modification. Our results demonstrate that Graphite can simulate target architectures containing over 1000 cores on ten 8-core servers. Performance scales well as more machines are added with near linear speedup in many cases. Simulation slowdown is as low as 41× versus native execution.

/pdf/graphite-a-distributed-parallel-simulator-for-multicores-1y50n9y9w6.pdf

Graphite: A distributed parallel simulator for multicores

Analysis of technology and application trends reveals a growing imbalance in the peak compute-to-memory-capacity ratio for future servers. At the same time, the fraction contributed by memory systems to total datacenter costs and power consumption during typical usage is increasing. In response to these trends, this paper re-examines traditional compute-memory co-location on a single system and details the design of a new general-purpose architectural building block-a memory blade-that allows memory to be "disaggregated" across a system ensemble. This remote memory blade can be used for memory capacity expansion to improve performance and for sharing memory across servers to reduce provisioning and power costs. We use this memory blade building block to propose two new system architecture solutions-(1) page-swapped remote memory at the virtualization layer, and (2) block-access remote memory with support in the coherence hardware-that enable transparent memory expansion and sharing on commodity-based systems. Using simulations of a mix of enterprise benchmarks supplemented with traces from live datacenters, we demonstrate that memory disaggregation can provide substantial performance benefits (on average 10X) in memory constrained environments, while the sharing enabled by our solutions can improve performance-per-dollar by up to 57% when optimizing memory provisioning across multiple servers.

/pdf/disaggregated-memory-for-expansion-and-sharing-in-blade-yf0xv0kfvz.pdf

Disaggregated memory for expansion and sharing in blade servers

In this paper we introduce CACTI-D, a significant enhancement of CACTI 5.0. CACTI-D adds support for modeling of commodity DRAM technology and support for main memory DRAM chip organization. CACTI-D enables modeling of the complete memory hierarchy with consistent models all the way from SRAM based L1 caches through main memory DRAMs on DIMMs. We illustrate the potential applicability of CACTI-D in the design and analysis of future memory hierarchies by carrying out a last level cache study for a multicore multithreaded architecture at the 32nm technology node. In this study we use CACTI-D to model all components of the memory hierarchy including L1, L2, last level SRAM, logic process based DRAM or commodity DRAM L3 caches, and main memory DRAM chips. We carry out architectural simulation using benchmarks with large data sets and present results of their execution time, breakdown of power in the memory hierarchy, and system energy-delay product for the different system configurations. We find that commodity DRAM technology is most attractive for stacked last level caches, with significantly lower energy-delay products.

A Comprehensive Memory Modeling Tool and Its Application to the Design and Analysis of Future Memory Hierarchies

Technology trends may soon favor building main memory as a hybrid between DRAM and non-volatile memory, such as flash or PC-RAM. We describe how the operating system might manage such hybrid memories, using semantic information not available in other layers. We describe preliminary experiments suggesting that this approach is viable.

/pdf/operating-system-support-for-nvm-dram-hybrid-main-memory-a56f5s9h84.pdf

Operating system support for NVM+DRAM hybrid main memory

A processing device includes a processor core and a number of calculation modules that each is configurable to perform any one of operations for a convolutional neuron network system. A first set of the calculation modules are configured to perform convolution operations, a second set of the calculation modules are reconfigured to perform averaging operations, and a third set of the calculation modules are reconfigured to perform dot product operations.

Reconfigurable processing unit

The high speed and faithfulness of state-of-the-art virtual machines (VMs) make them the ideal front-end for a system simulation framework. However, VMs only emulate the functional behavior and just provide the minimal timing for the system to run correctly. In a simulation framework supporting the exploration of different configurations, a timing backend is still necessary to accurately determine the performance of the simulated target. As it has been extensively researched, sampling is an excellent approach for fast timing simulation. However, existing sampling mechanisms require capturing information for every instruction and memory access. Hence, coupling a standard sampling technique to a VM implies disabling most of the "tricks" used by a VM to accelerate execution, such as the caching and linking of dynamically compiled code. Without code caching, the performance of a VM is severely impacted. In this paper we present a novel dynamic sampling mechanism that overcomes this problem and enables the use of VMs for timing simulation. By making use of the internal information collected by the VM during functional simulation, we can quickly assess important characteristics of the simulated applications (such as phase changes), and activate or deactivate the timing simulation accordingly. This allows us to run unmodified OS and applications over emulated hardware at near-native speed, yet providing a way to insert timing measurements that yield a final accuracy similar to state-of-the-art sampling methods

Combining Simulation and Virtualization through Dynamic Sampling

Systems and methods for performing convolution operations. An example processing system comprises: a processing core; and a convolver unit to apply a convolution filter to a plurality of input data elements represented by a two-dimensional array, the convolver unit comprising a plurality of multipliers coupled to two or more sets of latches, wherein each set of latches is to store a plurality of data elements of a respective one-dimensional section of the two-dimensional array.

Processing device for performing convolution operations

In superscalar processors, capable of issuing and executing multiple instructions per cycle, fetch performance represents an upper bound to the overall processor performance. Unless there is some form of instruction re-use mechanism, you cannot execute instructions faster than you can fetch them. Instruction Level Parallelism, represented by wide issue out oforder superscalar processors, was the trending topic during the end of the 90's and early 2000's. It is indeed the most promising way to continue improving processor performance in a way that does not impact application development, unlike current multicore architectures which require parallelizing the applications (a process that is still far from being automated in the general case). Widening superscalar processor issue was the promise of neverending improvements to single thread performance, as identified by Yale N. Patt et al. in the 1997 special issue of IEEE Computer about "Billion transistor processors" [1].However, instruction fetch performance is limited by the control flow of the program. The basic fetch stage implementation can read instructions from a single cache line, starting from the current fetch address and up to the next control flow instruction. That is one basic block per cycle at most.Given that the typical basic block size in SPEC integer benchmarks is 4-6 instructions, fetch performance was limited to those same 4-6 instructions per cycle, making 8-wide and 16-wide superscalar processors impractical. It became imperative to find mechanisms to fetch more than 8 instructions per cycle, and that meant fetching more than one basic block per cycle.

/pdf/author-retrospective-for-software-trace-cache-3uaterh35j.pdf

Author retrospective for software trace cache

This paper introduces the prophet/critic hybrid conditionalbranch predictor, which has two component predictorsthat play the role of either prophet or critic.Theprophet is a conventional predictor that uses branch historyto predict the direction of the current branch.Further accessesof the prophet yield predictions for the branches followingthe current one.Predictions for the current branchand the ones that follow are collectively known as thebranch's future.They are actually a prophecy, or predictedbranch future.The critic uses both the branch's history andfuture to give a critique of the prophet's prediction fo thecurrent branch.The critique, either agree or disagree, isused to generate the final prediction for the branch.Our results show an 8K + 8K byte prophet/critic hybridhas 39% fewer mispredicts than a 16K byte 2Bc-gskewpredictor-a predictor similar to that of the proposed Compaq*Alpha* EV8 processor-across a wide range of applications.The distance between pipeline flushes due to mispredictsincreases from one flush per 418 micro-operations(uops) to one per 680 uops.For gcc, the percentage of mispredictedbranches drops from 3.11% to 1.23%.On a machinebased on the Intel® Pentium® 4 processor, this improvesuPC (Uops Per Cycle) by 7.8% (18% for gcc) andreduces the number of uops fetched (along both correct andincorrect paths) by 8.6%.

/pdf/prophet-critic-hybrid-branch-prediction-4sklwwjnl6.pdf

Ayose Falcón

Papers

Reconfigurable processing unit

Combining Simulation and Virtualization through Dynamic Sampling

Processing device for performing convolution operations

Author retrospective for software trace cache

Prophet/Critic Hybrid Branch Prediction