Commodity computer systems contain more and more processor cores and exhibit increasingly diverse architectural tradeoffs, including memory hierarchies, interconnects, instruction sets and variants, and IO configurations. Previous high-performance computing systems have scaled in specific cases, but the dynamic nature of modern client and server workloads, coupled with the impossibility of statically optimizing an OS for all workloads and hardware variants pose serious challenges for operating system structures.We argue that the challenge of future multicore hardware is best met by embracing the networked nature of the machine, rethinking OS architecture using ideas from distributed systems. We investigate a new OS structure, the multikernel, that treats the machine as a network of independent cores, assumes no inter-core sharing at the lowest level, and moves traditional OS functionality to a distributed system of processes that communicate via message-passing.We have implemented a multikernel OS to show that the approach is promising, and we describe how traditional scalability problems for operating systems (such as memory management) can be effectively recast using messages and can exploit insights from distributed systems and networking. An evaluation of our prototype on multicore systems shows that, even on present-day machines, the performance of a multikernel is comparable with a conventional OS, and can scale better to support future hardware.

/pdf/the-multikernel-a-new-os-architecture-for-scalable-multicore-1zaxph0v39.pdf

The multikernel: a new OS architecture for scalable multicore systems

Energy efficiency is the new fundamental limiter of processor performance, way beyond numbers of processors.

/pdf/the-future-of-microprocessors-15i7iemtj8.pdf

The future of microprocessors

Recent advances in computing have led to an explosion in the amount of data being generated. Processing the ever-growing data in a timely manner has made throughput computing an important aspect for emerging applications. Our analysis of a set of important throughput computing kernels shows that there is an ample amount of parallelism in these kernels which makes them suitable for today's multi-core CPUs and GPUs. In the past few years there have been many studies claiming GPUs deliver substantial speedups (between 10X and 1000X) over multi-core CPUs on these kernels. To understand where such large performance difference comes from, we perform a rigorous performance analysis and find that after applying optimizations appropriate for both CPUs and GPUs the performance gap between an Nvidia GTX280 processor and the Intel Core i7-960 processor narrows to only 2.5x on average. In this paper, we discuss optimization techniques for both CPU and GPU, analyze what architecture features contributed to performance differences between the two architectures, and recommend a set of architectural features which provide significant improvement in architectural efficiency for throughput kernels.

/pdf/debunking-the-100x-gpu-vs-cpu-myth-an-evaluation-of-1m7y4u1n7m.pdf

Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU

GPU architectures are increasingly important in the multi-core era due to their high number of parallel processors. Programming thousands of massively parallel threads is a big challenge for software engineers, but understanding the performance bottlenecks of those parallel programs on GPU architectures to improve application performance is even more difficult. Current approaches rely on programmers to tune their applications by exploiting the design space exhaustively without fully understanding the performance characteristics of their applications.To provide insights into the performance bottlenecks of parallel applications on GPU architectures, we propose a simple analytical model that estimates the execution time of massively parallel programs. The key component of our model is estimating the number of parallel memory requests (we call this the memory warp parallelism) by considering the number of running threads and memory bandwidth. Based on the degree of memory warp parallelism, the model estimates the cost of memory requests, thereby estimating the overall execution time of a program. Comparisons between the outcome of the model and the actual execution time in several GPUs show that the geometric mean of absolute error of our model on micro-benchmarks is 5.4% and on GPU computing applications is 13.3%. All the applications are written in the CUDA programming language.

/pdf/an-analytical-model-for-a-gpu-architecture-with-memory-level-4930mwitke.pdf

An analytical model for a GPU architecture with memory-level and thread-level parallelism awareness

We present PacketShader, a high-performance software router framework for general packet processing with Graphics Processing Unit (GPU) acceleration. PacketShader exploits the massively-parallel processing power of GPU to address the CPU bottleneck in current software routers. Combined with our high-performance packet I/O engine, PacketShader outperforms existing software routers by more than a factor of four, forwarding 64B IPv4 packets at 39 Gbps on a single commodity PC. We have implemented IPv4 and IPv6 forwarding, OpenFlow switching, and IPsec tunneling to demonstrate the flexibility and performance advantage of PacketShader. The evaluation results show that GPU brings significantly higher throughput over the CPU-only implementation, confirming the effectiveness of GPU for computation and memory-intensive operations in packet processing.

/pdf/packetshader-a-gpu-accelerated-software-router-1cyii2z7z5.pdf

PacketShader: a GPU-accelerated software router

A method of obtaining an operand from a memory device includes reading a first operand from a first location in a memory device, the first operand including part of the operand specified by an instruction, shifting the first operand by a first shift amount, reading a second data operand from the memory device, the second operand having part of the operand specified by the instruction, shifting the second operand by a second shift amount, and combining the first shifted data entry and the second shifted data entry to produce an aligned operand, wherein shifting the first operand and shifting the second operand is performed by a shifter also used for floating point functions.

Unaligned memory operands

A technique to share execution resources. In one embodiment, a CPU and a GPU share resources according to workload, power considerations, or available resources by scheduling or transferring instructions and information between the CPU and GPU.

Sharing resources between a CPU and GPU

A method and apparatus to detect and filter out redundant cache line addresses in a prefetch input queue, and to adjust the detector window size dynamically according to the number of detector entries in the queue for the cache-to-memory controller bus. Detectors correspond to cache line addresses that may represent cache misses in various levels of cache memory.

Prefetching data in computer system

A method and apparatus for accessing memory comprising monitoring memory accesses from a hardware prefetcher; determining whether the memory accesses from the hardware prefetcher are used by an out-of-order core; and switching memory accesses from a first mode to a second mode if a percentage of the memory access generated by the hardware prefetcher are used by the out-of-order core.

Method and apparatus to control memory accesses

A method and apparatus for efficiently caching streaming and non-streaming data is described herein. Software, such as a compiler, identifies last use streaming instructions/operations that are the last instruction/operation to access streaming data for a number of instructions or an amount of time. As a result of performing an access to a cache line for a last use instruction/operation, the cache line is updated to a streaming data no longer needed (SDN) state. When control logic is to determine a cache line to be replaced, a modified Least Recently Used (LRU) algorithm is biased to select SDN state lines first to replace no longer needed streaming data.

Eric Sprangle

Papers

Unaligned memory operands

Sharing resources between a CPU and GPU

Prefetching data in computer system

Method and apparatus to control memory accesses

Mechanism for effectively caching streaming and non-streaming data patterns