Commodity computer systems contain more and more processor cores and exhibit increasingly diverse architectural tradeoffs, including memory hierarchies, interconnects, instruction sets and variants, and IO configurations. Previous high-performance computing systems have scaled in specific cases, but the dynamic nature of modern client and server workloads, coupled with the impossibility of statically optimizing an OS for all workloads and hardware variants pose serious challenges for operating system structures.We argue that the challenge of future multicore hardware is best met by embracing the networked nature of the machine, rethinking OS architecture using ideas from distributed systems. We investigate a new OS structure, the multikernel, that treats the machine as a network of independent cores, assumes no inter-core sharing at the lowest level, and moves traditional OS functionality to a distributed system of processes that communicate via message-passing.We have implemented a multikernel OS to show that the approach is promising, and we describe how traditional scalability problems for operating systems (such as memory management) can be effectively recast using messages and can exploit insights from distributed systems and networking. An evaluation of our prototype on multicore systems shows that, even on present-day machines, the performance of a multikernel is comparable with a conventional OS, and can scale better to support future hardware.

/pdf/the-multikernel-a-new-os-architecture-for-scalable-multicore-1zaxph0v39.pdf

The multikernel: a new OS architecture for scalable multicore systems

Energy efficiency is the new fundamental limiter of processor performance, way beyond numbers of processors.

/pdf/the-future-of-microprocessors-15i7iemtj8.pdf

The future of microprocessors

Recent advances in computing have led to an explosion in the amount of data being generated. Processing the ever-growing data in a timely manner has made throughput computing an important aspect for emerging applications. Our analysis of a set of important throughput computing kernels shows that there is an ample amount of parallelism in these kernels which makes them suitable for today's multi-core CPUs and GPUs. In the past few years there have been many studies claiming GPUs deliver substantial speedups (between 10X and 1000X) over multi-core CPUs on these kernels. To understand where such large performance difference comes from, we perform a rigorous performance analysis and find that after applying optimizations appropriate for both CPUs and GPUs the performance gap between an Nvidia GTX280 processor and the Intel Core i7-960 processor narrows to only 2.5x on average. In this paper, we discuss optimization techniques for both CPU and GPU, analyze what architecture features contributed to performance differences between the two architectures, and recommend a set of architectural features which provide significant improvement in architectural efficiency for throughput kernels.

/pdf/debunking-the-100x-gpu-vs-cpu-myth-an-evaluation-of-1m7y4u1n7m.pdf

Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU

GPU architectures are increasingly important in the multi-core era due to their high number of parallel processors. Programming thousands of massively parallel threads is a big challenge for software engineers, but understanding the performance bottlenecks of those parallel programs on GPU architectures to improve application performance is even more difficult. Current approaches rely on programmers to tune their applications by exploiting the design space exhaustively without fully understanding the performance characteristics of their applications.To provide insights into the performance bottlenecks of parallel applications on GPU architectures, we propose a simple analytical model that estimates the execution time of massively parallel programs. The key component of our model is estimating the number of parallel memory requests (we call this the memory warp parallelism) by considering the number of running threads and memory bandwidth. Based on the degree of memory warp parallelism, the model estimates the cost of memory requests, thereby estimating the overall execution time of a program. Comparisons between the outcome of the model and the actual execution time in several GPUs show that the geometric mean of absolute error of our model on micro-benchmarks is 5.4% and on GPU computing applications is 13.3%. All the applications are written in the CUDA programming language.

/pdf/an-analytical-model-for-a-gpu-architecture-with-memory-level-4930mwitke.pdf

An analytical model for a GPU architecture with memory-level and thread-level parallelism awareness

We present PacketShader, a high-performance software router framework for general packet processing with Graphics Processing Unit (GPU) acceleration. PacketShader exploits the massively-parallel processing power of GPU to address the CPU bottleneck in current software routers. Combined with our high-performance packet I/O engine, PacketShader outperforms existing software routers by more than a factor of four, forwarding 64B IPv4 packets at 39 Gbps on a single commodity PC. We have implemented IPv4 and IPv6 forwarding, OpenFlow switching, and IPsec tunneling to demonstrate the flexibility and performance advantage of PacketShader. The evaluation results show that GPU brings significantly higher throughput over the CPU-only implementation, confirming the effectiveness of GPU for computation and memory-intensive operations in packet processing.

/pdf/packetshader-a-gpu-accelerated-software-router-1cyii2z7z5.pdf

PacketShader: a GPU-accelerated software router

A method and apparatus for a microprocessor with multiple memory read opportunity ports in a pipeline is disclosed. In one embodiment, a register file may have only one read port. When a statistically rare instruction requires two operands to be read from the register file, a spacer may be introduced into the pipeline, permitting the use of a second opportunity port to read its second operand from the register file at a later time. The spacer may be a nop, or it may be another instruction that receives its operands from a bypass path. In other embodiments, a register alias table may have only one read port, and a second opportunity port may be used to read a second physical register address.

Method and apparatus for utilizing multiple opportunity ports in a processor pipeline

A technique to increase memory bandwidth for throughput applications. In one embodiment, memory bandwidth can be increased, particularly for throughput applications, without increasing interconnect trace or pin count by pipelining pages between one or more memory storage areas on half cycles of a memory access clock.

Partition-free multi-socket memory system architecture

A system and method for generating a single compressed vector including two or more predetermined attribute values. For each of a plurality of data points such as pixels, if a first and a second attribute values of the data point are equal to a first and a second, respectively, of the two or more predetermined attribute values, the compressed vector is used to operate on the data point. Other embodiments are described and claimed.

Device, system, and method for improving processing efficiency by collectively applying operations

PROBLEM TO BE SOLVED: To prevent deterioration in performance or increase in power consumption by switching threads at an optimum timing or by optimum frequency. SOLUTION: A pre-fetch buffer bank 101 includes a plurality of buffer entries and stores a plurality of instructions concerning respective threads. The instruction fetched about each thread is transmitted to a multiplexer (mux) 105 via interconnects T0, T1, T2, T3. The multiplexer 105 selects one instruction corresponding to the selected thread, on the basis of a selection line from thread picker logic 110. The thread picker logic may determines which thread is to be selected, i.e., which instruction of the thread is to be performed, on the basis of the expression of the fetched instruction which is to be provided by a thread block indicator 115 and/or a decoded indication or a decoder 120. COPYRIGHT: (C)2010,JPO&INPIT

Technique for scheduling threads

A method and apparatus for efficiently handling texture sampling is described herein. A compiler or other software is capable of breaking a texture sampling operation for a pixel into a pre-fetch operation and a use operation. A processing element, in response to executing the pre-fetch operation, delegates computation of the texture sample of the pixel to a hardware texture sample unit. In parallel to the hardware texture sample unit performing a texture sample for the pixel and providing the result, i.e. a textured pixel (texel), to a destination address, the processing element is capable of executing other independent code. After an amount of time, the processing element executes the use operation, such as a load operation to load the texel from the destination address.

Eric Sprangle

Papers

Method and apparatus for utilizing multiple opportunity ports in a processor pipeline

Partition-free multi-socket memory system architecture

Device, system, and method for improving processing efficiency by collectively applying operations

Technique for scheduling threads

Mechanism for effectively handling texture sampling