Dune is a system that provides applications with direct but safe access to hardware features such as ring protection, page tables, and tagged TLBs, while preserving the existing OS interfaces for processes. Dune uses the virtualization hardware in modern processors to provide a process, rather than a machine abstraction. It consists of a small kernel module that initializes virtualization hardware and mediates interactions with the kernel, and a user-level library that helps applications manage privileged hardware features. We present the implementation of Dune for 64- bit x86 Linux. We use Dune to implement three user-level applications that can benefit from access to privileged hardware: a sandbox for untrusted code, a privilege separation facility, and a garbage collector. The use of Dune greatly simplifies the implementation of these applications and provides significant performance advantages.

/pdf/dune-safe-user-level-access-to-privileged-cpu-features-3asykpax4p.pdf

Dune: safe user-level access to privileged CPU features

Published in 1996, Richard Joness Garbage Collection was a milestone in the area of automatic memory management. The field has grown considerably since then, sparking a need for an updated look at the latest state-of-the-art developments. The Garbage Collection Handbook: The Art of Automatic Memory Management brings together a wealth of knowledge gathered by automatic memory management researchers and developers over the past fifty years. The authors compare the most important approaches and state-of-the-art techniques in a single, accessible framework. The book addresses new challenges to garbage collection made by recent advances in hardware and software. It explores the consequences of these changes for designers and implementers of high performance garbage collectors. Along with simple and traditional algorithms, the book covers parallel, incremental, concurrent, and real-time garbage collection. Algorithms and concepts are often described with pseudocode and illustrations. The nearly universal adoption of garbage collection by modern programming languages makes a thorough understanding of this topic essential for any programmer. This authoritative handbook gives expert insight on how different collectors work as well as the various issues currently facing garbage collectors. Armed with this knowledge, programmers can confidently select and configure the many choices of garbage collectors. Web ResourceThe books online bibliographic database at www.gchandbook.org includes over 2,500 garbage collection-related publications. Continually updated, it contains abstracts for some entries and URLs or DOIs for most of the electronically available ones. The database can be searched online or downloaded as BibTeX, PostScript, or PDF.

The Garbage Collection Handbook: The Art of Automatic Memory Management

Chip-multiprocessors (CMPs) must often execute workload mixes with different performance requirements. On one hand, user-facing, latency-critical applications (e.g., web search) need low tail (i.e., worst-case) latencies, often in the millisecond range, and have inherently low utilization. On the other hand, compute-intensive batch applications (e.g., MapReduce) only need high long-term average performance. In current CMPs, latency-critical and batch applications cannot run concurrently due to interference on shared resources. Unfortunately, prior work on quality of service (QoS) in CMPs has focused on guaranteeing average performance, not tail latency. In this work, we analyze several latency-critical workloads, and show that guaranteeing average performance is insufficient to maintain low tail latency, because microarchitectural resources with state, such as caches or cores, exert inertia on instantaneous workload performance. Last-level caches impart the highest inertia, as workloads take tens of milliseconds to warm them up. When left unmanaged, or when managed with conventional QoS frameworks, shared last-level caches degrade tail latency significantly. Instead, we propose Ubik, a dynamic partitioning technique that predicts and exploits the transient behavior of latency-critical workloads to maintain their tail latency while maximizing the cache space available to batch applications. Using extensive simulations, we show that, while conventional QoS frameworks degrade tail latency by up to 2.3x, Ubik simultaneously maintains the tail latency of latency-critical workloads and significantly improves the performance of batch applications.

/pdf/ubik-efficient-cache-sharing-with-strict-qos-for-latency-1kdrpwzvjv.pdf

Ubik: efficient cache sharing with strict qos for latency-critical workloads

What fundamental opportunities for scalability are latent in interfaces, such as system call APIs? Can scalability opportunities be identified even before any implementation exists, simply by considering interface specifications? To answer these questions this paper introduces the following rule: Whenever interface operations commute, they can be implemented in a way that scales. This rule aids developers in building more scalable software starting from interface design and carrying on through implementation, testing, and evaluation.To help developers apply the rule, a new tool named Commuter accepts high-level interface models and generates tests of operations that commute and hence could scale. Using these tests, Commuter can evaluate the scalability of an implementation. We apply Commuter to 18 POSIX calls and use the results to guide the implementation of a new research operating system kernel called sv6. Linux scales for 68% of the 13,664 tests generated by Commuter for these calls, and Commuter finds many problems that have been observed to limit application scalability. sv6 scales for 99% of the tests.

/pdf/the-scalable-commutativity-rule-designing-scalable-software-4ilrf4n7cl.pdf

The scalable commutativity rule: designing scalable software for multicore processors

Virtual machines in the cloud typically run existing general-purpose operating systems such as Linux. We notice that the cloud's hypervisor already provides some features, such as isolation and hardware abstraction, which are duplicated by traditional operating systems, and that this duplication comes at a cost.

We present the design and implementation of OSv, a new guest operating system designed specifically for running a single application on a virtual machine in the cloud. It addresses the duplication issues by using a low-overhead library-OS-like design. It runs existing applications written for Linux, as well as new applications written for OSv. We demonstrate that OSv is able to efficiently run a variety of existing applications. We demonstrate its sub-second boot time, small OS image and how it makes more memory available to the application. For unmodified network-intensive applications, we demonstrate up to 25% increase in throughput and 47% decrease in latency. By using non-POSIX network APIs, we can further improve performance and demonstrate a 290% increase in Memcached throughput.

/pdf/os-v-optimizing-the-operating-system-for-virtual-machines-3h3gfey0xf.pdf

OS v : optimizing the operating system for virtual machines

C4, the Continuously Concurrent Compacting Collector, an updated generational form of the Pauseless GC Algorithm [7], is introduced and described, along with details of its implementation on modern X86 hardware. It uses a read barrier to support concur- rent compaction, concurrent remapping, and concurrent incremental update tracing. C4 differentiates itself from other generational garbage collectors by supporting simultaneous-generational concurrency: the different generations are collected using concurrent (non stop-the-world) mechanisms that can be simultaneously and independently active. C4 is able to continuously perform concurrent young generation collections, even during long periods of concurrent full heap collection, allowing C4 to sustain high allocation rates and maintain the efficiency typical to generational collectors, without sacrificing response times or reverting to stop-the-world operation. Azul systems has been shipping a commercial implementation of the Pauseless GC mechanism, since 2005. Three successive generations of Azul's Vega series systems relied on custom multi-core processors and a custom OS kernel to deliver both the scale and features needed to support Pauseless GC. In 2010, Azul released its first software-only commercial implementation of C4 for modern commodity X86 hardware, using Linux kernel enhancements to support the required feature set. We discuss implementa- tion details of C4 on X86, including the Linux virtual and physical memory management enhancements that were used to support the high rate of virtual memory operations required for sustained pauseless operation. We discuss updates to the collector's manage- ment of the heap for efficient generational collection and provide throughput and pause time data while running sustained workloads.

C4: the continuously concurrent compacting collector

We describe the Collie collector, a fully concurrent compacting collector that uses transactional memory techniques to achieve wait-free compaction. The collector uses compaction as the primary means of reclaiming unused memory, and performs "individual object transplantations" as transactions. We introduce new terms and requirements useful for analyzing concurrent relocating collectors, including definitions of referrer sets, object transplantation and the notion of individually transplantable objects. The Collie collector builds on these terms and on a detailed analysis of an object's legal states during compaction.Collie uses a combination of read barriers, write barriers and transactional memory operations. Its read-barrier supports fast, direct object referencing while using a bound, constant time, wait- free triggering path. Collie thereby avoids the constant indirection cost of Brooks [9] style barriers or handle-based heaps [25]. Collie is demonstrated using speculative multi-address atomicity [11], a form of hardware transactional memory supported by the Azul Vega architecture [2].We evaluate the Collie collector on the Azul platform, on which previous concurrent collectors such as the Pauseless Collector [12] and its generational variant [30] have been commercially available for several years. We discuss Collie's performance while running sustained workloads, and compare it to the Pauseless collector on the same platform. The Collie collector provides significant MMU [5] improvements even in the 1-msec time windows compared to the Pauseless collector. At the same time, it matches Pauseless in throughput and in the ability to scale to large heap sizes.We believe that the Collie collector is the first garbage collector to leverage hardware-assisted transactional memory. While Collie directly leverages Vega's speculative multi-address atomicity feature (SMA) [11], its design can be easily adapted to other hardware- assisted transactional memory systems. Specifically, the upcoming Intel TSX instruction set extensions [21] include capabilities similar to SMA. We expect Collie to be easily implementable on future commodity servers based on Intel Haswell processors and following processor generations.

The Collie: a wait-free compacting collector

Parallel marking algorithms use multiple threads to walk through the object heap graph and mark each reachable object as live. Parallel marker threads mark an object "live" by atomically setting a bit in a mark-bitmap or a bit in the object header. Most of these parallel algorithms strive to improve the marking throughput by using work-stealing algorithms for load-balancing and to ensure that all participating threads are kept busy. A purely "processor-centric" load-balancing approach in conjunction with a need to atomically set the mark bit, results in significant contention during parallel marking. This limits the scalability and throughput of parallel marking algorithms.We describe a new non-blocking and lock-free, work-sharing algorithm, the primary goal being to reduce contention during atomic updates of the mark-bitmap by parallel task-threads. Our work-sharing mechanism uses the address of a word in the mark-bitmap as the key to stripe work among parallel task-threads, with only a subset of the task-threads working on each stripe. This filters out most of the contention during parallel marking with 20% improvements in performance.In case of concurrent and on-the-fly collector algorithms, mutator threads also generate marking-work for the marking task-threads. In these schemes, mutator threads are also provided with thread-local marking stacks where they collect references to potentially "gray" objects, i.e., objects that haven't been "marked-through" by the collector. We note that since this work is generated by mutators when they reference these objects, there is a high likelihood that these objects continue to be present in the processor cache. We describe and evaluate a scheme to distribute mutator generated marking work among the collector's task-threads that is cognizant of the processor and cache topology. We prototype both our algorithms within the C4 [28] collector that ships as part of an industrial strength JVM for the Linux-X86 platform.

Scalable concurrent and parallel mark

For languages supporting concurrency the implementation of synchronization primitives is important for achieving high-performance. Many concurrent languages use object based locks to control access to critical regions. When lock ownership doesn't change for most of its lifetime, lock biasing allows a thread to take ownership of an object so that atomic operations aren't necessary on lock entry and exit. Revoking ownership of locks biased to a thread is an expensive operation compared to the atomic operation, as the thread that holds the lock must be suspended.When lock revocation occurs it is common for the object being locked to be modified so that future lock attempts use atomic operations. When repeated revocations occur the locking policy can reduce the amount of lock biasing that the system performs. Factors that can drive this include the type of the object being revoked and how recently it was allocated. The system must achieve a balance between being pessimistic about biased lock use and avoiding revocations.This work introduces a new locking protocol where revocations can be sampled by the locker without having to bias. The mechanism provides locking information specific to a particular instance that can be used to avoid unprofitable bias lock speculation and create a better locking policy. We demonstrate a new instance specific locking policy implemented in the Zing Virtual Machine, an extension of the HotSpot Java Virtual Machine. We present results on how the sampling window effects the number of atomic lock operations and revocations for the SPECjvm2008 and DaCapo Bach benchmark suites.

Balaji Iyengar

Papers

C4: the continuously concurrent compacting collector

The Collie: a wait-free compacting collector

Scalable concurrent and parallel mark

Reducing biased lock revocation by learning