Robust and powerful software instrumentation tools are essential for program analysis tasks such as profiling, performance evaluation, and bug detection. To meet this need, we have developed a new instrumentation system called Pin. Our goals are to provide easy-to-use, portable, transparent, and efficient instrumentation. Instrumentation tools (called Pintools) are written in C/C++ using Pin's rich API. Pin follows the model of ATOM, allowing the tool writer to analyze an application at the instruction level without the need for detailed knowledge of the underlying instruction set. The API is designed to be architecture independent whenever possible, making Pintools source compatible across different architectures. However, a Pintool can access architecture-specific details when necessary. Instrumentation with Pin is mostly transparent as the application and Pintool observe the application's original, uninstrumented behavior. Pin uses dynamic compilation to instrument executables while they are running. For efficiency, Pin uses several techniques, including inlining, register re-allocation, liveness analysis, and instruction scheduling to optimize instrumentation. This fully automated approach delivers significantly better instrumentation performance than similar tools. For example, Pin is 3.3x faster than Valgrind and 2x faster than DynamoRIO for basic-block counting. To illustrate Pin's versatility, we describe two Pintools in daily use to analyze production software. Pin is publicly available for Linux platforms on four architectures: IA32 (32-bit x86), EM64T (64-bit x86), Itanium®, and ARM. In the ten months since Pin 2 was released in July 2004, there have been over 3000 downloads from its website.

/pdf/pin-building-customized-program-analysis-tools-with-dynamic-1cvwqey5sb.pdf

Pin: building customized program analysis tools with dynamic instrumentation

Aspect] is a simple and practical aspect-oriented extension to Java With just a few new constructs, AspectJ provides support for modular implementation of a range of crosscutting concerns. In AspectJ's dynamic join point model, join points are well-defined points in the execution of the program; pointcuts are collections of join points; advice are special method-like constructs that can be attached to pointcuts; and aspects are modular units of crosscutting implementation, comprising pointcuts, advice, and ordinary Java member declarations. AspectJ code is compiled into standard Java bytecode. Simple extensions to existing Java development environments make it possible to browse the crosscutting structure of aspects in the same kind of way as one browses the inheritance structure of classes. Several examples show that AspectJ is powerful, and that programs written using it are easy to understand.

An overview of AspectJ

The Rise of the Creative Class

Timely and cost-effective analytics over “Big Data” is now a key ingredient for success in many businesses, scientific and engineering disciplines, and government endeavors. The Hadoop software stack—which consists of an extensible MapReduce execution engine, pluggable distributed storage engines, and a range of procedural to declarative interfaces—is a popular choice for big data analytics. Most practitioners of big data analytics—like computational scientists, systems researchers, and business analysts—lack the expertise to tune the system to get good performance. Unfortunately, Hadoop’s performance out of the box leaves much to be desired, leading to suboptimal use of resources, time, and money (in payas-you-go clouds). We introduce Starfish, a self-tuning system for big data analytics. Starfish builds on Hadoop while adapting to user needs and system workloads to provide good performance automatically, without any need for users to understand and manipulate the many tuning knobs in Hadoop. While Starfish’s system architecture is guided by work on self-tuning database systems, we discuss how new analysis practices over big data pose new challenges; leading us to different design choices in Starfish.

/pdf/starfish-a-self-tuning-system-for-big-data-analytics-1c4ab07m8g.pdf

Starfish: A Self-tuning System for Big Data Analytics.

Tools to understand complex system behaviour are essential for many performance analysis and debugging tasks, yet there are many open research problems in their development. Magpie is a toolchain for automatically extracting a system's workload under realistic operating conditions. Using low-overhead instrumentation, we monitor the system to record fine-grained events generated by kernel, middleware and application components. The Magpie request extraction tool uses an application-specific event schema to correlate these events, and hence precisely capture the control flow and resource consumption of each and every request. By removing scheduling artefacts, whilst preserving causal dependencies, we obtain canonical request descriptions from which we can construct concise workload models suitable for performance prediction and change detection. In this paper we describe and evaluate the capability of Magpie to accurately extract requests and construct representative models of system behaviour.

/pdf/using-magpie-for-request-extraction-and-workload-modelling-1gfx849p4j.pdf

Using magpie for request extraction and workload modelling

A method for tracing an instrumented program, including associating an instrumentation provider with a trace point to provide a probe in the instrumented program, selectively enabling the probe to obtain an enabled probe, wherein enabling the probe includes defining an action to perform when the enabled probe is fired, firing the enabled probe during execution of the instrumented program, and performing the action when the enabled probe is fired, wherein the action is performed by a tracing framework.

Mechanism for disjoint instrumentation providers in a tracing framework

Instruction set architectures for compute-centric object stores. An exemplary method may include receiving a request from a user, the request identifying parameters of a compute operation that is to be executed against one or more objects in a distributed object store, generating a set of tasks from the request that comprise instructions for a daemon, locating the one or more objects within the distributed object store, the one or more objects being stored on a physical node. The method includes providing the set of tasks to a daemon, the daemon controlling execution of the compute operation by a virtual operating system container based upon the set of tasks, and storing an output of the virtual operating system container in the distributed object store.

Instruction set architecture for compute-based object stores

A method for storing and processing data includes providing an operating system (OS) virtualization running on a processor and having a plurality of containers. Each container may prevent privilege escalation by a user to an administrator of a global zone running the OS virtualization. The method may also include providing a hardware virtual machine (HVM) for the user, the HVM encapsulated in one of the containers. A system for storing and processing data is provided that includes an operating system (OS) virtualization stored in a memory and running on a processor. The OS virtualization has a plurality of containers, and each container prevents privilege escalation by a user to an administrator of a global zone running the OS virtualization. The HVM may be encapsulated in one of the containers. A non-transitory computer readable storage medium having a program recorded thereon is provided.

Systems and methods for double hulled virtualization operations

A method of a speculative tracing, including defining the speculative tracing using a plurality of probes, firing at least one of the plurality of probes defined by the speculative tracing, allocating at least one instance of a first speculative buffer arranged to transfer data to a first principal buffer, if one of the plurality of probes comprises a first speculation function, and determining a first state value associated with the first speculative buffer.

Method and apparatus for speculative tracing

Methods and apparatus for enabling timeouts with arbitrary resolutions to be implemented are disclosed. According to one aspect of the present invention, a method for enabling a device driver to communicate with a processor in a computing system includes exchanging information between the device driver and a clock system, and exchanging information between the clock system and a cyclic system. Information is also exchanged between the cyclic system and the processor. Although the clock system indirectly exchanges information with the processor, the clock system does not directly exchange information with the processor. In one embodiment, the clock system includes a callout system and a system clock, and exchanging information between the device driver and the clock system includes exchanging information between the system clock and the callout system, and exchanging information between the callout system and the device driver.

Bryan M. Cantrill

Papers

Mechanism for disjoint instrumentation providers in a tracing framework

Instruction set architecture for compute-based object stores

Systems and methods for double hulled virtualization operations

Method and apparatus for speculative tracing

Method and apparatus for arbitrary resolution interval timeouts