While the use of MapReduce systems (such as Hadoop) for large scale data analysis has been widely recognized and studied, we have recently seen an explosion in the number of systems developed for cloud data serving. These newer systems address "cloud OLTP" applications, though they typically do not support ACID transactions. Examples of systems proposed for cloud serving use include BigTable, PNUTS, Cassandra, HBase, Azure, CouchDB, SimpleDB, Voldemort, and many others. Further, they are being applied to a diverse range of applications that differ considerably from traditional (e.g., TPC-C like) serving workloads. The number of emerging cloud serving systems and the wide range of proposed applications, coupled with a lack of apples-to-apples performance comparisons, makes it difficult to understand the tradeoffs between systems and the workloads for which they are suited. We present the "Yahoo! Cloud Serving Benchmark" (YCSB) framework, with the goal of facilitating performance comparisons of the new generation of cloud data serving systems. We define a core set of benchmarks and report results for four widely used systems: Cassandra, HBase, Yahoo!'s PNUTS, and a simple sharded MySQL implementation. We also hope to foster the development of additional cloud benchmark suites that represent other classes of applications by making our benchmark tool available via open source. In this regard, a key feature of the YCSB framework/tool is that it is extensible--it supports easy definition of new workloads, in addition to making it easy to benchmark new systems.

/pdf/benchmarking-cloud-serving-systems-with-ycsb-3u47swhmki.pdf

Benchmarking cloud serving systems with YCSB

This paper presents an online feature selection mechanism for evaluating multiple features while tracking and adjusting the set of features used to improve tracking performance. Our hypothesis is that the features that best discriminate between object and background are also best for tracking the object. Given a set of seed features, we compute log likelihood ratios of class conditional sample densities from object and background to form a new set of candidate features tailored to the local object/background discrimination task. The two-class variance ratio is used to rank these new features according to how well they separate sample distributions of object and background pixels. This feature evaluation mechanism is embedded in a mean-shift tracking system that adaptively selects the top-ranked discriminative features for tracking. Examples are presented that demonstrate how this method adapts to changing appearances of both tracked object and scene background. We note susceptibility of the variance ratio feature selection method to distraction by spatially correlated background clutter and develop an additional approach that seeks to minimize the likelihood of distraction.

/pdf/online-selection-of-discriminative-tracking-features-4rt4q4qm9g.pdf

Online selection of discriminative tracking features

PlanetLab is a global overlay network for developing and accessing broad-coverage network services. Our goal is to grow to 1000 geographically distributed nodes, connected by a disverse collection of links. PlanetLab allows multiple service to run concurrently and continuously, each in its own slice of PlanetLab. This paper discribes our initial implementation of PlanetLab, including the mechanisms used to impelment virtualization, and the collection of core services used to manage PlanetLab.

/pdf/planetlab-an-overlay-testbed-for-broad-coverage-services-2h7fbdqwx2.pdf

PlanetLab: an overlay testbed for broad-coverage services

Software Defined Networking (SDN) is an exciting technology that enables innovation in how we design and manage networks. Although this technology seems to have appeared suddenly, SDN is part of a long history of efforts to make computer networks more programmable. In this paper, we trace the intellectual history of programmable networks, including active networks, early efforts to separate the control and data plane, and more recent work on OpenFlow and network operating systems. We highlight key concepts, as well as the technology pushes and application pulls that spurred each innovation. Along the way, we debunk common myths and misconceptions about the technologies and clarify the relationship between SDN and related technologies such as network virtualization.

The road to SDN: an intellectual history of programmable networks

Scalable management and self-organizational capabilities are emerging as central requirements for a generation of large-scale, highly dynamic, distributed applications. We have developed an entirely new distributed information management system called Astrolabe. Astrolabe collects large-scale system state, permitting rapid updates and providing on-the-fly attribute aggregation. This latter capability permits an application to locate a resource, and also offers a scalable way to track system state as it evolves over time. The combination of features makes it possible to solve a wide variety of management and self-configuration problems. This paper describes the design of the system with a focus upon its scalability. After describing the Astrolabe service, we present examples of the use of Astrolabe for locating resources, publish-subscribe, and distributed synchronization in large systems. Astrolabe is implemented using a peer-to-peer protocol, and uses a restricted form of mobile code based on the SQL query language for aggregation. This protocol gives rise to a novel consistency model. Astrolabe addresses several security considerations using a built-in PKI. The scalability of the system is evaluated using both simulation and experiments; these confirm that Astrolabe could scale to thousands and perhaps millions of nodes, with information propagation delays in the tens of seconds.

/pdf/astrolabe-a-robust-and-scalable-technology-for-distributed-2681334o7d.pdf

Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining

Three experimental environments traditionally support network and distributed systems research: network emulators, network simulators, and live networks. The continued use of multiple approaches highlights both the value and inadequacy of each. Netbed, a descendant of Emulab, provides an experimentation facility that integrates these approaches, allowing researchers to configure and access networks composed of emulated, simulated, and wide-area nodes and links. Netbed's primary goals are ease of use, control, and realism, achieved through consistent use of virtualization and abstraction.By providing operating system-like services, such as resource allocation and scheduling, and by virtualizing heterogeneous resources, Netbed acts as a virtual machine for network experimentation. This paper presents Netbed's overall design and implementation and demonstrates its ability to improve experimental automation and efficiency. These, in turn, lead to new methods of experimentation, including automated parameter-space studies within emulation and straightforward comparisons of simulated, emulated, and wide-area scenarios.

An integrated experimental environment for distributed systems and networks

Today there exist three environments in which to perform experimental network and distributed systems research: network emulators, simulators, and live networks. The continued use of multiple approaches highlights both the importance and inadequacies of each. Emulab is an experimentation facility that seamlessly integrates these approaches. Emulab’s primary goals are ease of use, control, and realism. Unlike the constituent experimental platforms it leverages, Emulab achieves these goals simultaneously.

/pdf/an-integrated-experimental-environment-for-distributed-5ceauoist2.pdf

An Integrated Experimental Environment for Distributed Systems and Networks (full report)

A web-enabled resin panel customization website, hosted through a web portal, provides, through a client computer system, one or more user interfaces configured to receive a plurality of user selections, and provide a realistic display of the resulting output. In one implementation, a processing module overlays transparent portions of images representing a translucent resin substrate with one or more images representing decorative objects, films, or the like. The processing module can also combine pixel information from each of the added/overlain layers. In either case, the processing module sends a combined pixel data stream that can be rendered by a client system, and thus provide a user with a realistic depiction of the user's resin panel selections. The customization website further includes one or more user interfaces that enable a user to store specific resin panel design profiles, and order resin panels corresponding to those specific design profiles.

Viewing and ordering customized resin panels through web-based interfaces

The diverse requirements of network and distributed systems research are not well met by any single experimental environment. Competing approaches remain popular because each offers different levels of ease of use , control, andrealism. These include packet-level discrete event simulation, live network experimentation, and emulation, which subjects real hardware, protocols, and workloads to a synthetic network environment. Netbed offers a new alternative. It is software that, when deployed on local and wide-area machines, provides a platform for research, education, or development in distributed systems and networks. Netbed’s primary contribution is the seamless integrationof the above disparate techniques into a common architectural framework that preserves the control and ease of use of simulation, without sacrificing the realism of emulation and live network experimentation. This framework provides common abstractions, namespaces, services, and user interfaces to all three environments. Netbed’s integration allows tools such as topology and traffic generators that were originally targeted for one domain to apply to all. Netbed leverages and extends its ancestor, Emulab, which focused on making emulationas easy to use and control as simulation. Our results show that Emulab speeds up experiment configuration on a cluster by two orders of magnitude, and through timeand space-sharing, improves cluster utilization by another two orders of magnitude. Architecture: Netbed revolves around interacting state machines, monitored by a state management daemon. The central state machine is the “experiment.” Subsidiary state machines include those handling node allocation, configuration, and disk reloading. This approach copes well with the reliability challenges of large-scale distributed systems, composed of often unstable commodity hardware. The state daemon catches illegal or tardy state transitions. For example, if a node hangs while rebooting, the state daemon times out and attempts an alternate reboot mechanism. An experiment is generated from a user’s n script or via a GUI, resulting in hard state in Netbed’s relational database. It represents a network configuration, such as switch VLAN mappings or IP tunnels, and node configurations, including OS images. A node’s local disk is currently considered soft state; this allows an experiment to be “swapped out” and later regenerated from the database onto other hardware. The database provides a single namespace and abstractions for heterogeneous resources. The resulting resourcetransparency enables simulated, emulated, live,

Netbed: an integrated experimental environment

Complex distributed systems have many components that fail from time to time, adversely affecting the system’s overall reliability. Our advanced network testbed, Emulab, also known as Netbed, is a complex timeand space-shared distributed system, composed of thousands of hardware and software components. Early versions of the Emulab system had reliability problems, could only support experiments of limited size, and were inefficient in resource use. Our thesis is that enhancing Emulab with a flexible framework based on state machines provides better monitoring and control and improves the reliability, scalability, performance, efficiency, and generality of the system. Reliability is key, because of its effect on scalability, performance and efficiency, and any solution used to address these issues must be general and portable, across a variety of workloads and software and hardware configurations. Our results show improvement through faster error detection and recovery, better monitoring of testbed nodes, and prevention of race conditions. The state machine framework has also contributed to Emulab’s generality and portability by expanding the accepted workload, providing graceful degradation for less capable node configurations, and making it easier to add new node types to the testbed’s hardware base. To . . .

Mac Newbold

Papers

An integrated experimental environment for distributed systems and networks

An Integrated Experimental Environment for Distributed Systems and Networks (full report)

Viewing and ordering customized resin panels through web-based interfaces

Netbed: an integrated experimental environment

Reliability and state machines in an advanced network testbed