One measure of the usefulness of a general-purpose distributed computing system is the system's ability to provide a level of performance commensurate to the degree of multiplicity of resources present in the system. A taxonomy of approaches to the resource management problem is presented in an attempt to provide a common terminology and classification mechanism necessary in addressing this problem. The taxonomy, while presented and discussed in terms of distributed scheduling, is also applicable to most types of resource management. >

/pdf/a-taxonomy-of-scheduling-in-general-purpose-distributed-2ar4ly9zfl.pdf

A taxonomy of scheduling in general-purpose distributed computing systems

The problem of designing a logical topology over a wavelength-routed all-optical network (AON) physical topology is studied. The physical topology consists of the nodes and fiber links in the network. On an AON physical topology, we can set up lightpaths between pairs of nodes, where a lightpath represents a direct optical connection without any intermediate electronics. The set of lightpaths along with the nodes constitutes the logical topology. For a given network physical topology and traffic pattern, our objective is to design the logical topology and the routing algorithm so as to minimize the network congestion while constraining the average delay seen by a source-destination pair and the amount of processing required at the nodes (degree of the logical topology). Ignoring the delay constraints can result in fairly convoluted logical topologies with very long delays. On the other hand, in all our examples, imposing it results in a minimal increase in congestion. While the number of wavelengths required to imbed the resulting logical topology on the physical all optical topology is also a constraint in general, we find that in many cases of interest this number can be quite small. We formulate the combined logical topology design and routing problem described above as a mixed integer linear programming problem which we then solve for a number of cases of a six-node network. This programming problem is split into two subproblems: logical topology design, and routing. We then compare the performance of several heuristic topology design algorithms against that of randomly generated topologies, as well as lower bounds.

Design of logical topologies for wavelength-routed optical networks

Internet-wide network scanning has numerous security applications, including exposing new vulnerabilities and tracking the adoption of defensive mechanisms, but probing the entire public address space with existing tools is both difficult and slow. We introduce ZMap, a modular, open-source network scanner specifically architected to perform Internet-wide scans and capable of surveying the entire IPv4 address space in under 45 minutes from user space on a single machine, approaching the theoretical maximum speed of gigabit Ethernet. We present the scanner architecture, experimentally characterize its performance and accuracy, and explore the security implications of high speed Internet-scale network surveys, both offensive and defensive. We also discuss best practices for good Internet citizenship when performing Internet-wide surveys, informed by our own experiences conducting a long-term research survey over the past year.

/pdf/zmap-fast-internet-wide-scanning-and-its-security-3utgzu4gwq.pdf

ZMap: fast internet-wide scanning and its security applications

To support bursty traffic on the Internet (and especially WWW) efficiently, optical burst switching (OBS) is proposed as a way to streamline both protocols and hardware in building the future gener...

Optical burst switching (OBS) - a new paradigm for an optical Internet

Modern Internet systems often combine different applications (e.g., DNS, web, and database), span different administrative domains, and function in the context of network mechanisms like tunnels, VPNs, NATs, and overlays. Diagnosing these complex systems is a daunting challenge. Although many diagnostic tools exist, they are typically designed for a specific layer (e.g., traceroute) or application, and there is currently no tool for reconstructing a comprehensive view of service behavior. In this paper we propose X-Trace, a tracing framework that provides such a comprehensive view for systems that adopt it. We have implemented X-Trace in several protocols and software systems, and we discuss how it works in three deployed scenarios: DNS resolution, a three-tiered photo-hosting website, and a service accessed through an overlay network.

/pdf/x-trace-a-pervasive-network-tracing-framework-2j9x5xvue1.pdf

X-trace: a pervasive network tracing framework

Prior measurement studies of the Internet have explored traffic and topology, but have largely ignored edge hosts. While the number of Internet hosts is very large, and many are hidden behind firewalls or in private address space, there is much to be learned from examining the population of visible hosts, those with public unicast addresses that respond to messages. In this paper we introduce two new approaches to explore the visible Internet. Applying statistical population sampling, we use censuses to walk the entire Internet address space, and surveys to probe frequently a fraction of that space. We then use these tools to evaluate address usage, where we find that only 3.6% of allocated addresses are actually occupied by visible hosts, and that occupancy is unevenly distributed, with a quarter of responsive /24 address blocks (subnets) less than 5% full, and only 9% of blocks more than half full. We show about 34 million addresses are very stable and visible to our probes (about 16% of responsive addresses), and we project from this up to 60 million stable Internet-accessible computers. The remainder of allocated addresses are used intermittently, with a median occupancy of 81 minutes. Finally, we show that many firewalls are visible, measuring significant diversity in the distribution of firewalled block size. To our knowledge, we are the first to take a census of edge hosts in the visible Internet since 1982, to evaluate the accuracy of active probing for address census and survey, and to quantify these aspects of the Internet.

/pdf/census-and-survey-of-the-visible-internet-vnskeitdms.pdf

Census and survey of the visible internet

The twofold purpose of this research is to develop mathematical programming tools for the optimal or near-optimal design of a new type of network architecture called the wavelength-division optical network (WON) and to discover underlying design principles for such networks via experimentation using these tools. The WON, a multichannel, multihop lightwave network with tunable transceivers, is suitable for use as a metropolitan area network. A unique feature of the WON is that a great number of virtual topologies can be mapped onto a given physical topology. The problems of virtual- and physical-topology design in the WON are described, techniques for their solution are presented, and the interplay between these problems is studied. >

Topological design of the wavelength-division optical network

This paper examines task allocation in fault-tolerant distributed systems. The problem is formulated as a constrained sum of squares minimization problem. The computational complexity of this problem prompts us to consider an efficient approximation algorithm. We show that the ratio of the performance of the approximation algorithm to that of the optimal solution is bounded by 9m/(8m?r+1)), wherem is the number of processors to be allocated andr is the number of times each task is to be replicated. Experience with the algorithm suggests that even better performance ratios can be expected.

Task allocation in fault-tolerant distributed systems

When nodes of a communication network have identical input- and output-link capacities, it is possible to use as few as one packet buffer per link, if one is willing to deflect-or misroute-a subset of simultaneously arriving fixed-length packets from preferred to alternate output links. This scheme, known as deflection routing, can achieve very fast packet switching in regular networks and has been proposed as the basic routing and switching protocol of several all-optical networks. The performance models of deflection-routing networks that have appeared in the literature have assumed that time is slotted and packets arrive at nodes on time-slot boundaries. In practice, however, slotted operation is difficult to implement in all-optical networks. The present authors evaluate by simulation the performance of deflection routing in unslotted networks. The evaluations show a surprising degradation of throughput in unslotted deflection routing networks, compared to slotted networks, and reveal situations where severe congestion occurs. To overcome these limitations they propose the use of specific control mechanisms in unslotted networks that allow one to eliminate congestion and to improve substantially the network throughput. >

Unslotted deflection routing in all-optical networks

Optical CDMA local area networks allow shared access to a broadcast medium. Every node on the network is assigned an optical orthogonal codeword (OOC) to transmit or receive on. OOCs are designed to be pseudo-orthogonal, i.e., the correlation (and therefore the interference) between pairs of codewords is constrained. This paper demonstrates that the use of optical CDMA does not preclude the need for a media access control (MAC) layer protocol to resolve contention for the shared media. OOCs have low spectral efficiency. As more codewords are transmitted simultaneously, the interference between codewords increases and the network throughput falls. This paper analyzes a network architecture where there is virtually no MAC layer, except for choice of the codeset, and shows that its throughput degrades and collapses under moderate to heavy load. We propose an alternate architecture called interference avoidance where nodes on the network use media access mechanisms to avoid causing interference on the line, thereby improving network throughput. Interference avoidance is analyzed and it is shown that it can provide up to 30% improvement in throughput with low delays and no throughput collapse. We validate our analysis through simulation with realistic network traffic traces.

Joseph Bannister

Papers

Census and survey of the visible internet

Topological design of the wavelength-division optical network

Task allocation in fault-tolerant distributed systems

Unslotted deflection routing in all-optical networks

The need for media access control in optical CDMA networks