To support bursty traffic on the Internet (and especially WWW) efficiently, optical burst switching (OBS) is proposed as a way to streamline both protocols and hardware in building the future generation Optical Internet. By leveraging the attractive properties of optical communications and at the same time, taking into account its limitations, OBS combines the best of optical circuit-switching and packet/cell switching. In this paper, the general concept of OBS protocols and in particular, those based on Just-Enough-Time (JET), is described, along with the applicability of OBS protocols to IP over WDM. Specific issues such as the use of fiber delay-lines (FDLs) for accommodating processing delay and/or resolving conflicts are also discussed. In addition, the performance of JET-based OBS protocols which use an offset time along with delayed reservation to achieve efficient utilization of both bandwidth and FDLs as well as to support priority-based routing is evaluated.

Optical burst switching (OBS) - a new paradigm for an optical Internet

We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech-two vastly different languages. Because it replaces entire pipelines of hand-engineered components with neural networks, end-to-end learning allows us to handle a diverse variety of speech including noisy environments, accents and different languages. Key to our approach is our application of HPC techniques, enabling experiments that previously took weeks to now run in days. This allows us to iterate more quickly to identify superior architectures and algorithms. As a result, in several cases, our system is competitive with the transcription of human workers when benchmarked on standard datasets. Finally, using a technique called Batch Dispatch with GPUs in the data center, we show that our system can be inexpensively deployed in an online setting, delivering low latency when serving users at scale.

/pdf/end-to-end-speech-recognition-in-english-and-mandarin-3d91x54v7w.pdf

End to end speech recognition in English and Mandarin

Mobile ad hoc networks (MANETs) represent complex distributed systems that comprise wireless mobile nodes that can freely and dynamically self-organize into arbitrary and temporary, ‘‘ad-hoc’’ network topologies, allowing people and devices to seamlessly internetwork in areas with no pre-existing communication infrastructure, e.g., disaster recovery environments. Ad hoc networking concept is not a new one, having been around in various forms for over 20 years. Traditionally, tactical networks have been the only communication networking application that followed the ad hoc paradigm. Recently, the introduction of new technologies such as the Bluetooth, IEEE 802.11 and Hyperlan are helping enable eventual commercial MANET deployments outside the military domain. These recent evolutions have been generating a renewed and growing interest in the research and development of MANET. This paper attempts to provide a comprehensive overview of this dynamic field. It first explains the important role that mobile ad hoc networks play in the evolution of future wireless technologies. Then, it reviews the latest research activities in these areas, including a summary of MANETs characteristics, capabilities, applications, and design constraints. The paper concludes by presenting a set of challenges and problems requiring further research in the future. � 2003 Elsevier B.V. All rights reserved.

/pdf/mobile-ad-hoc-networking-imperatives-and-challenges-56mord86q0.pdf

Mobile ad hoc networking: imperatives and challenges

Training modern deep learning models requires large amounts of computation, often provided by GPUs. Scaling computation from one GPU to many can enable much faster training and research progress but entails two complications. First, the training library must support inter-GPU communication. Depending on the particular methods employed, this communication may entail anywhere from negligible to significant overhead. Second, the user must modify his or her training code to take advantage of inter-GPU communication. Depending on the training library's API, the modification required may be either significant or minimal. 
Existing methods for enabling multi-GPU training under the TensorFlow library entail non-negligible communication overhead and require users to heavily modify their model-building code, leading many researchers to avoid the whole mess and stick with slower single-GPU training. In this paper we introduce Horovod, an open source library that improves on both obstructions to scaling: it employs efficient inter-GPU communication via ring reduction and requires only a few lines of modification to user code, enabling faster, easier distributed training in TensorFlow. Horovod is available under the Apache 2.0 license at this https URL

Horovod: fast and easy distributed deep learning in TensorFlow

Experience froman operational Map-Reduce cluster reveals that outliers significantly prolong job completion. The causes for outliers include run-time contention for processor, memory and other resources, disk failures, varying bandwidth and congestion along network paths and, imbalance in task workload. We present Mantri, a system that monitors tasks and culls outliers using cause- and resource-aware techniques. Mantri's strategies include restarting outliers, network-aware placement of tasks and protecting outputs of valuable tasks. Using real-time progress reports, Mantri detects and acts on outliers early in their lifetime. Early action frees up resources that can be used by subsequent tasks and expedites the job overall. Acting based on the causes and the resource and opportunity cost of actions lets Mantri improve over prior work that only duplicates the laggards. Deployment in Bing's production clusters and trace-driven simulations show that Mantri improves job completion times by 32%.

/pdf/reining-in-the-outliers-in-map-reduce-clusters-using-mantri-35h1fwhz5f.pdf

Reining in the outliers in map-reduce clusters using Mantri

Bandwidth optimal all-reduce algorithms for clusters of workstations

Multiconstrained quality-of-service (QoS) routing deals with finding routes that satisfy multiple independent QoS constraints. This problem is NP-hard. In this paper, two heuristics, the limited granularity heuristic and the limited path heuristic, are investigated. Both heuristics extend the Bellman-Ford shortest path algorithm and solve general k-constrained QoS routing problems. Analytical and simulation studies are conducted to compare the time/space requirements of the heuristics and the effectiveness of the heuristics in finding paths that satisfy the QoS constraints. The major results of this paper are the following. For an N-nodes and E-edges network with k (a small constant) independent QoS constraints, the limited granularity heuristic must maintain a table of size O(|N|k- 1) in each node to be effective, which results in a time complexity of O (|N|K|E|); while the limited path heuristic can achieve very high performance by maintaining O (|N|2lg(|N|)) entries in each node. These results indicate that the limited path heuristic is relatively insensitive to the number of constraints and is superior to the limited granularity heuristic in solving k-constrained QoS routing problems when k > 3.

/pdf/heuristic-algorithms-for-multiconstrained-quality-of-service-1roqfzi341.pdf

Heuristic algorithms for multiconstrained quality-of-service routing

In order for collective communication routines to achieve high performance on different platforms, they must be able to adapt to the system architecture and use different algorithms for different situations. Current Message Passing Interface (MPI) implementations, such as MPICH and LAM/MPI, are not fully adaptable to the system architecture and are not able to achieve high performance on many platforms. In this paper, we present a system that produces efficient MPI collective communication routines. By automatically generating topology specific routines and using an empirical approach to select the best implementations, our system adapts to a given platform and constructs routines that are customized for the platform. The experimental results show that the tuned routines consistently achieve high performance on clusters with different network topologies.

/pdf/automatic-generation-and-tuning-of-mpi-collective-28vww86dce.pdf

Automatic generation and tuning of MPI collective communication routines

Multi-constrained quality of service (QoS) routing finds a route in the network that satisfies multiple independent quality of service constraints. This problem is NP-hard and a number of heuristic algorithms have been proposed to solve the problem. This paper studies two heuristics, the limited granularity heuristic and the limited path heuristic, for solving general k-constrained problems. Analytical and simulation studies are conducted to compare the time/space requirements of the heuristics and the effectiveness of the heuristics in finding the paths that satisfy the QoS constraints. We prove analytically that for an N nodes and E edges network with k (a small constant) independent QoS constraints, the limited granularity heuristic must maintain a table of size O(|N|/sup k-1/) in each node to be effective, which results in a time complexity of O(|N|/sup k/|E|). We also prove that the limited path heuristic can achieve very high performance by maintaining O(|N|/sup 2/lg(|N|)) entries in each node, which indicates that the performance of the limited path heuristic is not sensitive to the number of constraints. We conclude that although both the limited granularity heuristic and the limited path heuristic can efficiently solve 2-constrained QoS routing problems, the limited path heuristic is superior to the limited granularity heuristic in solving k-constrained QoS routing problems when k>3. Our simulation study further confirms this conclusion.

/pdf/heuristic-algorithms-for-multi-constrained-quality-of-ovrljrg67s.pdf

Heuristic algorithms for multi-constrained quality of service routing

This paper describes distributed (or decentralized) protocols for establishing wavelength paths in point-to-point WDM networks. Distributed control improves reliability as well as scalability and reduces implementation cost of a network, but also presents major challenges in managing/allocating wavelengths efficiently. Two types of distributed wavelength reservation protocols are proposed and evaluated, namely forward and backward. The benefit of wavelength conversion is assessed first based on the evaluation of the forward reservation protocols, and it is found that wavelength conversion can result in a performance advantage in a distributed environment. For networks without wavelength conversion, a class of backward reservation protocols is studied and shown to generally perform better than their forward counterparts.

Xin Yuan

Papers

Bandwidth optimal all-reduce algorithms for clusters of workstations

Heuristic algorithms for multiconstrained quality-of-service routing

Automatic generation and tuning of MPI collective communication routines

Heuristic algorithms for multi-constrained quality of service routing

Distributed control protocols for wavelength reservation and their performance evaluation