scispace - formally typeset
Search or ask a question

Showing papers on "Scalability published in 2003"


Proceedings ArticleDOI
05 Nov 2003
TL;DR: It is argued that TPSN roughly gives a 2x better performance as compared to Reference Broadcast Synchronization (RBS) and verify this by implementing RBS on motes and use simulations to verify its accuracy over large-scale networks.
Abstract: Wireless ad-hoc sensor networks have emerged as an interesting and important research area in the last few years. The applications envisioned for such networks require collaborative execution of a distributed task amongst a large set of sensor nodes. This is realized by exchanging messages that are time-stamped using the local clocks on the nodes. Therefore, time synchronization becomes an indispensable piece of infrastructure in such systems. For years, protocols such as NTP have kept the clocks of networked systems in perfect synchrony. However, this new class of networks has a large density of nodes and very limited energy resource at every node; this leads to scalability requirements while limiting the resources that can be used to achieve them. A new approach to time synchronization is needed for sensor networks.In this paper, we present Timing-sync Protocol for Sensor Networks (TPSN) that aims at providing network-wide time synchronization in a sensor network. The algorithm works in two steps. In the first step, a hierarchical structure is established in the network and then a pair wise synchronization is performed along the edges of this structure to establish a global timescale throughout the network. Eventually all nodes in the network synchronize their clocks to a reference node. We implement our algorithm on Berkeley motes and show that it can synchronize a pair of neighboring motes to an average accuracy of less than 20ms. We argue that TPSN roughly gives a 2x better performance as compared to Reference Broadcast Synchronization (RBS) and verify this by implementing RBS on motes. We also show the performance of TPSN over small multihop networks of motes and use simulations to verify its accuracy over large-scale networks. We show that the synchronization accuracy does not degrade significantly with the increase in number of nodes being deployed, making TPSN completely scalable.

2,215 citations


Proceedings ArticleDOI
25 Aug 2003
TL;DR: This work proposes several modifications to Gnutella's design that dynamically adapt the overlay topology and the search algorithms in order to accommodate the natural heterogeneity present in most peer-to-peer systems.
Abstract: Napster pioneered the idea of peer-to-peer file sharing, and supported it with a centralized file search facility. Subsequent P2P systems like Gnutella adopted decentralized search algorithms. However, Gnutella's notoriously poor scaling led some to propose distributed hash table solutions to the wide-area file search problem. Contrary to that trend, we advocate retaining Gnutella's simplicity while proposing new mechanisms that greatly improve its scalability. Building upon prior research [1, 12, 22], we propose several modifications to Gnutella's design that dynamically adapt the overlay topology and the search algorithms in order to accommodate the natural heterogeneity present in most peer-to-peer systems. We test our design through simulations and the results show three to five orders of magnitude improvement in total system capacity. We also report on a prototype implementation and its deployment on a testbed.

1,184 citations


Journal ArticleDOI
TL;DR: The design of theAstrolabe system is described, with a focus upon its scalability, and examples of the use of Astrolabe for locating resources, publish-subscribe, and distributed synchronization in largesystems are presented.
Abstract: Scalable management and self-organizational capabilities are emerging as central requirements for a generation of large-scale, highly dynamic, distributed applications. We have developed an entirely new distributed information management system called Astrolabe. Astrolabe collects large-scale system state, permitting rapid updates and providing on-the-fly attribute aggregation. This latter capability permits an application to locate a resource, and also offers a scalable way to track system state as it evolves over time. The combination of features makes it possible to solve a wide variety of management and self-configuration problems. This paper describes the design of the system with a focus upon its scalability. After describing the Astrolabe service, we present examples of the use of Astrolabe for locating resources, publish-subscribe, and distributed synchronization in large systems. Astrolabe is implemented using a peer-to-peer protocol, and uses a restricted form of mobile code based on the SQL query language for aggregation. This protocol gives rise to a novel consistency model. Astrolabe addresses several security considerations using a built-in PKI. The scalability of the system is evaluated using both simulation and experiments; these confirm that Astrolabe could scale to thousands and perhaps millions of nodes, with information propagation delays in the tens of seconds.

875 citations


Proceedings Article
31 Mar 2003
TL;DR: The mechanisms in Plutus to reduce the number of cryptographic keys exchanged between users by using filegroups, distinguish file read and write access, handle user revocation efficiently, and allow an untrusted server to authorize file writes are explained.
Abstract: Plutus is a cryptographic storage system that enables secure file sharing without placing much trust on the file servers. In particular, it makes novel use of cryptographic primitives to protect and share files. Plutus features highly scalable key management while allowing individual users to retain direct control over who gets access to their files. We explain the mechanisms in Plutus to reduce the number of cryptographic keys exchanged between users by using filegroups, distinguish file read and write access, handle user revocation efficiently, and allow an untrusted server to authorize file writes. We have built a prototype of Plutus on OpenAFS. Measurements of this prototype show that Plutus achieves strong security with overhead comparable to systems that encrypt all network traffic.

781 citations


Journal ArticleDOI
26 Oct 2003
TL;DR: It is argued that these problems can be addressed by moving to a declarative style of concurrency control in which programmers directly indicate the safety properties that they require, which is easier for mainstream programmers to use, prevents lock-based priority-inversion and deadlock problems and can offer performance advantages.
Abstract: Concurrent programming is notoriously difficult. Current abstractions are intricate and make it hard to design computer systems that are reliable and scalable. We argue that these problems can be addressed by moving to a declarative style of concurrency control in which programmers directly indicate the safety properties that they require. In our scheme the programmer demarks sections of code which execute within lightweight software-based transactions that commit atomically and exactly once. These transactions can update shared data, instantiate objects, invoke library features and so on. They can also block, waiting for arbitrary boolean conditions to become true. Transactions which do not access the same shared memory locations can commit concurrently. Furthermore, in general, no performance penalty is incurred for memory accesses outside transactions.We present a detailed design of this proposal along with an implementation and evaluation. We argue that the resulting system (i) is easier for mainstream programmers to use, (ii) prevents lock-based priority-inversion and deadlock problems and (iii) can offer performance advantages.

735 citations


Patent
09 Jan 2003
TL;DR: In this paper, a power management architecture for an electrical power distribution system, or portion thereof, is disclosed, which includes multiple intelligent electronic devices (IEDs) distributed throughout the power distribution systems to manage the flow and consumption of power from the system using real time communications.
Abstract: A power management architecture for an electrical power distribution system, or portion thereof, is disclosed. The architecture includes multiple intelligent electronic devices (“IED's”) distributed throughout the power distribution system to manage the flow and consumption of power from the system using real time communications. Power management application software and/or hardware components operate on the IED's and the back-end servers and inter-operate via the network to implement a power management application. The architecture provides a scalable and cost effective framework of hardware and software upon which such power management applications can operate to manage the distribution and consumption of electrical power by one or more utilities/suppliers and/or customers which provide and utilize the power distribution system. Autonomous communication on the network between IED's, back-end servers and other entities coupled with secure networks, themselves interconnected, via firewalls, by one or more unsecure networks, is facilitated by the use of a back-channel protocol. The back-channel protocol allows a device coupled with a secure network to solicit communications from a device on the unsecure network, thereby opening a back-channel through the firewall through which the unsecure network device may send unsolicited messages to the secure network device. Communications between multiple secure networks is accomplished using a unsecure device on an intermediary unsecure network to relay communications between the secure network devices using the protocol described above.

707 citations


Journal ArticleDOI
TL;DR: The main algorithmic features in the software package SuperLU_DIST, a distributed-memory sparse direct solver for large sets of linear equations, are presented, with an innovative static pivoting strategy proposed earlier by the authors.
Abstract: We present the main algorithmic features in the software package SuperLU_DIST, a distributed-memory sparse direct solver for large sets of linear equations. We give in detail our parallelization strategies, with a focus on scalability issues, and demonstrate the software's parallel performance and scalability on current machines. The solver is based on sparse Gaussian elimination, with an innovative static pivoting strategy proposed earlier by the authors. The main advantage of static pivoting over classical partial pivoting is that it permits a priori determination of data structures and communication patterns, which lets us exploit techniques used in parallel sparse Cholesky algorithms to better parallelize both LU decomposition and triangular solution on large-scale distributed machines.

670 citations


Proceedings ArticleDOI
24 Aug 2003
TL;DR: CLOSET+ integrates the advantages of the previously proposed effective strategies as well as some ones newly developed here, and develops a winning algorithm CLOSET+.
Abstract: Mining frequent closed itemsets provides complete and non-redundant results for frequent pattern analysis. Extensive studies have proposed various strategies for efficient frequent closed itemset mining, such as depth-first search vs. breadthfirst search, vertical formats vs. horizontal formats, tree-structure vs. other data structures, top-down vs. bottom-up traversal, pseudo projection vs. physical projection of conditional database, etc. It is the right time to ask "what are the pros and cons of the strategies?" and "what and how can we pick and integrate the best strategies to achieve higher performance in general cases?"In this study, we answer the above questions by a systematic study of the search strategies and develop a winning algorithm CLOSET+. CLOSET+ integrates the advantages of the previously proposed effective strategies as well as some ones newly developed here. A thorough performance study on synthetic and real data sets has shown the advantages of the strategies and the improvement of CLOSET+ over existing mining algorithms, including CLOSET, CHARM and OP, in terms of runtime, memory usage and scalability.

654 citations


Proceedings ArticleDOI
09 Jul 2003
TL;DR: This work proposes a content location solution in which peers loosely organize themselves into an interest- based structure on top of the existing Gnutella network, and demonstrates the existence of interest-based locality in five diverse traces of content distribution applications, two of which are traces of popular peer-to-peer file-sharing applications.
Abstract: Locating content in decentralized peer-to-peer systems is a challenging problem. Gnutella, a popular file-sharing application, relies on flooding queries to all peers. Although flooding is simple and robust, it is not scalable. We explore how to retain the simplicity of Gnutella, while addressing its inherent weakness: scalability. We propose a content location solution in which peers loosely organize themselves into an interest-based structure on top of the existing Gnutella network. Our approach exploits a simple, yet powerful principle called interest-based locality, which posits that if a peer has a particular piece of content that one is interested in, it is very likely that it will have other items that one is interested in as well. When using our algorithm, called interest-based shortcuts, a significant amount of flooding can be avoided, making Gnutella a more competitive solution. In addition, shortcuts are modular and can be used to improve the performance of other content location mechanisms including distributed hash table schemes. We demonstrate the existence of interest-based locality in five diverse traces of content distribution applications, two of which are traces of popular peer-to-peer file-sharing applications. Simulation results show that interest-based shortcuts often resolve queries quickly in one peer-to-peer hop, while reducing the total load in the system by a factor of 3 to 7.

632 citations


Proceedings Article
01 Jan 2003
TL;DR: The architectural challenges facing the design of large-scale distributed stream processing systems are described, and novel approaches for addressing load management, high availability, and federated operation issues are discussed.
Abstract: Stream processing fits a large class of new applications for which conventional DBMSs fall short. Because many stream-oriented systems are inherently geographically distributed and because distribution offers scalable load management and higher availability, future stream processing systems will operate in a distributed fashion. They will run across the Internet on computers typically owned by multiple cooperating administrative domains. This paper describes the architectural challenges facing the design of large-scale distributed stream processing systems, and discusses novel approaches for addressing load management, high availability, and federated operation issues. We describe two stream processing systems, Aurora* and Medusa, which are being designed to explore complementary solutions to these challenges. This paper discusses the architectural issues facing the design of large-scale distributed stream processing systems. We begin in Section 2 with a brief description of our centralized stream processing system, Aurora [4]. We then discuss two complementary efforts to extend Aurora to a distributed environment: Aurora* and Medusa. Aurora* assumes an environment in which all nodes fall under a single administrative domain. Medusa provides the infrastructure to support federated operation of nodes across administrative boundaries. After describing the architectures of these two systems in Section 3, we consider three design challenges common to both: infrastructures and protocols supporting communication amongst nodes (Section 4), load sharing in response to variable network conditions (Section 5), and high availability in the presence of failures (Section 6). We also discuss high-level policy specifications employed by the two systems in Section 7. For all of these issues, we believe that the push-based nature of stream-based applications not only raises new challenges but also offers the possibility of new domain-specific solutions.

624 citations


Journal ArticleDOI
TL;DR: GHT, a Geographic Hash Table system for DCS on sensornets, is described, and it is demonstrated that GHT is the preferable approach for the application workloads, analytically predict, offers high data availability, and scales to large sensornet deployments, even when nodes fail or are mobile.
Abstract: Making effective use of the vast amounts of data gathered by large-scale sensor networks (sensornets) will require scalable, self-organizing, and energy-efficient data dissemination algorithms. For sensornets, where the content of the data is more important than the identity of the node that gathers them, researchers have found it useful to move away from the Internet's point-to-point communication abstraction and instead adopt abstractions that are more data-centric. This approach entails naming the data and using communication abstractions that refer to those names rather than to node network addresses [1,11]. Previous work on data-centric routing has shown it to be an energy-efficient data dissemination method for sensornets [12]. Herein, we argue that a companion method, data-centric storage (DCS), is also a useful approach. Under DCS, sensed data are stored at a node determined by the name associated with the sensed data. In this paper, we first define DCS and predict analytically where it outperforms other data dissemination approaches. We then describe GHT, a Geographic Hash Table system for DCS on sensornets. GHT hashes keys into geographic coordinates, and stores a key-value pair at the sensor node geographically nearest the hash of its key. The system replicates stored data locally to ensure persistence when nodes fail. It uses an efficient consistency protocol to ensure that key-value pairs are stored at the appropriate nodes after topological changes. And it distributes load throughout the network using a geographic hierarchy. We evaluate the performance of GHT as a DCS system in simulation against two other dissemination approaches. Our results demonstrate that GHT is the preferable approach for the application workloads we analytically predict, offers high data availability, and scales to large sensornet deployments, even when nodes fail or are mobile.

Journal ArticleDOI
TL;DR: This paper presents lightweight probabilistic broadcast (lpbcast), a novel gossip-based broadcast algorithm, which complements the inherent throughput scalability of traditional probabilism broadcast algorithms with a scalable memory management technique.
Abstract: Gossip-based broadcast algorithms, a family of probabilistic broadcast algorithms, trade reliability guarantees against "scalability" properties. Scalability in this context has usually been expressed in terms of message throughput and delivery latency, but there has been little work on how to reduce the memory consumption for membership management and message buffering at large scale.This paper presents lightweight probabilistic broadcast (lpbcast), a novel gossip-based broadcast algorithm, which complements the inherent throughput scalability of traditional probabilistic broadcast algorithms with a scalable memory management technique. Our algorithm is completely decentralized and based only on local information: in particular, every process only knows a fixed subset of processes in the system and only buffers fixed "most suitable" subsets of messages. We analyze our broadcast algorithm stochastically and compare the analytical results both with simulations and concrete implementation measurements.

Book ChapterDOI
09 Sep 2003
TL;DR: This paper presents the initial design of PIER, a massively distributed query engine based on overlay networks, which is intended to bring database query processing facilities to new, widely distributed environments.
Abstract: The database research community prides itself on scalable technologies. Yet database systems traditionally do not excel on one important scalability dimension: the degree of distribution. This limitation has hampered the impact of database technologies on massively distributed systems like the Internet. In this paper, we present the initial design of PIER, a massively distributed query engine based on overlay networks, which is intended to bring database query processing facilities to new, widely distributed environments. We motivate the need for massively distributed queries, and argue for a relaxation of certain traditional database research goals in the pursuit of scalability and widespread adoption. We present simulation results showing PIER gracefully running relational queries across thousands of machines, and show results from the same software base in actual deployment on a large experimental cluster.

Journal ArticleDOI
01 Jan 2003
TL;DR: In this paper, the GPSR geographic routing algorithm and a new generation of efficient peer-to-peer lookup systems (such as Chord, CAN, Pastry, Tapestry, etc.).
Abstract: Sensornets are large-scale distributed sensing networks comprised of many small sensing devices equipped with memory, processors, and short-range wireless communication. Making effective use of sensornet data will require scalable, self-organizing, and energy-efficient data dissemination algorithms. Recent work has identified data-centric routing as one such method. In this paper we suggest that a companion method, data-centric storage, may also be a useful approach. While there are many ways to achieve data-centric storage, this paper proposes a mechanism that builds upon two recent advances; (1) the GPSR geographic routing algorithm and (2) a new generation of efficient peer-to-peer lookup systems (such as Chord, CAN, Pastry, Tapestry, etc.). We evaluate the performance of data-centric storage and two other dissemination approaches in several sensornet scenarios and identify the conditions under which the various approaches are preferable.

Journal ArticleDOI
TL;DR: An overview of the CDN architecture and popular CDN service providers can be found in this paper, where the authors offer an overview of some of the most popular service providers and their architecture.
Abstract: CDNs improve network performance and offer fast and reliable applications and services by distributing content to cache servers located close to users. The Web's growth has transformed communications and business services such that speed, accuracy, and availability of network-delivered content has become absolutely critical - both on their own terms and in terms of measuring Web performance. Proxy servers partially address the need for rapid content delivery by providing multiple clients with a shared cache location. In this context, if a requested object exists in a cache (and the cached version has not expired), clients get a cached copy, which typically reduces delivery time. CDNs act as trusted overlay networks that offer high-performance delivery of common Web objects, static data, and rich multimedia content by distributing content load among servers that are close to the clients. CDN benefits include reduced origin server load, reduced latency for end users, and increased throughput. CDNs can also improve Web scalability and disperse flash-crowd events. Here we offer an overview of the CDN architecture and popular CDN service providers.

Proceedings ArticleDOI
05 Mar 2003
TL;DR: A dataflow operator called flux is introduced that encapsulates adaptive state partitioning and dataflow routing that can be used for CQ operators under shifting processing and memory loads and can provide several factors improvement in throughput and orders of magnitude improvement in average latency over the static case.
Abstract: The long-running nature of continuous queries poses new scalability challenges for dataflow processing. CQ systems execute pipelined dataflows that may be shared across multiple queries. The scalability of these dataflows is limited by their constituent, stateful operators - e.g. windowed joins or grouping operators. To scale such operators, a natural solution is to partition them across a shared-nothing platform. But in the CQ context, traditional, static techniques for partitioned parallelism can exhibit detrimental imbalances as workload and runtime conditions evolve. Long-running CQ dataflows must continue to function robustly in the face of these imbalances. To address this challenge, we introduce a dataflow operator called flux that encapsulates adaptive state partitioning and dataflow routing. Flux is placed between producer-consumer stages in a dataflow pipeline to repartition stateful operators while the pipeline is still executing. We present the flux architecture, along with repartitioning policies that can be used for CQ operators under shifting processing and memory loads. We show that the flux mechanism and these policies can provide several factors improvement in throughput and orders of magnitude improvement in average latency over the static case.

01 Jan 2003
TL;DR: This work presents the software architecture of mpiBLAST, an open-source parallelization of BLAST that achieves superlinear speed-up by segmenting a BLAST database and then having each node in a computational cluster search a unique portion of the database.
Abstract: mpiBLAST is an open-source parallelization of BLAST that achieves superlinear speed-up by segmenting a BLAST database and then having each node in a computational cluster search a unique portion of the database. Database segmentation permits each node to search a smaller portion of the database, eliminating disk I/O and vastly improving BLAST performance. Because database segmentation does not create heavy communication demands, BLAST users can take advantage of low-cost and efficient Linux cluster architectures such as the bladed Beowulf. In addition to presenting the software architecture of mpiBLAST we present a detailed performance analysis of mpiBLAST to demonstrate its scalability.

Journal ArticleDOI
TL;DR: A joint encryption and compression framework in which video data are scrambled efficiently in the frequency domain by employing selective bit scrambling, block shuffling and block rotation of the transform coefficients and motion vectors is presented.
Abstract: Multimedia data security is very important for multimedia commerce on the Internet such as video-on-demand and real-time video multicast. Traditional cryptographic algorithms/systems for data security are often not fast enough to process the vast amount of data generated by multimedia applications to meet real-time constraints. This paper presents a joint encryption and compression framework in which video data are scrambled efficiently in the frequency domain by employing selective bit scrambling, block shuffling and block rotation of the transform coefficients and motion vectors. The new approach is very simple to implement, yet provides considerable levels of security and different levels of transparency, and has a very limited adverse impact on compression efficiency and no adverse impact on error resiliency. Furthermore, it allows transcodability/scalability, and other content processing functionalities without having to access the cryptographic key and perform decryption and re-encryption.

Proceedings ArticleDOI
11 May 2003
TL;DR: The proposed protocol, Distributed Predictive Tracking, is robust against node or prediction failures which may result in temporary loss of the target and recovers from such scenarios quickly and with very little additional energy use.
Abstract: With recent advances in device fabrication technology, economical deployment of large scale sensor networks, capable of pervasive monitoring and control of physical systems have become possible. Scalability, low overhead anti distributed functionality are some of the key requirements for any protocol designed for such large scale sensor networks. In this paper, we present a protocol, Distributed Predictive Tracking, for one of the most likely applications for sensor networks: tracking moving targets. The protocol uses a clustering based approach for scalability and a prediction based tracking mechanism to provide a distributed and energy efficient solution. The protocol is robust against node or prediction failures which may result in temporary loss of the target and recovers from such scenarios quickly and with very little additional energy use. Using simulations we show that the proposed architecture is able to accurately track targets with random movement patterns with accuracy over a wide range of target speeds.

Proceedings ArticleDOI
20 May 2003
TL;DR: These RDF-based P2P networks are able to support sophisticated routing and clustering strategies based on the metadata schemas, attributes and ontologies used, and the use of super-peer based topologies for these networks is described.
Abstract: RDF-based P2P networks have a number of advantages compared with simpler P2P networks such as Napster, Gnutella or with approaches based on distributed indices such as CAN and CHORD. RDF-based P2P networks allow complex and extendable descriptions of resources instead of fixed and limited ones, and they provide complex query facilities against these metadata instead of simple keyword-based searches.In previous papers, we have described the Edutella infrastructure and different kinds of Edutella peers implementing such an RDF-based P2P network. In this paper we will discuss these RDF-based P2P networks as a specific example of a new type of P2P networks, schema-based P2P networks, and describe the use of super-peer based topologies for these networks. Super-peer based networks can provide better scalability than broadcast based networks, and do provide perfect support for inhomogeneous schema-based networks, which support different metadata schemas and ontologies (crucial for the Semantic Web). Furthermore, as we will show in this paper, they are able to support sophisticated routing and clustering strategies based on the metadata schemas, attributes and ontologies used. Especially helpful in this context is the RDF functionality to uniquely identify schemas, attributes and ontologies. The resulting routing indices can be built using dynamic frequency counting algorithms and support local mediation and transformation rules, and we will sketch some first ideas for implementing these advanced functionalities as well.

Journal ArticleDOI
TL;DR: This work describes the Entropia distributed computing system as a case study, detailing its internal architecture and philosophy in attacking key problems of efficiency, robustness, scalability, manageability, unobtrusiveness, and openness/ ease of application integration.

Proceedings ArticleDOI
25 Aug 2003
TL;DR: This work introduces a new network simulation environment, developed by the research group, called the Georgia Tech Network Simulator (GTNetS), designed specifically to allow much larger-scale simulations than can easily be created by existing network simulation tools.
Abstract: We introduce a new network simulation environment, developed by our research group, called the Georgia Tech Network Simulator (GTNetS). Our simulator is designed specifically to allow much larger-scale simulations than can easily be created by existing network simulation tools. The design of the simulator very closely matches the design of real network protocol stacks and hardware. Thus, anyone with a good understanding of networking in general can easily understand how the simulations are constructed. Further, our simulator is implemented completely in object-oriented C++, which leads to easy extension by users to experiment with new or modified behavior of existing simulation models. Our tool is designed from the beginning with scalability in mind, including the support for distributed simulations on a network of workstations as part of the basic design.We give an overview of the features of GTNetS, and present some preliminary scalability results we have obtained by running GTNetS on a computing cluster at the Pittsburgh Supercomputer Center.

Journal ArticleDOI
19 Oct 2003
TL;DR: Capriccio is presented, a scalable thread package for use with high-concurrency servers and introduced linked stack management, which minimizes the amount of wasted stack space by providing safe, small, and non-contiguous stacks that can grow or shrink at run time.
Abstract: This paper presents Capriccio, a scalable thread package for use with high-concurrency servers. While recent work has advocated event-based systems, we believe that thread-based systems can provide a simpler programming model that achieves equivalent or superior performance.By implementing Capriccio as a user-level thread package, we have decoupled the thread package implementation from the underlying operating system. As a result, we can take advantage of cooperative threading, new asynchronous I/O mechanisms, and compiler support. Using this approach, we are able to provide three key features: (1) scalability to 100,000 threads, (2) efficient stack management, and (3) resource-aware scheduling.We introduce linked stack management, which minimizes the amount of wasted stack space by providing safe, small, and non-contiguous stacks that can grow or shrink at run time. A compiler analysis makes our stack implementation efficient and sound. We also present resource-aware scheduling, which allows thread scheduling and admission control to adapt to the system's current resource usage. This technique uses a blocking graph that is automatically derived from the application to describe the flow of control between blocking points in a cooperative thread package. We have applied our techniques to the Apache 2.0.44 web server, demonstrating that we can achieve high performance and scalability despite using a simple threaded programming model.

Proceedings ArticleDOI
01 Sep 2003
TL;DR: An efficient two-step genetic algorithm is described that has been used to build a tool for mapping an application, described by a parameterized task graph, on to a NoC architecture with a two dimensional mesh of switches as a communication backbone.
Abstract: Network on Chip (NoC) is a new paradigm for designing core based System on Chip which supports high degree of reusability and is scalable. In this paper we describe an efficient two-step genetic algorithm that has been used to build a tool for mapping an application, described by a parameterized task graph, on to a NoC architecture with a two dimensional mesh of switches as a communication backbone. The computational resources in NoC consist of a set of heterogeneous IP cores. Our algorithm finds a mapping of the vertices of the task graph to available cores so that the overall execution time of the task graph is minimized. We have developed a NoC architecture specific communication delay model to estimate the execution time. Our algorithm is able to handle large task graphs and provide near optimal mapping in a few minutes on a PC platform. Our tool also provides facilities for specifying NoC architecture, generation and viewing synthetic task graphs and viewing the progress of the genetic algorithm as it converges to a solution.

Book ChapterDOI
21 Feb 2003
TL;DR: Lighthouse as mentioned in this paper is a scalable location mechanism for wide-area networks, which can be used to avoid the communication bottlenecks and single-points-of-failure that otherwise limit the practicality of such systems.
Abstract: This paper introduces Lighthouse, a scalable location mechanism for wide-area networks. Unlike existing vector-based systems such as GNP, we show how network-location can be established without using a fixed set of reference points. This lets us avoid the communication bottlenecks and single-points-of-failure that otherwise limit the practicality of such systems.

Proceedings ArticleDOI
08 Sep 2003
TL;DR: SoCIN is presented, a scalable network based on a parametric router architecture to be used in the synthesis of customized low cost NoCs, and some synthesis results are presented.
Abstract: Networks-on-chip (NoCs) interconnection architectures, to be used in future billion-transistor systems-on-chip (SoCs), meet the major communication requirements of these systems, offering, at the same time, reusability, scalability and parallelism in communication. Furthermore, they cope with other issues like power constraints and clock distribution. Currently, there are a number of research works which explore different features of NoCs. In this paper, we present SoCIN, a scalable network based on a parametric router architecture to be used in the synthesis of customized low cost NoCs. The architecture of SoCIN and its router are described, and some synthesis results are presented.

Proceedings ArticleDOI
25 Aug 2003
TL;DR: This paper considers how optics can be used to scale capacity and reduce power in a router, and describes two different implementations based on technology available within the next three years.
Abstract: Routers built around a single-stage crossbar and a centralized scheduler do not scale, and (in practice) do not provide the throughput guarantees that network operators need to make efficient use of their expensive long-haul links. In this paper we consider how optics can be used to scale capacity and reduce power in a router. We start with the promising load-balanced switch architecture proposed by C-S. Chang. This approach eliminates the scheduler, is scalable, and guarantees 100% throughput for a broad class of traffic. But several problems need to be solved to make this architecture practical: (1) Packets can be mis-sequenced, (2) Pathological periodic traffic patterns can make throughput arbitrarily small, (3) The architecture requires a rapidly configuring switch fabric, and (4) It does not work when linecards are missing or have failed. In this paper we solve each problem in turn, and describe new architectures that include our solutions. We motivate our work by designing a 100Tb/s packet-switched router arranged as 640 linecards, each operating at 160Gb/s. We describe two different implementations based on technology available within the next three years.

Proceedings Article
18 May 2003
TL;DR: This work uses a simple resource usage model to measured behavior from the Gnutella file-sharing network to argue that large-scale cooperative storage is limited by likely dynamics and cross-system bandwidth -- not by local disk space.
Abstract: Peer-to-peer storage aims to build large-scale, reliable and available storage from many small-scale unreliable, low-availability distributed hosts. Data redundancy is the key to any data guarantees. However, preserving redundancy in the face of highly dynamic membership is costly. We use a simple resource usage model to measured behavior from the Gnutella file-sharing network to argue that large-scale cooperative storage is limited by likely dynamics and cross-system bandwidth -- not by local disk space. We examine some bandwidth optimization strategies like delayed response to failures, admission control, and load-shifting and find that they do not alter the basic problem. We conclude that when redundancy, data scale, and dynamics are all high, the needed cross-system bandwidth is unreasonable.

Journal ArticleDOI
TL;DR: A model of organizational networks according to which links are added incrementally to a hierarchical backbone is introduced and it is found that multiscale networks attain most of their robustness with surprisingly few link additions, suggesting that ultrarobust organizational networks can be generated in an efficient and scalable manner.
Abstract: The dynamics of information exchange is an important but understudied aspect of collective communication, coordination, and problem solving in a wide range of distributed systems, both physical (e.g., the Internet) and social (e.g., business firms). In this paper, we introduce a model of organizational networks according to which links are added incrementally to a hierarchical backbone and test the resulting networks under variable conditions of information exchange. Our main result is the identification of a class of multiscale networks that reduce, over a wide range of environments, the likelihood that individual nodes will suffer congestion-related failure and that the network as a whole will disintegrate when failures do occur. We call this dual robustness property of multiscale networks “ultrarobustness.” Furthermore, we find that multiscale networks attain most of their robustness with surprisingly few link additions, suggesting that ultrarobust organizational networks can be generated in an efficient and scalable manner. Our results are directly relevant to the relief of congestion in communication networks and also more broadly to activities, like distributed problem solving, that require individuals to exchange information in an unpredictable manner.

Proceedings ArticleDOI
19 Sep 2003
TL;DR: A hybrid authenticated key establishment scheme, which exploits the difference in capabilities between security managers and sensors, and put the cryptographic burden where the resources are less constrained, which is better than all the other public-key based key establishment protocols studied.
Abstract: In this paper, we consider efficient authenticated key establishment protocols between a sensor and a security manager in a self-organizing sensor network. We propose a hybrid authenticated key establishment scheme, which exploits the difference in capabilities between security managers and sensors, and put the cryptographic burden where the resources are less constrained. The hybrid scheme reduces the high cost public-key operations at the sensor side and replaces them with efficient symmetric-key based operations. Meanwhile, the scheme authenticates the two identities based on public-key certificates to avoid the typical key management problem in pure symmetric-key based protocols and maintain a good amount of scalability. The proposed scheme can be efficiently implemented on Mitsubishi's M16C microprocessor in 5.2Kbyte code/data size, and achieve a total processing time of 760 ms on sensor side, which is better than all the other public-key based key establishment protocols we have studied. We also present its modified version with a faster speed but more communication overhead.