Showing papers on "Performance metric published in 2003"

Performance management for cluster based Web services

Abstract: We present an architecture and prototype implementation of a performance management system for cluster-based Web services. The system supports multiple classes of Web services traffic and allocates server resources dynamically so to maximize the expected value of a given cluster utility function in the face of fluctuating loads. The cluster utility is a function of the performance delivered to the various classes, and this leads to Differentiated Service. In this paper we use the average response time as the performance metric. The management system is transparent: it requires no changes in the client code, the server code, or the network interface between them. The system performs three performance management tasks: resource allocation, load balancing, and server overload protection. We use two nested levels of management mechanism. The inner level centers on queuing and scheduling of request messages. The outer level is a feedback control loop that periodically adjusts the scheduling weights and server allocations of the inner level. The feedback controller is based on an approximate first-principles model of the system, with parameters derived from continuous monitoring. We focus on SOAP-based Web services. We report experimental results that show the dynamic behavior of the system.

Classifying scheduling policies with respect to unfairness in an M/GI/1

Abstract: It is common to evaluate scheduling policies based on their mean response times. Another important, but sometimes opposing, performance metric is a scheduling policy's fairness. For example, a policy that biases towards small job sizes so as to minimize mean response time may end up being unfair to large job sizes. In this paper we define three types of unfairness and demonstrate large classes of scheduling policies that fall into each type. We end with a discussion on which jobs are the ones being treated unfairly.

Method and system for storing and reporting network performance metrics using histograms

Abstract: A method for reporting data network monitoring information. The method includes accessing performance metric values for a network component and generating a trace of graph data points for the performance metric values. For a range of the trace, a histogram is built and displayed corresponding to the graph data points (step 430). For a user interface, a performance monitoring display is generated including a graph of the trace relative to an x-axis and a y-axis and a representation of the histogram. Using the graphical user interface (GUI), the user can access a selection mechanism by a moving the range selector to define the selected histogram range (steps 440 and 470). The graph data points in the trace corresponds to a histogram previously built from the performance metric values, and the trace is generated by determining and plotting an average value of each of the graph data point histograms. The building of the histogram for the performance monitoring display involves combining the graph data point histograms corresponding to the graph data points in selected histogram range (step 460).

Software system and methods for analyzing the performance of a server

Abstract: A system for monitoring and evaluating the performance of a server comprises a component that monitors a plurality of performance metrics of the server over a period of time while the server responds to requests from clients. An analysis component analyzes the collected performance metric data values to evaluate degrees to which specific performance metrics are related. Causal relationships and dependencies that affect the performance of the server are thereby identified.

Heterogeneous processor core systems for improved throughput

Abstract: A computer system for maximizing system and individual job throughput includes a number of computer hardware processor cores that differ amongst themselves in at least in their respective resource requirements and processing capabilities. A monitor gathers performance metric information from each of the computer hardware processor cores that are specific to a particular run of application software then executing. Based on these metrics, a workload assignment mechanism assigns jobs to processor cores in order to maximize overall system throughput and the throughput of individual jobs.

The flexible time-triggered (FTT) paradigm: an approach to QoS management in distributed real-time systems

Abstract: Real-time distributed systems are becoming pervasive, supporting a broad range of applications such as avionics, automotive, adaptive control, robotics, computer vision and multimedia. In such systems, several activities are executed on different nodes and cooperate via message passing. One increasingly important concept is that of quality-of-service (QoS), i.e. a system performance metric from the application point-of-view. Concerning the communication system, the QoS delivered to the application is a function of communication parameters such as the rates of message streams. In previous work, the authors have developed two network protocols, FTT-CAN and FTT-Ethernet, which allow online changes to the set of message streams under guaranteed timeliness. The specific network protocols are abstracted away leading to a generic communication paradigm named Flexible Time-Triggered (FTT), which supports online QoS management, with arbitrary policies, in distributed real-time systems. Two possible QoS management policies are referred, priority-based and another one based on the elastic task model, and their use is illustrated with a case study.

Optimal partitioning in distributed model predictive control

Abstract: In this paper we develop the algorithms for optimal partitioning of a distributed control system into subsystems of manageable size for which control actions are found using model predictive control (MPC) technology. We will first define a realization-invariant weighting matrix to represent the distributed system as a directed graph. We then develop a formulation in which an open loop performance metric is used to partition the distributed system into subsystem in which local MPC problems will be solved. This partitioning however is balanced against the closed loop cost of the control actions for the overall distributed system. Effective algorithms for the distributed control of the large-scale systems are then proposed. Future work will include the study of the effect of the constraints in the partitioning, and the development of efficient problem formulations aimed at improving numerical properties of the proposed control algorithms.

Network performance monitoring at small time scales

Abstract: SNMP statistics are usually collected over intervals of 5 minutes and correspond to average activity of IP links and network elements for the duration of the interval. Nevertheless, reports of traffic performance across periods of minutes can mask out performance degradation due to short-lived events, such as micro-congestion episodes, that manifest themselves at smaller time scales. In this paper we perform a measurement study of packet traces collected inside the Sprint IP network to identify the time scales over which micro-congestion episodes occur. We characterize these episodes with respect to their amplitude, frequency and duration. We define a new performance metric that could be easily computed by a router and reported every 5 minutes through SNMP to shed light into the micro-behavior of the carried traffic. We show that the proposed performance metric is well suited to track the time scales over which micro-congestion episodes occur, and may be useful for a variety of network provisioning tasks.

Redeeming IPC as a performance metric for multithreaded programs

Abstract: Recent work has shown that multithreaded workloads running in execution-driven, full-system simulation environments cannot use instructions per cycle (IPC) as a valid performance metric due to non-deterministic program behavior. Unfortunately, invalidating IPC as a performance metric introduces its own host of difficulties: special workload setup, consideration of cold-startand end-effects, statistical methodologies leading to increased simulation bandwidth, and workload-specific, higher-level metrics to measure performance. This paper explores the non-determinism problem in multithreaded programs, describes a method to eliminate non-determinism across simulations of different experimental machine models, and demonstrates the suitability of this methodology for performing architectural performance analysis, thus redeeming IPC as a performance metric for multithreaded programs.

Methods and systems for control discovery in computing systems

Abstract: Techniques are provided for generically controlling one or more resources associated with at least one computing system. In one aspect of the invention, the technique comprises evaluating one or more performance metrics associated with the one or more resources given one or more configurations of the one or more resources. The technique then causes a change in the one or more configurations of the one or more resources based on the performance metric evaluating step. The one or more performance metrics and the one or more configurations are expressed in generic formats.

Evaluating and allocating system resources to improve resource utilization

Abstract: Methods and systems for evaluating the utilization of resources in a system. Information that characterizes an allocation of resources in the system is received. Information that characterizes requests to the system is also received. Workflow in the system in response to the requests is simulated. Processes that are executed by the system in response to the requests are also simulated. Utilization of the allocation of resources is determined. The resources can be reallocated to improve the utilization as measured against a selected performance metric.

Dynamically balancing load for servers

Abstract: Dynamically balancing load for servers. Servers that are able to respond to requests directed at the system are monitored for. After discovery of a server, a performance metric is obtained for the discovered server. The performance metric for the discovered server is stored in a table. When a request is received, the request is routed to a selected server in the system of servers based on the performance metric, wherein the system of servers comprises the discovered servers.

Classifying scheduling policies with respect to unfairness in an M/GI/1

Abstract: It is common to evaluate scheduling policies based on their mean response times. Another important, but sometimes opposing, performance metric is a scheduling policy's fairness. For example, a poli...

Method and apparatus to select a channel using performance metrics

Abstract: A method and apparatus to select a channel of operation for a wireless network device may examine channels supported by the device to gather information regarding present use of the channels. A best channel may be determined for the wireless network device by identifying a channel that maximizes a predetermined performance metric.

Investigating performance insights of segment-based proxy caching of streaming media strategies

Abstract: In general, existing segment-based caching strategies target one of the following two performance objectives: (1) reducing client startup delay by giving a high priority to cache the beginning segments of media objects, or (2) reducing server traffic by caching popular segments of media objects. Our previous study has shown that the approach targeting the second objective has several advantages over the first one. However, we have also observed that the effort of improving server traffic reduction can increase client startup delay, which may potentially offset the overall performance gain. Little work so far has considered these two objectives in concert. In this paper, we first build an analytical model for these two types of typical segment-based caching approaches. The analysis on the model reveals the nature of the trade-off between two performance objectives and the bounds of each are given under certain circumstances. To provide a feasible way to evaluate different strategies, we propose a new comprehensive performance metric based on the analysis. To understand this performance trade-off, we restructure the adaptive-lazy segmentation strategy with a heuristic replacement policy to improve overall performance. The evaluation results confirm our analysis and show the effectiveness of our proposed new performance metric.

Method and apparatus for modifying design constraints based on observed performance

Abstract: A method for modifying design constraints based on observed performance includes measuring a characteristic of a plurality of devices manufactured in a process flow. A design constraint associated with the characteristic is defined. A performance metric relating the performance of the devices as a function of the measured characteristic and the design constraint is generated. The design constraint is modified based on the performance metric. A manufacturing system includes a metrology tool and a design rule monitor. The metrology tool is configured to measure a characteristic of a plurality of devices manufactured in a process flow. The design rule monitor is configured to receive a design constraint associated with the characteristic, generate a performance metric relating the performance of the devices as a function of the measured characteristic and the design constraint, and modify the design constraint based on the performance metric.

Effect of power amplifier nonlinearity on system performance metric, bit-error-rate (BER)

Abstract: In today's high-speed wireless communication applications, the simultaneous demand for bandwidth-efficient modulation schemes such as M-QAM and high power-efficiency of electronic components in the system imposes a contradicting tradeoff. On one hand, with the RF power amplifier (PA) at the transmitter antenna being the major power "hungry" block in many such systems, high power-efficiency for the system is obtained when the PA consumes most of the little supplied dc power (e.g. from a battery) as RF power, which is the power delivered to the load. On the other hand, at high powers, the PA becomes nonlinear, which becomes problematic when the input to the PA has a time-varying envelope. More precisely, when an M-QAM modulated signal (which has varying envelope and phase) is transmitted, the PA nonlinearity characteristics adversely affect the system performance, such as degradation of the Bit Error Rate (BER). Although linearization and compensation techniques can be used to minimize the above mentioned degradation in system performance, such techniques are mostly not suitable for low-cost and small-size integrated-circuit implementations. As a result, to obtain an optimal solution to this linearity/efficiency tradeoff, the objective of this thesis is to qualitatively and quantitatively characterize the effect of the PA nonlinearity on the system performance metric, BER. To gain an intuitive understanding of the effect of the PA nonlinearity, a system simulation is developed. In addition, to achieve a simple relationship between this circuit nonlinearity and system performance, an appropriate model (AM-AM/AM-PM) to characterize the PA nonlinearity is found. In fact, the results of this thesis in characterizing the effect of the PA nonlinearity on system performance can be useful to PA designers to design cost-effective and efficient PAs by finding a PA operating point that lies in the nonlinear region as much as possible while still meeting the system performance requirements. Thesis Supervisor: Charles G. Sodini Title: Professor of Electrical Engineering and Computer Science

Performance of iCAR systems: a simplified analysis technique

Abstract: In this paper, a simplified analysis technique for the integrated cellular and Ad hoc relay (iCAR) systems is presented. First, a simple two-cell system is analyzed using a multi-dimensional Markov-chain. The performance metric employed is the call blocking probability of each cell in the system. To this end, first a closed-form expression for the call blocking probability in the two-cell system is provided. Then, it is shown that these closed-form expressions could be used to analyze more practical systems. The accuracy of the developed simple analytical expressions is checked and verified by comparing the results predicted by these analytical expressions with simulation results. It is shown that there is an excellent match between analytical and simulation results.

Analytical performance of closed loop power control quantized under fast fading for CDMA techniques

Abstract: The performance of SIR (signal to interference ratio)-based closed loop power control (CLPC) is analytically analysed. The evaluation work has been carried out using the standard deviation of the power control error (PCE) as the performance metric. A non-linear control theory method is applied to the feedback system under fast fading. An analytical expression of the CLPC under fast fading is also produced. Finally a quantized-step size power control algorithm, replacing the hard limiter is considered. The proposed method is found to work considerably better for high speed MS as well as being a powerful tool to optimise the loop performance.

An Algorithm Designer’s Workbench for Platform FPGAs

Abstract: Growing gate density, availability of embedded multipliers and memory, and integration of traditional processors are some of the key advantages of Platform FPGAs. Such FPGAs are attractive for implementing compute intensive signal processing kernels used in wired as well as wireless mobile devices. However, algorithm design using Platform FPGAs, with energy dissipation as an additional performance metric for mobile devices, poses significant challenges. In this paper, we propose an algorithm designer’s workbench that addresses the above issues. The workbench supports formal modeling of the signal processing kernels, evaluation of latency, energy, and area of a design, and performance tradeoff analysis to facilitate optimization. The workbench includes a high-level estimator for rapid performance estimation and widely used low-level simulators for detailed simulation. Features include a confidence interval based technique for accurate power estimation and facility to store algorithm designs as library of models for reuse. We demonstrate the use of the workbench through design of matrix multiplication algorithm for Xilinx Virtex-II Pro.

Fuzzy based paths ordering algorithm in networks with imperfect QoS information

Abstract: In this paper, we study the problem of path selection under additive QoS (quality of service) constraints, where the information available for making routing decisions is inaccurate. The goal of the path selection process is to identify a feasible path while minimizing the overall setup time required for establishing a successful connection. Under the assumption that a list of candidate paths already exists, we explore the fuzzy logic approach to minimize the overall setup time by finding an optimal preference order of the given paths. A simulation program was used to evaluate the performance of the proposed algorithm. In the simulation, we compare between the fuzzy logic approach and another approach suggested in the literature to solve a similar problem, where the main performance metric is the average setup time for successfully establishing a feasible connection. The results show that the fuzzy approach has a lower average setup time for establishing a connection than the existing approach.

Performance optimization for a mobile small-unit-operation situational awareness (SUO-SAS) radio network

Abstract: This paper concerns performance optimization using the Taguchi parameter optimization methodology[M.S. Phaadke, May 12 1989] in a network based on SUO-SAS (situational awareness system for small unit operations)-a new state-of-the-art communication system developed by ITT Industries. For the subject experiment, the network consisted of 20 nodes. The nodes are mobile and move in accordance with an operationally relevant scenario based upon a topology provided by DARPA. The scenario specifies the input traffic variations and other noise factors such as message size, message types and mobility. The throughput is the most important performance metric and is selected as the output characteristic. The criterion for throughput optimization amounts to generating as large a throughput as possible. Four crucial parameters associated with the intranet layer and link layer, and an L/sub 9/ orthogonal array are selected for optimization through OPNET modeling and simulation. Parameter optimization yields a 34% improvement in message completion rate and a 35% improvement in system throughput.

Conflict resolution strategies and their performance models for large-scale multiagent systems

Abstract: Distributed, collaborative agents are promising to play an important role in large-scale multiagent applications, such as distributed sensors and distributed spacecraft. Since no single agent can have complete global knowledge in such large scale applications, conflicts are inevitable even among collaborative agents over shared resources, plans, or tasks. Fast conflict resolution techniques are required in many multiagent systems under soft or hard time constraints. In resolving conflicts, we focus on the approaches based on DCSP (distributed constraint satisfaction problems), a major paradigm in multiagent conflict resolution. We aim to speed up conflict resolution convergence via developing efficient DCSP strategies. We focus on multiagent systems characterized by agents that are collaborative, homogeneous, arranged in regular networks, and relying on local communication (found in many multiagent applications). This thesis provides the followings major contributions. First, we develop novel DCSP strategies that significantly speed up conflict resolution convergence. The novel strategies are based on the extra communication of local information between neighboring agents. We formalize a set of DCSP strategies which exploit the extra communication: in selecting a new choice of actions, plans, or resources to resolve conflicts, each agent takes into account how much flexibility is given to neighboring agents. Second, we provide a new run-time model for performance measurement of DCSP strategies since a popular existing DCSP performance metric does not consider the extra communication overhead. The run-time model enables us to evaluate the strategy performance in various computing and networking environments. Third, the analysis of message processing and communication overhead of the novel strategies shows that such overhead caused by the novel strategy is not overwhelming. Thus, despite extra communication, the novel strategies indeed show big speedups in a significant range of problems (particularly for harder problems). Fourth, we provide categorization of problem settings with big speedups by the novel strategies Finally, to select the right strategy in a given domain, we develop performance modeling techniques based on distributed POMDP (Partially Observable Markov Decision Process) based model where scalability issue is addressed with a new decomposition technique.

Web-enabled metrics and automation

Abstract: A method and system for monitoring enterprise system performance in which a datastore receives and stores performance metric data from different enterprise systems and provides the performance metric data to a web-based browser interface for presentation to a user as one or more linked web pages. In a preferred embodiment, the linked web pages are accessible on an enterprise intranet.

Performance analysis of leaky-bucket controlled Pareto-distributed on/off sources

Abstract: An approximate performance analysis of a leaky bucket with Pareto-distributed on/off sources is presented. The aggregate of Pareto-distributed on/off sources is a self-similar traffic. This analysis is applicable to the leaky bucket policing device at the ingresses of (1) a base station in a wireless ATM network, (2) a wireline ATM network, and (3) future generations of IP-based networks with quality of service (QoS) differentiation and with call admission control. The performance metric is packet loss. Numerical results show good agreement between the approximate analysis and simulation results.

NOT(Faster Implementation ==> Better Algorithm), A Case Study

Abstract: : Given two algorithms that perform the same task, one may ask which is better. One simple answer is that the algorithm that delivers the "best" answer is the better algorithm. But what if both algorithms deliver results of similar quality? In this case, a common metric that is utilized to differentiate between the two algorithms is the time to find a solution. Measurements, however, must be performed using an implementation of an algorithm (not an abstract algorithm) and must be taken using specific test data. Because the effects of implementation quality and test data selection may be large, the measured time metric is an insufficient measure of algorithm performance and quality. In this paper we present the specific case of several different implementations of the same Dijkstra graph search algorithm applied to graphs with various branching factors. Our experimental results show that quality rankings based on time may be heavily influenced by the choice of operational scenario and code quality. In addition, we explore possible alternative ranking schemes for the specific case of Dijkstra graph search algorithms.

A new availability concept for (n,k)-way cluster systems regarding waiting time

Abstract: It is necessary to have the precise definition of available performance of high availability systems that can represent the availability and performability of the systems altogether. However, the difference between numeric scales of availability and performance metrics such as waiting time makes quantitative evaluation difficult. A number of previous studies on availability do not include a performance metric in their availability modeling. In this paper, we propose a new availability model for (n,k)-way cluster systems which compose of n primary servers and k backup servers and compute the redundancy levels for the systems to satisfy both the requirements of availability and waiting time performance.

Building scalable and adaptive network services

Abstract: The lack of infrastructural functionality in IP-based networks has led to the creation of many overlay network algorithms where application-level nodes self-organize, forming an overlay atop the underlying IP substrate. The process of overlay design, development, and evaluation is plagued by a number of challenges. Current overlay algorithms are designed toward specific application requirements and tend to either be one-dimensional, optimizing for a single performance metric while sacrificing others, or not scalable to large numbers of nodes. Development of overlays is tedious, complex, and redundant since these algorithms make use of similar system and network functionality. Disparate evaluation techniques expose implementation artifacts rather than differences in algorithmic principles, thus leading to unfair and inconsistent evaluation. This dissertation presents a collection of overlay services geared at simplifying the creation of scalable and adaptive overlay networks. First, RanSub is a scalable mechanism for providing overlay nodes with random subsets of global participants. Second, TreeMaint is a protocol for scalable and adaptive overlay tree maintenance with guaranteed loop-free properties. Finally, MACEDON provides a language for the high-level representation of overlay algorithms from which high-performance implementations are generated. It leverages functional similarities of overlays to decrease development and evaluation complexity and to promote consistent evaluation. In addition, this dissertation describes how these tools can create high-performance versions of prominent overlays and design novel algorithms that overcome current one-dimensional approaches by optimizing for multiple performance metrics.

Performance analysis of a generic proxy-based client-server system for World-Wide Web services using a generalized Markov chain model

Abstract: We investigate the performance of a network system on the World-Wide Web employing proxies that interact with a pool of servers to service a pool of client requests. We focus on modelling the interaction of a proxy with the server system and its performance in maximizing the number of requests that can be served. We formulate the problem using a discrete time Markov chain model capturing the state of the client (equivalently the state of the proxy) and quantify the performance in terms of number of requests that are successfully processed by the server system. We derive a closed-form solution to determine the percentage of requests that can be admitted by the servers, referred to as admission control factor, which is our performance metric of interest. We conduct a systematic and rigorous simulation study to demonstrate the behavior of our performance metric with respect to the number of client requests and the number of available servers in the system. We also study the effect of server response time and propose three different strategies that a proxy may adopt to tune the performance of the system. The behavior of the three strategies - Conservative, Greedy, and Incremental Tuning, with respect to our performance metric is rigorously studied under a variety of system dependent parameters, with respect to the probability of a client request, and the probability of a new request arrival. Further, we perform an asymptotic analysis to quantify the ultimate performance limits of the admission control factor with respect to the number of client requests than can be supported by the system and the number of servers in the system. In the study with respect to the number of servers, we derive an exact bound on the behavior of our performance metric and testify this finding with our rigorous simulation experiments. Several interesting features are highlighted and the model is conclusively shown to be robust and elegant in capturing the behavior of the system.