scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Real-time estimation of the parameters of long-range dependence

TL;DR: An on-line version of the Abry-Veitch wavelet-based estimator of the Hurst parameter has very low memory and computational requirements and scales naturally to arbitrarily high data rates, enabling its use in real-time applications such as admission control, and avoiding the need to store huge data sets for off-line analysis.
Abstract: An on-line version of the Abry-Veitch (see IEEE GLOBECOM'98, Sydney, Australia,p.3716-21, 1998) wavelet-based estimator of the Hurst parameter is presented. It has very low memory and computational requirements and scales naturally to arbitrarily high data rates, enabling its use in real-time applications such as admission control, and avoiding the need to store huge data sets for off-line analysis. The performance of the estimator as a function of the length of data processed is demonstrated using simulated data. An implementation for 10-Mb/s Ethernet based on standard hardware supporting sampling rates of 1 data point per millisecond is described, and results of its operation presented, as is an implementation for 155-Mb/s asynchronous transfer mode networks. Finally we illustrate the power of on-line measurements by collecting measurements over a period of five months, and using them to look for diurnal trends in scaling properties of the data.

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI
07 Aug 2002
TL;DR: This paper discusses some of the pitfalls associated with applying traditional performance evaluation techniques to highly-interacting, large-scale networks such as the Internet, and presents one promising approach based on chaotic maps to capture and model the dynamics of TCP-type feedback control in such networks.
Abstract: One of the most significant findings of traffic measurement studies over the last decade has been the observed self-similarity in packet network traffic. Subsequent research has focused on the origins of this self-similarity, and the network engineering significance of this phenomenon. This paper reviews what is currently known about network traffic self-similarity and its significance. We then consider a matter of current research, namely, the manner in which network dynamics (specifically, the dynamics of transmission control protocol (TCP), the predominant transport protocol used in today's Internet) can affect the observed self-similarity. To this end, we first discuss some of the pitfalls associated with applying traditional performance evaluation techniques to highly-interacting, large-scale networks such as the Internet. We then present one promising approach based on chaotic maps to capture and model the dynamics of TCP-type feedback control in such networks. Not only can appropriately chosen chaotic map models capture a range of realistic source characteristics, but by coupling these to network state equations, one can study the effects of network dynamics on the observed scaling behavior We consider several aspects of TCP feedback, and illustrate by examples that while TCP-type feedback can modify the self-similar scaling behavior of network traffic, it neither generates it nor eliminates it.

147 citations

Proceedings ArticleDOI
20 Oct 2003
TL;DR: This paper utilizes energy distribution based on wavelet analysis to detect DDoS attack traffic and shows that energy distribution variance changes markedly causing a "spike" when traffic behaviors affected by DDoS attacked, making it an effective attack detection.
Abstract: This paper presents a systematic method for DDoS attack detection. DDoS attack can be considered system anomaly or misuse from which abnormal behavior is imposed on network traffic. Attack detection can be performed via abnormal behavior identification. Network traffic characterization with behavior modeling could be a good guidance of attack detection. Aggregated traffic has been found to be strong bursty across a wide range of time scales. Wavelet analysis is able to capture complex temporal correlation across multiple time scales with very low computational complexity. We utilize energy distribution based on wavelet analysis to detect DDoS attack traffic. Energy distribution over time would have limited variation if the traffic keeps its behavior over time (i.e. attack-free situation); while an introduction of attack traffic in the network would elicit significant energy distribution deviation in short time period. Our experimental results with typical Internet traffic trace show that energy distribution variance changes markedly causing a "spike" when traffic behaviors affected by DDoS attack In contrast, normal traffic exhibits a remarkably stationary energy distribution. In addition, this spike in energy distribution variance can be captured in early stage of attack, for ahead of congestion build-up, making it an effective attack detection.

90 citations

Journal ArticleDOI
TL;DR: In this article, the local Whittle wavelet estimator of the memory parameter d 0 was studied and it was shown that the estimator is consistent and rate optimal if X is a linear process, and is asymptotically normal if X are Gaussian.
Abstract: We consider a time series X = {X k , k ∈ Z} with memory parameter do ∈ R. This time series is either stationary or can be made stationary after differencing a finite number of times. We study the "local Whittle wavelet estimator" of the memory parameter d 0 . This is a wavelet-based semiparametric pseudo-likelihood maximum method estimator. The estimator may depend on a given finite range of scales or on a range which becomes infinite with the sample size. We show that the estimator is consistent and rate optimal if X is a linear process, and is asymptotically normal if X is Gaussian.

79 citations


Cites background from "Real-time estimation of the paramet..."

  • ...Another interesting application involves considering online estimators of d0: online computation of wavelet coefficients is easier when the number of scales is fixed; see [19]....

    [...]

Journal ArticleDOI
TL;DR: The experimental results with typical Internet traffic trace show that energy distribution variance markedly changes, causing a “spike” when traffic behaviors are affected by DDoS attack, making it an effective detection of the attack.
Abstract: This paper presents a systematic method for DDoS attack detection. DDoS attack can be considered a system anomaly or misuse from which abnormal behavior is imposed on network traffic. Attack detection can be performed via abnormal behavior identification. Network traffic characterization with behavior modeling could be a good indication of attack detection. Aggregated traffic has been found to be strong bursty across a wide range of time scales. Wavelet analysis is able to capture complex temporal correlation across multiple time scales with very low computational complexity. We utilize energy distribution based on wavelet analysis to detect DDoS attack traffic. Energy distribution over time will have limited variation if the traffic keeps its behavior over time (i.e. attack-free situation) while an introduction of attack traffic in the network will elicit significant energy distribution deviation in a short time period. Our experimental results with typical Internet traffic trace show that energy distribution variance markedly changes, causing a "spike" when traffic behaviors are affected by DDoS attack. In contrast, normal traffic exhibits a remarkably stationary energy distribution. In addition, this spike in energy distribution variance can be captured in the early stages of an attack, far ahead of congestion build-up, making it an effective detection of the attack.

74 citations


Cites background from "Real-time estimation of the paramet..."

  • ...Roughan et al. [ 21 ] suggested the upper bound of scale (j � max ) should be less than log2 n, the largest scale in sampling data, where n is the length of sampling data....

    [...]

BookDOI
01 Jan 2001

58 citations


Cites methods from "Real-time estimation of the paramet..."

  • ...This idea is investigated in Schonlau et al. [1999]. Another paper that looks at this problem is Lane and Brodley [1999]....

    [...]

  • ...3 The results (in percent) for the six algorithms reported in Schonlau et al. [1999]. The algorithms aimed at a false alarm rate of 1 %....

    [...]

  • ...Schonlau et al. [1999] report on a set of six methods for detecting masqueraders....

    [...]

References
More filters
Book
01 Jan 1998
TL;DR: An introduction to a Transient World and an Approximation Tour of Wavelet Packet and Local Cosine Bases.
Abstract: Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour. Estimations are Approximations. Transform Coding. Appendix A: Mathematical Complements. Appendix B: Software Toolboxes.

17,693 citations

Book
01 May 1992
TL;DR: This paper presents a meta-analyses of the wavelet transforms of Coxeter’s inequality and its applications to multiresolutional analysis and orthonormal bases.
Abstract: Introduction Preliminaries and notation The what, why, and how of wavelets The continuous wavelet transform Discrete wavelet transforms: Frames Time-frequency density and orthonormal bases Orthonormal bases of wavelets and multiresolutional analysis Orthonormal bases of compactly supported wavelets More about the regularity of compactly supported wavelets Symmetry for compactly supported wavelet bases Characterization of functional spaces by means of wavelets Generalizations and tricks for orthonormal wavelet bases References Indexes.

16,073 citations


"Real-time estimation of the paramet..." refers background in this paper

  • ...Wavelet transforms in general can be understood as a more flexible form of a Fourier transform, where is transformed, not into a fre­quency domain, but into a time-scale wavelet domain....

    [...]

Journal ArticleDOI
TL;DR: In this article, the regularity of compactly supported wavelets and symmetry of wavelet bases are discussed. But the authors focus on the orthonormal bases of wavelets, rather than the continuous wavelet transform.
Abstract: Introduction Preliminaries and notation The what, why, and how of wavelets The continuous wavelet transform Discrete wavelet transforms: Frames Time-frequency density and orthonormal bases Orthonormal bases of wavelets and multiresolutional analysis Orthonormal bases of compactly supported wavelets More about the regularity of compactly supported wavelets Symmetry for compactly supported wavelet bases Characterization of functional spaces by means of wavelets Generalizations and tricks for orthonormal wavelet bases References Indexes.

14,157 citations

Journal ArticleDOI
TL;DR: It is demonstrated that Ethernet LAN traffic is statistically self-similar, that none of the commonly used traffic models is able to capture this fractal-like behavior, and that such behavior has serious implications for the design, control, and analysis of high-speed, cell-based networks.
Abstract: Demonstrates that Ethernet LAN traffic is statistically self-similar, that none of the commonly used traffic models is able to capture this fractal-like behavior, that such behavior has serious implications for the design, control, and analysis of high-speed, cell-based networks, and that aggregating streams of such traffic typically intensifies the self-similarity ("burstiness") instead of smoothing it. These conclusions are supported by a rigorous statistical analysis of hundreds of millions of high quality Ethernet traffic measurements collected between 1989 and 1992, coupled with a discussion of the underlying mathematical and statistical properties of self-similarity and their relationship with actual network behavior. The authors also present traffic models based on self-similar stochastic processes that provide simple, accurate, and realistic descriptions of traffic scenarios expected during B-ISDN deployment. >

5,567 citations


"Real-time estimation of the paramet..." refers methods in this paper

  • ...Finally we illustrate the power of on-line measurements by collecting measurements over a period of five months, and using them to look for diurnal trends in scaling properties of the data....

    [...]

Journal ArticleDOI
TL;DR: It is found that user-initiated TCP session arrivals, such as remote-login and file-transfer, are well-modeled as Poisson processes with fixed hourly rates, but that other connection arrivals deviate considerably from Poisson.
Abstract: Network arrivals are often modeled as Poisson processes for analytic simplicity, even though a number of traffic studies have shown that packet interarrivals are not exponentially distributed. We evaluate 24 wide area traces, investigating a number of wide area TCP arrival processes (session and connection arrivals, FTP data connection arrivals within FTP sessions, and TELNET packet arrivals) to determine the error introduced by modeling them using Poisson processes. We find that user-initiated TCP session arrivals, such as remote-login and file-transfer, are well-modeled as Poisson processes with fixed hourly rates, but that other connection arrivals deviate considerably from Poisson; that modeling TELNET packet interarrivals as exponential grievously underestimates the burstiness of TELNET traffic, but using the empirical Tcplib interarrivals preserves burstiness over many time scales; and that FTP data connection arrivals within FTP sessions come bunched into "connection bursts", the largest of which are so large that they completely dominate FTP data traffic. Finally, we offer some results regarding how our findings relate to the possible self-similarity of wide area traffic. >

3,915 citations