Statistical approaches to DDoS attack detection and response
Summary (5 min read)
1. Introduction
- Powerful DDoS toolkits are available to potential attackers, and essential networks are ill prepared for defense.
- False positives can lead to inappropriate responses that cause denial of service to legitimate users.
- The PHAD component clusters observed values and then compares the size of the clusters to accepted thresholds to determine anomalies.
2. Detection Algorithms
- The authors detection algorithms measure statistical properties of specific fields in the packet headers at various points in the Internet.
- If a detector captures 1000 consecutive packets at a peering point and computes the frequency of occurrence of each unique source IP address in those 1000 packets, then the detector will have a model of the distribution of the source address.
- Further computations with this distribution allow us to measure the randomness or uniformity of the addresses as well as the “goodness-of-fit” of the distribution with respect to prior measurements.
2.1. Entropy
- Let an information source have n independent symbols each with probability of choice pi.
- The authors have observed through experimentation that while a network is not under attack, the entropy values for various header fields each fall in a narrow range.
- Isolate the term in the summation corresponding to the probability of the symbol acquired from shifting the window.
- Using the values computed in step 6, add the two terms missing from the entropy summation back in and compare this new entropy value to the previous entropy computations.
- Increasing W will reduce the variation in entropy and may reduce the rate of falsepositives resulting from brief and presumably insignificant anomalies.
2.2. Chi-Square Statistic
- Pearson’s chi-square (χ2) Test is used for distribution comparison in cases where the measurements involved are discrete values.
- Hence, comparison with the chi-square distribution is of limited utility.
- Apply exponential decay to the stored frequency for v based on its age (time since last update).
- Group the attribute values into bins based on frequency.
- Each time the current-traffic bin frequencies are computed, the average is updated as follows: 1. Exponential decay is applied to the stored binfrequency averages, using a significantly longer half-life than is used for the current-traffic profile.
3. Detector Evaluation
- In order to evaluate thoroughly the potential effectiveness of DDoS detection methods such as those described in Section 2, the authors must address the following questions.
- Ideally, a detector should pick up not only attacks generated by tools found “in the wild” to date, but also more stealthy attacks using more sophisticated tools wielded by attackers familiar with the detection method and detector’s network environment.
- Characteristics of the monitored network traffic will vary significantly depending on where detectors are deployed.
- The remainder of this section describes attempts answer these questions for the entropy and chi-square DDoS detection methods.
3.1. Prototype Implementation
- To evaluate the DDoS attack detection methods described in Section 2 under realistic conditions, the authors implemented prototype detector modules as plug-ins for Snort, the popular, open-source network intrusion detection system [13], [14].
- In addition to real-time traffic monitoring, Snort supports off-line processing of previously captured network traffic, making it possible to conduct reproducible detection experiments with traffic data from a variety of environments.
- Conf configuration file, and can trigger alarms through Snort’s modular alerting facility.
- The chi-square detector logs the periodically computed chi-square statistics for each of the specified packet attributes, along with the current and baseline bin frequency values used to compute those statistics.
- This data can be useful for manual or automatic detector tuning and alert threshold setting.
3.2. Network Trace Data
- This allows us to determine how stable the traffic statistics monitored by the detectors are in those environments, and how effectively the detectors can identify DDoS attack traffic in different contexts.
- To test the effects of DDoS attacks, the authors simulate these attacks by overlaying the kind of attack traffic generated by some existing DDoS attack tools onto the traces at various concentrations [10].
- The traces used were drawn from a variety of network environments, as described below, and most have IP addresses that have been transformed via an unknown but one-to-one function for privacy purposes.
- This trace, from July 2000, includes five consecutive days of IP headers sent through the New Zealand Internet Exchange (NZIX), a peering point for several major New Zealand ISPs and the University of Waikato; throughput ranges roughly from 4 to 12 Mbits/s. One 24-hour weekday trace was used for experimentation.
3.3. Detection Example
- To illustrate the effects of an attack on the entropy and chi-square statistics, the authors examined a 1,000,000- packet excerpt from the NZIX data set with a simulated DDoS attack comprising 25% of all packets, starting at packet number 700,000 and ending at packet number 800,000.
- Before the attack begins, source address entropy measurements fall entirely within the range 7.0-7.5.
- Any maximum-entropy threshold setting between 7.5 and 8.75 would detect this attack without generating any false-positives in this example.
- In Figure 2, the bin frequency profile for a source address chi-square detector (.
4096, and the remainder) is displayed for the same example. The six colored regions represent the percent-
- When the attack begins at packet 700,000, the total frequency of bin 6 (representing packets whose source addresses are least frequently seen) grows noticeably, as the authors would expect since the source addresses in the attack traffic are drawn from a uniform, rather than power-law, distribution.
- The chi-square values for this trace are shown in Figure 3, using a baseline profile taken from the previous day’s traffic.
- Any chi-square threshold between 1500 and 5000 would catch the attack without generating false positives.
- An attack in which the source addresses were fixed or drawn from a small set would produce similarly dramatic results for both entropy and chi-square detectors.
3.4. Distribution of Statistics
- The authors now look more closely at the distribution of chisquare and entropy measurements for legitimate traffic traces and for the same traces with different kinds of simulated DDoS attack traffic overlaid.
- The simulated attack traffic thus has the same source-address frequency distribution as the legitimate traffic, but uses a different set of source addresses.
- This way, an attacker armed with this knowledge of the detector environment could produce attack traffic that would produce little change in the entropy observed at the detector.
- Distribution of chi-square values for source address under normal and stealthyDDoS attack conditions Tables 1 and 2 show the results of running similar experiments on the different traffic traces described in Section 3.2, with a variety of attack and detector combinations, also known as Figure 7.
- The authors contend that the prospects for detecting stealthy attacks are not as bleak as they might appear, for several reasons.
3.5. Detector Performance
- Since the authors are proposing to use these detection methods in high-speed core routers, it is imperative that they have low computational cost, especially for the operations that must be carried out for each packet.
- The prototype Snort detector implementation exhibits adequate performance for its purposes: on a 1GHz Pentium-III-based machine, a Snort process running a single chi-square detector observing source addresses can process 240,000-270,000 packets per second (pps) offline.
- A single-attribute entropy detector can manage about 294,000 pps, while adding six others yields 130,000 pps.
- These speeds are roughly in the OC3 range.
- The authors expect to achieve improved performance by implementing some optimizations that approximate the true frequency profile while reducing or eliminating floating-point operations in the packet-handling code.
4. Response
- The authors defense approach involves response modules that use a characterization of the attack provided by the detection module to take defensive measures.
- The response module classifies individual packets as benign or suspect based on the attack characteristics provided by the detector.
- Once identified, the suspect packets are subjected to rate limiting or packet-filtering methods based on the intensity of the attack or pre-defined response policies.
- In the case of stealthy DDoS attacks, the response module should communicate with the detector and share the data structures and statistical models maintained by the detector to identify the attack packets with high confidence; the prototype described below does not yet offer such coordination.
4.1. Prototype DDoS Response Module
- It uses netfilter and Linux Advanced Routing and Traffic Control to filter and rate-limit packets [15],[4].
- Currently, the response module implements three packet-filtering rules.
- The random filter rule can be applied to the IP header fields, TCP source and destination ports, UDP source and destination ports, and ICMP type and code fields.
- Since this simple approach allows all packets with a given value to pass after the threshold is reached for that value, an attacker could choose a distribution of attack packets that limits the filter’s effectiveness.
- The clear option is used to remove filter rules.
4.2. Extending detectors to recommend response
- Both chi-square and entropy DDoS detectors can be extended to provide attack characterization information that can be used to target packet-filtering or ratelimiting responses to mitigate the effects of DDoS attacks.
- In order to determine the most anomalous bin, the detector need only find the largest terms in the chi-square sum.
- Conversely, an unusually high entropy value suggests that the low-frequency values are causing trouble, so the detector might suggest that packets having highfrequency values be given preferential treatment.
- Second, the authors modified the Snort-based chi-square and entropy detectors to issue rate-limiting directives to the iptables-based response module described in Section 4.1.
- This approach would allow response decisions to take full advantage of the information already collected by the detector.
4.3. DDoS Response Module Evaluation
- The current response prototype is an initial implementation of the response system.
- Initial experimental results have indicated that the response prototype blocks substantial DDoS attack traffic generated by the Stacheldraht attack tool.
- The random rule has the basic drawback of dropping the first few packets of every new good connection.
- These two rules could potentially increase the false negatives.
- The allow rule could allow through some of the DDoS attack traffic that matches the rule, increasing the false positives.
5. Summary and Future Extensions
- The focus thus far has been on detection and response algorithms and the implementation of these algorithms in software.
- Against today’s relatively unsophisticated DDoS toolkits, their prototype detector is able to determine that the network is under attack and deploy accurate filtering rules.
- The filtering effort is immediate and reduces the impact of the attack downstream almost instantly.
- Another approach to providing more narrowly targeted response while avoiding computationally expensive analysis would be to enable detectors to dynamically tune themselves and “drill down” to investigate detected anomalies more closely.
- The Linux implementation of this system has been appropriate for demonstration environments and evaluation of alternative detection approaches.
7. References
- [1] D. Dittrich, “The ‘Stacheldraht’ Distributed Denial of Service Attack Tool”, http://staff.washington.edu/dittrich/ misc/stacheldraht.analysis, 1999. [2].
- D. Knuth, The Art of Computer Programming: Seminumerical Algorithms, Third edition, Vol. 2, AddisonWesley, Reading, Massachusetts, 1997. [6].
- O. Pomerantz, “Linux Kernel Module Programming Guide”, http://www.tldp.org/LDP/lkmpg/mpg.html. [12].
- M. Roesch, “Snort - Lightweight Intrusion Detection for Networks” Proceedings of the 13th Systems Administration Conference (LISA'99), USENIX Association, 1999, pp. 229- 238, http://www.snort.org/docs/lisapaper.txt.
- Proceedings of the DARPA Information Survivability Conference and Exposition (DISCEX’03).
Did you find this useful? Give us your feedback
Citations
1,228 citations
Cites background from "Statistical approaches to DDoS atta..."
...for example for problems in intrusion detection by [26], and to detect DOS attacks [9]....
[...]
433 citations
Cites background from "Statistical approaches to DDoS atta..."
...[35], [36], but mainly in the context of the wired Internet....
[...]
421 citations
351 citations
328 citations
Cites methods from "Statistical approaches to DDoS atta..."
...Categories and Subject Descriptors C.2.3 [Computer-Communication-Networks]: Network Operations network management, network monitoring General Terms Management, Measurement Keywords Entropy, Anomaly Detection 1....
[...]
References
65,425 citations
10,215 citations
5,023 citations
"Statistical approaches to DDoS atta..." refers background in this paper
...Like many network characteristics [2], source address frequency for this trace follows roughly a power-law distribution, so the bins of exponentially increasing size have roughly equal frequencies....
[...]
3,793 citations
3,490 citations
"Statistical approaches to DDoS atta..." refers methods in this paper
...To evaluate the DDoS attack detection methods described in Section 2 under realistic conditions, we implemented prototype detector modules as plug-ins for Snort, the popular, open-source network intrusion detection system [13], [14]....
[...]
Related Papers (5)
Frequently Asked Questions (2)
Q2. What are the future works in "Statistical approaches to ddos attack detection and response" ?
Future research and development will focus on tighter integration of detection and response modules. In the current implementation, detectors generate concise recommended rules for responders to impose, and there is no further detector/responder coordination. By implementing detection and response methods on this platform and testing their performance, the authors can validate the claim that they are appropriate for use in future high-speed routers.