Experimental evaluation of BBR congestion control
01 Oct 2017-pp 1-10
TL;DR: An independent and extensive experimental evaluation of BBR at higher speeds and considers throughput, queuing delay, packet loss, and fairness, as well as some severe inherent issues such as increased queuing delays, unfairness, and massive packet loss.
Abstract: BBR is a recently proposed congestion control. Instead of using packet loss as congestion signal, like many currently used congestion controls, it uses an estimate of the available bottleneck link bandwidth to determine its sending rate. BBR tries to provide high link utilization while avoiding to create queues in bottleneck buffers. The original publication of BBR shows that it can deliver superior performance compared to Cubic TCP in some environments. This paper provides an independent and extensive experimental evaluation of BBR at higher speeds. The experimental setup uses BBR's Linux kernel 4.9 implementation and typical data rates of 10Gbit/s and 1 Gbit/s at the bottleneck link. The experiments vary the flows' round-trip times, the number of flows, and buffer sizes at the bottleneck. The evaluation considers throughput, queuing delay, packet loss, and fairness. On the one hand, the intended behavior of BBR could be observed with our experiments. On the other hand, some severe inherent issues such as increased queuing delays, unfairness, and massive packet loss were also detected. The paper provides an in-depth discussion of BBR's behavior in different experiment setups.
Citations
More filters
14 May 2018
TL;DR: A publicly available framework for reproducible TCP measurements based on network emulation is presented and an analysis of BBR’s inter-flow synchronization behavior is contributed, showing that it reaches fairness equilibrium for long lived flows.
Abstract: In 2016, Google published the bottleneck bandwidth and round-trip time (BBR) congestion control algorithm. Unlike established loss- or delay-based algorithms like CUBIC or Vegas, BBR claims to operate without creating packet loss or filling buffers. Because of these prospects and promising initial performance results, BBR has gained wide-spread attention. As such it has been subject to behavior and performance analysis, which confirmed the results, but also revealed critical flaws.Because BBR is still work in progress, measurement results have limited validity for the future. In this paper we present our publicly available framework for reproducible TCP measurements based on network emulation. In a case study, we analyze the TCP BBR algorithm, reproduce and confirm weaknesses of the current BBR implementation, and provide further insights. We also contribute an analysis of BBR’s inter-flow synchronization behavior, showing that it reaches fairness equilibrium for long lived flows.
89 citations
Cites background or methods from "Experimental evaluation of BBR cong..."
...[4, 5, 6])....
[...]
...jitter) occur [4], but works well for the data rates we use....
[...]
...evaluated BBR in an experimental setup with 10 Gbit/s links and software-based switches [4]....
[...]
...The size of the bottleneck buffer is crucial for the fairness between competing BBR and CUBIC flows [1, 4]....
[...]
...[1] and we refer to their work for a detailed description of the congestion control algorithm or [4] for a formal analysis....
[...]
TL;DR: This paper surveys the main novelties related to transport protocols that have been recently proposed, identifying three main research trends: the evolution of congestion control algorithms, to target optimal performance in challenging scenarios, and the introduction of multipath capabilities at the transport layer.
Abstract: Over the years, the Internet has been enriched with new available communication technologies, for both fixed and mobile networks and devices, exhibiting an impressive growth in terms of performance, with steadily increasing available data rates. The Internet research community has kept trying to evolve the transport layer protocols to match the capabilities of modern networks, in order to fully reap the benefits of the new communication technologies. This paper surveys the main novelties related to transport protocols that have been recently proposed, identifying three main research trends: (i) the evolution of congestion control algorithms, to target optimal performance in challenging scenarios, possibly with the application of machine learning techniques; (ii) the proposal of brand new transport protocols, alternative to the Transmission Control Protocol (TCP) and implemented in the user-space; and (iii) the introduction of multipath capabilities at the transport layer.
88 citations
Cites background from "Experimental evaluation of BBR cong..."
...as previously mentioned, the mechanism becomes too aggressive when buffers are very small [144] and too conservative when the network suffers from bufferbloat....
[...]
...BBR also works well with high-capacity connections; however, the presence of short buffers causes the BBR operating point to increase too much, thus leading to massive packet losses and unfairness towards loss-based flows [144]....
[...]
TL;DR: The delay signal and the algorithms that completely or partially utilize this type of signal are described and modern active queue management (AQM) principles are illustrated and the interaction between AQM and the delay signal is discussed.
Abstract: Congestion control (CC) has a significant influence on the performance of transmission control protocol (TCP) connections. Over the last three decades, many researchers have extensively studied and proposed a multitude of enhancements to standard TCP CC. However, this topic still inspires both academic and industrial research communities due to the change in Internet application requirements and the evolution of Internet technologies. The standard TCP CC infers network congestion based on packet loss events which leads to long queuing delay when bottleneck buffer size is large. A promising solution to this problem is to use the delay signal (RTT or one-way delay measurements) to infer congestion earlier and react to the congestion before the queuing delay reaches a high value. In this survey paper, we describe the delay signal and the algorithms that completely or partially utilize this type of signal. Additionally, we illustrate standard CC and modern active queue management (AQM) principles and discus the interaction between AQM and the delay signal.
57 citations
Cites background from "Experimental evaluation of BBR cong..."
...[123] conduct experimental evaluation of TCP BBR in an emulated environment and conclude that BBR is able to achieve its goals....
[...]
21 Oct 2019
TL;DR: The first model capturing BBR's behavior in competition with loss-based CCAs is provided and it is shown that under competition, BBR becomes window-limited by its 'in-flight cap' which then determines B BR's bandwidth consumption.
Abstract: BBR is a new congestion control algorithm (CCA) deployed for Chromium QUIC and the Linux kernel. As the default CCA for YouTube (which commands 11+% of Internet traffic), BBR has rapidly become a major player in Internet congestion control. BBR's fairness or friendliness to other connections has recently come under scrutiny as measurements from multiple research groups have shown undesirable outcomes when BBR competes with traditional CCAs. One such outcome is a fixed, 40% proportion of link capacity consumed by a single BBR flow when competing with as many as 16 loss-based algorithms like Cubic or Reno. In this short paper, we provide the first model capturing BBR's behavior in competition with loss-based CCAs. Our model is coupled with practical experiments to validate its implications. The key lesson is this: under competition, BBR becomes window-limited by its 'in-flight cap' which then determines BBR's bandwidth consumption. By modeling the value of BBR's in-flight cap under varying network conditions, we can predict BBR's throughput when competing against Cubic flows with a median error of 5%, and against Reno with a median of 8%.
51 citations
Cites background from "Experimental evaluation of BBR cong..."
...The first independent study of BBR was presented by Hock et al. [11]....
[...]
...Early presentations [8] imply that it primarily resolves the fairness issues discussed by Hock et al [11], but does not touch on the fixed proportion of link capacity as discussed in this paper....
[...]
...This phenomena was first explored in [11] and BBRv2 is expected to patch the problem [7]....
[...]
01 Nov 2018
TL;DR: This paper develops the insight that one should “Keep the pipe just full, but no fuller” and shows this is equivalent to loading the system so that in many cases the optimized average number in the pipe is exactly equal to the Bandwidth-Delay Product.
Abstract: Recently there has been considerable interest in a key paper [1] describing a new approach to congestion control in Internet traffic which has resulted in significant network performance improvement. The approach is based on a 1978 paper [2] and a companion 1979 paper [3] which identified a system operating point that was optimal in that it maximized delivered throughput while minimizing delay and loss. This operating point is simply characterized by the insight that one should “Keep the pipe just full, but no fuller” and we show this is equivalent to loading the system so that in many cases (including those relevant to TCP connections) the optimized average number in the pipe is exactly equal to the Bandwidth-Delay Product. It is important to understand the reasoning and intuition behind this early insight and why it provides such improved behavior of systems and networks. In this paper, we first develop this insight using purely deterministic reasoning. We then extend this reasoning by examining far more complex stochastic queueing systems and networks using a function called Power to mathematically and graphically extract exact and surprising results that support the insight and allow us to identify the optimum operating point for a broad class of systems. These observations allow us to study the impact of Power on networks leading eventually to supporting the statements about steady state congestion and flow control as presented in [1] for today’s Internet. We point out that the discussions about the latest congestion control algorithms [ 1 , 4, 5, 6, 7, 8, 9, 10, 11] address the dynamics of tracking flow, dealing with multiple intersecting flows, fairness, and more, and which focus on the dynamic behavior of data networks whereas our work here focuses only on the steady state behavior.
39 citations
Cites background from "Experimental evaluation of BBR cong..."
...It is this latter, more difficult problem, that [1, 5, 6, 7, 8, 9, 10, 11] and its variations seek to solve....
[...]
...In October, 2017, Hock, et al, [6], looked deeper into the issue of multi-...
[...]
...We point out that the discussions about the latest congestion control algorithms [1, 4, 5, 6, 7, 8, 9, 10, 11] address the dynamics of...
[...]
References
More filters
01 Apr 1999
TL;DR: This document defines TCP's four intertwined congestion control algorithms: slow start, congestion avoidance, fast retransmit, and fast recovery, as well as discussing various acknowledgment generation methods.
Abstract: This document defines TCP's four intertwined congestion control algorithms: slow start, congestion avoidance, fast retransmit, and fast recovery. In addition, the document specifies how TCP should begin transmission after a relatively long idle period, as well as discussing various acknowledgment generation methods.
2,237 citations
TL;DR: The CUBIC protocol modifies the linear window growth function of existing TCP standards to be a cubic function in order to improve the scalability of TCP over fast and long distance networks.
Abstract: CUBIC is a congestion control protocol for TCP (transmission control protocol) and the current default TCP algorithm in Linux. The protocol modifies the linear window growth function of existing TCP standards to be a cubic function in order to improve the scalability of TCP over fast and long distance networks. It also achieves more equitable bandwidth allocations among flows with different RTTs (round trip times) by making the window growth to be independent of RTT -- thus those flows grow their congestion window at the same rate. During steady state, CUBIC increases the window size aggressively when the window is far from the saturation point, and the slowly when it is close to the saturation point. This feature allows CUBIC to be very scalable when the bandwidth and delay product of the network is large, and at the same time, be highly stable and also fair to standard TCP flows. The implementation of CUBIC in Linux has gone through several upgrades. This paper documents its design, implementation, performance and evolution as the default TCP algorithm of Linux.
2,088 citations
"Experimental evaluation of BBR cong..." refers background in this paper
...Loss-based congestion controls (such as CUBIC TCP [12] or TCP Reno [2]) use packet loss as congestion signal....
[...]
01 Oct 1994
TL;DR: This paper motivates and describes the three key techniques employed by Vegas, and presents the results of a comprehensive experimental performance study—using both simulations and measurements on the Internet— of the Vegas and Reno implementations of TCP.
Abstract: Vegas is a new implementation of TCP that achieves between 40 and 70% better throughput, with one-fifth to one-half the losses, as compared to the implementation of TCP in the Reno distribution of BSD Unix. This paper motivates and describes the three key techniques employed by Vegas, and presents the results of a comprehensive experimental performance study—using both simulations and measurements on the Internet—of the Vegas and Reno implementations of TCP.
1,399 citations
TL;DR: FAST TCP is described, a new TCP congestion control algorithm for high-speed long-latency networks, from design to implementation, and its equilibrium and stability properties are characterized.
Abstract: We describe FAST TCP, a new TCP congestion control algorithm for high-speed long-latency networks, from design to implementation. We highlight the approach taken by FAST TCP to address the four difficulties which the current TCP implementation has at large windows. We describe the architecture and summarize some of the algorithms implemented in our prototype. We characterize its equilibrium and stability properties. We evaluate it experimentally in terms of throughput, fairness, stability, and responsiveness
1,214 citations
"Experimental evaluation of BBR cong..." refers methods in this paper
...FAST TCP [16] is also based on TCP Vegas and aims toward high speed wide-area networks but does not have low queuing delay as design goal....
[...]
TL;DR: When bottleneck buffers are large, loss-based congestion control keeps them full, causing bufferbloat, leading to low throughput, which requires an alternative to loss- based congestion control.
Abstract: When bottleneck buffers are large, loss-based congestion control keeps them full, causing bufferbloat When bottleneck buffers are small, loss-based congestion control misinterprets loss as a signa
850 citations
"Experimental evaluation of BBR cong..." refers background or methods in this paper
...The following analysis is mainly based on the concepts described in [5], [7], some details were taken from the Linux implementation....
[...]
..., built-in traffic policing detection) are provided in [5], [7], [9]....
[...]
...BBR was originally described in [5], [7] and updates were presented by presentations [6], [8]....
[...]
...Evaluation results shown in [5], [7] consist of scenarios with 10 Mbit/s and 100 Mbit/s bottleneck link bandwidth, RTTs are mostly 40 ms....
[...]