Data for Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness

Introduction

This is the dataset for the paper titled ‘Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness’ that has been accepted for publication at IEEE INFOCOM 2024. The paper’s accepted version will be available following publication in May 2024 at https://sussex.figshare.com/articles/conference_contribution/Reinforcement_learningbased_congestion_control_a_systematic_evaluation_of_fairness_efficiency_and_responsiveness/24711033. The dataset is meant to be used in conjunction with the codebase that is also made available at https://doi.org/10.25377/sussex.24978162.

However, the dataset itself is of value to researchers as it contains an extensive set of metrics captured during experimentation with Reinforcement Learning-based Congestion control as discussed in the ‘Experimental Evaluation’ section of the paper. Our study is the result of a 160-hour long experimentation during which 1950 Orca, Aurora and TCP Cubic flows were measured. We have collected approximately 500GB of data encompassing diverse metrics related to network interfaces (e.g., utilisation, retransmissions, packet drops), CPU and memory parameters (such as CPU load and memory usage), as well as the data transport layer (e.g., congestion window, round trip time). Reinforcement learning (RL)-based congestion control (CC) promises efficient CC in a fast-changing networking landscape, where evolving communication technologies, applications and traffic workloads pose severe challenges to human-derived, static CC algorithms. RL-based CC is in its early days and substantial research is required to understand existing limitations, identify research challenges and, eventually, yield deployable solutions for real-world networks. In this paper we present the first reproducible and systematic study of RL-based CC with the aim to highlight strengths and uncover fundamental limitations of the state-of-the-art. We identify challenges in evaluating RL-based CC, establish a methodology for studying said approaches and perform large-scale experimentation with RL-based CC approaches that are publicly available. We show that existing approaches can acquire all available bandwidth swiftly and are resistant to non-congestive loss, however, this is commonly at the cost of excessive packet loss in normal operation. We show that, as fairness is not embedded directly into reward functions, existing approaches exhibit unfairness in almost all tested network setups. Finally, we provide evidence that existing RL-based CC approaches under-perform when the available bandwidth and end-to-end latency dynamically change. Our experimentation codebase and datasets are publicly available with the aim to galvanise the community towards transparency and reproducibility, which have been recognised as crucial for researching and evaluating machine-generated policies.

The dataset

The dataset contains an extensive set of metrics captured during experimentation. It is composed of eight different experiments. Please refer to the paper for a detailed explanation of the experimental set-up. Note that some experiments are the source of more than one plot in the paper. Refer to the codebase for the relationship between plots and experiments.

The data from each experiment is organised into multiple folders and files. Just under the root folder, the data is divided by the type of packet scheduling adopted by the bottleneck queue, i.e. fifo, codel, fq, fq-codel.

The next folder contains all variations of bottleneck bandwidth, propagation delay, and buffer size for the same experiment. Each of the folders is named following the same pattern:{topology}_{bottleneck_bandwidth}mbps_{one_way_delay}ms_{buffer_size}pkts_{loss_rate}loss_{number_of_flows}flows_22tcpbuf_{protocol}

For example: Dumbell_100mbit_80ms_279pkts_0loss_2flows_22tcpbuf_orca contains all the runs of two flows in a dumbbell topology with 100mbps bottlenck bandwidth, 160ms rtt and a buffer size of 279 MSS. Note that the semantic meaning of {protocol} depends on the specific experimental setup. In the intra-protocol fairness experiment, all flows will be {protocol}, whereas in the friendliness experiment, one flow will always be cubic and one {protocol}.

Under the experiment variation folder, multiple folders contain different runs of the same variation. Depending on the experiment, a different seed may be used. However, in most of the cases, all runs are identical.

Finally, each run folder contains all the raw data captured during the experiment, plus some processed data.

The files and folders contained in each run folder are:

• tcp_probe.txt: The raw output of tcp_probe module
• x[N]_output.txt: The raw output of the Nth server application
• c[N]_output.txt: The raw output of the Nth client application.
• emulation_info.json: JSON representation of all flows in the experiment. Each flow is represented by the source and destination IP addresses, the protocol used, and the starting time.
• queues (folder): Contains a text file for each of the bottleneck queue. Each file contains the output of ‘tc -s qdisc show dev’ applied to the dev in the filename.
• sysstat (folder): Contains the raw binary files (datafile_*.log) generated with sysstat. It also contains some log files generated using sadf processing utility on the binary datafiles. ]
• csvs (folder): contains processed data in csv format.