Showing papers by "Thomas Clausen published in 2022"

Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

Abstract: This paper presents the network load balancing problem, a challenging real-world task for multi-agent reinforcement learning (MARL) methods. Conventional heuristic solutions like Weighted-Cost Multi-Path (WCMP) and Local Shortest Queue (LSQ) are less flexible to the changing workload distributions and arrival rates, with a poor balance among multiple load balancers. The cooperative network load balancing task is formulated as a Dec-POMDP problem, which naturally induces the MARL methods. To bridge the reality gap for applying learning-based methods, all models are directly trained and evaluated on a real-world system from moderate- to large-scale setups. Experimental evaluations show that the independent and "selfish'' load balancing strategies are not necessarily the globally optimal ones, while the proposed MARL solution has a superior performance over different realistic settings. Additionally, the potential difficulties of the application and deployment of MARL methods for network load balancing are analysed, which helps draw the attention of the learning and network communities to such challenges.

Vedvarende bruk av opioider og samtidig bruk av andre vanedannende legemidler

Abstract: BACKGROUND Long-term use of opioids may have undesirable consequences. We have investigated long-term opioid use in patient groups that were prescribed opioids for various indications (chronic pain, palliative care, other (white prescriptions, not generally covered by the Norwegian National Insurance Scheme)) as well as the groups' concomitant use of some other addictive medications. MATERIAL AND METHOD Persons registered in the Norwegian Prescription Database with at least one filled prescription of an opioid in the period 2011-19 were included. Long-term use in a calendar year was defined as the dispensing of > 180 defined daily doses or > 4 500 mg oral morphine equivalents distributed over at least 3 periods of 3 months. RESULTS The number of long-term opioid users was 50 422 in 2011 and 59 996 in 2019 (10.1 and 10.7 % of all opioid users). The number who received opioids on blue prescription (partly covered by the Norwegian National Insurance Scheme) for chronic pain increased in the period by 9 952 persons, but the majority (n=38 006, 63.3 %) continued to receive opioids exclusively on white prescription in 2019. A total of 15 623 (41.1 %) and 14 881 (39.2 %), respectively, of the long-term opioid users who received opioids solely on white prescription in 2019 also received benzodiazepines and Z-hypnotics in the same year. Of the 23 967 long-term users who also received benzodiazepines, 88 % were dispensed opioids and benzodiazepines on the same day at least once in 2019. INTERPRETATION Prolonged prescribing of opioids on white prescription and concurrent prescribing of other addictive drugs may indicate undesirable use with no clear indication.

Aquarius—Enable Fast, Scalable, Data-Driven Service Management in the Cloud

Abstract: In order to dynamically manage and update networking policies in cloud data centers, Virtual Network Functions (VNFs) use, and therefore actively collect, networking state information - and in the process, incur additional control signaling and management overhead, especially in larger data centers. In the meantime, VNFs in production prefer distributed and straightforward heuristics over advanced learning algorithms to avoid intractable additional processing latency under high-performance and low-latency networking constraints. This paper identifies the challenges of deploying learning algorithms in the context of cloud data centers, and proposes Aquarius to bridge the application of machine learning (ML) techniques on distributed systems and service management. Aquarius passively yet efficiently gathers reliable observations, and enables the use of ML techniques to collect, infer, and supply accurate networking state information—without incurring additional signaling and management overhead. It offers fine-grained and programmable visibility to distributed VNFs, and enables both open- and close-loop control over networking systems. This paper illustrates the use of Aquarius with a traffic classifier, an auto-scaling system, and a load balancer—and demonstrates the use of three different ML paradigms—unsupervised, supervised, and reinforcement learning, within Aquarius, for network state inference and service management. Testbed evaluations show that Aquarius suitably improves network state visibility and brings notable performance gains for various scenarios with low overhead.

HLB: Toward Load-Aware Load Balancing

Abstract: The purpose of network load balancers is to optimize quality of service to the users of a set of servers– basically, to improve response times and to reducing computing resources– by properly distributing workloads. This paper proposes a distributed, application-agnostic, Hybrid Load Balancer (HLB) that– without explicit monitoring or signaling– infers server occupancies and processing speeds, which allows making optimised workload placement decisions. This approach is evaluated both through simulations and extensive experiments, including synthetic workloads and Wikipedia replays on a real-world testbed. Results show significant performance gains, in terms of both response time and system utilisation, when compared to existing load-balancing algorithms.

Reinforced Cooperative Load Balancing in Data Center

Abstract: Network load balancers are central components in modern data centers, that cooperatively distribute workloads of high arrival rates across application servers, thereby contribute to offering scalable services. The independent and “selfish” load balancing strategy is not necessarily the globally optimal one. This paper represents the load balancing problem as a cooperative team-game with limited observations over system states, and adopts multiagent reinforcement learning methods to make fair load balancing decisions without inducing additional processing latency. On both a simulation and an emulation system, the proposed method is evaluated against other load balancing algorithms, including state-of-the-art heuristics and learningbased strategies. Experiments under different settings and complexities show the advantageous performance of the proposed method.

Efficient Data-Driven Network Functions

Abstract: Cloud environments require dynamic and adaptive networking policies. It is preferred to use heuristics over advanced learning algorithms in Virtual Network Functions (VNFs) in production because of high-performance constraints. This paper proposes Aquarius to passively yet efficiently gather observations and enable the use of machine learning to collect, infer, and supply accurate networking state information - without incurring additional signaling and management overhead. This paper illustrates the use of Aquarius with a traffic classifier, an auto-scaling system, and a load balancer - and demonstrates the use of three different machine learning paradigms - unsupervised, supervised, and reinforcement learning, within Aquarius, for inferring network state. Testbed evaluations show that Aquarius increases network state visibility and brings notable performance gains with low overhead.

Optimal deployment of indoor wireless local area networks

Abstract: We present a two‐phase methodology to address the problem of optimally deploying indoor wireless local area networks. In the first phase, we use Helmholtz's equation to simulate electromagnetic fields in a typical environment such as an office floor. The linear system which results from the discretization of this partial differential equation is solved with a state‐of‐the‐art library for sparse linear algebra. In the second phase, we formulate the network deployment problem in the setting of binary linear programming. This formulation employs the simulator output as input parameters, and jointly optimizes the number of access points, their locations, and their emission channels. We prove that this optimization problem is NP‐Hard, and use mathematical programming based techniques and heuristics to solve it. We present numerical experiments on medium‐sized buildings.