M
Malte Schwarzkopf
Researcher at Massachusetts Institute of Technology
Publications - 49
Citations - 2907
Malte Schwarzkopf is an academic researcher from Massachusetts Institute of Technology. The author has contributed to research in topics: Computer science & Scheduling (computing). The author has an hindex of 18, co-authored 40 publications receiving 2193 citations. Previous affiliations of Malte Schwarzkopf include University of Cambridge & Brown University.
Papers
More filters
Proceedings ArticleDOI
Omega: flexible, scalable schedulers for large compute clusters
TL;DR: This work presents a novel approach to address increasing scale and the need for rapid response to changing requirements using parallelism, shared state, and lock-free optimistic concurrency control to address monolithic cluster scheduler architectures.
Proceedings ArticleDOI
Learning scheduling algorithms for data processing clusters
TL;DR: Decima as discussed by the authors uses reinforcement learning and neural networks to learn workload-specific scheduling algorithms without any human instruction beyond a high-level objective, such as minimizing average job completion time, and shows that RL techniques can generate highly-efficient policies automatically.
Posted Content
Learning Scheduling Algorithms for Data Processing Clusters
TL;DR: It is shown that modern machine learning techniques can generate highly-efficient policies automatically and improve average job completion time by at least 21% over hand-tuned scheduling heuristics, achieving up to 2x improvement during periods of high cluster load.
Proceedings ArticleDOI
CIEL: a universal execution engine for distributed data-flow computing
Derek G. Murray,Malte Schwarzkopf,Christopher Smowton,Steven G. Smith,Anil Madhavapeddy,Steven Hand +5 more
TL;DR: The execution engine provides transparent fault tolerance and distribution to Skywriting scripts and high-performance code written in other programming languages, and achieves scalable performance for both iterative and non-iterative algorithms.
Proceedings ArticleDOI
Firmament: fast, centralized cluster scheduling at scale
TL;DR: Firmament is described, a centralized scheduler that scales to over ten thousand machines at sub-second placement latency even though it continuously reschedules all tasks via a min-cost max-flow (MCMF) optimization, and exceeds the placement quality of four widely-used centralized and distributed schedulers on a real-world cluster.