scispace - formally typeset
Open AccessProceedings Article

Hidden technical debt in Machine learning systems

Reads0
Chats0
TLDR
It is found it is common to incur massive ongoing maintenance costs in real-world ML systems, and several ML-specific risk factors to account for in system design are explored.
Abstract
Machine learning offers a fantastically powerful toolkit for building useful complex prediction systems quickly. This paper argues it is dangerous to think of these quick wins as coming for free. Using the software engineering framework of technical debt, we find it is common to incur massive ongoing maintenance costs in real-world ML systems. We explore several ML-specific risk factors to account for in system design. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns.

read more

Content maybe subject to copyright    Report

Citations
More filters
Proceedings ArticleDOI

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

TL;DR: In this article, the authors propose LIME, a method to explain models by presenting representative individual predictions and their explanations in a non-redundant way, framing the task as a submodular optimization problem.
Posted Content

Towards A Rigorous Science of Interpretable Machine Learning

TL;DR: This position paper defines interpretability and describes when interpretability is needed (and when it is not), and suggests a taxonomy for rigorous evaluation and exposes open questions towards a more rigorous science of interpretable machine learning.
Proceedings Article

Anchors: High-Precision Model-Agnostic Explanations

TL;DR: This work introduces a novel model-agnostic system that explains the behavior of complex models with high-precision rules called anchors, representing local, “sufficient” conditions for predictions, and proposes an algorithm to efficiently compute these explanations for any black-box model with high probability guarantees.
Proceedings ArticleDOI

Software engineering for machine learning: a case study

TL;DR: A study conducted on observing software teams at Microsoft as they develop AI-based applications finds that various Microsoft teams have united this workflow into preexisting, well-evolved, Agile-like software engineering processes, providing insights about several essential engineering challenges that organizations may face in creating large-scale AI solutions for the marketplace.
Journal ArticleDOI

Massive MIMO is a reality—What is next?: Five promising research directions for antenna arrays

TL;DR: In this paper, the authors explain how the first chapter of the massive MIMO research saga has come to an end, while the story has just begun, and outline five new massive antenna array related research directions.
References
More filters
Book

Refactoring: Improving the Design of Existing Code

TL;DR: Almost every expert in Object-Oriented Development stresses the importance of iterative development, but how do you add function to the existing code base while still preserving its design integrity?
Proceedings ArticleDOI

Refactoring improving the design of existing code

TL;DR: The present document details the how, why and when to apply refactoring in computer systems that have been poorly designed, this in order to a better performance and maintenance of the constituent components.
Proceedings ArticleDOI

Scaling Distributed Machine Learning with the Parameter Server

Mu Li
TL;DR: View on new challenges identified are shared, and some of the application scenarios such as micro-blog data analysis and data processing in building next generation search engines are covered.
Proceedings ArticleDOI

Scaling distributed machine learning with the parameter server

TL;DR: In this paper, the authors propose a parameter server framework for distributed machine learning problems, where both data and workloads are distributed over worker nodes, while the server nodes maintain globally shared parameters, represented as dense or sparse vectors and matrices.
Book

AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis

TL;DR: An entertaining and often enlightening text that defines what seasoned developers have long suspected: despite advances in software engineering, most software projects still fail to meet expectations--and about a third are cancelled altogether.
Related Papers (5)