Edge Impulse is a cloud-based machine learning operations (MLOps) platform for developing embedded and edge ML (TinyML) systems that can be deployed to a wide range of hardware targets. Current TinyML workflows are plagued by fragmented software stacks and heterogeneous deployment hardware, making ML model optimizations difficult and unportable. We present Edge Impulse, a practical MLOps platform for developing TinyML systems at scale. Edge Impulse addresses these challenges and streamlines the TinyML design cycle by supporting various software and hardware optimizations to create an extensible and portable software stack for a multitude of embedded systems. As of Oct. 2022, Edge Impulse hosts 118,185 projects from 50,953 developers.

Edge Impulse: An MLOps Platform for Tiny Machine Learning

Deploying machine learning applications on edge devices can bring clear benefits such as improved reliability, latency and privacy but it also introduces its own set of challenges. Most works focus on the limited computational resources of edge platforms but this is not the only bottleneck standing in the way of widespread adoption. In this paper we list several other challenges that a Tiny ML practitioner might need to consider when operationalizing an application on edge devices. We focus on tasks such as monitoring and managing the application, common functionality for a MLOps platform, and show how they are complicated by the distributed nature of edge deployment. We also discuss issues that are unique to edge applications such as protecting a model's intellectual property and verifying its integrity.

TinyMLOps: Operational Challenges for Widespread Edge AI Adoption

The role of Machine Learning (ML) engineers in mobile development has become increasingly important in recent years, as more and more business-critical mobile applications depend on AI components. Many development teams already include dedicated ML engineers who aim to follow agile development practices in their work, as part of the larger MLOps concept. However, the availability of MLOps tools tailored specifically towards mobile platforms is scarce, often due the limited support for non-native programming languages such as Python, as well as the unsuitability of native programming languages such as Java and Kotlin to support ML-related programming tasks. This paper aims to address this gap and describes a plug-in architecture for developing, deploying and running data ingestion and processing components written in Python on the Android platform. With the possibility to pass a user-defined schema with the data format and structure, the proposed architecture ensures that time-series datasets are correctly interpreted by multiple ML modules dealing with both data ingestion and processing,. The proposed approach benefits from modularity, extensibility, customisation, and separation of concerns, which enable ML engineers to be fully involved in a mobile development lifecycle following agile MLOps practices.

Bridging the Gap Between Java and Python in Mobile Software Development to Enable MLOps

Smartphones are increasingly used as universal IoT gateways collecting data from connected sensors in a wide range of industrial applications. With the increasing computing capabilities, they are used not just for simple data aggregation and transferring, but have now become capable of performing advanced data analytics. As AI has become a key element in enterprise software systems, many software development teams rely on dedicated Machine Learning (ML) engineers who often follow agile development practices in their work. However, in the context of mobile app development, there is still limited tooling support for MLOps, mainly due to unsuitability of native programming languages such as Java and Kotlin to support ML-related programming tasks. This paper aims to address this gap and describes a plug-in architecture for developing, deploying and running ML modules for data analytics on the Android platform. The proposed approach advocates for modularity, extensibility, customisation, and separation of concerns, allowing ML engineers to develop their components independently from the main application in an agile and incremental manner.

Towards MLOps in Mobile Development with a Plug-in Architecture for Data Analytics

Although deep neural networks are typically computationally expensive to use, technological advances in both the design of hardware platforms and of neural network architectures, have made it possible to use powerful models on edge devices. To enable widespread adoption of edge based machine learning, we introduce a set of open-source tools that make it easy to deploy, update and monitor machine learning models on a wide variety of edge devices. Our tools bring the concept of containerization to the TinyMLworld.We propose to package ML and application logic as containers called Runes to deploy onto edge devices. The containerization allows us to target a fragmented Internet-of-Things (IoT) ecosystem by providing a common platform for Runes to run across devices.

A VM/Containerized Approach for Scaling TinyML Applications

Software development is a sequential process where the allied steps in the development lifecycle involve planning and modularization, requirement engineering, analysis of product viability, profits estimation, strategic decision making, maintenance strategies etc. Often, most of these phases are pretty complex and thereby, extremely difficult to handle solely through human intervention, mainly due to the size of the project, the number of factors to be taken into consideration at each modular level and the rapidly changing external environment. In this paper we aim to provide an intuition on using Artificial Intelligence (AI) in the different phases of the software development lifecycle. Our paper focuses on a specific software development example for clarity and precision, but most of the techniques are highly general and scalable to any software development process.

/pdf/use-of-artificial-intelligence-in-automation-of-sequential-2ud7e4ilry.pdf

Akshay Sharma

Papers

A VM/Containerized Approach for Scaling TinyML Applications

TinyMLOps: Operational Challenges for Widespread Edge AI Adoption

Use of artificial intelligence in automation of sequential steps of software development / production