The design of embedded systems warrants a new perspective because of the following two reasons: Firstly, slow and energy inefficient memory hierarchies have already become the bottleneck of the embedded systems. It is documented in the literature as the memory wall problem. Secondly, the software running on the contemporary embedded devices is becoming increasingly complex. It is also well understood that no silver bullet exists to solve the memory wall problem. Therefore, this book explores a collaborative approach by proposing novel memory hierarchies and software optimization techniques for the optimal utilization of these memory hierarchies. Linking memory architecture design with memory-architecture aware compilation results in fast, energy-efficient and timing predictable memory accesses. The evaluation of the optimization techniques using real-life benchmarks for a single processor system, a multiprocessor system-on-chip (SoC) and for a digital signal processor system, reports significant reductions in the energy consumption and performance improvement of these systems. The book presents a wide range of optimizations, progressively increasing in the complexity of analysis and of memory hierarchies. The final chapter covers optimization techniques for applications consisting of multiple processes found in most modern embedded devices. Advanced Memory Optimization Techniques for Low Power Embedded Processorsis designed for researchers, complier writers and embedded system designers / architects who wish to optimize the energy and performance characteristics of the memory subsystem.

Memory optimization techniques for low-power embedded processors

Which is the best metric for the space of collider events? Motivated by the success of the Energy Mover's Distance in characterizing collider events, we explore the larger space of unbalanced optimal transport distances, of which the Energy Mover's Distance is a particular case. Geometric and computational considerations favor an unbalanced optimal transport distance known as the Hellinger-Kantorovich distance, which possesses a Riemannian structure that lends itself to efficient linearization. We develop the particle linearized unbalanced Optimal Transport (pluOT) framework for collider events based on the linearized Hellinger-Kantorovich distance and demonstrate its efficacy in boosted jet tagging. This provides a flexible and computationally efficient optimal transport framework ideally suited for collider physics applications.

Which Metric on the Space of Collider Events

Which is the best metric for the space of collider events? Motivated by the success of the energy mover's distance in characterizing collider events, we explore the larger space of unbalanced optimal transport distances, of which the energy mover's distance is a particular case. Geometric and computational considerations favor an unbalanced optimal transport distance known as the Hellinger-Kantorovich distance, which possesses a Riemannian structure that lends itself to efficient linearization. We develop the particle linearized unbalanced optimal transport framework for collider events based on the linearized Hellinger-Kantorovich distance and demonstrate its efficacy in boosted jet tagging. This provides a flexible and computationally efficient optimal transport framework ideally suited for collider physics applications. 

http://link.aps.org/pdf/10.1103/PhysRevD.105.076003

Which metric on the space of collider events?

Large fractions of today’s embedded systems’ power consumption can be attributed to the memory subsystem. In order to reduce this fraction, we propose a mathematical model to optimize on-chip memory configurations for minimal power. We exploit the power reduction effect of splitting memory into subunits with frequently accessed addresses mapped to small memories. The definition of an integer linear programming model enables us to solve the twofold problem of allocating an optimal set of memory instances with varying size on the one hand and finding an optimal mapping of application segments to allocated memories on the other hand. Experimental results yield power reductions of up to 82 % for instruction memory and 73 % for data memory. Area usage, at the same time, deteriorates by only 2.1 %, respectively, 1.2 % on average and even improves in some cases. Flexibility and performance of our model make it a valuable tool for low power system-on-chip design, either for efficient design space exploration or as part of a HW/SW codesign synthesis flow.

/pdf/low-power-memory-allocation-and-mapping-for-area-constrained-ss7vsgaq3b.pdf

Low power memory allocation and mapping for area-constrained systems-on-chips

Cutting edge detectors push sensing technology by further improving spatial and temporal resolution, increasing detector area and volume, and generally reducing backgrounds and noise. This has led to a explosion of more and more data being generated in next-generation experiments. Therefore, the need for near-sensor, at the data source, processing with more powerful algorithms is becoming increasingly important to more eﬃciently capture the right experimental data, reduce downstream system complexity, and enable faster and lower-power feedback loops. In this paper, we discuss the motivations and potential applications for on-detector AI. Furthermore, the unique requirements of particle physics can uniquely drive the development of novel AI hardware and design tools. We describe existing modern work for particle physics in this area. Finally, we out-line a number of areas of opportunity where we can advance machine learning techniques, codesign workﬂows, and future microelectronics technologies which will accelerate design, performance, and implementations for next generation experiments. Submitted to Proceedings the Future of Particle Physics (Snowmass

Smart sensors using artificial intelligence for on-detector electronics and ASICs

Despite advances in the programmable logic capabilities of modern trigger systems, a significant bottleneck remains in the amount of data to be transported from the detector to off-detector logic where trigger decisions are made. We demonstrate that a neural network (NN) autoencoder model can be implemented in a radiation-tolerant application-specific integrated circuit (ASIC) to perform lossy data compression alleviating the data transmission problem while preserving critical information of the detector energy profile. For our application, we consider the high-granularity calorimeter from the Compact Muon Solenoid (CMS) experiment at the CERN Large Hadron Collider. The advantage of the machine learning approach is in the flexibility and configurability of the algorithm. By changing the NN weights, a unique data compression algorithm can be deployed for each sensor in different detector regions and changing detector or collider conditions. To meet area, performance, and power constraints, we perform quantization-aware training to create an optimized NN hardware implementation. The design is achieved through the use of high-level synthesis tools and the hls4ml framework and was processed through synthesis and physical layout flows based on a low-power (LP)-CMOS 65-nm technology node. The flow anticipates 200 Mrad of ionizing radiation to select gates and reports a total area of 3.6 mm2 and consumes 95 mW of power. The simulated energy consumption per inference is 2.4 nJ. This is the first radiation-tolerant on-detector ASIC implementation of an NN that has been designed for particle physics applications.

/pdf/a-reconfigurable-neural-network-asic-for-detector-front-end-1posbodx2x.pdf

A Reconfigurable Neural Network ASIC for Detector Front-End Data Compression at the HL-LHC

This paper presents an energy-aware electronic design automation (EDA) methodology for the system-level exploration of hierarchical storage organizations, focusing mainly on data-intensive signal processing applications. Starting from the high-level behavioral specification of a given application, several memory management tasks are addressed in a common algebraic framework, using data-dependence analysis techniques similar to those used in modern compilers. Within this memory management software system, the designer can explore different algorithmic specifications functionally equivalent, by computing their minimum storage requirements. The system can perform an exploration based on energy consumption of signal assignments to the off- and on-chip memory layers, followed by a storage-efficient mapping of signals to the physical memories. The last phase of the methodology is an exploration approach for energy-aware banking of the on-chip memory, which takes into account both the static and dynamic energy consumption.

Design space exploration for low-power memory systems in embedded signal processing applications

Hierarchical memory organizations are used in embedded systems to reduce energy consumption and improve performance by exploiting the non-uniformity of memory accesses and assigning the frequently-accessed data to the low levels of memory hierarchy. Moreover, within a given level, energy can be further reduced and performance further enhanced by memory partitioning - whose principle is to divide the address space in several smaller blocks and to map these blocks to physical memory banks. Scratch-pad memories (SPMs) offer a good compromise - as on-chip storage in embedded systems - when taking into account performance, energy consumption, and die area. This paper addresses the problem of optimizing the partitioning of SPMs. Different from previous techniques, the cost function takes into account all the three major design objectives - energy consumption, performance, and die area, letting the designers decide on their relative importance for a specific project. The proposed approach proved to be computationally fast and very efficient when tested for several data-intensive applications, whose behavioral specifications contain multidimensional arrays as main data structures.

Energy-aware scratch-pad memory partitioning for embedded systems

In real-time data-intensive multimedia processing applications, data transfer and storage significantly influence, if not dominate, all the major cost parameters of the design space - namely power consumption, performance, and chip area. This paper presents an electronic design automation (EDA) methodology for the high-level design of hierarchical memory architectures in embedded data-intensive applications, mainly in the area of multidimensional signal processing. This framework employs a formal model operating with integral polyhedra, using techniques specific to the data-dependence analysis employed in modern compilers. Different from the previous works, the problems of data assignment to the memory layers and of banking the on-chip memory are addressed in a consistent way, based on the same formal model. The main design target is the reduction of the static and dynamic energy consumption in the memory subsystem, but the same formal model and algorithmic flow can be also applied to reduce the overall time of access to memories.

Compiler-directed memory hierarchy design for low-energy embedded systems

Partitioning a memory into multiple banks that can be independently accessed is an approach mainly used for the reduction of the dynamic energy consumption When leakage energy comes into play as well, the idle memory banks must be put in a low-leakage `dormant' state to save static energy when not accessed The energy savings must be large enough to compensate the energy overhead spent by changing the bank status from active to dormant, then back to active again This paper addresses the problem of energy-aware on-chip memory banking, taking into account - during the exploration of the search space - the idleness time intervals of the data mapped into the memory banks As on-chip storage, we target scratch-pad memories (SPMs) since they are commonly used in embedded systems as an alternative to caches The proposed approach proved to be computationally fast and very efficient when tested for several data-intensive applications, whose behavioral specifications contain multidimensional arrays as main data structures

Cristian V. Gingu

Papers

A Reconfigurable Neural Network ASIC for Detector Front-End Data Compression at the HL-LHC

Design space exploration for low-power memory systems in embedded signal processing applications

Energy-aware scratch-pad memory partitioning for embedded systems

Compiler-directed memory hierarchy design for low-energy embedded systems

Leakage-aware scratch-pad memory banking for embedded multidimensional signal processing