Extremely high data rates at modern synchrotron and X-ray free-electron lasers (XFELs) light source beamlines motivate the use of machine learning methods for data reduction, feature detection, and other purposes. Regardless of the application, the basic concept is the same: data collected in early stages of an experiment, data from past similar experiments, and/or data simulated for the upcoming experiment are used to train machine learning models that, in effect, learn specific characteristics of those data; these models are then used to process subsequent data more efficiently than would general-purpose models that lack knowledge of the specific dataset or data class. Thus, a key challenge is to be able to train models with sufficient rapidity that they can be deployed and used within useful timescales. We describe here how specialized data center AI systems can be used for this purpose.

Bridge Data Center AI Systems with Edge Computing for Actionable Information Retrieval.

Extracting actionable information rapidly from data produced by instruments such as the Linac Coherent Light Source (LCLS-II) and Advanced Photon Source Upgrade (APS-U) is becoming ever more challenging due to high (up to TB/s) data rates. Conventional physics-based information retrieval methods are hard-pressed to detect interesting events fast enough to enable timely focusing on a rare event or correction of an error. Machine learning (ML) methods that learn cheap surrogate classifiers present a promising alternative, but can fail catastrophically when changes in instrument or sample result in degradation in ML performance. To overcome such difficulties, we present a new data storage and ML model training architecture designed to organize large volumes of data and models so that when model degradation is detected, prior models and/or data can be queried rapidly and a more suitable model retrieved and fine-tuned for new conditions. We show that our approach can achieve up to 100x data labelling speedup compared to the current state-of-the-art, 200x improvement in training speed, and 92x speedup in-terms of end-to-end model updating time.

fairDMS: Rapid Model Training by Data and Model Reuse

In a world heading towards applications, in science and industry, based on big data processing, the ability to elaborate streams of data is becoming more important every day. Applications of various natures, ranging from biology to chemistry, from medical imaging to spectroscopy, need systems able to detect, process and store huge amounts of data in real-time. In this context, techniques such as histogramming come into play. Histograms are able to represent data shapes and retrieve statistical information, favoring further processing. This kind of processing is usually done with the help of general purpose processors, relying on temporal computing, with their pros (simplicity and fast operating frequencies) and cons (inability to exploit parallel computation). Both Industry and Academia have, however, proposed many solutions to this need, delivering histogram generators both in full-custom Application-Specific Integrated Circuits (ASICs) and Field-Programmable Gate Array (FPGA) IP-Cores, preferred over processors thanks to their superior parallel computing power. In this work, we present a full FPGA approach to this issue, resulting in a high-performance IP-Core for generating and managing multiple histograms in real time, each supporting a data flow up to 224 MSps, with a strong focus on overall flexibility and area efficiency, using as low as < 250 LUTs and 300 FFs resources for a 16 bit-wide, 4096-bin histogram implementation. The IP-Core has been successfully integrated and validated in a high-performance time-mode application: an FPGA-based Time-to-Digital Converter. The resulting IP-Core is, more in general, a single solution perfectly adaptable to any field of application and FPGA device. 

High-Performance Computing of Real-Time and Multichannel Histograms: A Full FPGA Approach

High throughput and low latency data processing is essential for systems requiring live decision making, control, and machine learning-optimized data reduction. We focus on two distinct use cases for in-flight streaming data processing for a) X-ray pulse reconstruction at SLAC’s LCLS-II Free-Electron Laser and b) control diagnostics at the DIII-D tokamak fusion reactor. Both cases exemplify high throughput and low latency control feedback and motivate our focus on machine learning at the edge where data processing and machine learning algorithms can be implemented in field programmable gate array based hardware immediately after the diagnostic sensors. We present our recent work on a data preprocessing chain which requires fast featurization for information encoding. We discuss several options for such algorithms with the primary focus on our discrete cosine and sine transform-based approach adapted for streaming data. These algorithms are primarily aimed at implementation in field programmable gate arrays, favoring linear algebra operations that are also aligned with the recent advances in inference accelerators for the computational edge. 

At-the-Edge Data Processing for Low Latency High Throughput Machine Learning Algorithms

New developments in radiation and photonic detectors improve resolution, sensitivity, size, and rate, all of which contribute to a gigantic increase in the data production rate. Moving data analysis and compression adjacent or even embedded within the detector hardware will reduce the data volumes generated, thereby reducing material cost, power, and data management requirements. Several solutions are already being developed both on the hardware and on the software side to facilitate the use of machine learning as a real-time data analysis solution.

Potential of edge machine learning for instrumentation.

Several large physics experiments face an increasingly large "data firehose". Raw data generation exceeds TB/s rates for several existing and planned experiments, generating untenable data sets in very little time. The data often contain limited information and extracting this relevant information online would reduce the offline storage requirements by several orders of magnitude. Additionally, ultra low latency data analysis can be used to drive a fast feedback control system to adjust the experiment in real time, including decisions on data acquisition conditions, detector parameter adjustments and source operation modifications. However, most data state-of-the-art algorithms use computationally expensive operations and require uploading the data to a CPU or GPU compute node. With appropriate training, machine learning can categorize data samples and extract relevant information from raw data using simple arithmetic operations. Placing these fast inference models on FPGAs near the detector— at the edge—would reduce the data velocity at the source. We demonstrate this approach with an initial proof of concept targeting the CookieBox, an angular streaking detector developed for LCLS-II placed upstream as an online beam diagnostic tool. Data is streamed to the FPGA where a parallel and pipelined inference model extracts the relevant information in less than 20 µs at a streaming rate of 77 million events per second.

Omar Quijano

Papers

Machine Learning at the Edge for Ultra High Rate Detectors