There exists an urgent need for determining the right amount and type of specialization while making a heterogeneous system as programmable and flexible as possible. Therefore, in this paper, we pioneer a self-optimizing and self-programming computing system (SOSPCS) design framework that achieves both programmability and flexibility and exploits computing heterogeneity [e.g., CPUs, GPUs, and hardware accelerators (HWAs)]. First, at compile time, we form a task pool consisting of hybrid tasks with different processing element (PE) affinities according to target applications. Tasks preferred to be executed on GPUs or accelerators are detected from target applications by neural networks. Tasks suitable to run on CPUs are formed by community detection to minimize data movement overhead. Next, a distributed reinforcement learning-based approach is used at runtime to allow agents to map the tasks onto the network-on-chip-based heterogeneous PEs by learning an optimal policy based on $Q$ values in the environment. We have conducted experiments on a heterogeneous platform consisting of CPUs, GPUs, and HWAs with deep learning algorithms such as matrix multiplication, ReLU, and sigmoid functions. We concluded that SOSPCS provides performance improvement up to $4.12\times $ and energy reduction up to $3.24\times $ compared to the state-of-the-art approaches.

Self-Optimizing and Self-Programming Computing Systems: A Combined Compiler, Complex Networks, and Machine Learning Approach

The IARPA SuperTools program requires the development of superconducting electronic design automation (S-EDA) and superconducting technology computer-aided design (S-TCAD) tools aimed at enabling the reliable design of complex superconducting digital circuits with millions of Josephson junctions. Within the SuperTools program, the ColdFlux project addresses S-EDA and S-TCAD tool research and development in four areas: 1) RTL synthesis, architectures and verification; 2) analog design and layout synthesis; 3) physical design and test; and 4) device and process modeling/simulation and cell library design. Capabilities include, but are not limited to, the following: device level modeling and simulation of Josephson junctions, modeling and simulation of the superconducting process manufacturing processes, powerful new electrical circuit simulation, parameterized schematic and layout libraries, optimization, compact SPICE-like model extraction, timing analysis, behavioral, register-transfer-level and logic syntheses, clock tree synthesis, placement and routing, layout-versus-schematic extraction, functional verification, and the evaluation of designs in the presence of magnetic fields and trapped flux. ColdFlux consists of six research groups from four continents. Here, we present an overview of the current and planned activities related to the project and justify the design assumptions and decisions that were made to allow the development of design tools for million-gate circuits.

ColdFlux Superconducting EDA and TCAD Tools Project: Overview and Progress

Near-memory computing: Past, present, and future

The conventional approach of moving data to the CPU for computation has become a significant performance bottleneck for emerging scale-out data-intensive applications due to their limited data reuse. At the same time, the advancement in 3D integration technologies has made the decade-old concept of coupling compute units close to the memory --- called near-memory computing (NMC) --- more viable. Processing right at the "home" of data can significantly diminish the data movement problem of data-intensive applications. 
In this paper, we survey the prior art on NMC across various dimensions (architecture, applications, tools, etc.) and identify the key challenges and open issues with future research directions. We also provide a glimpse of our approach to near-memory computing that includes i) NMC specific microarchitecture independent application characterization ii) a compiler framework to offload the NMC kernels on our target NMC platform and iii) an analytical model to evaluate the potential of NMC.

Near-Memory Computing: Past, Present, and Future.

Fog computing is an emerging architectural paradigm for the implementation of the Internet of Things, where computation moves from cloud servers to network edges. Fog computing systems are with three characteristics: 1) low latency; 2) strong presence of real-time applications; and 3) reusability of end devices. Most existing designs of fog computing systems concentrate on reducing application processing latency, but neglect real-time requirements of applications and reusability of end devices, which may drastically degrade both functionality and quality-of-service (QoS) of applications. In this article, we investigate QoS optimization of real-time applications in fog computing systems equipped with reusable end devices and powered by hybrid energy of renewable generations and grid electricity. We propose a renewable-adaptive computation offloading approach. At the end device layer, local energy allocation schemes are designed at the application-level and component-level, where techniques of the cooperative game and mixed-integer linear programming (MILP) are leveraged, respectively. At the fog layer, the local energy allocation method is augmented to a local-remote scheduling solution by judiciously judging whether or not the computation offloading of an application needs to be triggered. The experimental results demonstrate that compared to benchmarking algorithms, our approach improves the overall and individual application QoS by up to 101.93% and 59.30%, respectively.

Exploring Renewable-Adaptive Computation Offloading for Hierarchical QoS Optimization in Fog Computing

Many-core multi-threaded performance is plagued by on-chip communication nonidealities, limited memory bandwidth, and critical sections. Inspired by complex network theory of social communities, we propose a novel methodology to model the dynamic execution of an application and partition the application into an optimal number of clusters for parallel execution. We first adopt an LLVM IR compiler analysis of a specific application and construct a dynamic application dependency graph encoding its computational and memory operations. Next, based on this graph, we propose an optimization model to find the optimal clusters such that (1) the intra-cluster edges are maximized, (2) the execution times of the clusters are nearly equalized, for load balancing, and (3) the cluster size does not exceed the core count. Our novel approach confines data movement to be mainly inside a cluster for power reduction and congestion prevention. Finally, we propose an algorithm to sort the graph of connected clusters topologically and map the clusters onto NoC. Experimental results on a 32-core NoC demonstrate a maximum speedup of 131.82% when compared to thread-based execution. Furthermore, the scalability of our framework makes it a promising software design automation platform.

A load balancing inspired optimization framework for exascale multicore systems: a complex networks approach

With increasing demand for distributed intelligent physical systems performing big data analytics on the field and in real-time, processing-in-memory (PIM) architectures integrating 3D-stacked memory and logic layers could provide higher performance and energy efficiency. Towards this end, the PIM design requires principled and rigorous optimization strategies to identify interactions and manage data movement across different vaults. In this paper, we introduce Prometheus, a novel PIM-based framework that constructs a comprehensive model of computation and communication (MoCC) based on a static and dynamic compilation of an application. Firstly, by adopting a low level virtual machine (LLVM) intermediate representation (IR), an input application is modeled as a two-layered graph consisting of (i) a computation layer in which the nodes denote computation IR instructions and edges denote data dependencies among instructions, and (ii) a communication layer in which the nodes denote memory operations (e.g., load/store) and edges represent memory dependencies detected by alias analysis. Secondly, we develop an optimization framework that partitions the multi-layer network into processing communities within which the computational workload is maximized while balancing the load among computational clusters. Thirdly, we propose a community-to-vault mapping algorithm for designing a scalable hybrid memory cube (HMC)-based system where vaults are interconnected through a network-on-chip (NoC) approach rather than a crossbar architecture. This ensures scalability to hundreds of vaults in each cube. Experimental results demonstrate that Prometheus consisting of 64 HMC-based vaults improves system performance by 9.8x and achieves 2.3x energy reduction, compared to conventional systems.

Prometheus: Processing-in-memory heterogeneous architecture design from a multi-layer network theoretic strategy

With increasing design complexity, the correlation between test transactions and functional properties becomes non-intuitive, hence impacting the reliability of test generation. This paper presents a modified coverage directed test generation based on an Artificial Neural Network (ANN). The ANN extracts features of test transactions and only those which are learned to be critical, will be sent to the design under verification. Furthermore, the priority of coverage groups is dynamically learned based on the previous test iterations. With ANN-based screening, low-coverage or redundant assertions will be filtered out, which helps accelerate the verification process. This allows our framework to learn from the results of the previous vectors and use that knowledge to select the following test vectors. Our experimental results confirm that our learning-based framework can improve the speed of existing function verification techniques by 24.5x and also also deliver assertion coverage improvement, ranging from 4.3x to 28.9x, compared to traditional coverage directed test generation, implemented in UVM.

Accelerating Coverage Directed Test Generation for Functional Verification: A Neural Network-based Framework

With the increasing demand for distributed big data analytics and data-intensive programs which contribute to large volumes of packets among processing elements (PEs) and memory banks, we witness a pressing need for new mathematical models and algorithms that can engineer a brain-inspired plasticity into the computing platforms by mining the topological complexity of high-level programs (HLPs) and exploiting their self-similar and fractal characteristics for designing reconfigurable domain-specific computing architectures. In this article, we present Plasticity-on-Chip (PoC) by engineering plasticity into ”artificial brains” to mine and exploit the self-similarity of HLPs. First, we present a communication modeling of HLPs (e.g., C/C++ implementations of various applications) that relies on static and dynamic compiler analysis of programs with varying input seeds, performing comprehensive program analysis of all traces, and representing the HLPs as weighted directed acyclic graphs while capturing the intrinsic timing constraints and data/control flow requirements. Second, we propose a rigorous mathematical framework for determining the optimal parallel degree of executing a set of interacting HLPs (by partitioning them into clusters of densely interconnected supernodes - tasks) which helps us decide the number of available heterogeneous PEs, the amount of required memory and the structure of the synthesized deadlock-free irregular NoC topology that offers an efficient communication medium. These clusters serve as abstract models of computation for the synthesized PEs within the parallel execution model. Finally, exploiting the fractal and complex networks concepts, we extract in-depth features from graphs that serve as inputs for distributed reinforcement learning. Our experimental results on synthesized PEs and NoCs show performance improvements as high as 7.61x when compared to the traditional NoC and 2.6x compared to gem5-Aladdin.

Yao Xiao

Papers

Self-Optimizing and Self-Programming Computing Systems: A Combined Compiler, Complex Networks, and Machine Learning Approach

A load balancing inspired optimization framework for exascale multicore systems: a complex networks approach

Prometheus: Processing-in-memory heterogeneous architecture design from a multi-layer network theoretic strategy

Accelerating Coverage Directed Test Generation for Functional Verification: A Neural Network-based Framework

Plasticity-on-Chip Design: Exploiting Self-Similarity for Data Communications