Deep learning is a field in artificial intelligence that works well in computer vision, natural language processing and audio recognition. Deep neural network architectures has number of layers to conceive the features well, by itself. The hyperparameter tuning plays a major role in every dataset which has major effect in the performance of the training model. Due to the large dimensionality of data it is impossible to tune the parameters by human expertise. In this paper, we have used the CIFAR-10 Dataset and applied the Bayesian hyperparameter optimization algorithm to enhance the performance of the model. Bayesian optimization can be used for any noisy black box function for hyperparameter tuning. In this work Bayesian optimization clearly obtains optimized values for all hyperparameters which saves time and improves performance. The results also show that the error has been reduced in graphical processing unit than in CPU by 6.2% in the validation. Achieving global optimization in the trained model helps transfer learning across domains as well.

Automatic tuning of hyperparameters using Bayesian optimization

Automatic transistor sizing is a challenging problem in circuit design due to the large design space, complex performance tradeoffs, and fast technology advancements. Although there have been plenty of work on transistor sizing targeting on one circuit, limited research has been done on transferring the knowledge from one circuit to another to reduce the re-design overhead. In this paper, we present GCN-RL Circuit Designer, leveraging reinforcement learning (RL) to transfer the knowledge between different technology nodes and topologies. Moreover, inspired by the simple fact that circuit is a graph, we learn on the circuit topology representation with graph convolutional neural networks (GCN). The GCN-RL agent extracts features of the topology graph whose vertices are transistors, edges are wires. Our learning-based optimization consistently achieves the highest Figures of Merit (FoM) on four different circuits compared with conventional black box optimization methods (Bayesian Optimization, Evolutionary Algorithms), random search and human expert designs. Experiments on transfer learning between five technology nodes and two circuit topologies demonstrate that RL with transfer learning can achieve much higher FoMs than methods without knowledge transfer. Our transferable optimization method makes transistor sizing and design porting more effective and efficient.

GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning

Automatic transistor sizing is a challenging problem in circuit design due to the large design space, complex performance trade-offs, and fast technological advancements. Although there has been plenty of work on transistor sizing targeting on one circuit, limited research has been done on transferring the knowledge from one circuit to another to reduce the re-design overhead. In this paper, we present GCN-RL Circuit Designer, leveraging reinforcement learning (RL) to transfer the knowledge between different technology nodes and topologies. Moreover, inspired by the simple fact that circuit is a graph, we learn on the circuit topology representation with graph convolutional neural networks (GCN). The GCN-RL agent extracts features of the topology graph whose vertices are transistors, edges are wires. Our learning-based optimization consistently achieves the highest Figures of Merit (FoM) on four different circuits compared with conventional black-box optimization methods (Bayesian Optimization, Evolutionary Algorithms), random search, and human expert designs. Experiments on transfer learning between five technology nodes and two circuit topologies demonstrate that RL with transfer learning can achieve much higher FoMs than methods without knowledge transfer. Our transferable optimization method makes transistor sizing and design porting more effective and efficient.

We present Circuit-GNN, a graph neural network (GNN) model for designing distributed circuits. Today, designing distributed circuits is a slow process that can take months from an expert engineer. Our model both automates and speeds up the process. The model learns to simulate the electromagnetic (EM) properties of distributed circuits. Hence, it can be used to replace traditional EM simulators, which typically take tens of minutes for each design iteration. Further, by leveraging neural networks’ differentiability, we can use our model to solve the inverse problem – i.e., given desirable EM specifications, we propagate the gradient to optimize the circuit parameters and topology to satisfy the specifications. We exploit the flexibility of GNN to create one model that works for different circuit topologies. We compare our model with a commercial simulator showing that it reduces simulation time by four orders of magnitude. We also demonstrate the value of our model by using it to design a Terahertz channelizer, a difficult task that requires a specialized expert. The results show that our model produces a channelizer whose performance is as good as a manually optimized design, and can save the expert several weeks of topology and parameter optimization. Most interestingly, our model comes up with new designs that differ from the limited templates commonly used by engineers in the field, hence significantly expanding the design space.

/pdf/circuit-gnn-graph-neural-networks-for-distributed-circuit-5ep4u92mmj.pdf

Circuit-GNN: Graph Neural Networks for Distributed Circuit Design

The expected improvement (EI) algorithm is a very popular method for expensive optimization problems. In the past twenty years, the EI criterion has been extended to deal with a wide range of expensive optimization problems. This paper gives a comprehensive review of the EI extensions designed for parallel optimization, multiobjective optimization, constrained optimization, noisy optimization, multi-fidelity optimization and high-dimensional optimization. The main challenges of extending the EI approach to solve these complex optimization problems are pointed out, and the ideas proposed in literature to tackle these challenges are highlighted. For each reviewed algorithm, the surrogate modeling method, the computation of the infill criterion and the internal optimization of the infill criterion are carefully studied and compared. In addition, the monotonicity properties of the multiobjective EI criteria and constrained EI criteria are analyzed in detail. Through this review, we give an organized summary about the EI developments in the past twenty years and show a clear picture about how the EI approach has advanced. In the end of this paper, several interesting problems and future research topics about the EI developments are given.

Expected improvement for expensive optimization: a review

The computation-intensive circuit simulation makes the analog circuit sizing challenging for large-scale/complicated analog/RF circuits. A Bayesian optimization approach has been proposed recently for the optimization problems involving the evaluations of black-box functions with high computational cost in either objective functions or constraints. In this paper, we propose a weighted expected improvement-based Bayesian optimization approach for automated analog circuit sizing. Gaussian processes (GP) are used as the online surrogate models for circuit performances. Expected improvement is selected as the acquisition function to balance the exploration and exploitation during the optimization procedure. The expected improvement is weighted by the probability of satisfying the constraints. In this paper, we propose a complete Bayesian optimization framework for the optimization of analog circuits with constraints for the first time. The existing GP model-based optimization methods for analog circuits take the GP models as either offline models or as assistance for the evolutionary algorithms. We also extend the Bayesian optimization algorithm to handle multi-objective optimization problems. Compared with the state-of-the-art approaches listed in this paper, the proposed Bayesian optimization method achieves better optimization results with significantly less number of simulations.

An Efficient Bayesian Optimization Approach for Automated Optimization of Analog Circuits

Bayesian optimization methods are promising for the optimization of black-box functions that are expensive to evaluate. In this paper, a novel batch Bayesian optimization approach is proposed. The parallelization is realized via a multi-objective ensemble of multiple acquisition functions. In each iteration, the multi-objective optimization of the multiple acquisition functions is performed to search for the Pareto front of the acquisition functions. The batch of inputs are then selected from the Pareto front. The Pareto front represents the best trade-off between the multiple acquisition functions. Such a policy for batch Bayesian optimization can significantly improve the efficiency of optimization. The proposed method is compared with several state-of-the-art batch Bayesian optimization algorithms using analytical benchmark functions and real-world analog integrated circuits. The experimental results show that the proposed method is competitive compared with the state-of-the-art algorithms.

/pdf/batch-bayesian-optimization-via-multi-objective-acquisition-2vgrmr6aeb.pdf

Batch Bayesian Optimization via Multi-objective Acquisition Ensemble for Automated Analog Circuit Design

In this paper, a novel multi-objective Bayesian optimization method is proposed for the sizing of analog/RF circuits. The proposed approach follows the framework of Bayesian optimization to balance the exploitation and exploration. Gaussian processes (GP) are used as the online surrogate models for the multiple objective functions. The lower confidence bound (LCB) functions are taken as the acquisition functions to select the data point with best Pareto-dominance and diversity. A modified non-dominated sorting based evolutionary multi-objective algorithm is proposed to find the Pareto Front (PF) of the multiple LCB functions, and the next simulation point is chosen from the PF of the multiple LCB functions. Compared with the multi-objective evolutionary algorithms (MOEA) and the state-of-the-art online surrogate model based circuit optimization method, our method can better approximate the Pareto Front while significantly reduce the number of circuit simulations.

Multi-objective bayesian optimization for analog/RF circuit synthesis

This paper presents an efficient multi-fidelity Bayesian optimization approach for analog circuit synthesis. The proposed method can significantly reduce the overall computational cost by fusing the simple but potentially inaccurate low-fidelity model and a few accurate but expensive high-fidelity data. Gaussian Process (GP) models are employed to model the low- and high-fidelity black-box functions separately. The nonlinear map between the low-fidelity model and high-fidelity model is also modelled as a Gaussian process. A fusing GP model which combines the low- and high-fidelity models can thus be built. An acquisition function based on the fusing GP model is used to balance the exploitation and exploration. The fusing GP model is evolved gradually as new data points are selected sequentially by maximizing the acquisition function. Experimental results show that our proposed method reduces up to 65.5% of the simulation time compared with the state-of-the-art single-fidelity Bayesian optimization method, while exhibiting more stable performance and a more promising practical prospect.

An Efficient Multi-fidelity Bayesian Optimization Approach for Analog Circuit Synthesis

Bayesian optimization with Gaussian process as surrogate model has been successfully applied to analog circuit synthesis. In the traditional Gaussian process regression model, the kernel functions are defined explicitly. The computational complexity of training is O(N3), and the computation complexity of prediction is O(N2), where N is the number of training data. Gaussian process model can also be derived from a weight space view, where the original data are mapped to feature space, and the kernel function is defined as the inner product of nonlinear features. In this paper, we propose a Bayesian optimization approach for analog circuit synthesis using neural network. We use deep neural network to extract good feature representations, and then define Gaussian process using the extracted features. Model averaging method is applied to improve the quality of uncertainty prediction. Compared to Gaussian process model with explicitly defined kernel functions, the neural-network-based Gaussian process model can automatically learn a kernel function from data, which makes it possible to provide more accurate predictions and thus accelerate the follow-up optimization procedure. Also, the neural-network-based model has O(N) training time and constant prediction time. The efficiency of the proposed method has been verified by two real-world analog circuits.

Wenlong Lyu

Papers

An Efficient Bayesian Optimization Approach for Automated Optimization of Analog Circuits

Batch Bayesian Optimization via Multi-objective Acquisition Ensemble for Automated Analog Circuit Design

Multi-objective bayesian optimization for analog/RF circuit synthesis

An Efficient Multi-fidelity Bayesian Optimization Approach for Analog Circuit Synthesis

Bayesian Optimization Approach for Analog Circuit Synthesis Using Neural Network