Deep learning is a field in artificial intelligence that works well in computer vision, natural language processing and audio recognition. Deep neural network architectures has number of layers to conceive the features well, by itself. The hyperparameter tuning plays a major role in every dataset which has major effect in the performance of the training model. Due to the large dimensionality of data it is impossible to tune the parameters by human expertise. In this paper, we have used the CIFAR-10 Dataset and applied the Bayesian hyperparameter optimization algorithm to enhance the performance of the model. Bayesian optimization can be used for any noisy black box function for hyperparameter tuning. In this work Bayesian optimization clearly obtains optimized values for all hyperparameters which saves time and improves performance. The results also show that the error has been reduced in graphical processing unit than in CPU by 6.2% in the validation. Achieving global optimization in the trained model helps transfer learning across domains as well.

Automatic tuning of hyperparameters using Bayesian optimization

We present Circuit-GNN, a graph neural network (GNN) model for designing distributed circuits. Today, designing distributed circuits is a slow process that can take months from an expert engineer. Our model both automates and speeds up the process. The model learns to simulate the electromagnetic (EM) properties of distributed circuits. Hence, it can be used to replace traditional EM simulators, which typically take tens of minutes for each design iteration. Further, by leveraging neural networks’ differentiability, we can use our model to solve the inverse problem – i.e., given desirable EM specifications, we propagate the gradient to optimize the circuit parameters and topology to satisfy the specifications. We exploit the flexibility of GNN to create one model that works for different circuit topologies. We compare our model with a commercial simulator showing that it reduces simulation time by four orders of magnitude. We also demonstrate the value of our model by using it to design a Terahertz channelizer, a difficult task that requires a specialized expert. The results show that our model produces a channelizer whose performance is as good as a manually optimized design, and can save the expert several weeks of topology and parameter optimization. Most interestingly, our model comes up with new designs that differ from the limited templates commonly used by engineers in the field, hence significantly expanding the design space.

/pdf/circuit-gnn-graph-neural-networks-for-distributed-circuit-5ep4u92mmj.pdf

Circuit-GNN: Graph Neural Networks for Distributed Circuit Design

Bayesian optimization methods are promising for the optimization of black-box functions that are expensive to evaluate. In this paper, a novel batch Bayesian optimization approach is proposed. The parallelization is realized via a multi-objective ensemble of multiple acquisition functions. In each iteration, the multi-objective optimization of the multiple acquisition functions is performed to search for the Pareto front of the acquisition functions. The batch of inputs are then selected from the Pareto front. The Pareto front represents the best trade-off between the multiple acquisition functions. Such a policy for batch Bayesian optimization can significantly improve the efficiency of optimization. The proposed method is compared with several state-of-the-art batch Bayesian optimization algorithms using analytical benchmark functions and real-world analog integrated circuits. The experimental results show that the proposed method is competitive compared with the state-of-the-art algorithms.

/pdf/batch-bayesian-optimization-via-multi-objective-acquisition-2vgrmr6aeb.pdf

Batch Bayesian Optimization via Multi-objective Acquisition Ensemble for Automated Analog Circuit Design

A multistage collaborative machine learning (MS-CoML) method that can be applied to efficient multiobjective antenna modeling and optimization is proposed. Machine learning methods, including single-output Gaussian process regression (SOGPR) and symmetric and asymmetric multioutput GPR (MOGPR) methods, are introduced to collaboratively build highly accurate multitask surrogate models for antennas. Variable-fidelity electromagnetic (EM) models are simulated, with their responses utilized to build separate MOGPR surrogate models. By combining the three machine-learning methods in a multistage framework, mappings between the same and different responses of the EM models with variable fidelity are learned, therein helping to substantially reduce the computational effort under a negligible loss of predictive power. Three antenna designs aiming at single-band, broadband, and multiband applications are selected as examples. And, for illustrating the applicability and superiority of the proposed MS-CoML method, a reference point-based multiobjective antenna optimization algorithm is used to optimize these three antennas. Simulation results show that using the MS-CoML method can significantly reduce the total optimization time without compromising modeling accuracy and optimized performance.

Multistage Collaborative Machine Learning and its Application to Antenna Modeling and Optimization

Analog IC design relies on human experts to search for parameters that satisfy circuit specifications with their experience and intuitions, which is highly labor intensive, time consuming and suboptimal. Machine learning is a promising tool to automate this process. However, supervised learning is difficult for this task due to the low availability of training data: 1) Circuit simulation is slow, thus generating large-scale dataset is time-consuming; 2) Most circuit designs are propitiatory IPs within individual IC companies, making it expensive to collect large-scale datasets. We propose Learning to Design Circuits (L2DC) to leverage reinforcement learning that learns to efficiently generate new circuits data and to optimize circuits. We fix the schematic, and optimize the parameters of the transistors automatically by training an RL agent with no prior knowledge about optimizing circuits. After iteratively getting observations, generating a new set of transistor parameters, getting a reward, and adjusting the model, L2DC is able to optimize circuits. We evaluate L2DC on two transimpedance amplifiers. Trained for a day, our RL agent can achieve comparable or better performance than human experts trained for a quarter. It first learns to meet hard-constraints (eg. gain, bandwidth), and then learns to optimize good-to-have targets (eg. area, power). Compared with grid search-aided human design, L2DC can achieve $\mathbf{250}\boldsymbol{\times}$ higher sample efficiency with comparable performance. Under the same runtime constraint, the performance of L2DC is also better than Bayesian Optimization.

Learning to Design Circuits

The computation-intensive circuit simulation makes the analog circuit sizing challenging for large-scale/complicated analog/RF circuits. A Bayesian optimization approach has been proposed recently for the optimization problems involving the evaluations of black-box functions with high computational cost in either objective functions or constraints. In this paper, we propose a weighted expected improvement-based Bayesian optimization approach for automated analog circuit sizing. Gaussian processes (GP) are used as the online surrogate models for circuit performances. Expected improvement is selected as the acquisition function to balance the exploration and exploitation during the optimization procedure. The expected improvement is weighted by the probability of satisfying the constraints. In this paper, we propose a complete Bayesian optimization framework for the optimization of analog circuits with constraints for the first time. The existing GP model-based optimization methods for analog circuits take the GP models as either offline models or as assistance for the evolutionary algorithms. We also extend the Bayesian optimization algorithm to handle multi-objective optimization problems. Compared with the state-of-the-art approaches listed in this paper, the proposed Bayesian optimization method achieves better optimization results with significantly less number of simulations.

Pan Xue

Papers

An Efficient Bayesian Optimization Approach for Automated Optimization of Analog Circuits