Hybrid neural network

About: Hybrid neural network is a research topic. Over the lifetime, 1305 publications have been published within this topic receiving 18223 citations.

Parana: A Parallel Neural Architecture Considering Thermal Problem of 3D Stacked Memory

Abstract: Recent advances in deep learning (DL) have stimulated increasing interests in neural networks (NN). From the perspective of operation type and network architecture, deep neural networks can be categorized into full convolution-based neural network (ConvNet), recurrent neural network (RNN), and fully-connected neural network (FCNet). Different types of neural networks are usually cascaded and combined as a hybrid neural network (Hybrid-NN) to complete real-life cognitive tasks. Such hybrid-NN implementation is memory-intensive with large number of memory accesses, hence the performance of hybrid-NN is often limited by the insufficient memory bandwidth. A “3D + 2.5D” integration system, which integrates a high-bandwidth 3D stacked DRAM side-by-side with a highly-parallel neural processing unit (NPU) on a silicon interposer, overcomes the bandwidth bottleneck in hybrid-NN acceleration. However, intensive concurrent 3D DRAM accesses produced by the NPU lead to a serious thermal problem in 3D DRAM. In this paper, we propose a neural processor called Parana for hybrid-NN acceleration in consideration of thermal problem of 3D DRAM. Parana solves the thermal problem of 3D memory by optimizing both the total number of memory accesses and memory accessing behaviors. For memory accessing behaviors, Parana balances the memory bandwidth by spatial division mapping hybrid-NN onto computing resources, which efficiently avoids that masses of memory accesses are issued in a short time period. To reduce the total number of memory accesses, we design a new NPU architecture and propose a memory-oriented tiling and scheduling mechanism to exploit the maximum utilization of on-chip buffer. Experimental results show that Parana reduces the peak temperature by up to 54.72 $^\circ$ C and the steady temperature by up to 32.27 $^\circ$ C over state-of-the-art accelerators with 3D memory without performance degradation.

Development of genetic algorithm based hybrid neural network model for predicting the ultimate flexural strength of ferrocement elements

Abstract: This paper demonstrates the applicability of Hybrid Neural Networks that combine simple back propagation networks (BPN) and Genetic Algorithms (GAs) for predicting the ultimate flexural strength of ferrocement elements. A hybrid neural network model has been developed to predict the ultimate flexural strength of ferrocement elements with span to depth ratios (3, 6, 9 & 11.45), number of mesh layers (0, 1, 3 & 5) and percentage replacement of silicafume (0, 5, 10, 15, 20 & 25) as input parameters. The network has been trained with experimental data obtained from experimental work. The hybrid neural network model learned the relationship for predicting the ultimate flexural strength in just 300 training epochs. After successful learning, the model predicted the ultimate flexural strength satisfying all the constraints with an accuracy of 95%. The

Neural network applications in non-linear modelling of (bio)chemical processes

Abstract: In recent years, neural networks have attracted much attention for their potential to address a number of difficult problems in modelling and controlling nonlinear dynamic systems, especially in (bio) chemical engineering. The objective of this paper is to review some of the most widely used approaches to neural-network-based modelling, including plain black box as well as hybrid neural network — first principles modelling. Two specific application examples are used for illustration purposes: a simple tank level-control system is studied in simulation while a challenging bioprocess application is investigated based on experimental data. These applications allow some original concepts and techniques to be introduced.

Hybrid Neural Network mixed with Random Forests and Perlin noise

Abstract: Data mixed up with discrete and continuous features makes negative affect to the classification for existing models which discretizes the continuous features or even without any treatment is not able to deal with it. The noise which is not subject to Gaussian distribution also severely affected the result of classification. In this paper, a deep neural network mixed with random forests, Stacked Denoising Autoencoder and Multilayer Perception is proposed to improve the classification effect of mixed features input. Furthermore, to improve pre-training effect of the Stacked Denoising Autoencoder, Perlin noise is added to the Gaussian Noise in noise layer. A certain promotion of this method has been proved through the experiments with the crime data between 2003 and 2015 in San Francisco, and comparisons by 10-fold cross-validation with other common methods of classification shows that this method has advantages on data mixed up with discrete and continuous features.

A hybrid model of neural network and classification in wine

Abstract: After studying the disadvantage of BP neural network which has low convergent speed and trap into local minima easily, an idea of designing a new hybrid neural network model. By using Artificial Bee Colony Algorithm (ABC) to expand the updated space of weight and using the fitness functions to decide the better weight. On the basis, make the acquired better value as the weight of BP neural network. Both are coordination makes the optimized BP network not to trap into the local minima, and good generalization characteristic. Simulation analysis demonstrated that this network model can attain higher categories of precision.