Showing papers on "Hybrid neural network published in 2017"

A 1.06-to-5.09 TOPS/W reconfigurable hybrid-neural-network processor for deep learning applications

Abstract: An energy-efficient hybrid neural network (NN) processor is implemented in a 65nm technology. It has two 16×16 reconfigurable heterogeneous processing elements (PEs)arrays. To accelerate a hybrid-NN, the PE array is designed to support on demand partitioning and reconfiguration for parallel processing different NNs. To improve energy efficiency, each PE supports bit-width adaptive computing to meet variant bit-width of different neural layers. Measurement results show that this processor achieves a peak 409.6GOPS running at 200MHz and at most 5.09TOPS/W energy efficiency. This processor outperforms the state-of-the-art up to 5.2X in energy efficiency.

Volatility of main metals forecasted by a hybrid ANN-GARCH model with regressors

Abstract: A hybrid model is analyzed to predict the price volatility of gold, silver and copperThe hybrid model used is a ANN-GARCH model with regressors.APGARCH with exogenous variables is used as benchmark.The benchmark is better than the classical GARCH used in previous studies.The incorporation of ANN into the best Garch with regressors increases the accuracy. In this article, we analyze volatility forecasts associated with the price of gold, silver, and copper, three of the most important metals in the world market. First, a group of GARCH models are used to forecast volatility, including explanatory variables like the US Dollar-Euro and US Dollar-Yen exchange rates, the oil price, and the Chinese, Indian, British, and American stock market indexes. Subsequently, these model predictions are used as inputs for a neural network in order to analyze the increase in hybrid predictive power. The results obtained show that for these three metals, using the hybrid neural network model increases the forecasting power of out-of-sample volatility. In order to optimize the results, we conducted a series of sensitizations of the artificial neural network architecture and analyses for different cases, finding that the best models to forecast the price return volatility of these main metals are the ANN-GARCH model with regressors. Due to the heteroscedasticity in the financial series, the loss function used is Heteroskedasticity-adjusted Mean Squared Error (HMSE), and to test the superiority of the models, the Model Confidence Set is used.

A 3.43TOPS/W 48.9pJ/pixel 50.1nJ/classification 512 analog neuron sparse coding neural network with on-chip learning and classification in 40nm CMOS

Abstract: A digital-analog hybrid neural network exploits efficient analog computation and digital intra-network communication for feature extraction and classification. Taking advantage of the inherently low SNR requirements of the Locally Competitive Algorithm (LCA), the internally-analog neuron is 3x smaller and 7.5x more energy efficient than an equivalent digital design. This work demonstrates large-scale integration of 512 analog neurons using a traditional scalable digital workflow to achieve a best-of-class power efficiency of 3.43TOPS/W for object classification. At 48.9pJ/pixel and 50.1nJ/classification, the prototype 512-neuron IC achieves 2x efficiency over the digital design while maintaining reliable classification results over PVT.

A hybrid neural network-gray wolf optimization algorithm for melanoma detection

Abstract: Melanoma is one of the most dangerous tumors in the human kind cancers. Nonetheless, early detection of this cancer can help the doctors to cure it perfectly. In this paper, a new efficient method is proposed to detect the malignant melanoma images from the images. In the proposed method, a hybrid technique is utilized. We first eliminate the extra scales by using edge detection and smoothing. Afterwards, the main hybrid technique is applied to segment the cancer images. Finally by using the morphological operations, the extra information is eliminated and used to focus on the area which melanoma boundary potentially exists. Here, Gray Wolf Optimization algorithm is utilized to optimize an MLP neural Networks (ANN). Gray Wolf Optimization is a new evolutionary algorithm which recently introduced and has a good performance in some optimization problems. GWO is a derivative-free, Meta Heuristic algorithm, mimicking the ecological behaviour of colonizing weeds. Gray wolf optimization is a global search algorithm while gradient-based back propagation method is local search. In this proposed algorithm, Multi-Layer Perceptron Network (MLP) employs the problem's constraints and GWO algorithm tries to minimize the root mean square error. Experimental results show that the proposed method can develop the performance of the standard MLP algorithm significantly.

Bootstrapping Social Emotion Classification with Semantically Rich Hybrid Neural Networks

Abstract: Social emotion classification aims to predict the aggregation of emotional responses embedded in online comments contributed by various users. Such a task is inherently challenging because extracting relevant semantics from free texts is a classical research problem. Moreover, online comments are typically characterized by a sparse feature space, which makes the corresponding emotion classification task very difficult. On the other hand, though deep neural networks have been shown to be effective for speech recognition and image analysis tasks because of their capabilities of transforming sparse low-level features to dense high-level features, their effectiveness on emotion classification requires further investigation. The main contribution of our work reported in this paper is the development of a novel model of semantically rich hybrid neural network (HNN) which leverages unsupervised teaching models to incorporate semantic domain knowledge into the neural network to bootstrap its inference power and interpretability. To our best knowledge, this is the first successful work of incorporating semantics into neural networks to enhance social emotion classification and network interpretability. Through empirical studies based on three real-world social media datasets, our experimental results confirm that the proposed hybrid neural networks outperform other state-of-the-art emotion classification methods.

Weather forecasting based on hybrid neural model

Abstract: Making deductions and expectations about climate has been a challenge all through mankind’s history. Challenges with exact meteorological directions assist to foresee and handle problems well in time. Different strategies have been investigated using various machine learning techniques in reported forecasting systems. Current research investigates climate as a major challenge for machine information mining and deduction. Accordingly, this paper presents a hybrid neural model (MLP and RBF) to enhance the accuracy of weather forecasting. Proposed hybrid model ensure precise forecasting due to the specialty of climate anticipating frameworks. The study concentrates on the data representing Saudi Arabia weather forecasting. The main input features employed to train individual and hybrid neural networks that include average dew point, minimum temperature, maximum temperature, mean temperature, average relative moistness, precipitation, normal wind speed, high wind speed and average cloudiness. The output layer composed of two neurons to represent rainy and dry weathers. Moreover, trial and error approach is adopted to select an appropriate number of inputs to the hybrid neural network. Correlation coefficient, RMSE and scatter index are the standard yard sticks adopted for forecast accuracy measurement. On individual standing MLP forecasting results are better than RBF, however, the proposed simplified hybrid neural model comes out with better forecasting accuracy as compared to both individual networks. Additionally, results are better than reported in the state of art, using a simple neural structure that reduces training time and complexity.

Image based skin disease detection using hybrid neural network coupled bag-of-features

Abstract: The current work proposes a neural based detection method of two different skin diseases using skin imaging. Skin images of two diseases namely Basel Cell Carcinoma and Skin Angioma are utilized. SIFT feature extractor has been employed followed by a clustering phase on feature space in order to reduce the number of features suitable for neural based models. The extracted bag-of-features modified dataset is used to train metaheuristic supported hybrid Artificial Neural Networks to classify the skin images in order to detect the diseases under study. A well-known multi objective optimization technique called Non-dominated Sorting Genetic Algorithm — II is used to train the ANN (NN-NSGA-II). The proposed model is further compared with two other well-known metaheuristic based classifier namely NN-PSO (ANN trained with PSO) and NN-CS (ANN trained with Cuckoo Search) in terms of testing phase confusion matrix based performance measuring metrics such as accuracy, precision, recall and F-measure. Experimental results indicated towards the superiority of the proposed bag-of-features enabled NN-NSGA-II model.

Joint Extraction of Multiple Relations and Entities by Using a Hybrid Neural Network

Abstract: This paper proposes a novel end-to-end neural model to jointly extract entities and relations in a sentence. Unlike most existing approaches, the proposed model uses a hybrid neural network to automatically learn sentence features and does not rely on any Natural Language Processing (NLP) tools, such as dependency parser. Our model is further capable of modeling multiple relations and their corresponding entity pairs simultaneously. Experiments on the CoNLL04 dataset demonstrate that our model using only word embeddings as input features achieves state-of-the-art performance.

Software maintainability prediction using hybrid neural network and fuzzy logic approach with parallel computing concept

Abstract: In present day scenario, majority of software companies use object-oriented concept to develop software systems as it enables effective design, development, testing and maintenance, in addition to the optimal characterization of the software system. With the increase in number of these software systems, their effective maintenance aspect becomes very important day by day. In this study, Neuro-Fuzzy approach: hybrid neural network and fuzzy logic approach has been considered to develop a maintainability model using ten different object-oriented static source code metrics as input. This method is applied on maintainability data of two commercial software products such as UIMS and QUES. Rough set analysis (RSA) and principal component analysis (PCA) are used to select suitable set of metrics from the ten metrics employed to improve performance of maintainability prediction model. From experimental results, it is observed that Neuro-Fuzzy model can effectively predict the maintainability of object-oriented software systems. After implementing parallel computing concept, it is observed that the training time gets reduced to a significant amount when the number of computing nodes were increased. Further it is observed that selected subset of metrics using feature selection techniques i.e., PCA, and RSA was able to predict maintainability with higher accuracy.

A Hybrid Path Loss Prediction Model based on Artificial Neural Networks using Empirical Models for LTE And LTE-A at 800 MHz and 2600 MHz

Abstract: This article presents the development and analysis of a hybrid, error correction-based, neural network to predict the path loss for suburban areas at 800 MHz and 2600 MHz, obtained by combining empirical propagation models, ECC-33, Ericsson 9999, Okumura Hata, and 3GPP's TR 36.942, with a backpropagation Artificial Neural Network (ANN). The network performance was tested along with two optimization techniques - Genetic Algorithm (GA) and Least Mean Square (LMS). Results were compared with data obtained by measurements performed in the vicinity of the Federal University of Rio Grande do Norte (UFRN), in the city of Natal, Brazil. In the end, the hybrid neural network presented the best results, indicating greater similarity with experimental data. The results developed in this research will help to achieve better signal estimation, reducing errors in planning and implementation of LTE and LTE-A systems.

Non-invasive load identification algorithm based on hybrid neural network and ensemble learning

Abstract: The invention belongs to the data mining and machine learning field and relates to a non-invasive load identification algorithm based on a hybrid neural network and ensemble learning. According to the method, experimental data are processed, so that the format of the data conforms to the input formats of models; after the data are processed, a hybrid neural network model is established; the data are input into the model; the model is trained and tested, identification results are obtained; and voting is performed for the results of three different models based on the idea of ensemble learning, so that a final identification result is obtained. With the method adopted, the feature extraction effect and load identification effect of the hybrid neural network are better than the effects of a traditional neural network; an ensemble learning idea-based method is provided, a plurality of feature subsets are selected from a total feature set so as to train a plurality of base classifiers, and the base classifiers are combined, and therefore, variance can be decreased, and the identification effect of the final identification result can be improved, and the problem of adverse influence of the introduction of harmonic features on an identification effect can be solved.

A hybrid Neural Network Model for Joint Prediction of Presence and Period Assertions of Medical Events in Clinical Notes.

Abstract: In this paper, we propose a novel neural network architecture for clinical text mining. We formulate this hybrid neural network model (HNN), composed of recurrent neural network and deep residual network, to jointly predict the presence and period assertion values associated with medical events in clinical texts. We evaluate the effectiveness of our model on a corpus of expert-annotated longitudinal Electronic Health Records (EHR) notes from Cancer patients. Our experiments show that HNN improves the joint assertion classification accuracy as compared to conventional baselines.

Depth estimation from single monocular images using deep hybrid network

Abstract: Depth estimation is a significant task in the robotics vision. In this paper, we address the depth estimation from a single monocular image, which is a challenging problem in automated vision systems since a single image alone does not carry any additional measurements. To tackle our main objective, we design a deep hybrid neural network, which is composed of convolutional and recurrent layers (ReNet), where each ReNet layer is composed of the Long Short-Term Memory unit (LSTM), which is famous for the ability to memorize long-range context. In the proposed network, ReNet layers aim to enrich the features representation by directly capturing global context. The effective integration of ReNet and convolutional layers in the common CNN framework allows us to train the hybrid network in the end-to-end fashion. Experimental evaluation on the benchmarks dataset demonstrated, that hybrid network achieves the state-of-the-art results without any post-processing steps. Moreover, the composition of recurrent and convolutional layers provide more satisfying results.

Hybrid Neural Network Alignment and Lexicon Model in Direct HMM for Statistical Machine Translation

Abstract: Recently, the neural machine translation systems showed their promising performance and surpassed the phrase-based systems for most translation tasks. Retreating into conventional concepts machine translation while utilizing effective neural models is vital for comprehending the leap accomplished by neural machine translation over phrase-based methods. This work proposes a direct HMM with neural network-based lexicon and alignment models, which are trained jointly using the Baum-Welch algorithm. The direct HMM is applied to rerank the n-best list created by a state-of-the-art phrase-based translation system and it provides improvements by up to 1.0% Bleu scores on two different translation tasks.

Reliability assessment of component based software systems using fuzzy and ANFIS techniques

Abstract: Software reliability is one of the most commonly discussed research issue in the field of software engineering. In this paper we evaluate the reliability of component based software using Adaptive Neuro-Fuzzy inference system. The proposed model considers the factors particular to component based software that affects its reliability. The hybrid neural network used in ANFIS is trained using the data set obtained from a survey. This neural network in turn guides the rule base of the fuzzy inference system. Our ANFIS model is validated against the data obtained from survey of various existing component based software designs. An evaluation model based on Mamdani fuzzy inference system is also proposed. The performance analysis of ANFIS model is done by comparing its accuracy in determining correct outputs with that of the FIS model. The ANFIS model is optimized to obtain evaluation near to the empirical results. Experimental results show that ANFIS based evaluation model performs better than the corresponding FIS model.

Estimation of minimum horizontal stress, geomechanical modeling and hybrid neural network based on conventional well logging data – a case study

Abstract: The minimum horizontal stress (Shmin) is one of the three principal stresses and is required for evaluation of the hydraulic fracturing, sand production, and well stability. Shmin is obtained using direct methods such as the leak-off and mini-frac tests or using some equations like the poroelastic equation. These equations require some information including the elastic parameters, shear sonic logs, core data and the pore pressure. In this study, a geomechanical model is constructed to obtain the minimum horizontal stress; then, an artificial neural network (ANN) with multilayer perceptron and feedforward backpropagation algorithm based on the conventional well logging data is applied to predict the Shmin. Cuckoo optimization algorithm (COA), imperialist competitive algorithm, particle swarm optimization and genetic algorithm are also utilized to optimize the ANN. The proposed methodology is applied in two wells in the reservoir rock located at the southwest of Iran, one for training, and the other...

Time series forecasting based on wavelet decomposition and feature extraction

Abstract: Time series forecasting is one of the most important issues in numerous applications in real life. The objective of this study was to propose a hybrid neural network model based on wavelet transform (WT) and feature extraction for time series forecasting. The motivation of the proposed model, which is called PCA-WCCNN, is to establish a single simplified model with shorter training time and satisfactory forecasting performance. This model combines the principal component analysis (PCA) and WT with artificial neural networks (ANNs). Given a forecasting sequence, order of the original forecasting model is determined firstly. Secondly, the original time series is decomposed into approximation and detail components by employing WT technique. Then, instead of using all the components as inputs, feature inputs are extracted from all the sub-series obtained from the above step. Finally, based on the extracted features and all the sub-series, a famous neural network construction method called cascade-correlation algorithm is applied to train neural network model to learn the dynamics. As an illustration, the proposed model is compared with two classical models and two hybrid models, respectively. They are the traditional cascade-correlation neural network, back-propagation neural network, wavelet-based cascade-correlation network using all the wavelet components as inputs to establish one model (WCCNN) and wavelet-based cascade-correlation network with combination of each sub-series model (WCCNN multi-models). Results obtained from this study indicate that the proposed method improves the accuracy of ANN and can yield better efficiency than other four neural network models.

Hybrid neural network‐based prediction model for tribological properties of polyamide6‐based friction materials

Abstract: The hybrid neural network employed to predict the tribological properties of solid lubricants reinforced nano-TiO2/polyamide6 composites was established based on the back propagation and radial basis function networks, and optimized by adaptive genetic algorithm. With such three factors as material composition, testing loads, and velocities, orthogonal tests were designed and the data obtained was used for training the neural network. The correlation index between the predicted and the experimental values for friction coefficient and ware rate were 0.992 and 0.998, respectively, and 3D plots for the predicted friction coefficient and wear rate as a function of material compositions and testing conditions were established. It shows that the results are in good agreement with the data measured. It is demonstrated that the well-optimized neural network had remarkable capability for modeling concern. POLYM. COMPOS., 2015. © 2015 Society of Plastics Engineers

Automatic leaf segmentation using grey wolf optimizer based neural network

Abstract: This study proposes a hybrid neural network model for the segmentation of leaf images with various illumination conditions. Segmentation of images with different illumination conditions is a quite challenging process. In particular, the shadows and dark regions in the image can be quite misleading for traditional segmentation algorithms. Using a single feature or reviewing them in a single colour space may work for some images, but this approach does not work on the entire dataset that have different colour. For this reason, automatic segmentation method is proposed in this study by using components from four different colour spaces. Firstly, the image is converted into RGB, HSV, XYZ and YIQ channels. Then, B, S, Z and I components are used to train hybrid neural network. Grey wolf optimizer is used for neural network optimization. The segmentation results of proposed method are compared with the well-known segmentation algorithms and are more successful. The results of proposed method are that sensitivity is 99.66 %, specificity is 98.42 % and accuracy is 99.31 %.

A hybrid neural network hidden Markov model approach for automatic story segmentation

Abstract: We propose a hybrid neural network hidden Markov model (NN-HMM) approach for automatic story segmentation. A story is treated as an instance of an underlying topic (a hidden state) and words are generated from the distribution of the topic. The transition from one topic to another indicates a story boundary. Different from the traditional HMM approach, in which the emission probability of each state is calculated from a topic-dependent language model, we use deep neural network (DNN) to directly map the word distribution into topic posterior probabilities. DNN is known to be able to learn meaningful continuous features for words and hence has better discriminative and generalization capability than n-gram models. Specifically, we investigate three neural network structures: a feed-forward neural network, a recurrent neural network with long short-term memory cells (LSTM-RNN) and a modified LSTM-RNN with multi-task learning ability. Experimental results on the TDT2 corpus show that the proposed NN-HMM approach outperforms the traditional HMM approach significantly and achieves state-of-the-art performance in story segmentation.

Construction method of hybrid neural network model for dialogue generation

Abstract: The invention discloses a construction method of hybrid neural network model for dialogue generation. The construction method of hybrid neural network model for dialogue generation includes the steps: acquiring a data set in a mode of dialogue statement pairs, and constructing a glossary; generating a word embedded table; initializing the convolution neural network with special structure, generating a vocabulary recommending table corresponding to the input statement, determining whether real output is provided, and if so, training the parameters of the convolution neural network in the step; initializing the recurrent neural network with special structure, using the last step to output, generating a vocabulary identity list with word order, determining whether real output is provided, and if so, training the parameters of the recurrent neural network in the step; after the training result satisfies the set index, saving the glossary and the word embedded table, and saving the parameters of the convolution neural network and the recurrent neural network, thus completing construction of the whole model. The construction method of hybrid neural network model for dialogue generation solves the problems that a current neural network dialogue model is slow in the training speed, low in the accuracy and general in statement generation because the glossary is too long.

A geographically distributed bio-hybrid neural network with memristive plasticity.

Abstract: Throughout evolution the brain has mastered the art of processing real-world inputs through networks of interlinked spiking neurons. Synapses have emerged as key elements that, owing to their plasticity, are merging neuron-to-neuron signalling with memory storage and computation. Electronics has made important steps in emulating neurons through neuromorphic circuits and synapses with nanoscale memristors, yet novel applications that interlink them in heterogeneous bio-inspired and bio-hybrid architectures are just beginning to materialise. The use of memristive technologies in brain-inspired architectures for computing or for sensing spiking activity of biological neurons8 are only recent examples, however interlinking brain and electronic neurons through plasticity-driven synaptic elements has remained so far in the realm of the imagination. Here, we demonstrate a bio-hybrid neural network (bNN) where memristors work as "synaptors" between rat neural circuits and VLSI neurons. The two fundamental synaptors, from artificial-to-biological (ABsyn) and from biological-to- artificial (BAsyn), are interconnected over the Internet. The bNN extends across Europe, collapsing spatial boundaries existing in natural brain networks and laying the foundations of a new geographically distributed and evolving architecture: the Internet of Neuro-electronics (IoN).

Diagnosis System Of Internal Combustion Engine Development

Abstract: The design of an automated system for the diagnosis of internal combustion engines during its operation is described. A software module for engine diagnosing based on algorithms for information processing from sensor signals and the executive mechanisms of the control system was developed and implemented by external speed characteristic creation. The algorithms are developed on the capabilities of fuzzy inference systems for dependency identification. The method of fuzzy systems of Sugeno type was used for a chosen hybrid neural network. The possibility of complex dependences of engine parameter approximation is shown by hybrid neural networks. The adequacy of the developed diagnostic system was assessed by testing the serviceable ICE. Before the tests the ICE was prepared, which consisted in its adjusting parameter normalization. After the conduct of a series of engine tests, five engine parameters were selected, on the basis of which they evaluated the adequacy of the developed diagnostic system (spark angle, effective power, air flow, torque, fuel consumption). Еру statistical analysis of the data was carried out, which revealed greater accuracy and repeatability of engine values during testing. The values of the relative errors were calculated during the determination of external velocity characteristics.

A Hybrid Network Traffic Prediction Model Based on Optimized Neural Network

Abstract: With growth of networks, it’s demanding to predict the development of network traffic. In this paper, we analyze the network traffic based on the hybrid neural network model. The chaotic property of traffic data is verified by analyzing the chaos characteristics of the data. Based on the study of artificial neural network, wavelet transform theory and quantum genetic algorithm, we propose a neural network optimization method based on efficient global search capability of quantum genetic algorithm. The proposed quantum genetic artificial neural network model can predict the network traffic more accurately. The prediction results can be used to monitor the network anomaly in network security field, and improve the quality of service. The results will also benefit to search efficient network optimization solutions by predicting network behavior.

Optimizing signal decomposition techniques in artificial neural network-based rainfall-runoff model

Abstract: The proper function of artificial neural networks (ANNs) depends on several factors including the suitability of input variables and the amount of information they can add to the model in order to produce the required target output(s). Wavelet transforms and to lesser extent singular spectrum analysis (SSA) are well known and widely applied pre-processing methods to enhance ANN models. An important step in the SSA algorithm and wavelet transform method is choosing the window length (L) and determining the suitable number of decomposition stages, respectively. In most past research, these parameters have been used as granted. Moreover, a research to show the impact of using a combination of wavelet and SSA is absent. This study addresses an approach to optimize window length for SSA and number of decomposition stages for wavelet transform applied in a rainfall-runoff model. Moreover, a hybrid neural network is developed to take the advantage of wavelet and SSA-based ANN models. The results show a s...

Intuitionistic fuzzy radial basis functions network for modeling of nonlinear dynamics

Abstract: This paper deals with a design methodology for a neural network with improved robust qualities in notion to handling uncertain input data space variations. The proposed network topology combines the simplicity of the radial basis functions networks to interpret or classify data pairs and the abilities of the intuitionistic fuzzy logic to deal with the vagueness of the data space. A simplified gradient optimization procedure as a learning approach for the designed hybrid neural network is proposed. To investigate the effects of the generated structure throughout varying network parameters, the modeling of a two benchmark chaotic time series — Mackey-Glass and Rossler under uncertain conditions is investigated. The obtained results prove the flexibility of the approach and its potentials to cope with data variations.