Showing papers in "IEEE Transactions on Neural Networks in 2018"
TL;DR: Compared with existing deep neural networks, experimental results on the Modified National Institute of Standards and Technology database and NYU NORB object recognition dataset benchmark data demonstrate the effectiveness of the proposed Broad Learning System.
Abstract: Broad Learning System (BLS) that aims to offer an alternative way of learning in deep structure is proposed in this paper. Deep structure and learning suffer from a time-consuming training process because of a large number of connecting parameters in filters and layers. Moreover, it encounters a complete retraining process if the structure is not sufficient to model the system. The BLS is established in the form of a flat network, where the original inputs are transferred and placed as “mapped features” in feature nodes and the structure is expanded in wide sense in the “enhancement nodes.” The incremental learning algorithms are developed for fast remodeling in broad expansion without a retraining process if the network deems to be expanded. Two incremental learning algorithms are given for both the increment of the feature nodes (or filters in deep structure) and the increment of the enhancement nodes. The designed model and algorithms are very versatile for selecting a model rapidly. In addition, another incremental learning is developed for a system that has been modeled encounters a new incoming input. Specifically, the system can be remodeled in an incremental way without the entire retraining from the beginning. Satisfactory result for model reduction using singular value decomposition is conducted to simplify the final structure. Compared with existing deep neural networks, experimental results on the Modified National Institute of Standards and Technology database and NYU NORB object recognition dataset benchmark data demonstrate the effectiveness of the proposed BLS.
1,061 citations
TL;DR: An improvement version of kTree method is proposed, which enables to conduct kNN classification using a subset of the training samples in the leaf nodes rather than all training samples used in the newly kNN methods.
Abstract: ${k}$ nearest neighbor (kNN) method is a popular classification method in data mining and statistics because of its simple implementation and significant classification performance. However, it is impractical for traditional kNN methods to assign a fixed ${k}$ value (even though set by experts) to all test samples. Previous solutions assign different $k$ values to different test samples by the cross validation method but are usually time-consuming. This paper proposes a kTree method to learn different optimal $k$ values for different test/new samples, by involving a training stage in the kNN classification. Specifically, in the training stage, kTree method first learns optimal $k$ values for all training samples by a new sparse reconstruction model, and then constructs a decision tree (namely, kTree) using training samples and the learned optimal $k$ values. In the test stage, the kTree fast outputs the optimal $k$ value for each test sample, and then, the kNN classification can be conducted using the learned optimal $k$ value and all training samples. As a result, the proposed kTree method has a similar running cost but higher classification accuracy, compared with traditional kNN methods, which assign a fixed ${k}$ value to all test samples. Moreover, the proposed kTree method needs less running cost but achieves similar classification accuracy, compared with the newly kNN methods, which assign different ${k}$ values to different test samples. This paper further proposes an improvement version of kTree method (namely, k*Tree method) to speed its test stage by extra storing the information of the training samples in the leaf nodes of kTree, such as the training samples located in the leaf nodes, their kNNs, and the nearest neighbor of these kNNs. We call the resulting decision tree as k*Tree, which enables to conduct kNN classification using a subset of the training samples in the leaf nodes rather than all training samples used in the newly kNN methods. This actually reduces running cost of test stage. Finally, the experimental results on 20 real data sets showed that our proposed methods (i.e., kTree and k*Tree) are much more efficient than the compared methods in terms of classification tasks.
765 citations
TL;DR: This paper provides a comprehensive survey on the application of DL, RL, and deep RL techniques in mining biological data and compares the performances of DL techniques when applied to different data sets across various application domains.
Abstract: Rapid advances in hardware-based technologies during the past decades have opened up new possibilities for life scientists to gather multimodal data in various application domains, such as omics , bioimaging , medical imaging , and (brain/body)–machine interfaces . These have generated novel opportunities for development of dedicated data-intensive machine learning techniques. In particular, recent research in deep learning (DL), reinforcement learning (RL), and their combination (deep RL) promise to revolutionize the future of artificial intelligence. The growth in computational power accompanied by faster and increased data storage, and declining computing costs have already allowed scientists in various fields to apply these techniques on data sets that were previously intractable owing to their size and complexity. This paper provides a comprehensive survey on the application of DL, RL, and deep RL techniques in mining biological data. In addition, we compare the performances of DL techniques when applied to different data sets across various application domains. Finally, we outline open issues in this challenging research area and discuss future development perspectives.
622 citations
TL;DR: Q-learning and the integral RL algorithm as core algorithms for discrete time (DT) and continuous-time (CT) systems, respectively are discussed, and a new direction of off-policy RL for both CT and DT systems is discussed.
Abstract: This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal $\mathcal {H}_{2}$ and $\mathcal {H}_\infty $ control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.
536 citations
TL;DR: In this article, a cost sensitive deep neural network (CoSen) is proposed to learn robust feature representations for both the majority and minority classes, which jointly optimizes the class-dependent costs and the neural network parameters.
Abstract: Class imbalance is a common problem in the case of real-world object detection and classification tasks. Data of some classes are abundant, making them an overrepresented majority, and data of other classes are scarce, making them an underrepresented minority. This imbalance makes it challenging for a classifier to appropriately learn the discriminating boundaries of the majority and minority classes. In this paper, we propose a cost-sensitive (CoSen) deep neural network, which can automatically learn robust feature representations for both the majority and minority classes. During training, our learning procedure jointly optimizes the class-dependent costs and the neural network parameters. The proposed approach is applicable to both binary and multiclass problems without any modification. Moreover, as opposed to data-level approaches, we do not alter the original data distribution, which results in a lower computational cost during the training process. We report the results of our experiments on six major image classification data sets and show that the proposed approach significantly outperforms the baseline algorithms. Comparisons with popular data sampling techniques and CoSen classifiers demonstrate the superior performance of our proposed method.
524 citations
TL;DR: With the proposed control, the stability of the closed-loop system is achieved via Lyapunov’s stability theory, and the tracking performance is guaranteed under the condition of state constraints and uncertainty.
Abstract: This paper investigates adaptive fuzzy neural network (NN) control using impedance learning for a constrained robot, subject to unknown system dynamics, the effect of state constraints, and the uncertain compliant environment with which the robot comes into contact. A fuzzy NN learning algorithm is developed to identify the uncertain plant model. The prominent feature of the fuzzy NN is that there is no need to get the prior knowledge about the uncertainty and a sufficient amount of observed data. Also, impedance learning is introduced to tackle the interaction between the robot and its environment, so that the robot follows a desired destination generated by impedance learning. A barrier Lyapunov function is used to address the effect of state constraints. With the proposed control, the stability of the closed-loop system is achieved via Lyapunov’s stability theory, and the tracking performance is guaranteed under the condition of state constraints and uncertainty. Some simulation studies are carried out to illustrate the effectiveness of the proposed scheme.
498 citations
TL;DR: Zhang et al. as mentioned in this paper proposed a coattention mechanism using a deep neural network (DNN) architecture to jointly learn the attentions for both the image and the question, which can reduce the irrelevant features effectively and obtain more discriminative features for image and question representations.
Abstract: Visual question answering (VQA) is challenging, because it requires a simultaneous understanding of both visual content of images and textual content of questions. To support the VQA task, we need to find good solutions for the following three issues: 1) fine-grained feature representations for both the image and the question; 2) multimodal feature fusion that is able to capture the complex interactions between multimodal features; and 3) automatic answer prediction that is able to consider the complex correlations between multiple diverse answers for the same question. For fine-grained image and question representations, a “coattention” mechanism is developed using a deep neural network (DNN) architecture to jointly learn the attentions for both the image and the question, which can allow us to reduce the irrelevant features effectively and obtain more discriminative features for image and question representations. For multimodal feature fusion, a generalized multimodal factorized high-order pooling approach (MFH) is developed to achieve more effective fusion of multimodal features by exploiting their correlations sufficiently, which can further result in superior VQA performance as compared with the state-of-the-art approaches. For answer prediction, the Kullback–Leibler divergence is used as the loss function to achieve precise characterization of the complex correlations between multiple diverse answers with the same or similar meaning, which can allow us to achieve faster convergence rate and obtain slightly better accuracy on answer prediction. A DNN architecture is designed to integrate all these aforementioned modules into a unified model for achieving superior VQA performance. With an ensemble of our MFH models, we achieve the state-of-the-art performance on the large-scale VQA data sets and win the runner-up in VQA Challenge 2017.
437 citations
TL;DR: This framework outperforms the best classification method in the literature on the BCI competition IV-2a 4-class MI data set by 7% increase in average subject accuracy and by studying the convolutional weights of the trained networks, it gains an insight into the temporal characteristics of EEG.
Abstract: Deep learning (DL) methods and architectures have been the state-of-the-art classification algorithms for computer vision and natural language processing problems. However, the successful application of these methods in motor imagery (MI) brain–computer interfaces (BCIs), in order to boost classification performance, is still limited. In this paper, we propose a classification framework for MI data by introducing a new temporal representation of the data and also utilizing a convolutional neural network (CNN) architecture for classification. The new representation is generated from modifying the filter-bank common spatial patterns method, and the CNN is designed and optimized accordingly for the representation. Our framework outperforms the best classification method in the literature on the BCI competition IV-2a 4-class MI data set by 7% increase in average subject accuracy. Furthermore, by studying the convolutional weights of the trained networks, we gain an insight into the temporal characteristics of EEG.
399 citations
TL;DR: An unsupervised deep convolutional coupling network for change detection based on two heterogeneous images acquired by optical sensors and radars on different dates that demonstrates the promising performance of the proposed network compared with several existing approaches.
Abstract: We propose an unsupervised deep convolutional coupling network for change detection based on two heterogeneous images acquired by optical sensors and radars on different dates. Most existing change detection methods are based on homogeneous images. Due to the complementary properties of optical and radar sensors, there is an increasing interest in change detection based on heterogeneous images. The proposed network is symmetric with each side consisting of one convolutional layer and several coupling layers. The two input images connected with the two sides of the network, respectively, are transformed into a feature space where their feature representations become more consistent. In this feature space, the different map is calculated, which then leads to the ultimate detection map by applying a thresholding algorithm. The network parameters are learned by optimizing a coupling function. The learning process is unsupervised, which is different from most existing change detection methods based on heterogeneous images. Experimental results on both homogenous and heterogeneous images demonstrate the promising performance of the proposed network compared with several existing approaches.
354 citations
TL;DR: An iterative multiview agreement strategy by minimizing the divergence objective among all factorized latent data-cluster representations during each iteration of optimization process, where such latent representation from each view serves to regulate those from other views, and such an intuitive process iteratively coordinates all views to be agreeable.
Abstract: Multiview data clustering attracts more attention than their single-view counterparts due to the fact that leveraging multiple independent and complementary information from multiview feature spaces outperforms the single one. Multiview spectral clustering aims at yielding the data partition agreement over their local manifold structures by seeking eigenvalue–eigenvector decompositions. Among all the methods, low-rank representation (LRR) is effective, by exploring the multiview consensus structures beyond the low rankness to boost the clustering performance. However, as we observed, such classical paradigm still suffers from the following stand-out limitations for multiview spectral clustering of overlooking the flexible local manifold structure, caused by aggressively enforcing the low-rank data correlation agreement among all views, and such a strategy, therefore, cannot achieve the satisfied between-views agreement; worse still, LRR is not intuitively flexible to capture the latent data clustering structures. In this paper, first, we present the structured LRR by factorizing into the latent low-dimensional data-cluster representations, which characterize the data clustering structure for each view. Upon such representation, second, the Laplacian regularizer is imposed to be capable of preserving the flexible local manifold structure for each view. Third, we present an iterative multiview agreement strategy by minimizing the divergence objective among all factorized latent data-cluster representations during each iteration of optimization process, where such latent representation from each view serves to regulate those from other views, and such an intuitive process iteratively coordinates all views to be agreeable. Fourth, we remark that such data-cluster representation can flexibly encode the data clustering structure from any view with an adaptive input cluster number. To this end, finally, a novel nonconvex objective function is proposed via the efficient alternating minimization strategy. The complexity analysis is also presented. The extensive experiments conducted against the real-world multiview data sets demonstrate the superiority over the state of the arts.
337 citations
TL;DR: A deep HSI sharpening method is presented for the fusion of an LR-HSI with an HR-MSI, which directly learns the image priors via deep convolutional neural network-based residual learning.
Abstract: Hyperspectral image (HSI) sharpening, which aims at fusing an observable low spatial resolution (LR) HSI (LR-HSI) with a high spatial resolution (HR) multispectral image (HR-MSI) of the same scene to acquire an HR-HSI, has recently attracted much attention. Most of the recent HSI sharpening approaches are based on image priors modeling, which are usually sensitive to the parameters selection and time-consuming. This paper presents a deep HSI sharpening method (named DHSIS) for the fusion of an LR-HSI with an HR-MSI, which directly learns the image priors via deep convolutional neural network-based residual learning. The DHSIS method incorporates the learned deep priors into the LR-HSI and HR-MSI fusion framework. Specifically, we first initialize the HR-HSI from the fusion framework via solving a Sylvester equation. Then, we map the initialized HR-HSI to the reference HR-HSI via deep residual learning to learn the image priors. Finally, the learned image priors are returned to the fusion framework to reconstruct the final HR-HSI. Experimental results demonstrate the superiority of the DHSIS approach over existing state-of-the-art HSI sharpening approaches in terms of reconstruction accuracy and running time.
TL;DR: A new no-reference (NR) IQA model is developed and a robust image enhancement framework is established based on quality optimization, which can well enhance natural images, low-contrast images,Low-light images, and dehazed images.
Abstract: In this paper, we investigate into the problem of image quality assessment (IQA) and enhancement via machine learning. This issue has long attracted a wide range of attention in computational intelligence and image processing communities, since, for many practical applications, e.g., object detection and recognition, raw images are usually needed to be appropriately enhanced to raise the visual quality (e.g., visibility and contrast). In fact, proper enhancement can noticeably improve the quality of input images, even better than originally captured images, which are generally thought to be of the best quality. In this paper, we present two most important contributions. The first contribution is to develop a new no-reference (NR) IQA model. Given an image, our quality measure first extracts 17 features through analysis of contrast, sharpness, brightness and more, and then yields a measure of visual quality using a regression module, which is learned with big-data training samples that are much bigger than the size of relevant image data sets. The results of experiments on nine data sets validate the superiority and efficiency of our blind metric compared with typical state-of-the-art full-reference, reduced-reference and NA IQA methods. The second contribution is that a robust image enhancement framework is established based on quality optimization. For an input image, by the guidance of the proposed NR-IQA measure, we conduct histogram modification to successively rectify image brightness and contrast to a proper level. Thorough tests demonstrate that our framework can well enhance natural images, low-contrast images, low-light images, and dehazed images. The source code will be released at https://sites.google.com/site/guke198701/publications .
TL;DR: A formalization of the fraud-detection problem is proposed that realistically describes the operating conditions of FDSs that everyday analyze massive streams of credit card transactions and a novel learning strategy is designed and assessed that effectively addresses class imbalance, concept drift, and verification latency.
Abstract: Detecting frauds in credit card transactions is perhaps one of the best testbeds for computational intelligence algorithms. In fact, this problem involves a number of relevant challenges, namely: concept drift (customers’ habits evolve and fraudsters change their strategies over time), class imbalance (genuine transactions far outnumber frauds), and verification latency (only a small set of transactions are timely checked by investigators). However, the vast majority of learning algorithms that have been proposed for fraud detection rely on assumptions that hardly hold in a real-world fraud-detection system (FDS). This lack of realism concerns two main aspects: 1) the way and timing with which supervised information is provided and 2) the measures used to assess fraud-detection performance. This paper has three major contributions. First, we propose, with the help of our industrial partner, a formalization of the fraud-detection problem that realistically describes the operating conditions of FDSs that everyday analyze massive streams of credit card transactions. We also illustrate the most appropriate performance measures to be used for fraud-detection purposes. Second, we design and assess a novel learning strategy that effectively addresses class imbalance, concept drift, and verification latency. Third, in our experiments, we demonstrate the impact of class unbalance and concept drift in a real-world data stream containing more than 75 million transactions, authorized over a time window of three years.
TL;DR: In this article, the output reachable estimation and safety verification problems for multilayer perceptron (MLP) neural networks are addressed, and an automated safety verification is developed based on the output reachedable set estimation result.
Abstract: In this brief, the output reachable estimation and safety verification problems for multilayer perceptron (MLP) neural networks are addressed. First, a conception called maximum sensitivity is introduced, and for a class of MLPs whose activation functions are monotonic functions, the maximum sensitivity can be computed via solving convex optimization problems. Then, using a simulation-based method, the output reachable set estimation problem for neural networks is formulated into a chain of optimization problems. Finally, an automated safety verification is developed based on the output reachable set estimation result. An application to the safety verification for a robotic arm model with two joints is presented to show the effectiveness of the proposed approaches.
TL;DR: In this paper, the authors considered the consensus problem of hybrid multiagent systems and proposed three kinds of consensus protocols for the hybrid multi-agent system based on matrix theory and graph theory.
Abstract: In this brief, we consider the consensus problem of hybrid multiagent systems. First, the hybrid multiagent system is proposed, which is composed of continuous-time and discrete-time dynamic agents. Then, three kinds of consensus protocols are presented for the hybrid multiagent system. The analysis tool developed in this brief is based on the matrix theory and graph theory. With different restrictions of the sampling period, some necessary and sufficient conditions are established for solving the consensus of the hybrid multiagent system. The consensus states are also obtained under different protocols. Finally, simulation examples are provided to demonstrate the effectiveness of our theoretical results.
TL;DR: This work shows that in a feedforward spiking network that uses a temporal coding scheme where information is encoded in spike times instead of spike rates, the network input–output relation is differentiable almost everywhere and this relation is piecewise linear after a transformation of variables.
Abstract: Gradient descent training techniques are remarkably successful in training analog-valued artificial neural networks (ANNs). Such training techniques, however, do not transfer easily to spiking networks due to the spike generation hard nonlinearity and the discrete nature of spike communication. We show that in a feedforward spiking network that uses a temporal coding scheme where information is encoded in spike times instead of spike rates, the network input–output relation is differentiable almost everywhere. Moreover, this relation is piecewise linear after a transformation of variables. Methods for training ANNs thus carry directly to the training of such spiking networks as we show when training on the permutation invariant MNIST task. In contrast to rate-based spiking networks that are often used to approximate the behavior of ANNs, the networks we present spike much more sparsely and their behavior cannot be directly approximated by conventional ANNs. Our results highlight a new approach for controlling the behavior of spiking networks with realistic temporal dynamics, opening up the potential for using these networks to process spike patterns with complex temporal information.
TL;DR: The results ascertain that the proposed encryption algorithm based on the piecewise linear chaotic map and the chaotic inertial neural network is efficient and reliable for secure communication applications.
Abstract: In this paper, synchronization of an inertial neural network with time-varying delays is investigated. Based on the variable transformation method, we transform the second-order differential equations into the first-order differential equations. Then, using suitable Lyapunov–Krasovskii functionals and Jensen’s inequality, the synchronization criteria are established in terms of linear matrix inequalities. Moreover, a feedback controller is designed to attain synchronization between the master and slave models, and to ensure that the error model is globally asymptotically stable. Numerical examples and simulations are presented to indicate the effectiveness of the proposed method. Besides that, an image encryption algorithm is proposed based on the piecewise linear chaotic map and the chaotic inertial neural network. The chaotic signals obtained from the inertial neural network are utilized for the encryption process. Statistical analyses are provided to evaluate the effectiveness of the proposed encryption algorithm. The results ascertain that the proposed encryption algorithm is efficient and reliable for secure communication applications.
TL;DR: This paper investigates adaptive neural control methods for robotic manipulators, subject to uncertain plant dynamics and constraints on the joint position, in which the barrier Lyapunov function is employed to guarantee that the joint constraints are not violated.
Abstract: This paper investigates adaptive neural control methods for robotic manipulators, subject to uncertain plant dynamics and constraints on the joint position. The barrier Lyapunov function is employed to guarantee that the joint constraints are not violated, in which the Moore–Penrose pseudo-inverse term is used in the control design. To handle the unmodeled dynamics, the neural network (NN) is adopted to approximate the uncertain dynamics. The NN control based on full-state feedback for robots is proposed when all states of the closed loop are known. Subsequently, only the robot joint is measurable in practice; output feedback control is designed with a high-gain observer to estimate unmeasurable states. Through the Lyapunov stability analysis, system stability is achieved with the proposed control, and the system output achieves convergence without violation of the joint constraints. Simulation is conducted to approve the feasibility and superiority of the proposed NN control.
TL;DR: A snapshot of the fast-growing deep learning field for microscopy image analysis, which explains the architectures and the principles of convolutional neural networks, fully Convolutional networks, recurrent neural Networks, stacked autoencoders, and deep belief networks and their formulations or modelings for specific tasks on various microscopy images.
Abstract: Computerized microscopy image analysis plays an important role in computer aided diagnosis and prognosis. Machine learning techniques have powered many aspects of medical investigation and clinical practice. Recently, deep learning is emerging as a leading machine learning tool in computer vision and has attracted considerable attention in biomedical image analysis. In this paper, we provide a snapshot of this fast-growing field, specifically for microscopy image analysis. We briefly introduce the popular deep neural networks and summarize current deep learning achievements in various tasks, such as detection, segmentation, and classification in microscopy image analysis. In particular, we explain the architectures and the principles of convolutional neural networks, fully convolutional networks, recurrent neural networks, stacked autoencoders, and deep belief networks, and interpret their formulations or modelings for specific tasks on various microscopy images. In addition, we discuss the open challenges and the potential trends of future research in microscopy image analysis using deep learning.
TL;DR: Ristretto is a CNN approximation framework that enables empirical investigation of the tradeoff between various number representation and word width choices and the classification accuracy of the model and is used to demonstrate that three ImageNet networks can be condensed to use 8-bit dynamic fixed point for network weights and activations.
Abstract: Convolutional neural networks (CNNs) have led to remarkable progress in a number of key pattern recognition tasks, such as visual scene understanding and speech recognition, that potentially enable numerous applications. Consequently, there is a significant need to deploy trained CNNs to resource-constrained embedded systems. Inference using pretrained modern deep CNNs, however, requires significant system resources, including computation, energy, and memory space. To enable efficient implementation of trained CNNs, a viable approach is to approximate the network with an implementation-friendly model with only negligible degradation in classification accuracy. We present Ristretto, a CNN approximation framework that enables empirical investigation of the tradeoff between various number representation and word width choices and the classification accuracy of the model. Specifically, Ristretto analyzes a given CNN with respect to numerical range required to represent weights, activations, and intermediate results of convolutional and fully connected layers, and subsequently, it simulates the impact of reduced word width or lower precision arithmetic operators on the model accuracy. Moreover, Ristretto can fine-tune a quantized network to further improve its classification accuracy under a given number representation and word width configuration. Given a maximum classification accuracy degradation tolerance of 1%, we use Ristretto to demonstrate that three ImageNet networks can be condensed to use 8-bit dynamic fixed point for network weights and activations. Ristretto is available as a popular open-source software project 1 and has already been viewed over 1 000 times on Github as of the submission of this brief. 1 https://github.com/pmgysel/caffe
TL;DR: In the proposed method, the feedback signals and the NN weights are aperiodically updated only when the event-triggered condition is violated, and the number of transmissions can be significantly reduced.
Abstract: This paper is concerned with the adaptive event-triggered control problem of nonlinear continuous-time systems in strict-feedback form. By using the event-sampled neural network (NN) to approximate the unknown nonlinear function, an adaptive model and an associated event-triggered controller are designed by exploiting the backstepping method. In the proposed method, the feedback signals and the NN weights are aperiodically updated only when the event-triggered condition is violated. A positive lower bound on the minimum intersample time is guaranteed to avoid accumulation point. The closed-loop stability of the resulting nonlinear impulsive dynamical system is rigorously proved via Lyapunov analysis under an adaptive event sampling condition. In comparing with the traditional adaptive backstepping design with a fixed sample period, the event-triggered method samples the state and updates the NN weights only when it is necessary. Therefore, the number of transmissions can be significantly reduced. Finally, two simulation examples are presented to show the effectiveness of the proposed control method.
TL;DR: This paper proposes a novel semisupervised NMF learning framework, called robust structured NMF, that learns a robust discriminative representation by leveraging the block-diagonal structure and the inline-formula-norm loss function, which addresses the problems of noise and outliers.
Abstract: Dimensionality reduction has attracted increasing attention, because high-dimensional data have arisen naturally in numerous domains in recent years. As one popular dimensionality reduction method, nonnegative matrix factorization (NMF), whose goal is to learn parts-based representations, has been widely studied and applied to various applications. In contrast to the previous approaches, this paper proposes a novel semisupervised NMF learning framework, called robust structured NMF, that learns a robust discriminative representation by leveraging the block-diagonal structure and the $\ell _{2,p}$ -norm (especially when $0 ) loss function. Specifically, the problems of noise and outliers are well addressed by the $\ell _{2,p}$ -norm ( $0 ) loss function, while the discriminative representations of both the labeled and unlabeled data are simultaneously learned by explicitly exploring the block-diagonal structure. The proposed problem is formulated as an optimization problem with a well-defined objective function solved by the proposed iterative algorithm. The convergence of the proposed optimization algorithm is analyzed both theoretically and empirically. In addition, we also discuss the relationships between the proposed method and some previous methods. Extensive experiments on both the synthetic and real-world data sets are conducted, and the experimental results demonstrate the effectiveness of the proposed method in comparison to the state-of-the-art methods.
TL;DR: The sampled-data feedback control, which is assumed to allow norm-bounded uncertainty and involves a constant signal transmission delay, is constructed for the first time in this paper, and a sufficient condition is developed, such that the nonfragile exponential stability of the error system is guaranteed.
Abstract: This paper considers nonfragile exponential synchronization for complex dynamical networks (CDNs) with time-varying coupling delay. The sampled-data feedback control, which is assumed to allow norm-bounded uncertainty and involves a constant signal transmission delay, is constructed for the first time in this paper. By constructing a suitable augmented Lyapunov function, and with the help of introduced integral inequalities and employing the convex combination technique, a sufficient condition is developed, such that the nonfragile exponential stability of the error system is guaranteed. As a result, for the case of sampled-data control free of norm-bound uncertainties, some sufficient conditions of sampled-data synchronization criteria for the CDNs with time-varying coupling delay are presented. As the formulations are in the framework of linear matrix inequality, these conditions can be easily solved and implemented. Two illustrative examples are presented to demonstrate the effectiveness and merits of the proposed feedback control.
TL;DR: By using the Lyapunov analysis, it is proven that all the signals of the closed-loop systems are semiglobally uniformly ultimately bounded.
Abstract: This paper studies the zero-error tracking control problem of Euler-Lagrange systems subject to full-state constraints and nonparametric uncertainties. By blending an error transformation with barrier Lyapunov function, a neural adaptive tracking control scheme is developed, resulting in a solution with several salient features: 1) the control action is continuous and $\mathscr C^{1}$ smooth; 2) the full-state tracking error converges to a prescribed compact set around origin within a given finite time at a controllable rate of convergence that can be uniformly prespecified; 3) with Nussbaum gain in the loop, the tracking error further shrinks to zero as $t\to \infty $ ; and 4) the neural network (NN) unit can be safely included in the loop during the entire system operational envelope without the danger of violating the compact set precondition imposed on the NN training inputs. Furthermore, by using the Lyapunov analysis, it is proven that all the signals of the closed-loop systems are semiglobally uniformly ultimately bounded. The effectiveness and benefits of the proposed control method are validated via computer simulation.
TL;DR: The control design problem of nonlinear interconnected switched systems with unknown switching signals can be solved by the proposed scheme, and only a tuning parameter is needed for each subsystem, and the effectiveness of the proposed control approach is verified.
Abstract: In this paper, an adaptive neural net- works (NNs)-based decentralized control scheme with the prescribed performance is proposed for uncertain switched nonstrict-feedback interconnected nonlinear systems. It is assumed that nonlinear interconnected terms and nonlinear functions of the concerned systems are unknown, and also the switching signals are unknown and arbitrary. A linear state estimator is constructed to solve the problem of unmeasured states. The NNs are employed to approximate unknown interconnected terms and nonlinear functions. A new output feedback decentralized control scheme is developed by using the adaptive backstepping design technique. The control design problem of nonlinear interconnected switched systems with unknown switching signals can be solved by the proposed scheme, and only a tuning parameter is needed for each subsystem. The proposed scheme can ensure that all variables of the control systems are semi-globally uniformly ultimately bounded and the tracking errors converge to a small residual set with the prescribed performance bound. The effectiveness of the proposed control approach is verified by some simulation results.
TL;DR: Some novel sufficient conditions are obtained to guarantee that the closed-loop system reaches a specified cost value under the designed jumping state feedback control law in terms of linear matrix inequalities.
Abstract: This paper is concerned with the guaranteed cost control problem for a class of Markov jump discrete-time neural networks (NNs) with event-triggered mechanism, asynchronous jumping, and fading channels. The Markov jump NNs are introduced to be close to reality, where the modes of the NNs and guaranteed cost controller are determined by two mutually independent Markov chains. The asynchronous phenomenon is considered, which increases the difficulty of designing required mode-dependent controller. The event-triggered mechanism is designed by comparing the relative measurement error with the last triggered state at the process of data transmission, which is used to eliminate dispensable transmission and reduce the networked energy consumption. In addition, the signal fading is considered for the effect of signal reflection and shadow in wireless networks, which is modeled by the novel Rice fading models. Some novel sufficient conditions are obtained to guarantee that the closed-loop system reaches a specified cost value under the designed jumping state feedback control law in terms of linear matrix inequalities. Finally, some simulation results are provided to illustrate the effectiveness of the proposed method.
TL;DR: A weighted kernel-based SMOTE (WK-SMOTE) that overcomes the limitation of SMOTE for nonlinear problems by oversampling in the feature space of support vector machine (SVM) classifier is proposed and shown to improve performance when compared to other baseline methods on multiple benchmark imbalanced data sets.
Abstract: Historical data sets for fault stage diagnosis in industrial machines are often imbalanced and consist of multiple categories or classes. Learning discriminative models from such data sets is challenging due to the lack of representative data and the bias of traditional classifiers toward the majority class. Sampling methods like synthetic minority oversampling technique (SMOTE) have been traditionally used for such problems to artificially balance the data set before being trained by a classifier. This paper proposes a weighted kernel-based SMOTE (WK-SMOTE) that overcomes the limitation of SMOTE for nonlinear problems by oversampling in the feature space of support vector machine (SVM) classifier. The proposed oversampling algorithm along with a cost-sensitive SVM formulation is shown to improve performance when compared to other baseline methods on multiple benchmark imbalanced data sets. In addition, a hierarchical framework is developed for multiclass imbalanced problems that have a progressive class order. The proposed WK-SMOTE and hierarchical framework are validated on a real-world industrial fault detection problem to identify deterioration in insulation of high-voltage equipments.
TL;DR: Comparisons of the approximation performance between radial basis function NN, RNN, and DLRNN show that theDLRNN can accurately estimate the unknown dynamics with a fast speed while the internal states of DLRnn are more stable.
Abstract: In this paper, an adaptive sliding mode control system using a double loop recurrent neural network (DLRNN) structure is proposed for a class of nonlinear dynamic systems. A new three-layer RNN is proposed to approximate unknown dynamics with two different kinds of feedback loops where the firing weights and output signal calculated in the last step are stored and used as the feedback signals in each feedback loop. Since the new structure has combined the advantages of internal feedback NN and external feedback NN, it can acquire the internal state information while the output signal is also captured, thus the new designed DLRNN can achieve better approximation performance compared with the regular NNs without feedback loops or the regular RNNs with a single feedback loop. The new proposed DLRNN structure is employed in an equivalent controller to approximate the unknown nonlinear system dynamics, and the parameters of the DLRNN are updated online by adaptive laws to get favorable approximation performance. To investigate the effectiveness of the proposed controller, the designed adaptive sliding mode controller with the DLRNN is applied to a $z$ -axis microelectromechanical system gyroscope to control the vibrating dynamics of the proof mass. Simulation results demonstrate that the proposed methodology can achieve good tracking property, and the comparisons of the approximation performance between radial basis function NN, RNN, and DLRNN show that the DLRNN can accurately estimate the unknown dynamics with a fast speed while the internal states of DLRNN are more stable.
TL;DR: A novel bounded real lemma (BRL) for the resultant error system is derived and is applied to present a method for designing suitable Luenberger estimators in terms of solutions of linear matrix inequalities with two tuning parameters.
Abstract: This brief is concerned with the problem of neural state estimation for static neural networks with time-varying delays. Notice that a Luenberger estimator can produce an estimation error irrespective of the neuron state trajectory. This brief provides a method for designing such an estimator for static neural networks with time-varying delays. First, in-depth analysis on a well-used reciprocally convex approach is made, leading to an improved reciprocally convex inequality. Second, the improved reciprocally convex inequality and some integral inequalities are employed to provide a tight upper bound on the time-derivative of some Lyapunov–Krasovskii functional. As a result, a novel bounded real lemma (BRL) for the resultant error system is derived. Third, the BRL is applied to present a method for designing suitable Luenberger estimators in terms of solutions of linear matrix inequalities with two tuning parameters. Finally, it is shown through a numerical example that the proposed method can derive less conservative results than some existing ones.
TL;DR: A novel distributed primal–dual dynamical multiagent system is designed in a smart grid scenario to seek the saddle point of dynamical economic dispatch, which coincides with the optimal solution.
Abstract: The resource allocation problem is studied and reformulated by a distributed interior point method via a $\theta$ - logarithmic barrier. By the facilitation of the graph Laplacian, a fully distributed continuous-time multiagent system is developed for solving the problem. Specifically, to avoid high singularity of the $\theta$ - logarithmic barrier at boundary, an adaptive parameter switching strategy is introduced into this dynamical multiagent system. The convergence rate of the distributed algorithm is obtained. Moreover, a novel distributed primal–dual dynamical multiagent system is designed in a smart grid scenario to seek the saddle point of dynamical economic dispatch, which coincides with the optimal solution. The dual decomposition technique is applied to transform the optimization problem into easily solvable resource allocation subproblems with local inequality constraints. The good performance of the new dynamical systems is, respectively, verified by a numerical example and the IEEE six-bus test system-based simulations.