Showing papers in "IEEE Transactions on Neural Networks in 2019"
TL;DR: In this article, a review of deep learning-based object detection frameworks is provided, focusing on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further.
Abstract: Due to object detection’s close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles that combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy, and optimization function. In this paper, we provide a review of deep learning-based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely, the convolutional neural network. Then, we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection, and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network-based learning systems.
3,097 citations
TL;DR: In this paper, the authors review recent findings on adversarial examples for DNNs, summarize the methods for generating adversarial samples, and propose a taxonomy of these methods.
Abstract: With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks (DNNs) have been recently found vulnerable to well-designed input samples called adversarial examples . Adversarial perturbations are imperceptible to human but can easily fool DNNs in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying DNNs in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for DNNs, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.
1,203 citations
TL;DR: Six learning algorithms including biogeography-based optimization, particle swarm optimization, genetic algorithm, ant colony optimization, evolutionary strategy, and population-based incremental learning are used to train a new dendritic neuron model (DNM) and are suggested to make DNM more powerful in solving classification, approximation, and prediction problems.
Abstract: An artificial neural network (ANN) that mimics the information processing mechanisms and procedures of neurons in human brains has achieved a great success in many fields, e.g., classification, prediction, and control. However, traditional ANNs suffer from many problems, such as the hard understanding problem, the slow and difficult training problems, and the difficulty to scale them up. These problems motivate us to develop a new dendritic neuron model (DNM) by considering the nonlinearity of synapses, not only for a better understanding of a biological neuronal system, but also for providing a more useful method for solving practical problems. To achieve its better performance for solving problems, six learning algorithms including biogeography-based optimization, particle swarm optimization, genetic algorithm, ant colony optimization, evolutionary strategy, and population-based incremental learning are for the first time used to train it. The best combination of its user-defined parameters has been systemically investigated by using the Taguchi’s experimental design method. The experiments on 14 different problems involving classification, approximation, and prediction are conducted by using a multilayer perceptron and the proposed DNM. The results suggest that the proposed learning algorithms are effective and promising for training DNM and thus make DNM more powerful in solving classification, approximation, and prediction problems.
517 citations
TL;DR: A 3-D deep neural network for automatic diagnosing lung cancer from computed tomography scans that selects the top five nodules based on the detection confidence, evaluates their cancer probabilities, and combines them with a leaky noisy-OR gate to obtain the probability of lung cancer for the subject.
Abstract: Automatic diagnosing lung cancer from computed tomography scans involves two steps: detect all suspicious lesions (pulmonary nodules) and evaluate the whole-lung/pulmonary malignancy. Currently, there are many studies about the first step, but few about the second step. Since the existence of nodule does not definitely indicate cancer, and the morphology of nodule has a complicated relationship with cancer, the diagnosis of lung cancer demands careful investigations on every suspicious nodule and integration of information of all nodules. We propose a 3-D deep neural network to solve this problem. The model consists of two modules. The first one is a 3-D region proposal network for nodule detection, which outputs all suspicious nodules for a subject. The second one selects the top five nodules based on the detection confidence, evaluates their cancer probabilities, and combines them with a leaky noisy-OR gate to obtain the probability of lung cancer for the subject. The two modules share the same backbone network, a modified U-net. The overfitting caused by the shortage of the training data is alleviated by training the two modules alternately. The proposed model won the first place in the Data Science Bowl 2017 competition.
378 citations
TL;DR: This paper develops several methods to represent modulated signals in data formats with gridlike topologies for the CNN and demonstrates the significant performance advantage and application feasibility of the DL-based approach for modulation classification.
Abstract: Deep learning (DL) is a new machine learning (ML) methodology that has found successful implementations in many application domains. However, its usage in communications systems has not been well explored. This paper investigates the use of the DL in modulation classification, which is a major task in many communications systems. The DL relies on a massive amount of data and, for research and applications, this can be easily available in communications systems. Furthermore, unlike the ML, the DL has the advantage of not requiring manual feature selections, which significantly reduces the task complexity in modulation classification. In this paper, we use two convolutional neural network (CNN)-based DL models, AlexNet and GoogLeNet. Specifically, we develop several methods to represent modulated signals in data formats with gridlike topologies for the CNN. The impacts of representation on classification performance are also analyzed. In addition, comparisons with traditional cumulant and ML-based algorithms are presented. Experimental results demonstrate the significant performance advantage and application feasibility of the DL-based approach for modulation classification.
355 citations
TL;DR: A mathematical proof of the universal approximation property of BLS is provided and the framework of several BLS variants with their mathematical modeling is given, which include cascade, recurrent, and broad–deep combination structures.
Abstract: After a very fast and efficient discriminative broad learning system (BLS) that takes advantage of flatted structure and incremental learning has been developed, here, a mathematical proof of the universal approximation property of BLS is provided In addition, the framework of several BLS variants with their mathematical modeling is given The variations include cascade, recurrent, and broad–deep combination structures From the experimental results, the BLS and its variations outperform several exist learning algorithms on regression performance over function approximation, time series prediction, and face recognition databases In addition, experiments on the extremely challenging data set, such as MS-Celeb-1M, are given Compared with other convolutional networks, the effectiveness and efficiency of the variants of BLS are demonstrated
327 citations
TL;DR: The adaptive backstepping control method and Lyapunov stability theory are used to prove the proposed controller can ensure all the signals in the systems are semiglobally uniformly ultimately bounded, and the output of the systems can track the reference signal closely.
Abstract: In this paper, the adaptive neural network (NN) tracking control problem is addressed for robot manipulators subject to dead-zone input. The control objective is to design an adaptive NN controller to guarantee the stability of the systems and obtain good performance. Different from the existing results, which used NN to approximate the nonlinearities directly, NNs are employed to identify the originally designed virtual control signals with unknown nonlinear items in this paper. Moreover, a sequence of virtual control signals and real controller are designed. The adaptive backstepping control method and Lyapunov stability theory are used to prove the proposed controller can ensure all the signals in the systems are semiglobally uniformly ultimately bounded, and the output of the systems can track the reference signal closely. Finally, the proposed adaptive control strategy is applied to the Puma 560 robot manipulator to demonstrate its effectiveness.
276 citations
TL;DR: A novel EEG-based spatial–temporal convolutional neural network (ESTCNN) to detect driver fatigue that could automatically learn valid features from EEG signals, which outperforms the classical two-step machine learning algorithms.
Abstract: Driver fatigue evaluation is of great importance for traffic safety and many intricate factors would exacerbate the difficulty. In this paper, based on the spatial–temporal structure of multichannel electroencephalogram (EEG) signals, we develop a novel EEG-based spatial–temporal convolutional neural network (ESTCNN) to detect driver fatigue. First, we introduce the core block to extract temporal dependencies from EEG signals. Then, we employ dense layers to fuse spatial features and realize classification. The developed network could automatically learn valid features from EEG signals, which outperforms the classical two-step machine learning algorithms. Importantly, we carry out fatigue driving experiments to collect EEG signals from eight subjects being alert and fatigue states. Using 2800 samples under within-subject splitting, we compare the effectiveness of ESTCNN with eight competitive methods. The results indicate that ESTCNN fulfills a better classification accuracy of 97.37% than these compared methods. Furthermore, the spatial–temporal structure of this framework advantages in computational efficiency and reference time, which allows further implementations in the brain–computer interface online systems.
268 citations
TL;DR: This paper introduces a technique, inversion, to project data samples, specifically images, to the latent space using a pretrained GAN, and demonstrates how the proposed inversion technique may be used to quantitatively compare the performance of various GAN models trained on three image data sets.
Abstract: Generative adversarial networks (GANs) learn a deep generative model that is able to synthesize novel, high-dimensional data samples. New data samples are synthesized by passing latent samples, drawn from a chosen prior distribution, through the generative model. Once trained, the latent space exhibits interesting properties that may be useful for downstream tasks such as classification or retrieval. Unfortunately, GANs do not offer an “inverse model,” a mapping from data space back to latent space, making it difficult to infer a latent representation for a given data sample. In this paper, we introduce a technique, inversion , to project data samples, specifically images, to the latent space using a pretrained GAN. Using our proposed inversion technique, we are able to identify which attributes of a data set a trained GAN is able to model and quantify GAN performance, based on a reconstruction loss. We demonstrate how our proposed inversion technique may be used to quantitatively compare the performance of various GAN models trained on three image data sets. We provide codes for all of our experiments in the website ( https://github.com/ToniCreswell/InvertingGAN ).
260 citations
TL;DR: A novel low tensor-train (TT) rank (LTTR)-based HSI super-resolution method is proposed, where an LTTR prior is designed to learn the correlations among the spatial, spectral, and nonlocal modes of the nonlocal similar high-spatial-resolution HSI (HR-HSI) cubes.
Abstract: Hyperspectral images (HSIs) with high spectral resolution only have the low spatial resolution. On the contrary, multispectral images (MSIs) with much lower spectral resolution can be obtained with higher spatial resolution. Therefore, fusing the high-spatial-resolution MSI (HR-MSI) with low-spatial-resolution HSI of the same scene has become the very popular HSI super-resolution scheme. In this paper, a novel low tensor-train (TT) rank (LTTR)-based HSI super-resolution method is proposed, where an LTTR prior is designed to learn the correlations among the spatial, spectral, and nonlocal modes of the nonlocal similar high-spatial-resolution HSI (HR-HSI) cubes. First, we cluster the HR-MSI cubes as many groups based on their similarities, and the HR-HSI cubes are also clustered according to the learned cluster structure in the HR-MSI cubes. The HR-HSI cubes in each group are much similar to each other and can constitute a 4-D tensor, whose four modes are highly correlated. Therefore, we impose the LTTR constraint on these 4-D tensors, which can effectively learn the correlations among the spatial, spectral, and nonlocal modes because of the well-balanced matricization scheme of TT rank. We formulate the super-resolution problem as TT rank regularized optimization problem, which is solved via the scheme of alternating direction method of multipliers. Experiments on HSI data sets indicate the effectiveness of the LTTR-based method.
252 citations
TL;DR: In this article, the sparsity of neuron activations in CNNs is exploited to accelerate the computation and reduce memory requirements for low-power and low-latency application scenarios.
Abstract: Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving many state-of-the-art (SOA) visual processing tasks. Even though graphical processing units are most often used in training and deploying CNNs, their power efficiency is less than 10 GOp/s/W for single-frame runtime inference. We propose a flexible and efficient CNN accelerator architecture called NullHop that implements SOA CNNs useful for low-power and low-latency application scenarios. NullHop exploits the sparsity of neuron activations in CNNs to accelerate the computation and reduce memory requirements. The flexible architecture allows high utilization of available computing resources across kernel sizes ranging from $1\times 1$ to $7\times 7$ . NullHop can process up to 128 input and 128 output feature maps per layer in a single pass. We implemented the proposed architecture on a Xilinx Zynq field-programmable gate array (FPGA) platform and presented the results showing how our implementation reduces external memory transfers and compute time in five different CNNs ranging from small ones up to the widely known large VGG16 and VGG19 CNNs. Postsynthesis simulations using Mentor Modelsim in a 28-nm process with a clock frequency of 500 MHz show that the VGG19 network achieves over 450 GOp/s. By exploiting sparsity, NullHop achieves an efficiency of 368%, maintains over 98% utilization of the multiply–accumulate units, and achieves a power efficiency of over 3 TOp/s/W in a core area of 6.3 mm2. As further proof of NullHop’s usability, we interfaced its FPGA implementation with a neuromorphic event camera for real-time interactive demonstrations.
TL;DR: This paper proposes an enhanced robot skill learning system considering both motion generation and trajectory tracking, and a neural-network-based controller is designed for the robot to track the trajectories generated from the motion model.
Abstract: This paper proposes an enhanced robot skill learning system considering both motion generation and trajectory tracking. During robot learning demonstrations, dynamic movement primitives (DMPs) are used to model robotic motion. Each DMP consists of a set of dynamic systems that enhances the stability of the generated motion toward the goal. A Gaussian mixture model and Gaussian mixture regression are integrated to improve the learning performance of the DMP, such that more features of the skill can be extracted from multiple demonstrations. The motion generated from the learned model can be scaled in space and time. Besides, a neural-network-based controller is designed for the robot to track the trajectories generated from the motion model. In this controller, a radial basis function neural network is used to compensate for the effect caused by the dynamic environments. The experiments have been performed using a Baxter robot and the results have confirmed the validity of the proposed methods.
TL;DR: A CNN-based NR-IQA framework that can effectively solve the challenge of applying a deep CNN to no-reference image quality assessment and a way to visualize perceptual error maps to analyze what was learned by the deep CNN model is proposed.
Abstract: Image recognition based on convolutional neural networks (CNNs) has recently been shown to deliver the state-of-the-art performance in various areas of computer vision and image processing. Nevertheless, applying a deep CNN to no-reference image quality assessment (NR-IQA) remains a challenging task due to critical obstacles, i.e., the lack of a training database. In this paper, we propose a CNN-based NR-IQA framework that can effectively solve this problem. The proposed method—deep image quality assessor (DIQA)—separates the training of NR-IQA into two stages: 1) an objective distortion part and 2) a human visual system-related part. In the first stage, the CNN learns to predict the objective error map, and then the model learns to predict subjective score in the second stage. To complement the inaccuracy of the objective error map prediction on the homogeneous region, we also propose a reliability map. Two simple handcrafted features were additionally employed to further enhance the accuracy. In addition, we propose a way to visualize perceptual error maps to analyze what was learned by the deep CNN model. In the experiments, the DIQA yielded the state-of-the-art accuracy on the various databases.
TL;DR: Wang et al. as mentioned in this paper proposed a generative approach, referred to as multimodal stochastic recurrent neural networks (MS-RNNs), which models the uncertainty observed in the data using latent variables.
Abstract: Video captioning, in essential, is a complex natural process, which is affected by various uncertainties stemming from video content, subjective judgment, and so on. In this paper, we build on the recent progress in using encoder-decoder framework for video captioning and address what we find to be a critical deficiency of the existing methods that most of the decoders propagate deterministic hidden states. Such complex uncertainty cannot be modeled efficiently by the deterministic models. In this paper, we propose a generative approach, referred to as multimodal stochastic recurrent neural networks (MS-RNNs), which models the uncertainty observed in the data using latent stochastic variables. Therefore, MS-RNN can improve the performance of video captioning and generate multiple sentences to describe a video considering different random factors. Specifically, a multimodal long short-term memory (LSTM) is first proposed to interact with both visual and textual features to capture a high-level representation. Then, a backward stochastic LSTM is proposed to support uncertainty propagation by introducing latent variables. Experimental results on the challenging data sets, microsoft video description and microsoft research video-to-text, show that our proposed MS-RNN approach outperforms the state-of-the-art video captioning benchmarks.
TL;DR: In this paper, an adaptive neural tracking control of underactuated surface vessels with modeling uncertainties and time-varying external disturbances is presented, where the tracking errors consisting of position and orientation errors are required to keep inside their predefined feasible regions in which the controller singularity problem does not happen.
Abstract: This paper presents adaptive neural tracking control of underactuated surface vessels with modeling uncertainties and time-varying external disturbances, where the tracking errors consisting of position and orientation errors are required to keep inside their predefined feasible regions in which the controller singularity problem does not happen. To provide the preselected specifications on the transient and steady-state performances of the tracking errors, the boundary functions of the predefined regions are taken as exponentially decaying functions of time. The unknown external disturbances are estimated by disturbance observers and then are compensated in the feedforward control loop to improve the robustness against the disturbances. Based on the dynamic surface control technique, backstepping procedure, logarithmic barrier functions, and control Lyapunov synthesis, singularity-free controllers are presented to guarantee the satisfaction of predefined performance requirements. In addition to the nominal case when the accurate model of a marine vessel is known a priori , the modeling uncertainties in the form of unknown nonlinear functions are also discussed. Adaptive neural control with the compensations of modeling uncertainties and external disturbances is developed to achieve the boundedness of the signals in the closed-loop system with guaranteed transient and steady-state tracking performances. Simulation results show the performance of the vessel control systems.
TL;DR: A novel HDA method that can optimize both feature discrepancy and distribution divergence in a unified objective function is proposed, which first learns a new transferable feature space by dictionary-sharing coding, and then aligns the distribution gaps on the new space.
Abstract: In real-world transfer learning tasks, especially in cross-modal applications, the source domain and the target domain often have different features and distributions, which are well known as the heterogeneous domain adaptation (HDA) problem. Yet, existing HDA methods focus on either alleviating the feature discrepancy or mitigating the distribution divergence due to the challenges of HDA. In fact, optimizing one of them can reinforce the other. In this paper, we propose a novel HDA method that can optimize both feature discrepancy and distribution divergence in a unified objective function. Specifically, we present progressive alignment , which first learns a new transferable feature space by dictionary-sharing coding, and then aligns the distribution gaps on the new space. Different from previous HDA methods that are limited to specific scenarios, our approach can handle diverse features with arbitrary dimensions. Extensive experiments on various transfer learning tasks, such as image classification, text categorization, and text-to-image recognition, verify the superiority of our method against several state-of-the-art approaches.
TL;DR: A valid adaptive neural state-feedback controller design algorithm is presented such that all the signals of the switched closed-loop system are in probability semiglobally uniformly ultimately bounded, and the tracking error eventually converges to a small neighborhood of the origin in probability.
Abstract: In this paper, the problem of adaptive neural state-feedback tracking control is considered for a class of stochastic nonstrict-feedback nonlinear switched systems with completely unknown nonlinearities. In the design procedure, the universal approximation capability of radial basis function neural networks is used for identifying the unknown compounded nonlinear functions, and a variable separation technique is employed to overcome the design difficulty caused by the nonstrict-feedback structure. The most outstanding novelty of this paper is that individual Lyapunov function of each subsystem is constructed by flexibly adopting the upper and lower bounds of the control gain functions of each subsystem. Furthermore, by combining the average dwell-time scheme and the adaptive backstepping design, a valid adaptive neural state-feedback controller design algorithm is presented such that all the signals of the switched closed-loop system are in probability semiglobally uniformly ultimately bounded, and the tracking error eventually converges to a small neighborhood of the origin in probability. Finally, the availability of the developed control scheme is verified by two simulation examples.
TL;DR: The asymptotic synchronization of coupled reaction–diffusion neural networks with proportional delay and Markovian switching topologies is considered in this brief where the diffusion space does not need to contain the origin.
Abstract: The asymptotic synchronization of coupled reaction–diffusion neural networks with proportional delay and Markovian switching topologies is considered in this brief where the diffusion space does not need to contain the origin. The main objectives of this brief are to save communication resources and to reduce the conservativeness of the obtained synchronization criteria, which are carried out from the following two aspects: 1) mode-dependent quantized control technique is designed to reduce control cost and save communication channels and 2) Wirtinger inequality is utilized to deal with the reaction–diffusion terms in a matrix form and reciprocally convex technique combined with new Lyapunov–Krasovskii functional is used to derive delay-dependent synchronization criteria. The obtained results are general and formulated by linear matrix inequalities. Moreover, combined with an optimal algorithm, control gains with the least magnitude are designed.
TL;DR: An evolutionary cost-sensitive deep belief network (ECS-DBN) for imbalanced classification that uses adaptive differential evolution to optimize the misclassification costs based on the training data that presents an effective approach to incorporating the evaluation measure into the objective function.
Abstract: Imbalanced data with a skewed class distribution are common in many real-world applications. Deep Belief Network (DBN) is a machine learning technique that is effective in classification tasks. However, conventional DBN does not work well for imbalanced data classification because it assumes equal costs for each class. To deal with this problem, cost-sensitive approaches assign different misclassification costs for different classes without disrupting the true data sample distributions. However, due to lack of prior knowledge, the misclassification costs are usually unknown and hard to choose in practice. Moreover, it has not been well studied as to how cost-sensitive learning could improve DBN performance on imbalanced data problems. This paper proposes an evolutionary cost-sensitive deep belief network (ECS-DBN) for imbalanced classification. ECS-DBN uses adaptive differential evolution to optimize the misclassification costs based on the training data that presents an effective approach to incorporating the evaluation measure (i.e., G-mean) into the objective function. We first optimize the misclassification costs, and then apply them to DBN. Adaptive differential evolution optimization is implemented as the optimization algorithm that automatically updates its corresponding parameters without the need of prior domain knowledge. The experiments have shown that the proposed approach consistently outperforms the state of the art on both benchmark data sets and real-world data set for fault diagnosis in tool condition monitoring.
TL;DR: The leader-following consensus issue with event/self-triggered schemes under an unreliable network environment is investigated and a self- Triggered communication scheme is proposed in which the next triggering instant can be determined by computing with the most updated information.
Abstract: This paper investigates the leader-following consensus issue with event/self-triggered schemes under an unreliable network environment. First, we characterize network communication and control protocol update in the presence of denial-of-service (DoS) attacks. In this situation, an event-triggered communication scheme is first proposed to effectively schedule information transmission over the network possibly subject to malicious attacks. In this communication framework, synchronous and asynchronous updated strategies of control protocols are constructed to achieve leader-following consensus in the presence of DoS attacks. Moreover, to further reduce the cost induced by event detection, a self-triggered communication scheme is proposed in which the next triggering instant can be determined by computing with the most updated information. Finally, a numerical example is provided to verify the effectiveness of the proposed communication schemes and updated strategies in the unreliable network environment.
TL;DR: A neural network layer architecture that incorporates the idea of bilinear projection as well as an attention mechanism that enables the layer to detect and focus on crucial temporal information is proposed, which outperforms by a large margin all existing state-of-the-art results coming from much deeper architectures while requiring far fewer computations.
Abstract: Financial time-series forecasting has long been a challenging problem because of the inherently noisy and stochastic nature of the market. In the high-frequency trading, forecasting for trading purposes is even a more challenging task, since an automated inference system is required to be both accurate and fast. In this paper, we propose a neural network layer architecture that incorporates the idea of bilinear projection as well as an attention mechanism that enables the layer to detect and focus on crucial temporal information. The resulting network is highly interpretable, given its ability to highlight the importance and contribution of each temporal instance, thus allowing further analysis on the time instances of interest. Our experiments in a large-scale limit order book data set show that a two-hidden-layer network utilizing our proposed layer outperforms by a large margin all existing state-of-the-art results coming from much deeper architectures while requiring far fewer computations.
TL;DR: Based on the backstepping recursive technique and the common Lyapunov function method, a finite-time switching control method is presented and the effectiveness of the proposed method is given through its application to a mass-spring-damper system.
Abstract: This paper solves the finite-time switching control issue for the nonstrict-feedback nonlinear switched systems. The controlled plants contain immeasurable states, arbitrarily switchings, and the unknown functions which are constructed with the whole states. Neural network is used to simulate the uncertain systems and a filter-based state observer is designed to estimate the immeasurable states in this paper, respectively. Based on the backstepping recursive technique and the common Lyapunov function method, a finite-time switching control method is presented. Due to the developed finite-time control strategy, the closed-loop signals can be ensured to be bounded under arbitrarily switchings, and the outputs of systems can quickly track the desired reference signals in finite time. The effectiveness of the proposed method is given through its application to a mass-spring-damper system.
TL;DR: A novel multiview clustering method is proposed by using t-product in the third-order tensor space, to which the tensor-tensor product can be applied and which outperforms the state-of-the-art methods for a range of criteria.
Abstract: The ubiquitous information from multiple-view data, as well as the complementary information among different views, is usually beneficial for various tasks, for example, clustering, classification, denoising, and so on. Multiview subspace clustering is based on the fact that multiview data are generated from a latent subspace. To recover the underlying subspace structure, a successful approach adopted recently has been sparse and/or low-rank subspace clustering. Despite the fact that existing subspace clustering approaches may numerically handle multiview data, by exploring all possible pairwise correlation within views, high-order statistics that can only be captured by simultaneously utilizing all views are often overlooked. As a consequence, the clustering performance of the multiview data is compromised. To address this issue, in this paper, a novel multiview clustering method is proposed by using t-product in the third-order tensor space. First, we propose a novel tensor construction method to organize multiview tensorial data, to which the tensor-tensor product can be applied. Second, based on the circular convolution operation, multiview data can be effectively represented by a t-linear combination with sparse and low-rank penalty using “self-expressiveness.” Our extensive experimental results on face, object, digital image, and text data demonstrate that the proposed method outperforms the state-of-the-art methods for a range of criteria.
TL;DR: An adaptive neural control method is investigated for the first time to address the problems of the time-varying full-state constraints and time-Varying delays in a unified framework with Lyapunov–Krasovskii functions.
Abstract: This paper proposes an adaptive neural control method for a class of nonlinear time-varying delayed systems with time-varying full-state constraints. To address the problems of the time-varying full-state constraints and time-varying delays in a unified framework, an adaptive neural control method is investigated for the first time. The problems of time delay and constraint are the main factors of limiting the system performance severely and even cause system instability. The effect of unknown time-varying delays is eliminated by using appropriate Lyapunov–Krasovskii functionals. In addition, the constant constraint is the only special case of time-varying constraint which leads to more complex and difficult tasks. To guarantee the full state always within the time-varying constrained interval, the time-varying asymmetric barrier Lyapunov function is employed. Finally, two simulation examples are given to confirm the effectiveness of the presented control scheme.
TL;DR: By establishing a key lemma specifically tackling the SPP, sufficient conditions are obtained such that, for any SPP less than or equal to a predefined upper bound, the error dynamics of the state estimation is asymptotically stable and satisfies a prescribed performance requirement.
Abstract: This paper investigates the $H_{\infty }$ state estimation problem for a class of discrete-time nonlinear singularly perturbed complex networks (SPCNs) under the Round-Robin (RR) protocol. A discrete-time nonlinear SPCN model is first devised on two time scales with their discrepancies reflected by a singular perturbation parameter (SPP). The network measurement outputs are transmitted via a communication network where the data transmissions are scheduled by the RR protocol with hope to avoid the undesired data collision. The error dynamics of the state estimation is governed by a switched system with a periodic switching parameter. A novel Lyapunov function is constructed that is dependent on both the transmission order and the SPP. By establishing a key lemma specifically tackling the SPP, sufficient conditions are obtained such that, for any SPP less than or equal to a predefined upper bound, the error dynamics of the state estimation is asymptotically stable and satisfies a prescribed $H_{\infty }$ performance requirement. Furthermore, the explicit parameterization of the desired state estimator is given by means of the solution to a set of matrix inequalities, and the upper bound of the SPP is then evaluated in the feasibility of these matrix inequalities. Moreover, the corresponding results for linear discrete-time SPCNs are derived as corollaries. A numerical example is given to illustrate the effectiveness of the proposed state estimator design scheme.
TL;DR: Simulations illustrate the effectiveness of the proposed bounded controller for tracking a moving target, which is designed based on the neural estimation model and a saturated function and bounded with the bounds known as a priori.
Abstract: This paper is concerned with the target tracking of underactuated autonomous surface vehicles with unknown dynamics and limited control torques. The velocity of the target is unknown, and only the measurements of line-of-sight range and angle are obtained. First, a kinematic control law is designed based on an extended state observer, which is utilized to estimate the uncertain target dynamics due to the unknown velocities. Next, an estimation model based on a single-hidden-layer neural network is developed to approximate the unknown follower dynamics induced by uncertain model parameters, unmodeled dynamics, and environmental disturbances. A bounded control law is designed based on the neural estimation model and a saturated function. The salient feature of the proposed controller is twofold. First, only the measured line-of-sight range and angle are used, and the velocity information of the target is not required. Second, the control torques are bounded with the bounds known as a priori . The input-to-state stability of the closed-loop system is analyzed via cascade theory. Simulations illustrate the effectiveness of the proposed bounded controller for tracking a moving target.
TL;DR: A new solution for a multistage game between the attacker and the defender based on reinforcement learning to identify the optimal attack sequences given certain objectives (e.g., transmission line outages or generation loss) is proposed.
Abstract: Existing smart grid security research investigates different attack techniques and cascading failures from the attackers’ viewpoints, while the defenders’ or the operators’ protection strategies are somehow neglected. Game theoretic methods are applied for the attacker–defender games in the smart grid security area. Yet, most of the existing works only use the one-shot game and do not consider the dynamic process of the electric power grid. In this paper, we propose a new solution for a multistage game (also called a dynamic game) between the attacker and the defender based on reinforcement learning to identify the optimal attack sequences given certain objectives (e.g., transmission line outages or generation loss). Different from a one-shot game, the attacker here learns a sequence of attack actions applying for the transmission lines and the defender protects a set of selected lines. After each time step, the cascading failure will be measured, and the line outage (and/or generation loss) will be used as the feedback for the attacker to generate the next action. The performance is evaluated on W&W 6-bus and IEEE 39-bus systems. A comparison between a multistage attack and a one-shot attack is conducted to show the significance of the multistage attack. Furthermore, different protection strategies are evaluated in simulation, which shows that the proposed reinforcement learning solution can identify optimal attack sequences under several attack objectives. It also indicates that attacker’s learned information helps the defender to enhance the security of the system.
TL;DR: The dissipative synchronization control problem for Markovian jump memristive neural networks (MNNs) is addressed with fully considering the time-varying delays and the fragility problem in the process of implementing the gain-scheduled controller.
Abstract: In this paper, the dissipative synchronization control problem for Markovian jump memristive neural networks (MNNs) is addressed with fully considering the time-varying delays and the fragility problem in the process of implementing the gain-scheduled controller. A Markov jump model is introduced to describe the stochastic changing among the connection of MNNs and it makes the networks under consideration suitable for some actual circumstances. By utilizing some improved integral inequalities and constructing a proper Lyapunov–Krasovskii functional, several delay-dependent synchronization criteria with less conservatism are established to ensure the dynamic error system is strictly stochastically dissipative. Based on these criteria, the procedure of designing the desired nonfragile gain-scheduled controller is established, which can well handle the fragility problem in the process of implementing the controller. Finally, an illustrated example is employed to explain that the developed method is efficient and available.
TL;DR: This paper solves the problem of nonrigid point set registration by designing a robust transformation learning scheme and applies the proposed method to learning motion flows between image pairs of similar scenes for visual homing, which is a specific type of mobile robot navigation.
Abstract: This paper solves the problem of nonrigid point set registration by designing a robust transformation learning scheme. The principle is to iteratively establish point correspondences and learn the nonrigid transformation between two given sets of points. In particular, the local feature descriptors are used to search the correspondences and some unknown outliers will be inevitably introduced. To precisely learn the underlying transformation from noisy correspondences, we cast the point set registration into a semisupervised learning problem, where a set of indicator variables is adopted to help distinguish outliers in a mixture model. To exploit the intrinsic structure of a point set, we constrain the transformation with manifold regularization which plays a role of prior knowledge. Moreover, the transformation is modeled in the reproducing kernel Hilbert space, and a sparsity-induced approximation is utilized to boost efficiency. We apply the proposed method to learning motion flows between image pairs of similar scenes for visual homing, which is a specific type of mobile robot navigation. Extensive experiments on several publicly available data sets reveal the superiority of the proposed method over state-of-the-art competitors, particularly in the context of the degenerated data.
TL;DR: The results show that the compensation and hardware impairment mitigation capabilities of the ARVTDNN are superior to the existing state-of-the-art real-valued focused time-delay neural network (RVFTDNN) by 3–4 dB for the adjacent channel power ratio and by 2–3 dB in terms of the normalized mean square error.
Abstract: A digital predistorter, modeled by an augmented real-valued time-delay neural network (ARVTDNN), has been proposed and found suitable to mitigate the nonlinear distortions of the power amplifier (PA) along with modulator imperfections for a wideband direct-conversion transmitter. The input signal of the proposed ARVTDNN consists of Cartesian in-phase and quadrature phase ( $I/Q$ ) components, as well as envelope-dependent terms. Theoretical analysis shows that the proposed model is able to produce a richer basis function containing both the desired odd- and even-order terms, resulting in improved modeling capability and distortion mitigation. Its actual performance has been validated through extensive simulations and experiments. The results show that the compensation and hardware impairment mitigation capabilities of the ARVTDNN are superior to the existing state-of-the-art real-valued focused time-delay neural network (RVFTDNN) by 3–4 dB for the adjacent channel power ratio and by 2–3 dB in terms of the normalized mean square error. Other important features of the proposed model are its reduced complexity, in terms of the number of parameters and floating-point operations, and its improved numerical stability compared to the RVFTDNN model.