Showing papers in "IEEE Transactions on Neural Networks in 2014"

Classification in the Presence of Label Noise: A Survey

Abstract: Label noise is an important issue in classification, with many potential negative consequences. For example, the accuracy of predictions may decrease, whereas the complexity of inferred models and the number of necessary training samples may increase. Many works in the literature have been devoted to the study of label noise and the development of techniques to deal with label noise. However, the field lacks a comprehensive survey on the different types of label noise, their consequences and the algorithms that consider label noise. This paper proposes to fill this gap. First, the definitions and sources of label noise are considered and a taxonomy of the types of label noise is proposed. Second, the potential consequences of label noise are discussed. Third, label noise-robust, label noise cleansing, and label noise-tolerant algorithms are reviewed. For each category of approaches, a short discussion is proposed to help the practitioner to choose the most suitable technique in its own particular field of application. Eventually, the design of experiments is also discussed, what may interest the researchers who would like to test their own algorithms. In this paper, label noise consists of mislabeled instances: no additional information is assumed to be available like e.g., confidences on labels.

Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems

Abstract: This paper is concerned with a new discrete-time policy iteration adaptive dynamic programming (ADP) method for solving the infinite horizon optimal control problem of nonlinear systems. The idea is to use an iterative ADP technique to obtain the iterative control law, which optimizes the iterative performance index function. The main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete-time nonlinear systems for the first time. It shows that the iterative performance index function is nonincreasingly convergent to the optimal solution of the Hamilton-Jacobi-Bellman equation. It is also proven that any of the iterative control laws can stabilize the nonlinear systems. Neural networks are used to approximate the performance index function and compute the optimal control law, respectively, for facilitating the implementation of the iterative ADP algorithm, where the convergence of the weight matrices is analyzed. Finally, the numerical results and analysis are presented to illustrate the performance of the developed method.

Adaptive Consensus Control for a Class of Nonlinear Multiagent Time-Delay Systems Using Neural Networks

Abstract: Because of the complicity of consensus control of nonlinear multiagent systems in state time-delay, most of previous works focused only on linear systems with input time-delay. An adaptive neural network (NN) consensus control method for a class of nonlinear multiagent systems with state time-delay is proposed in this paper. The approximation property of radial basis function neural networks (RBFNNs) is used to neutralize the uncertain nonlinear dynamics in agents. An appropriate Lyapunov–Krasovskii functional, which is obtained from the derivative of an appropriate Lyapunov function, is used to compensate the uncertainties of unknown time delays. It is proved that our proposed approach guarantees the convergence on the basis of Lyapunov stability theory. The simulation results of a nonlinear multiagent time-delay system and a multiple collaborative manipulators system show the effectiveness of the proposed consensus control algorithm.

A Comprehensive Review of Stability Analysis of Continuous-Time Recurrent Neural Networks

Abstract: Stability problems of continuous-time recurrent neural networks have been extensively studied, and many papers have been published in the literature The purpose of this paper is to provide a comprehensive review of the research on stability of continuous-time recurrent neural networks, including Hopfield neural networks, Cohen-Grossberg neural networks, and related models Since time delay is inevitable in practice, stability results of recurrent neural networks with different classes of time delays are reviewed in detail For the case of delay-dependent stability, the results on how to deal with the constant/variable delay in recurrent neural networks are summarized The relationship among stability results in different forms, such as algebraic inequality forms, \(M\) -matrix forms, linear matrix inequality forms, and Lyapunov diagonal stability forms, is discussed and compared Some necessary and sufficient stability conditions for recurrent neural networks without time delays are also discussed Concluding remarks and future directions of stability analysis of recurrent neural networks are given

Short-Term Load and Wind Power Forecasting Using Neural Network-Based Prediction Intervals

Abstract: Electrical power systems are evolving from today's centralized bulk systems to more decentralized systems. Penetrations of renewable energies, such as wind and solar power, significantly increase the level of uncertainty in power systems. Accurate load forecasting becomes more complex, yet more important for management of power systems. Traditional methods for generating point forecasts of load demands cannot properly handle uncertainties in system operations. To quantify potential uncertainties associated with forecasts, this paper implements a neural network (NN)-based method for the construction of prediction intervals (PIs). A newly introduced method, called lower upper bound estimation (LUBE), is applied and extended to develop PIs using NN models. A new problem formulation is proposed, which translates the primary multiobjective problem into a constrained single-objective problem. Compared with the cost function, this new formulation is closer to the primary problem and has fewer parameters. Particle swarm optimization (PSO) integrated with the mutation operator is used to solve the problem. Electrical demands from Singapore and New South Wales (Australia), as well as wind power generation from Capital Wind Farm, are used to validate the PSO-based LUBE method. Comparative results show that the proposed method can construct higher quality PIs for load and wind power generation forecasts in a short time.

On the complexity of neural network classifiers: a comparison between shallow and deep architectures.

Abstract: Recently, researchers in the artificial neural network field have focused their attention on connectionist models composed by several hidden layers. In fact, experimental results and heuristic considerations suggest that deep architectures are more suitable than shallow ones for modern applications, facing very complex problems, e.g., vision and human language understanding. However, the actual theoretical results supporting such a claim are still few and incomplete. In this paper, we propose a new approach to study how the depth of feedforward neural networks impacts on their ability in implementing high complexity functions. First, a new measure based on topological concepts is introduced, aimed at evaluating the complexity of the function implemented by a neural network, used for classification purposes. Then, deep and shallow neural architectures with common sigmoidal activation functions are compared, by deriving upper and lower bounds on their complexity, and studying how the complexity depends on the number of hidden units and the used activation function. The obtained results seem to support the idea that deep networks actually implements functions of higher complexity, so that they are able, with the same number of resources, to address more difficult problems.

Reacting to Different Types of Concept Drift: The Accuracy Updated Ensemble Algorithm

Abstract: Data stream mining has been receiving increased attention due to its presence in a wide range of applications, such as sensor networks, banking, and telecommunication. One of the most important challenges in learning from data streams is reacting to concept drift, i.e., unforeseen changes of the stream's underlying data distribution. Several classification algorithms that cope with concept drift have been put forward, however, most of them specialize in one type of change. In this paper, we propose a new data stream classifier, called the Accuracy Updated Ensemble (AUE2), which aims at reacting equally well to different types of drift. AUE2 combines accuracy-based weighting mechanisms known from block-based ensembles with the incremental nature of Hoeffding Trees. The proposed algorithm is experimentally compared with 11 state-of-the-art stream methods, including single classifiers, block-based and online ensembles, and hybrid approaches in different drift scenarios. Out of all the compared algorithms, AUE2 provided best average classification accuracy while proving to be less memory consuming than other ensemble approaches. Experimental results show that AUE2 can be considered suitable for scenarios, involving many types of drift as well as static environments.

Memristor Crossbar-Based Neuromorphic Computing System: A Case Study

Abstract: By mimicking the highly parallel biological systems, neuromorphic hardware provides the capability of information processing within a compact and energy-efficient platform. However, traditional Von Neumann architecture and the limited signal connections have severely constrained the scalability and performance of such hardware implementations. Recently, many research efforts have been investigated in utilizing the latest discovered memristors in neuromorphic systems due to the similarity of memristors to biological synapses. In this paper, we explore the potential of a memristor crossbar array that functions as an autoassociative memory and apply it to brain-state-in-a-box (BSB) neural networks. Especially, the recall and training functions of a multianswer character recognition process based on the BSB model are studied. The robustness of the BSB circuit is analyzed and evaluated based on extensive Monte Carlo simulations, considering input defects, process variations, and electrical fluctuations. The results show that the hardware-based training scheme proposed in the paper can alleviate and even cancel out the majority of the noise issue.

Event-Based Visual Flow

Abstract: This paper introduces a new methodology to compute dense visual flow using the precise timings of spikes from an asynchronous event-based retina. Biological retinas, and their artificial counterparts, are totally asynchronous and data-driven and rely on a paradigm of light acquisition radically different from most of the currently used frame-grabber technologies. This paper introduces a framework to estimate visual flow from the local properties of events' spatiotemporal space. We will show that precise visual flow orientation and amplitude can be estimated using a local differential approach on the surface defined by coactive events. Experimental results are presented; they show the method adequacy with high data sparseness and temporal resolution of event-based acquisition that allows the computation of motion flow with microsecond accuracy and at very low computational cost.

Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems

Abstract: This paper studies the robust optimal control design for a class of uncertain nonlinear systems from a perspective of robust adaptive dynamic programming (RADP). The objective is to fill up a gap in the past literature of adaptive dynamic programming (ADP) where dynamic uncertainties or unmodeled dynamics are not addressed. A key strategy is to integrate tools from modern nonlinear control theory, such as the robust redesign and the backstepping techniques as well as the nonlinear small-gain theorem, with the theory of ADP. The proposed RADP methodology can be viewed as an extension of ADP to uncertain nonlinear systems. Practical learning algorithms are developed in this paper, and have been applied to the controller design problems for a jet engine and a one-machine power system.

Neural network-based motion control of an underactuated wheeled inverted pendulum model.

Abstract: In this paper, automatic motion control is investigated for one of wheeled inverted pendulum (WIP) models, which have been widely applied for modeling of a large range of two wheeled modern vehicles. First, the underactuated WIP model is decomposed into a fully actuated second order subsystem Σa consisting of planar movement of vehicle forward and yaw angular motions, and a nonactuated first order subsystem Σb of pendulum motion. Due to the unknown dynamics of subsystem Σa and the universal approximation ability of neural network (NN), an adaptive NN scheme has been employed for motion control of subsystem Σa . The model reference approach has been used whereas the reference model is optimized by the finite time linear quadratic regulation technique. The pendulum motion in the passive subsystem Σb is indirectly controlled using the dynamic coupling with planar forward motion of subsystem Σa , such that satisfactory tracking of a set pendulum tilt angle can be guaranteed. Rigours theoretic analysis has been established, and simulation studies have been performed to demonstrate the developed method.

Active Learning With Drifting Streaming Data

Abstract: In learning to classify streaming data, obtaining true labels may require major effort and may incur excessive cost. Active learning focuses on carefully selecting as few labeled instances as possible for learning an accurate predictive model. Streaming data poses additional challenges for active learning, since the data distribution may change over time (concept drift) and models need to adapt. Conventional active learning strategies concentrate on querying the most uncertain instances, which are typically concentrated around the decision boundary. Changes occurring further from the boundary may be missed, and models may fail to adapt. This paper presents a theoretically supported framework for active learning from drifting data streams and develops three active learning strategies for streaming data that explicitly handle concept drift. They are based on uncertainty, dynamic allocation of labeling efforts over time, and randomization of the search space. We empirically demonstrate that these strategies react well to changes that can occur anywhere in the instance space and unexpectedly.

Feature Learning for Image Classification Via Multiobjective Genetic Programming

Abstract: Feature extraction is the first and most critical step in image classification. Most existing image classification methods use hand-crafted features, which are not adaptive for different image domains. In this paper, we develop an evolutionary learning methodology to automatically generate domain-adaptive global feature descriptors for image classification using multiobjective genetic programming (MOGP). In our architecture, a set of primitive 2-D operators are randomly combined to construct feature descriptors through the MOGP evolving and then evaluated by two objective fitness criteria, i.e., the classification error and the tree complexity. After the entire evolution procedure finishes, the best-so-far solution selected by the MOGP is regarded as the (near-)optimal feature descriptor obtained. To evaluate its performance, the proposed approach is systematically tested on the Caltech-101, the MIT urban and nature scene, the CMU PIE, and Jochen Triesch Static Hand Posture II data sets, respectively. Experimental results verify that our method significantly outperforms many state-of-the-art hand-designed features and two feature learning techniques in terms of classification accuracy.

Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach

Abstract: In this paper, using a neural-network-based online learning optimal control approach, a novel decentralized control strategy is developed to stabilize a class of continuous-time nonlinear interconnected large-scale systems. First, optimal controllers of the isolated subsystems are designed with cost functions reflecting the bounds of interconnections. Then, it is proven that the decentralized control strategy of the overall system can be established by adding appropriate feedback gains to the optimal control policies of the isolated subsystems. Next, an online policy iteration algorithm is presented to solve the Hamilton-Jacobi-Bellman equations related to the optimal control problem. Through constructing a set of critic neural networks, the cost functions can be obtained approximately, followed by the control policies. Furthermore, the dynamics of the estimation errors of the critic networks are verified to be uniformly and ultimately bounded. Finally, a simulation example is provided to illustrate the effectiveness of the present decentralized control scheme.

Adaptive Neural Tracking Control for a Class of Nonstrict-Feedback Stochastic Nonlinear Systems With Unknown Backlash-Like Hysteresis

Abstract: This paper considers the problem of adaptive neural control of stochastic nonlinear systems in nonstrict-feedback form with unknown backlash-like hysteresis nonlinearities. To overcome the design difficulty of nonstrict-feedback structure, variable separation technique is used to decompose the unknown functions of all state variables into a sum of smooth functions of each error dynamic. By combining radial basis function neural networks' universal approximation capability with an adaptive backstepping technique, an adaptive neural control algorithm is proposed. It is shown that the proposed controller guarantees that all the signals in the closed-loop system are four-moment semiglobally uniformly ultimately bounded, and the tracking error eventually converges to a small neighborhood of the origin in the sense of mean quartic value. Simulation results further show the effectiveness of the presented control scheme.

PANFIS: A Novel Incremental Learning Machine

Abstract: Most of the dynamics in real-world systems are compiled by shifts and drifts, which are uneasy to be overcome by omnipresent neuro-fuzzy systems. Nonetheless, learning in nonstationary environment entails a system owning high degree of flexibility capable of assembling its rule base autonomously according to the degree of nonlinearity contained in the system. In practice, the rule growing and pruning are carried out merely benefiting from a small snapshot of the complete training data to truncate the computational load and memory demand to the low level. An exposure of a novel algorithm, namely parsimonious network based on fuzzy inference system (PANFIS), is to this end presented herein. PANFIS can commence its learning process from scratch with an empty rule base. The fuzzy rules can be stitched up and expelled by virtue of statistical contributions of the fuzzy rules and injected datum afterward. Identical fuzzy sets may be alluded and blended to be one fuzzy set as a pursuit of a transparent rule base escalating human's interpretability. The learning and modeling performances of the proposed PANFIS are numerically validated using several benchmark problems from real-world or synthetic datasets. The validation includes comparisons with state-of-the-art evolving neuro-fuzzy methods and showcases that our new method can compete and in some cases even outperform these approaches in terms of predictive fidelity and model complexity.

Distributed neural network control for adaptive synchronization of uncertain dynamical multiagent systems.

Abstract: This paper addresses the leader-follower synchronization problem of uncertain dynamical multiagent systems with nonlinear dynamics. Distributed adaptive synchronization controllers are proposed based on the state information of neighboring agents. The control design is developed for both undirected and directed communication topologies without requiring the accurate model of each agent. This result is further extended to the output feedback case where a neighborhood observer is proposed based on relative output information of neighboring agents. Then, distributed observer-based synchronization controllers are derived and a parameter-dependent Riccati inequality is employed to prove the stability. This design has a favorable decouple property between the observer and the controller designs for nonlinear multiagent systems. For both cases, the developed controllers guarantee that the state of each agent synchronizes to that of the leader with bounded residual errors. Two illustrative examples validate the efficacy of the proposed methods.

Reinforcement Learning Output Feedback NN Control Using Deterministic Learning Technique

Abstract: In this brief, a novel adaptive-critic-based neural network (NN) controller is investigated for nonlinear pure-feedback systems. The controller design is based on the transformed predictor form, and the actor-critic NN control architecture includes two NNs, whereas the critic NN is used to approximate the strategic utility function, and the action NN is employed to minimize both the strategic utility function and the tracking error. A deterministic learning technique has been employed to guarantee that the partial persistent excitation condition of internal states is satisfied during tracking control to a periodic reference orbit. The uniformly ultimate boundedness of closed-loop signals is shown via Lyapunov stability analysis. Simulation results are presented to demonstrate the effectiveness of the proposed control.

A Survey on CPG-Inspired Control Models and System Implementation

Abstract: This paper surveys the developments of the last 20 years in the field of central pattern generator (CPG) inspired locomotion control, with particular emphasis on the fast emerging robotics-related applications. Functioning as a biological neural network, CPGs can be considered as a group of coupled neurons that generate rhythmic signals without sensory feedback; however, sensory feedback is needed to shape the CPG signals. The basic idea in engineering endeavors is to replicate this intrinsic, computationally efficient, distributed control mechanism for multiple articulated joints, or multi-DOF control cases. In terms of various abstraction levels, existing CPG control models and their extensions are reviewed with a focus on the relative advantages and disadvantages of the models, including ease of design and implementation. The main issues arising from design, optimization, and implementation of the CPG-based control as well as possible alternatives are further discussed, with an attempt to shed more light on locomotion control-oriented theories and applications. The design challenges and trends associated with the further advancement of this area are also summarized.

Global and Local Structure Preservation for Feature Selection

Abstract: The recent literature indicates that preserving global pairwise sample similarity is of great importance for feature selection and that many existing selection criteria essentially work in this way. In this paper, we argue that besides global pairwise sample similarity, the local geometric structure of data is also critical and that these two factors play different roles in different learning scenarios. In order to show this, we propose a global and local structure preservation framework for feature selection (GLSPFS) which integrates both global pairwise sample similarity and local geometric data structure to conduct feature selection. To demonstrate the generality of our framework, we employ methods that are well known in the literature to model the local geometric data structure and develop three specific GLSPFS-based feature selection algorithms. Also, we develop an efficient optimization algorithm with proven global convergence to solve the resulting feature selection problem. A comprehensive experimental study is then conducted in order to compare our feature selection algorithms with many state-of-the-art ones in supervised, unsupervised, and semisupervised learning scenarios. The result indicates that: 1) our framework consistently achieves statistically significant improvement in selection performance when compared with the currently used algorithms; 2) in supervised and semisupervised learning scenarios, preserving global pairwise similarity is more important than preserving local geometric data structure; 3) in the unsupervised scenario, preserving local geometric data structure becomes clearly more important; and 4) the best feature selection performance is always obtained when the two factors are appropriately integrated. In summary, this paper not only validates the advantages of the proposed GLSPFS framework but also gains more insight into the information to be preserved in different feature selection tasks.

Dynamic Learning From Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems

Abstract: This paper studies the problem of learning from adaptive neural network (NN) control of a class of nonaffine nonlinear systems in uncertain dynamic environments. In the control design process, a stable adaptive NN tracking control design technique is proposed for the nonaffine nonlinear systems with a mild assumption by combining a filtered tracking error with the implicit function theorem, input-to-state stability, and the small-gain theorem. The proposed stable control design technique not only overcomes the difficulty in controlling nonaffine nonlinear systems but also relaxes constraint conditions of the considered systems. In the learning process, the partial persistent excitation (PE) condition of radial basis function NNs is satisfied during tracking control to a recurrent reference trajectory. Under the PE condition and an appropriate state transformation, the proposed adaptive NN control is shown to be capable of acquiring knowledge on the implicit desired control input dynamics in the stable control process and of storing the learned knowledge in memory. Subsequently, an NN learning control design technique that effectively exploits the learned knowledge without re-adapting to the controller parameters is proposed to achieve closed-loop stability and improved control performance. Simulation studies are performed to demonstrate the effectiveness of the proposed design techniques.

Synchronization on complex networks of networks.

Abstract: In this paper, pinning synchronization on complex networks of networks is investigated, where there are many subnetworks with the interactions among them. The subnetworks and their connections can be regarded as the nodes and interactions of the networks, respectively, which form the networks of networks. In this new setting, the aim is to design pinning controllers on the chosen nodes of each subnetwork so as to reach synchronization behavior. Some synchronization criteria are established for reaching pinning control on networks of networks. Furthermore, the pinning scheme is designed, which shows that the nodes with very low degrees and large degrees are good candidates for applying pinning controllers. Then, the attack and robustness of the pinning scheme are discussed. Finally, a simulation example is presented to verify the theoretical analysis in this paper.

Delay-Dependent Stability Criteria for Generalized Neural Networks With Two Delay Components

Abstract: This paper investigates the delay-dependent stability for generalized continuous neural networks with time-varying delays. A novel Lyapunov–Krasovskii functional (LKF) that considers more information on activation functions of delayed neural networks and delay upper bounds is developed. Simultaneously, most commonly used techniques for treating the derivative of the LKF are reviewed and compared with each other. With the way of introducing slack matrices, those techniques are classified into two categories, including free-weighting matrix (FWM)-based techniques and reciprocally convex combination-based techniques. It is found that the introduced slack matrices play an important role in conservatism reducing and those four types of FWM-based methods lead to same results and are equivalent. Moreover, the obtained criteria are extended to the system with a single time-varying delay. Two numerical examples are given to verify the effectiveness of the proposed method.

Multilinear Sparse Principal Component Analysis

Abstract: In this brief, multilinear sparse principal component analysis (MSPCA) is proposed for feature extraction from the tensor data. MSPCA can be viewed as a further extension of the classical principal component analysis (PCA), sparse PCA (SPCA) and the recently proposed multilinear PCA (MPCA). The key operation of MSPCA is to rewrite the MPCA into multilinear regression forms and relax it for sparse regression. Differing from the recently proposed MPCA, MSPCA inherits the sparsity from the SPCA and iteratively learns a series of sparse projections that capture most of the variation of the tensor data. Each nonzero element in the sparse projections is selected from the most important variables/factors using the elastic net. Extensive experiments on Yale, Face Recognition Technology face databases, and COIL-20 object database encoded the object images as second-order tensors, and Weizmann action database as third-order tensors demonstrate that the proposed MSPCA algorithm has the potential to outperform the existing PCA-based subspace learning algorithms.

Sparse Bayesian Extreme Learning Machine for Multi-classification

Abstract: Extreme learning machine (ELM) has become a popular topic in machine learning in recent years. ELM is a new kind of single-hidden layer feedforward neural network with an extremely low computational cost. ELM, however, has two evident drawbacks: 1) the output weights solved by Moore-Penrose generalized inverse is a least squares minimization issue, which easily suffers from overfitting and 2) the accuracy of ELM is drastically sensitive to the number of hidden neurons so that a large model is usually generated. This brief presents a sparse Bayesian approach for learning the output weights of ELM in classification. The new model, called Sparse Bayesian ELM (SBELM), can resolve these two drawbacks by estimating the marginal likelihood of network outputs and automatically pruning most of the redundant hidden neurons during learning phase, which results in an accurate and compact model. The proposed SBELM is evaluated on wide types of benchmark classification problems, which verifies that the accuracy of SBELM model is relatively insensitive to the number of hidden neurons; and hence a much more compact model is always produced as compared with other state-of-the-art neural network classifiers.

Lagrange Stability of Memristive Neural Networks With Discrete and Distributed Delays

Abstract: Memristive neuromorphic system is a good candidate for creating artificial brain. In this paper, a general class of memristive neural networks with discrete and distributed delays is introduced and studied. Some Lagrange stability criteria dependent on the network parameters are derived via nonsmooth analysis and control theory. In particular, several succinct criteria are provided to ascertain the Lagrange stability of memristive neural networks with and without delays. The proposed Lagrange stability criteria are the improvement and extension of the existing results in the literature. Three numerical examples are given to show the superiority of theoretical results.

Attractivity Analysis of Memristor-Based Cellular Neural Networks With Time-Varying Delays

Abstract: This paper presents new theoretical results on the invariance and attractivity of memristor-based cellular neural networks (MCNNs) with time-varying delays. First, sufficient conditions to assure the boundedness and global attractivity of the networks are derived. Using state-space decomposition and some analytic techniques, it is shown that the number of equilibria located in the saturation regions of the piecewise-linear activation functions of an n-neuron MCNN with time-varying delays increases significantly from 2n to 22n2+n (22n2 times) compared with that without a memristor. In addition, sufficient conditions for the invariance and local or global attractivity of equilibria or attractive sets in any designated region are derived. Finally, two illustrative examples are given to elaborate the characteristics of the results in detail.

PCA Feature Extraction for Change Detection in Multidimensional Unlabeled Data

Abstract: When classifiers are deployed in real-world applications, it is assumed that the distribution of the incoming data matches the distribution of the data used to train the classifier. This assumption is often incorrect, which necessitates some form of change detection or adaptive classification. While there has been a lot of work on change detection based on the classification error monitored over the course of the operation of the classifier, finding changes in multidimensional unlabeled data is still a challenge. Here, we propose to apply principal component analysis (PCA) for feature extraction prior to the change detection. Supported by a theoretical example, we argue that the components with the lowest variance should be retained as the extracted features because they are more likely to be affected by a change. We chose a recently proposed semiparametric log-likelihood change detection criterion that is sensitive to changes in both mean and variance of the multidimensional distribution. An experiment with 35 datasets and an illustration with a simple video segmentation demonstrate the advantage of using extracted features compared to raw data. Further analysis shows that feature extraction through PCA is beneficial, specifically for data with multiple balanced classes.

Multiclass From Binary: Expanding One-Versus-All, One-Versus-One and ECOC-Based Approaches

Abstract: Recently, there has been a lot of success in the development of effective binary classifiers. Although many statistical classification techniques have natural multiclass extensions, some, such as the support vector machines, do not. The existing techniques for mapping multiclass problems onto a set of simpler binary classification problems run into serious efficiency problems when there are hundreds or even thousands of classes, and these are the scenarios where this paper's contributions shine. We introduce the concept of correlation and joint probability of base binary learners. We learn these properties during the training stage, group the binary leaner's based on their independence and, with a Bayesian approach, combine the results to predict the class of a new instance. Finally, we also discuss two additional strategies: one to reduce the number of required base learners in the multiclass classification, and another to find new base learners that might best complement the existing set. We use these two new procedures iteratively to complement the initial solution and improve the overall performance. This paper has two goals: finding the most discriminative binary classifiers to solve a multiclass problem and keeping up the efficiency, i.e., small number of base learners. We validate and compare the method with a diverse set of methods of the literature in several public available datasets that range from small (10 to 26 classes) to large multiclass problems (1000 classes) always using simple reproducible scenarios.

Synchronization in an array of output-coupled Boolean networks with time delay.

Abstract: This brief presents an analytical study of synchronization in an array of coupled deterministic Boolean networks (BNs) with time delay. Two kinds of models are considered. In one model, the outputs contain time delay, while in another one, the outputs do not. One restriction in this brief is that the state delay and output delay are restricted to be equal. By referring to the algebraic representations of logical dynamics and using the techniques of semitensor product of matrices, some necessary and sufficient conditions are derived for the synchronization of delay-coupled BNs. Examples including a practical epigenetic example are given for illustration.