scispace - formally typeset
Search or ask a question

Showing papers in "International Journal of Applied Mathematics and Computer Science in 2011"


Journal ArticleDOI
TL;DR: It is shown that every electrical circuit is a singular fractional system if it contains at least one mesh consisting of branches only with an ideal supercapacitor and voltage sources or at leastone node with branches with supercoils.
Abstract: Singular fractional linear systems and electrical circuitsA new class of singular fractional linear systems and electrical circuits is introduced. Using the Caputo definition of the fractional derivative, the Weierstrass regular pencil decomposition and the Laplace transformation, the solution to the state equation of singular fractional linear systems is derived. It is shown that every electrical circuit is a singular fractional system if it contains at least one mesh consisting of branches only with an ideal supercapacitor and voltage sources or at least one node with branches with supercoils.

88 citations


Journal ArticleDOI
TL;DR: This paper proposes a random projections based formal concept analysis for knowledge discovery in data and demonstrates the implementation of the proposed method on two real world healthcare datasets.
Abstract: Knowledge discovery in data using formal concept analysis and random projectionsIn this paper our objective is to propose a random projections based formal concept analysis for knowledge discovery in data. We demonstrate the implementation of the proposed method on two real world healthcare datasets. Formal Concept Analysis FCA is a mathematical framework that offers a conceptual knowledge representation through hierarchical conceptual structures called concept lattices. However, during the design of a concept lattice, complexity plays a major role.

65 citations


Journal ArticleDOI
TL;DR: The design of a neural network based feedback linearization (NNFBL) controller for a two degree-of-freedom, quarter-car, servo-hydraulic vehicle suspension system is presented to improve the system's ride comfort and handling quality.
Abstract: This paper presents the design of a neural network based feedback linearization (NNFBL) controller for a two degree-of-freedom (DOF), quarter-car, servo-hydraulic vehicle suspension system. The main objective of the direct adaptive NNFBL controller is to improve the system's ride comfort and handling quality. A feedforward, multi-layer perceptron (MLP) neural network (NN) model that is well suited for control by discrete input-output linearization (NNIOL) is developed using input-output data sets obtained from mathematical model simulation. The NN model is trained using the Levenberg-Marquardt optimization algorithm. The proposed controller is compared with a constant-gain PID controller (based on the Ziegler-Nichols tuning method) during suspension travel setpoint tracking in the presence of deterministic road disturbance. Simulation results demonstrate the superior performance of the proposed direct adaptive NNFBL controller over the generic PID controller in rejecting the deterministic road disturbance. This superior performance is achieved at a much lower control cost within the stipulated constraints.

65 citations


Journal ArticleDOI
TL;DR: Stability and Hopf bifurcation analysis for a two-species Lotka-Volterra predator-prey model with two delays and the linear stability of the positive equilibrium is investigated and hopf bIfurcation is demonstrated.
Abstract: In this paper, a two-species Lotka-Volterra predator-prey model with two delays is considered. By analyzing the associated characteristic transcendental equation, the linear stability of the positive equilibrium is investigated and Hopf bifurcation is demonstrated. Some explicit formulae for determining the stability and direction of Hopf bifurcation periodic solutions bifurcating from Hopf bifurcations are obtained by using normal form theory and center manifold theory. Some numerical simulations for supporting the theoretical results are also included.

54 citations


Journal ArticleDOI
TL;DR: In the context of norm based residual evaluation, the residual generator and evaluator are designed together in an integrated form, and, based on it, a trade-off FD system is finally achieved in the sense that, for a given Fault Detection Rate, the False Alarm Rate is minimized.
Abstract: Integrated design of observer based Fault Detection (FD) for a class of uncertain nonlinear systems with Lipschitz nonlinearities is studied. In the context of norm based residual evaluation, the residual generator and evaluator are designed together in an integrated form, and, based on it, a trade-off FD system is finally achieved in the sense that, for a given Fault Detection Rate (FDR), the False Alarm Rate (FAR) is minimized. A numerical example is given to illustrate the effectiveness of the proposed design method.

50 citations


Journal ArticleDOI
TL;DR: The Haar wavelet approximating method is used to reduce the fractional Volterra and Abel integral equations to a system of algebraic equations to demonstrate the validity and applicability of the developed method.
Abstract: A Haar wavelet operational matrix is applied to fractional integration, which has not been undertaken before The Haar wavelet approximating method is used to reduce the fractional Volterra and Abel integral equations to a system of algebraic equations A global error bound is estimated and some numerical examples with smooth, nonsmooth, and singular solutions are considered to demonstrate the validity and applicability of the developed method

47 citations


Journal ArticleDOI
TL;DR: A new design method for microprogrammed controllers is proposed, which allows reducing the total number of logic elements needed for implementation in programmable structures, especially Field Programmable Gate Arrays (FPGAs).
Abstract: In the article we propose a new design method for microprogrammed controllers. The traditional structure is improved by modifying internal modules and connections. Such a solution allows reducing the total number of logic elements needed for implementation in programmable structures, especially Field Programmable Gate Arrays (FPGAs). Detailed results of experiments show that on the average the application of the proposed methods yields up to 30% savings as far as the destination device is considered.

43 citations


Journal ArticleDOI
TL;DR: Analysis of AQM queues with queue size based packet dropping Queueing systems in which an arriving job is blocked and lost with a probability that depends on the queue size are studied.
Abstract: Analysis of AQM queues with queue size based packet droppingQueueing systems in which an arriving job is blocked and lost with a probability that depends on the queue size are studied. The study is motivated by the popularity of Active Queue Management AQM algorithms proposed for packet queueing in Internet routers. AQM algorithms often exploit the idea of queue-size based packet dropping. The main results include analytical solutions for queue size distribution, loss ratio and throughput. The analytical results are illustrated via numerical examples that include some commonly used blocking probabilities dropping functions.

41 citations


Journal ArticleDOI
TL;DR: This analysis shows the relationship between reliability and control reconfigurability defined generally through Gramian controllability and proposes an admissible solution according to reliability evaluation based on energy consumption under degraded functional conditions.
Abstract: In this paper the integration of reliability evaluation in reconfigurability analysis of a fault-tolerant control system is considered. The aim of this work is to contribute to reliable fault-tolerant control design. The admissibility of control reconfigurability is analyzed with respect to reliability requirements. This analysis shows the relationship between reliability and control reconfigurability defined generally through Gramian controllability. An admissible solution for reconfigurability is proposed according to reliability evaluation based on energy consumption under degraded functional conditions. The proposed study is illustrated with a flight control application.

38 citations


Journal ArticleDOI
TL;DR: Several new results on the inversion of full normal rank nonsquare polynomial matrices are presented, and applications in inverse-model control, in particular robust minimum variance control, are exploited, and possible applications in signal transmission/recovery in various types of MIMO channels are indicated.
Abstract: This paper presents several new results on the inversion of full normal rank nonsquare polynomial matrices. New analytical right/left inverses of polynomial matrices are introduced, including the so-called i¾?-inverses, i¾?-inverses and, in particular, S-inverses, the latter providing the most general tool for the design of various polynomial matrix inverses. The applicationoriented problem of selecting stable inverses is also solved. Applications in inverse-model control, in particular robust minimum variance control, are exploited, and possible applications in signal transmission/recovery in various types of MIMO channels are indicated.

38 citations


Journal ArticleDOI
TL;DR: In this article, a new recursive optimal filter structure with transformation of the original system is proposed, which is based on the singular value decomposition of the direct feed-through matrix distribution of the fault which is assumed to be of arbitrary rank.
Abstract: This paper studies recursive optimal filtering as well as robust fault and state estimation for linear stochastic systems with unknown disturbances. It proposes a new recursive optimal filter structure with transformation of the original system. This transformation is based on the singular value decomposition of the direct feedthrough matrix distribution of the fault which is assumed to be of arbitrary rank. The resulting filter is optimal in the sense of the unbiased minimum-variance criteria. Two numerical examples are given in order to illustrate the proposed method, in particular to solve the estimation of the simultaneous actuator and sensor fault problem and to make a comparison with the existing literature results.

Journal ArticleDOI
TL;DR: This paper presents two general non-cooperative game approaches, namely, the symmetric non-zero sum game and the asymmetric Stackelberg game for modeling grid user behavior defined as user requirements, and designs and implemented GA-based hybrid schedulers for approximating the equilibrium points for both games.
Abstract: Tasks scheduling and resource allocation are among crucial issues in any large scale distributed system, including Computational Grids (CGs). These issues are commonly investigated using traditional computational models and resolution methods that yield near-optimal scheduling strategies. One drawback of such approaches is that they cannot effectively tackle the complex nature of CGs. On the one hand, such systems account for many administrative domains with their own access policies, user privileges, etc. On the other, CGs have hierarchical nature and therefore any computational model should be able to effectively express the hierarchical architecture in the optimization model. Recently, researchers have been investigating the use of game theory for modeling user requirements regarding task and resource allocation in grid scheduling problems. In this paper we present two general non-cooperative game approaches, namely, the symmetric non-zero sum game and the asymmetric Stackelberg game for modeling grid user behavior defined as user requirements. In our game-theoretic approaches we are able to cast new requirements arising in allocation problems, such as asymmetric users relations, security and reliability restrictions in CGs. For solving the games, we designed and implemented GA-based hybrid schedulers for approximating the equilibrium points for both games. The proposed hybrid resolution methods are experimentally evaluated through the grid simulator under heterogeneity, and large-scale and dynamics conditions. The relative performance of the schedulers is measured in terms of the makespan and flowtime metrics. The experimental analysis showed high efficiency of meta-heuristics in solving the game-based models, especially in the case of an additional cost of secure task scheduling to be paid by the users.

Journal ArticleDOI
TL;DR: The concept of a Lyapunov game as a subclass of strictly dominated games and potential games is introduced and it is shown that, for repeated games with bounded nonnegative cost functions within the class of differentiable vector functions whose derivatives satisfy the Lipschitz condition, a complex vector-function can be built, where each component is a function of the corresponding cost value and satisfies the condition of the Lyap unov-like function.
Abstract: We introduce the concept of a Lyapunov game as a subclass of strictly dominated games and potential games. The advantage of this approach is that every ergodic system (repeated game) can be represented by a Lyapunov-like function. A direct acyclic graph is associated with a game. The graph structure represents the dependencies existing between the strategy profiles. By definition, a Lyapunov-like function monotonically decreases and converges to a single Lyapunov equilibrium point identified by the sink of the game graph. It is important to note that in previous works this convergence has not been guaranteed even if the Nash equilibrium point exists. The best reply dynamics result in a natural implementation of the behavior of a Lyapunov-like function. Therefore, a Lyapunov game has also the benefit that it is common knowledge of the players that only best replies are chosen. By the natural evolution of a Lyapunov-like function, no matter what, a strategy played once is not played again. As a construction example, we show that, for repeated games with bounded nonnegative cost functions within the class of differentiable vector functions whose derivatives satisfy the Lipschitz condition, a complex vector-function can be built, where each component is a function of the corresponding cost value and satisfies the condition of the Lyapunov-like function. The resulting vector Lyapunov-like function is a monotonic function which can only decrease over time. Then, a repeated game can be represented by a one-shot game. The functionality of the suggested method is successfully demonstrated by a simulated experiment.

Journal ArticleDOI
TL;DR: It is shown that a semantic wiki is a flexible tool for engineering knowledge on the knowledge formalization continuum, and the agile use of knowledge representations within a knowledge engineering project is proposed, as well as transitions between the different representations, when required.
Abstract: In spite of their industrial success, the development of intelligent systems is still a complex and risky task. When building intelligent systems, we see that domain knowledge is often present at different levels of formalization-ranging from text documents to explicit rules. In this paper, we describe the knowledge formalization continuum as a metaphor to help domain specialists during the knowledge acquisition phase. To make use of the knowledge formalization continuum, the agile use of knowledge representations within a knowledge engineering project is proposed, as well as transitions between the different representations, when required. We show that a semantic wiki is a flexible tool for engineering knowledge on the knowledge formalization continuum. Case studies are taken from one industrial and one academic project, and they illustrate the applicability and benefits of semantic wikis in combination with the knowledge formalization continuum.

Journal ArticleDOI
TL;DR: The principal idea of the paper is focused on the concept of needs, with a certain instrumental role of emotions, in a fully autonomous robotic decision-making system based on a mathematical model of human cognitive-behavioural psychology.
Abstract: The paper gives an account of research results concerning a project on creating a fully autonomous robotic decision-making system, able to interact with its environment and based on a mathematical model of human cognitive-behavioural psychology, with some key elements of personality psychology included. The principal idea of the paper is focused on the concept of needs, with a certain instrumental role of emotions.

Journal ArticleDOI
TL;DR: This paper presents a P2P-based infrastructure that leverages semantic technologies to support a scalable and accurate service discovery process and proposes an overlay network organized in several semantic groups of peers, each specialized in answering queries pertaining to specific applicative domains.
Abstract: An intense standardization process is favouring the convergence of grids and Service Oriented Architectures (SOAs). One of the benefits of such technological convergence is that grid resources and applications can be virtualized by services and offered through the SOA paradigm. In the broad and interoperable scenarios enabled by the SOA, involving the participation of several grid infrastructures across many administrative domains, service discovery can be a serious issue. In this paper we present a P2P-based infrastructure that leverages semantic technologies to support a scalable and accurate service discovery process. The key concept of the presented idea is the creation of an overlay network organized in several semantic groups of peers, each specialized in answering queries pertaining to specific applicative domains. Groups are formed by clustering together peers offering services that are semantically related. The architecture details of the proposed solution are presented. A system prototype has also been implemented and validated through a case study deployed on the PlanetLab testbed.

Journal ArticleDOI
TL;DR: It is shown that linear minimum-phase systems with real negative poles and zeros always have positive stable realizations.
Abstract: Conditions for the existence of positive stable realizations with system Metzler matrices for fractional continuous-time linear systems are established. A procedure based on the Gilbert method for computation of positive stable realizations of proper transfer matrices is proposed. It is shown that linear minimum-phase systems with real negative poles and zeros always have positive stable realizations.

Journal ArticleDOI
TL;DR: An approach is developed that characterizes the optimal control and leads to a numerical algorithm that solves the regional control problem for infinite dimensional bilinear systems.
Abstract: Regional control problem for distributed bilinear systems: Approach and simulationsThis paper investigates the regional control problem for infinite dimensional bilinear systems. We develop an approach that characterizes the optimal control and leads to a numerical algorithm. The obtained results are successfully illustrated by simulations.

Journal ArticleDOI
TL;DR: The approach to path planning is formulated as a "quasi-dynamic" NonLinear Programming (NLP) problem with equality and inequality constraints in terms of the joint variables, to find joint paths which satisfy the given constraints and minimize the proposed performance index.
Abstract: An application of advanced optimization techniques to solve the path planning problem for closed chain robot systems is proposed. The approach to path planning is formulated as a "quasi-dynamic" NonLinear Programming (NLP) problem with equality and inequality constraints in terms of the joint variables. The essence of the method is to find joint paths which satisfy the given constraints and minimize the proposed performance index. For numerical solution of the NLP problem, the IPOPT solver is used, which implements a nonlinear primal-dual interior-point method, one of the leading techniques for large-scale nonlinear optimization.

Journal ArticleDOI
TL;DR: New delay-dependent stochastic stability criteria are obtained in terms of linear matrix inequalities (LMIs) by employing a Lyapunov functional approach and significantly improve the allowable upper bounds of delays over some results existing in the literature.
Abstract: The paper is concerned with stability analysis for a class of impulsive Hopfield neural networks with Markovian jumping parameters and time-varying delays. The jumping parameters considered here are generated from a continuous-time discrete-state homogenous Markov process. By employing a Lyapunov functional approach, new delay-dependent stochastic stability criteria are obtained in terms of linear matrix inequalities (LMIs). The proposed criteria can be easily checked by using some standard numerical packages such as the Matlab LMI Toolbox. A numerical example is provided to show that the proposed results significantly improve the allowable upper bounds of delays over some results existing in the literature.

Journal ArticleDOI
TL;DR: A fuzzy augmented state and fault observer is designed to achieve the system state and sensor fault estimates simultaneously and an observer-based dynamic output feedback fault-tolerant controller is developed to compensate for the effect of faults by stabilizing the resulting closed-loop system.
Abstract: This paper addresses the problems of robust fault estimation and fault-tolerant control for Takagi-Sugeno (T-S) fuzzy systems with time delays and unknown sensor faults. A fuzzy augmented state and fault observer is designed to achieve the system state and sensor fault estimates simultaneously. Furthermore, based on the information of on-line fault estimates, an observer-based dynamic output feedback fault-tolerant controller is developed to compensate for the effect of faults by stabilizing the resulting closed-loop system. Sufficient conditions for the existence of both a state observer and a fault-tolerant controller are given in terms of linear matrix inequalities. A simulation example is given to illustrate the effectiveness of the proposed approach.

Journal ArticleDOI
TL;DR: The proposed design methodology can constitute a reliable and robust approach for application to real nonlinear processes and is compared with a different approach relying on sliding mode control, in order to evaluate benefits and drawbacks of both techniques.
Abstract: This paper describes the design of fault diagnosis and active fault tolerant control schemes that can be developed for nonlinear systems. The methodology is based on a fault detection and diagnosis procedure relying on adaptive filters designed via the nonlinear geometric approach, which allows obtaining the disturbance de-coupling property. The controller reconfiguration exploits directly the on-line estimate of the fault signal. The classical model of an inverted pendulum on a cart is considered as an application example, in order to highlight the complete design procedure, including the mathematical aspects of the nonlinear disturbance de-coupling method based on the nonlinear differential geometry, as well as the feasibility and efficiency of the proposed approach. Extensive simulations of the benchmark process and Monte Carlo analysis are practical tools for assessing experimentally the robustness and stability properties of the developed fault tolerant control scheme, in the presence of modelling and measurement errors. The fault tolerant control method is also compared with a different approach relying on sliding mode control, in order to evaluate benefits and drawbacks of both techniques. This comparison highlights that the proposed design methodology can constitute a reliable and robust approach for application to real nonlinear processes.

Journal ArticleDOI
TL;DR: The relationship between D CCA and LDA is analyzed, showing that the projective directions by DCCA are equal to the ones obtained from LDA with respect to an orthogonal transformation.
Abstract: Analysis of correlation based dimension reduction methodsDimension reduction is an important topic in data mining and machine learning. Especially dimension reduction combined with feature fusion is an effective preprocessing step when the data are described by multiple feature sets. Canonical Correlation Analysis CCA and Discriminative Canonical Correlation Analysis DCCA are feature fusion methods based on correlation. However, they are different in that DCCA is a supervised method utilizing class label information, while CCA is an unsupervised method. It has been shown that the classification performance of DCCA is superior to that of CCA due to the discriminative power using class label information. On the other hand, Linear Discriminant Analysis LDA is a supervised dimension reduction method and it is known as a special case of CCA. In this paper, we analyze the relationship between DCCA and LDA, showing that the projective directions by DCCA are equal to the ones obtained from LDA with respect to an orthogonal transformation. Using the relation with LDA, we propose a new method that can enhance the performance of DCCA. The experimental results show that the proposed method exhibits better classification performance than the original DCCA.

Journal ArticleDOI
TL;DR: Simulation results show that a PA with an IFF allocation algorithm for mesh systems and a torus-based NoC with express-virtual-channel flow control are very energy efficient and combination of these two solutions is a clear choice for modern CMPs.
Abstract: Energy consumption in a Chip MultiProcessor (CMP) is one of the most important costs. It is related to design aspects such as thermal and power constrains. Besides efficient on-chip processing elements, a well-designed Processor Allocator (PA) and a Network-on-Chip (NoC) are also important factors in the energy budget of novel CMPs. In this paper, the authors propose an energy model for NoCs with 2D-mesh and 2D-torus topologies. All important NoC architectures are described and discussed. Energy estimation is presented for PAs. The estimation is based on synthesis results for PAs targeting FPGA. The PAs are driven by allocation algorithms that are studied as well. The proposed energy model is employed in a simulation environment, where exhaustive experiments are performed. Simulation results show that a PA with an IFF allocation algorithm for mesh systems and a torus-based NoC with express-virtual-channel flow control are very energy efficient. Combination of these two solutions is a clear choice for modern CMPs.

Journal ArticleDOI
V. Prasath1
TL;DR: The well-posedness of the corresponding minimization problem is proved in the space of functions of bounded variation and the proposed adaptive weight based scheme provides a better multiscale edge map, which in turn produces better restoration.
Abstract: We propose an edge adaptive digital image denoising and restoration scheme based on space dependent regularization. Traditional gradient based schemes use an edge map computed from gradients alone to drive the regularization. This may lead to the oversmoothing of the input image, and noise along edges can be amplified. To avoid these drawbacks, we make use of a multiscale descriptor given by a contextual edge detector obtained from local variances. Using a smooth transition from the computed edges, the proposed scheme removes noise in flat regions and preserves edges without oscillations. By incorporating a space dependent adaptive regularization parameter, image smoothing is driven along probable edges and not across them. The well-posedness of the corresponding minimization problem is proved in the space of functions of bounded variation. The corresponding gradient descent scheme is implemented and further numerical results illustrate the advantages of using the adaptive parameter in the regularization scheme. Compared with similar edge preserving regularization schemes, the proposed adaptive weight based scheme provides a better multiscale edge map, which in turn produces better restoration.

Journal ArticleDOI
TL;DR: Findings indicate that a significant decrease in computing times is observed under these conditions and highlight how overheads and virtualisation in a distributed environment hinder the possibility of achieving the maximum (peak) performance.
Abstract: This work analyses the performance of Hadoop, an implementation of the MapReduce programming model for distributed parallel computing, executing on a virtualisation environment comprised of 1+16 nodes running the VMWare workstation software. A set of experiments using the standard Hadoop benchmarks has been designed in order to determine whether or not significant reductions in the execution time of computations are experienced when using Hadoop on this virtualisation platform on a departmental cloud. Our findings indicate that a significant decrease in computing times is observed under these conditions. They also highlight how overheads and virtualisation in a distributed environment hinder the possibility of achieving the maximum (peak) performance.

Journal ArticleDOI
TL;DR: A new approach regarding a reconfigured system is proposed to improve the performance of an active fault tolerant control system to avoid actuator saturation and reduce the control energy spent in closed loop dynamics.
Abstract: In this paper, a new approach regarding a reconfigured system is proposed to improve the performance of an active fault tolerant control system. The system performance is evaluated with an intelligent index of performance. The reconfiguration mechanism is based on a model predictive controller and reference trajectory management techniques. When an actuator fault occurs in the system, a new degraded reference trajectory is generated and the controller calculates new admissible controls. A constraint set and cost function are established to avoid actuator saturation and reduce the control energy spent in closed loop dynamics. The effectiveness of the proposed method is illustrated using a hydrothermal system subject to actuator faults and constraints on actuator dynamic ranges.

Journal ArticleDOI
TL;DR: This paper presents an approach to fault tolerant control based on the sensor masking principle in the case of wireless networked control systems, and facilitates fault compensation based on an interacting multiple model approach developed in the framework of channel errors or network congestion equivalent to multiple sensors failures.
Abstract: This paper presents an approach to fault tolerant control based on the sensor masking principle in the case of wireless networked control systems. With wireless transmission, packet losses act as sensor faults. In the presence of such faults, the faulty measurements corrupt directly the behaviour of closed-loop systems. Since the controller aims at cancelling the error between the measurement and its reference input, the real outputs will, in such a networked control system, deviate from the desired value and may drive the system to its physical limitations or even to instability. The proposed method facilitates fault compensation based on an interacting multiple model approach developed in the framework of channel errors or network congestion equivalent to multiple sensors failures. The interacting multiple model method involved in a networked control system provides simultaneously detection and isolation of on-line packet losses, and also performs a suitable state estimation. Based on particular knowledge of packet losses, sensor fault-tolerant controls are obtained by computing a new control law using fault-free estimation of the faulty element to avoid intermittent observations that might develop into failures and to minimize the effects on system performance and safety.

Journal ArticleDOI
TL;DR: The position control is shown to be capable of accommodating serious failures and preventing breakage of a mooring line, or a loss of a buoyancy element, from causing subsequent failures, as demonstrated by high fidelity simulations.
Abstract: This paper addresses fault-tolerant control for position mooring of a shuttle or floating production storage and offloading vessels. A complete framework for fault diagnosis is presented. A loss of a sub-sea mooring line buoyancy element and line breakage are given particular attention, since such failures might cause high-risk abortion of an oil-loading operation. With significant drift forces from waves, non-Gaussian elements dominate forces and the residuals designed for fault diagnosis. Hypothesis testing is designed using dedicated change detection for the type of distribution encountered. A new position recovery algorithm is proposed as a means of fault accommodation in order to keep the mooring system in a safe state, despite faults. The position control is shown to be capable of accommodating serious failures and preventing breakage of a mooring line, or a loss of a buoyancy element, from causing subsequent failures. Properties of the detection and fault-tolerant control algorithms are demonstrated by high fidelity simulations.

Journal ArticleDOI
TL;DR: The archived approach to Coevolutionary Temporal Difference Learning is superior to other techniques considered here and produces strategies that outperform a handcrafted weighted piece counter strategy and simple liberty-based heuristics.
Abstract: We apply Coevolutionary Temporal Difference Learning (CTDL) to learn small-board Go strategies represented as weighted piece counters. CTDL is a randomized learning technique which interweaves two search processes that operate in the intra-game and inter-game mode. Intra-game learning is driven by gradient-descent Temporal Difference Learning (TDL), a reinforcement learning method that updates the board evaluation function according to differences observed between its values for consecutively visited game states. For the inter-game learning component, we provide a coevolutionary algorithm that maintains a sample of strategies and uses the outcomes of games played between them to iteratively modify the probability distribution, according to which new strategies are generated and added to the sample. We analyze CTDL's sensitivity to all important parameters, including the trace decay constant that controls the lookahead horizon of TDL, and the relative intensity of intra-game and inter-game learning. We also investigate how the presence of memory (an archive) affects the search performance, and find out that the archived approach is superior to other techniques considered here and produces strategies that outperform a handcrafted weighted piece counter strategy and simple liberty-based heuristics. This encouraging result can be potentially generalized not only to other strategy representations used for small-board Go, but also to various games and a broader class of problems, because CTDL is generic and does not rely on any problem-specific knowledge.