Showing papers by "Hamid R. Tizhoosh published in 2007"

A novel population initialization method for accelerating evolutionary algorithms

Abstract: Population initialization is a crucial task in evolutionary algorithms because it can affect the convergence speed and also the quality of the final solution. If no information about the solution is available, then random initialization is the most commonly used method to generate candidate solutions (initial population). This paper proposes a novel initialization approach which employs opposition-based learning to generate initial population. The conducted experiments over a comprehensive set of benchmark functions demonstrate that replacing the random initialization with the opposition-based population initialization can accelerate convergence speed.

Quasi-oppositional Differential Evolution

Abstract: In this paper, an enhanced version of the opposition-based differential evolution (ODE) is proposed. ODE utilizes opposite numbers in the population initialization and generation jumping to accelerate differential evolution (DE). Instead of opposite numbers, in this work, quasi opposite points are used. So, we call the new extension quasi- oppositional DE (QODE). The proposed mathematical proof shows that in a black-box optimization problem quasi- opposite points have a higher chance to be closer to the solution than opposite points. A test suite with 15 benchmark functions has been employed to compare performance of DE, ODE, and QODE experimentally. Results confirm that QODE performs better than ODE and DE in overall. Details for the proposed approach and the conducted experiments are provided.

Applying Opposition-Based Ideas to the Ant Colony System

Abstract: This paper presents several extensions to an algorithm in the family of ant colony optimization, the ant colony system. The proposed extensions are based on the idea of opposition and attempt to increase the exploration efficiency of the solution space. The modifications focus on the solution construction phase of the ant colony system. Three of the proposed methods work by pairing the ants and synchronizing their path selection. The two other approaches modify the decisions of the ants by using an opposite-pheromone content. Results on the application of these algorithms on travelling salesman problem instances demonstrate that the concept of opposition is not easily applied to the ant algorithm. Only one of the pheromone-based methods showed performance improvements that were statistically significant. The quality of the solutions increased and more optimal solutions were found. The other extensions showed no clear improvement. Further work must be conducted to explore the successful pheromone-based approach, as well as to determine if opposition should be applied to a different phase of the algorithm

Opposition-Based Differential Evolution (ODE) with Variable Jumping Rate

Abstract: In this paper, a time varying jumping rate (TVJR) model for opposition-based differential evolution (ODE) has been proposed. According to this model, the jumping rate changes linearly during the evolution based on the number of function evaluations. A test suite with 15 well-known benchmark functions has been employed to compare performance of the DE and ODE with variable jumping rate settings. Results show that a higher jumping rate is more desirable during the exploration than during the exploitation. Details for the proposed approach and the conducted experiments are provided

Simulated Annealing with Opposite Neighbors

Abstract: This paper presents an improvement to the vanilla version of the simulated annealing algorithm by using opposite neighbors. This new technique, is based on the recently proposed idea of opposition based learning, as such our proposed algorithm is termed opposition-based simulated annealing (OSA). In this paper we provide a theoretical basis for the algorithm as well as its practical implementation. In order to examine the efficacy of the approach we compare the new algorithm to SA on six common real optimization problems. Our findings confirm the theoretical predictions as well as show a significant improvement in accuracy and convergence rate over traditional SA. We also provide experimental evidence for the use of opposite neighbors over purely random ones

Opposite Transfer Functions and Backpropagation Through Time

Abstract: Backpropagation through time is a very popular discrete-time recurrent neural network training algorithm. However, the computational time associated with the learning process to achieve high accuracy is high. While many approaches have been proposed that alter the learning algorithm, this paper presents a computationally inexpensive method based on the concept of opposite transfer functions to improve learning in the backpropagation through time algorithm. Specifically, we will show an improvement in the accuracy, stability as well as an acceleration in learning time. We will utilize three common benchmarks to provide experimental evidence of the improvements

Application of Opposition-Based Reinforcement Learning in Image Segmentation

Abstract: In this paper a method for image segmentation using an opposition-based reinforcement learning scheme is introduced. We use this agent-based approach to optimally find the appropriate local values and segment the object. The agent uses an image and its manually segmented version and takes some actions to change the environment (the quality of segmented image). The agent is provided with a scalar reinforcement signal as reward/punishment. The agent uses this information to explore/exploit the solution space. The values obtained can be used as valuable knowledge to fill the Q-matrix. The results demonstrate potential for applying this new method in the field of medical image segmentation

Opposition-Based Reinforcement Learning in the Management of Water Resources

Abstract: Opposition-based learning (OBL) is a new scheme in machine intelligence. In this paper, an OBL version Q-learning which exploits opposite quantities to accelerate the learning is used for management of single reservoir operations. In this method, an agent takes an action, receives reward, and updates its knowledge in terms of action-value functions. Furthermore, the transition function which is the balance equation in the optimization model determines the next state and updates the action-value function pertinent to opposite action. Two type of opposite actions will be defined. It will be demonstrated that using OBL can significantly improve the efficiency of the operating policy within limited iterations. It is also shown that this technique is more robust than Q-Learning

Fuzzy Classification Using Pattern Discovery

Abstract: Rule-based classifiers allow rationalization of classifications made. This in turn improves understanding which is essential for effective decision support. As a rule based classifier, the pattern discovery (PD) algorithm functions well in discrete, nominal and continuous data domains. A drawback when using PD as a classifier for decision support is that it has an unbounded decision space that confounds the understanding of the degree of support for a decision. Incorporating PD into a fuzzy inference system (FIS) allows the the degree of support for a decision to be expressed with intuitively understandable terms. In addition, using discrete algorithms in continuous domains can result in reduced accuracy due to quantization. Fuzzification reduces this ldquocost of quantizationrdquo and improves classification performance. In this work, the PD algorithm was used as a source of rules for a series of FISs implemented using different rule weighting and defuzzification schemes, each providing a linguistic basis for rule description and a bounded space for expression of decision support. The output of each FIS consists of a suggested outcome, a strong confidence metric describing suggestions within this space and a linguistic expression of the rules. This constitutes a stronger basis for decision making than that provided by PD alone. A variety of synthetic, continuous class distributions with varying degrees of separation was used to evaluate the performance of fuzzy, PD, back-propagation and Bayesian classifiers. Overall, the accuracy of the fuzzy system was found to be similar, but slightly below, that of the inherently continuous valued classifiers and was somewhat improved with respect to the PD classifiers. For the difficult spiral class distributions studied, the fuzzy classifiers were able to make more classifications than the PD classifiers. The correct classification rates for the fuzzy classifiers were similar across the various rule weighting and defuzzification schemes, demonstrating the strength of the statistical method for rule generation. Analysis of several real-world data sets shows that a PD-based FIS has comparable performance to a neuro-fuzzy system. The use of a PD based FIS however, provides insight into the structure of the data analyzed not available through the other approaches.

Opposition-Based Q(λ) with Non-Markovian Update

Abstract: The OQ(λ) algorithm benefits from an extension of eligibility traces introduced as opposition trace. This new technique is a combination of the idea of opposition and eligibility traces to deal with large state space problems in reinforcement learning applications. In our previous works the comparison of the results of OQ(λ) and conventional Watkins' Q(λ) reflected a remarkable increase in performance for the OQ(λ) algorithm. However the Markovian update of opposition traces is an issue which is investigated in this paper. It has been assumed that the opposite state can be presented to the agent. This may limit the usability of the technique to deterministic environments. In order to relax this assumption the non-Markovian opposition-based Q(λ) (NOQ(λ)) is introduced in this work. The new method is a hybrid of Markovian update for eligibility traces and non-Markovian-based update for opposition traces. The experimental results show improvements of learning speed for the proposed technique compared to Q(λ) and OQ(λ). The new technique performs faster than OQ(λ) algorithm with the same success rate and can be employed for broader range of applications since it does not require determining state transition

Opposition-Based Window Memoization for Morphological Algorithms

Abstract: In this paper we combine window memoization, a performance optimization technique for image processing, with opposition-based learning, a new learning scheme where the opposite of data under study is also considered in solving a problem. Window memoization combines memoization techniques from software and hardware with the repetitive nature of image data to reduce the number of calculations required for an image processing algorithm. We applied window memoization and opposition-based learning to a morphological edge detector and found that a large portion of the calculations performed on pixels neighborhoods can be skipped and instead, previously calculated results can be reused. The typical speedup for window memoization was 1.42. Combining window memoization with opposition-based learning yielded a typical increase of 5% in speedups

Reservoir Operation Optimization by Reinforcement Learning

Abstract: Plann ing of reservoir management and optimal operations of surface water resources has always been a critical and strategic concern of all governments. Today, many equipments, facilities, and substantial budgets have been assigned to carry out an optimal scheduling of water and energy resources over long or short periods. Many researchers have been working on these areas to improve the performance of such a system. They usually attempt to apply new mathematical and heuristic techniques to tackle a wide variety of complexities in real-world applications and especially large-scale problems. Stochasticity, nonlinearity/nonconvexity and dimensionality are the main sources of complexity. In other words, there are many techniques, which could circumvent these complexities via some kind of approximations in uncertain environments with complex and unknown relations between various system parameters. In fact, using different methods to optimize the operations of large-scale problems coming along with much unrealistic estimations makes the final solution very imprecise and usually too far from real optimal solution. Moreover, the existing limitations of hardware or software cause some important physical constraints, which prevent various relations between variables and parameters from being considered. In other words, even if all possible relations between parameters in a problem are known and definable, considering all of them simultaneously might make the problem very difficult to solve.

Accelerating Image Processing Algorithms Based on the Reuse of Spatial Patterns

Abstract: This paper presents window memoization, a performance optimization technique for convolution-based image processing algorithms. Window memoization exploits the repetitive nature of image data to reduce the number of calculations required for image processing algorithms and hence, it improves the performance. We applied window memoization to a chain of image processing algorithms that includes median filter, Kirsch edge detector and local edge filling. We found that a large portion of the calculations performed on pixel neighborhoods can be skipped and instead, previously calculated results can be reused. The typical speedups were in the range of 1.6times to 2.8times.

Reinforcement Agents for E-Learning Applications

Abstract: Advanced computer systems have become pivotal components for learning. However, we are still faced with many challenges in e-learning environments when developing reliable tools to assist users and facilitate and enhance the learning process. For instance, the problem of creating a user-friendly system that can learn from interaction with dynamic learning requirements and deal with largescale information is still widely unsolved. We need systems that have the ability to communicate and cooperate with the users, learn their preferences and increase the learning efficiency of individual users. Reinforcement learning (RL) is an intelligent technique with the ability to learn from interaction with the environment. It learns from trial and error and generally does not need any training data or a user model. At the beginning of the learning process, the RL agent does not have any knowledge about the actions it should take. After a while, the agent learns which actions yield the maximum reward. The ability of learning from interaction with a dynamic environment and using reward and punishment independent of any training data set makes reinforcement learning a suitable tool for e-learning situations, where subjective user feedback can easily be translated into a reinforcement signal.

Active exploratory q-learning for large problems

Abstract: Although reinforcement learning (RL) emerged more than a decade ago, it is still under extensive investigation in application to large problems, where the states and actions are multi-dimensional and continuous and result in the so- called curse of dimensionality. Conventional RL methods are still not efficient enough in huge state-action spaces, while value-function generalization-based approaches require a very large number of good training examples. This paper presents an active exploratory approach to address the challenge of RL in large problems. The core principle of this approach is that the agent does not rush to the next state. Instead, it attempts a number of actions at the current state first, and then selects the action that returns the greatest immediate reward. The state resulting from performing the action is considered as the next state. Four active exploration algorithms for good actions are proposed: random-based search, opposition-based random search, search by cyclical adjustment, and opposition-based cyclical adjustment of each action dimension. The efficiency of these algorithms is determined by a visual-servoing experiment with a 6-axis robot.