Classification problem solving using multi-objective optimization approach and local search
01 Mar 2016-pp 243-247
TL;DR: Pareto approach is used to optimize neural network to solve classification problem and gives set of Pareto optimal solutions which represent different solutions for given classification problem.
Abstract: Classification is important task of data mining used to extract knowledge from huge volume of data. By nature, classification is multi-objective problem, as it required optimization of multiple objectives simultaneously like accuracy, sensitivity, squared error, precision etc. Traditionally, evolutionary algorithms were used to solve multi-objective classification problem by considering it as single-objective problem, but this approach gives single solution to problem. Therefore, multi-objective evolutionary algorithms are used to solve classification problem. In this paper, we have used Pareto approach to optimize neural network to solve classification problem. Non-dominated sorting genetic algorithm is used to simultaneously optimize accuracy and mean squared error objectives of neural network along with local search. As slow convergence to optimal solutions is major disadvantage of evolutionary algorithm. To speed up convergence to optimal solutions hybrid technique is adopted by augmenting evolutionary technique with local search algorithm. This proposed approach gives set of Pareto optimal solutions which represent different solutions for given classification problem.
Citations
More filters
TL;DR: This study provides a review on the basic theories and main recent algorithms for optimizing the ANN, and different types of nature-inspired meta-heuristic algorithms are presented; outlining the concepts and components that are used in order to give a summary and ease of the state-of-the-arts to find suitable methods in real world applications for the readers.
47 citations
TL;DR: In this paper , a hybrid of ACO-BP has been used for initialization of CNN weights using Ant Colony Optimization (ACO) and its further optimization using Backpropagation (BP) to overcome local minima.
Abstract: Convolution Neural Network (CNN) has been widely used in pattern recognition for various applications. Convolution neural network performs non-linear transformation on input to generate the global abstract feature vector. The resulting global feature vector is input to Fully connected Neural Network (FNN) and the activation value at the neuron in the output layer classifies the input data vector. During training of CNN on a given dataset, error at the output layer is minimized using backpropagation with stochastic gradient descent. The weights optimization using backpropagation has a drawback of local minima. Thus, in this research paper hybrid of ACO-BP has been used for initialization of CNN weights using Ant Colony Optimization (ACO) and its further optimization using Backpropagation (BP) to overcome local minima. The performance of CNN shows the improvement since the ability of deep learning architecture to generalize depends on the weight configuration during training phase. Experiment was conducted on MINST data set using k-fold cross validation method to confirm the effectiveness of CNN with hybrid of ACO-BP in pattern recognition. The results show the improvement in the classification accuracy-using hybrid of ACO-BP with CNN in comparison to CNN with BP only.
1 citations
Dissertation•
29 Jan 2018
TL;DR: In this paper, the authors proposed to add the ramping costs into the generation scheduling procedure with the aim of compensating the economic and technical losses of generation units, which may result in common damage mechanisms such as thermal shock, metal fatigue, corrosion, erosion and heat decay.
Abstract: The augmented renewable penetration due to the growth of environmental concerns increases the necessity of additional flexibility because of supply variability, and it reduces the existing flexibility level by displacing with conventional units due to the priority in dispatch for the renewable resources. Therefore, conventional units have to start-up, shut-down and ramp up/down more frequently to preserve the system balance in real-time which may result in common damage mechanisms such as thermal shock, metal fatigue, corrosion, erosion and heat decay [1]. To overcome this issue, it has been proposed to add the ramping costs into the generation scheduling procedure with the aim of compensating the economic and technical losses of generation units [2]-[4]. The inclusion of ramping costs in the day-ahead scheduling has been studied in [2].The effects of the variable nature of renewable generations on ramping costs of thermal units have been evaluated in [3]. The ramping costs have been incorporated into the generation scheduling problem in the presence of uncertain renewable generations [4]. However, the mentioned works have not modeled the ramp market. Practically, in order to compensate a partial loss of conventional generators and incentivize them to provide both upward and downward flexible ramp, a well-functioning market has been developed so-called "flexiramp" in California ISO (CAISO) [5] and "ramp capability" in Midcontinent ISO (MISO) [6] along with energy and reserve markets in order to ensure the rampability of reserve capacity provided by its generation mixture to cope with sudden net load variations.The creation of flexible ramp products in real-time ISO markets has been investigated in [7] with a set of simplified assumptions such as ignoring the transmission constraints and supposing predefined day-ahead decisions. Formulations for the day-ahead energy and flexible ramp markets' clearing have been proposed in [8] and [9], respectively. The authors in [9] discussed the role of electric vehicles participation in the ramp market, whereas [9] dealt with the evaluation of the impacts of natural gas delivery system modeling and demand response on flexible ramp deployment. Despite the mentioned reports in the literature, finding the optimal demand response programs in the flexible ramp markets has not been addressed.
References
More filters
TL;DR: This paper suggests a non-dominated sorting-based MOEA, called NSGA-II (Non-dominated Sorting Genetic Algorithm II), which alleviates all of the above three difficulties, and modify the definition of dominance in order to solve constrained multi-objective problems efficiently.
Abstract: Multi-objective evolutionary algorithms (MOEAs) that use non-dominated sorting and sharing have been criticized mainly for: (1) their O(MN/sup 3/) computational complexity (where M is the number of objectives and N is the population size); (2) their non-elitism approach; and (3) the need to specify a sharing parameter. In this paper, we suggest a non-dominated sorting-based MOEA, called NSGA-II (Non-dominated Sorting Genetic Algorithm II), which alleviates all of the above three difficulties. Specifically, a fast non-dominated sorting approach with O(MN/sup 2/) computational complexity is presented. Also, a selection operator is presented that creates a mating pool by combining the parent and offspring populations and selecting the best N solutions (with respect to fitness and spread). Simulation results on difficult test problems show that NSGA-II is able, for most problems, to find a much better spread of solutions and better convergence near the true Pareto-optimal front compared to the Pareto-archived evolution strategy and the strength-Pareto evolutionary algorithm - two other elitist MOEAs that pay special attention to creating a diverse Pareto-optimal front. Moreover, we modify the definition of dominance in order to solve constrained multi-objective problems efficiently. Simulation results of the constrained NSGA-II on a number of test problems, including a five-objective, seven-constraint nonlinear problem, are compared with another constrained multi-objective optimizer, and the much better performance of NSGA-II is observed.
37,111 citations
"Classification problem solving usin..." refers methods in this paper
...The NSGA II [6] is used to optimize TTBPN by simultaneously reducing complexity in terms of error and hidden nodes [4]....
[...]
...Droguett proposed NSGA II [6] multi-objective algorithm for training of neural network to solve the problem of predicting the scale deposition rate in oil and gas utensils....
[...]
...Curteanu proposed multiobjective optimization approach to optimize a polysiloxane synthesis process by using elitist NSGA II [6] algorithm and Neural network....
[...]
Book•
08 Sep 2000TL;DR: This book presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects, and provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data.
Abstract: The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data
23,600 citations
"Classification problem solving usin..." refers background in this paper
...Evolutionary algorithms have been found to be useful in automatic processing of large volume of raw noisy data due to their inherent parallel architecture [1], [2]....
[...]
...Minimize Mean Squared Error Mean Squared Error (E) [2] = (2)...
[...]
...[2] Han and Kamber, Data Mining: Concepts and Techniques, San Francisco, CA, USA: Morgan Kaufmann, 2006....
[...]
Book•
01 Jan 2001
TL;DR: This text provides an excellent introduction to the use of evolutionary algorithms in multi-objective optimization, allowing use as a graduate course text or for self-study.
Abstract: From the Publisher:
Evolutionary algorithms are relatively new, but very powerful techniques used to find solutions to many real-world search and optimization problems. Many of these problems have multiple objectives, which leads to the need to obtain a set of optimal solutions, known as effective solutions. It has been found that using evolutionary algorithms is a highly effective way of finding multiple effective solutions in a single simulation run. · Comprehensive coverage of this growing area of research · Carefully introduces each algorithm with examples and in-depth discussion · Includes many applications to real-world problems, including engineering design and scheduling · Includes discussion of advanced topics and future research · Features exercises and solutions, enabling use as a course text or for self-study · Accessible to those with limited knowledge of classical multi-objective optimization and evolutionary algorithms The integrated presentation of theory, algorithms and examples will benefit those working and researching in the areas of optimization, optimal design and evolutionary computing. This text provides an excellent introduction to the use of evolutionary algorithms in multi-objective optimization, allowing use as a graduate course text or for self-study.
12,134 citations
TL;DR: The proof-of-principle results obtained on two artificial problems as well as a larger problem, the synthesis of a digital hardware-software multiprocessor system, suggest that SPEA can be very effective in sampling from along the entire Pareto-optimal front and distributing the generated solutions over the tradeoff surface.
Abstract: Evolutionary algorithms (EAs) are often well-suited for optimization problems involving several, often conflicting objectives. Since 1985, various evolutionary approaches to multiobjective optimization have been developed that are capable of searching for multiple solutions concurrently in a single run. However, the few comparative studies of different methods presented up to now remain mostly qualitative and are often restricted to a few approaches. In this paper, four multiobjective EAs are compared quantitatively where an extended 0/1 knapsack problem is taken as a basis. Furthermore, we introduce a new evolutionary approach to multicriteria optimization, the strength Pareto EA (SPEA), that combines several features of previous multiobjective EAs in a unique manner. It is characterized by (a) storing nondominated solutions externally in a second, continuously updated population, (b) evaluating an individual's fitness dependent on the number of external nondominated points that dominate it, (c) preserving population diversity using the Pareto dominance relationship, and (d) incorporating a clustering procedure in order to reduce the nondominated set without destroying its characteristics. The proof-of-principle results obtained on two artificial problems as well as a larger problem, the synthesis of a digital hardware-software multiprocessor system, suggest that SPEA can be very effective in sampling from along the entire Pareto-optimal front and distributing the generated solutions over the tradeoff surface. Moreover, SPEA clearly outperforms the other four multiobjective EAs on the 0/1 knapsack problem.
7,512 citations
"Classification problem solving usin..." refers background or methods in this paper
...Following algorithms are used for multi-objective optimization: Non-dominated Sorting Genetic Algorithm (NSGA) [10], Strength Pareto Evolutionary Algorithm (SPEA) [11], Strength Pareto Evolutionary Algorithm 2 (SPEA 2) [12], Pareto archived evolution strategy (PAES) [13], Pareto Envelope-based Selection Algorithm (PESA) [14], Pareto Envelope-based Selection Algorithm II (PESA II) [15]....
[...]
...it requires longer search time and evolutionary approach becomes slow [11]....
[...]
...[12] E. Zitzler, L. Thiele and M. Laumanns, SPEA2: Improving the strength Pareto evolutionary algorithm, in Proc....
[...]
TL;DR: Goldberg's notion of nondominated sorting in GAs along with a niche and speciation method to find multiple Pareto-optimal points simultaneously are investigated and suggested to be extended to higher dimensional and more difficult multiobjective problems.
Abstract: In trying to solve multiobjective optimization problems, many traditional methods scalarize the objective vector into a single objective. In those cases, the obtained solution is highly sensitive to the weight vector used in the scalarization process and demands that the user have knowledge about the underlying problem. Moreover, in solving multiobjective problems, designers may be interested in a set of Pareto-optimal points, instead of a single point. Since genetic algorithms (GAs) work with a population of points, it seems natural to use GAs in multiobjective optimization problems to capture a number of solutions simultaneously. Although a vector evaluated GA (VEGA) has been implemented by Schaffer and has been tried to solve a number of multiobjective problems, the algorithm seems to have bias toward some regions. In this paper, we investigate Goldberg's notion of nondominated sorting in GAs along with a niche and speciation method to find multiple Pareto-optimal points simultaneously. The proof-of-principle results obtained on three problems used by Schaffer and others suggest that the proposed method can be extended to higher dimensional and more difficult multiobjective problems. A number of suggestions for extension and application of the algorithm are also discussed.
6,411 citations
"Classification problem solving usin..." refers methods in this paper
...Following algorithms are used for multi-objective optimization: Non-dominated Sorting Genetic Algorithm (NSGA) [10], Strength Pareto Evolutionary Algorithm (SPEA) [11], Strength Pareto Evolutionary Algorithm 2 (SPEA 2) [12], Pareto archived evolution strategy (PAES) [13], Pareto Envelope-based Selection Algorithm (PESA) [14], Pareto Envelope-based Selection Algorithm II (PESA II) [15]....
[...]