Genetic programming (GP) is a systematic, domain-independent method for getting computers to solve problems automatically starting from a high-level statement of what needs to be done. Using ideas from natural evolution, GP starts from an ooze of random computer programs, and progressively refines them through processes of mutation and sexual recombination, until high-fitness solutions emerge. All this without the user having to know or specify the form or structure of solutions in advance. GP has generated a plethora of human-competitive results and applications, including novel scientific discoveries and patentable inventions. This unique overview of this exciting technique is written by three of the most active scientists in GP. See www.gp-field-guide.org.uk for more information on the book.

/pdf/a-field-guide-to-genetic-programming-3wkkmsw81n.pdf

A Field Guide to Genetic Programming

This paper presents a new form of Genetic Programming called Cartesian Genetic Programming in which a program is represented as an indexed graph. The graph is encoded in the form of a linear string of integers. The inputs or terminal set and node outputs are numbered sequentially. The node functions are also separately numbered. The genotype is just a list of node connections and functions. The genotype is then mapped to an indexed graph that can be executed as a program. Evolutionary algorithms are used to evolve the genotype in a symbolic regression problem (sixth order polynomial) and the Santa Fe Ant Trail. The computational effort is calculated for both cases. It is suggested that hit effort is a more reliable measure of computational efficiency. A neutral search strategy that allows the fittest genotype to be replaced by another equally fit genotype (a neutral genotype) is examined and compared with non-neutral search for the Santa Fe ant problem. The neutral search proves to be much more effective.

Cartesian Genetic Programming

The Artificial Intelligence field continues to be plagued by what can only be described as ‘bold promises for the future syndrome’, often perpetrated by researchers who should know better. While impartial assessment can point to concrete contributions over the past 50 years (such as automated theorem proving, games strategies, the LISP and Prolog high-level computer languages, Automatic Speech Recognition, Natural Language Processing, mobile robot path planning, unmanned vehicles, humanoid robots, data mining, and more), the more cynical argue that AI has witnessed more than its fair share of ‘unmitigated disasters’ during this time – see, for example [3,58,107,125,186]. The general public becomes rapidly jaded with such ‘bold predictions’ that fail to live up to their original hype, and which ultimately render the zealots’ promises as counter-productive.

Computational Intelligence: An Introduction

We apply Convolutional Networks (ConvNets) to the task of traffic sign classification as part of the GTSRB competition. ConvNets are biologically-inspired multi-stage architectures that automatically learn hierarchies of invariant features. While many popular vision approaches use hand-crafted features such as HOG or SIFT, ConvNets learn features at every level from data that are tuned to the task at hand. The traditional ConvNet architecture was modified by feeding 1st stage features in addition to 2nd stage features to the classifier. The system yielded the 2nd-best accuracy of 98.97% during phase I of the competition (the best entry obtained 98.98%), above the human performance of 98.81%, using 32×32 color input images. Experiments conducted after phase 1 produced a new record of 99.17% by increasing the network capacity, and by using greyscale images instead of color. Interestingly, random features still yielded competitive results (97.33%).

/pdf/traffic-sign-recognition-with-multi-scale-convolutional-4xfzgakcol.pdf

Traffic sign recognition with multi-scale Convolutional Networks

EvoCOMNET Contributions.- New Research in Nature Inspired Algorithms for Mobility Management in GSM Networks.- Adaptive Local Search for a New Military Frequency Hopping Planning Problem.- SS vs PBIL to Solve a Real-World Frequency Assignment Problem in GSM Networks.- Reconstruction of Networks from Their Betweenness Centrality.- A Self-learning Optimization Technique for Topology Design of Computer Networks.- A Comparative Study of Fuzzy Inference Systems, Neural Networks and Adaptive Neuro Fuzzy Inference Systems for Portscan Detection.- EvoFIN Contributions.- Evolutionary Single-Position Automated Trading.- Genetic Programming in Statistical Arbitrage.- Evolutionary System for Generating Investment Strategies.- Horizontal Generalization Properties of Fuzzy Rule-Based Trading Models.- Particle Swarm Optimization for Tackling Continuous Review Inventory Models.- Option Model Calibration Using a Bacterial Foraging Optimization Algorithm.- A SOM and GP Tool for Reducing the Dimensionality of a Financial Distress Prediction Problem.- Quantum-Inspired Evolutionary Algorithms for Financial Data Analysis.- EvoHOT Contributions.- Analysis of Reconfigurable Logic Blocks for Evolvable Digital Architectures.- Analogue Circuit Control through Gene Expression.- Discovering Several Robot Behaviors through Speciation.- Architecture Performance Prediction Using Evolutionary Artificial Neural Networks.- Evolving a Vision-Driven Robot Controller for Real-World Indoor Navigation.- Evolving an Automatic Defect Classification Tool.- Deterministic Test Pattern Generator Design.- An Evolutionary Methodology for Test Generation for Peripheral Cores Via Dynamic FSM Extraction.- Exploiting MOEA to Automatically Geneate Test Programs for Path-Delay Faults in Microprocessors.- EvoIASP Contributions.- Evolutionary Object Detection by Means of Naive Bayes Models Estimation.- An Evolutionary Framework for Colorimetric Characterization of Scanners.- Artificial Creatures for Object Tracking and Segmentation.- Automatic Recognition of Hand Gestures with Differential Evolution.- Optimizing Computed Tomographic Angiography Image Segmentation Using Fitness Based Partitioning.- A GA-Based Feature Selection Algorithm for Remote Sensing Images.- An Evolutionary Approach for Ontology Driven Image Interpretation.- Hybrid Genetic Algorithm Based on Gene Fragment Competition for Polyphonic Music Transcription.- Classification of Seafloor Habitats Using Genetic Programming.- Selecting Local Region Descriptors with a Genetic Algorithm for Real-World Place Recognition.- Object Detection Using Neural Networks and Genetic Programming.- Direct 3D Metric Reconstruction from Multiple Views Using Differential Evolution.- Discrete Tomography Reconstruction through a New Memetic Algorithm.- A Fuzzy Hybrid Method for Image Decomposition Problem.- Triangulation Using Differential Evolution.- Fast Multi-template Matching Using a Particle Swarm Optimization Algorithm for PCB Inspection.- EvoMUSART Contributions.- A Generative Representation for the Evolution of Jazz Solos.- Automatic Invention of Fitness Functions with Application to Scene Generation.- Manipulating Artificial Ecosystems.- Evolved Diffusion Limited Aggregation Compositions.- Scaffolding for Interactively Evolving Novel Drum Tracks for Existing Songs.- AtomSwarm: A Framework for Swarm Improvisation.- Using DNA to Generate 3D Organic Art Forms.- Towards Music Fitness Evaluation with the Hierarchical SOM.- Evolutionary Pointillist Modules: Evolving Assemblages of 3D Objects.- An Artificial-Chemistry Approach to Generating Polyphonic Musical Phrases.- Implicit Fitness Functions for Evolving a Drawing Robot.- Free Flight in Parameter Space: A Dynamic Mapping Strategy for Expressive Free Impro.- Modelling Video Games' Landscapes by Means of Genetic Terrain Programming - A New Approach for Improving Users' Experience.- Virtual Constructive Swarm Compositions and Inspirations.- New-Generation Methods in an Interpolating EC Synthesizer Interface.- Composing Music with Neural Networks and Probabilistic Finite-State Machines.- TransFormer #13: Exploration and Adaptation of Evolution Expressed in a Dynamic Sculpture.- EvoNUM Contributions.- Multiobjective Tuning of Robust PID Controllers Using Evolutionary Algorithms.- Truncation Selection and Gaussian EDA: Bounds for Sustainable Progress in High-Dimensional Spaces.- Scalable Continuous Multiobjective Optimization with a Neural Network-Based Estimation of Distribution Algorithm.- Cumulative Step Length Adaptation for Evolution Strategies Using Negative Recombination Weights.- Computing Surrogate Constraints for Multidimensional Knapsack Problems Using Evolution Strategies.- A Critical Assessment of Some Variants of Particle Swarm Optimization.- An Evolutionary Game-Theoretical Approach to Particle Swarm Optimisation.- A Hybrid Particle Swarm Optimization Algorithm for Function Optimization.- EvoSTOC Contributions.- Memory Based on Abstraction for Dynamic Fitness Functions.- A Memory Enhanced Evolutionary Algorithm for Dynamic Scheduling Problems.- Compound Particle Swarm Optimization in Dynamic Environments.- An Evolutionary Algorithm for Adaptive Online Services in Dynamic Environment.- EvoTHEORY Contributions.- A Study of Some Implications of the No Free Lunch Theorem.- Negative Slope Coefficient and the Difficulty of Random 3-SAT Instances.- EvoTRANSLOG Contributions.- A Memetic Algorithm for the Team Orienteering Problem.- Decentralized Evolutionary Optimization Approach to the p-Median Problem.- Genetic Computation of Road Network Design and Pricing Stackelberg Games with Multi-class Users.- Constrained Local Search Method for Bus Fleet Scheduling Problem with Multi-depot with Line Change.- Evolutionary System with Precedence Constraints for Ore Harbor Schedule Optimization.

Applications of evolutionary computing: EvoWorkshops 2008: EvoCOMNET, EvoFIN, EvoHOT, EvoIASP, EvoMUSART, EvoNUM, EvoSTOC, and EvoTransLog, Naples, Italy, March 26-28, 2008. Proceedings.

Approximate circuits and approximate circuit design methodologies attracted a significant attention of researchers as well as industry in recent years. In order to accelerate the approximate circuit and system design process and to support a fair benchmarking of circuit approximation methods, we propose a library of approximate adders and multipliers called EvoApprox8b. This library contains 430 non-dominated 8-bit approximate adders created from 13 conventional adders and 471 non-dominated 8-bit approximate multipliers created from 6 conventional multipliers. These implementations were evolved by a multi-objective Cartesian genetic programming. The EvoApprox8b library provides Verilog, Matlab and C models of all approximate circuits. In addition to standard circuit parameters, the error is given for seven different error metrics. The EvoApprox8b library is available at: www.fit.vutbr.cz/research/groups/ehw/approxlib

EvoApproxSb: Library of approximate adders and multipliers for circuit design and benchmarking of approximation methods

Artificial neural networks (NN) have shown a significant promise in difficult tasks like image classification or speech recognition. Even well-optimized hardware implementations of digital NNs show significant power consumption. It is mainly due to non-uniform pipeline structures and inherent redundancy of numerous arithmetic operations that have to be performed to produce each single output vector. This paper provides a methodology for the design of well-optimized power-efficient NNs with a uniform structure suitable for hardware implementation. An error resilience analysis was performed in order to determine key constraints for the design of approximate multipliers that are employed in the resulting structure of NN. By means of a search based approximation method, approximate multipliers showing desired tradeoffs between the accuracy and implementation cost were created. Resulting approximate NNs, containing the approximate multipliers, were evaluated using standard benchmarks (MNIST dataset) and a real-world classification problem of Street-View House Numbers. Significant improvement in power efficiency was obtained in both cases with respect to regular NNs. In some cases, 91% power reduction of multiplication led to classification accuracy degradation of less than 2.80%. Moreover, the paper showed the capability of the back propagation learning algorithm to adapt with NNs containing the approximate multipliers.

Design of power-efficient approximate multipliers for approximate artificial neural networks

The paper introduces a new method for the design of real-world applications of evolvable hardware using common FPGAs (Field Programmable Gate Arrays). In order to avoid "reconfiguration problems" of current FPGAs a new virtual reconfigurable circuit, whose granularity and configuration schema exactly fit to requirements of a given application, is designed on the top of an ordinary FPGA. As an example, a virtual reconfigurable circuit is constructed to speed up the software model, which was utilized for the evolutionary design of image operators.

/pdf/virtual-reconfigurable-circuits-for-real-world-applications-2px7t7ypks.pdf

Virtual reconfigurable circuits for real-world applications of evolvable hardware

In approximate computing, the requirement of perfect functional behavior can be relaxed because some applications are inherently error resilient. Approximate circuits, which fall into the approximate computing paradigm, are designed in such a way that they do not fully implement the logic behavior given by the specification and, hence, their accuracy can be exchanged for lower area, delay or power consumption. In order to automate the design process, we propose to evolve approximate digital circuits that show a minimal error for a supplied amount of resources. The design process, which is based on Cartesian genetic programming (CGP), can be repeated many times in order to obtain various tradeoffs between the accuracy and area. A heuristic seeding mechanism is introduced to CGP, which allows for improving not only the quality of evolved circuits, but also reducing the time of evolution. The efficiency of the proposed method is evaluated for the gate as well as the functional level evolution. In particular, approximate multipliers and median circuits that show very good parameters in comparison with other available implementations were constructed by means of the proposed method.

Evolutionary Approach to Approximate Digital Circuits Design

1 Introduction.- 1.1 Natural Computing.- 1.1.1 Soft Computing.- 1.1.2 Quantum Computing.- 1.1.3 DNA Computing.- 1.1.4 Membrane Computing.- 1.2 Bioinspired Hardware.- 1.3 Motivation for Research.- 2 Reconfigurable Hardware.- 2.1 Digital Cicuits.- 2.2 Digital Circuit Design.- 2.3 Field Programmable Gate arrays.- 2.3.1 Architecture of FPGAs.- 2.3.2 The XC4000 Family.- 2.3.3 ThE Virtex Family.- 2.3.4 The XC6200 Family.- 2.3.5 Atmel FPGAs.- 2.3.6 Features of FPGAs.- 2.4 Hardware Reused as Software.- 2.5 Reconfigurable Computing.- 2.6 Nanotechnology.- 2.7 Cell Matrix.- 2.8 Summary.- 3 Evolutionary Algorithms.- 3.1 Introduction.- 3.2 Variant of Evolutionary Algorithms.- 3.2.1 Genetic Algorithms.- 3.2.2 Genetic Programming.- 3.2.3 Evolutionary Strategies.- 3.2.4 Evolutionary Programming.- 3.3 Some Other Features of Evolutionary Algorithms.- 3.3.1 Parallel Implementations.- 3.3.2 Dynamic Fitness Function.- 3.4 Evolutionary Design and Optimization.- 3.5 The Evolutionary Algorithm Design.- 3.5.1 Missing Theories.- 3.5.2 The Design Strategies.- 3.6 Formal Approach.- 3.7 Summary.- 4 Evolvable Hardware.- 4.1 Basic Concept.- 4.2 Cartesian Genetic Programming.- 4.3 Features of Cartesian Genetic Programming.- 4.3.1 Redundancy and Neutrality.- 4.3.2 Fitness Landscape Analysis.- 4.3.3 Implementation Issues.- 4.4 From Chromosome to Fitness Value.- 4.4.1 Representation.- 4.4.2 Platforms for Circuit Evolution.- 4.4.3 Circuit Evaluation.- 4.5 Fitness Function.- 4.5.1 Fitness Function and Circuit Behavior.- 4.5.2 Evolutionary Circuit Design: Static Fitness Function.- 4.5.3 Evolvable Hardware: Dynamic Fitness Function.- 4.5.4 Discussion.- 4.6 Applications and Degree of Hardware Implementation.- 4.7 Promising Results.- 4.8 Major Current Problems and Potential Solutions.- 4.8.1 Scalability of Representaion.- 4.8.2 SCalability of Fitnes Evaluation.- 4.8.3 Robustness of the Evolved Circuits.- 4.8.4 Applications in Dynamic Environments.- 4.9 Summary.- 5 Towards Evolvable Components.- 5.1 Component Approach to Problem Solving.- 5.2 Evolvable Components.- 5.2.1 System Decomposition.- 5.2.2 Interface.- 5.3 Hardware Implementation.- 5.3.1 Evolvable Componenets.- 5.3.2 Environment.- 5.3.3 Communication Betweem Evolvable Component and Environment.- 5.4 Extension of Evolvable Components.- 5.5 Summary.- 6 Evolvable Computational Machines.- 6.1 Computational Machines and Evolutionary Design.- 6.2 Cellular Automata.- 6.2.1 Basic Model.- 6.2.2 Evolvable Non-Uniform CEllular Automaton.- 6.2.3 An example: Evolvable Non-Uniform Cellular Automaton as a Sequence Generator.- 6.3 General Evolvable Computational Machine.- 6.4 Dynamic Environment.- 6.5 Evolvable Computational System.- 6.5.1 Formal Definition.- 6.5.2 An example: Formal Description of a Simple Image Compression.- 6.6 Properties of Evolvable Machines.- 6.6.1 On the Computation of Evolvable Machines.- 6.6.2 Mappings g and f.- 6.6.3 Changing Fitness Fuction.- 6.7 The Computational Power.- 6.7.1 The Turing Machine and the Church Turing Thesis.- 6.7.2 Beyond the Turing Machines.- 6.7.3 A New Paradigm.- 6.7.4 Site Machine.- 6.7.5 the Power of an Evolvable System.- 6.7.6 Discussion.- 6.8 Summary.- 7 An Evolvable Component for Image Pre-processing.- 7.1 Motivation and Problem Specification.- 7.2 The Image Filter Design.- 7.2.1 Types of Noise Considered for Testing.- 7.2.2 Convnetional Approaches.- 7.2.3 Implementation of FPGAs.- 7.2.4 A Brief Survey of Evolutionary Approaches.- 7.3 Analysis of Reconfigurability and Size of the Search Space.- 7.3.1 Elementary Measures.- 7.3.2 Cartesian Genetic Programming in Hardware.- 7.3.3 Cartesian Genetic Programming at the Fuctional Level.- 7.4 Evolutionary Design: Experimental Framework.- 7.4.1 Reconfigurable Circuit.- 7.4.2 Evolutionary Algorithms.- 7.4.3 Fitness Function.- 7.5 Filters for Smoothing.- 7.5.1 The Results.- 7.5.2 Discussion.- 7.6 Other Image Operators.- 7.6.1 "Salt and Pepper" Noise Filters.- 7.6.2 Random Shot-Noise Filters.- 7.6.3 Edge Detectors.- 7.7 Dynamics Environment.- 7.7.1 Experimental Setup.- 7.7.2 The Results in Tables 7.9 and 7.10.- 7.7.3 Discussion.- 7.8 A Note on a Single Filter Design.- 7.9 Summary.- Virtual Reconfigurable Devices.- 8.1 Chip on Top of a Chip.- 8.2 Architecture of Virtual Reconfigurable Circuits.- 8.2.1 Overview.- 8.2.2 Routing Logic and Configuration Memory.- 8.2.3 Configuration Options.- 8.3 Implementation Costs.- 8.4 Speeding up the Evolutionary Design.- 8.5 Genetic Unit.- 8.6 Physical Realization.- 8.7 Discussion.- 8.8 Summary.- 9 Concluding Statements.- 9.1 The Approach.- 9.2 The Obtained Results.- 9.3 Future Work.- References.

Lukas Sekanina

Papers

EvoApproxSb: Library of approximate adders and multipliers for circuit design and benchmarking of approximation methods

Design of power-efficient approximate multipliers for approximate artificial neural networks

Virtual reconfigurable circuits for real-world applications of evolvable hardware

Evolutionary Approach to Approximate Digital Circuits Design

Evolvable Components: From Theory to Hardware Implementations