1 Basic Concepts.- 1.1 Industrial Vision Systems.- 1.1.1 Justification.- 1.1.2 Limitations of Present Systems.- 1.1.3 Flexible Manufacturing Systems.- 1.1.4 Process Control.- 1.2 Systems Engineering.- 1.2.1 Importance of Context.- 1.2.2 Industrial Examples.- 1.3 Intelligent Vision.- 1.3.1 Heuristics and Algorithms.- 1.3.2 Artificial Intelligence (AI) Languages.- 1.4 Book Outline.- 2 Basic Machine Vision Techniques.- 2.1 Representations of Images.- 2.2 Elementary Image Processing Functions.- 2.2.1 Monadic, Point-by-point Operators.- 2.2.2 Dyadic, Point-by-point Operators.- 2.2.3 Local Operators.- 2.2.4 Linear Local Operators.- 2.2.5 Non-linear Local Operators.- 2.2.6 N-tuple Operators.- 2.2.7 Edge Effects.- 2.2.8 Intensity Histogram.- 2.3 Binary Images.- 2.3.1 Measurements on Binary Images.- 2.3.2 Shape Descriptors.- 2.4 Binary Mathematical Morphology.- 2.4.1 Opening and Closing Operations.- 2.4.2 Structuring Element Decomposition.- 2.5 Grey Scale Morphology.- 2.6 Global Image Transforms.- 2.6.1 Hough Transform.- 2.6.2 Two-dimensional Discrete Fourier Transform.- 2.7 Texture Analysis.- 2.7.1 Statistical Approaches.- 2.7.2 Co-occurrence Matrix Approach.- 2.7.3 Structural Approaches.- 2.7.4 Morphological Texture Analysis.- 2.8 Implementation Considerations.- 2.8.1 Morphological System Implementation.- 2.9 Commercial Devices.- 2.9.1 Plug-in Boards: Frame-stores.- 2.9.2 Plug-in Boards: Dedicated Function.- 2.9.3 Self-contained Systems.- 2.9.4 Turn-key Systems.- 2.9.5 Software.- 2.10 Further Remarks.- 3 Intelligent Image Processing.- 3.1 Interactive Image Processing.- 3.1.1 Modus Operandi.- 3.1.2 Prototyping Inspection Systems.- 3.1.3 Building Simple Programs.- 3.1.4 Interaction and Prolog.- 3.2 Introducing PrologH-.- 3.3 Review of Prolog.- 3.3.1 Sample Program.- 3.3.2 Sample Queries.- 3.4 The Nature of Prolog+.- 3.5 Prolog+Programs.- 3.5.1 Recognising Bakewell Tarts.- 3.5.2 Recognising Printed Letters.- 3.5.3 Identifying Table Cutlery.- 3.5.4 Analysing all Visible Objects.- 3.5.5 Recognising a Table Place Setting.- 3.6 Abstract Concepts in Prolog+.- 3.6.1 Describing a Simple Package.- 3.6.2 Abstract Spatial Relationships.- 3.6.3 Geometric Figures.- 3.7 Implementation of Prolog+.- 3.7.1 The # Operator.- 3.8 Comments.- 4 Enhanced Intelligent Systems.- 4.1 Prolog+ Environment: A Tool-box for Machine Vision.- 4.1.1 Defining New Predicate Names.- 4.1.2 Default Values for Arguments.- 4.1.3 Useful Operators.- 4.1.4 Program Library.- 4.1.5 Auto-start.- 4.1.6 Interactive Mode.- 4.1.7 User Extendible Pull-down Menus.- Mechanism for Extending Menus.- 4.1.8 Command Keys.- 4.1.9 Graphical Display of a Robot Work Cell.- 4.1.10 Speech Synthesis and Recorded Speech.- 4.1.11 On-line HELP.- 4.1.12 Cursor.- 4.1.13 Automatic Script Generation and Optimisation.- 4.1.14 Linking to Other Programs.- HyperCard Controller for a Flexible Inspection Cell.- 4.2 Understanding Simple Spoken Instructions.- 4.2.1 Speech Recognition.- 4.2.2 Natural Language Understanding.- 4.2.3 Automatically Building a Pull-down Menu.- 4.2.4 Understanding NL Commands for an (X, Y, )-table.- 4.2.5 Sample Sentences.- 4.2.6 Interpreting the Parser Output.- 4.2.7 Review.- 4.3 Aids for Designing Vision Systems.- 4.3.1 Lighting Advisor.- Stack Structure.- Search Mechanisms.- Remarks About the Lighting Advisor.- 4.3.2 Other Design Aids for Machine Vision.- 4.4 Multi-camera Systems.- 4.4.1 Multiplexed-video Systems.- 4.4.2 Networked Vision Systems.- 4.4.3 Master-Slave System Organisation.- 4.4.4 Remote Queries.- Interactive Operation of the Remote Process.- 4.4.5 Blackboard.- Master and Slave Program Elements.- 4.4.6 Controlling the Master-Slave System.- Starting the System.- Stopping a Slave.- Passing a Message to the Slave.- Receiving Data from a Slave.- Slave Program.- Blackboard (Snapshot of Database, Changing Constantly).- 4.4.7 Crash Recovery.- Programming the Slave from the Master.- 4.5 Comments.- 5 Controlling External Devices.- 5.1 Devices and Signals.- 5.2 Protocols and Signals.- 5.2.1 Interfacing to Commercial Systems.- 5.3 Programmable Logic Controller.- 5.4 General Purpose Interface Unit.- 5.4.1 Motivation for the Design.- 5.4.2 Hardware Organisation.- 5.4.3 Programs.- 5.4.4 Digression on Lighting.- 5.4.5 Languages for Robotics.- 5.5 Flexible Inspection Cell, Design Issues.- 5.5.1 Lighting Arrangement.- 5.5.2 Mechanical Handling.- 5.5.3 Cameras and Lenses.- 5.5.4 MMB-Host Interface Protocol.- 5.5.5 Additional Remarks.- 5.5.6 HyperCard Control Software for the FIC.- 5.6 Prolog+ Predicates for Device Control.- 5.7 System Calibration.- 5.7.1 FIC Calibration Procedure (Overhead Camera).- 5.7.2 Calibration, SCARA and Gantry Robots (Overhead Camera).- 5.7.3 Calibration Procedure (Overhead Narrow-view Camera).- 5.7.4 Calibration Procedure (Side Camera).- 5.8 Picking up a Randomly Placed Object (Overhead Camera).- 5.8.1 Program.- 5.9 Grippers.- 5.9.1 Suction Gripper.- 5.9.2 Magnetic Gripper.- 5.9.3 Multi-Finger Gripper.- 5.9.4 Further Remarks.- 5.10 Summary.- 6 Colour Image Recognition.- 6.1 Introduction.- 6.2 Applications of Coarse Colour Discrimination.- 6.3 Why is a Banana Yellow?.- 6.4 Machines for Colour Discrimination.- 6.4.1 Optical Filters.- 6.4.2 Colour Cameras.- 6.4.3 Light Sources for Colour Vision.- 6.4.4 Colour Standards.- 6.5 Ways of Thinking about Colour.- 6.5.1 Opponent Process Representation of Colour.- 6.5.2 YIQ Colour Representation.- 6.5.3 HSI, Hue Saturation and Intensity.- 6.5.4 RGB Colour Space: Colour Triangle.- 6.5.5 1-Dimensional Histograms of RGB Colour Separations.- 6.5.6 2-Dimensional Scattergrams.- 6.5.7 Colour Scattergrams.- 6.6 Programmable Colour Filter (PCF).- 6.6.1 Implementation of the PCF.- 6.6.2 Programming the PCF.- 6.6.3 Recognising a Single Colour.- 6.6.4 Noise Effects.- 6.6.5 Recognising Multiple Colours.- 6.6.6 Pseudo-Colour Display for the PCF.- 6.6.7 Recent Teaching of the PCF Dominates.- 6.6.8 Prolog+Software for Operating the PCF.- Plot Colour Scattergram.- Draw Colour Triangle Outline.- Clear LUT.- Store Current LUT.- Reload Stored PCF.- Reverting to Monochrome Operation.- 6.6.9 Programming the PCF using the Colour Scattergram.- 6.6.10 Programming the PCF by Image Processing.- 6.6.11 "Hue" PCF.- 6.6.12 Analysing Output of the Hue PCF.- 6.6.13 "Segmented" PCF.- 6.6.14 Measuring Colour Similarity and Saturation.- 6.6.15 Detecting Local Colour Changes.- 6.6.16 Colour Generalisation.- 6.7 Colour Recognition in Prolog+ Programs.- 6.7.1 Counting Coloured Objects.- 6.7.2 Recognising a Polychromatic Logo, Program 1.- 6.7.3 Recognising a Polychromatic Logo, Program 2.- 6.7.4 Recognising a Polychromatic Logo, Program 3.- 6.7.5 Multiple Exemplar Approach to Recognition.- 6.7.6 Learning Proportions of Colours in a Scene.- 6.7.7 Superior Program for Learning Colour Proportions.- 6.7.8 Teaching the PCF by Showing.- 6.7.9 Template Matching of Colour Images.- 6.7.10 Using Colour for Object Orientation.- 6.7.11 Approximating an Image by a Set of Overlapping Discs.- 6.7.12 Interpreting Resistor and Capacitor Colour Codes.- 6.8 Discussion and Conclusions.- 7 Applications of Intelligent Vision.- 7.1 Recognition of Printed Patterns.- 7.1.1 Non-picture Playing Cards.- 7.1.2 "Stars".- 7.1.3 "Smiley Faces".- 7.1.4 Alphanumeric Characters.- Program.- Comments.- Logical and Analogue Shape Measurements.- 7.2 Manipulation of Planar Objects.- 7.2.1 Assumptions.- 7.2.2 Significance.- 7.2.3 Simple Shape Measurements.- 7.2.4 Learning and Recognition.- 7.2.5 Program Listing.- 7.2.6 Sample Output of Recognition Phase.- 7.3 Packing and Depletion.- 7.3.1 Geometric Packer Implementation.- 7.3.2 Heuristic Packing Techniques.- Blob Packing.- Polygon Packing.- 7.3.3 Performance Measures.- Predicates.- 7.3.4 Robot Gripper Considerations.- 7.3.5 Packing Scenes with Defective Regions.- 7.3.6 Discussion.- 7.4 Handedness of Mirror-Image Components.- 7.4.1 Handedness and Chirality.- Relating Chirality and Handedness.- 7.4.2 Concavity Trees.- Formal Definition.- Generating Concavity Trees.- Sample Concavity Trees.- Canonical Form of Concavity Trees.- Program to find Chirality.- 7.4.3 Properties of Concavity Trees.- Instability.- 7.4.4 Simpler Tests for Chirality.- Second Program.- Third Program.- Fourth Program.- Fifth Program.- 7.5 Telling the Time.- 7.5.1 Significance.- 7.5.2 Simplifying Assumptions.- 7.5.3 Lighting.- 7.5.4 First Program.- 7.5.5 Other Methods.- 7.5.6 Concluding Remarks.- 7.6 Food and Agricultural Products.- 7.6.1 Objective.- 7.6.2 Industrial Relevance.- 7.6.3 Product Shape, Two-dimensions.- Image Acquisition.- Rectangular and Circular Biscuits.- Slices of Bread.- Locating the Base and Determining Orientation.- Locating Straight Sides.- Measuring Overspill.- Radius of Curvature of Top Edge.- 7.6.4 Analysing the 3D Structure of an Uncut Loaf.- 8 Concluding Remarks.- References.- A Proverbs, Opinions and Folklore.- B Factors to be Considered when Designing a Vision System.- C General Reference Material.- D PIP - Software Implementation of Prolog+.- E Prolog+ and PIP Commands.- Glossary of Terms.- Index of Predicates, Operators and Grammar Rules.

Intelligent Vision Systems for Industry

This paper describes the application of SKIPSM (Separated-Kernel Image Processing using Finite State Machines) to binary morphology. In comparison with conventional hardware-based and software-based approaches, SKIPSM allow implementation at higher speeds and/or lower hardware cost. The key theoretical developments upon which this improved performance is based are the separation of 2D binary morphological image processing operations into a row operation followed by a column operation, the formulation of these row and column operations in a form compatible with pipelined operation, the implementation of the resulting operations as simple finite-state machine, and the automated generation of the finite-state machine configuration data. Some features of SKIPSM, as applied to binary morphology, are as follows: (1) The structuring elements (SEs) can be large (25 X 25 and larger) and arbitrary (with 'holes' and nonconvex shapes). (2) All types of morphology operations can be performed. (3) Multiple related or unrelated SEs can be applied simultaneously in a single pipeline pass. (4) Speed increases and/or neighborhood size increases by factors of 100 or more can be achieved. (5) Corresponding 'speedups' can be achieved in software-based implementations. (6) Inexpensive off-the-shelf 'chips' can be configured to carry out the same operations as expensive conventional hardware. (7) The user specifies the Se or set of simultaneous SEs. All other steps are automated. This paper includes some simple examples of the results and give implementation guidelines based on Se size and shape.

Application of SKIPSM to binary morphology

An overview of SKIPSM (eparated-eme1 Jmage j.rocessing using Finite state Machines), a powerful new way to implement many standard image processing operations, is presented in a companion paper.' Other applications are presented in four other companion aper235 In comparison with conventional hardware-based and software-based approaches, SKIPSM allows implementation at higher speeds and/or lower hardware cost. The key theoretical developments upon which this improved performance is based ai . the separation of 2.-D binary image processing operations into a row operation followed by a column operation, . the formulation of these row and column operations in a form compatible with pipelined operation, . the implementation of the resulting operations as simple finite-state machines, and I the automated generation of the finite-state machine configuration data. This paper presents a general method for carrying out binary template matching, which is useful for image analysis in general and automated visual inspection and quality control in particular. Some key features of SKIPSM, as applied to binary template matching, are as follows: . Binary template matching with large, arbitrary templates can be implemented. Templates up to 35x35 and even larger are readily applied in a single pipelined pass. S Multipletemplates can be applied simultaneously in a single pass. . The user specifies the template or templates. All other steps can be automated. Speed increases and/or neighborhood size increases by factors of 100 or more can be achieved using conventional pipelined hardware in this new way. Alternatively, inexpensive off-the-shelf "chips" can be configured to carry out the same operations as more expensive conventional image processing hardware. Corresponding "speedups" are achieved in software-based implementations. This paper includes some simple examples of the results and gives implementation feasibility guidelines.

 KEYWORDS: image processing, binary template matching, real-time, implementations, finite-state machines, inspection

Application of SKIPSM to binary template matching

Binary correlation is often used for finding specified patterns in complex binary images, especially in industrial inspection tasks such as locating the corners and/or edges of parts. As such, it is an important tool for higher-level 'intelligent' vision systems. Binary correlation is a form of binary template matching which provides a numerical value corresponding to 'degree of fit' rather than an 'all or nothing' answer. Commercially available high-speed image processing systems can readily perform this operation using linear convolvers, but such convolvers are very expensive except for very small kernels. Furthermore, linear convolvers constitute a gross 'overkill' for the relatively simple operation of binary correlation. Specialized binary convolvers have been built, but are not part of standard commercial systems. This paper describes a new pipelined implementation of binary correlation which fits into the standard SKIPSM (separated-kernel image processing using finite state machines) architecture and which can be built using standard ICs costing less than $500 total. The same approach can also be implemented in software, providing an order-of-magnitude increase in speed at no extra cost. Furthermore, this same SKIPSM architecture is highly versatile and programmable, allowing it to be software-reconfigured to perform hundreds of other pipelined image processing operations.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Application of SKIPSM to binary correlation

A new method of implementing a wide range of standard image processing operations in a real-time finite-state-machine architecture has been presented at various conferences in the past year. This architecture, under the generic name SKIPSM (separated-kernel image processing using finite state machines), has been shown to be capable of carrying out binary morphology with very large arbitrary structuring elements, simultaneous application of many binary structuring elements, gray-level morphology, binary and gray-level template matching, binary skeletonization, binary correlation, row and column summations, and many other operations. This paper describes inexpensive hardware implementations of the SKIPSM architecture, including a daughter board compatible with commercially available pipelined image processing hardware.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

SKIPSM implementations: morphology and much, much more

Binary morphological processing operations have been shown to be useful in a range of industrial inspection applications. These operations have been available in two forms: As software-based operations on general-purpose algorithm development workstations, such as Bruce Batchelor's SUSIE and its many descendants, and as hardware-based operations on dedicated "morphology engines." Obstacles to wider use of these methods include • Limited familiarity with binary morphological processing techniques on the part of general users, • The slow operation of software-based implementations, and • The relatively high initial cost of fast special-purpose binary morphological processing hardware. This paper describes another alternative: A near-real-time implementation of binary morphological processing as part of a very large set of operations on a moderately-priced general-purpose image processing and algorithm development workstation. The workstation is based on commercially-available image processing boards, and provides a high-level operator interface. With this system, the speed depends on the size of the structuring element, but migration to real-time implementations using the same hardware family is straightforward. This implementation allows arbitrary specification of the final structuring element, with no constraints as to symmetry, connectedness, concavities, holes, etc. This eliminates one of the obstacles to the use of complex structuring elements on some dedicated morphological processing machines -- the need to find a set of simple "primitives" which, when applied sequentially, will yield the desired structuring element. The nature of the implementation also allows for a new possibility: By a change in one register, operation is converted from a strict binary morphology into "fuzzy" morphology (not grey-level morphology), in which the user can specify a "percentage fit" of structuring element to image. By this means, the structuring element can still be made to "fit" even in the presence of some "noise" pixels, without the need for separate noise-removal steps. This greatly increases the robustness and practicality of morphological methods in "real world" industrial inspection applications.

Frederick M. Waltz

Papers

Fast Implementation Of Standard And "Fuzzy" Binary Morphological Operations With Large, Arbitrary Structuring Elements