Showing papers in "Autonomous Robots in 2000"

Multiagent Systems: A Survey from a Machine Learning Perspective

Abstract: Distributed Artificial Intelligence (DAI) has existed as a subfield of AI for less than two decades. DAI is concerned with systems that consist of multiple independent entities that interact in a domain. Traditionally, DAI has been divided into two sub-disciplines: Distributed Problem Solving (DPS) focuses on the information management aspects of systems with several components working together towards a common goals Multiagent Systems (MAS) deals with behavior management in collections of several independent entities, or agents. This survey of MAS is intended to serve as an introduction to the field and as an organizational framework. A series of general multiagent scenarios are presented. For each scenario, the issues that arise are described along with a sampling of the techniques that exist to deal with them. The presented techniques are not exhaustive, but they highlight how multiagent systems can be and have been used to build complex systems. When options exist, the techniques presented are biased towards machine learning approaches. Additional opportunities for applying machine learning to MAS are highlighted and robotic soccer is presented as an appropriate test bed for MAS. This survey does not focus exclusively on robotic systems. However, we believe that much of the prior research in non-robotic MAS is relevant to robotic MAS, and we explicitly discuss several robotic MAS, including all of those presented in this issue.

A Probabilistic Approach to Collaborative Multi-Robot Localization

Abstract: This paper presents a statistical algorithm for collaborative mobile robot localization. Our approach uses a sample-based version of Markov localization, capable of localizing mobile robots in an any-time fashion. When teams of robots localize themselves in the same environment, probabilistic methods are employed to synchronize each robot's belief whenever one robot detects another. As a result, the robots localize themselves faster, maintain higher accuracy, and high-cost sensors are amortized across multiple robot platforms. The technique has been implemented and tested using two mobile robots equipped with cameras and laser range-finders for detecting other robots. The results, obtained with the real robots and in series of simulation runs, illustrate drastic improvements in localization speed and accuracy when compared to conventional single-robot localization. A further experiment demonstrates that under certain conditions, successful localization is only possible if teams of heterogeneous robots collaborate during localization.

Design and Control of Autonomous Underwater Robots: A Survey

Abstract: During the 1990s, numerous worldwide research and development activities have occurred in underwater robotics, especially in the area of autonomous underwater vehicles (AUVs). As the ocean attracts great attention on environmental issues and resources as well as scientific and military tasks, the need for and use of underwater robotic systems has become more apparent. Great efforts have been made in developing AUVs to overcome challenging scientific and engineering problems caused by the unstructured and hazardous ocean environment. In the 1990s, about 30 new AUVs have been built worldwide. With the development of new materials, advanced computing and sensory technology, as well as theoretical advancements, R&D activities in the AUV community have increased. However, this is just the beginning for more advanced, yet practical and reliable AUVs. This paper surveys some key areas in current state-of-the-art underwater robotic technologies. It is by no means a complete survey but provides key references for future development. The new millennium will bring advancements in technology that will enable the development of more practical, reliable AUVs.

Using Real-Time Stereo Vision for Mobile Robot Navigation

Abstract: This paper describes a working vision-based mobile robot that navigates and autonomously explores its environment while building occupancy grid maps of the environment. We present a method for reducing stereo vision disparity images to two-dimensional map information. Stereo vision has several attributes that set it apart from other sensors more commonly used for occupancy grid mapping. We discuss these attributes, the errors that some of them create, and how to overcome them. We reduce errors by segmenting disparity images based on continuous disparity surfaces to reject “spikes” caused by stereo mismatches. Stereo vision processing and map updates are done at 5 Hz and the robot moves at speeds of 300 cm/s.

Coverage of Known Spaces: The Boustrophedon Cellular Decomposition

Abstract: Coverage path planning is the determination of a path that a robot must take in order to pass over each point in an environment. Applications include de-mining, floor scrubbing, and inspection. We developed the boustrophedon cellular decomposition, which is an exact cellular decomposition approach, for the purposes of coverage. Essentially, the boustrophedon decomposition is a generalization of the trapezoidal decomposition that could allow for non-polygonal obstacles, but also has the side effect of having more “efficient” coverage paths than the trapezoidal decomposition. Each cell in the boustrophedon decomposition is covered with simple back and forth motions. Once each cell is covered, then the entire environment is covered. Therefore, coverage is reduced to finding an exhaustive path through a graph which represents the adjacency relationships of the cells in the boustrophedon decomposition. This approach is provably complete and experiments on a mobile robot validate this approach.

A Gesture Based Interface for Human-Robot Interaction

Abstract: Service robotics is currently a highly active research area in robotics, with enormous societal potential. Since service robots directly interact with people, finding “natural” and easy-to-use user interfaces is of fundamental importance. While past work has predominately focussed on issues such as navigation and manipulation, relatively few robotic systems are equipped with flexible user interfaces that permit controlling the robot by “natural” means. This paper describes a gesture interface for the control of a mobile robot equipped with a manipulator. The interface uses a camera to track a person and recognize gestures involving arm motion. A fast, adaptive tracking algorithm enables the robot to track and follow a person reliably through office environments with changing lighting conditions. Two alternative methods for gesture recognition are compared: a template based approach and a neural network approach. Both are combined with the Viterbi algorithm for the recognition of gestures defined through arm motion (in addition to static arm poses). Results are reported in the context of an interactive clean-up task, where a person guides the robot to specific locations that need to be cleaned and instructs the robot to pick up trash.

CONRO: Towards Deployable Robots with Inter-Robots Metamorphic Capabilities

Abstract: Metamorphic robots are modular robots that can reconfigure their shape. Such capability is desirable in tasks such as earthquake search and rescue and battlefield surveillance and scouting, where robots must go through unexpected situations and obstacles and perform tasks that are difficult for fixed-shape robots. The capabilities of the robots are determined by the design specification of their modules. In this paper, we present the design specification of a CONRO module, a small, self-sufficient and relatively homogeneous module that can be connected to other modules to form complex robots. These robots have not only the capability of changing their shape (intra-robot metamorphing) but also can split into smaller robots or merge with other robots to create a single larger robot (inter-robot metamorphing), i.e., CONRO robots can alter their shape and their size. Thus, heterogeneous robot teams can be built with homogeneous components. Furthermore, the CONRO robots can separate the reconfiguration stage from the locomotion stage, allowing the selection of configuration-dependent gaits. The locomotion and automatic inter-module docking capabilities of such robots were tested using tethered prototypes that can be reconfigured manually. We conclude the paper discussing the future work needed to fully realize the construction of these robots.

Heterogeneous Teams of Modular Robots for Mapping and Exploration

Abstract: In this article, we present the design of a team of heterogeneous, centimeter-scale robots that collaborate to map and explore unknown environments The robots, called Millibots, are configured from modular components that include sonar and IR sensors, camera, communication, computation, and mobility modules Robots with different configurations use their special capabilities collaboratively to accomplish a given task For mapping and exploration with multiple robots, it is critical to know the relative positions of each robot with respect to the others We have developed a novel localization system that uses sonar-based distance measurements to determine the positions of all the robots in the group With their positions known, we use an occupancy grid Bayesian mapping algorithm to combine the sensor data from multiple robots with different sensing modalities Finally, we present the results of several mapping experiments conducted by a user-guided team of five robots operating in a room containing multiple obstacles

“Gastrobots”—Benefits and Challenges of Microbial Fuel Cells in FoodPowered Robot Applications

Abstract: The present paper introduces the concept of i>Gastrobots, a class of intelligent machines that derive their operational power by exploiting the digestion of real food. Robots of this type could potentially be made self sufficient with just an input of natural food, water and air, and as such would be ideal for a host of applications that demand “living off the land” during fully autonomous “start and forget” missions. One possible method of powering such machines is through the use of a i>Microbial Fuel Cell (MFC), which can directly convert various food substrates into electricity. MFCs offer specific advantages over other renewable energy conversion methods, such as photovoltaic panels, principally in terms of compact configuration and 24 hour operation. Incorporating such a MFC into a robot is not a trivial matter, and many challenges have to be addressed and overcome. Previous work has concentrated on improving MFC efficiency, and has not emphasized long term fully autonomous operation, an essential requirement for adoption in mobile robots. The present paper attempts to discuss the applicability of current MFC technology to mobile robot operation, and enumerates many of the existing shortcomings, while suggesting possible ways ahead.

Hierarchic Social Entropy: An Information Theoretic Measure of Robot Group Diversity

Abstract: As research expands in multiagent intelligent systems, investigators need new tools for evaluating the artificial societies they study. It is impossible, for example, to correlate heterogeneity with performance in multiagent robotics without a quantitative metric of diversity. Currently diversity is evaluated on a bipolar scale with systems classified as either heterogeneous or homogeneous, depending on whether any of the agents differ. Unfortunately, this labeling doesn't tell us much about the extent of diversity in heterogeneous teams. How can it be determined if one system is more or less diverse than anotherq Heterogeneity must be evaluated on a continuous scale to enable substantive comparisons between systems. To enable these types of comparisons, we introduce: (1) a continuous measure of robot behavioral difference, and (2) hierarchic social entropy, an application of Shannon's information entropy metric to robotic groups that provides a continuous, quantitative measure of robot team diversity. The metric captures important components of the meaning of diversity, including the number and size of behavioral groups in a society and the extent to which agents differ. The utility of the metrics is demonstrated in the experimental evaluation of multirobot soccer and multirobot foraging teams.

A Short History of Cleaning Robots

Abstract: The definition of the desired functions and the design of an ultimate versatile personal robot is an ongoing debate. Meanwhile, however, precursors of this yet to evolve species are well on their way to become commercial products. Cleaning robots for public environments as well as for private households seem to be able to provide the breakthrough which the designers of non-industrial robot systems have long awaited. This survey describes a selection of 30 different cleaning robots, with the first developments reaching back more than 15 years. With a few exceptions we have focused on floor cleaning, in particular indoor floor cleaning. We describe a variety of scrubbing and vacuuming robots which were developed for this task. The described systems range from heavy, large, and expensive industrial cleaning vehicles to small-size, light-weight, low-cost household devices. The survey does not include, however, systems for cleaning facades of buildings, or windows, or production tools. Although not all of the 30 cleaning robots abovementioned have yet reached the state of commercial products, their number alone certainly reflects the expectations regarding the economic value associated with the automation of cleaning tasks. In Europe only the estimates for the market for cleaning services range up to the order of USd 100 billion per year. It is therefore not surprising that the cleaning industry and the manufacturers of cleaning devices are rather enthusiastic with respect to the automation of cleaning tasks using (semi-)autonomous mobile robot systems.

An Experimental Study of a Cooperative Positioning System

Abstract: Several position identification methods are being used for mobile robots. Dead reckoning is a popular method but due to the error accumulation from wheel slippage, its reliability is low for measurement of long distances especially on uneven surfaces. Another popular method is the landmark method, which estimates current position relative to known landmarks, but the landmark method's limitation is that it cannot be used in an uncharted environment. Thus, this paper proposes a new method called “Cooperative Positioning System (CPS)” that is able to overcome these shortcomings. The main concept of CPS is to divide the robots into two groups, A and B where group A remains stationary and acts as a landmark while group B moves and then group B stops and acts as a landmark for group A. This process is repeated until the target position is reached. Compared with dead reckoning, CPS has a far lower accumulation of positioning errors, and can also work in three dimensions. Furthermore, CPS employs inherent landmarks and therefore can be used in uncharted environments unlike the landmark method. In this paper, we introduce the basic concept of CPS and its positioning principle. Next, we outline a second prototype CPS machine model (CPS-II) and discuss the method of position estimation using the variance of positioning error and weighted least squares method. Position identification experiments using the CPS-II model give a positioning accuracy of 0.12% for position and 0.32 degree for attitude after the robots traveled a distance of 21.5 m.

Grounded Symbolic Communication between Heterogeneous Cooperating Robots

Abstract: In this paper, we describe the implementation of a heterogeneous cooperative multi-robot system that was designed with a goal of engineering a grounded symbolic representation in a bottom-up fashion. The system comprises two autonomous mobile robots that perform cooperative cleaning. Experiments demonstrate successful purposive navigation, map building and the symbolic communication of locations in a behavior-based system. We also examine the perceived shortcomings of the system in detail and attempt to understand them in terms of contemporary knowledge of human representation and symbolic communication. From this understanding, we propose the Adaptive Symbol Grounding Hypothesis as a conception for how symbolic systems can be envisioned.

An Autonomous Water Vapor Plume Tracking Robot Using Passive Resistive Polymer Sensors

Abstract: A simple reactive robot is described which is capable of tracking a water vapor plume to its source The robot acts completely within the plume and is endowed with no deliberate information about wind direction or speed, yet accurately tracks the plume upstream The robot's behavior, results from the behavior of simple resistive polymer sensors and their strategic placement on the robot's body

Lifelong Adaptation in Heterogeneous Multi-Robot Teams: Response to Continual Variation in Individual Robot Performance

Abstract: Generating teams of robots that are able to perform their tasks over long periods of time requires the robots to be responsive to continual changes in robot team member capabilities and to changes in the state of the environment and mission. In this article, we describe the L-ALLIANCE architecture, which enables teams of heterogeneous robots to dynamically adapt their actions over time. This architecture, which is an extension of our earlier work on ALLIANCE, is a distributed, behavior-based architecture aimed for use in applications consisting of a collection of independent tasks. The key issue addressed in L-ALLIANCE is the determination of which tasks robots should select to perform during their mission, even when multiple robots with heterogeneous, continually changing capabilities are present on the team. In this approach, robots monitor the performance of their teammates performing common tasks, and evaluate their performance based upon the time of task completion. Robots then use this information throughout the lifetime of their mission to automatically update their control parameters. After describing the L-ALLIANCE architecture, we discuss the results of implementing this approach on a physical team of heterogeneous robots performing proof-of-concept box pushing experiments. The results illustrate the ability of L-ALLIANCE to enable lifelong adaptation of heterogeneous robot teams to continuing changes in the robot team member capabilities and in the environment.

The Inchworm Robot: A Multi-Functional System

Abstract: We wish for robots to manipulate objects and move flexibly in three-dimensional spaces. We describe a robot that can move on a web of surfaces oriented around arbitrary directions in three-space and a set of control algorithms that implements motion in three-dimensions. The robot can manipulate objects in three dimensions while moving, by using the same set of physical resources and control algorithms. This robot is an inchworm-like robot with a simple, modular, and flexible design. Finally, we discuss our experiments.

Tracking Multiple Moving Objects for Real-Time Robot Navigation

Abstract: This paper proposes a method for detecting and tracking the motion of a large number of dynamic objects in crowded environments, such as concourses in railway stations or airports, shopping malls, or convention centers. With this motion information, a mobile vehicle is able to navigate autonomously among moving obstacles, operating at higher speeds and using more informed locomotion strategies that perform better than simple reactive manoeuvering strategies. Unlike many of the methods for motion detection and tracking discussed in the literature, our approach is not based on visual imagery but uses 2D range data obtained using a laser rangefinder. The direct availability of range information contributes to the real-time performance of our approach, which is a primary goal of the project, since the purpose of the vehicle is the transport of humans in crowded areas. Motion detection and tracking of dynamic objects is done by constructing a sequence of temporal lattice maps. These capture the time-varying nature of the environment, and are denoted as time-stamp maps. A time-stamp map is a projection of range information obtained over a short interval of time (a scan) onto a two-dimensional grid, where each cell which coincides with a specific range value is assigned a time stamp. Based on this representation, we devised two algorithms for motion detection and motion tracking. The approach is very efficient, with a complete cycle involving both motion detection and tracking taking 6 ms on a Pentium 166 MHz. The system has been demonstrated on an intelligent wheelchair operating in railway stations and convention centers during rush hour.

Outdoor Visual Position Estimation for Planetary Rovers

Abstract: This paper describes (1) a novel, effective algorithm for outdoor visual position estimations (2) the implementation of this algorithm in the Viper systems and (3) the extensive tests that have demonstrated the superior accuracy and speed of the algorithm. The Viper system (i>Visual i>Position i>Estimator for i>Rovers) is geared towards robotic space missions, and the central purpose of the system is to increase the situational awareness of a rover operator by presenting accurate position estimates. The system has been extensively tested with terrestrial and lunar imagery, in terrains ranging from moderate—the rounded hills of Pittsburgh and the high deserts of Chile—to rugged—the dramatic relief of the Apollo 17 landing site—to extreme—the jagged peaks of the Rockies. Results have consistently demonstrated that the visual estimation algorithm estimates position with an accuracy and reliability that greatly surpass previous work.

Cleaning and Household Robots: A Technology Survey

Abstract: This paper describes some of the main technology areas that have been actually used in the development of cleaning robots. The approach taken in this survey is to examine the characteristics of cleaning robots that have made successful laboratory demonstration or have become commercial products. We then identify the technology approach followed by the authors, and group their contributions in a few general areas. The result is a summary of used approaches to the solution of difficult, albeit very practical, problems in the area of autonomous execution of cleaning tasks.

Intelligent Learning for Deformable Object Manipulation

Abstract: The majority of manipulation systems are designed with the assumption that the objects being handled are rigid and do not deform when grasped. This paper addresses the problem of robotic grasping and manipulation of 3-D deformable objects, such as rubber balls or bags filled with sand. Specifically, we have developed a generalized learning algorithm for handling of 3-D deformable objects in which prior knowledge of object attributes is not required and thus it can be applied to a large class of object types. Our methodology relies on the implementation of two main tasks. Our first task is to calculate deformation characteristics for a non-rigid object represented by a physically-based model. Using nonlinear partial differential equations, we model the particle motion of the deformable object in order to calculate the deformation characteristics. For our second task, we must calculate the minimum force required to successfully lift the deformable object. This minimum lifting force can be learned using a technique called ‘iterative lifting’. Once the deformation characteristics and the associated lifting force term are determined, they are used to train a neural network for extracting the minimum force required for subsequent deformable object manipulation tasks. Our developed algorithm is validated with two sets of experiments. The first experimental results are derived from the implementation of the algorithm in a simulated environment. The second set involves a physical implementation of the technique whose outcome is compared with the simulation results to test the real world validity of the developed methodology.

Robot Awareness in Cooperative Mobile Robot Learning

Abstract: Most of the straight-forward learning approaches in cooperative robotics imply for each learning robot a state space growth exponential in the number of team members. To remedy the exponentially large state space, we propose to investigate a less demanding cooperation mechanism—i.e., various levels of awareness—instead of communication. We define awareness as the perception of other robots locations and actions. We recognize four different levels (or degrees) of awareness which imply different amounts of additional information and therefore have different impacts on the search space size (T(0), T(1), T(N), o(N),^1 where N is the number of robots in the team). There are trivial arguments in favor of avoiding binding the increase of the search space size to the number of team members. We advocate that, by studying the maximum number of neighbor robots in the application context, it is possible to tune the parameters associated with a T(1) increase of the search space size and allow good learning performance. We use the cooperative multi-robot observation of multiple moving targets (CMOMMT) application to illustrate our method. We verify that awareness allows cooperation, that cooperation shows better performance than a purely collective behavior and that learned cooperation shows better results than learned collective behavior.

Development of a Cleaning Robot System with Cooperative Positioning System

Abstract: For the development of an automatic floor cleaning robot system, an accurate positioning method in unstructured and dynamically changing environments is indispensable Dead reckoning is a popular method, but is not reliable for measurement over long distances especially on uneven and slippery floors due to the accumulation error of wheel diameter and slippage The landmark method, which estimates current position relative to landmarks, cannot be used in an uncharted and an unfamiliar environment We have proposed a new method called “Cooperative Positioning System (CPS)” The main concept of CPS is to divide the robots into two groups, A and B where group A remains stationary and acts as a landmark while group B moves, then group B stops and acts as a landmark for group A This process is repeated until the target position is reached CPS has a far lower accumulation of positioning error than dead reckoning, and can work in three-dimensions Also, CPS has inherent landmarks and therefore works in uncharted environments In previous papers, we have introduced the prototype CPS machine models, CPS-I and CPS-II and demonstrated high performance as a positioning system in an unknown and uneven environment In this paper, we report on the third prototype CPS model, CPS-III, that is designed specifically as an automatic floor-cleaning robot system, and the results of a floor cleaning experiment In this system, we categorize these robots for two tasks, that is, an accurate positioning task achieved with 3 robots using the CPS strategy, and a floor-cleaning task executed by an omni-directional vehicle, so as to improve the efficiency of the floor-cleaning system Experiments show that these robots can perform a floor-cleaning task in a corridor within a positioning error of 1408 mm even after robots move over a distance of 1017 m

Optimal Selection of Uncertain Actions by Maximizing Expected Utility

Abstract: A new means of action selection via utility fusion is introduced as an alternative to both sensor fusion and command fusion. Distributed asynchronous behaviors indicate the utility of various possible states and their associated uncertainty. A centralized arbiter then combines these utilities and probabilities to determine the optimal action based on the maximization of expected utility. The construction of a utility map allows the system being controlled to be modeled and compensated fors experimental results verify that this approach improves performance.

Cooperative Navigation of Micro-Rovers Using Color Segmentation

Abstract: This paper addresses position estimation of a micro-rover mobile robot (called the “daughter”) as a larger robot (the “mother”) tracks it through large spaces with unstructured lighting. Position estimation is necessary for localization, where the mother extracts the relative position of the daughter for mapping purposes, and for cooperative navigation, where the mother controls the daughter in real-time. The approach taken is to employ the Spherical Coordinate Transform color segmenter developed for medical applications as a low computational and hardware cost solution. Data was collected from 50 images taken in five types of lighting: fluorescent, tungsten, daylight lamp, natural daylight indoors and outdoors. The results show that average pixel error was 1.5, with an average error in distance estimation of 6.3 cm. The size of the error did not vary greatly with the type of lighting. The segmentation and distance tracking have also been implemented as a real-time tracking system. Using this system, the mother robot is able to autonomously control the micro-rover and display a map of the daughter's path in real-time using only a Pentium class processor and no specialized hardware.

Coupled Oscillator Control of Autonomous Mobile Robots

Abstract: This paper introduces a nonlinear oscillator scheme to control autonomous mobile robots. The method is based on observations of a successful control mechanism used in nature, the Central Pattern Generator. Simulations were used to assess the performance of oscillator controller when used to implement several behaviors in an autonomous robot operating in a closed arena. A sequence of basic behaviors (random wandering, obstacle avoidance and light following) was coordinated in the robot to produce the higher behavior of foraging for light. The controller is explored in simulations and tests on physical robots. It is shown that the oscillator—based controller outperforms a reactive controller in the tasks of exploring an arena with irregular walls and in searching for light.

A Navigation System for Cleaning Robots

Abstract: Free navigation in indoor environments is one of the main enabling technologies for many service robot applications. The SIEMENS navigation system SINAS which is primarily targeted towards cleaning robot applications, has proved its suitability for tough everyday operation since August 1996 on several occasions, e.g., in several chain store supermarkets. This paper discusses the main requirements of a navigation system for cleaning robots, presents the architecture and main modules of the SINAS system, and reports on real-world experiences.

Autonomous Pool Cleaning: Self Localization and Autonomous Navigation for Cleaning

Abstract: Cleaning is a major problem associated with pools. Since the manual cleaning is tedious and boring there is an interest in automating the task. This paper presents methods for autonomous localization and navigation for a pool cleaner to enable full coverage of pools. Path following cannot be ensured through use of internal position estimation methods alones therefore sensing is needed. Sensor based estimation enable automatic correction of slippage. For this application we use ultrasonic sonars. Based on an analysis of the overall task and performance of the system a strategy for cleaning/navigation is developed. For the automatic localization a Kalman filtering technique is proposed: the Kalman filter uses sonar measurements and a dynamic model of the robot to provide estimates of the pose of the pool cleaner. Using this localization method we derive an optimal control strategy for traversal of a pool. The system has been implemented and successfully tested on the “WEDA B400” pool cleaner.

Functional Programming of Behavior-Based Systems

Abstract: In this paper, I describe a simple functional programming language, GRL, in which most of the characteristic features of the popular behavior-based robot architectures can be concisely written as reusable software abstractions. This makes it easier to write clear, modular code, to “mix and match” arbitration mechanisms, and to experiment with variations on existing mechanisms. I describe the compilation process for the language, our experiences with it, and issues of efficiency, expressiveness, and code size relative to other languages.

Moore Automata for the Supervisory Control of Robotic Manufacturing Workcells

Abstract: The potential of flexible-manufacturing workcells (FMCs) to produce a family of parts in many possible orders of operations and choices of different machines is advantageous. Despite intensive research on the theoretical control of discrete-event systems (DESs), however, current techniques can still only be used for the supervisory control of simple cells. In this paper, a novel modeling and control synthesis technique is presented for FMCs that allow part-routing flexibility. Our proposed methodology combines Extended Moore Automata (EMA) and Controlled-Automata theories to synthesize supervisors for such FMCs.

Target Reaching by Using Visual Information and Q-learning Controllers

Abstract: This paper presents a solution to the problem of manipulation control: target identification and grasping. The proposed controller is designed for a real platform in combination with a monocular vision system. The objective of the controller is to learn an optimal policy to reach and to grasp a spherical object of known size, randomly placed in the environment. In order to accomplish this, the task has been treated as a reinforcement problem, in which the controller learns by a trial and error approach the situation-action mapping. The optimal policy is found by using the Q-Learning algorithm, a model free reinforcement learning technique, that rewards actions that move the arm closer to the target. The vision system uses geometrical computation to simplify the segmentation of the moving target (a spherical object) and determines an estimate of the target parameters. To speed-up the learning time, the simulated knowledge has been ported on the real platform, an industrial robot manipulator PUMA 560. Experimental results demonstrate the effectiveness of the adaptive controller that does not require an explicit global target position using direct perception of the environment.