The situation recognition system, to which this paper is devoted, receives as input a stream of time-stamped events; it performs recognition of instances of occurring situations, as they are developing, and it generates as output deduced events and actions to trigger. It is mainly a temporal reasoning system. It is predictive in the sense that it predicts forthcoming events relevant to its task, it focuses its attention on them and it maintains their temporal windows of relevance. Its main functionality is to recognize efficiently complex temporal patterns on the fly, while they are taking place. This system has been tested for the surveillance of an environment by a multisensory perception machine; it is being applied to monitoring a complex dynamic system.

https://ijcai.org/Proceedings/93-1/Papers/024.pdf

Situation recognition: representation and algorithms

In this paper, we propose an approach to solve the problem of finding an object in a polygon which may contain holes. We define an optimal solution as the route that minimizes the expected time it takes to find said object. The object search problem is shown to be NP-hard by reduction, therefore, we propose the heuristic of an utility function, defined as the ratio of a gain over a cost and a greedy algorithm in a reduced search space that is able to explore several steps ahead without incurring in too high computational cost. This approach was implemented and simulation results are shown.

/pdf/an-efficient-strategy-for-rapidly-finding-an-object-in-a-26457euhed.pdf

An efficient strategy for rapidly finding an object in a polygonal world

This paper presents our approach to retrieve a dependable three-dimensional description of a partially known indoor environment. We describe the way the sensor data from a video camera is preprocessed by contour tracing to extract the boundary lines of the objects and how this information is transformed into a three-dimensional environmental model of the world. We introduce a dynamic map that operates in a closed loop with various sensor systems improving their performance by filtering and contributing certain knowledge. The filtering relies on the capability of a mobile robot to gather sensor readings from different positions. An important part of our approach is the interaction between the dynamic map, storing and filtering the incoming information, and a module predicting missing sensor features based on structures and reference objects. This interaction helps to generate a more accurate model containing also poor detectable features, that are impossible to extract from a single sensor view.

Vision based model generation for indoor environments

Robot intelligence requires a real-time connection between sensing and action. A new computation principle of robotics that efficiently implements such a connection is utmost important for the new generation of robotics. In this paper, a perception–action network is presented as a means of efficiently integrating sensing, knowledge, and action for sensor fusion and planning. The network consists of a number of heterogeneous computational units, representing feature transformation and decision-making for action, which are interconnected as a dynamic system. New input stimuli to the network invoke the evolution of network states to a new equilibrium, through which a real-time integration of sensing, knowledge, and action can be accomplished. The network provides a formal, yet general and efficient, method of achieving sensor fusion and planning. This is because the uncertainties of signals, propagated in the network, can be controlled by modifying sensing parameters and robot actions. Algorithms for sensor planning based on the proposed network are established and applied to robot self-localization. Simulation and experimental results are shown.

Sensor Fusion and Planning with Perception–Action Network

The authors present a method for selecting viewpoints and sensing tasks to confirm by a multisensory perception machine, an identification hypothesis previously generated. The determination relies on the use of a compiled knowledge base that links object and sensor models, and defines a priori the best sensing tasks to be performed. The method is fully detailed, and its use under dynamic constraints due to a real environment is explained, along with a control strategy to activate the search. Result of experiments integrating other modules for environment modeling, path planning and execution control, and object recognition are presented. >

Perception planning for a multi-sensory interpretation machine

The authors propose an approach to generalize the hypothesis and test recognition paradigm for multisensory environments and fairly generic object models. Matching, prediction and localization procedures are based on a generic representation of feature accuracy. This generic approach performs fusion both at the numeric (geometric) and at the symbolic (recognition) levels. Its reliability is illustrated by several real-world examples demonstrating recognition of real objects in complex cluttered environments using four types of sensory data: contour images (two viewpoints), stereovision 3-D line segments, range 3-D faces, and color images. >

M. Ghallab

Papers

Perception planning for a multi-sensory interpretation machine

Multisensory scene interpretation: model-based object recognition