scispace - formally typeset
SciSpace - Your AI assistant to discover and understand research papers | Product Hunt

Proceedings ArticleDOI

Keeping multiple objects in the field of view of a single PTZ camera

10 Jun 2009-pp 5259-5264

TL;DR: A set of task functions are developed to regulate the mean and variance of a set of image features to deter feature points from leaving the camera field-of-view.

AbstractThis paper introduces a novel visual servo controller designed to keep multiple moving objects in the camera field-of-view using a pan/tilt/zoom camera. In contrast to most visual servo controllers, there is no goal pose or goal image. In this paper, a set of task functions are developed to regulate the mean and variance of a set of image features. Regulating these task functions will deter feature points from leaving the camera field-of-view. An additional task function is used to maintain a high level of motion perceptibility, which ensures that desired feature point velocities can be achieved. To provide proper control of a pan/tilt/zoom camera, an image Jacobian is developed utilizing actuation of the focal length. Simulations of several object tracking tasks have verified the performance of the proposed method.

Topics: Pan–tilt–zoom camera (70%), Camera auto-calibration (66%), Camera resectioning (65%), Smart camera (61%), Digital zoom (61%)

...read more

Content maybe subject to copyright    Report

Citations
More filters

Proceedings ArticleDOI
21 Oct 2013
TL;DR: This work presents a method to generate aesthetic video from a robotic camera by incorporating a virtual camera operating on a delay, and a hybrid controller which uses feedback from both the robotic and virtual cameras.
Abstract: We present a method to generate aesthetic video from a robotic camera by incorporating a virtual camera operating on a delay, and a hybrid controller which uses feedback from both the robotic and virtual cameras. Our strategy employs a robotic camera to follow a coarse region-of-interest identified by a realtime computer vision system, and then resamples the captured images to synthesize the video that would have been recorded along a smooth, aesthetic camera trajectory. The smooth motion trajectory is obtained by operating the virtual camera on a short delay so that perfect knowledge of immediate future events is known. Previous autonomous camera installations have employed either robotic cameras or stationary wide-angle cameras with subregion cropping. Robotic cameras track the subject using realtime sensor data, and regulate a smoothness-latency trade-off through control gains. Fixed cameras post-process the data and suffer significant reductions in image resolution when the subject moves freely over a large area.Our approach provides a solution for broadcasting events from locations where camera operators cannot easily access. We can also offer broadcasters additional actuated camera angles without the overhead of additional human operators. Experiments on our prototype system for college basketball illustrate how our approach better mimics human operators compared to traditional robotic control approaches, while avoiding the loss in resolution that occurs from fixed camera system.

72 citations


Cites methods from "Keeping multiple objects in the fie..."

  • ...[10] use taskpriority kinematic control to keep a set of interest points within the camera field of view....

    [...]

  • ...As a result, we employ assumptions common in visual servoing [9, 10] and approxi-...

    [...]


Journal ArticleDOI
TL;DR: A novel visual servo controller that is designed to control the pose of the camera to keep multiple objects in the field of view (FOV) of a mobile camera and a proof of stability is presented for tracking three or fewer targets.
Abstract: This study introduces a novel visual servo controller that is designed to control the pose of the camera to keep multiple objects in the field of view (FOV) of a mobile camera. In contrast with other visual servo methods, the control objective is not formulated in terms of a goal pose or a goal image. Rather, a set of underdetermined task functions are developed to regulate the mean and variance of a set of image features. Regulating these task functions inhibits feature points from leaving the camera FOV. An additional task function is used to maintain a high level of motion perceptibility, which ensures that desired feature point velocities can be achieved. These task functions are mapped to camera velocity, which serves as the system input. A proof of stability is presented for tracking three or fewer targets. Experiments of tracking eight or more targets have verified the performance of the proposed method.

45 citations


Cites background or methods from "Keeping multiple objects in the fie..."

  • ...Specifically, Chebyshev’s inequality proves that at least 75% of all values are within two standard deviations of the mean, and at least 89% of values are within three standard deviations....

    [...]

  • ...The camera is a Matrix Vision BlueFox, with a resolution of 1024 × 768 pixels....

    [...]


Journal ArticleDOI
TL;DR: A robust visual servo system for object tracking applications of a nonholonomic mobile robot using a Lyapunov-based stability analysis and a sliding mode control technique is presented.
Abstract: In this paper, we present the development of a robust visual servo system for object tracking applications of a nonholonomic mobile robot. The system mainly consists of an adaptive shape tracking algorithm and a robust visual servo controller. The adaptive shape tracking algorithm is designed to automatically detect the shape contours of moving objects, extract the shape parameters, and continuously track the object in shape parameter space. Based on direct measurements of the shape parameters, the visual servo controller is designed using the sliding mode control technique. Through a Lyapunov-based stability analysis, a sufficient condition on the selection of control gains to achieve the tracking goal in finite time is provided, and simulation and experimental tests of the proposed approach are illustrated.

33 citations


Cites methods from "Keeping multiple objects in the fie..."

  • ...Recently, for tracking multiple objects, an image Jacobian was developed in [18] for controlling the desired feature points within the field of view of a single pan-tilt-zoom camera and a configuration-homography-based analysis is proposed in [19] for deriving related...

    [...]


Proceedings Article
01 Jan 2014
TL;DR: This work reviews autonomous camera systems developed over the past twenty years and discusses a trend towards more datadriven approaches fueled by continuous improvements in underlying sensing and signal processing technology.
Abstract: Autonomous cameras allow live events, such as lectures and sports matches, to be broadcast to larger audiences. In this work, we review autonomous camera systems developed over the past twenty years. Quite often, these systems were demonstrated on scripted stage productions (typically cooking shows), lectures, or team sports. We organize the discussion in terms of three core tasks: (1) planning where the cameras should look, (2) controlling the cameras as they transition from one parameter configuration to another, and (3) selecting which camera to put “on-air” in multi-camera systems. We conclude by discussing a trend towards more datadriven approaches fueled by continuous improvements in underlying sensing and signal processing technology.

20 citations


Cites methods from "Keeping multiple objects in the fie..."

  • ...Gans et al. (2009) used a task-priority kinematic controller to keep a set of interest points within the camera field of view....

    [...]


Proceedings ArticleDOI
13 Jun 2017
Abstract: This paper proposes a guidance algorithm of a UAV for multiple targets tracking, considering physical constraints of the platform and its sensor. The on-board sensor used for target tracking is a camera. To relax the need of a gimbal system and provide flexibility in utilisation of the vision sensor, a new rotorcraft type of UAV is developed. The main focus in designing tracking guidance is to develop an image based visual servoing approach, appropriate to the newly developed platform. A complementary to the guidance system a control system for the unconventional UAV is also presented. Both control and guidance algorithms are based on the PID control techniques and this paper shows that the tracking guidance can be significantly simplified with the new type of UAV developed. The performance of the proposed tracking guidance along with the controller designed is validated by numerical simulations and flight tests.

8 citations


Cites background or result from "Keeping multiple objects in the fie..."

  • ...In papers [21], [23], [22], variance of position of the targets in the image is used to keep objects away from its edges....

    [...]

  • ...In [21], [22], [23], a theoretical investigation of their guidance algorithms was presented....

    [...]

  • ...There have been also some studies on multi-target tracking [21], [22], [23], [24]....

    [...]


References
More filters

Proceedings ArticleDOI
20 Sep 1999
TL;DR: Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.
Abstract: An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low residual least squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.

15,597 citations


"Keeping multiple objects in the fie..." refers background in this paper

  • ...The extraction of such points could come from a variety of algorithms, such as tracking distinguishable points on moving targets [10], [11], the centroid of segmented blobs [12] or the corner points of bounding rectangle of tracked moving targets [13]....

    [...]


Proceedings ArticleDOI
21 Jun 1994
TL;DR: A feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world are proposed.
Abstract: No feature-based vision system can work unless good features can be identified and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under affine image transformations. We test performance with several simulations and experiments. >

8,046 citations


"Keeping multiple objects in the fie..." refers background in this paper

  • ...The extraction of such points could come from a variety of algorithms, such as tracking distinguishable points on moving targets [10], [11], the centroid of segmented blobs [12] or the corner points of bounding rectangle of tracked moving targets [13]....

    [...]


Journal ArticleDOI
01 Oct 1996
TL;DR: This article provides a tutorial introduction to visual servo control of robotic manipulators by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process.
Abstract: This article provides a tutorial introduction to visual servo control of robotic manipulators. Since the topic spans many disciplines our goal is limited to providing a basic conceptual framework. We begin by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process. We then present a taxonomy of visual servo control systems. The two major classes of systems, position-based and image-based systems, are then discussed in detail. Since any visual servo system must be capable of tracking image features in a sequence of images, we also include an overview of feature-based and correlation-based methods for tracking. We conclude the tutorial with a number of observations on the current directions of the research field of visual servo control.

3,456 citations


"Keeping multiple objects in the fie..." refers background or methods in this paper

  • ...The feature point depths must be known or accurately estimated for six-DOF IBVS, and IBVS is brittle to depth estimation errors [14]....

    [...]

  • ...The method is rooted in classic IBVS [1]–[4], however there is no goal image or goal feature trajectory....

    [...]

  • ...There are many approaches, including image based visual servoing (IBVS), position based visual servoing, partitioned methods and switching methods....

    [...]

  • ...The first two columns of Lλ(t) are the same as the fourth and fifth columns of the classic six -DOF image Jacobian [1], [3]....

    [...]

  • ...The time-varying image feature velocity is mapped to camera motions through IBVS methods....

    [...]


Journal ArticleDOI
30 Nov 2006
TL;DR: This paper is the first of a two-part series on the topic of visual servo control using computer vision data in the servo loop to control the motion of a robot using basic techniques that are by now well established in the field.
Abstract: This paper is the first of a two-part series on the topic of visual servo control using computer vision data in the servo loop to control the motion of a robot. In this paper, we describe the basic techniques that are by now well established in the field. We first give a general overview of the formulation of the visual servo control problem. We then describe the two archetypal visual servo control schemes: image-based and position-based visual servo control. Finally, we discuss performance and stability issues that pertain to these two schemes, motivating the second article in the series, in which we consider advanced techniques

1,775 citations


Book ChapterDOI
01 Jun 1992
TL;DR: Vision-based control in robotics based on considering a vision system as a specific sensor dedicated to a task and included in a control servo loop is described, and stability and robustness questions arise.
Abstract: Vision-based control in robotics based on considering a vision system as a specific sensor dedicated to a task and included in a control servo loop is described. Once the necessary modeling stage is performed, the framework becomes one of automatic control, and stability and robustness questions arise. State-of-the-art visual servoing is reviewed, and the basic concepts for modeling the concerned interactions are given. The interaction screw is thus defined in a general way, and the application to images follows. Starting from the concept of task function, the general framework of the control is described, and stability results are recalled. The concept of the hybrid task is presented and then applied to visual sensors. Simulation and experimental results are presented, and guidelines for future work are drawn in the conclusion. >

1,425 citations