Showing papers on "Smart camera published in 2007"

MonoSLAM: Real-Time Single Camera SLAM

Abstract: We present a real-time algorithm which can recover the 3D trajectory of a monocular camera, moving rapidly through a previously unknown scene. Our system, which we dub MonoSLAM, is the first successful application of the SLAM methodology from mobile robotics to the "pure vision" domain of a single uncontrolled camera, achieving real time but drift-free performance inaccessible to structure from motion approaches. The core of the approach is the online creation of a sparse but persistent map of natural landmarks within a probabilistic framework. Our key novel contributions include an active approach to mapping and measurement, the use of a general motion model for smooth camera movement, and solutions for monocular feature initialization and feature orientation estimation. Together, these add up to an extremely efficient and robust algorithm which runs at 30 Hz with standard PC and camera hardware. This work extends the range of robotic systems in which SLAM can be usefully applied, but also opens up new areas. We present applications of MonoSLAM to real-time 3D localization and mapping for a high-performance full-size humanoid robot and live augmented reality with a hand-held camera

MeshEye: a hybrid-resolution smart camera mote for applications in distributed intelligent surveillance

Abstract: Surveillance is one of the promising applications to which smart camera motes forming a vision-enabled network can add increasing levels of intelligence. We see a high degree of in-node processing in combination with distributed reasoning algorithms as the key enablers for such intelligent surveillance systems. To put these systems into practice still requires a considerable amount of research ranging from mote architectures, pixel-processing algorithms, up to distributed reasoning engines. This paper introduces MeshEye, an energy-efficient smart camera mote architecture that has been designed with intelligent surveillance as the target application in mind. Special attention is given to MeshEye's unique vision system: a low-resolution stereo vision system continuously determines position, range, and size of moving objects entering its field of view. This information triggers a color camera module to acquire a high-resolution image sub-array containing the object, which can be efficiently processed in subsequent stages. It offers reduced complexity, response time, and power consumption over conventional solutions. Basic vision algorithms for object detection, acquisition, and tracking are described and illustrated on real- world data. The paper also presents a basic power model that estimates lifetime of our smart camera mote in battery-powered operation for intelligent surveillance event processing.

Design and application of industrial machine vision systems

Abstract: In this paper, the role and importance of the machine vision systems in the industrial applications are described. First understanding of the vision in terms of a universal concept is explained. System design methodology is discussed and a generic machine vision model is reported. Such a machine includes systems and sub-systems, which of course depend on the type of applications and required tasks. In general, expected functions from a vision machine are the exploitation and imposition of the environmental constraint of a scene, the capturing of the images, analysis of those captured images, recognition of certain objects and features within each image, and the initiation of subsequent actions in order to accept or reject the corresponding objects. After a vision system performs all these stages, the task in hand is almost completed. Here, the sequence and proper functioning of each system and sub-systems in terms of high-quality images is explained. In operation, there is a scene with some constraint, first step for the machine is the image acquisition, pre-processing of image, segmentation, feature extraction, classification, inspection, and finally actuation, which is an interaction with the scene under study. At the end of this report, industrial image vision applications are explained in detail. Such applications include the area of automated visual inspection (AVI), process control, parts identification, and important role in the robotic guidance and control. Vision developments in manufacturing that can result in improvements in the reliability, in the product quality, and enabling technology for a new production process are presented. The key points in design and applications of a machine vision system are also presented. Such considerations can be generally classified into the six different categories such as the scene constraints, image acquisition, image pre-processing, image processing, machine vision justification, and finally the systematic considerations. Each aspect of such processes is described here and the proper condition for an optimal design is reported.

Camera system for large vehicles

Abstract: A vehicle-borne camera-based observation system for monitoring areas adjacent a vehicle or passenger vehicle, such as a bus or schoolbus, is disclosed, to provide safer operation for passersby, including for children, and driver convenience. The system includes several cameras, and several monitors in a driver's area displaying all of the fields of view from the cameras, such that each monitor may be controllable to show either the field of view of a first camera or a the field of view of a second camera, according to a driver selection or according to an automatic selection. Night vision, automatic tracking, and illumination systems are also provided.

Extrinsic self calibration of a camera and a 3D laser range finder from natural scenes

Abstract: In this paper, we describe a new approach for the extrinsic calibration of a camera with a 3D laser range finder, that can be done on the fly. This approach does not require any calibration object. Only few point correspondences are used, which are manually selected by the user from a scene viewed by the two sensors. The proposed method relies on a novel technique to visualize the range information obtained from a 3D laser scanner. This technique converts the visually ambiguous 3D range information into a 2D map where natural features of a scene are highlighted. We show that by enhancing the features the user can easily find the corresponding points of the camera image points. Therefore, visually identifying laser- camera correspondences becomes as easy as image pairing. Once point correspondences are given, extrinsic calibration is done using the well-known PnP algorithm followed by a noninear refinement process. We show the performance of our approach through experimental results. In these experiments, we will use an omnidirectional camera. The implication of this method is important because it brings 3D computer vision systems out of the laboratory and into practical use.

Dual mode camera solution apparatus, system, and method

Abstract: A camera solution includes an image sensor and an image processing and control system. At least two different operating modes are supported, with one of the modes having a higher dynamic range. Control of the dynamic range is provided at the system level. The system supports statically or dynamically selecting an operating mode that determines the dynamic range of a camera. In one implementation, the system supports the use of either a conventional image sensor that does not natively support a high dynamic range or a dual-mode image sensor.

Aging in place: fall detection and localization in a distributed smart camera network

Abstract: This paper presents the design, implementation and evaluation of a distributed network of smart cameras whose function is to detect and localize falls, an important application in elderly living environments. A network of overlapping smart cameras uses a decentralized procedure for computing inter-image homographies that allows the location of a fall to be reported in 2D world coordinates by calibrating only one camera. Also, we propose a joint routing and homography transformation scheme for multi-hop localization that yields localization errors of less than 2 feet using very low resolution images. Our goal is to demonstrate that such a distributed low-power system can perform adequately in this and related applications. A prototype implementation is given for low-power Agilent/UCLA Cyclops cameras running on the Crossbow MICAz platform. We demonstrate the effectiveness of the fall detection as well as the precision of the localization using a simulation of our sample implementation.

Autonomous multicamera tracking on embedded smart cameras

Abstract: There is currently a strong trend towards the deployment of advanced computer vision methods on embedded systems. This deployment is very challenging since embedded platforms often provide limited resources such as computing performance, memory, and power. In this paper we present a multicamera tracking method on distributed, embedded smart cameras. Smart cameras combine video sensing, processing, and communication on a single embedded device which is equipped with a multiprocessor computation and communication infrastructure. Our multicamera tracking approach focuses on a fully decentralized handover procedure between adjacent cameras. The basic idea is to initiate a single tracking instance in the multicamera system for each object of interest. The tracker follows the supervised object over the camera network, migrating to the camera which observes the object. Thus, no central coordination is required resulting in an autonomous and scalable tracking approach. We have fully implemented this novel multicamera tracking approach on our embedded smart cameras. Tracking is achieved by the well-known CamShift algorithm; the handover procedure is realized using a mobile agent system available on the smart camera network. Our approach has been successfully evaluated on tracking persons at our campus.

Optimal Camera Placement for Automated Surveillance Tasks

Abstract: Camera placement has an enormous impact on the performance of vision systems, but the best placement to maximize performance depends on the purpose of the system. As a result, this paper focuses largely on the problem of task-specific camera placement. We propose a new camera placement method that optimizes views to provide the highest resolution images of objects and motions in the scene that are critical for the performance of some specified task (e.g. motion recognition, visual metrology, part identification, etc.). A general analytical formulation of the observation problem is developed in terms of motion statistics of a scene and resolution of observed actions resulting in an aggregate observability measure. The goal of this system is to optimize across multiple cameras the aggregate observability of the set of actions performed in a defined area. The method considers dynamic and unpredictable environments, where the subject of interest changes in time. It does not attempt to measure or reconstruct surfaces or objects, and does not use an internal model of the subjects for reference. As a result, this method differs significantly in its core formulation from camera placement solutions applied to problems such as inspection, reconstruction or the Art Gallery class of problems. We present tests of the system's optimized camera placement solutions using real-world data in both indoor and outdoor situations and robot-based experimentation using an all terrain robot vehicle-Jr robot in an indoor setting.

Wireless endoscopic camera

Abstract: A system and method for wirelessly transmitting a video image signal from an endoscopic camera to a receiver or control unit for storage and/or display on a video monitor. Use of a frame-specific, variable compression algorithm capable of progressively encoding a data stream provides for a better performing and higher quality wireless endoscopic camera system capable of generating images at varying resolutions. Use of a short-range, high-performance wireless technology, such as Ultrawideband (UWB), improves the performance capabilities of the system, while minimizing power consumption and extending battery life. Implementations of error correcting codes, as well as the use of multiple transmitting and receiving antennas, further improve the fidelity of the wireless communication.

3D Imaging Camera for Gaming Application

Abstract: We describe a camera which is designed as a novel user interface for gaming applications Such a camera, developed and patented by 3DV Systems, is capable of separating objects in space or time, thus be an ideal tool for interfacing in free space gaming consoles or other gaming platforms by body or hand gestures only The basic feature of such a camera is real time depth imaging by capturing the shape of a light-pulse front as it is reflected of a three dimensional object This depth sensing is very versatile and may be applied to many fields, such as security, robotics and gestures recognition, to name a few We describe the basic architecture and main building blocks of a high-speed depth camera, for real time gaming interfacing

Camera Mote with a High-Performance Parallel Processor for Real-Time Frame-Based Video Processing

Abstract: This paper describes a new smart camera mote with a high performance SIMD (single-instruction multiple-data) processor. Previous versions of our camera mote were equipped with IC3D, a line-based processor. The mote described in this paper is equipped with Xetal-II, a processor designed for frame-based real-time video analysis. The processor uses 320 processing elements in parallel to achieve performance figures of more than 100GOPS with a power consumption of 600 mWatt at peak performance. The IC has a 10Mbit internal memory cache to store and work on 4 VGA frames. The internal bandwidth to this memory is more than 1.5 Tbit/s allowing multiple passes over the images within frametime. Augmented with hardware tools for object processing, the new mote opens the door for embedded active vision applications and other iterative techniques such as watershedding and distance transforms in collaborative camera networks.

System for 3D monitoring and analysis of motion behavior of targets

Abstract: The present invention relates to a system for the 3-D monitoring and analysis of motion-related behavior of test subjects The system comprises an actual camera, at least one virtual camera, a computer connected to the actual camera and the computer is preferably installed with software capable of capturing the stereo images associated with the 3-D motion-related behavior of test subjects as well as processing these acquired image frames for the 3-D motion parameters of the subjects The system of the invention comprises hardware components as well as software components The hardware components preferably comprise a hardware setup or configuration, a hardware-based noise elimination component, an automatic calibration device component, and a lab animal container component The software components preferably comprise a software-based noise elimination component, a basic calibration component, an extended calibration component, a linear epipolar structure derivation component, a non-linear epipolar structure derivation component, an image segmentation component, an image correspondence detection component, a 3-D motion tracking component, a software-based target identification and tagging component, a 3-D reconstruction component, and a data post-processing component In a particularly preferred embodiment, the actual camera is a digital video camera, the virtual camera is the reflection of the actual camera in a planar reflective mirror Therefore, the preferred system is a catadioptric stereo computer vision system

Determining vision graphs for distributed camera networks using feature digests

Abstract: We propose a decentralized method for obtaining the vision graph for a distributed, ad-hoc camera network, in which each edge of the graph represents two cameras that image a sufficiently large part of the same environment Each camera encodes a spatially well-distributed set of distinctive, approximately viewpoint-invariant feature points into a fixed-length "feature digest" that is broadcast throughout the network Each receiver camera robustly matches its own features with the decompressed digest and decides whether sufficient evidence exists to form a vision graph edge We also show how a camera calibration algorithm that passes messages only along vision graph edges can recover accurate 3D structure and camera positions in a distributed manner We analyze the performance of different message formation schemes, and show that high detection rates (>08) can be achieved while maintaining low false alarm rates (<005) using a simulated 60-node outdoor camera network

Real-time camera tracking using sports pitch markings

Abstract: When broadcasting sports events such as football, it is useful to be able to place virtual annotations on the pitch, to indicate things such as distances between players and the goal, or whether a player is offside. This requires the camera position, orientation, and focal length to be estimated in real time, so that the graphics can be rendered to match the camera view. Whilst this can be achieved by using sensors on the camera mount and lens, they can be impractical or expensive to install, and often the broadcaster only has access to the video feed itself. This paper presents a method for computing the position, orientation and focal length of a camera in real time, using image analysis. The method uses markings on the pitch, such as arcs and lines, to compute the camera pose. A novel feature of the method is the use of multiple images to improve the accuracy of the camera position estimate. A means of automatically initialising the tracking process is also presented, which makes use of a modified form of Hough transform. The paper shows how a carefully chosen set of algorithms can provide fast, robust and accurate tracking for this real-world application.

Integrating 3D Time-of-Flight Camera Data and High Resolution Images for 3DTV Applications

Abstract: Applying the machine-learning technique of inference in Markov random fields we build improved 3D models by integrating two different modalities. Visual input from a standard color camera delivers high-resolution texture data but also enables us to enhance the 3D data calculated from the range output of a 3D time-of-flight camera in terms of noise and spatial resolution. The proposed method to increase the visual quality of the 3D data makes this kind of camera a promising device for various upcoming 3DTV applications. With our two-camera setup we believe that the design of low-cost, fast and highly portable 3D scene acquisition systems will be possible in the near future.

Method and apparatus for powering on an electronic device with a video camera that detects motion

Abstract: A method and apparatus for controlling power up of an electronic device with a video camera is provided. The present invention provides for using a video camera attached to an electronic device, such as a computer system, to cause the electronic device to be powered up from sleep mode when motion is detected. The electronic device may also be powered up from being shut down. In one embodiment, the video camera includes a processor and memory that compare consecutive frames captured by the video camera. When the electronic device is in sleep mode, if consecutive frames are the same, the video camera continues to monitor the scene without generating an output signal. If the frames are different, motion is detected and the video camera generates a signal that is used to determine whether the electronic device should power up. In this manner, the electronic device may begin the powering up process before the user of the device interacts with the device.

Camera system and methods

Abstract: A system and method for providing visible indications of an area of interest for assisting the visual orientation of a user using a digital camera is provided. The system includes a digital camera having a built in illumination projector. The illumination projector optical axis is related to the camera optical axis. Typically the illumination projector is collimated with the camera. Images produced by the illumination projector are viewed by camera user and are used as a reference for understanding the camera orientation and for selecting the desirable direction and field of view (FOV) for the camera. The system and method is extremely useful for portable and wearable digital cameras as well as for setup of fixed camera applications.

Object Reacquisition and Tracking in Large-Scale Smart Camera Networks

Abstract: Object reacquisition or reidentification is the process of matching objects between images taken from separate cameras. In this paper, we present our work on feature based object reidentification performed on autonomous embedded smart cameras and applied to traffic scenarios. We present a novel approach based on PCA-SIFT features and a vocabulary tree. By building unique object signatures from visual features, reidentification can be done efficiently coevally minimizing the communication overhead between separate camera nodes. Applied to large-scale traffic scenarios, important parameters including travel time, travel time variability, section density, and partial dynamic origin/destination demands can be obtained. The proposed approach works on spatially separated, un-calibrated, non-overlapping cameras, is highly scalable and solely based on appearance-based optical features. The entire system is implemented and evaluated with regard to a typical embedded smart camera platform featuring one single Texas Instruments trade fixed-point DSP.

Dynamic shared visual spaces: experimenting with automatic camera control in a remote repair task

Abstract: We present an experimental study of automatic camera control in the performance of collaborative remote repair tasks using video-mediated communication. Twelve pairs of participants, one "helper" and one "worker," completed a series of Lego puzzle tasks using both a static camera and an automatic camera system that was guided in part by tracking the worker's hand position. Results show substantial performance benefits for the automatic system, particularly for complex tasks. The implications of these results are discussed, along with some lessons for the use of motion tracking as a driver for camera control.

Mesh communication wireless camera system and method

Abstract: A system includes a camera and a monitoring system that are capable of wireless communication. The camera is operable to send a message to the monitoring system via a plurality of wireless communication paths. The message includes information relating to an operational status of the camera. The camera may generate and analyze a video signal and the message may include information relating to the analysis of the video signal. The camera may be one of a plurality of cameras capable of generating video signals. The system may further include an intermediate node capable of wireless communication with the cameras and the monitoring system. The intermediate node may multiplex the video signals from the cameras to the monitoring system.

Smart Camera Networks in Virtual Reality

Abstract: We present smart camera network research in the context of a unique new synthesis of advanced computer graphics and vision simulation technologies. We design and experiment with simulated camera networks within visually and behaviorally realistic virtual environments. Specifically, we demonstrate a smart camera network comprising static and active simulated video surveillance cameras that provides perceptive coverage of a large virtual public space, a train station populated by autonomously self-animating virtual pedestrians. In the context of human surveillance, we propose a camera network control strategy that enables a collection of smart cameras to provide perceptive scene coverage and perform persistent surveillance with minimal intervention. Our novel control strategy naturally addresses camera aggregation and camera handoff, it does not require camera calibration, a detailed world model, or a central controller, and it is robust against camera failures and communication.

Warning method and system for preventing collision for vehicle on high standard highway

Abstract: The invention relates to a vehicle bias alarm method and relative system, for alarming driver to avoid bias and accidence. The output of inverter power and one end of relay is connected with an output signal interface of an intelligent camera, the picture information collecting module, channel mark line recognizing module, vehicle direction parameter evaluation module, vehicle position parameter evaluation module, channel width evaluation module and channel bias alarm module are embedded in the chip of the camera, while the intelligent camera via a mounting support is fixed in the front windscreen glass of cabin, and said six modules should started in serially to realize the alarm method.

Hardware, Design and Implementation Issues on a Fpga-Based Smart Camera

Abstract: Processing images to extract useful information in real-time is a complex task, dealing with large amounts of iconic data and requiring intensive computation. Smart cameras use embedded processing to save the host system from the low-level processing load and to reduce communication flows and overheads. Field programmable devices present special interest for smart cameras design: flexibility, reconfigurability and parallel processing skills are some specially important features. In this paper we present a FPGA-based smart camera research platform. The hardware architecture is described, and some design issues are discussed. Our goal is to use the possibility to reconfigure the FPGA device in order to adapt the system architecture to a given application. To that, a design methodology, based on pre-programmed processing elements, is proposed and sketched. Some implementation issues are discussed and a template tracking application is given as example, with its experimental results.

An adaptive focus-of-attention model for video surveillance and monitoring

Abstract: In current video surveillance systems, commercial pan/tilt/zoom (PTZ) cameras typically provide naive (or no) automatic scanning functionality to move a camera across its complete viewable field. However, the lack of scene-specific information inherently handicaps these scanning algorithms. We address this issue by automatically building an adaptive, focus-of-attention, scene-specific model using standard PTZ camera hardware. The adaptive model is constructed by first detecting local human activity (i.e., any translating object with a specific temporal signature) at discrete locations across a PTZ camera’s entire viewable field. The temporal signature of translating objects is extracted using motion history images (MHIs) and an original, efficient algorithm based on an iterative candidacy-classification-reduction process to separate the target motion from noise. The target motion at each location is then quantified and employed in the construction of a global activity map for the camera. We additionally present four new camera scanning algorithms which exploit this activity map to maximize a PTZ camera’s opportunity of observing human activity within the camera’s overall field of view. We expect that these efficient and effective algorithms are implementable within current commercial camera systems.

Fast vision-based minimum distance determination between known and unkown objects

Abstract: We present a method for quickly determining the minimum distance between multiple known and multiple unkown objects within a camera image. Known objects are objects with known geometry, position, orientation, and configuration. Unkown objects are objects which have to be detected by a vision sensor but with unkown geometry, position, orientation and configuration. The known objects are modeled and expanded in 3D and then projected into a camera image. The camera image is classified into object areas including known and unknown objects and into non-object areas. The distance is conservatively estimated by searching for the largest expansion radius where the projected model does not intersect the object areas classified as unknown in the camera image. The method requires only minimal computation times and can be used for surveillance and safety applications.

Monitoring system, camera, and video encoding method

Abstract: A monitor system is provided for making it possible to efficiently determine a coding quantity of each camera in accordance with the movement of an object. In each camera ( 101 ) composing the monitor system, based on a position of the object detected by a sensor ( 106 ) and position specifying information of neighboring cameras and cooperation parameters stored at a neighboring camera information memory unit ( 405 ), ( 1 ) a distribution pattern indicative of a distribution of values of the cooperation parameter in a space where a plurality of the cameras exist forms a concentric circle defined from the object as a starting point, and ( 2 ) a cooperation parameter renewal unit ( 413 ) for renewing the cooperation parameters so that a target coding quantity of the camera, which picks up an image of the object, can be larger than a target quantity of the camera, which has not picked up an image of the object, and the camera is provided with a target coding quantity determining unit ( 410 ) for determining the target coding quantity allocated to its own camera based on the cooperation parameters of the neighboring camera and the target coding quantity stored at the neighboring camera information memory unit ( 405 ).

Camera mote with a high-performance parallel processor for real-time frame-based video processing

Abstract: This paper describes a new smart camera mote with a high performance SIMD (single-instruction multiple-data) processor. Previous versions of our camera mote were equipped with IC3D, a line-based processor. The mote described in this paper is equipped with Xetal-II, a processor designed for frame-based real-time video analysis. The processor uses 320 processing elements in parallel to achieve performance figures of more than 100 GOPS with a power consumption of 600 mWatt at peak performance. The IC has a 10 Mbit internal memory cache to store and work on 4 VGA frames. The internal bandwidth to this memory is more than 1.5 Tbit/s allowing multiple passes over the images within frametime. Augmented with hardware tools for object processing, the new mote opens the door for embedded active vision applications and other iterative techniques such as watershedding and distance transforms in collaborative camera networks.

An efficient extrinsic calibration of a multiple laser scanners and cameras' sensor system on a mobile platform

Abstract: This work is motivated by a development of a portable and low-cost solution for road mapping in downtown area using a number of laser scanners and video cameras that are mounted on an intelligent vehicle. Sensors on the vehicle platform are considered to be removable, so that extrinsic calibrations are required after each sensors' setting up. Extrinsic calibration might always happen at or near measuremental sites, so that the facilities such as specially marked large environment could not be supposed as a given in the process. In this research, we present a practical method for extrinsic calibration of multiple laser scanners and video cameras that are mounted on a vehicle platform. Referring to a fiducial coordinate system on vehicle platform, a constraint between the data of a laser scanner and of a video camera is established. It is solved in an iterative way to find a best solution from the laser scanner and from the video camera to the fiducial coordinate system. On the other hand, all laser scanners and video cameras are calibrated for each laser scanner and video camera pair that has common in feature points in a sequential way. An experiment is conducted using the data measured on a normal street road. Calibration results are demonstrated by fusing the sensor data into a global coordinate system.

Extrinsic Camera Calibration for an On-board Two-Camera System without overlapping Field of View

Abstract: Most recent developments in car technology promise that future cars will be equipped with many cameras facing different directions (e.g.: headlights, wing mirrors, break lights etc.). This work investigates the possibility of letting the cameras calibrate and localize themselves relative to each other by tracking one arbitrary and fixed calibration object (e.g.: a traffic sign). Since the fields of view for each camera may not be overlapping, the calibration object serves as logical connection between different views. Under the assumption that the intrinsic camera parameters and the vehicle's speed are known, we suggest a method for computing the extrinsic camera parameters (rotation, translation) for a two-camera system, where one camera is defined as the origin.