Showing papers on "Smart camera published in 2009"

Systems and methods for suggesting meta-information to a camera user

Abstract: According to one embodiment of the invention, a camera captures an image. The image is transmitted to a server for image recognition processing. The camera receives information from the server, including an indication of information to suggest to a user for meta-tagging the image. The suggested information may be based, for example, on a comparison of the image with meta-information stored by the server and/or a database of stored images. The camera asks the user if the user would like to meta-tag the image with the information. Optionally, the camera receives an indication from the user that the user would like to meta-tag the image with the suggested information, and the camera meta-tags the image with the information.

Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment

Abstract: The advent of high-resolution digital cameras and sophisticated multi-view stereo algorithms offers the promise of unprecedented geometric fidelity in image-based modeling tasks, but it also puts unprecedented demands on camera calibration to fulfill these promises. This paper presents a novel approach to camera calibration where top-down information from rough camera parameter estimates and the output of a multi-view-stereo system on scaled-down input images is used to effectively guide the search for additional image correspondences and significantly improve camera calibration parameters using a standard bundle adjustment algorithm (Lourakis and Argyros 2008). The proposed method has been tested on six real datasets including objects without salient features for which image correspondences cannot be found in a purely bottom-up fashion, and objects with high curvature and thin structures that are lost in visual hull construction even with small errors in camera parameters. Three different methods have been used to qualitatively assess the improvements of the camera parameters. The implementation of the proposed algorithm is publicly available at Furukawa and Ponce (2008b).

Multi-Camera Networks: Principles and Applications

Abstract: The first book, by the leading experts, on this rapidly developing field with applications to security, smart homes, multimedia, and environmental monitoringComprehensive coverage of fundamentals, algorithms, design methodologies, system implementation issues, architectures, and applicationsPresents in detail the latest developments in multi-camera calibration, active and heterogeneous camera networks, multi-camera object and event detection, tracking, coding, smart camera architecture and middlewareThis book is the definitive reference in multi-camera networks. It gives clear guidance on the conceptual and implementation issues involved in the design and operation of multi-camera networks, as well as presenting the state-of-the-art in hardware, algorithms and system development. The book is broad in scope, covering smart camera architectures, embedded processing, sensor fusion and middleware, calibration and topology, network-based detection and tracking, and applications in distributed and collaborative methods in camera networks. This book will be an ideal reference for university researchers, R&D engineers, computer engineers, and graduate students working in signal and video processing, computer vision, and sensor networks.Hamid Aghajan is a Professor of Electrical Engineering (consulting) at Stanford University. His research is on multi-camera networks for smart environments with application to smart homes, assisted living and well being, meeting rooms, and avatar-based communication and social interactions. He is Editor-in-Chief of Journal of Ambient Intelligence and Smart Environments, and was general chair of ACM/IEEE ICDSC 2008.Andrea Cavallaro is Reader (Associate Professor) at Queen Mary, University of London (QMUL). His research is on target tracking and audiovisual content analysis for advanced surveillance and multi-sensor systems. He serves as Associate Editor of the IEEE Signal Processing Magazine and the IEEE Trans. on Multimedia, and has been general chair of IEEE AVSS 2007, ACM/IEEE ICDSC 2009 and BMVC 2009. The first book, by the leading experts, on this rapidly developing field with applications to security, smart homes, multimedia, and environmental monitoringComprehensive coverage of fundamentals, algorithms, design methodologies, system implementation issues, architectures, and applicationsPresents in detail the latest developments in multi-camera calibration, active and heterogeneous camera networks, multi-camera object and event detection, tracking, coding, smart camera architecture and middleware

Systems and methods for remote camera control

Abstract: A camera capable of capturing still images and video and included in a portable media device can be controlled remotely using an accessory. The accessory can register with the PMD to automatically receive notifications whenever there is a change in the camera state. The camera states can include mode, operation status, and configuration settings. The accessory can send instructions to a camera application that interfaces with the camera to control the camera. The accessory can remotely activate the camera, change camera mode, and send instructions to operate the camera. The accessory and the PMD can concurrently control the camera. The PMD can send the captured still images and recorded video to the accessory for preview and receive instructions from the accessory on disposition of the still images and the video.

Multiple View and Multiple Object Processing in Wide-Angle Video Camera

Abstract: Methods and systems of transmitting a plurality of views from a video camera are disclosed. A multi object processing camera captures a wide-angle field of view in high resolution and a processing circuit executes a plurality of software tasks on a plurality of regions extracted from the wide-angle view. Multiple objects can be processed by the camera to detect various events, and the results of the processing transmitted to a base station. The camera removes the need for mechanical pan, tilt, and zoom apparatus by correcting distortions in the electronic image introduced by the wide-angle optical system and image sensor to electronically emulate the pan, tilt, and zoom movement.

Camera applications in a handheld device

Abstract: A hand held device containing at least one camera can perform various functions. In some embodiments, digitized images taken with two or more camera lenses having different fixed focal lengths may be combined to emulate a high-resolution optical zoom, and may also permit the device to perform 3D applications. In other embodiments, a device containing a camera may perform as a bar code reader, and may wirelessly transmit and/or visually present a bar code to other devices. Movable optics may permit the same camera to focus on either distant objects or on a close-up bar code.

Systems and methods for providing immersive displays of video camera information from a plurality of cameras

Abstract: A system for providing stitched video from a first camera and a second camera to an electronic display system includes a processing circuit configured to associate a view a first camera with an approximate location. The processing circuit is further configured to build relationship data between the first camera and a second camera using the approximate location. The processing circuit is further configured to transform video from the first camera relative to video from the second camera, the transformation based on the relationship data. The processing circuit is further configured to use the transformed video to cause the stitched video to be provided to the electronic display system.

Light field video stabilization

Abstract: We describe a method for producing a smooth, stabilized video from the shaky input of a hand-held light field video camera—specifically, a small camera array. Traditional stabilization techniques dampen shake with 2D warps, and thus have limited ability to stabilize a significantly shaky camera motion through a 3D scene. Other recent stabilization techniques synthesize novel views as they would have been seen along a virtual, smooth 3D camera path, but are limited to static scenes. We show that video camera arrays enable much more powerful video stabilization, since they allow changes in viewpoint for a single time instant. Furthermore, we point out that the straightforward approach to light field video stabilization requires computing structure-from-motion, which can be brittle for typical consumer-level video of general dynamic scenes. We present a more robust approach that avoids input camera path reconstruction. Instead, we employ a spacetime optimization that directly computes a sequence of relative poses between the virtual camera and the camera array, while minimizing acceleration of salient visual features in the virtual image plane. We validate our novel method by comparing it to state-of-the-art stabilization software, such as Apple iMovie and 2d3 SteadyMove Pro, on a number of challenging scenes.

Camera housing with integrated expansion module

Abstract: A camera housing includes an integrated expansion module for providing expanded functionality to a camera (e.g., a display screen). Different embodiments of the housing include different expansion modules for adding a variety of features to the camera when the camera is placed in the housing. Thus, a user may modify features of a camera by swapping the housing. Furthermore, a user may add a feature to an existing camera without needing to purchase a new camera.

SCOPES: Smart Cameras Object Position Estimation System

Abstract: Wireless camera sensor networks have to balance the conflicting challenges imposed by the detection performance, latency and lifetime requirements in surveillance applications. While previous studies for camera sensor networks have addressed these issues separately, they have not quantified the trade-offs between these requirements. In this paper, we discuss the design and implementation of SCOPES, a distributed Smart Camera Object Position Estimation System that balances the trade-offs associated with camera sensor networks. The main contribution of the paper is the extensive evaluation of parameters affecting the performance of the system through analysis, simulation and experimentation in real-life conditions. Our results demonstrates the effectiveness of SCOPES, which achieves detection probabilities ranging from 84% to 98% and detection latencies from 10 seconds to 18 seconds. Moreover, by using coordination schemes, the detection performance of SCOPES was improved with increased system lifetime. SCOPES highlights that intelligent system design can compensate for resource-constrained hardware and computationally simple data processing algorithms.

HealthAware: Tackling obesity with health aware smart phone systems

Abstract: Obesity prevention requires individuals to have healthy eating and physical activity awareness in their daily lives. New technological products have been explored to help monitoring individual health behaviors. Smart phones are ubiq­uitous hand-held tools with rich embedded sensors and limited computing power. A smart phone is a prospective candidate to integrate physical exercise analysis and food intake monitoring. This paper presents a novel health-aware smart phone system (HealthAware) which utilizes the embedded accelerometer to monitor daily physical activities and the built-in camera to analyze food items. The system presents the user's physical activity counts at real time to remind how much daily activity is needed to keep healthy. The user takes pictures of food items and the system provides health related information regarding to the food item intakes. The system is composed of an on-device database which holds the user specific data and food item information. HealthAware is a real time practical system to prevent obesity by enhancing individual daily healthy behavior awareness.

Camera data management and user interface apparatuses, systems, and methods

Abstract: In certain embodiments, a graphical user interface ("GUI") including a live camera sensor view is displayed and, in response to the capture of a camera image, an image manager pane is displayed together with the live camera sensor view in the graphical user interface. The image manager pane includes a visual indicator representative of the captured camera image. In certain embodiments, a camera image is captured and automatically assigned to a session based on a predefined session grouping heuristic. In certain embodiments, data representative of a captured camera image is provided to a content distribution subsystem over a network, and the content distribution subsystem is configured to distribute data representative of the camera image to a plurality of predefined destinations.

System and method for runtime determination of camera miscalibration

Abstract: This invention provides a system and method for runtime determination (self-diagnosis) of camera miscalibration (accuracy), typically related to camera extrinsics, based on historical statistics of runtime alignment scores for objects acquired in the scene, which are defined based on matching of observed and expected image data of trained object models. This arrangement avoids a need to cease runtime operation of the vision system and/or stop the production line that is served by the vision system to diagnose if the system's camera(s) remain calibrated. Under the assumption that objects or features inspected by the vision system over time are substantially the same, the vision system accumulates statistics of part alignment results and stores intermediate results to be used as indicator of current system accuracy. For multi-camera vision systems, cross validation is illustratively employed to identify individual problematic cameras. The system and method allows for faster, less-expensive and more-straightforward diagnosis of vision system failures related to deteriorating camera calibration.

Visibility transition planning for dynamic camera control

Abstract: We present a real-time camera control system that uses a global planning algorithm to compute large, occlusion free camera paths through complex environments. The algorithm incorporates the visibility of a focus point into the search strategy, so that a path is chosen along which the focus target will be in view. The efficiency of our algorithm comes from a visibility-aware roadmap data structure that permits the precomputation of a coarse representation of all collision-free paths through an environment, together with an estimate of the pair-wise visibility between all portions of the scene. Our runtime system executes a path planning algorithm using the precomputed roadmap values to find a coarse path, and then refines the path using a sequence of occlusion maps computed on-the-fly. An iterative smoothing algorithm, together with a physically-based camera model, ensures that the path followed by the camera is smooth in both space and time. Our global planning strategy on the visibility-aware roadmap enables large-scale camera transitions as well as a local third-person camera module that follows a player and avoids obstructed viewpoints. The data structure itself adapts at run-time to dynamic occluders that move in an environment. We demonstrate these capabilities in several realistic game environments.

The Multiple-Camera 3-D Production Studio

Abstract: Multiple-camera systems are currently widely used in research and development as a means of capturing and synthesizing realistic 3-D video content. Studio systems for 3-D production of human performance are reviewed from the literature, and the practical experience gained in developing prototype studios is reported across two research laboratories. System design should consider the studio backdrop for foreground matting, lighting for ambient illumination, camera acquisition hardware, the camera configuration for scene capture, and accurate geometric and photometric camera calibration. A ground-truth evaluation is performed to quantify the effect of different constraints on the multiple-camera system in terms of geometric accuracy and the requirement for high-quality view synthesis. As changing camera height has only a limited influence on surface visibility, multiple-camera sets or an active vision system may be required for wide area capture, and accurate reconstruction requires a camera baseline of 25deg, and the achievable accuracy is 5-10-mm at current camera resolutions. Accuracy is inherently limited, and view-dependent rendering is required for view synthesis with sub-pixel accuracy where display resolutions match camera resolutions. The two prototype studios are contrasted and state-of-the-art techniques for 3-D content production demonstrated.

Digital Camera Imaging System Simulation

Abstract: A digital camera is a complex system including a lens, a sensor (physics and circuits), and a digital image processor, where each component is a sophisticated system on its own. Since prototyping a digital camera is very expensive, it is highly desirable to have the capability to explore the system design tradeoffs and preview the system output ahead of time. An empirical digital imaging system simulation that aims to achieve such a goal is presented. It traces the photons reflected by the objects in a scene through the optics and color filter array, converts photons into electrons with consideration of noise introduced by the system, quantizes the accumulated voltage to digital counts by an analog-to-digital converter, and generates a Bayer raw image just as a real camera does. The simulated images are validated against real system outputs and show a close resemblance to the images captured under similar condition at all illumination levels.

Directing camera behavior in 3-D imaging system

Abstract: A method of associating a computer generated camera with an object in a three- dimensional computer generated space. The method receives a command to associate the camera with an object in the simulated space. Based on the command the method determines a path for moving the camera to a position near the object and aiming the camera at the object such that the object. The method creates a video from the simulated camera's perspective of the three- dimensional simulated space.

Camera system with autonomous miniature camera and light source assembly and method for image enhancement

Abstract: The present Invention relates to a camera system suitable for use in minimally invasive surgery (MIS ), among other applications. In at least one embodiment, the camera system includes an autonomous miniature camera, a light source assembly providing features such as steerable illumination and a variable radiation angle, and a control and processing unit for processing images acquired by the camera Io generate improved images having reduced blurring using a deblurring algorithm.

Wireless smart camera system and method for 3-D visualization of surveillance

Abstract: Systems and methods having a 3D model of a space provides a 3D context for the inputs from the ICDs; inputs from the ICDs, including direct cross-communication information, location, settings, environment conditions, and inputs (video, audio, temperature, other sensors), being visually represented on a GUI independently and in the 3D context for simultaneous display of all the info, and analytics based on the info, including activity density within the 3D context based on the inputs, for surveillance and analysis of target environment(s).

Vision-based rain sensing with an in-vehicle camera

Abstract: An intelligent wiper speed adjustment system can be found in most middle and upper class cars. A core piece of this gadget is the rain sensor on the windshield. With the upcoming number of cars being equipped with an in-vehicle camera for vision-based applications the call for integrating all sensors in the area of the rearview mirror into one device rises to reduce the number of parts and variants. In this paper, functionality of standard rain sensors and different vision-based approaches are explained and a novel rain sensing concept based on an automotive in-vehicle camera for Driver Assistance Systems (DAS) is developed to enhance applicability. Hereby, the region at the bottom of the field of view (FOV) of the imager is used to detect raindrops, while the upper part of the image is still usable for other vision-based applications. A simple algorithm is set up to keep the additional processing time low and to quantitatively gather the rain intensity. Mechanisms to avoid false activations of the wipers are introduced. First experimental experiences based on real scenarios show promising results.

Camera control in computer graphics: models, techniques and applications

Abstract: This course summarizes the motivations and requirements for camera control, presents an overview of the state of the art, and examines promising avenues and hot topics for future research. It classifies the various techniques and identifies the representational limits and commitments of each. Approaches range from completely interactive techniques based on the possible mappings between a user's input and the camera parameters to completely automated paradigms in which the camera moves and jumps according to high-level, scenario-oriented goals. Between these extremes lie approaches with more limited expressiveness that use a range of algebraic and constraint-based optimization techniques. The course includes a number of live examples from both commercial systems and research prototypes, and it emphasizes the tough issues facing application developers, such as real-time handling of visibility for complex multiple targets in dynamic environments (multi-object tracking).

EYEWATCHME—3D Hand and object tracking for inside out activity analysis

Abstract: This paper investigates the “inside-out” recognition of everyday manipulation tasks using a gaze-directed camera, which is a camera that actively directs at the visual attention focus of the person wearing the camera. We present EYEWATCHME, an integrated vision and state estimation system that at the same time tracks the positions and the poses of the acting hands, the pose that the manipulated object, and the pose of the observing camera. Taken together, EYEWATCHME provides comprehensive data for learning predictive models of vision-guided manipulation that include the objects people are attending, the interaction of attention and reaching/grasping, and the segmentation of reaching and grasping using visual attention as evidence. Key technical contributions of this paper include an ego view hand tracking system that estimates 27 DOF hand poses. The hand tracking system is capable of detecting hands and estimating their poses despite substantial self-occlusion caused by the hand and occlusions caused by the manipulated object. EYEWATCHME can also cope with blurred images that are caused by rapid eye movements. The second key contribution is the of the integrated activity recognition system that simultaneously tracks the attention of the person, the hand poses, and the poses of the manipulated objects in terms of a global scene coordinates. We demonstrate the operation of EYEWATCHME in the context of kitchen tasks including filling a cup with water.

The graph camera

Abstract: A conventional pinhole camera captures only a small fraction of a 3-D scene due to occlusions. We introduce the graph camera, a non-pinhole with rays that circumvent occluders to create a single layer image that shows simultaneously several regions of interest in a 3-D scene. The graph camera image exhibits good continuity and little redundancy. The graph camera model is literally a graph of tens of planar pinhole cameras. A fast projection operation allows rendering in feed-forward fashion, at interactive rates, which provides support for dynamic scenes. The graph camera is an infrastructure level tool with many applications. We explore the graph camera benefits in the contexts of virtual 3-D scene exploration and summarization, and in the context of real-world 3-D scene visualization. The graph camera allows integrating multiple video feeds seamlessly, which enables monitoring complex real-world spaces with a single image.

Adaptive power control for solar harvesting multimodal wireless smart camera

Abstract: Energy efficiency for wireless smart camera networks is one of the major efforts in the distributed monitoring and surveillance community. If video cameras are equipped with circuits that receive and convert energy from regenerative sources such as solar cells, an effective power management becomes essential for the design of small sized and perpetually powered devices, which can be deployed unattended for years and feature smart vision applications. In this paper we present a simple but optimal power management tailored for multi-modal video sensor nodes and based on model predictive controller (MPC) principles. The system is designed for low-power and low-cost video surveillance and exploits small solar cells for battery recharging and Pyroelectric InfraRed (PIR) sensors to provide low-power monitoring when the camera is not needed. The aim of this work is to show how an adaptive controller helps the system to improve its performance while outperforming naive power management policies. Simulation results and measurements on the video sensor node demonstrate the efficiency of our approach.

Providing camera-based services using a portable communication device

Abstract: Camera-based services are provided to a user of a portable communication device by recognizing text contained in an image. An image of an environment is captured using a camera within the portable communication device so as to obtain image data. The image data is processed such that text data is recognized and extracted from the image data. Data related to the text data is then output in a form recognizable by a user of the portable communication device. The text data can be processed on the portable communication device to obtain the data related to the text data. Alternatively, the processing is performed by a processing unit external to the portable communication device. Translated and audio versions of the text data are output to the user. One camera-based service provides price and product information related to a product described in an image captured by the camera.

Camera pod that captures images or video when triggered by a mobile device

Abstract: The present invention provides methods and systems for remotely recording an image using a mobile device. In one embodiment, a user transmits a message using a mobile device to activate a camera pod to record an image. The user makes adjustments to the settings of the camera pod using the mobile device, and in some instances, initiates a trigger sequence using the mobile device. The camera pod records the image and transmits the image to the mobile device, enabling the user to preview the image. In some instances, the camera pod transmits the image to a storage server, where it is stored in an album. In one embodiment, the storage server creates a webpage with the captured visual record that the user can retrieve using a personal computer.

Smart Cameras: Fundamentals and Classification

Abstract: Since the late 1990s smart cameras have gained significant popularity and market acceptance, especially in surveillance and machine vision industries. A smart camera is a vision system that can perform tasks far beyond simply taking photos and recording videos. Thanks to the purposely developed intelligent image processing and pattern recognition algorithms running on increasingly powerful micro-processors, smart cameras can detect motion, measure objects, read vehicle number plates, and even recognize human behavior. They are essential components for building active and automated control systems for many applications and hold great promise for being pervasive and intelligent sensors in the future. In this chapter we provide a technical definition of smart cameras and analyze the reasons behind their increasing popularity. We also discuss the characteristics and advantages of smart cameras. Later in the chapter a tentative classification of smart cameras is provided based on their system architectures.

Intelligent headlight control using camera sensors

Abstract: This paper describes our recent work on intelligently controlling a vehicle's headlights using a forward-facing camera sensor. Specifically, we aim to automatically control its beam state (high beam or low beam) during a night-time drive based on the detection of oncoming/overtaking/leading traffics as well as urban areas from the videos captured by the camera. A three-level decision framework is proposed which includes various types of image and video content analysis, an SVM-based learning mechanism and a frame-level decision making mechanism. Both video and context information have been exploited to accomplish the task. Online test drives as well as offline evaluations on tens of videos have validated the robustness and effectiveness of the proposed system.

Multiple-view object recognition in band-limited distributed camera networks

Abstract: In this paper, we study the classical problem of object recognition in low-power, low-bandwidth distributed camera networks. The ability to perform robust object recognition is crucial for applications such as visual surveillance to track and identify objects of interest, and compensate visual nuisances such as occlusion and pose variation between multiple camera views. We propose an effective framework to perform distributed object recognition using a network of smart cameras and a computer as the base station. Due to the limited bandwidth between the cameras and the computer, the method utilizes the available computational power on the smart sensors to locally extract and compress SIFT-type image features to represent individual camera views. In particular, we show that between a network of cameras, high-dimensional SIFT histograms share a joint sparse pattern corresponding to a set of common features in 3-D. Such joint sparse patterns can be explicitly exploited to accurately encode the distributed signal via random projection, which is unsupervised and independent to the sensor modality. On the base station, we study multiple decoding schemes to simultaneously recover the multiple-view object features based on the distributed compressive sensing theory. The system has been implemented on the Berkeley CITRIC smart camera platform. The efficacy of the algorithm is validated through extensive simulation and experiments.