Showing papers on "Smart camera published in 2017"

Deep learning for decentralized parking lot occupancy detection

Abstract: We propose an effective CNN architecture for visual parking occupancy detectionThe CNN architecture is small enough to run on smart camerasThe proposed solution performs and generalizes better than other SotA approachesWe provide a new training/validation dataset for parking occupancy detection A smart camera is a vision system capable of extracting application-specific information from the captured images The paper proposes a decentralized and efficient solution for visual parking lot occupancy detection based on a deep Convolutional Neural Network (CNN) specifically designed for smart cameras This solution is compared with state-of-the-art approaches using two visual datasets: PKLot, already existing in literature, and CNRPark-EXT The former is an existing dataset, that allowed us to exhaustively compare with previous works The latter dataset has been created in the context of this research, accumulating data across various seasons of the year, to test our approach in particularly challenging situations, exhibiting occlusions, and diverse and difficult viewpoints This dataset is public available to the scientific community and is another contribution of our research Our experiments show that our solution outperforms and generalizes the best performing approaches on both datasets The performance of our proposed CNN architecture on the parking lot occupancy detection task, is comparable to the well-known AlexNet, which is three orders of magnitude larger

Cross-Correlation-Based Structural System Identification Using Unmanned Aerial Vehicles.

Abstract: Computer vision techniques have been employed to characterize dynamic properties of structures, as well as to capture structural motion for system identification purposes. All of these methods leverage image-processing techniques using a stationary camera. This requirement makes finding an effective location for camera installation difficult, because civil infrastructure (i.e., bridges, buildings, etc.) are often difficult to access, being constructed over rivers, roads, or other obstacles. This paper seeks to use video from Unmanned Aerial Vehicles (UAVs) to address this problem. As opposed to the traditional way of using stationary cameras, the use of UAVs brings the issue of the camera itself moving; thus, the displacements of the structure obtained by processing UAV video are relative to the UAV camera. Some efforts have been reported to compensate for the camera motion, but they require certain assumptions that may be difficult to satisfy. This paper proposes a new method for structural system identification using the UAV video directly. Several challenges are addressed, including: (1) estimation of an appropriate scale factor; and (2) compensation for the rolling shutter effect. Experimental validation is carried out to validate the proposed approach. The experimental results demonstrate the efficacy and significant potential of the proposed approach.

Towards Full-System Energy-Accuracy Tradeoffs: A Case Study of An Approximate Smart Camera System

Abstract: The intrinsic error resilience exhibited by emerging application domains enables a new dimension for energy optimization of computing systems, namely the introduction of a controlled amount of approximations during system operation in exchange for substantial energy savings. Prior work in the area of approximate computing has focused on individual subsystems of a computing system, e.g., the computational subsystem or the memory subsystem. Since they focus only on individual subsystems, these techniques are unable to exploit the large energy-saving opportunities that stem from adopting a full-system perspective and approximating multiple subsystems of a computing platform simultaneously in a coordinated manner. This paper proposes a systematic methodology to perform joint approximations across different subsystems, leading to significant energy benefits compared to approximating individual subsystems in isolation. We use the example of a smart camera system that executes various computer vision and image processing applications to illustrate how the sensing, memory, and processing subsystems can all be approximated synergistically. We have implemented such an approximate smart camera system using an Altera Stratix IV GX FPGA development board, a Terasic TRDBD5M 5 Megapixel camera module, and a 1GB DDR3 SODIMM module. Experimental results obtained using six application benchmarks demonstrate significant energy savings (around 7.5× on average) for minimal (< 1%) loss in application quality. Compared to approximating a single subsystem, the proposed full-system approximation methodology achieves additional energy benefits of 3.5× – 5.5× on average for minimal (< 1%) quality loss.

Optical Camera Communication: Motion over Camera

Abstract: OCC is a pragmatic version of VLC based on a smart device camera that allows easier implementation of various services in smart devices. This article presents a flexible and novel motion detection scheme over a smart device camera in OCC. The motion detection is conducted in conjunction with a static downlink optical camera communication, where a mobile phone front camera is employed as the receiver and an 8 x 8 dot matrix LED as the transmitter. In addition to the 8 x 8 dot matrix LED for data transmission, 10 white LEDs are also employed for providing illumination, acquiring camera focus, and light metering. The motion detection or MoC is designed to detect the user's finger movement through the OCC link via the camera. A simple but efficient quadrant division based motion detection algorithm is proposed for reliable and accurate detection of motion. The experiment and simulation results demonstrate that the proposed scheme is able to detect motion with a success probability of up to 96 percent in the mobile phone camera based OCC. It is envisioned that the proposed motion detection can facilitate cost-effective and convenient smart home environments in the OCC, where the provision of illumination and short-range wireless communications has already been addressed.

Automatic Camera Calibration Using Active Displays of a Virtual Pattern.

Abstract: Camera calibration plays a critical role in 3D computer vision tasks. The most commonly used calibration method utilizes a planar checkerboard and can be done nearly fully automatically. However, it requires the user to move either the camera or the checkerboard during the capture step. This manual operation is time consuming and makes the calibration results unstable. In order to solve the above problems caused by manual operation, this paper presents a full-automatic camera calibration method using a virtual pattern instead of a physical one. The virtual pattern is actively transformed and displayed on a screen so that the control points of the pattern can be uniformly observed in the camera view. The proposed method estimates the camera parameters from point correspondences between 2D image points and the virtual pattern. The camera and the screen are fixed during the whole process; therefore, the proposed method does not require any manual operations. Performance of the proposed method is evaluated through experiments on both synthetic and real data. Experimental results show that the proposed method can achieve stable results and its accuracy is comparable to the standard method by Zhang.

Erratum to: Geocasting-based synchronization of Almanac on the maritime cloud for distributed smart surveillance

Abstract: A number of recent maritime accidents strongly imply the need of distributed smart surveillance. The maritime cloud, proposed as communications infrastructure of e-Navigation, is one of the most optimal infrastructure systems in the smart surveillance environment. To maintain the safe maritime environment, security in the distributed smart surveillance environment is critical, but research on security of the maritime cloud, which will be adopted as major communications infrastructure in the smart surveillance system, is still in the fledging stage. In this regard, this paper suggested a safe synchronization method of Almanac, which is necessary to provide unimpeded maritime cloud service. Almanac plays a role of a telephone directory and it should be shared in the latest version in communicating between vessels or a vessel and land. In other words, synchronization of Almanac between offshore and vessels is required to safely deliver major video information collected by the distributed smart camera. The method proposed in this paper enables geocasting based synchronization between vessels, which is suitable for maritime conditions, and does not expose information in the course of synchronization even in the case of broadcasting through an unsafe channel. In addition, the method ensures integrity based on block ID and supports delta update, thereby minimizing bandwidth and boosting performance.

A Real Time Object Recognition and Counting System for Smart Industrial Camera Sensor

Abstract: In the industry 4.0, factories around the world grow automated and intelligent, and where smart camera plays an important role. Smart camera is equipped with processor, memory, communication interface, and operating system, so it can process large amounts of data in advance to assist follow-up automatic inspection and judgment. Additionally, since smart camera is an independent system, it will not affect the original system of factories, which is an immense advantage in troubleshooting. Besides, thanks to technology breakthroughs in recent years, using Graphics Processing Unit (GPU) to implementing tons of parallel computing helps to significantly boost the overall efficiency. Therefore, when a rising number of factories consider improving production capacity of production lines, how to use GPU to assist the improvement is an important issue. Based on this scenario, this paper used NVidia Tegra TX1 platform with 256 GPU CUDA cores and Quad-core ARM Cortex A57 processor and Basler USB 3.0 industrial camera to simulate a smart industrial camera, which has GPU and can perform a myriad of complex computations. This paper designed how to recognize and count objects in a real time manner in a high-speed industrial inspection environment with large volumes of data, so as to verify the concept (smart camera with GPU cores) we proposed. The experimental results proved our ideas, and the software design architecture provided in this paper is a simple and efficient design. In the future application in the Internet of Things or the Internet of Everything, this structure can be a valuable reference.

A Novel Approach for Smart and Cost Effective IoT Based Elderly Fall Detection System Using Pi Camera

Abstract: In our society, yearly 33% of senior citizens fall down in their residential area. When our family members come to old age, it becomes essential to monitor them for their in-home health and safety. Although they dwell in home, because of illness and weedy joints they have a big threat of falling down in any corner of home premises. In such circumstances, it becomes significant to recognize if an elderly person has fallen so that he/she can get quick help on time. Physically handicapped person on wheelchair also requires to be monitored for fall detection. Currently CCTV Camera-based monitoring systems are in existence but these systems are very expensive. Common man cannot afford such system. For these reasons we propose a novel, smart and cost-effective fall-detection system by merging latest technology as Internet of Things and existing algorithms like Motion History Image and C-Motion. This system uses low-cost Pi Camera mounted on Raspberry Pi to monitor and detect person's fall-like movements. Pi Camera is a smart camera can be easily fixed on windows and walls of living room. System will be watching keenly for fall detection and unexpected motion changes in targeted person. An unexpected abrupt change with peak in the system is treated as a fall. In case of the person did not fall and alarm was false, then the system will have a provision to stop the alert within 5 seconds. If within stipulated time, person does not press stop button, system will consider it's as fall- activity and automatically sends an alert message via Wi-Fi to victim's take-givers for quick medical help.

Smart Cameras Are Making Our Beaches Safer: A 5G-Envisioned Distributed Architecture for Safe, Connected Coastal Areas

Abstract: With the explosive growth of smart city and Internet of Things (IoT) applications in recent years, a series of efforts has been undertaken to bring more intelligence to the smart cities and public transportation to solve essential problems like human surveillance for safety, traffic, and congestion management. In this article, we define a fifth-generation (5G)-envisioned architecture to enable cooperative sensing in the IoT for smart beaches and intelligent transportation systems and to develop a reference end-to-end implementation exploiting big data. Currently, many beaches are able to offer urban sophistication, but they lack a safety and connectivity infrastructure. Similarly, the dependence on road transport in our daily lives has grown massively, along with the problems arising from its use: permanent congestion, energy waste, and excessive carbon dioxide (CO2) emissions. We aim to address these safety and connectivity challenges and thus make a qualitative leap toward the future IoT and big-data application for smart cities and mobility. We present an architecture that raises the implementation of a platform to merge and integrate heterogeneous data sources into a common system and provide a set of advanced tools for the monitoring, simulation, and prediction to achieve a safer, less congested, and better connected beach. Our results will enable advanced surveillance at beaches along with traffic and travel management strategies based on reliable, real-time input data. The effectiveness of such new strategies, together with the proposed system, is assessed in field trials.

Unsupervised Adaptive Re-identification in Open World Dynamic Camera Networks

Abstract: Person re-identification is an open and challenging problem in computer vision. Existing approaches have concentrated on either designing the best feature representation or learning optimal matching metrics in a static setting where the number of cameras are fixed in a network. Most approaches have neglected the dynamic and open world nature of the re-identification problem, where a new camera may be temporarily inserted into an existing system to get additional information. To address such a novel and very practical problem, we propose an unsupervised adaptation scheme for re-identification models in a dynamic camera network. First, we formulate a domain perceptive re-identification method based on geodesic flow kernel that can effectively find the best source camera (already installed) to adapt with a newly introduced target camera, without requiring a very expensive training phase. Second, we introduce a transitive inference algorithm for re-identification that can exploit the information from best source camera to improve the accuracy across other camera pairs in a network of multiple cameras. Extensive experiments on four benchmark datasets demonstrate that the proposed approach significantly outperforms the state-of-the-art unsupervised learning based alternatives whilst being extremely efficient to compute.

A Sub-mW IoT-Endnode for Always-On Visual Monitoring and Smart Triggering

Abstract: This paper presents a fully programmable Internet of Things visual sensing node that targets sub-mW power consumption in always-on monitoring scenarios. The system features a spatial-contrast ${128\times 64}$ binary pixel imager with focal-plane processing. The sensor, when working at its lowest power mode ( ${10 ~\mu }\text{W}$ at 10 frames/s), provides as output the number of changed pixels. Based on this information, a dedicated camera interface, implemented on a low-power field-programmable gate array, wakes up an ultralow-power parallel processing unit to extract context-aware visual information. We evaluate the smart sensor on three always-on visual triggering application scenarios. Triggering accuracy comparable to RGB image sensors is achieved at nominal lighting conditions, while consuming an average power between 193 and ${277 ~\mu }\text{W}$ , depending on context activity. The digital subsystem is extremely flexible, thanks to a fully programmable digital signal processing engine, but still achieves $19 {\times }$ lower power consumption compared to MCU-based cameras with significantly lower on-board computing capabilities.

Trajectory-based vehicle tracking at low frame rates

Abstract: A new vehicle tracking method is proposed for an embedded traffic surveillance system.The proposed method demonstrates efficient tracking performance at a low frame rate.The proposed method employs greedy data association based on appearance and position similarities.To manage abrupt appearance changes, manifold learning is used.To manage abrupt motion changes, trajectory information is used to predict the next probable position. In smart cities, an intelligent traffic surveillance system plays a crucial role in reducing traffic jams and air pollution, thus improving the quality of life. An intelligent traffic surveillance should be able to detect and track multiple vehicles in real-time using only limited resources. Conventional tracking methods usually run at a high video-sampling rate, assuming that the same vehicles in successive frames are similar and move only slightly. However, in cost effective embedded surveillance systems (e.g., a distributed wireless network of smart cameras), video frame rates are typically low because of limited system resources. Therefore, conventional tracking methods perform poorly in embedded surveillance systems because of discontinuity of the moving vehicles in the captured recordings. In this study, we present a fast and light algorithm that is suitable for an embedded real-time visual surveillance system to detect effectively and track multiple moving vehicles whose appearance and/or position changes abruptly at a low frame rate. For effective tracking at low frame rates, we propose a new matching criterion based on greedy data association using appearance and position similarities between detections and trackers. To manage abrupt appearance changes, manifold learning is used to calculate appearance similarity. To manage abrupt changes in motion, the next probable centroid area of the tracker is predicted using trajectory information. The position similarity is then calculated based on the predicted next position and progress direction of the tracker. The proposed method demonstrates efficient tracking performance during rapid feature changes and is tested on an embedded platform (ARM with DSP-based system).

Energy Consumption Models for Smart Camera Networks

Abstract: Camera networks require heavy visual data processing and high-bandwidth communication. In this paper, we identify key factors underpinning the development of resource-aware algorithms and we propose a comprehensive energy consumption model for the resources employed by smart camera networks, which are composed of cameras that process data locally and collaborate with their neighbors. We account for the main parameters that influence consumption when sensing (frame size and frame rate), processing (dynamic frequency scaling and task load), and communication (output power and bandwidth) are considered. Next, we define an abstraction based on clock frequency and duty cycle that accounts for active, idle, and sleep operational states. We demonstrate the importance of the proposed model for a multicamera tracking task and show how one may significantly reduce consumption with only minor performance degradation when choosing to operate with an appropriately reduced hardware capacity. Moreover, we quantify the dependency on local computation resources and bandwidth availability. The proposed consumption model can be easily adjusted to account for new platforms, thus providing a valuable tool for the design of resource-aware algorithms and further research in resource-aware camera networks.

Panoptes: servicing multiple applications simultaneously using steerable cameras

Abstract: Steerable surveillance cameras offer a unique opportunity to support multiple vision applications simultaneously. However, state-of-art camera systems do not support this as they are often limited to one application per camera. We believe that we should break the one-to-one binding between the steerable camera and the application. By doing this we can quickly move the camera to a new view needed to support a different vision application. When done well, the scheduling algorithm can support a larger number of applications over an existing network of surveillance cameras. With this in mind we developed Panoptes, a technique that virtualizes a camera view and presents a different fixed view to different applications. A scheduler uses camera controls to move the camera appropriately providing the expected view for each application in a timely manner, minimizing the impact on application performance. Experiments with a live camera setup demonstrate that Panoptes can support multiple applications, capturing up to 80% more events of interest in a wide scene, compared to a fixed view camera.

Feature encoding in band-limited distributed surveillance systems

Abstract: Distributed surveillance systems have become popular in recent years due to security concerns. However, transmitting high dimensional data in bandwidth-limited distributed systems becomes a major challenge. In this paper, we address this issue by proposing a novel probabilistic algorithm based on the divergence between the probability distributions of the visual features in order to reduce their dimensionality and thus save the network bandwidth in distributed wireless smart camera networks. We demonstrate the effectiveness of the proposed approach through extensive experiments on two surveillance recognition tasks.

Online Multi-object k-coverage with Mobile Smart Cameras

Abstract: In this paper, we combine k-coverage with the Cooperative Multi-robot Observation of Multiple Moving Targets problem, defining the new problem of online multi-object k-coverage. We demonstrate the benefits of mobility in tackling this and propose a decentralised multi-camera coordination that improves this further. We show that coordination exploiting shared visual features is more effective than coordination based on Euclidean distance. When coordinating k-coverage in a distributed way, our results suggest that the design of coordination mechanisms should shift towards decisions being made by potential responders with up-to-date knowledge of their own state, rather than a coordinating camera.

Smart services: A case study on smarter public safety by a mobile app for University of São Paulo

Abstract: The University of Sao Paulo has faced public safety issues a long the years. Due to its size preventive surveillance by the campus security guard cannot be effective all the times. In order to bring a safer environment to its public of more than 60,000 daily users, a smart public safety system is being developed. This is a complex system, spread throughout all the University's campuses. It is composed of a smart surveillance cameras system, a back office system with a workflow engine and a mobile application within a collaborative concept. The smart cameras system is being deployed and the mobile application together with the back office system is being used this past year with satisfactory results. The mobile application is the user entry point to report several security and campus maintenance related issues that are automatically directed to the responder team for immediate action in the case of security or enters an automated workflow engine in the case of campus maintenance. This paper presents the structure created towards achieving a smarter public safety environment, details of the implementation, presents statistical data collected by the system showing its effectiveness and concludes showing the improvements introduced in the university community safety and welfare.

Joint face-detection and head-pose-angle-estimation using small-scale convolutional neural network (CNN) modules for embedded systems

Abstract: Embodiments described herein provide various examples of a joint face-detection and head-pose-angle-estimation system based on using a small-scale hardware CNN module such as the built-in CNN module in HiSilicon Hi3519 system-on-chip. In some embodiments, the disclosed joint face-detection and head-pose-angle-estimation system is configured to jointly perform multiple tasks of detecting most or all faces in a sequence of video frames, generating pose-angle estimations for the detected faces, tracking detected faces of a same person across the sequence of video frames, and generating “best-pose” estimation for the person being tracked. The disclosed joint face-detection and pose-angle-estimation system can be implemented on resource-limited embedded systems such as smart camera systems that are only integrated with one or more small-scale CNN modules. The proposed system in conjunction with using a subimage-based technique has made it possible to performance multiple face detection and face recognition tasks on high-resolution input images with small-scale low-cost CNN modules.

Embedded architecture for noise-adaptive video object detection using parameter-compressed background modeling

Abstract: Video processing algorithms are computationally intensive and place stringent requirements on performance and efficiency of memory bandwidth and capacity. As such, efficient hardware accelerations are inevitable for fast video processing systems. In this paper, we propose resource- and power-optimized FPGA-based configurable architecture for video object detection by integrating noise estimation, Mixture-of-Gaussian background modeling, motion detection, and thresholding. Due to large amount of background modeling parameters, we propose a novel Gaussian parameter compression technique suitable for resource- and power-constraint embedded video systems. The proposed architecture is simulated, synthesized and verified for its functionality, accuracy and performance on a Virtex-5 FPGA-based embedded platform by directly interfacing to a digital video input. Intentional exploitation of heterogeneous resources in FPGAs, and advanced design techniques such as heavy pipelining and data parallelism yield real-time processing of HD-1080p video streams at 30 frames per second. Objective and subjective evaluations to existing hardware-based methods show that the proposed architecture obtains orders of magnitude performance improvements, while utilizing minimal hardware resources. This work is an early attempt to devise a complete video surveillance system onto a stand-alone resource-constraint FPGA-based smart camera.

The Third Eye: A Shopping Assistant for the Visually Impaired

Abstract: This video describes HCI work that is part of a multi-university NSF Expedition in Computing "Visual Cortex on Silicon" (CCF 1317560) spanning vision science, computer vision, processor architecture, information retrieval, and human-computer interaction. Our driving application is a smart camera prosthetic enabling people with visual impairment to shop on their own. The video was originally shown during Big Ten television broadcasts in Fall 2016.

Smart cameras with onboard signcryption for securing IoT applications

Abstract: Cameras are expected to become key sensor devices for various internet of things (IoT) applications. Since cameras often capture highly sensitive information, security is a major concern. Our approach towards data security for smart cameras is rooted on protecting the captured images by signcryption based on elliptic curve cryptography (ECC). Signcryption achieves resource-efficiency by performing data signing and encryption in a single step. By running the signcryption on the sensing unit, we can relax some security assumptions for the camera host unit which typically runs a complex software stack. We introduce our system architecture motivated by a typical case study for camera-based IoT applications, evaluate security properties and present performance results of an ARM-based implementation.

Efficient real-time video stabilization for UAVs using only IMU data

Abstract: While some unmanned aerial vehicles (UAVs) have the capacity to carry mechanically stabilized camera equipment, weight limits or other problems may make mechanical stabilization impractical. As a result many UAVs rely on fixed cameras to provide a video stream to an operator or observer. With a fixed camera, the video stream is often unsteady due to the multirotor's movement from wind and acceleration. These video streams are often analyzed by both humans and machines, and the unwanted camera movement can cause problems for both. For a human observer, unwanted movement may simply make it harder to follow the video, while for computer algorithms, it may severely impair the algorithm's intended function. There has been significant research on how to stabilize videos using feature tracking to determine camera movement, which in turn is used to manipulate frames and stabilize the camera stream. We believe, however, that this process could be greatly simplified by using data from a UAV's on-board inertial measurement unit (IMU) to stabilize the camera feed. In this paper we present an algorithm for video stabilization based only on IMU data from a UAV platform. Our results show that our algorithm successfully stabilizes the camera stream with the added benefit of requiring less computational power.

Vision algorithm performance using low level sensor fusion

Abstract: A method and system (10) that performs low level fusion of radar (200) or LiDAR data with an image (402) from a camera (908). The system (10) includes a radar-sensor (904), a camera (908), and a controller (912). The radar-sensor (904) is used to detect a radar-signal (926) reflected by an object (902) in a radar-field-of-view (906). The camera (908) is used to capture an image (402) of the object (902) in a camera-field-of-view (910) that overlaps the radar (200)-field of view. The controller (912) is in communication with the radar-sensor (904) and the camera (908). The controller (912) is configured to determine a location of a radar-detection (400) in the image (402) indicated by the radar-signal (926), determine a parametric-curve (404) of the image (402) based on the radar (200) detections, define a region-of-interest (406) of the image (402) based on the parametric-curve (404) derived from the radar-detection (400), and process the region-of-interest (406) of the image (402) to determine an identity (914) of the object (902). The region-of-interest (406) may be a subset of the camera-field-of-view (910).

Investigation of Customer Behavior Analysis Based on Top-View Depth Camera

Abstract: Human behavior analysis based on surveillance camera is one of hot topics in security, marketing as well as computer vision and pattern recognition, and these are useful for commercial facilities such as convenience stores or book stores. In general, since surveillance camera is placed on the ceiling near store wall to monitor customer behaviors, the majority of this research utilize human model adapted to a front view of the person because the human shape has high discriminative power for human detection, pose estimation, etc. However, this approach has a problem that customers are often occluded by others in the store. To solve this occlusion problem for behavior analysis, we propose a new human behavior analysis method with top-view depth camera. In this research, for the first step to investigate the effectiveness of the analysis, we suppose the book store situation. Our proposed method is composed two behavior estimators. The first estimator is based on height of hand with depth information and the second is based on SVM with depth and PSA (Pixel State Analysis) based features. The characteristic point of our proposed method is that we utilize these estimators by cascading them. From experimental results with 10 behaviors of 3 subjects, although our research is still exploratory, we have confirmed that our proposed method allows us to obtain 89.5% of average estimation accuracy.

Self-learning Smart Cameras - Harnessing the Generalization Capability of XCS.

Abstract: In this paper, we show how an evolutionary rule-based machine learning technique can be applied to tackle the task of self-configuration of smart camera networks. More precisely, the Extended Classifier System (XCS) is utilized to learn a configuration strategy for the pan, tilt, and zoom of smart cameras. Thereby, we extend our previous approach, which is based on Q-Learning, by harnessing the generalization capability of Learning Classifier Systems (LCS), i.e. avoiding to separately approximate the quality of each possible (re-)configuration (action) in reaction to a certain situation (state). Instead, situations in which the same reconfiguration is adequate are grouped to one single rule. We demonstrate that our XCS-based approach outperforms the Q-learning method on the basis of empirical evaluations on scenarios of different severity.

Opportunistic Image Acquisition of Individual and Group Activities in a Distributed Camera Network

Abstract: The decreasing cost and size of video sensors has led to camera networks becoming pervasive in our lives. However, the ability to analyze these images effectively is very much a function of the quality of the acquired images. In this paper, we consider the problem of automatically controlling the fields of view of individual pan–tilt–zoom (PTZ) cameras in a camera network leading to improved situation awareness (e.g., where and what are the critical targets and events) in a region of interest. The network of cameras attempts to observe the entire region of interest at some minimum resolution while opportunistically acquiring high resolution images of critical events in real time. Since many activities involve groups of people interacting, an important decision that the network needs to make is whether to focus on individuals or groups of them. This is achieved by understanding the performance of video analysis tasks and designing camera control strategies to improve a metric that quantifies the quality of the source imagery. Optimization strategies, along with a distributed implementation, are proposed, and their theoretical properties analyzed. The proposed methods bring together computer vision and network control ideas. The performance of the proposed methodologies discussed herein has been evaluated on a real-life wireless network of PTZ capable cameras.