Architecture for implementing a perceptual user interface. The architecture comprises alternative modalities for controlling computer application programs and manipulating on-screen objects through hand gestures or a combination of hand gestures and verbal commands. The perceptual user interface system includes a tracking component that detects object characteristics of at least one of a plurality of objects within a scene, and tracks the respective object. Detection of object characteristics is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images. A seeding component iteratively seeds the tracking component with object hypotheses based upon the presence of the object characteristics and the image comparison. A filtering component selectively removes the tracked object from the object hypotheses and/or at least one object hypothesis from the set of object hypotheses based upon predetermined removal criteria.

Architecture for controlling a computer using hand gestures

Map-matching is the process of aligning a sequence of observed user positions with the road network on a digital map. It is a fundamental pre-processing step for many applications, such as moving object management, traffic flow analysis, and driving directions. In practice there exists huge amount of low-sampling-rate (e.g., one point every 2--5 minutes) GPS trajectories. Unfortunately, most current map-matching approaches only deal with high-sampling-rate (typically one point every 10--30s) GPS data, and become less effective for low-sampling-rate points as the uncertainty in data increases. In this paper, we propose a novel global map-matching algorithm called ST-Matching for low-sampling-rate GPS trajectories. ST-Matching considers (1) the spatial geometric and topological structures of the road network and (2) the temporal/speed constraints of the trajectories. Based on spatio-temporal analysis, a candidate graph is constructed from which the best matching path sequence is identified. We compare ST-Matching with the incremental algorithm and Average-Frechet-Distance (AFD) based global map-matching algorithm. The experiments are performed both on synthetic and real dataset. The results show that our ST-matching algorithm significantly outperform incremental algorithm in terms of matching accuracy for low-sampling trajectories. Meanwhile, when compared with AFD-based global algorithm, ST-Matching also improves accuracy as well as running time.

/pdf/map-matching-for-low-sampling-rate-gps-trajectories-4hstm6zk4l.pdf

Map-matching for low-sampling-rate GPS trajectories

Advances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring

An automatics system 100 that uses one to three grids 20 cm of overhead cameras 20 c to first video an event area 2. Overall bandwidth is greatly reduced by intelligent hubs 26 that extract foreground blocks 10 m based upon initial and continuously updated background images 2 r. The hubs also analyze current images 10 c to constantly locate, classify and track in 3D the limited number of expected foreground objects 10. As objects 10 of interest are tracked, the system automatically directs ptz perspective view cameras 40 c to follow the activities. These asynchronous cameras 40 c limit their images to defined repeatable pt angles and zoom depths. Pre-captured venue backgrounds 2 r at each repeatable ptz setting facilitate perspective foreground extraction. The moving background, such as spectators 13, is removed with various techniques including stereoscopic side cameras 40 c-b and 40 c-c flanking each perspective camera 40 c. The tracking data 101 derived from the overhead view 102 establishes event performance measurement and analysis data 701. The analysis results in statistics and descriptive performance tokens 702 translatable via speech synthesis into audible descriptions of the event activities corresponding to overhead 102 and perspective video 202.

Automatic event videoing, tracking and content generation

Image analysis techniques, including object recognition analysis, are applied to images obtained by one or more image capture devices deployed within inventory environments The object recognition analysis provides object recognition data (that may include one or more recognized product instances) based on stored product (training) images In turn, a variety of functionalities may be enabled based on the object recognition data For example, a planogram may be extracted and compared to a target planogram, or at least one product display parameter for a product can be determined and used to assess presence of the product within the inventory environment, or to determine compliance of display of the product with a promotional objective In yet another embodiment, comparisons may be made within a single image or between multiple images over time to detect potential conditions requiring response In this manner, efficiency and effectiveness of many previously manually-implemented tasks may be improved

Planogram extraction based on image processing

Presents a real-time computer vision system that tracks the motion of a tennis ball in 3D using multiple cameras. Ball tracking enables virtual replays, new game statistics, and other visualizations which result in very new ways of experiencing and analyzing tennis matches. The system has been used in international television broadcasts and webcasts of more than 15 matches. Six cameras around a stadium, divided into four pairs, are currently used to track the ball on serves which sometimes exceed speeds of 225 kmph. A multi-threaded approach is taken to tracking where each thread tracks the ball in a pair of cameras based on motion, intensity and shape, performs stereo matching to obtain the 3D trajectory, detects when a ball goes out of view of its camera pair, and initializes and triggers a subsequent thread. This efficient approach is scalable to many more cameras tracking multiple objects. The ready acceptance of the system indicates the growing potential for multi-camera based real-time tracking in broadcast applications.

Ball tracking and virtual replays for innovative tennis broadcasts

Techniques for indexing multimedia data substantially concurrently or contemporaneously with its capture to convert a real world event into an accessible database in real time are provided. The present invention introduces a new paradigm of converting a real world event in real time into a rich multimedia database by processing data from multiple sensors observing the event. Real time analysis of the sensor data, coupled with domain knowledge, results in instant indexing of multimedia data at capture time itself. This yields the semantic information to answer complex queries about the content, and the ability to extract portions of data that correspond to complex actions performed in the real world.

Instantly indexed databases for multimedia content analysis and retrieval

We introduce a new paradigm for real-time conversion of a real world event into a rich multimedia database by processing data from multiple sensors observing the event. A real-time analysis of the sensor data, tightly coupled with domain knowledge, results in instant indexing of multimedia data at capture time. This yields semantic information to answer complex queries about the content and the ability to extract portions of data that correspond to complex actions performed in the real world. The power of such an instantly indexed multimedia database system, in content-based retrieval of multimedia data or in semantic analysis and visualization of the data, far exceeds that of systems which index multimedia data only after it is produced. We present LucentVision, an instantly indexed multimedia database system developed for the sport of tennis. This system analyzes video from multiple cameras in real time and captures the activity of the players and the ball in the form of motion trajectories. The system stores these trajectories in a database along with video, 3D models of the environment, scores, and other domain-specific information. LucentVision has been used to enhance live television and Internet broadcasts with game analyses and virtual replays in more than 250 international tennis matches.

https://www.lehman.edu/faculty/yves/Publications_files/01017739.pdf

Instantly indexed multimedia databases of real world events

Remote experience of sporting events has thus far been limited mostly to watching video and the scores and statistics associated with the sport. However, a fast-developing trend is the use of visualization techniques to give new insights into performance, style, and strategy of the players. Automated techniques can extract accurate information from video about player performance that not even the most skilled observer is able to discern. When presented as static images or as a three-dimensional virtual replay, this information makes viewing a game an entirely new and exciting experience. This paper presents one such sports visualization system called LucentVision, which has been developed for the sport of tennis. LucentVision uses real-time video analysis to obtain motion trajectories of players and the ball, and offers a rich set of visualization options based on this trajectory data. The system has been used extensively in the broadcast of international tennis tournaments, both on television and the Internet.

Visualization of sports using motion trajectories: providing insights into performance, style, and strategy

Techniques for intelligent and automatic control of sensors for capturing data associated with real time events. Preferably, the sensors are associated with a multimedia database system. For example, a technique for controlling one or more sensors used to capture data associated with an event comprises the following steps/operations. First, sensor data captured in accordance with the event is processed. Then, the one or more sensors are automatically controlled based on information pertaining to the continual activity of at least one of one or more objects and one or more persons associated with the event in real time obtained using at least a portion of the processed data captured in accordance with the event.

Agata Opalach

Papers

Ball tracking and virtual replays for innovative tennis broadcasts

Instantly indexed databases for multimedia content analysis and retrieval

Instantly indexed multimedia databases of real world events

Visualization of sports using motion trajectories: providing insights into performance, style, and strategy

Method and apparatus for intelligent and automatic sensor control using multimedia database system