In the real world, a realistic setting for computer vision or multimedia recognition problems is that we have some classes containing lots of training data and many classes contain a small amount of training data. Therefore, how to use frequent classes to help learning rare classes for which it is harder to collect the training data is an open question. Learning with Shared Information is an emerging topic in machine learning, computer vision and multimedia analysis. There are different level of components that can be shared during concept modeling and machine learning stages, such as sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, etc. Regarding the specific methods, multi-task learning, transfer learning and deep learning can be seen as using different strategies to share information. These learning with shared information methods are very effective in solving real-world large-scale problems. This special issue aims at gathering the recent advances in learning with shared information methods and their applications in computer vision and multimedia analysis. Both state-of-the-art works, as well as literature reviews, are welcome for submission. Papers addressing interesting real-world computer vision and multimedia applications are especially encouraged. Topics of interest include, but are not limited to:  • Multi-task learning or transfer learning for large-scale computer vision and multimedia analysis • Deep learning for large-scale computer vision and multimedia analysis • Multi-modal approach for large-scale computer vision and multimedia analysis • Different sharing strategies, e.g., sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, • Real-world computer vision and multimedia applications based on learning with shared information, e.g., event detection, object recognition, object detection, action recognition, human head pose estimation, object tracking, location-based services, semantic indexing. • New datasets and metrics to evaluate the benefit of the proposed sharing ability for the specific computer vision or multimedia problem. • Survey papers regarding the topic of learning with shared information.  Authors who are unsure whether their planned submission is in scope may contact the guest editors prior to the submission deadline with an abstract, in order to receive feedback.

IEEE transactions on pattern analysis and machine intelligence

A review of vision-based systems for soccer video analysis

The method for generating a 3D representation of a dynamically changing 3D scene, comprises the steps of acquiring at least two synchronised video streams (120) from at least two cameras located at different locations and observing the same 3D scene (102); determining camera parameters, which comprise the orientation and zoom setting, for said at least two cameras (103); tracking the movement of objects (310a,b, 312a,b; 330a,b, 331a,b, 332a,b; 410a,b, 411a,b; 430a,b, 431a,b; 420a,b, 421a,b) in the at least two video streams (104); determining the identity of said objects in the at least two video streams (105); determining the 3D position of the objects by combining the information from the at least two video streams (106); wherein the step of tracking (104) the movement of objects in the at least two video streams uses position information derived from the 3D position of the objects in one or more earlier instants in time. As a result, the quality, speed and robustness of the 2D tracking in the video streams is improved.

Method and system for generating a 3D representation of a dynamically changing 3D scene

An autonomous computer based method and system is described for personalized production of videos such as team sport videos such as basketball videos from multi- sensored data under limited display resolution. Embodiments of the present invention relate to the selection of a view to display from among the multiple video streams captured by the camera network. Technical solutions are provided to provide perceptual comfort as well as an efficient integration of contextual information, which is implemented, for example, by smoothing generated viewpoint/camera sequences to alleviate flickering visual artefacts and discontinuous story-telling artefacts. A design and implementation of the viewpoint selection process is disclosed that has been verified by experiments, which shows that the method and system of the present invention efficiently distribute the processing load across cameras, and effectively selects viewpoints that cover the team action at hand while avoiding major perceptual artefacts.

Systems and methods for the autonomous production of videos from multi-sensored data

Visual mixed and augmented realities have historically been applied to the gaming application domain. This article provides a survey of visual mixed and augmented reality gaming in both the academic and commercial contexts. There is an exploration of both indoor and outdoor mixed and augmented reality gaming. The different games are presented via the three major display technologies: head-mounted display, handheld display, and spatial immersive display. A number of academic mixed and augmented reality research projects are described that provide an overview of the current state of the art. A set of example commercial games are also examined to provide the context for the state of the games on the market.

A survey of visual, mixed, and augmented reality gaming

This paper presents a novel method for virtual view synthesis that allows viewers to virtually fly through real soccer scenes, which are captured by multiple cameras in a stadium. The proposed method generates images of arbitrary viewpoints by view interpolation of real camera images near the chosen viewpoints. In this method, cameras do not need to be strongly calibrated since projective geometry between cameras is employed for the interpolation. For avoiding the complex and unreliable process of 3-D recovery, object scenes are segmented into several regions according to the geometric property of the scene. Dense correspondence between real views, which is necessary for intermediate view generation, is automatically obtained by applying projective geometry to each region. By superimposing intermediate images for all regions, virtual views for the entire soccer scene are generated. The efforts for camera calibration are reduced and correspondence matching requires no manual operation; hence, the proposed method can be easily applied to dynamic events in a large space. An application for fly-through observations of soccer match replays is introduced along with the algorithm of view synthesis and experimental results. This is a new approach for providing arbitrary views of an entire dynamic event.

/pdf/virtual-viewpoint-replay-for-a-soccer-match-by-view-2bqskwphya.pdf

Virtual Viewpoint Replay for a Soccer Match by View Interpolation From Multiple Cameras

This paper presents a novel observation system for immersive soccer match taken by multiple video cameras at a real stadium. The user sees the soccer field model in front of his/her eyes from the viewpoint through head-mounted display, while the images of players and a soccer ball are also rendered onto the display. For geometric registration between the soccer field model in the real world and the dynamic soccer scene in the rendered images, the viewpoint position of the user is computed by using only natural feature lines in the HMD camera image. Since it is difficult to strongly calibrate the HMD camera and the multiple cameras that capture the real soccer scene, we employ projective geometry for the registration and the rendering. For demonstrating the efficacy of the proposed system, video images of soccer matches taken at real stadium are rendered onto the HMD camera images of a tabletop stadium model. This is a completely new challenge to apply augmented reality to a dynamic event in a large-space.

Immersive Observation of Virtualized Soccer Match at Real Stadium Model

This paper introduces a novel method for generating an intermediate view of soccer scenes taken by multiple video cameras. In the proposed method, a soccer scene is classified into dynamic regions, a field region, and a background region. Using an epipolar geometry in the first region and homography in the second, dense correspondence is obtained to interpolate views. For the third region, partial area images are extracted from the panoramic image compounded from the background of multiple views. Finally synthesizing them completes intermediate view images of the whole object. Applying this method to actual scenes of a soccer match captured at the stadium, we succeeded in generating natural intermediate view videos.

Intermediate view generation of soccer scene from multiple videos

Intermediate View Generation of Soccer Scene from Multiple Videos

We introduce methods for sports scene analysis and visualization from multiple videos captured with multiple cameras. As the scene analysis, we present a method for tracking multiple soccer players. Tracking is done by integrating the tracking data from all cameras, using the geometrical relationship between cameras called homography. Integrating information from all cameras enables stable tracking on the scene, where the tracking by a single camera often fails in the case of occlusion. We also present a method for free-viewpoint visualization of a soccer game as a technique for visualizing the soccer scene. The free-viewpoint image is synthesized by view interpolation between actual cameras near the virtual viewpoint at each frame. Such free-viewpoint video can also be presented in a system of augmented reality (AR) for immersive visualization of soccer matches

http://hvrl.ics.keio.ac.jp/paper/pdf/international_Conference/2004/ICME2004.pdf

Naho Inamoto

Papers

Virtual Viewpoint Replay for a Soccer Match by View Interpolation From Multiple Cameras

Immersive Observation of Virtualized Soccer Match at Real Stadium Model

Intermediate view generation of soccer scene from multiple videos

Intermediate View Generation of Soccer Scene from Multiple Videos

Sports scene analysis and visualization from multiple-view video