The PASCAL Visual Object Classes Challenge

In the real world, a realistic setting for computer vision or multimedia recognition problems is that we have some classes containing lots of training data and many classes contain a small amount of training data. Therefore, how to use frequent classes to help learning rare classes for which it is harder to collect the training data is an open question. Learning with Shared Information is an emerging topic in machine learning, computer vision and multimedia analysis. There are different level of components that can be shared during concept modeling and machine learning stages, such as sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, etc. Regarding the specific methods, multi-task learning, transfer learning and deep learning can be seen as using different strategies to share information. These learning with shared information methods are very effective in solving real-world large-scale problems. This special issue aims at gathering the recent advances in learning with shared information methods and their applications in computer vision and multimedia analysis. Both state-of-the-art works, as well as literature reviews, are welcome for submission. Papers addressing interesting real-world computer vision and multimedia applications are especially encouraged. Topics of interest include, but are not limited to:  • Multi-task learning or transfer learning for large-scale computer vision and multimedia analysis • Deep learning for large-scale computer vision and multimedia analysis • Multi-modal approach for large-scale computer vision and multimedia analysis • Different sharing strategies, e.g., sharing generic object parts, sharing attributes, sharing transformations, sharing regularization parameters and sharing training examples, • Real-world computer vision and multimedia applications based on learning with shared information, e.g., event detection, object recognition, object detection, action recognition, human head pose estimation, object tracking, location-based services, semantic indexing. • New datasets and metrics to evaluate the benefit of the proposed sharing ability for the specific computer vision or multimedia problem. • Survey papers regarding the topic of learning with shared information.  Authors who are unsure whether their planned submission is in scope may contact the guest editors prior to the submission deadline with an abstract, in order to receive feedback.

IEEE transactions on pattern analysis and machine intelligence

Given audio of President Barack Obama, we synthesize a high quality video of him speaking with accurate lip sync, composited into a target video clip. Trained on many hours of his weekly address footage, a recurrent neural network learns the mapping from raw audio features to mouth shapes. Given the mouth shape at each time instant, we synthesize high quality mouth texture, and composite it with proper 3D pose matching to change what he appears to be saying in a target video to match the input audio track. Our approach produces photorealistic results.

Synthesizing Obama: learning lip sync from audio

This paper describes our concept of Intelligent Space. Intelligent Spaces are rooms or areas that are equipped with sensors, which enable the spaces to perceive and understand what is happening in them. With such features, people or systems in the Intelligent Space can use additional functions that are afforded by the space. This research field itself is not particularly related to robotics, but we believe robots under Intelligent Spaces have many interesting features. Intelligent Spaces are expected to have a broad range of applications such as in homes, offices, factories, etc. Thus, we can consider that a robot, which will be used in our general living environment, is related to the concept of Intelligent Space. Several applications of Intelligent Space are shown in this paper with descriptions.

Intelligent Space — concept and contents

In recent years, considerable amount of research has been dedicated to the integration of artificial cognitive functionalities into informatics. With the immense growth in volume of cognitive content handled by both artificial and natural cognitive systems, the scientific treatment of new and efficient communication forms between such cognitive systems is inevitable. In this paper, we provide the first definition of cognitive infocommunications, a multidisciplinary field which aims to expand the information space between communicating cognitive systems (artificial or otherwise). Following this definition, we specify the modes and types of communication which make up cognitive infocommunications. Through a number of examples, we describe what is expected from this new discipline in further detail.

Cognitive Infocommunications (Coginfocom)

The robots that will be needed in the near future are human-friendly robots that are able to coexist with humans and support humans effectively. To realize this, humans and robots need to be in close proximity to each other as much as possible. Moreover, it is necessary for their interactions to occur naturally. It is desirable for a robot to carry out human following, as one of the human-affinitive movements. The human-following robot requires several techniques: the recognition of the target human, the recognition of the environment around the robot, and the control strategy for following a human stably. In this research, an intelligent environment is used in order to achieve these goals. An intelligent environment is a space in which many sensors and intelligent devices are distributed. Mobile robots exist in this space as physical agents providing humans with services. A mobile robot is controlled to follow a walking human using distributed intelligent sensors as stably and precisely as possible. The control law based on the virtual spring model is proposed to mitigate the difference of movement between the human and the mobile robot. The proposed control law is applied to the intelligent environment and its performance is verified by the computer simulation and the experiment.

Human-following mobile robot in a distributed intelligent sensor network

Mobile robots need sufficient sensors and information on the environment in order to navigate. In this paper, we propose a system of mobile robots, which is controlled in a distributed intelligent sensor network. In such a networked space, the environment is divided by distributed sensors. Each area is monitored by a distributed sensor device, which connects with other distributed sensor devices and robots throughout the network. As a result, the mobile robots are able to accomplish tasks simply by following orders from the sensor devices in the networked environment, although the mobile robots are not self-contained with information on the environment and sensors for self-positioning and control. We test several situations to verify the proposed system.

Controlling mobile robots in distributed intelligent sensor network

Intelligent spaces are rooms or areas that are equipped with sensors such as microphones or cameras that enable them to perceive what is happening in them. In such spaces that have an intelligence of their own a world model no longer is something the robot has alone but a service offered by the information infrastructure of the space. In this article we show how such an intelligent space can generate a topological map for robots by looking at the movements of people in the room. We describe the stereo vision system that is capable of tracking the 3D movements of several humans in real time and give experimental results obtained in a real-world environment with several people.

Building topological maps by looking at people: an example of cooperation between intelligent spaces and robots

We propose an architecture of intelligent space based on distributed intelligent sensors. Intelligent space is an environmental system able to support humans in informative and physical ways. Since an intelligent space should adapt to the various sizes and shapes of an environment, an architecture based on distributed intelligent sensors is designed. The proposed architecture satisfies not only scalability but also reconfigurability, modularity, easy maintenance, and affinity problems in building an intelligent space. Intelligent sensors are distributed among a space and they provide functions based on position information. According to the particular situation, cooperation among intelligent sensors or cooperation among function modules in the intelligent sensors are performed. Selected demonstrations are described in the paper.

Joo-Ho Lee

Papers

Intelligent Space — concept and contents

Human-following mobile robot in a distributed intelligent sensor network

Controlling mobile robots in distributed intelligent sensor network

Building topological maps by looking at people: an example of cooperation between intelligent spaces and robots

Cooperation of distributed intelligent sensors in intelligent environment