The aim of this paper is to address recognition of natural human actions in diverse and realistic video settings. This challenging but important subject has mostly been ignored in the past due to several problems one of which is the lack of realistic and annotated video datasets. Our first contribution is to address this limitation and to investigate the use of movie scripts for automatic annotation of human actions in videos. We evaluate alternative methods for action retrieval from scripts and show benefits of a text-based classifier. Using the retrieved action samples for visual learning, we next turn to the problem of action classification in video. We present a new method for video classification that builds upon and extends several recent ideas including local space-time features, space-time pyramids and multi-channel non-linear SVMs. The method is shown to improve state-of-the-art results on the standard KTH action dataset by achieving 91.8% accuracy. Given the inherent problem of noisy labels in automatic annotation, we particularly investigate and show high tolerance of our method to annotation errors in the training set. We finally apply the method to learning and classifying challenging action classes in movies and show promising results.

/pdf/learning-realistic-human-actions-from-movies-3draj9xusg.pdf

Learning realistic human actions from movies

VLFeat is an open and portable library of computer vision algorithms. It aims at facilitating fast prototyping and reproducible research for computer vision scientists and students. It includes rigorous implementations of common building blocks such as feature detectors, feature extractors, (hierarchical) k-means clustering, randomized kd-tree matching, and super-pixelization. The source code and interfaces are fully documented. The library integrates directly with MATLAB, a popular language for computer vision research.

/pdf/vlfeat-an-open-and-portable-library-of-computer-vision-5bicdsdo9n.pdf

Vlfeat: an open and portable library of computer vision algorithms

The traditional SPM approach based on bag-of-features (BoF) requires nonlinear classifiers to achieve good image classification performance. This paper presents a simple but effective coding scheme called Locality-constrained Linear Coding (LLC) in place of the VQ coding in traditional SPM. LLC utilizes the locality constraints to project each descriptor into its local-coordinate system, and the projected coordinates are integrated by max pooling to generate the final representation. With linear classifier, the proposed approach performs remarkably better than the traditional nonlinear SPM, achieving state-of-the-art performance on several benchmarks. Compared with the sparse coding strategy [22], the objective function used by LLC has an analytical solution. In addition, the paper proposes a fast approximated LLC method by first performing a K-nearest-neighbor search and then solving a constrained least square fitting problem, bearing computational complexity of O(M + K2). Hence even with very large codebooks, our system can still process multiple frames per second. This efficiency significantly adds to the practical values of LLC for real applications.

http://people.ee.duke.edu/~lcarin/cvpr2010_0618.pdf

Locality-constrained Linear Coding for image classification

Recently SVMs using spatial pyramid matching (SPM) kernel have been highly successful in image classification. Despite its popularity, these nonlinear SVMs have a complexity O(n2 ~ n3) in training and O(n) in testing, where n is the training size, implying that it is nontrivial to scaleup the algorithms to handle more than thousands of training images. In this paper we develop an extension of the SPM method, by generalizing vector quantization to sparse coding followed by multi-scale spatial max pooling, and propose a linear SPM kernel based on SIFT sparse codes. This new approach remarkably reduces the complexity of SVMs to O(n) in training and a constant in testing. In a number of image categorization experiments, we find that, in terms of classification accuracy, the suggested linear SPM based on sparse coding of SIFT descriptors always significantly outperforms the linear SPM kernel on histograms, and is even better than the nonlinear SPM kernels, leading to state-of-the-art performance on several benchmarks by using a single type of descriptors.

/pdf/linear-spatial-pyramid-matching-using-sparse-coding-for-4sjbmgk16d.pdf

Linear spatial pyramid matching using sparse coding for image classification

Domain adaptation is an important emerging topic in computer vision. In this paper, we present one of the first studies of domain shift in the context of object recognition. We introduce a method that adapts object models acquired in a particular visual domain to new imaging conditions by learning a transformation that minimizes the effect of domain-induced changes in the feature distribution. The transformation is learned in a supervised manner and can be applied to categories for which there are no labeled examples in the new domain. While we focus our evaluation on object recognition tasks, the transform-based adaptation technique we develop is general and could be applied to nonimage data. Another contribution is a new multi-domain object database, freely available for download. We experimentally demonstrate the ability of our method to improve recognition on categories with few or no target domain labels and moderate to large changes in the imaging conditions.

/pdf/adapting-visual-category-models-to-new-domains-2a256r4qsc.pdf

Adapting visual category models to new domains

This work reports the experience gathered along 6 years in developing a team-based project activity to promote engineering programs among secondary school students. The aim of the activity is to increase the interest of students for science and technology in general, but also to promote engineering skills, capabilities and values, leading to attract more secondary school students to enrolment for engineering programs. Simple theoretical concepts related to physics and engineering are illustrated through hands-on experimentation. To achieve this goal, the students build a remotely operated underwater robot in a workshop. The robot is built using low-cost materials and the students customize their own design over the different phases of the workshop. Once the activity is completed, every team understands that with teamwork, effort and a good working strategy, every problem can be overcome. This paper explains in detail the structure and operation of the workshop and introduces a first initiative towards the internationalization of the project, which was carried out in two schools for underprivileged children in South India. Through this initiative in India, the workshop takes a step forward, and in addition to promoting science and technology, it also adds a new dimension of education for global citizenship to the project.

Team-based workshop to engage young students in engineering and science: Building and driving a ROV (R2B2). Two special editions in India

In this paper an object learning system for image understanding is proposed. The knowledge acquisition system is designed as a supervised learning task, which emphasises the role of the user as teacher of the system and allows to obtain the object description as well as to select the best recognition strategy for each specific object. From several representative examples in training images, an object description is acquired by considering different model representations. Moreover, different recognition strategies are built and applied to obtain initial results. Next, teacher evaluates these results and the system automatically selects the specific strategy which best recognise each object. Experimental results are shown and discussed.

/pdf/a-learning-framework-for-object-recognition-on-image-47n37w12et.pdf

A learning framework for object recognition on image understanding

ResumenLa perspectiva ecologica sobre el desarrollo humano (Bronfenbrenner, 1979) pone el acento en la relacion entre los diferentes contextos de vida de los ninos. En la educacion infantil, los contextos de vida mas importantes son la familia y la escuela. En este articulo mostramos los aspectos de continuidad y discontinuidad entre ambos contextos en relacion a las ideas de las familias y las educadoras sobre el desarrollo infantil y tambien a sus creencias y expectativas sobre la educacion. Los resultados muestran que existe un acuerdo notable en muchos aspectos entre las familias y las educadoras, si bien tambien existen aspectos de desacuerdo principalment e entre las educadoras de parvulario y las familias.

/pdf/expectativas-evolutivas-ideas-y-creencias-sobre-la-educacion-3e6f42of03.pdf

Expectativas evolutivas, ideas y creencias sobre la educación infantil de familias y maestros: ¿continuidad o discontinuidad?

We propose an hybrid and probabilistic classification of image regions belonging to scenes primarily containing natural objects, e.g. sky, trees, etc. as a first step in solving the problem of scene context generation. Therefore, we will focus our work in the problem of image regions labeling to classify every pixel of a given image into one of several predefined classes. Our proposal begins with a top-down control to find the core of objects, which allow us to update the learned models. Moreover, they become the starting seeds for the growing of a set of concurrent active regions which, considering the own region model as well as region and boundary information, obtain an accurate recognition of known regions. Next, a general segmentation extracts the unknown regions by a bottom-up strategy. Finally, a last stage exploits the contextual information to classify initially unknown segmented objects. The result is both a segmentation of the image and a recognition of each segment as a given object class or as an unknown segmented object. Experimental results on a wide set of outdoor scene images are shown to evaluate and compare our proposal.

/pdf/classifying-natural-objects-on-outdoor-scenes-lm54usmt0e.pdf

Classifying Natural Objects on Outdoor Scenes

An object learning system for image understanding is proposed in this paper. The knowledge acquisition system is designed as a supervised learning task. Therefore, the role of the user as teacher of the system is emphasized, which allows to obtain the object description as well as to select the best recognition strategy for each specific object. An object description is acquired by considering different model representations provided from several representative examples in training images. Moreover, different recognition strategies are built and applied to obtain initial results. Next, teacher evaluates these results and the system automatically selects the specific strategy which best recognise each object. We provide a feedback with the user until he/she is satisfied with the knowledge achieved by the system. Experimental results are shown and discussed.

/pdf/supervised-object-knowledge-learning-for-image-understanding-28r2l3k9im.pdf

Anna Bosch

Papers

Team-based workshop to engage young students in engineering and science: Building and driving a ROV (R2B2). Two special editions in India

A learning framework for object recognition on image understanding

Expectativas evolutivas, ideas y creencias sobre la educación infantil de familias y maestros: ¿continuidad o discontinuidad?

Classifying Natural Objects on Outdoor Scenes

Supervised Object Knowledge Learning for Image Understanding