A visual attention system, inspired by the behavior and the neuronal architecture of the early primate visual system, is presented. Multiscale image features are combined into a single topographical saliency map. A dynamical neural network then selects attended locations in order of decreasing saliency. The system breaks down the complex problem of scene understanding by rapidly selecting, in a computationally efficient manner, conspicuous locations to be analyzed in detail.

/pdf/a-model-of-saliency-based-visual-attention-for-rapid-scene-wv0n3yqxix.pdf

A model of saliency-based visual attention for rapid scene analysis

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Data Mining Practical Machine Learning Tools and Techniques

This paper presents a method for recognizing scene categories based on approximate global geometric correspondence. This technique works by partitioning the image into increasingly fine sub-regions and computing histograms of local features found inside each sub-region. The resulting "spatial pyramid" is a simple and computationally efficient extension of an orderless bag-of-features image representation, and it shows significantly improved performance on challenging scene categorization tasks. Specifically, our proposed method exceeds the state of the art on the Caltech-101 database and achieves high accuracy on a large database of fifteen natural scene categories. The spatial pyramid framework also offers insights into the success of several recently proposed image descriptions, including Torralbas "gist" and Lowes SIFT descriptors.

/pdf/beyond-bags-of-features-spatial-pyramid-matching-for-4aa5cbe2fz.pdf

Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

An approach for implementing affective information as tags for multimedia content indexing and retrieval is presented. The approach can be used for implicit as well as explicit tags and is presented here using data recorded during the viewing of movie fragments containing annotations and physiological signal recordings. For retrieval based on affective queries, a representation of the query-words is defined in the arousalvalence space in the form of a Gaussian probability distribution and a retrieval method based on this representation is presented. Validation of retrieval accuracy is performed using Precision and Recall parameters. Results show that the use of arousal and valence as affective tags can improve retrieval results.

/pdf/queries-and-tags-in-affect-based-multimedia-retrieval-1bps3h4lkp.pdf

Queries and tags in affect-based multimedia retrieval

Recognizing emotional reactions of movie audiences to affective movie content is a challenging task in affective computing. Previous research on induced emotion recognition has mainly focused on using audio-visual movie content. Nevertheless, the relationship between the perceptions of the affective movie content (perceived emotions) and the emotions evoked in the audiences (induced emotions) is unexplored. In this work, we studied the relationship between perceived and induced emotions of movie audiences. Moreover, we investigated multimodal modelling approaches to predict movie induced emotions from movie content based features, as well as physiological and behavioral reactions of movie audiences. To carry out analysis of induced and perceived emotions, we first extended an existing database for movie affect analysis by annotating perceived emotions in a crowd-sourced manner. We find that perceived and induced emotions are not always consistent with each other. In addition, we show that perceived emotions, movie dialogues, and aesthetic highlights are discriminative for movie induced emotion recognition besides spectators’ physiological and behavioral reactions. Also, our experiments revealed that induced emotion recognition could benefit from including temporal information and performing multimodal fusion. Moreover, our work deeply investigated the gap between affective content analysis and induced emotion recognition by gaining insight into the relationships between aesthetic highlights, induced emotions, and perceived emotions.

/pdf/recognizing-induced-emotions-of-movie-audiences-from-33keexbejt.pdf

Recognizing Induced Emotions of Movie Audiences from Multimodal Information

This work advocates the formulation of digital watermarking as a communication problem. We consider watermarking as communication with side information available for both encoder and decoder. A generalized watermarking channel is considered that includes geometrical attacks, fading and additive non-Gaussian noise. The optimal encoding/decoding scenario is discussed for the generalized watermarking channel.

/pdf/optimal-adaptive-diversity-watermarking-with-channel-state-3j9qpn0oua.pdf

Optimal adaptive diversity watermarking with channel state estimation

In this paper, we propose a new theoretical framework for the data-hiding problem of digital and printed text documents. We explain how this problem can be seen as an instance of the well-known Gel'fand-Pinsker problem. The main idea for this interpretation is to consider a text character as a data structure consisting of multiple quantifiable features such as shape, position, orientation, size, color, etc. We also introduce color quantization, a new semi-fragile text data-hiding method that is fully automatable, has high information embedding rate, and can be applied to both digital and printed text documents. The main idea of this method is to quantize the color or luminance intensity of each character in such a manner that the human visual system is not able to distinguish between the original and quantized characters, but it can be easily performed by a specialized reader machine. We also describe halftone quantization, a related method that applies mainly to printed text documents. Since these methods may not be completely robust to printing and scanning, an outer coding layer is proposed to solve this issue. Finally, we describe a practical implementation of the color quantization method and present experimental results for comparison with other existing methods.

/pdf/text-data-hiding-for-digital-and-printed-documents-3m1r6fpm5f.pdf

Text data-hiding for digital and printed documents: theoretical and practical considerations

In the Internet world, the widespread use of graphical user interfaces (GUIs) increasingly bars visually handicapped people from accessing digital information. In this context our project aims at providing sight handicapped people with alternative access modalities to various types of GUIs and graphic intensive programs, in order for instance to facilitate usage of Web services. We describe in this paper AB-Web, a 3D-audio Web browser that allows blind computer users to explore Web pages, fill in forms, etc., using a 3D sonic rendering. We also present WebSound, a generic tool that permits to associate with each HTML tag a given sonic object (earcon or auditory icon). Finally, we describe a series of associated programs composed of the family of sonic games From Dots to Shapes, as well as of IDEA, a tool that lets users comprehend simple drawings, as well as to create graphics.

/pdf/audio-haptic-internet-browser-and-associated-tools-for-blind-1715dx73a1.pdf

Thierry Pun

Papers

Queries and tags in affect-based multimedia retrieval

Recognizing Induced Emotions of Movie Audiences from Multimodal Information

Optimal adaptive diversity watermarking with channel state estimation

Text data-hiding for digital and printed documents: theoretical and practical considerations

Audio-Haptic Internet Browser and Associated Tools for Blind Users and Visually Impaired Computer Users