The literature reports research efforts allowing the editing of interactive TV multimedia documents by end-users. In this article we propose complementary contributions relative to end-user generated interactive video, video tagging, and collaboration. In earlier work we proposed the watch-and-comment (WaC) paradigm as the seamless capture of an individual's comments so that corresponding annotated interactive videos be automatically generated. As a proof of concept, we implemented a prototype application, the WaCTool, that supports the capture of digital ink and voice comments over individual frames and segments of the video, producing a declarative document that specifies both: different media stream structure and synchronization.In this article, we extend the WaC paradigm in two ways. First, user-video interactions are associated with edit commands and digital ink operations. Second, focusing on collaboration and distribution issues, we employ annotations as simple containers for context information by using them as tags in order to organize, store and distribute information in a P2P-based multimedia capture platform. We highlight the design principles of the watch-and-comment paradigm, and demonstrate related results including the current version of the WaCTool and its architecture. We also illustrate how an interactive video produced by the WaCTool can be rendered in an interactive video environment, the Ginga-NCL player, and include results from a preliminary evaluation.

Watch-and-comment as a paradigm toward ubiquitous interactive video editing

Given the very ambiguous and imprecise nature of sentiments and of their expressions, this survey focuses on approaches making use of components of graduality in the task of automatic sentiments analysis. To that aim, we review methods taking account of intrinsic psychological models components of graduality as well as extrinsic components issued from computational intelligence approaches. In particular, beyond psychological models of sentiments that define affective states as multidimensional vectors in affective continuous spaces, we identify three components of graduality, namely composition or blending, intensity and inheritance. In our discussion, we review how fuzzy set theory as well as other gradual structures based on a vectorial representation are employed to describe affective states as complex or imprecise entities. Finally, we focus on verbal expressions of sentiments and more specifically, we discuss the use of components of graduality in order to deal with sentiments complex and subtle expressions issued from the expressive power of natural languages.

Expressions of graduality for sentiments analysis — A survey

LSA and Information Retrieval

Considering that, when users watch a video with someone else, they are used to make comments regarding its contents --- such as a comment with respect to someone appearing in the video --- in previous work we exploited ubiquitous computing concepts to propose the watching-and-commentingauthoring paradigm in which a user's comments are automatically captured so as to automatically generate a corresponding annotated interactive video. In this paper we revisit and extend our previous work and detail our prototype that supports the watching-and-editingparadigm, discussing how a ubiquitous computing platform may explore digital ink and associated gestures to support the authoring of multimedia content while enhancing the social aspects of video watching.

Ubiquitous Interactive Video Editing Via Multimodal Annotations

Supporting most aspects of a media provider's real workflows such as production, distribution, content description, archiving, and re-use of video items, we developed a holistic framework to solve issues such as lack of human resources, necessity of parallel media distribution, and retrieving previously archived content through editors or consumers.

Produce. annotate. archive. repurpose --: accelerating the composition and metadata accumulation of tv content

The large amounts of TV, radio, games, music tracks or other IP based content becoming available in DVB-H mobile digital broadcast, offering more than 50 channels when adapted to the screen size of a handheld device, requires that the selection of media can be personalized according to user preferences. This paper presents an approach to model user preferences that could be used as a fundament for filtering content listed in the ESG electronic service guide, based on the TVA TV-Anytime metadata associated with the consumed content. The semantic modeling capabilities are assessed based on examples of BBC program listings using TVA classification schema vocabularies. Similarites between programs are identified using attributes from different knowledge domains, and the potential for increasing similarity knowledge through second level associations between terms belonging to separate TVA domain-specific vocabularies is demonstrated.

Semantic modelling using TV-anytime genre metadata

Outlining a high level cognitive approach to how we select media based on affective user preferences, we model the latent semantics of lyrics as patterns of emotional components. Using a selection of affective last.fm tags as top-down emotional buoys, we apply LSA latent semantic analysis to bottom-up represent the correlation of terms and song lyrics in a vector space that reflects the emotional context. Analyzing the resulting patterns of affective components, by comparing them against last.fm tag clouds describing the corresponding songs, we propose that it might be feasible to automatically generate affective user preferences based on song lyrics.

/pdf/semantic-contours-in-tracks-based-on-emotional-tags-2sfvyz6eej.pdf

Semantic Contours in Tracks Based on Emotional Tags

In order to extract the atmosphere in media, we model the latent semantics of TV synopses and affective terms as patterns of emotional components. Using a selection of affective last.fm tags and TV-Anytime atmosphere terms as emotional buoys, we apply LSA latent semantic analysis to represent the correlation of terms and descriptions in a vector space that reflects the emotional context. Analyzing the resulting patterns of affective components, we propose that this approach could be applied to automatically generate affective user preferences based on synopsis, subtitles or other textual representations associated with media.

Modeling emotional context from latent semantics

The increasing amounts of media becoming available in converged
digital broadcast and mobile broadband networks will require intelligent interfaces capable of personalizing
the selection of content. Aiming to capture the mood in the content, we construct a semantic space based on tags,
frequently used to describe emotions associated with music in the last.fm social network. Implementing latent semantic analysis (LSA), we model the affective context of songs based on their lyrics, and apply a similar approach to extract
moods from BBC synopsis descriptions of TV episodes using TV-Anytime atmosphere terms. Based on our early results,
we propose that LSA could be implemented as machinelearning method to extract emotional context and model
affective user preferences.

/pdf/extracting-moods-from-songs-and-bbc-programs-based-on-1od3aoe9z1.pdf

Extracting Moods from Songs and BBC Programs Based on Emotional Context

The increasing amounts of streaming and downloadable me- dia becoming available in converged digital broadcast and next genera- tion mobile broadband networks will require intelligent interfaces capable of personalizing the selection of content according to user preferences and moods. We propose an approach to automatically generate atmosphere-like metadata from BBC synopsis descriptions, by applying LSA latent semantic analysis to define the degree of similarity between textual program descriptions and emotional tags in a semantic space.

/pdf/modeling-moods-in-bbc-programs-based-on-emotional-context-52j0zlmsha.pdf

Andrius Butkus

Papers

Semantic modelling using TV-anytime genre metadata

Semantic Contours in Tracks Based on Emotional Tags

Modeling emotional context from latent semantics

Extracting Moods from Songs and BBC Programs Based on Emotional Context

Modeling Moods in BBC Programs Based on Emotional Context