Text-Based Semantic Video Annotation for Interactive Cooking Videos

TLDR: A text-based semantic video annotation system for interactive cooking videos to facilitate user interactions and is superior to existing alignment algorithms and effective in semantic cooking video annotation.

Abstract: Videos represent one of the most frequently used forms of multimedia applications. In addition to watching videos, people control slider bars of video players to find specific scenes and want detailed information on certain objects in scenes. However, it is difficult to support user interactions in current video formats because of a lack of metadata for facilitating such interactions. This paper proposes a text-based semantic video annotation system for interactive cooking videos to facilitate user interactions. The proposed annotation process includes three parts: the synchronization of recipes and corresponding cooking videos based on a caption-recipe alignment algorithm; the information extraction of food recipes based on lexico-syntactic patterns; and the semantic interconnection between recognized entities and web resources. The experimental results show that the proposed system is superior to existing alignment algorithms and effective in semantic cooking video annotation.