Jesse S. Jin

Bio: Jesse S. Jin is an academic researcher from University of Newcastle. The author has contributed to research in topics: Image retrieval & Image texture. The author has an hindex of 19, co-authored 61 publications receiving 1117 citations. Previous affiliations of Jesse S. Jin include Tianjin University & University of Sydney.

Hierarchical movie affective content analysis based on arousal and valence features

Abstract: Emotional factors directly reflect audiences' attention, evaluation and memory. Affective contents analysis not only create an index for users to access their interested movie segments, but also provide feasible entry for video highlights. Most of the work focus on emotion type detection. Besides emotion type, emotion intensity is also a significant clue for users to find their interested content. For some film genres (Horror, Action, etc), the segments with high emotion intensity have the most possibilities to be video highlights. In this paper, we propose a hierarchical structure for emotion categories and analyze emotion intensity and emotion type by using arousal and valence related features hierarchically. Firstly, High, Medium and Low are detected as emotion intensity levels by using fuzzy c-mean clustering on arousal features. Fuzzy clustering provides a mathematical model to represent vagueness, which is close to human perception. After that, valence related features are used to detect emotion types (Anger, Sad, Fear, Happy and Neutral). Considering video is continuous time series data and the occurrence of a certain emotion is affected by recent emotional history, Hidden Markov Models (HMMs) are used to capture the context information. Experimental results shows the movie segments with high emotion intensity cover over 80% of the movie highlights in Horror and Action movies and the hierarchical method outperforms the one-step method on emotion type detection. Meanwhile, it is flexible for user to pick up their favorite affective content by choosing both emotion intensity levels and emotion types.

Segmentation, categorization, and identification of commercial clips from TV streams using multimodal analysis

Abstract: TV advertising is ubiquitous, perseverant, and economically vital. Millions of people's living and working habits are affected by TV commercials. In this paper, we present a multimodal ("visual + audio + text") commercial video digest scheme to segment individual commercials and carry out semantic content analysis within a detected commercial segment from TV streams.Two challenging issues are addressed. Firstly, we propose a multimodal approach to robustly detect the boundaries of individual commercials. Secondly, we attempt to classify a commercial with respect to advertised products/services. For the first, the boundary detection of individual commercials is reduced to the problem of binary classification of shot boundaries via the mid-level features derived from two concepts: Image Frames Marked with Product Information (FMPI) and Audio Scene Change Indicator (ASCI). Moreover, the accurate individual boundary enables us to perform commercial identification by clip matching via a spatial-temporal signature. For the second, commercial classification is formulated as the task of text categorization by expanding sparse texts from ASR/OCR with external knowledge. Our boundary detection has achieved a good result of F1 = 93.7% on the dataset comprising 499 individual commercials from TRECVID'05 video corpus. Commercial classification has obtained a promising accuracy of 80.9% on 141 distinct ones. Based on these achievements, various applications such as an intelligent digital TV set-top box can be accomplished to enhance the TV viewer's capabilities in monitoring and managing commercials from TV streams.

A Useful Visualization Technique: A Literature Review for Augmented Reality and its Application, limitation & future direction.

Abstract: Augmented reality (AR), a useful visualization technique, is reviewed based literatures. The AR research methods and applications are surveyed since AR was first developed over forty years ago. Recent and future AR researches are proposed which could help researchers of decide which topics should be developed when they are beginning their own researches in the field.

A Useful Visualization Technique: A Literature Review for Augmented Reality and its Application, limitation & future direction

Abstract: Augmented reality (AR), a useful visualization technique, is reviewed based literatures. The AR research methods and applications are surveyed since AR was first developed over forty years ago. Recent and future AR researches are proposed which could help researchers of decide which topics should be developed when they are beginning their own researches in the field.

Computer and Robot Vision

Abstract: Computer and Robot Vision Vol. 1, by R.M. Haralick and Linda G. Shapiro, Addison-Wesley, 1992, ISBN 0-201-10887-1.

A Dataset and Taxonomy for Urban Sound Research

Abstract: Automatic urban sound classification is a growing area of research with applications in multimedia retrieval and urban informatics. In this paper we identify two main barriers to research in this area - the lack of a common taxonomy and the scarceness of large, real-world, annotated data. To address these issues we present a taxonomy of urban sounds and a new dataset, UrbanSound, containing 27 hours of audio with 18.5 hours of annotated sound event occurrences across 10 sound classes. The challenges presented by the new dataset are studied through a series of experiments using a baseline classification system.

Alzheimer's & Dementia: the Journal of the Alzheimer's Association.

Abstract: i n c y h h i t i a R Last month, Alzheimer’s & Dementia: The Journal of the lzheimer’s Association received acceptance for inclusion n MEDLINE, the bibliographic database of the U.S. Naional Library of Medicine (NLM). Three years since the aunch, this achievement marks an important recognition of he Journal’s scientific merit and contribution to the field of lzheimer’s disease research. The editors, our publishing artners from Elsevier, and our sponsoring colleagues from he Alzheimer’s Association are extremely thankful to the uthors, reviewers, Editorial Board members, and readers or their many valuable contributions. As the official journal of the Alzheimer’s Association, lzheimer’s & Dementia will now be circulated to the active embers of the Association’s new International Society to dvance Alzheimer Research and Treatment (ISTAART) imonthly, as well as other subscribers and libraries. The ournal will continue to cover critical scientific, medical, ocial, and policy issues that investigators and clinicians ace every day, on matters concerning healthy brain aging to ll forms of dementia. Unlike other journals in the field, lzheimer’s & Dementia bridges new thinking across dierse areas of investigation. This interdisciplinary journal rovides the impetus for new scientific initiatives and offers

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

Abstract: Digital twin can be defined as a virtual representation of a physical asset enabled through data and simulators for real-time prediction, optimization, monitoring, controlling, and improved decision making. Recent advances in computational pipelines, multiphysics solvers, artificial intelligence, big data cybernetics, data processing and management tools bring the promise of digital twins and their impact on society closer to reality. Digital twinning is now an important and emerging trend in many applications. Also referred to as a computational megamodel, device shadow, mirrored system, avatar or a synchronized virtual prototype, there can be no doubt that a digital twin plays a transformative role not only in how we design and operate cyber-physical intelligent systems, but also in how we advance the modularity of multi-disciplinary systems to tackle fundamental barriers not addressed by the current, evolutionary modeling practices. In this work, we review the recent status of methodologies and techniques related to the construction of digital twins mostly from a modeling perspective. Our aim is to provide a detailed coverage of the current challenges and enabling technologies along with recommendations and reflections for various stakeholders.