While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

/pdf/the-unreasonable-effectiveness-of-deep-features-as-a-10k66u5qas.pdf

The PASCAL Visual Object Classes Challenge

A morphable model for the synthesis of 3D faces

我的台灣, 看見心靈的故鄉 =2009林磐聳藝術與設計展

ICT platforms for news production, distribution, and consumption must exploit the ever-growing availability of digital data. These data originate from different sources and in different formats; they arrive at different velocities and in different volumes. Semantic knowledge graphs (KGs) is an established technique for integrating such heterogeneous information. It is therefore well-aligned with the needs of news producers and distributors, and it is likely to become increasingly important for the news industry. This article reviews the research on using semantic knowledge graphs for production, distribution, and consumption of news. The purpose is to present an overview of the field; to investigate what it means; and to suggest opportunities and needs for further research and development.

Semantic Knowledge Graphs for the News: A Review

Lifelog, Human Activity Recognition, Smartphones, Embedded Sensors, Smart-City, Heterogeneous Sensory Data Analytics.
This paper introduces a new idea for sensor data analytics, named PHASOR, that can recognize and stream individual human activities online. The proposed sensor concept can be utilized to solve some emerging problems in smartcity domain such as health care, urban mobility, or security by creating a lifelog of human activities. PHASOR is created from three ‘components’: ID, model, and Sensor. The first component is to identify which sensor is used to monitor which object (e.g., group of users, individual users, type of smart- phone). The second component decides suitable classifiers for human activities recognition. The last one includes two types: (1) physical sensors that utilize embedded sensors in smartphones to recognize human activities, (2) human factors that uses human interaction to personally increase the accuracy of the detection. The advantage of PHASOR is the error signal is inversely proportional to its lifetime, which is well-suited for lifelogging applications. The proposed concept is evaluated and compared to de-facto datasets as well as state-of-the-art of Human Activity Recognition (HAR) using smartphones, confirming that applying PHASOR can improves the accuracy of HAR.

Smart lifelogging: recognizing human activities using PHASOR

The rapid development of technology in social media platforms has led to abundant misinformation and fake news spreading in the community. One of the most prevalent ways to misleading information on social media is cheapfakes, which are more accessible and affordable than deepfakes. Most existing approaches extract features from text or concatenate visual and textual features and train with multimodal to classify news. This paper proposed several strategies to leverage object, textual, image captioning features. These strategies focus on utilizing image captioning to extract the correlation between images and captions. We also propose some boosting techniques to enhance the result. Our methods are evaluated on the "MMSys'21 Grand Challenge" dataset and have 86.75% accuracy.

Multimodal Cheapfakes Detection by Utilizing Image Captioning for Global Context

Nowadays, large-scale networked social media need better search technologies to achieve suitable performance Multimodal approaches are promising technologies to improve image ranking This is particularly true when metadata are not completely reliable, which is a rather common case as far as user annotation, time and location are concerned In this paper, we propose to properly combine visual information with additional multi-faceted information, to define a novel multimodal similarity measure More specifically, we combine visual features, which strongly relate to the image content, with semantic information represented by manually annotated concepts, and geo tagging, very often available in the form of object/subject location Furthermore, we propose a supervised machine learning approach, based on Support Vector Machines (SVMs), to automatically learn optimized weights to combine the above features The resulting models is used as a ranking function to sort the results of a multimodal query

/pdf/supervised-models-for-multimodal-image-retrieval-based-on-1on7uky6pp.pdf

Supervised models for multimodal image retrieval based on visual, semantic and geographic information

Lifelogging refers to the process of digitally capturing a continuous and detailed trace of life activities in a passive manner. In order to assist the research community to make progress in the organisation and retrieval of data from lifelog archives, a lifelog task was organised at NTCIR since edition 12. Lifelog-3 was the third running of the lifelog task (at NTCIR-14) and the Lifelog-3 task explored three different lifelog data access related challenges, the search challenge, the annotation challenge and the insights challenge. In this paper we review the dataset created for this activity, activities of participating teams who took part in these challenges and we highlight learnings for the community from the NTCIR-Lifelog challenges.

/pdf/advances-in-lifelog-data-organisation-and-retrieval-at-the-5dfobt80g6.pdf

Duc-Tien Dang-Nguyen

Papers

Semantic Knowledge Graphs for the News: A Review

Smart lifelogging: recognizing human activities using PHASOR

Multimodal Cheapfakes Detection by Utilizing Image Captioning for Global Context

Supervised models for multimodal image retrieval based on visual, semantic and geographic information

Advances in Lifelog Data Organisation and Retrieval at the NTCIR-14 Lifelog-3 Task