Detection of Disaster-Affected Cultural Heritage Sites from Social Media Images Using Deep Learning Techniques
TLDR
The results show that while the automatic classification is not perfect, it can greatly reduce the manual effort required to find photos of damaged cultural heritage by accurately detecting relevant candidates to be examined by a cultural heritage professional.Abstract:
This article describes a method for early detection of disaster-related damage to cultural heritage. It is based on data from social media, a timely and large-scale data source that is nevertheless quite noisy. First, we collect images posted on social media that may refer to a cultural heritage site. Then, we automatically categorize these images according to two dimensions: whether they are indeed a photo in which a cultural heritage resource is the main subject, and whether they represent damage. Both categorizations are challenging image classification tasks, given the ambiguity of these visual categories; we tackle both tasks using a convolutional neural network. We test our methodology on a large collection of thousands of images from the web and social media, which exhibit the diversity and noise that is typical of these sources, and contain buildings and other architectural elements, heritage and not-heritage, damaged by disasters as well as intact. Our results show that while the automatic classification is not perfect, it can greatly reduce the manual effort required to find photos of damaged cultural heritage by accurately detecting relevant candidates to be examined by a cultural heritage professional.read more
Citations
More filters
Journal ArticleDOI
Risk protection for cultural heritage and historic centres: Current knowledge and further research needs
TL;DR: The Special Issue on Risk Protection for Cultural Heritage and Historic Centres as mentioned in this paper reviewed the gaps in knowledge and practice related to disaster risk management of cultural heritage and highlighted specific issues that need to be addressed.
Journal ArticleDOI
Risk protection for cultural heritage and historic centres: Current knowledge and further research needs
TL;DR: The Special Issue on Risk Protection for Cultural Heritage and Historic Centres as mentioned in this paper reviewed the gaps in knowledge and practice related to disaster risk management of cultural heritage and highlighted specific issues that need to be addressed.
Multi-source Multimodal Data and Deep Learning for Disaster Response: A Systematic Review
TL;DR: In this paper, the authors conduct a systematic review of 83 articles to identify the successes, current and future challenges, and opportunities in using DL for disaster response tasks, focusing on the components of learning, a set of aspects that govern the application of Machine Learning (ML) for a given problem domain.
Journal ArticleDOI
CollabLearn: An Uncertainty-Aware Crowd-AI Collaboration System for Cultural Heritage Damage Assessment
TL;DR: CollabLearn as discussed by the authors is an uncertainty-aware crowd-AI collaborative assessment system that explicitly explores the human intelligence from crowdsourcing systems to identify and fix AI failure cases and boost the damage assessment accuracy.
Journal ArticleDOI
Cultural Perception of the Historical and Cultural Blocks of Beijing Based on Weibo Photos
TL;DR: Wang et al. as discussed by the authors investigated the cultural perception of historical and cultural blocks in Beijing, the capital of China, and constructed a system of cultural perception symbols based on the cultural connotations of the capital.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Proceedings Article
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan,Andrew Zisserman +1 more
TL;DR: In this paper, the authors investigated the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting and showed that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 layers.
Journal ArticleDOI
Deep learning
TL;DR: Deep learning is making major advances in solving problems that have resisted the best attempts of the artificial intelligence community for many years, and will have many more successes in the near future because it requires very little engineering by hand and can easily take advantage of increases in the amount of available computation and data.