Proceedings ArticleDOI
ASYSST: A Framework for Synopsis Synthesis Empowering Visually Impaired
TLDR
This work proposes an end to end framework (ASYSST) for textual description synthesis from digitized building floor plans and introduces a novel Bag of Decor feature to learn $5$ classes of a room from $1355$ samples under a supervised learning paradigm.Abstract:
In an indoor scenario, the visually impaired do not have the information about the surroundings and finds it difficult to navigate from room to room. The sensor-based solutions are expensive and may not always be comfortable for the end users. In this paper, we focus on the problem of synthesis of textual description from a given floor plan image to assist the visually impaired. The textual description, in addition to a text reading software, can aid the visually impaired person while moving inside a building. In this work, for the first time, we propose an end to end framework (ASYSST) for textual description synthesis from digitized building floor plans. We have introduced a novel Bag of Decor (BoD) feature to learn $5$ classes of a room from $1355$ samples under a supervised learning paradigm. These learned labels are fed into a description synthesis framework to yield a holistic description of a floor plan image. Experimental analysis of real publicly available floor plan data-set proves the superiority of our framework.read more
Citations
More filters
Proceedings ArticleDOI
BRIDGE: Building Plan Repository for Image Description Generation, and Evaluation
TL;DR: An extensive experimental study is presented for tasks like furniture localization in a floor plan, caption and description generation, on the proposed dataset showing the utility of BRIDGE.
Extending the flexibility of case-based design support tools: A use case in the architectural domain
Viktor Ayzenshtadt,Viktor Ayzenshtadt,Christoph Langenhan,Syed Saqib Bukhari,Klaus-Dieter Althoff,Klaus-Dieter Althoff,Frank Petzold,Andreas Dengel,Andreas Dengel +8 more
TL;DR: In this article, the authors present results of a user study into extending the functionality of an existing casebased search engine for similar architectural designs to a flexible process-oriented case-based support tool for the architectural conceptualization phase.
Journal ArticleDOI
Knowledge-driven description synthesis for floor plan interpretation
TL;DR: In this paper, the authors proposed two models, description synthesis from image cue (DSIC) and transformer-based description generation (TBDG), for text generation from floor plan images.
Posted Content
Knowledge driven Description Synthesis for Floor Plan Interpretation
TL;DR: In this paper, the authors proposed two models, Description Synthesis from Image Cue (DSIC) and Transformer Based Description Generation (TBDG), for floor plan image to text generation to fill the gaps in existing methods.
References
More filters
Journal ArticleDOI
Deep Visual-Semantic Alignments for Generating Image Descriptions
Andrej Karpathy,Li Fei-Fei +1 more
TL;DR: A model that generates natural language descriptions of images and their regions based on a novel combination of Convolutional Neural Networks over image regions, bidirectional Recurrent Neural networks over sentences, and a structured objective that aligns the two modalities through a multimodal embedding is presented.
Proceedings Article
Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems
Michael Denkowski,Alon Lavie +1 more
TL;DR: Meteor 1.3 as discussed by the authors was the first submission to the 2011 EMNLP Workshop on Statistical Machine Translation automatic evaluation metric tasks, which included improved text normalization, higher-precision paraphrase matching, and discrimination between content and function words.
Posted Content
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
Raffaella Bernardi,Ruket Cakici,Desmond Elliott,Aykut Erdem,Erkut Erdem,Nazli Ikizler-Cinbis,Frank Keller,Adrian Muscat,Barbara Plank +8 more
TL;DR: This survey classify the existing approaches based on how they conceptualize this problem, viz., models that cast description as either generation problem or as a retrieval problem over a visual or multimodal representational space.
Proceedings ArticleDOI
Improved Automatic Analysis of Architectural Floor Plans
TL;DR: This paper proposes a novel complete system for automated floor plan analysis that outperforms previous systems and introduces novel preprocessing methods, e.g., the differentiation between thick, medium, and thin lines and the removal of components outside the convex hull of the outer walls.
Proceedings ArticleDOI
A system to detect rooms in architectural floor plan images
TL;DR: A primitive extraction algorithm based on an original coupling of classical Hough transform with image vectorization in order to perform robust and efficient line detection and the way it detects some door hypothesis thanks to the extraction of arcs is presented.