Accurate document layout analysis is a key requirement for high-quality PDF document conversion. With the recent availability of public,largeground-truthdatasetssuchas PubLayNet andDocBank, deep-learning models have proven to be very effective at layout detection and segmentation. While these datasets are of adequate size to train such models, they severely lack in layout variability since they are sourced from scientific article repositories such as PubMed and arXiv only. Consequently, the accuracy of the layout segmentation drops significantly when these models are applied on more challenging and diverse layouts. In this paper, we present DocLayNet , a new, publicly available, document-layout annotation dataset in COCO format. It contains 80863 manually annotated pages from diverse data sources to represent a wide variability in layouts. For each PDF page, the layout annotations provide labelled bounding-boxes with a choice of 11 distinct classes. DocLayNet also provides a subset of double- and triple-annotated pages to determine the inter-annotator agreement. In multiple experiments, we provide baseline accuracy scores (in mAP) for a set of popular object detection models. We also demonstrate that these models fall approximately 10% behind the inter-annotator agreement. Furthermore, we provide evidence that DocLayNet is of sufficient size. Lastly, we compare models trained on PubLayNet, DocBank and DocLayNet, showing that layout predictions of the DocLayNet-trained models are more robust and thus the preferred choice for general-purpose document-layout analysis.

DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis

Accurate document layout analysis is a key requirement for high-quality PDF document conversion. With the recent availability of public, large ground-truth datasets such as PubLayNet and DocBank, deep-learning models have proven to be very effective at layout detection and segmentation. While these datasets are of adequate size to train such models, they severely lack in layout variability since they are sourced from scientific article repositories such as PubMed and arXiv only. Consequently, the accuracy of the layout segmentation drops significantly when these models are applied on more challenging and diverse layouts. In this paper, we presentDocLayNet, a new, publicly available, document-layout annotation dataset in COCO format. It contains 80863 manually annotated pages from diverse data sources to represent a wide variability in layouts. For each PDF page, the layout annotations provide labelled bounding-boxes with a choice of 11 distinct classes. DocLayNet also provides a subset of double- and triple-annotated pages to determine the inter-annotator agreement. In multiple experiments, we provide baseline accuracy scores (in mAP) for a set of popular object detection models. We also demonstrate that these models fall approximately 10% behind the inter-annotator agreement. Furthermore, we provide evidence that DocLayNet is of sufficient size. Lastly, we compare models trained on PubLayNet, DocBank and DocLayNet, showing that layout predictions of the DocLayNet-trained models are more robust and thus the preferred choice for general-purpose document-layout analysis.

DocLayNet: A Large Human-Annotated Dataset for Document-Layout Segmentation

Document layout analysis is a crucial prerequisite for document understanding, including document retrieval and conversion. Most public datasets currently contain only PDF documents and lack realistic documents. Models trained on these datasets may not generalize well to real-world scenarios. Therefore, this paper introduces a large and diverse document layout analysis dataset called $M^{6}Doc$. The $M^6$ designation represents six properties: (1) Multi-Format (including scanned, photographed, and PDF documents); (2) Multi-Type (such as scientific articles, textbooks, books, test papers, magazines, newspapers, and notes); (3) Multi-Layout (rectangular, Manhattan, non-Manhattan, and multi-column Manhattan); (4) Multi-Language (Chinese and English); (5) Multi-Annotation Category (74 types of annotation labels with 237,116 annotation instances in 9,080 manually annotated pages); and (6) Modern documents. Additionally, we propose a transformer-based document layout analysis method called TransDLANet, which leverages an adaptive element matching mechanism that enables query embedding to better match ground truth to improve recall, and constructs a segmentation branch for more precise document image instance segmentation. We conduct a comprehensive evaluation of $M^{6}Doc$ with various layout analysis methods and demonstrate its effectiveness. TransDLANet achieves state-of-the-art performance on $M^{6}Doc$ with 64.5% mAP. The $M^{6}Doc$ dataset will be available at https://github.com/HCIILAB/M6Doc.

M6Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis

This paper presents a proposal for automatically detecting the structure of regulatory documents, tagging management and text segmentation in units able to be processed as entries in a database. The information stored in that way must be adequate to be efficiently managed by a chatter bot named PTAH, which aims to answer user questions in Natural Language. As part of this work, the usage, main difficulties and characteristics of managing the information in this context is explained. Then, there is a step by step derivation using the proposed method with a small test set of 27 documents. Finally, the results are analyzed and evaluated. The proposal is flexible and robust, based on simple processing, clustering and a rule-based automatic algorithm. 

A Mining approach for Automatic Processing of Regulatory Document

DAZeTD: Deep Analysis of Zones in Torn Documents

Segmentation for document layout analysis: not dead yet

Relation extraction is a fundamental topic in document information extraction. Traditionally, datasets for relation extraction have been annotated with named entities and classified with a subset of relation categories. Models then predict either the entities and relations (end-to-end) or assume the entities are given and only classify the relations. However, current approaches are limited by datasets with a narrow definition of entities and relations. We seek to remedy this by introducing our Descriptive Relation Dataset (DReD), which contains 3286 annotations for descriptions of relations between more general noun phrases inspired by linguistic theory. We benchmark our dataset using several seq2seq models and find that T5 achieves the best results with a ROUGE-1 score of 75.5. We verify the usefulness of DreD by collecting feedback on 100 predictions and comparing human judgment to automated scoring methods. Finally, we verify that relations can be described accurately by transforming the CoNLL04 and Re-TACRED datasets and mapping sentence templates to relation categories. T5 achieves competitive accuracy on CoNLL-04 and Re-TACRED with an F1 score of 78.6 and 90.4, respectively. With this paper, we prove that relations can be described, therefore overcoming the limitations set by previous datasets and approaches. We publicly provide our dataset and training code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/logan-markewich/DReD</uri> . 

Zhi Li

Papers

Segmentation for document layout analysis: not dead yet

Segmentation for document layout analysis: not dead yet

DReD - A Descriptive Relation Dataset for Expanding Relation Extraction