https://hal.archives-ouvertes.fr/hal-01388088/document

A comprehensive survey of mostly textual document segmentation algorithms since 2008

Scene text in the wild is commonly presented with high variant characteristics. Using quadrilateral bounding box to localize the text instance is nearly indispensable for detection methods. However, recent researches reveal that introducing quadrilateral bounding box for scene text detection will bring a label confusion issue which is easily overlooked, and this issue may significantly undermine the detection performance. To address this issue, in this paper, we propose a novel method called Sequential-free Box Discretization (SBD) by discretizing the bounding box into key edges (KE) which can further derive more effective methods to improve detection performance. Experiments showed that the proposed method can outperform state-of-the-art methods in many popular scene text benchmarks, including ICDAR 2015, MLT, and MSRA-TD500. Ablation study also showed that simply integrating the SBD into Mask R-CNN framework, the detection performance can be substantially improved. Furthermore, an experiment on the general object dataset HRSC2016 (multi-oriented ships) showed that our method can outperform recent state-of-the-art methods by a large margin, demonstrating its powerful generalization ability. Source code: this https URL.

/pdf/omnidirectional-scene-text-detection-with-sequential-free-49p1da4asf.pdf

Omnidirectional Scene Text Detection with Sequential-free Box Discretization.

Facial expressions are a significant part of non-verbal communication. Recognizing facial expressions of people with neurological disorders is essential because these people may have lost a significant amount of their verbal communication ability. Such an assessment requires time consuming examination involving medical personnel, which can be quite challenging and expensive. Automated facial expression recognition systems that are low-cost and non-invasive can help experts detect neurological disorders. In this study, an automated facial expression recognition system is developed using a novel deep learning approach. The architecture consists of four-stage networks. The first, second and third networks segment the facial components which are essential for facial expression recognition. Owing to the three networks, an iconize facial image is obtained. The fourth network classifies facial expressions using raw facial images and iconize facial images. This four-stage method combines holistic facial information with local part-based features to achieve more robust facial expression recognition. Preliminary experimental results achieved 94.44% accuracy for facial expression recognition on RaFD database. The proposed system produced 5% improvement than the facial expression recognition system by using raw images. This study presents a quantitative, objective and non-invasive facial expression recognition system to help in the monitoring and diagnosis of neurological disorders influencing facial expressions.

Facial expression recognition for monitoring neurological disorders based on convolutional neural network

Traditional text detection methods mostly focus on quadrangle text. In this study we propose a novel method named sliding line point regression (SLPR) in order to detect arbitrary-shape text in natural scene. SLPR regresses multiple points on the edge of text line and then utilizes these points to sketch the outlines of the text. The proposed SLPR can be adapted to many object detection architectures such as Faster R-CNN and R-FCN. Specifically, we first generate the smallest rectangular box including the text with region proposal network (RPN), then isometrically regress the points on the edge of text by using the vertically and horizontally sliding lines. To make full use of information and reduce redundancy, we calculate x-coordinate or y-coordinate of target point by the rectangular box position, and just regress the remaining y-coordinate or x-coordinate. Accordingly we can not only reduce the parameters of system, but also restrain the points which will generate more regular polygon. Our approach achieved competitive results on traditional ICDAR2015 Incidental Scene Text benchmark and curve text detection dataset CTW1500.

Sliding Line Point Regression for Shape Robust Scene Text Detection

The digital comic book market is growing every year now, mixing digitized and digital-born comics. Digitized comics suffer from a limited automatic content understanding which restricts online content search and reading applications. This study shows how to combine state-of-the-art image analysis methods to encode and index images into an XML-like text file. Content description file can then be used to automatically split comic book images into sub-images corresponding to panels easily indexable with relevant information about their respective content. This allows advanced search in keywords said by specific comic characters, action and scene retrieval using natural language processing. We get down to panel, balloon, text, comic character and face detection using traditional approaches and breakthrough deep learning models, and also text recognition using LSTM model. Evaluations on a dataset composed of online library content are presented, and a new public dataset is also proposed.

https://www.mdpi.com/2313-433X/4/7/89/pdf

Digital Comics Image Indexing Based on Deep Learning

Face detection of comic characters is a necessary step in most applications, such as comic character retrieval, automatic character classification and comic analysis However, the existing methods were developed for simple cartoon images or small size comic datasets, and detection performance remains to be improved In this paper, we propose a Faster R-CNN based method for face detection of comic characters Our contribution is twofold First, for the binary classification task of face detection, we empirically find that the sigmoid classifier shows a slightly better performance than the softmax classifier Second, we build two comic datasets, JC2463 and AEC912, consisting of 3375 comic pages in total for characters face detection evaluation Experimental results have demonstrated that the proposed method not only performs better than existing methods, but also works for comic images with different drawing styles

A Faster R-CNN Based Method for Comic Characters Face Detection

Automatic comic frame extraction is the core technique for comic content adaptation. Typical algorithms segment a comic page into a set of frames using connected components or division lines, but they cannot produce frames without blank margins and there is still room for improvement for complex comic images. We present a method that identifies frame polygons via connected component labeling and line segments combination. We analyze lines within each component and preliminarily judge the frame type, and then we optimize an energy-like score function constrained by several rules to choose frames. Experimental results indicate an increase of both accuracy and F-score compared with previous methods.

Comic frame extraction via line segments combination

Comic panel extraction, i.e., decomposing a comic page image into panels, has become a fundamental technique for meeting many practical needs of mobile comic reading such as comic content adaptation and comic animating. Most of existing approaches are based on handcrafted low-level visual patterns and heuristics rules, thus having limited ability to deal with irregular comic panels. Only one existing method is based on deep learning and achieves better experimental results, but its architecture is redundant and its time efficiency is not good. To address these problems, we propose an end-to-end, two-stage quadrilateral regressing network architecture for comic panel detection, which inherits the architecture of Faster R-CNN. At the first stage, we propose a quadrilateral region proposal network for generating panel proposals, based on a newly proposed quadrilateral regression method. At the second stage, we classify the proposals and refine their shapes with the proposed quadrilateral regression method again. Extensive experimental results demonstrate that the proposed method significantly outperforms the existing comic panel detection methods on multiple datasets by F1-score and page accuracy.

An End-to-End Quadrilateral Regression Network for Comic Panel Extraction

The invention discloses a cartoon image layout recognition method and an automatic recognition system. According to the recognition method, by use of an energy minimization model and an image segmentation and optimization method, polygons constituting image storyboard frames are automatically recognized; a reading order of the image storyboards is obtained according to the geometric position relation of the recognized polygons; The recognition method comprises following steps: foreground and background segmentation, outline detection, polygon fitting, straight-line segment detection, storyboard searching, and reading order detection. The recognition system comprises a foreground and background segmentation module, an outline detection module, a straight-line segment detection module, a storyboard searching module, and a post processing module. According to the position relation of recognized polygons, the reading order of storyboards is determined so that complicated cartoon layouts can be effectively processed, the processing efficiency is increased, and the reading experience of mobile device users is enriched.

Cartoon image layout recognition method and automatic recognition system

The goal of storyboard extraction is to decompose the comic image into storyboards, which is the fundamental step of comic image understanding and producing digital comic documents suitable for mobile reading. Most of existing approaches are based on hand crafted low-level visual patters like edge segments and line segments, which do not capture high-level vision information. To overcome this drawback of the existing approaches, we propose a novel architecture based on deep convolutional neural network, named Shape Regression Network (SReN), to detect storyboards within comic images. Firstly, we use Fast R-CNN to generate rectangle bounding boxes as storyboard proposals. Then we train a deep neural network to predict quadrangles for these proposals. Unlike existing object detection methods which only output rectangle bounding boxes, SReN can produce more precise quadrangle bounding boxes. Experimental results on 7382 comic pages, demonstrate that SReN outperforms the state-of-the-art methods by more than 10% in terms of F1score and page correction rate.

Yafeng Zhou

Papers

A Faster R-CNN Based Method for Comic Characters Face Detection

Comic frame extraction via line segments combination

An End-to-End Quadrilateral Regression Network for Comic Panel Extraction

Cartoon image layout recognition method and automatic recognition system

SReN: Shape Regression Network for Comic Storyboard Extraction.