[신간의 별자리x] 우리/미술, 그리고 ‘슬픔의 박물관’

RTFN: A robust temporal feature network for time series classification

The history of text can be traced back over thousands of years. Rich and precise semantic information carried by text is important in a wide range of vision-based application scenarios. Therefore, text recognition in natural scenes has been an active research field in computer vision and pattern recognition. In recent years, with the rise and development of deep learning, numerous methods have shown promising in terms of innovation, practicality, and efficiency. This paper aims to (1) summarize the fundamental problems and the state-of-the-art associated with scene text recognition; (2) introduce new insights and ideas; (3) provide a comprehensive review of publicly available resources; (4) point out directions for future work. In summary, this literature review attempts to present the entire picture of the field of scene text recognition. It provides a comprehensive reference for people entering this field, and could be helpful to inspire future research. Related resources are available at our Github repository: this https URL.

Text Recognition in the Wild: A Survey

We present a high-performance method that can achieve mask-level instance segmentation with only bounding-box annotations for training. While this setting has been studied in the literature, here we show significantly stronger performance with a simple design (e.g., dramatically improving previous best reported mask AP of 21.1% in Hsu et al. (2019) to 31.6% on the COCO dataset). Our core idea is to redesign the loss of learning masks in instance segmentation, with no modification to the segmentation network itself. The new loss functions can supervise the mask training without relying on mask annotations. This is made possible with two loss terms, namely, 1) a surrogate term that minimizes the discrepancy between the projections of the ground-truth box and the predicted mask; 2) a pairwise loss that can exploit the prior that proximal pixels with similar colors are very likely to have the same category label. Experiments demonstrate that the redesigned mask loss can yield surprisingly high-quality instance masks with only box annotations. For example, without using any mask annotations, with a ResNet-101 backbone and 3x training schedule, we achieve 33.2% mask AP on COCO test-dev split (vs. 39.1% of the fully supervised counterpart). Our excellent experiment results on COCO and Pascal VOC indicate that our method dramatically narrows the performance gap between weakly and fully supervised instance segmentation. 
Code is available at: this https URL

BoxInst: High-Performance Instance Segmentation with Box Annotations

Scene text recognition (STR) task has a common practice: All state-of-the-art STR models are trained on large synthetic data. In contrast to this practice, training STR models only on fewer real labels (STR with fewer labels) is important when we have to train STR models without synthetic data: for handwritten or artistic texts that are difficult to generate synthetically and for languages other than English for which we do not always have synthetic data. However, there has been implicit common knowledge that training STR models on real data is nearly impossible because real data is insufficient. We consider that this common knowledge has obstructed the study of STR with fewer labels. In this work, we would like to reactivate STR with fewer labels by disproving the common knowledge. We consolidate recently accumulated public real data and show that we can train STR models satisfactorily only with real labeled data. Subsequently, we find simple data augmentation to fully exploit real data. Furthermore, we improve the models by collecting unlabeled data and introducing semi- and self-supervised methods. As a result, we obtain a competitive model to state-of-the-art methods. To the best of our knowledge, this is the first study that 1) shows sufficient performance by only using real labels and 2) introduces semi- and self-supervised methods into STR with fewer labels. Our code and data are available: https://github.com/ku21fan/STR-Fewer-Labels.

/pdf/what-if-we-only-use-real-datasets-for-scene-text-recognition-33vsvtdvw3.pdf

What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels

Chinese scene text reading is one of the most challenging problems in computer vision and has attracted great interest. Different from English text, Chinese has more than 6000 commonly used characters and Chinese characters can be arranged in various layouts with numerous fonts. The Chinese signboards in street view are a good choice for Chinese scene text images since they have different backgrounds, fonts and layouts. We organized a competition called ICDAR2019-ReCTS, which mainly focuses on reading Chinese text on signboard. This report presents the final results of the competition. A large-scale dataset of 25,000 annotated signboard images, in which all the text lines and characters are annotated with locations and transcriptions, were released. Four tasks, namely character recognition, text line recognition, text line detection and end-to-end recognition were set up. Besides, considering the Chinese text ambiguity issue, we proposed a multi ground truth (multi-GT) evaluation method to make evaluation fairer. The competition started on March 1, 2019 and ended on April 30, 2019. 262 submissions from 46 teams are received. Most of the participants come from universities, research institutes, and tech companies in China. There are also some participants from the United States, Australia, Singapore, and Korea. 21 teams submit results for Task 1, 23 teams submit results for Task 2, 24 teams submit results for Task 3, and 13 teams submit results for Task 4. The official website for the competition is http://rrc.cvc.uab.es/?ch=12.

ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard

Chinese scene text reading is one of the most challenging problems in computer vision and has attracted great interest. Different from English text, Chinese has more than 6000 commonly used characters and Chinesecharacters can be arranged in various layouts with numerous fonts. The Chinese signboards in street view are a good choice for Chinese scene text images since they have different backgrounds, fonts and layouts. We organized a competition called ICDAR2019-ReCTS, which mainly focuses on reading Chinese text on signboard. This report presents the final results of the competition. A large-scale dataset of 25,000 annotated signboard images, in which all the text lines and characters are annotated with locations and transcriptions, were released. Four tasks, namely character recognition, text line recognition, text line detection and end-to-end recognition were set up. Besides, considering the Chinese text ambiguity issue, we proposed a multi ground truth (multi-GT) evaluation method to make evaluation fairer. The competition started on March 1, 2019 and ended on April 30, 2019. 262 submissions from 46 teams are received. Most of the participants come from universities, research institutes, and tech companies in China. There are also some participants from the United States, Australia, Singapore, and Korea. 21 teams submit results for Task 1, 23 teams submit results for Task 2, 24 teams submit results for Task 3, and 13 teams submit results for Task 4. The official website for the competition is this http URL.

ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard.

Scene text detection is one of the most challenging problems in computer vision and has attracted great interest. Different from generic object detection, scene text detection mainly suffers from the large variance of scale, aspect ratio, and orientation in scene text. In this paper, we propose an effective and efficient model (SEG-FPN) for scene text detection, which is based on Feature Pyramid Network (FPN) and Linking Segments (SegLink). We incorporate feature pyramid mechanism with Single Shot Detector (SSD) framework to deal with different scale texts, and link locally detectable elements to detect texts of different orientations and aspect ratios. Moreover, compared with SSD, we enlarge the feature map of deep layers to better localize the large texts and recognize the small texts accurately. Experiments on ICDAR2015 and ICDAR2013 datasets demonstrate that our method can achieve comparable performance in terms of both accuracy and time. Specifically, SEG-FPN achieves an f-measure of 0.820 at 10.3 fps for 1280*768 ICDAR 2015 Incidental text images, and an f-measure of 0.879 at 19.2 fps for 512*512 ICDAR 2013 focused scene text images.

Scene Text Detection with Feature Pyramid Network and Linking Segments

Chinese characters are always arranged in arbitrary orders and forms in scene images which are difficult to handle for text line based methods. Recently approaches adopt irregular text line reading to solve this problem but a text line in Chinese is often ambiguous. In this paper, we propose a character based framework to spotting Chinese text from another view. Specifically, the framework consists of three components for character detection, character recognition and character grouping respectively. It is worth mentioning that a novel Conditional Random Field(CRF) based character grouping algorithm is presented for arbitrary arrangement in Chinese text. Experiments on ReCTS-ARB549 dataset demonstrate that our framework achieves superior performance comparing with state of the art text line based approaches.

Reading Chinese Scene Text with Arbitrary Arrangement Based on Character Spotting

The invention discloses a shop arrival confirmation method. The method comprises the steps of obtaining real-time position information uploaded by a user and a first door face image; confirming a first merchant name of an actual merchant where the user is located currently according to the first door face image and a candidate merchant name corresponding to the real-time position information; anddetecting whether the first merchant name is matched with a second merchant name of a target merchant corresponding to the user currently, detecting whether the first door face image is matched with asecond door face image of the target merchant, and determining whether the user reaches the target merchant according to a detection result. The technical problem that in-store confirmation is inaccurate is solved. The beneficial effect of improving the in-store detection accuracy is achieved.

Dong Wang

Papers

ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard

ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard.

Scene Text Detection with Feature Pyramid Network and Linking Segments

Reading Chinese Scene Text with Arbitrary Arrangement Based on Character Spotting

In-store confirmation method and device, electronic equipment and readable storage medium