“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Optical Character Recognition (OCR) plays an important role in the retrieval of information from pixel-based images to searchable and machine-editable text formats. In old or poorly printed documents, printed characters are typically broken and blurred, making character recognition potentially far more complex. In this work, deep neural network using Inception V3 is used to train and perform OCR. The Inception V3 network is trained with 53,342 noisy character images, which were collected from receipts and newspapers. Our experimental results show that the proposed deep neural network achieved significantly better recognition accuracy on poor quality text images and resulted in an overall 21.5% reduction in error rate compared to existing OCRs.

Improved optical character recognition with deep neural network

Characters in historical documents are typically densely distributed and are difficult to localize and segment by directly applying classic proposal and regression based methods. In this paper, we propose a novel method called recognition guided detector (RGD) that achieves tight Chinese character detection in historical documents. The proposed RGD consists of two simultaneously trained convolutional neural networks: a recognition guided proposal network that provides context information of the text and a detection network that applies this information to localize each of the characters accurately. To train and test the proposed method, we established two new datasets with character-level annotations, comprising ground truth character bounding boxes and ground truth characters in each of the boxes. The data in our datasets are scanned images collected from nine different versions of Tripitaka in Han. Experimental results show that, guided by a text recognition network with a test accuracy of 97.25%, the detection network in our proposed method achieves a much higher F-score with fewer parameters under a highly constrained evaluation criterion of intersection of union (IoU) ≥ 0.7, when comparing to several state-of-the-art object detection and text detection methods. The datasets are publicly available at https://github.com/HCIILAB/TKH_MTH_Datasets_Release for non-commercial use.

Dense and Tight Detection of Chinese Characters in Historical Documents: Datasets and a Recognition Guided Detector

Since past few years, deep neural networks, because of their outstanding performance, are getting highly used in computer vision and machine learning tasks such as regression, segmentation, classification, detection, pattern recognition etc. Recognition of handwritten Devanagari characters is challenging task, but Deep learning can be effectively used as a solution for various such problems. Person to person variations in writing style makes handwritten character recognition one of the most difficult tasks. In this Experiment, we successfully tried to classify handwritten Devanagari characters using transfer learning mechanism with the help of Alexnet. Alexnet, a convolutional neural network, is trained over a dataset of around 16870 samples of 22 consonants of Devanagari script which shows impressive results. The transfer learning helps to learn faster and better even if the data samples are less as compared with the training a CNN from scratch.

Handwritten Devanagari Character Classification using Deep Learning.

Nowadays, deep learning methods are employed in a broad range of research fields. The analysis and recognition of historical documents, as we survey in this work, is not an exception. Our study analyzes the papers published in the last few years on this topic from different perspectives: we first provide a pragmatic definition of historical documents from the point of view of the research in the area, then we look at the various sub-tasks addressed in this research. Guided by these tasks, we go through the different input-output relations that are expected from the used deep learning approaches and therefore we accordingly describe the most used models. We also discuss research datasets published in the field and their applications. This analysis shows that the latest research is a leap forward since it is not the simple use of recently proposed algorithms to previous problems, but novel tasks and novel applications of state of the art methods are now considered. Rather than just providing a conclusive picture of the current research in the topic we lastly suggest some potential future trends that can represent a stimulus for innovative research directions.

Deep Learning for Historical Document Analysis and Recognition—A Survey

Historical Chinese character recognition has been suffering from the problem of lacking sufficient labeled training samples. A transfer learning method based on Convolutional Neural Network (CNN) for historical Chinese character recognition is proposed in this paper. A CNN model L is trained by printed Chinese character samples in the source domain. The network structure and weights of model L are used to initialize another CNN model T, which is regarded as the feature extractor and classifier in the target domain. The model T is then fine-tuned by a few labeled historical or handwritten Chinese character samples, and used for final evaluation in the target domain. Several experiments regarding essential factors of the CNNbased transfer learning method are conducted, showing that the proposed method is effective.

CNN Based Transfer Learning for Historical Chinese Character Recognition

Historical Chinese character recognition faces problems including low image quality and lack of labeled training samples. We propose a generative adversarial network (GAN) based transfer learning method to ease these problems. The proposed TH-GAN architecture includes a discriminator and a generator. The network structure of the discriminator is based on a convolutional neural network (CNN). Inspired by Wasserstein GAN, the loss function of the discriminator aims to measure the probabilistic distribution distance of the generated images and the target images. The network structure of the generator is a CNN based encoder-decoder. The loss function of the generator aims to minimize the distribution distance between the real samples and the generated samples. In order to preserve the complex glyph structure of a historical Chinese character, a weighted mean squared error (MSE) criterion by incorporating both the edge and the skeleton information in the ground truth image is proposed as the weighted pixel loss in the generator. These loss functions are used for joint training of the discriminator and the generator. Experiments are conducted on two tasks to evaluate the performance of the proposed TH-GAN. The first task is carried out on style transfer mapping for multi-font printed traditional Chinese character samples. The second task is carried out on transfer learning for historical Chinese character samples by adding samples generated by TH-GAN. Experimental results show that the proposed TH-GAN is effective.

TH-GAN: Generative Adversarial Network Based Transfer Learning for Historical Chinese Character Recognition

Although transfer learning has aroused researchers' great interest, how to utilize the unlabeled data is still an open and important problem in this area We propose a novel semi-supervised transfer learning (STL) method by incorporating Multi-Kernel Maximum Mean Discrepancy (MK-MMD) loss into the traditional fine-tuned Convolutional Neural Network (CNN) transfer learning framework for Chinese character recognition The proposed method includes three steps First, a CNN model is trained by massive labeled samples in the source domain Then the CNN model is fine-tuned by a few labeled samples in the target domain Finally, the CNN model is trained with both a large number of unlabeled samples and the limited labeled samples in the target domain to minimize the MK-MMD loss Experiments investigate detailed configurations and parameters of the proposed STL method with several frequently used CNN structures including AlexNet, GoogLeNet, and ResNet Experimental results on practical Chinese character transfer learning tasks, such as Dunhuang historical Chinese character recognition, indicate that the proposed method can significantly improve recognition accuracy in the target domain

Semi-Supervised Transfer Learning for Convolutional Neural Network Based Chinese Character Recognition

Recognition of handwritten mathematical expressions (HMEs) has become a cutting edge research topic recently, as there are increasingly needs for pen-inputting applications. In this paper, we presented a novel framework to analyse HME layout and semantic information. This framework includes three steps, namely symbol segmentation, symbol recognition and semantic relationship analysis. For symbol segmentation, a decomposition on strokes is operated, then dynamic programming is adopted to find the paths corresponding to the best segmentation manner and reduce the stroke searching complexity. For symbol recognition, spatial geometry and directional element features are classified by a Gaussian Mixture Model learnt through Expectation-Maximization algorithm. At last, in the semantic relationship analysis module, a ternary tree is utilized to to store the ranked symbols through calculating the operator priorities. The motivation for our work comes from the apparent difference in writing styles across western and Chinese populations. Our results are reasonable and show promise on the private dataset.

On-line Handwritten Mathematical Expression Recognition Method Based on Statistical and Semantic Analysis

The invention provides a semi-supervised-transfer-learning character recognition method based on a convolutional neural network. The method includes: using batch character image samples, which are ina target domain and have no class label, as a test sample set to input the same to the convolutional neural network after semi-supervised transfer learning, and identifying character images of the test sample set, wherein the convolutional neural network after semi-supervised transfer learning is obtained by joint training of batch character image samples which are in a source domain and have class labels, batch character image samples which are in the target domain and have class labels, and batch character image samples which are in the target domain and have no class label, and recognitionprecision is improved. According to the semi-supervised-transfer-learning character recognition method based on the convolutional neural network provided and a semi-supervised-transfer-learning character recognition system based on the convolutional neural network provided by the invention, the large number of source-domain samples with the class labels, the small number of target-domain sampleswith the class labels and the relatively easily obtained target-domain samples without a class label can be utilized for semi-supervised transfer learning, and adaptation ability of the convolutionalneural network on the target-domain samples is improved.

Yejun Tang

Papers

CNN Based Transfer Learning for Historical Chinese Character Recognition

TH-GAN: Generative Adversarial Network Based Transfer Learning for Historical Chinese Character Recognition

Semi-Supervised Transfer Learning for Convolutional Neural Network Based Chinese Character Recognition

On-line Handwritten Mathematical Expression Recognition Method Based on Statistical and Semantic Analysis

Semi-supervised-transfer-learning character recognition method and system based on convolutional neural network