Showing papers on "Handwriting recognition published in 2018"

Drawing and Recognizing Chinese Characters with Recurrent Neural Network

Abstract: Recent deep learning based approaches have achieved great success on handwriting recognition. Chinese characters are among the most widely adopted writing systems in the world. Previous research has mainly focused on recognizing handwritten Chinese characters. However, recognition is only one aspect for understanding a language, another challenging and interesting task is to teach a machine to automatically write (pictographic) Chinese characters. In this paper, we propose a framework by using the recurrent neural network (RNN) as both a discriminative model for recognizing Chinese characters and a generative model for drawing (generating) Chinese characters. To recognize Chinese characters, previous methods usually adopt the convolutional neural network (CNN) models which require transforming the online handwriting trajectory into image-like representations. Instead, our RNN based approach is an end-to-end system which directly deals with the sequential structure and does not require any domain-specific knowledge. With the RNN system (combining an LSTM and GRU), state-of-the-art performance can be achieved on the ICDAR-2013 competition database. Furthermore, under the RNN framework, a conditional generative model with character embedding is proposed for automatically drawing recognizable Chinese characters. The generated characters (in vector format) are human-readable and also can be recognized by the discriminative RNN model with high accuracy. Experimental results verify the effectiveness of using RNNs as both generative and discriminative models for the tasks of drawing and recognizing Chinese characters.

Convolutional Neural Network (CNN) for Image Detection and Recognition

Abstract: Deep Learning algorithms are designed in such a way that they mimic the function of the human cerebral cortex. These algorithms are representations of deep neural networks i.e. neural networks with many hidden layers. Convolutional neural networks are deep learning algorithms that can train large datasets with millions of parameters, in form of 2D images as input and convolve it with filters to produce the desired outputs. In this article, CNN models are built to evaluate its performance on image recognition and detection datasets. The algorithm is implemented on MNIST and CIFAR-10 dataset and its performance are evaluated. The accuracy of models on MNIST is 99.6 %, CIFAR-10 is using real-time data augmentation and dropout on CPU unit.

Improving CNN-RNN Hybrid Networks for Handwriting Recognition

Abstract: The success of deep learning based models have centered around recent architectures and the availability of large scale annotated data. In this work, we explore these two factors systematically for improving handwritten recognition for scanned off-line document images. We propose a modified CNN-RNN hybrid architecture with a major focus on effective training using: (i) efficient initialization of network using synthetic data for pretraining, (ii) image normalization for slant correction and (iii) domain specific data transformation and distortion for learning important invariances. We perform a detailed ablation study to analyze the contribution of individual modules and present state of art results for the task of unconstrained line and word recognition on popular datasets such as IAM, RIMES and GW.

Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition

Abstract: Handwritten mathematical expression recognition is a challenging problem due to the complicated two-dimensional structures, ambiguous handwriting input and variant scales of handwritten math symbols. To settle this problem, recently we propose the attention based encoder-decoder model that recognizes mathematical expression images from two-dimensional layouts to one-dimensional LaTeX strings. In this study, we improve the encoder by employing densely connected convolutional networks as they can strengthen feature extraction and facilitate gradient propagation especially on a small training set. We also present a novel multi-scale attention model which is employed to deal with the recognition of math symbols in different scales and restore the fine-grained details dropped by pooling operations. Validated on the CROHME competition task, the proposed method significantly outperforms the state-of-the-art methods with an expression recognition accuracy of 52.8% on CROHME 2014 and 50.1% on CROHME 2016, by only using the official training dataset.

Start, Follow, Read: End-to-End Full-Page Handwriting Recognition

Abstract: Despite decades of research, offline handwriting recognition (HWR) of degraded historical documents remains a challenging problem, which if solved could greatly improve the searchability of online cultural heritage archives. HWR models are often limited by the accuracy of the preceding steps of text detection and segmentation. Motivated by this, we present a deep learning model that jointly learns text detection, segmentation, and recognition using mostly images without detection or segmentation annotations. Our Start, Follow, Read (SFR) model is composed of a Region Proposal Network to find the start position of text lines, a novel line follower network that incrementally follows and preprocesses lines of (perhaps curved) text into dewarped images suitable for recognition by a CNN-LSTM network. SFR exceeds the performance of the winner of the ICDAR2017 handwriting recognition competition, even when not using the provided competition region annotations.

WordRecorder: Accurate Acoustic-based Handwriting Recognition Using Deep Learning

Abstract: This paper presents WordRecorder, an efficient and accurate handwriting recognition system that identifies words using acoustic signals generated by pens and paper, thus enabling ubiquitous handwriting recognition. To achieve this, we carefully craft a new deep-learning based acoustic sensing framework with three major components, i.e., segmentation, classification, and word suggestion. First, we design a dual-window approach to segment the raw acoustic signal into a series of words and letters by exploiting subtle acoustic signal features of handwriting. Then we integrate a set of simple yet effective signal processing techniques to further refine raw acoustic signals into normalized spectrograms which are suitable for deep-learning classification. After that, we customize a deep neural network that is suitable for smart devices. Finally, we incorporate a word suggestion module to enhance the recognition performance. Our framework achieves both computation efficiency and desirable classification accuracy simultaneously. We prototype our design using off-the-shelf smartwatches and conduct extensive evaluations. Our results demonstrate that WordRecorder robustly archives 81% accuracy rate for trained users, and 75% for users without training, across a range of different environment, users, and writing habits.

Pentelligence: Combining Pen Tip Motion and Writing Sounds for Handwritten Digit Recognition

Abstract: Digital pens emit ink on paper and digitize handwriting. The range of the pen is typically limited to a special writing surface on which the pen's tip is tracked. We present Pentelligence, a pen for handwritten digit recognition that operates on regular paper and does not require a separate tracking device. It senses the pen tip's motions and sound emissions when stroking. Pen motions and writing sounds exhibit complementary properties. Combining both types of sensor data substantially improves the recognition rate. Hilbert envelopes of the writing sounds and mean-filtered motion data are fed to neural networks for majority voting. The results on a dataset of 9408 handwritten digits taken from 26 individuals show that motion+sound outperforms single-sensor approaches at an accuracy of 78.4% for 10 test users. Retraining the networks for a single writer on a dataset of 2120 samples increased the precision to 100% for single handwritten digits at an overall accuracy of 98.3%.

Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks

Abstract: Unconstrained text recognition is an important computer vision task, featuring a wide variety of different sub-tasks, each with its own set of challenges. One of the biggest promises of deep neural networks has been the convergence and automation of feature extractors from input raw signals, allowing for the highest possible performance with minimum required domain knowledge. To this end, we propose a data-efficient, end-to-end neural network model for generic, unconstrained text recognition. In our proposed architecture we strive for simplicity and efficiency without sacrificing recognition accuracy. Our proposed architecture is a fully convolutional network without any recurrent connections trained with the CTC loss function. Thus it operates on arbitrary input sizes and produces strings of arbitrary length in a very efficient and parallelizable manner. We show the generality and superiority of our proposed text recognition architecture by achieving state of the art results on seven public benchmark datasets, covering a wide spectrum of text recognition tasks, namely: Handwriting Recognition, CAPTCHA recognition, OCR, License Plate Recognition, and Scene Text Recognition. Our proposed architecture has won the ICFHR2018 Competition on Automated Text Recognition on a READ Dataset.

Handwritten Hindi Digits Recognition Using Convolutional Neural Network with RMSprop Optimization

Abstract: An efficient handwritten Hindi numeral digit recognition structure based on Convolutional Neural Network (CNN) with RMSprop optimization technique is present in this paper. Convolutional Neural Networks as a powerful feature extraction do not use the predefined kernels, but instead they learn data from specific kernels. The structural design of the network consists of convolutional (Conv2D) layer, pooling (MaxPool2D) layer, Flatten layer and two fully-connected layers. Where a sliding window function is applied to a matrix of a numerical image. We evaluated our scheme on 20,000 handwritten samples of Hindi numerals from kaggle dataset and from this experiment we achieved 99.85% recognition rate by the proposed method Convolutional Neural Network with RMSprop (Root Mean Square Propagation) Optimization which is very promising results.

Offline Handwriting Recognition on Devanagari Using a New Benchmark Dataset

Abstract: Handwriting recognition (HWR) in Indic scripts, like Devanagari is very challenging due to the subtleties in the scripts, variations in rendering and the cursive nature of the handwriting. Lack of public handwriting datasets in Indic scripts has long stymied the development of offline handwritten word recognizers and made comparison across different methods a tedious task in the field. In this paper, we release a new handwritten word dataset for Devanagari, IIIT-HW-Dev to alleviate some of these issues. We benchmark the IIIT-HW-Dev dataset using a CNN-RNN hybrid architecture. Furthermore, using this architecture, we empirically show that usage of synthetic data and cross lingual transfer learning helps alleviate the issue of lack of training data. We use this proposed pipeline on a public dataset, RoyDB and achieve state of the art results.

Boosting Handwriting Text Recognition in Small Databases with Transfer Learning

Abstract: In this paper we deal with the offline handwriting text recognition (HTR) problem with reduced training data sets. Recent HTR solutions based on artificial neural networks exhibit remarkable solutions in referenced databases. These deep learning neural networks are composed of both convolutional (CNN) and long short-term memory recurrent units (LSTM). In addition, connectionist temporal classification (CTC) is the key to avoid segmentation at character level, greatly facilitating the labeling task. One of the main drawbacks of the CNN-LSTM-CTC (CRNN) solutions is that they need a considerable part of the text to be transcribed for every type of calligraphy, typically in the order of a few thousands of lines. Furthermore, in some scenarios the text to transcribe is not that long, e.g. in the Washington database. The CRNN typically overfits for this reduced number of training samples. Our proposal is based on the transfer learning (TL) from the parameters learned with a bigger database. We first investigate, for a reduced and fixed number of training samples, 350 lines, how the learning from a large database, the IAM, can be transferred to the learning of the CRNN of a reduced database, Washington. We focus on which layers of the network could not be re-trained. We conclude that the best solution is to re-train the whole CRNN parameters initialized to the values obtained after the training of the CRNN from the larger database. We also investigate results when the training size is further reduced. For the sake of comparison, we study the character error rate (CER) with no dictionary or any language modeling technique. The differences in the CER are more remarkable when training with just 350 lines, a CER of 3.3% is achieved with TL while we have a CER of 18.2% when training from scratch. As a byproduct, the learning times are quite reduced. Similar good results are obtained from the Parzival database when trained with this reduced number of lines and this new approach.

Does Deeper Network Lead to Better Accuracy: A Case Study on Handwritten Devanagari Characters

Abstract: Deep neural network architectures have been used successfully in various document analysis studies. Its strength in producing human like performance has already been explored in handwritten English numeral recognition task. In this context, a natural question that often arises in a practitioner's mind: does an increase in the depth of the network eventually lead to an improved recognition performance on unknown samples? A goal of the present work is to search for an answer of the same through a case study of a larger class handwriting recognition problem. Here, we have studied recognition of handwritten Devanagari characters. In this study, we have implemented convolutional neural network (CNN) architectures of five different depths. We have also implemented additional neural architectures by adding two Bidirectional Long Short Term Memory (BLSTM) layers between the convolutional stack and the fully connected part of each of these five CNN networks. Simulations have been performed on two different databases of handwritten Devanagari characters consisting of 30408 and 36172 samples and a combined set consisting of 58451 samples. The recognition accuracy obtained in the best case improves significantly the existing state-of-the-art of this handwriting recognition problem. Also, further analysis of our simulation results provides an answer to the above question. Additionally, we have trained a BLSTM network alone using the Histogram of Oriented Gradient (HOG) features. Performance of this architecture failed to compete with the performance of CNN-BLSTM hybrid architecture.

Handwriting Recognition of Historical Documents with Few Labeled Data

Abstract: Historical documents present many challenges for offline handwriting recognition systems, among them, the segmentation and labeling steps. Carefully annotated text lines are needed to train an HTR system. In some scenarios, transcripts are only available at the paragraph level with no text-line information. In this work, we demonstrate how to train an HTR system with few labeled data. Specifically, we train a deep convolutional recurrent neural network (CRNN) system on only 10% of manually labeled text-line data from a dataset and propose an incremental training procedure that covers the rest of the data. Performance is further increased by augmenting the training set with specially crafted multi scale data. We also propose a model-based normalization scheme which considers the variability in the writing scale at the recognition phase. We apply this approach to the publicly available READ dataset. Our system achieved the second best result during the ICDAR2017 competition [1].

Handwritten Devanagari Character Classification using Deep Learning.

Abstract: Since past few years, deep neural networks, because of their outstanding performance, are getting highly used in computer vision and machine learning tasks such as regression, segmentation, classification, detection, pattern recognition etc. Recognition of handwritten Devanagari characters is challenging task, but Deep learning can be effectively used as a solution for various such problems. Person to person variations in writing style makes handwritten character recognition one of the most difficult tasks. In this Experiment, we successfully tried to classify handwritten Devanagari characters using transfer learning mechanism with the help of Alexnet. Alexnet, a convolutional neural network, is trained over a dataset of around 16870 samples of 22 consonants of Devanagari script which shows impressive results. The transfer learning helps to learn faster and better even if the data samples are less as compared with the training a CNN from scratch.

Bangla Handwritten Digit Recognition Using Deep CNN for Large and Unbiased Dataset

Abstract: Bangla handwritten digit recognition is a convenient starting point for building an OCR in the Bengali language. Lack of large and unbiased dataset, Bangla digit recognition was not standardized previously. But in this paper, a large and unbiased dataset known as NumtaDB is used for Bangla digit recognition. The challenges of the NumtaDB dataset are highly unprocessed and augmented images. So different kinds of preprocessing techniques are used for processing images and deep convolutional neural network (CNN) is used as the classification model in this paper. The deep convolutional neural network model has shown an excellent performance, securing the 13th position with 92.72% testing accuracy in the Bengali handwritten digit recognition challenge 2018 among 57 participating teams. A study of the network performance on the MNIST and EMNIST datasets were performed in order to bolster the analysis.

Transfer Learning for Handwriting Recognition on Historical Documents.

Abstract: In this work, we investigate handwriting recognition on new historical handwritten documents using transfer learning. Establishing a manual ground-truth of a new collection of handwritten documents is time consuming but needed to train and to test recognition systems. We want to implement a recognition system without performing this annotation step. Our research deals with transfer learning from heterogeneous datasets with a ground-truth and sharing common properties with a new dataset that has no ground-truth. The main difficulties of transfer learning lie in changes in the writing style, the vocabulary, and the named entities over centuries and datasets. In our experiment, we show how a CNN-BLSTM-CTC neural network behaves, for the task of transcribing handwritten titles of plays of the Italian Comedy, when trained on combinations of various datasets such as RIMES, Georges Washington, and Los Esposalles. We show that the choice of the training datasets and the merging methods are determinant to the results of the transfer learning task.

Learning Features for Writer Retrieval and Identification using Triplet CNNs

Abstract: This paper presents a method for writer retrieval and identification using a feature descriptor learned by a Convolutional Neural Network. Instead of using a network for classification, we propose the use of a triplet network that learns a similarity measure for image patches. Patches of the handwriting are extracted and mapped into an embedding where this similarity measure is defined by the L_2 distance. The triplet network is trained by maximizing the interclass distance, while minimizing the intraclass distance in this embedding. The image patches are encoded using the learned feature descriptor. By applying the Vector of Locally Aggregated Descriptors encoding to these features, we generate a feature vector for each document image. A detailed parameter evaluation is given which shows that this method achieves a mean average precision of 86.1% on the ICDAR 2013 writer identification dataset, but future work has to be done to improve the performance on historic datasets. In addition, the strategy for clustering the feature space is investigated.

Boosting the Deep Multidimensional Long-Short-Term Memory Network for Handwritten Recognition Systems

Abstract: One of the main challenges in the handwriting recognition area lies in identifying complete lines of handwritten text. In this paper, we propose a handwriting recognition system based on a deep multidimensional long-short-term memory (MDLSTM) network within a hybrid hidden Markov model framework. The MDLSTM architecture was elaborated to enhance the recognition performance and decrease the recognition time. Accordingly, we present modifications regarding the layers order and the number of pooling layers compared to a standard MDLSTM model. Since the results reported in the literature for deeper MDLSTM architectures relies on optimizing the network width with a fixed depth, we investigate the trade-off between both these properties to obtain an optimal topology. The system was evaluated with English handwritten text lines from the IAM database and the experiments demonstrated that the proposed MDLSTM architecture was able to maintain a robust recognition performance (around 3.6% CER and 10.5% WER) and present significant speedups, approximately 48% and 32% faster than the state-of-the-art MDLSTM optical model, regarding the learning and classification times, respectively. The full system including a decoder with linguistic knowledge presents competitive results with the state-of-the-art.

Convolutional Feature Learning and CNN Based HMM for Arabic Handwriting Recognition

Abstract: In this paper, we present a model CNN based HMM for Arabic handwriting word recognition. The HMM have proved a powerful to model the dynamics of handwriting. Meanwhile, the CNN have achieved impressive performance in many computer vision tasks, including handwritten characters recognition. In this model, the trainable classifier of CNN is replacing by the HMM classifier. CNN works as a generic feature extractor and HMM performs as a recognizer. The suggested system outperforms a basic HMM based on handcrafted features. Experiments have been conducted on the well-known IFN/ENIT database. The results obtained show the robustness of the proposed approach.

A comprehensive survey on word recognition for non-Indic and Indic scripts

Abstract: The term handwriting recognition is used to describe the capability of a computer system to transform human handwriting into machine processable text. Handwriting recognition has many applications in various fields such as bank-cheque processing, postal-address interpretation, document archiving, mail sorting and form processing in administration, insurance offices. A collection of different scripts is employed in writing languages throughout the world. Many researchers have done work for handwriting recognition of various non-Indic and Indic scripts from the most recent couple of years. But, only a limited number of systems are offered for word recognition for these scripts. This paper presents an extensive systematic survey of word recognition techniques. This survey of word recognition is classified broadly based on different scripts in which a word is written. Experimental evaluation of word recognition tools/techniques is presented in this paper. Different databases have been surveyed to evaluate the performance of techniques used to recognize words, and the achieved recognition accuracies have been reported. The efforts in two directions (non-Indic and Indic scripts) are reflected in this paper. We increased awareness of the potential benefits of word recognition techniques and identify the need to develop an efficient word recognition technique. Recommendations are also provided for future research. It is also observed that the research in this area is quietly thin and still more research is to be done, particularly in the case of word recognition of printed/handwritten documents in Indic scripts.

Fully Convolutional Networks for Handwriting Recognition

Abstract: Handwritten text recognition is challenging because of the virtually infinite ways a human can write the same message. Our fully convolutional handwriting model takes in a handwriting sample of unknown length and outputs an arbitrary stream of symbols. Our dual stream architecture uses both local and global context and mitigates the need for heavy preprocessing steps such as symbol alignment correction as well as complex post processing steps such as connectionist temporal classification, dictionary matching or language models. Using over 100 unique symbols, our model is agnostic to Latin-based languages, and is shown to be quite competitive with state of the art dictionary based methods on the popular IAM and RIMES datasets. When a dictionary is known, we further allow a probabilistic character error rate to correct errant word blocks. Finally, we introduce an attention based mechanism which can automatically target variants of handwriting, such as slant, stroke width, or noise.

MotionHacker: Motion sensor based eavesdropping on handwriting via smartwatch

Abstract: Motion sensors may impose the danger of hand-writing leakage if the smartwatch installs a malicious app By presenting a proof-of-concept system: MotionHacker, we show the usage of smartwatch app to record the motions and extract handwriting-specific features for machine learning based analysis MotionHacker is targeted on user-independent handwriting recognition and word-level estimation for the content of the victim's handwriting, where the only limit is that victim's handwriting follows print style Furthermore, the targeted handwriting is lowercase letters, which are more difficult than capital letters Our experimental results show that the average accuracy of word recognition is 328% for 5 victims writing two graphs from a novel and a research paper each When counting the word prediction results of top-5 items, the word recognition accuracy reaches 488% The results confirm the danger of handwriting content leakage from smartwatches' motion sensors

Towards Spotting and Recognition of Handwritten Words in Indic Scripts

Abstract: Handwriting recognition (HWR) in Indic scripts is a challenging problem due to the inherent subtleties in the scripts, cursive nature of the handwriting and similar shape of the characters. Lack of publicly available handwriting datasets in Indic scripts has affected the development of handwritten word recognizers, and made direct comparisons across different methods an impossible task in the field. In this paper, we propose a framework for annotating large scale of handwritten word images with ease and speed. We also release a new handwritten word dataset for Telugu, which is collected and annotated using the proposed framework. We also benchmark major Indic scripts such as Devanagari, Bangla and Telugu for the tasks of word spotting and handwriting recognition using state of the art deep neural architectures. Finally, we evaluate the proposed pipeline on RoyDB, a public dataset, and achieve significant reduction in error rates.

Recognition and Solution for Handwritten Equation Using Convolutional Neural Network

Abstract: Robust handwritten character recognition is a tricky job in the area of image processing. Among all the problem handwritten mathematical expression recognition is one of the complicated issue in the area of computer vision research. Segmentation and classification of specific character makes the task more difficult. In this paper a group of handwritten quadratic equation as well as a single quadratic equation are considered to recognize and make a solution for those equation. Horizontal compact projection analysis and combined connected component analysis methods are used for segmentation. For classification of specific character we apply Convolutional Neural Network. Each of the correct detection, character string operation is used for the solution of the equation. Finally the experimental results shows the great effectiveness of our proposed system.

Online-KHATT: An Open-Vocabulary Database for Arabic Online-Text Processing

Abstract: Objective: The main objective of this work is to build a comprehensive benchmarking database of online Arabic text. Part of this objective is the development of tools, techniques and procedures for online text collection, verification and transliteration. Additionally, we built a dataset for segmented online Arabic characters and ligatures with ground truth labeling and present classification results of online Arabic characters using DBN-based HMM.