In this paper, we show how motion data collected with a smartwatch can be used to infer text written on a whiteboard. All data is gathered by a commercial, off the shelf smartwatch, without any additional hardware. We obtain single letters recognition rates of 94% in the first guess. Using three guesses, we recognize 99% of the letters. This result is achieved by transforming raw sensor measurements into a coordinate system reflecting the orientation of the whiteboard. Therefore, training data recorded on one whiteboard can be used on another even if the user is writing at different heights or body postures. We show that the built in microphone picks up the sounds caused by the pen which can help to segment the input into individual letters.

Recognizing text using motion data from a smartwatch

The cultural and regional diversity across the world and specifically in India has given birth to a large number of writing systems and scripts having a variety of character sets. For scripts having a larger character set, just a simple keyboard with limited character set is not the optimal way for providing inputs to the computer. Variations in individual handwriting due to mood swings, changes in medium of writing, changes in writing styles, etc. pose a challenge before the character recognition (CR) research community. Similar kinds of symbols in various scripts and languages act as a big barrier in multilingual CR. Lack of benchmark results and corpora for multilingual CR hinder the research in multilingual CR. There have been only a limited number of articles for optimal combination of features and classifiers to process multilingual data. Multilingual CR has least explored the Indic scripts. This paper presents a detailed review and analysis of the work done in multilingual online as well as offline CR for Indic and non-Indic scripts. The paper mainly contributes in two ways: Firstly, it provides a clear perspective about various phases of monolingual and multilingual CR; and secondly, identifies the major deficiencies in monolingual and multilingual CR for printed and handwritten text. It contributes by giving an in-depth view of work done at each phase including data acquisition, pre-processing, segmentation, feature extraction, recognition and post-processing of CR. Issues to be resolved at each phase have also been elaborated. The recent work done using Deep and Shallow architectures has been analysed. Tools used for these architectures have been compared to highlight their pros and cons. The present work also suggests how further research can be conducted in the field of monolingual and multilingual CR. The problems such as CR in hybrid documents, identifying more reliable features, resolving issues of similar characters, identifying optimal combination strategies for deep and shallow architectures, etc. need to be tackled in future research.

A survey of mono- and multi-lingual character recognition using deep and shallow architectures: indic and non-indic scripts

Methods based on learning from examples that have been widely applied to character recognition are considered as one of the classification methods. This class of methods includes statistical methods, artificial neural network, support vector machines, multiple classifier combinations, etc. among these, artificial neural network are quite popular and have gained more attention. The paper presents a survey on ANNs applied for recognizing characters and the features applied as inputs. The principal objective of this paper is to help researchers attempt to apply ANN for character recognition and analysis of the networks and the features extracted to improve classification accuracy.

Character Recognition using Approaches of Artificial Neural Network: A Review

Monitoring of ecosystems entails the evaluation of contributing factors by the expert ecologist. The aim of this study is to examine to what extent the quantitative variables, calculated solely by ...

https://www.tandfonline.com/doi/pdf/10.5721/EuJRS20164953

Multi-modal knowledge base generation from very high resolution satellite imagery for habitat mapping

Japanese is one of the languages that use different letters / alphabets in international standard writing. SMA Negeri 1 Garum is one of the SMA that uses Japanese as one of the compulsory subjects. Constraints experienced by high school students 1 Garum is on learning the procedure of writing the letters Hiragana and Katakana. Based on the obstacles experienced, the researcher aims to create a learning media application that focuses more to help students in learning the procedures of writing hururf Hiragana and Katakana. In this research, the researcher succeeded in implementing API Gesture and API Voice Recognition in making the application. API Gesture successfully implemented in letter writing feature while API Voice Recognition successfully implemented on feature of guessing picture of letters, so with the success of this application make students can practice to write and guess the letters Hiragana and Katakana by using this application. The results of this study are expected to help students who are still difficult to learn the writing and pronunciation of the letters Hiragana and Katakana, in addition this research is expected to develop the results of previous studies in the manufacture of learning media applications..

Aplikasi media pembelajaran huruf hiragana dan katakana menggunakan api gesture dan api voice recognition berbasis android (studi kasus : sma negeri 1 garum kabupaten blitar)

The wide usage of touch-screen based mobile devices has led to a large volume of the users preferring touch-based interaction with the machine, as opposed to traditional input via keyboards/mice. To exploit this, we focus on the Android platform to design a personalized handwriting recognition system that is acceptably fast, light-weight, possessing a user-friendly interface with minimally-intrusive correction and auto-personalization mechanisms. Since cursive writing on smaller screens is not usual, here we study non-cursive handwriting only. The recognition is done at character level using nearest-neighbor matching to a small, automatically user-adaptive and dynamically updating library of character-class template gestures.

Building a Personal Handwriting Recognizer on an Android Device

Here, we present our recent attempt to develop a lightweight handwriting recognizer suitable for resource constrained handheld devices. Such an application requires real-time recognition of handwritten characters produced on their touchscreens. The proposed approach is well suited for minimal user-lag on devices having only limited computing power in sharp contrast to standard laptops or desktop computers. Moreover, the approach is user-adaptive in the sense that it can adapt through user corrections to wrong predictions. With an increasing number of interactive corrections by the user, the recognition accuracy improves significantly. An input stroke is first re-sampled generating a fixed small number of sample points such that at most two critical points (points corresponding to high curvature) are preserved. We use their x- and y-coordinates as the feature vector and do not compute any other high-level feature vector. The squared Mahalanobis distance is used to identify each stroke of the input sample as one of several stroke categories pre-determined based on a large pool of training samples. The inverted covariance matrix and mean vector for a stroke class that are required for computing the Mahalanobis distance are pre-calculated and stored as Serialized Objects on the SD card of the device. A Look-Up Table (LUT) of stroke combinations as keys and corresponding character class as values is used for the final Unicode character output. In case of an incorrect character output, user corrections are used to automatically update the LUT adapting to the user's particular handwriting style.

Lightweight user-adaptive handwriting recognizer for resource constrained handheld devices

In this article, a novel lightweight user-adaptive online handwriting recognition scheme has been presented. The present recognition approach is stroke order free. It is based on prior identification of the set of various strokes of different shapes used in writing the characters of the underlying alphabet utilizing a representative sample database. In this approach a very small number of prototypes of each stroke shape is used along with certain weighted DTW distance based nearest neighbour classifier to recognize the strokes in the input character. Individual characters are identified using a look-up table (LUT) each row of which corresponds to one character composed of a distinct set of stroke shapes. This LUT is formed using the representative sample set of the underlying character set. If a stroke does not find a close match in the training set or if the set of strokes for an input character does not find a corresponding entry in the LUT, user adaptation takes place using a modified Learning Vector Quantization (LVQ) method. The proposed scheme has been implemented in an Android-based Bangla handwriting recognizer [1] for handheld devices and its recognition performance is encouraging.

http://ieeexplore.ieee.org/iel7/6979380/6980977/06981029.pdf

A. Roy Chowdhury

Papers

Building a Personal Handwriting Recognizer on an Android Device

Lightweight user-adaptive handwriting recognizer for resource constrained handheld devices

Stroke Level User-Adaptation for Stroke Order Free Online Handwriting Recognition