Iconic and multi-stroke gesture recognition

Socio-technical PSS development to improve functionality and usability—Sketch planning using a Maptable

Digital ink features drive recognition engines. Intuitively, we understand that particular features are of more value for some problems than others. Likewise, inclusion of poor features may be detrimental to recognition success. Many different ink features have been proposed for ink recognition, and most work well for the context that they are employed. However given a new problem it is not clear which of the already defined features will be most useful. We have assembled and categorized a comprehensive feature library and use this with attribute selection algorithms to choose the best features for a specified problem. To verify the effectiveness of this approach the selected features are used to train a Rubine's recognizer. We show that a set of complementary features is most effective: poor features adversely affect recognition as do two or more aliases of good features. We have composed a variant of a Rubine recognizer for 3 different datasets and compared these with the Rubine's original features, a variant on this InkRubine and $1. The results show that feature selection can significantly improve recognition rates with this simple algorithm thus verifying our hypothesis that the right combination of features for a problem is one key to recognition success.

/pdf/the-power-of-automatic-feature-selection-rubine-on-steroids-40nwatomct.pdf

The power of automatic feature selection: Rubine on steroids

Freehand sketching is widely regarded as an efficient and natural way for interaction between computers and humans. We present a robust computerized scheme to automatically segment freehand sketches into a series of components with specific geometric meaning regardless of whether these are generated online or offline. This task is a necessary first step toward sketch understanding. By exploiting the interpolation/extrapolation characteristic of radial basis functions (RBFs), a greedy algorithm consisting of forward and backward operations is proposed for finding the minimum set of segmentation points that can be used to reconstruct with high fitting accuracy freehand sketches in the form of implicit functions. To obtain segmentation points, a simple angle-based rule is used to remove ''bridging'' points that provide a smooth transition between consecutive sketch components. Feasibility of the proposed algorithm is demonstrated by a preliminary performance assessment study using ten computer generated drawings. These experiments show that in this dataset sensitivity of the segmentation was higher than 97.5% with a false positive (FP) rate of approximately 25%. The majority of false positive identifications are located on arc regions where a larger number of segmentation points are needed for reconstruction purposes. The primary contribution of this algorithm is that it transforms an ambiguous problem, namely, freehand sketch segmentation, into an implicit function fitting operation. Therefore, this proposed approach has several advantages, including independence of the actual sketching activity, and the ability for a satisfactory detection of the transition point between a line and an arc or between two arcs.

/pdf/automated-freehand-sketch-segmentation-using-radial-basis-4175f070ji.pdf

Automated freehand sketch segmentation using radial basis functions

In the Unified Modeling Language (UML) models are constructed graphically, by drawing diagrams. However, it is not always easy to manipulate diagrams using the CASE-tools of today. Typically, functionality and information are hidden behind complex menu or dialog hierarchies, diminishing the usability of the tools. Speech recognition can be used to promote usability of an existing tool as a complementary user interface that allows simultaneous use of other interfaces. In this paper we present an approach to develop speech interfaces to UML tools. We also show that UML is a favorable target to speech recognition and that speech recognition is applicable and mature enough to be used to enhance usage of UML tools. To support our claims, we present a spoken language created for editing UML models, a prototype of a speech control system integrated with Rational Rose, and the results of the early user tests on the system and the language.

Adding speech recognition support to UML tools

An experimental comparison of machine learning for adaptive sketch recognition

Sketching is a natural and effective way to capture designer's brainstorms ideas. This paper presents a sketch-based graphics input system for conceptual design (SketchGIS), which is mainly based on online graphic recognition and dynamic user modeling. Online graphic recognition can discover latent primitive shapes among user-drawn strokes and show the regularized shape on the screen immediately based on spatial-relation graph (SRG) representation and conditioned partial matching by shape preparing, feature recognition, shape filtering and regularization. Dynamic user modeling method is used to recognize and predict sketchy composite shapes before they are completely drawn for user adaptation, which is based on the SVM-based incremental active learning method and the incremental decision tree induction method. Experiments show that the proposed method in this paper can yield good recognition precision, and the prototype system proposed provides fine user interaction effect, especially for UML diagramming.

A freehand sketchy graphic input system: SketchGIS

This paper presents briefly an incremental learning method based on SVM for online sketchy shape recognition. It can collect all classified results corrected by user and select some important samples as the retraining data according to their distance to the hyper-plane of the SVM-classifier. The classifier can then do incremental learning quickly on the newly added samples, and the retrained classifier can be adaptive to the user's drawing styles. Experiment shows the effectiveness of the proposed method.

An incremental learning method based on SVM for online sketchy shape recognition

This paper presents a method of parameterization for online freehand drawing objects based on a piecewise cubic Bezier curve approximation. The target is to represent sketches in a compact format within a certain error tolerance with lower computation to be practically adaptable for the online graphics input. A set of user's intended breakpoints in digital ink is firstly produced in terms of pen speed and local curvatures. Each of strokes of a skechy shape is then parameterized by the optimization of piecewise Bezier curve approximation to minimize the fitting error between stroke path and the curve. The experimental results show both effective and efficient for a wide range of drawing graphic objects.

Sketch parameterization using curve approximation

This paper presents a novel approach for online multi-strokes composite sketchy shape recognition based on Bayesian Networks. By means of the definition of a double-level Bayesian networks, a classifier is designed to model the intrinsic temporal orders among the strokes effectively, where a sketchy shape is modeled with the relationships not only between a stroke and its neighbouring strokes, but also between a stroke and all of its subsequence.. The drawing-style tree is then adopted to capture the users' accustomed drawing styles and simplify the training and recognition of Bayesian network classifier. The experiments prove both effectiveness and efficiency of the proposed method.

Lisha Zhang

Papers

An experimental comparison of machine learning for adaptive sketch recognition

A freehand sketchy graphic input system: SketchGIS

An incremental learning method based on SVM for online sketchy shape recognition

Sketch parameterization using curve approximation

Online composite sketchy shape recognition based on bayesian networks