The cloud heralds a new era of computing where application services are provided through the Internet. Cloud computing can enhance the computing capability of mobile systems, but is it the ultimate solution for extending such systems' battery lifetimes?

Cloud Computing for Mobile Users: Can Offloading Computation Save Energy?

Motivated by an availability gap for visual media, where images and videos are uploaded from mobile devices well after they are generated, we explore the selective, timely retrieval of media content from a collection of mobile devices. We envision this capability being driven by similarity-based queries posed to a cloud search front-end, which in turn dynamically retrieves media objects from mobile devices that best match the respective queries within a given time limit. Building upon a crowd-sensing framework, we have designed and implemented a system called MediaScope that provides this capability. MediaScope is an extensible framework that supports nearest-neighbor and other geometric queries on the feature space (e.g. clusters, spanners), and contains novel retrieval algorithms that attempt to maximize the retrieval of relevant information. From experiments on a prototype, MediaScope is shown to achieve near-optimal query completeness and low to moderate overhead on mobile devices.

MediaScope: selective on-demand media retrieval from mobile devices

We propose a novel model, called stroke sequence-dependent deep convolutional neural network (SSDCNN), which uses the stroke sequence information and eight-directional features of Chinese characters for online handwritten Chinese character recognition (OLHCCR). SSDCNN learns the representation of OLHCCs by incorporating the natural sequence information of the strokes. Furthermore, it naturally incorporates the eight-directional features. First, SSDCNN inputs the stroke sequence and transforms it into stacks of feature maps following the writing order of the strokes. Second, the fixed-length, stroke sequence-dependent representations of OLHCC are derived through convolutional, residual, and max-pooling operations. Third, the stroke sequence-dependent representation is combined with the eight-directional features via a number of fully connected neural network layers. Finally, the Chinese characters are recognized using a softmax classifier. The SSDCNN is trained in two stages: 1) the whole architecture is pretrained using the training data until the performance converges to an acceptable degree. 2) The stroke sequence-dependent representation is combined with the eight-directional features by a fully connected neural network and a softmax layer for further training. The model was experimentally evaluated on the OLHCCR competition tasks of International Conference on Document Analysis and Recognition (ICDAR) 2013. The recognition error was a maximum 58.28% lower in SSDCNN than in a model using the eight-directional features alone (5.13% versus 2.14%). Owing to its high accuracy (97.86%), the proposed SSDCNN reduced the recognition error by approximately 18.0% as compared with that of the winning system in the ICDAR 2013 competition. SSDCNN integrated with an adaptation mechanism, called the SSDCNN+Adapt model, and reached a new state-of-the-art (SOTA) standard with an accuracy of 97.94%. The SSDCNN exploits the stroke sequence information to learn high-quality OLHCC representations. Moreover, the learned representation and the classical eight-directional features complement each other within the SSDCNN architecture.

Stroke Sequence-Dependent Deep Convolutional Neural Network for Online Handwritten Chinese Character Recognition

A computer image search system for rapid retrieval of media files images corresponding to a sequence of input display commands, the system comprising: a programmable data processor operating under the control of a program to convert the display commands into data entries in an array of multi-bit data characters and words, each entry of the array corresponding to a set of descriptions of the image to be displayed; and a scanned-raster display device for generating illuminated points on a display surface in response to applied data signals causing the programmable data processor to perform a set of operations.

Lexicon based systems and methods for intelligent media search

In this paper, we propose a novel model, named Stroke Sequence-dependent Deep Convolutional Neural Network (SSDCNN), using the stroke sequence information and eight-directional features for Online Handwritten Chinese Character Recognition (OLHCCR). On one hand, SSDCNN can learn the representation of Online Handwritten Chinese Character (OLHCC) by incorporating the natural sequence information of the strokes. On the other hand, SSDCNN can incorporate eight-directional features in a natural way. In order to train SSDCNN, we divide the process of training into two stages: 1) The training data is used to pre-train the whole architecture until the performance tends to converge. 2) Fully-connected neural network which is used to combine the stroke sequence-dependent representation with eight-directional features and softmax layer are further trained. Experiments were conducted on the OLHCCR competition tasks of ICDAR 2013. Results show that, SSDCNN can reduce the recognition error by 50\% (5.13\% vs 2.56\%) compared to the model which only use eight-directional features. The proposed SSDCNN achieves 97.44\% accuracy which reduces the recognition error by about 1.9\% compared with the best submitted system on ICDAR2013 competition. These results indicate that SSDCNN can exploit the stroke sequence information to learn high-quality representation of OLHCC. It also shows that the learnt representation and the classical eight-directional features complement each other within the SSDCNN architecture.

The development of mobile-based image retrieval system is required for the efficient management about various image data with fast growth of mobile devices. The resource of mobile phone is a bit limited due to low memory capacity and low CPU performance. Hence, the system which is applicable on mobile phone should overcome the problem of limited resource. In this paper, we propose the fast feature extraction method which is applicable on mobile environment. The proposed algorithm is based on the lookup table which converts RGB color space to one of 36 indices on HSV color space. By the lookup table-based feature extraction, we extract three histograms which is about color distribution and the location of the distribution. Then, we load the proposed algorithm to mobile phone. Our system can query by three different queries: query-by-image, query-by-color, and query-by-blob. The experimental result shows that the proposed method is a fast and stable engine enough to utilize on mobile platform.

A fast image retrieval system using index lookup table on mobile device

In previous sketch recognition systems, curve has been fitted by a bit heuristic method. In this paper, we solved the problem by finding the optimal parameter of quadratic Bezier curve and utilize the error minimization between an input curve and a fitting curve by using iterative error minimization. First, we interpolated the input curve to compute the distance because the input curve consists of a set of sparse points. Then, we define the objective function. To find the optimal parameter, we assume that the initial parameter is known. Then, we derive the gradient vector with respect to the current parameter, and the parameter is updated by the gradient vector. This two steps are repeated until the error is not reduced. From the experiment, the average approximation error of the proposed algorithm was 0.946433 about 1400 synthesized curves, and this result demonstrates that the given curve can be fitted very closely by using the proposed fitting algorithm.

Curve fitting algorithm using iterative error minimization for sketch beautification

We propose a powerful shape representation to recognize sketches drawn on a pen-based input device. The proposed method is robust to the sketching order by using the combination of distance map and direction histogram. A distance map created after normalizing a freehand sketch represents a spatial feature of shape regardless of the writing order. Moreover, a distance map which acts a spatial feature is more robust to shape variation than chamfer distance. Direction histogram is also able to extract a directional feature unrelated to the drawing order by using the alignment of the spatial location between two neighboring points of the stroke. The combination of these two features represents rich information to recognize an input sketch. The experiment result demonstrates the superiority of the proposed method more than previous works. It shows 96% recognition performance for the experimental database, which consists of 28 freehand sketches and 10 on-line handwritten digits.

https://www3.cs.stonybrook.edu/~kyun/papers/kiwon_sspr2008.pdf

Shape Representation Robust to the Sketching Order Using Distance Map and Direction Histogram

Freehand sketching is a very efficient means for us to communicate each other. As table PC is widely popularized, the research about sketch recognition became one of important research issue. To recognize sketch, the feature point should be extracted and then each feature point is analyzed as line or curve. However, most of feature extraction algorithms suffers from noise which is occurred from the bad drawing sketch. In this paper, we propose the feature extraction algorithm robust to noise. The proposed algorithm consists of three cascade steps: candidate feature point extraction, noise reduction, and hook elimination. At the candidate feature point extraction step, the feature points is selected among input points. Then, in second step, we reduce the noise which is occurred from the previous step by using noise reduction rule based on inner product between two neighbor vectors. Finally, the hook, which can not be eliminated from two previous steps, is eliminated by the proposed hook elimination method. The experimental result shows that the average approximation error is less than 1 about 1004 line-curve hybrid shapes, and the proposed algorithm is the good feature methods.

Feature extraction method based on cascade noise elimination for sketch recognition

The original Active Appearance Model(AAM) uses the mean matrix of gradient matrixes instead of a gradient matrix which should be recomputed with respect to a varying parameter at a fitting phase. By this property, the original AAM can guarantee a fast fitting speed because it avoids computation of a gradient matrix of which a computation complexity is high. However, the fixed gradient matrix is not a good choice when the distribution of a training database is nonlinear because the mean can not represent the variation of a training database. To overcome this problem, this paper proposes multi subspaces AAM. First, we divide a training database into multi subspaces along the illumination direction, and build the independent AAM for each subspace. At a fitting phase, we adaptively choose a subspace well fit to a target image. However, the parameter update problem is occurred because a subspace can be changed during a fitting phase. To solve this problem, we propose a linear transform matrix on an eigenspace. In experiments, we apply the proposed method to Yale Face Database B and demonstrate that the method is robust for facial images under various illuminations.

Junyeong Yang

Papers

A fast image retrieval system using index lookup table on mobile device

Curve fitting algorithm using iterative error minimization for sketch beautification

Shape Representation Robust to the Sketching Order Using Distance Map and Direction Histogram

Feature extraction method based on cascade noise elimination for sketch recognition

Multi Subspaces Active Appearance Models