Annual review of condensed matter physics

The goal of this work is text spotting in natural images. This is divided into two sequential tasks: detecting words regions in the image, and recognizing the words within these regions. We make the following contributions: first, we develop a Convolutional Neural Network (CNN) classifier that can be used for both tasks. The CNN has a novel architecture that enables efficient feature sharing (by using a number of layers in common) for text detection, character case-sensitive and insensitive classification, and bigram classification. It exceeds the state-of-the-art performance for all of these. Second, we make a number of technical changes over the traditional CNN architectures, including no downsampling for a per-pixel sliding window, and multi-mode learning with a mixture of linear models (maxout). Third, we have a method of automated data mining of Flickr, that generates word and character level annotations. Finally, these components are used together to form an end-to-end, state-of-the-art text spotting system. We evaluate the text-spotting system on two standard benchmarks, the ICDAR Robust Reading data set and the Street View Text data set, and demonstrate improvements over the state-of-the-art on multiple measures.

/pdf/deep-features-for-text-spotting-befcynr15n.pdf

Deep Features for Text Spotting

Music recommender systems (MRSs) have experienced a boom in recent years, thanks to the emergence and success of online streaming services, which nowadays make available almost all music in the world at the user’s fingertip. While today’s MRSs considerably help users to find interesting music in these huge catalogs, MRS research is still facing substantial challenges. In particular when it comes to build, incorporate, and evaluate recommendation strategies that integrate information beyond simple user–item interactions or content-based descriptors, but dig deep into the very essence of listener needs, preferences, and intentions, MRS research becomes a big endeavor and related publications quite sparse. The purpose of this trends and survey article is twofold. We first identify and shed light on what we believe are the most pressing challenges MRS research is facing, from both academic and industry perspectives. We review the state of the art toward solving these challenges and discuss its limitations. Second, we detail possible future directions and visions we contemplate for the further evolution of the field. The article should therefore serve two purposes: giving the interested reader an overview of current challenges in MRS research and providing guidance for young researchers by identifying interesting, yet under-researched, directions in the field.

/pdf/current-challenges-and-visions-in-music-recommender-systems-5djs7nnth9.pdf

Current challenges and visions in music recommender systems research

A range of applications, from predicting the spread of human and electronic viruses to city planning and resource management in mobile communications, depend on our ability to foresee the whereabouts and mobility of individuals, raising a fundamental question: To what degree is human behavior predictable? Here we explore the limits of predictability in human dynamics by studying the mobility patterns of anonymized mobile phone users. By measuring the entropy of each individual's trajectory, we find a 93% potential predictability in user mobility across the whole user base. Despite the significant differences in the travel patterns, we find a remarkable lack of variability in predictability, which is largely independent of the distance users cover on a regular basis.

http://absimage.aps.org/image/MAR10/MWS_MAR10-2009-007155.pdf

Limits of predictability in human mobility

The standard approach to recognizing text in images consists in first classifying local image regions into candidate characters and then combining them with high-level word models such as conditional random fields. This paper explores a new paradigm that departs from this bottom-up view. We propose to embed word labels and word images into a common Euclidean space. Given a word image to be recognized, the text recognition problem is cast as one of retrieval: find the closest word label in this space. This common space is learned using the Structured SVM framework by enforcing matching label-image pairs to be closer than non-matching pairs. This method presents several advantages: it does not require ad-hoc or costly pre-/post-processing operations, it can build on top of any state-of-the-art image descriptor (Fisher vectors in our case), it allows for the recognition of never-seen-before words (zero-shot recognition) and the recognition process is simple and efficient, as it amounts to a nearest neighbor search. Experiments are performed on challenging datasets of license plates and scene text. The main conclusion of the paper is that with such a frugal approach it is possible to obtain results which are competitive with standard bottom-up approaches, thus establishing label embedding as an interesting and simple to compute baseline for text recognition.

Label Embedding: A Frugal Baseline for Text Recognition

In this paper we present a novel recognition approach that results in a 15% decrease in word error rate on heavily degraded Indian language document images. OCRs have considerably good performance on good quality documents, but fail easily in presence of degradations. Also, classical OCR approaches perform poorly over complex scripts such as those for Indian languages. We address these issues by proposing to recognize character n-gram images, which are basically groupings of consecutive character/component segments. Our approach is unique, since we use the character n-grams as a primitive for recognition rather than for post processing. By exploiting the additional context present in the character n-gram images, we enable better disambiguation between confusing characters in the recognition phase. The labels obtained from recognizing the constituent n-grams are then fused to obtain a label for the word that emitted them. Our method is inherently robust to degradations such as cuts and merges which are common in digital libraries of scanned documents. We also present a reliable and scalable scheme for recognizing character n-gram images. Tests on English and Malayalam document images show considerable improvement in recognition in the case of heavily degraded documents.

/pdf/robust-recognition-of-degraded-documents-using-character-n-y2h6oh65cn.pdf

Robust Recognition of Degraded Documents Using Character N-Grams

This work addresses the problem of melodic motif spotting, given a query, in Carnatic music. Melody in Carnatic music is based on the concept of raga. Melodic motifs are signature phrases which give a raga its identity. They are also the fundamental units that enable extempore elaborations of a raga. In this paper, an attempt is made to spot typical melodic motifs of a raga queried in a musical piece using a two pass dynamic programming approach, with pitch as the basic feature. In the first pass, the rough longest common subsequence (RLCS) matching is performed between the saddle points of the pitch contours of the reference motif and the musical piece. These saddle points corresponding to quasi-stationary points of the motifs, are relevant entities of the raga. Multiple sequences are identified in this step, not all of which correspond to the the motif that is queried. To reduce the false alarms, in the second pass a fine search using RLCS is performed between the continuous pitch contours of the reference motif and the subsequences obtained in the first pass. The proposed methodology is validated by testing on Alapanas of 20 different musicians.

https://compmusic.upf.edu/system/files/static_files/Ishwar-ISMIR-2013.pdf

Motif Spotting in an Alapana in Carnatic Music.

There are at least 100 rāgas that are regularly performed in Carnatic music concerts. The audience determines the identity of rāgas within a few seconds of listening to an item. Most of the audience consists of people who are only avid listeners and not performers. In this paper, an attempt is made to mimic the listener. A rāga verification framework is therefore suggested. The rāga verification system assumes that a specific rāga is claimed based on similarity of movements and motivic patterns. The system then checks whether this claimed rāga is correct. For every rāga, a set of cohorts are chosen. A rāga and its cohorts are represented using pallavi lines of compositions. A novel approach for matching, called Longest Common Segment Set (LCSS), is introduced. The LCSS scores for a rāga are then normalized with respect to its cohorts in two different ways. The resulting systems and a baseline system are compared for two partitionings of a dataset. A dataset of 30 rāgas from Charsur Foundation 1 is used for analysis. An equal error rate (EER) of 12% is obtained.

/pdf/raga-verification-in-carnatic-music-using-longest-common-2gn017vm45.pdf

Raga Verification in Carnatic Music Using Longest Common Segment Set.

Typical motifs of a rāga can be found in the various songs that are composed in the same rāga by different composers. The compositions in Carnatic music have a definite structure, the one commonly seen being pallavi, anupallavi and charanam. The tala is also fixed for every song. Taking lines corresponding to one or more cycles of the pallavi, anupallavi and charanam as one-liners, one-liners across different songs are compared using a dynamic programming based algorithm. The density of match between the one-liners and normalized cost along-with a new measure, which uses the stationary points in the pitch contour to reduce the false alarms, are used to determine and locate the matched pattern. The typical motifs of a rāga are then filtered using compositions of various rāgas. Motifs are considered typical if they are present in the compositions of the given rāga and are not found in compositions of other rāgas.

Discovering Typical Motifs of a Raga from One-Liners of Songs in Carnatic Music.

Perception necessitates interaction among neuronal ensembles, the dynamics of which can be conceptualized as the emergent behavior of coupled dynamical systems. Here, we propose a detailed neurobiologically realistic model that captures the neural mechanisms of inter-individual variability observed in cross-modal speech perception. From raw EEG signals recorded from human participants when they were presented with speech vocalizations of McGurk-incongruent and congruent audio-visual (AV) stimuli, we computed the global coherence metric to capture the neural variability of large-scale networks. We identified that participants' McGurk susceptibility was negatively correlated to their alpha band global coherence. The proposed biophysical model conceptualized the global coherence dynamics emerge from coupling between the interacting neural masses-representing the sensory-specific auditory/visual areas and modality nonspecific associative/integrative regions. Subsequently, we could predict that an extremely weak direct AV coupling results in a decrease in alpha band global coherence-mimicking the cortical dynamics of participants with higher McGurk susceptibility. Source connectivity analysis also showed decreased connectivity between sensory-specific regions in participants more susceptible to McGurk effect, thus establishing an empirical validation to the prediction. Overall, our study provides an outline to link variability in structural and functional connectivity metrics to variability of performance that can be useful for several perception and action task paradigms.

Shrey Dutta

Papers

Robust Recognition of Degraded Documents Using Character N-Grams

Motif Spotting in an Alapana in Carnatic Music.

Raga Verification in Carnatic Music Using Longest Common Segment Set.

Discovering Typical Motifs of a Raga from One-Liners of Songs in Carnatic Music.

Biophysical mechanisms governing large-scale brain network dynamics underlying individual-specific variability of perception.