Showing papers by "Brno University of Technology published in 2013"

Efficient Estimation of Word Representations in Vector Space

Abstract: We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.

Efficient Estimation of Word Representations in Vector Space

Abstract: We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.

Linguistic Regularities in Continuous Space Word Representations

Abstract: Continuous space language models have recently demonstrated outstanding results across a variety of tasks. In this paper, we examine the vector-space word representations that are implicitly learned by the input-layer weights. We find that these representations are surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset. This allows vector-oriented reasoning based on the offsets between words. For example, the male/female relationship is automatically learned, and with the induced vector representations, “King Man + Woman” results in a vector very close to “Queen.” We demonstrate that the word vectors capture syntactic regularities by means of syntactic analogy questions (provided with this paper), and are able to correctly answer almost 40% of the questions. We demonstrate that the word vectors capture semantic regularities by using the vector offset method to answer SemEval-2012 Task 2 questions. Remarkably, this method outperforms the best previous systems.

On the difficulty of training recurrent neural networks

Abstract: There are two widely known issues with properly training recurrent neural networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994). In this paper we attempt to improve the understanding of the underlying issues by exploring these problems from an analytical, a geometric and a dynamical systems perspective. Our analysis is used to justify a simple yet effective solution. We propose a gradient norm clipping strategy to deal with exploding gradients and a soft constraint for the vanishing gradients problem. We validate empirically our hypothesis and proposed solutions in the experimental section.

Sequence-discriminative training of deep neural networks

Abstract: Sequence-discriminative training of deep neural networks (DNNs) is investigated on a standard 300 hour American En- glish conversational telephone speech task. Different sequence- discriminative criteria — maximum mutual information (MMI), minimum phone error (MPE), state-level minimum Bayes risk (sMBR), and boosted MMI — are compared. Two different heuristics are investigated to improve the performance of the DNNs trained using sequence-based criteria — lattices are re- generated after the first iteration of training; and, for MMI and BMMI, the frames where the numerator and denominator hy- potheses are disjoint are removed from the gradient compu- tation. Starting from a competitive DNN baseline trained us- ing cross-entropy, different sequence-discriminative criteria are shown to lower word error rates by 7-9% relative, on aver- age. Little difference is noticed between the different sequence- based criteria that are investigated. The experiments are done using the open-source Kaldi toolkit, which makes it possible for the wider community to reproduce these results. Index Terms: speech recognition, deep learning, sequence- criterion training, neural networks, reproducible research

Retinal vessel segmentation by improved matched filtering: evaluation on a new high-resolution fundus image database

Abstract: Automatic assessment of retinal vessels plays an important role in the diagnosis of various eye, as well as systemic diseases. A public screening is highly desirable for prompt and effective treatment, since such diseases need to be diagnosed at an early stage. Automated and accurate segmentation of the retinal blood vessel tree is one of the challenging tasks in the computer-aided analysis of fundus images today. We improve the concept of matched filtering, and propose a novel and accurate method for segmenting retinal vessels. Our goal is to be able to segment blood vessels with varying vessel diameters in high-resolution colour fundus images. All recent authors compare their vessel segmentation results to each other using only low-resolution retinal image databases. Consequently, we provide a new publicly available high-resolution fundus image database of healthy and pathological retinas. Our performance evaluation shows that the proposed blood vessel segmentation approach is at least comparable with recent state-of-the-art methods. It outperforms most of them with an accuracy of 95% evaluated on the new database.

Improved feature processing for Deep Neural Networks

Abstract: In this paper, we investigate alternative ways of processing MFCC-based features to use as the input to Deep Neural Networks (DNNs). Our baseline is a conventional feature pipeline that involves splicing the 13-dimensional front-end MFCCs across 9 frames, followed by applying LDA to reduce the dimension to 40 and then further decorrelation using MLLT. Confirming the results of other groups, we show that speaker adaptation applied on the top of these features using feature-space MLLR is helpful. The fact that the number of parameters of a DNN is not strongly sensitive to the input feature dimension (unlike GMM-based systems) motivated us to investigate ways to increase the dimension of the features. In this paper, we investigate several approaches to derive higher-dimensional features and verify their performance with DNN. Our best result is obtained from splicing our baseline 40-dimensional speaker adapted features again across 9 frames, followed by reducing the dimension to 200 or 300 using another LDA. Our final result is about 3% absolute better than our best GMM system, which is a discriminatively trained model.

Synthesis of surfactant-free electrostatically stabilized gold nanoparticles by plasma-induced liquid chemistry

Abstract: Plasma-induced non-equilibrium liquid chemistry is used to synthesize gold nanoparticles (AuNPs) without using any reducing or capping agents. The morphology and optical properties of the synthesized AuNPs are characterized by transmission electron microscopy (TEM) and ultraviolet‐visible spectroscopy. Plasma processing parameters affect the particle shape and size and the rate of the AuNP synthesis process. Particles of different shapes (e.g. spherical, triangular, hexagonal, pentagonal, etc) are synthesized in aqueous solutions. In particular, the size of the AuNPs can be tuned from 5 nm to several hundred nanometres by varying the initial gold precursor (HAuCl4) concentration from 2:5 M to 1 mM. In order to reveal details of the basic plasma‐liquid interactions that lead to AuNP synthesis, we have measured the solution pH, conductivity and hydrogen peroxide (H2O2) concentration of the liquid after plasma processing, and conclude that H2O2 plays the role of the reducing agent which converts Au C3 ions to Au 0 atoms, leading to nucleation growth of the AuNPs. S Online supplementary data available from stacks.iop.org/Nano/24/245604/mmedia (Some figures may appear in colour only in the online journal)

Automatic no-reference quality assessment for retinal fundus images using vessel segmentation

Abstract: Fundus imaging is the most commonly used modality to collect information about the human eye background. Objective and quantitative assessment of quality for the acquired images is essential for manual, computer-aided and fully automatic diagnosis. In this paper, we present a no-reference quality metric to quantify image noise and blur and its application to fundus image quality assessment. The proposed metric takes the vessel tree visible on the retina as guidance to determine an image quality score. In our experiments, the performance of this approach is demonstrated by correlation analysis with the established full-reference metrics peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM). We found a Spearman rank correlation for PSNR and SSIM of 0.89 and 0.91. For real data, our metric correlates reasonable to a human observer, indicating high agreement to human visual perception.

Semi-supervised training of Deep Neural Networks

Abstract: In this paper we search for an optimal strategy for semi-supervised Deep Neural Network (DNN) training. We assume that a small part of the data is transcribed, while the majority of the data is untranscribed. We explore self-training strategies with data selection based on both the utterance-level and frame-level confidences. Further on, we study the interactions between semi-supervised frame-discriminative training and sequence-discriminative sMBR training. We found it beneficial to reduce the disproportion in amounts of transcribed and untranscribed data by including the transcribed data several times, as well as to do a frame-selection based on per-frame confidences derived from confusion in a lattice. For the experiments, we used the Limited language pack condition for the Surprise language task (Vietnamese) from the IARPA Babel program. The absolute Word Error Rate (WER) improvement for frame cross-entropy training is 2.2%, this corresponds to WER recovery of 36% when compared to the identical system, where the DNN is built on the fully transcribed data.

AC Drive Observability Analysis

Abstract: AC induction motors and permanent magnet synchronous drives became very popular for motion control applications due to their simple and reliable construction. Sensorless drive control is required in many applications to reduce drive production costs. While many approaches to magnetic flux, rotor speed, or other quantities needed to control electrical machine were proposed, conditions under which these quantities can be estimated are not often sufficiently investigated. In this paper, induction machine and permanent-magnet-synchronous-machine drive state observability analysis is presented, together with conditions allowing reliable rotor speed and position estimation. A method based on the nonlinear dynamical system state observability theory is proposed, resulting in a unified approach to the ac drive observability analysis.

Adaptive Wavelet Wiener Filtering of ECG Signals

Abstract: In this study, we focused on the reduction of broadband myopotentials (EMG) in ECG signals using the wavelet Wiener filtering with noise-free signal estimation. We used the dyadic stationary wavelet transform (SWT) in the Wiener filter as well as in estimating the noise-free signal. Our goal was to find a suitable filter bank and to choose other parameters of the Wiener filter with respect to the signal-to-noise ratio (SNR) obtained. Testing was performed on artificially noised signals from the standard CSE database sampled at 500 Hz. When creating an artificial interference, we started from the generated white Gaussian noise, whose power spectrum was modified according to a model of the power spectrum of an EMG signal. To improve the filtering performance, we used adaptive setting parameters of filtering according to the level of interference in the input signal. We were able to increase the average SNR of the whole test database by about 10.6 dB. The proposed algorithm provides better results than the classic wavelet Wiener filter.

Dynamic switching of the spin circulation in tapered magnetic nanodisks

Abstract: Magnetic vortices are characterized by the sense of in-plane magnetization circulation and by the polarity of the vortex core. With each having two possible states, there are four possible stable magnetization configurations that can be utilized for a multibit memory cell. Dynamic control of vortex core polarity has been demonstrated using both alternating and pulsed magnetic fields and currents. Here, we show controlled dynamic switching of spin circulation in vortices using nanosecond field pulses by imaging the process with full-field soft X-ray transmission microscopy. The dynamic reversal process is controlled by far-from-equilibrium gyrotropic precession of the vortex core, and the reversal is achieved at significantly reduced field amplitudes when compared with static switching. We further show that both the field pulse amplitude and duration required for efficient circulation reversal can be controlled by appropriate selection of the disk geometry.

New resistorless and electronically tunable realization of dual-output VM all-pass filter using VDIBA

Abstract: In this paper, a new active element called voltage differencing inverting buffered amplifier (VDIBA) is presented Using single VDIBA and a capacitor, a new resistorless voltage-mode (VM) first-order all-pass filter (APF) is proposed, which provides both inverting and non-inverting outputs at the same configuration simultaneously The pole frequency of the filter can be electronically controlled by means of bias current of the internal transconductance No component-matching conditions are required and it has low sensitivity In addition, the parasitic and loading effects are also investigated By connecting two newly introduced APFs in open loop a novel second-order APF is proposed As another application, the proposed VM APF is connected in cascade to a lossy integrator in a closed loop to design a four-phase quadrature oscillator The theoretical results are verified by SPICE simulations using TSMC 018 μm level-7 CMOS process parameters with ±09 V supply voltages Moreover, the behavior of the proposed VM APF was also experimentally measured using commercially available integrated circuit OPA860 by Texas Instruments

Experiences in building a mOSAIC of clouds

Abstract: The diversity of Cloud computing services is challenging the application developers as various and non-standard interfaces are provided for these services. Few middleware solutions were developed until now to support the design, deployment and execution of service-independent applications as well as the management of resources from multiple Clouds. This paper focuses on one of these advanced middleware solutions, called mOSAIC. Written after the completion of its development, this paper presents an integrated overview of the mOSAIC approach and the use of its various software prototypes in a Cloud application development process. We are starting from the design concepts and arrive to various applications, as well as to the position versus similar initiatives.

Score normalization and system combination for improved keyword spotting

Abstract: We present two techniques that are shown to yield improved Keyword Spotting (KWS) performance when using the ATWV/MTWV performance measures: (i) score normalization, where the scores of different keywords become commensurate with each other and they more closely correspond to the probability of being correct than raw posteriors; and (ii) system combination, where the detections of multiple systems are merged together, and their scores are interpolated with weights which are optimized using MTWV as the maximization criterion. Both score normalization and system combination approaches show that significant gains in ATWV/MTWV can be obtained, sometimes on the order of 8-10 points (absolute), in five different languages. A variant of these methods resulted in the highest performance for the official surprise language evaluation for the IARPA-funded Babel project in April 2013.

Nanocarriers for anticancer drugs--new trends in nanomedicine.

Abstract: This review provides a brief overview of the variety of carriers employed for targeted drug delivery used in cancer therapy and summarizes advantages and disadvantages of each approach. Particularly, the attention was paid to polymeric nanocarriers, liposomes, micelles, polyethylene glycol, poly(lactic-co-glycolic acid), dendrimers, gold and magnetic nanoparticles, quantum dots, silica nanoparti- cles, and carbon nanotubes. Further, this paper briefly focuses on several anticancer agents (paclitaxel, docetaxel, camptothecin, doxoru- bicin, daunorubicin, cisplatin, curcumin, and geldanamycin) and on the influence of their combination with nanoparticulate transporters to their properties such as cytotoxicity, short life time and/or solubility.

Biogas Upgrading Techniques: State of Art Review in European Region

Abstract: Upgrading biogas to high energy biomethane is commonly utilized to produce SNG (substitute natural gas used for injecting into grid) and/or CNG (compressed natural gas employed as a fuel in transportation). In this process the most crucial operation is a separation of CO2 from biogas, which is carried out by technology based on unit operations like absorption or permeation. Aim of this text is to introduce state of art in development of these technologies within European region: assess their pros and cons in order to alleviate orientation in this issue and point out research tendencies. There were chosen three aspects for the assessment: (1) process-technology, (2) energy-economy and (3) environmental impact. The review is based on research of studies, reports and other informational sources that come from the work of some research institutions and engineering companies. From process-technology perspective specifications like methane purity and flow rate range were evaluated. Aspect regarding energy and economy is mainly focused on assessment of consistency of available data and on utilization of the data for comparative analysis of given technology. From environmental point of view there was considered an important specification called methane losses.The review clearly suggests that in European region the highest share refers to water scrubbing technology which is followed by PSA method and chemical absorption. All kinds of absorption technology (water scrubbing, absorption with chemical or physical solvents) seem to be well established and conventional. On the other hand PSA method and especially membrane technology offer progressive research potential.

Text-dependent speaker recognition using PLDA with uncertainty propagation

Abstract: In this paper, we apply and enhance the i-vector-PLDA paradigm to text-dependent speaker recognition. Due to its origin in text-independent speaker recognition, this paradigm does not make use of the phonetic content of each utterance. Moreover, the uncertainty in the i-vector estimates should be taken into account in the PLDA model, due to the short duration of the utterances. To bridge this gap, a phrase-dependent PLDA model with uncertainty propagation is introduced. We examined it on the RSR-2015 dataset and we show that despite its low channel variability, improved results over the GMM-UBM model are attained.

Off-axis setup taking full advantage of incoherent illumination in coherence-controlled holographic microscope

Abstract: Coherence-controlled holographic microscope (CCHM) combines off-axis holography and an achromatic grating interferometer allowing for the use of light sources of arbitrary degree of temporal and spatial coherence. This results in coherence gating and strong suppression of coherent noise and parasitic interferences enabling CCHM to reach high phase measurement accuracy and imaging quality. The achievable lateral resolution reaches performance of conventional widefield microscopes, which allows resolving up to twice smaller details when compared to typical off-axis setups. Imaging characteristics can be controlled arbitrarily by coherence between two extremes: fully coherent holography and confocal-like incoherent holography. The basic setup parameters are derived and described in detail and experimental validations of imaging characteristics are demonstrated.

All for the Price of Few

Abstract: We present a simple and efficient framework for automatic verification of systems with a parameteric number of communicating processes The processes may be organized in various topologies such as words, multisets, rings, or trees Our method needs to inspect only a small number of processes in order to show correctness of the whole system It relies on an abstraction function that views the system from the perspective of a fixed number of processes The abstraction is used during the verification procedure in order to dynamically detect cut-off points beyond which the search of the state space need not continue We show that the method is complete for a large class of well quasi-ordered systems including Petri nets Our experimentation on a variety of benchmarks demonstrate that the method is highly efficient and that it works well even for classes of systems with undecidable verification problems

A Survey of Non-conventional Techniques for Low-voltage Low-power Analog Circuit Design

Abstract: Designing integrated circuits able to work under low-voltage (LV) low-power (LP) condition is currently undergoing a very considerable boom. Reducing voltage supply and power consumption of integrated circuits is crucial factor since in general it ensures the device reli- ability, prevents overheating of the circuits and in particu- lar prolongs the operation period for battery powered devices. Recently, non-conventional techniques i.e. bulk- driven (BD), floating-gate (FG) and quasi-floating-gate (QFG) techniques have been proposed as powerful ways to reduce the design complexity and push the voltage supply towards threshold voltage of the MOS transistors (MOST). Therefore, this paper presents the operation principle, the advantages and disadvantages of each of these techniques, enabling circuit designers to choose the proper design technique based on application requirements. As an exam- ple of application three operational transconductance amplifiers (OTA) based on these non-conventional tech- niques are presented, the voltage supply is only ±0.4 V and the power consumption is 23.5 μW. PSpice simulation results using the 0.18 μm CMOS technology from TSMC are included to verify the design functionality and corre- spondence with theory.

The Tree Width of Separation Logic with Recursive Definitions

Abstract: Separation Logic is a widely used formalism for describing dynamically allocated linked data structures, such as lists, trees, etc. The decidability status of various fragments of the logic constitutes a long standing open problem. Current results report on techniques to decide satisfiability and validity of entailments for Separation Logic(s) over lists (possibly with data). In this paper we establish a more general decidability result. We prove that any Separation Logic formula using rather general recursively defined predicates is decidable for satisfiability, and moreover, entailments between such formulae are decidable for validity. These predicates are general enough to define (doubly-) linked lists, trees, and structures more general than trees, such as trees whose leaves are chained in a list. The decidability proofs are by reduction to decidability of Monadic Second Order Logic on graphs with bounded tree width.

A New Face Database Simultaneously Acquired in Visible, Near-Infrared and Thermal Spectrums

Abstract: In this paper, we present a new hand database called Tecnocampus Hand Image Database that includes right hand, palm and dorsal images. All the images have been acquired with three different sensors (visible, near-infrared and thermal). This database consists of 100 people acquired in five different acquisition sessions, two images per session and palm/dorsal sides. The total amount of pictures is 6.000, and it is mainly developed for hand image biometric recognition purposes. In addition, the database has been studied from the information theory point of view, and we found that this highest level of information is achieved in thermal spectrum. Furthermore, a low level of mutual information between different spectrums is also demonstrated. This opens an interesting research field in multi-sensor data fusion.

The tree width of separation logic with recursive definitions

Abstract: Separation Logic is a widely used formalism for describing dynamically allocated linked data structures, such as lists, trees, etc. The decidability status of various fragments of the logic constitutes a long standing open problem. Current results report on techniques to decide satisfiability and validity of entailments for Separation Logic(s) over lists (possibly with data). In this paper we establish a more general decidability result. We prove that any Separation Logic formula using rather general recursively defined predicates is decidable for satisfiability, and moreover, entailments between such formulae are decidable for validity. These predicates are general enough to define (doubly-) linked lists, trees, and structures more general than trees, such as trees whose leaves are chained in a list. The decidability proofs are by reduction to decidability of Monadic Second Order Logic on graphs with bounded tree width.