Editing audio narration using conventional software typically involves many painstaking low-level manipulations. Some state of the art systems allow the editor to work in a text transcript of the narration, and perform select, cut, copy and paste operations directly in the transcript; these operations are then automatically applied to the waveform in a straightforward manner. However, an obvious gap in the text-based interface is the ability to type new words not appearing in the transcript, for example inserting a new word for emphasis or replacing a misspoken word. While high-quality voice synthesizers exist today, the challenge is to synthesize the new word in a voice that matches the rest of the narration. This paper presents a system that can synthesize a new word or short phrase such that it blends seamlessly in the context of the existing narration. Our approach is to use a text to speech synthesizer to say the word in a generic voice, and then use voice conversion to convert it into a voice that matches the narration. Offering a range of degrees of control to the editor, our interface supports fully automatic synthesis, selection among a candidate set of alternative pronunciations, fine control over edit placements and pitch profiles, and even guidance by the editors own voice. The paper presents studies showing that the output of our method is preferred over baseline methods and often indistinguishable from the original voice.

VoCo: text-based insertion and replacement in audio narration

In this paper, use of Viterbi-based algorithm and spectral transition measure (STM)-based algorithm for the task of speech data labeling is being attempted. In the STM framework, we propose use of several spectral features such as recently proposed cochlear filter cepstral coefficients (CFCC), perceptual linear prediction cepstral coefficients (PLPCC) and RelAtive SpecTrAl (RASTA)-based PLPCC in addition to Mel frequency cepstral coefficients (MFCC) for phonetic segmentation task. To evaluate effectiveness of these segmentation algorithms, we require manual accurate phoneme-level labeled data which is not available for low resourced languages such as Gujarati (one of the official languages of India). In order to measure effectiveness of various segmentation algorithms, HMM-based speech synthesis system (HTS) for Gujarati has been built. From the subjective and objective evaluations, it is observed that Viterbi-based and STM with PLPCC-based segmentation algorithms work better than other algorithms.

Effectiveness of PLP-based phonetic segmentation for speech synthesis

The text to speech technology has achieved significant progress during the past decade and is an active area of research and development in providing different human–computer interactive systems. Even though a number of speech synthesis models are available for different languages focusing on the domain requirements with many motive applications, a source of information on current trends in Indian language speech synthesis is unavailable till date making it difficult for the beginners to initiate research for the development of TTS systems for the low-resourced languages. This paper provides a review of the contributions made by different researchers in the field of Indian language speech synthesis along with a study on the Indian language characteristics and the associated challenges in designing TTS systems. A set of available applications and tools results out of different projects undertaken by different organizations along with a set of possible future developments are also discussed to provide a single reference to an important strand of research in speech synthesis which may benefit anyone interested to initiate research in this area.

A survey on speech synthesis techniques in Indian languages

This paper describes the implementation of the IRISA unit selection-based TTS system for our participation in the Blizzard Challenge 2016. We describe the process followed to build the voices from given data and the architecture of our system. The search is based on a A* algorithm with preselection filters used to reduce the search space. A penalty is introduced in the concatenation cost to block some concatenations based on their phonological class. Moreover, a fuzzy function is used to relax this penalty based on the concatenation quality with respect to the cost distribution.

https://hal.inria.fr/hal-01375897/document

The IRISA Text-To-Speech System for the Blizzard Challenge 2016

The main objective of this work is to classify Hindi and Telugu stories based on their structure into three genres: Fable, Folk-tale and Legend In this work, each story is divided into three parts: (i) introduction, (ii) main and (iii) climax The objective of this work is to explore how story genre information is embedded in different parts of the story We are proposing a framework for story classification using keyword and Part-of-speech (POS) based features Keyword based features like Term Frequency (TF) and Term Frequency Inverse Document Frequency (TFIDF) are used Classification performance is analyzed for different story parts using various combinations of features with three classifiers: (i) Naive Bayes (NB), (ii) k-Nearest Neighbour (KNN) and (iii) Support Vector Machine (SVM) From the experimental studies, it has been observed that classification performance has not significantly improved by combining linguistic (POS) and keyword based features Among classifiers, SVM outperformed the other classifiers The main part of the story has the highest classification accuracy compared to introduction and climax parts of the story

Children story classification based on structure of the story

In this paper, we discuss a consortium effort on building text to speech (TTS) systems for 13 Indian languages. There are about 1652 Indian languages. A unified framework is therefore attempted required for building TTSes for Indian languages. As Indian languages are syllable-timed, a syllable-based framework is developed. As quality of speech synthesis is of paramount interest, unit-selection synthesizers are built. Building TTS systems for low-resource languages requires that the data be carefully collected an annotated as the database has to be built from the scratch. Various criteria have to addressed while building the database, namely, speaker selection, pronunciation variation, optimal text selection, handling of out of vocabulary words and so on. The various characteristics of the voice that affect speech synthesis quality are first analysed. Next the design of the corpus of each of the Indian languages is tabulated. The collected data is labeled at the syllable level using a semiautomatic labeling tool. Text to speech synthesizers are built for all the 13 languages, namely, Hindi, Tamil, Marathi, Bengali, Malayalam, Telugu, Kannada, Gujarati, Rajasthani, Assamese, Manipuri, Odia and Bodo using the same common framework. The TTS systems are evaluated using degradation Mean Opinion Score (DMOS) and Word Error Rate (WER). An average DMOS score of ≈3.0 and an average WER of about 20 % is observed across all the languages.

A syllable-based framework for unit selection synthesis in 13 Indian languages

With the development of high quality TTS systems, application area of synthetic speech is increasing rapidly. Beyond the communication aids for the visually impaired and vocally handicap, TTS voices are being used in various educational, telecommunication and multimedia applications. All around the world people are trying to build TTS voice for their regional languages. TTS voice building requires a number of steps to follow and involves use of multiple tools, which makes it time consuming, tedious and perplexing to a user. This paper describes a Toolkit developed for HMM-based TTS voice building that makes the process much easier and handy. The toolkit uses all required tools, viz. HTS, Festival, Festvox, Hybrid Segmentation Tool, etc. and handles each and every step starting from phone set creation, then prompt generation, hybrid segmentation, F0 range finding, voice building, and finally putting the built voice into Synthesis framework. Wherever possible it does parallel processing to reduce time. It saves manual effort and time to a large extent and enable a person to build TTS voice very easily. This toolkit is made available under Open Source license.

TBT (Toolkit to Build TTS): A High Performance Framework to Build Multiple Language HTS Voice.

India is home to about 1600 languages. Less than 5% of this population is English literate. A consortium effort on developing a common framework to build text-to-speech synthesis (TTS) in various Indian languages has already been realized. In this paper, we propose a parallel effort to integrate the various TTS systems developed into various applications. These applications are primarily aimed at bringing the marginalized sections of Indian society into the main stream. The applications integrated include: i) screen readers for the visually challenged on desktop platforms ii) web-based plug-in to read highlighted text for the language challenged iii) integration of TTS with optical character recognition systems iv) various applications for the Android platform v) On-line test and tutorial for visually challenged and vi) conversion of gesture to speech for persons with cerebral palsy. The performance of the integrated systems have been extensively tested by visually challenged persons for the last three years. About 180 students have learnt to use the Office Suite, Internet and e-mail over the last 3 years.

Seamless integration of common framework Indian language TTSes in various applications

Lots of efforts have been made in the development of Automatic Speech Recognition systems for Indian languages, and good progress has been made for Hindi, Tamil, Telugu & Bengali, and Indian English. However, practically usable robust systems are still not available for many regional Indian languages including Marathi. This paper presents an effort toward the development of a large vocabulary continuous speech recognition system for Marathi and its integration into practical application, such as Speech-based typing system. This speech recognition system has been developed using the Kaldi toolkit. The Kaldi toolkit facilitates the exploration of different acoustic models such as Gaussian Mixture Model with Hidden Markov Model (GMM-HMM) and Deep Neural Networks with Hidden Markov Model (DNN-HMM). In this paper we have developed Marathi Speech Recognition system which is trained around 95 hours of audio data. We have used the trigram language model which is having 29 lakhs of words and 3 lakhs of vocabulary size. This system gives average word error rate of 19%.

Pranaw Kumar

Papers

A syllable-based framework for unit selection synthesis in 13 Indian languages

TBT (Toolkit to Build TTS): A High Performance Framework to Build Multiple Language HTS Voice.

Seamless integration of common framework Indian language TTSes in various applications

Large Vocabulary Continuous Speech Recognition System for Marathi