Transfer learning with seasonal and trend adjustment for cross-building energy forecasting

In this paper, we introduce L2-ARCTIC, a speech corpus of non-native English that is intended for research in voice conversion, accent conversion, and mispronunciation detection. This initial release includes recordings from ten non-native speakers of English whose first languages (L1s) are Hindi, Korean, Mandarin, Spanish, and Arabic, each L1 containing recordings from one male and one female speaker. Each speaker recorded approximately one hour of read speech from the Carnegie Mellon University ARCTIC prompts, from which we generated orthographic and forced-aligned phonetic transcriptions. In addition, we manually annotated 150 utterances per speaker to identify three types of mispronunciation errors: substitutions, deletions, and additions, making it a valuable resource not only for research in voice conversion and accent conversion but also in computer-assisted pronunciation training. The corpus is publicly accessible at https://psi.engr.tamu.edu/l2-arctic-corpus/.

https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1227&context=engl_pubs

L2-ARCTIC: A Non-Native English Speech Corpus

This paper investigates the use of multidistribution deep neural networks DNNs for mispronunciation detection and diagnosis MDD, to circumvent the difficulties encountered in an existing approach based on extended recognition networks ERNs. The ERNs leverage existing automatic speech recognition technology by constraining the search space via including the likely phonetic error patterns of the target words in addition to the canonical transcriptions. MDDs are achieved by comparing the recognized transcriptions with the canonical ones. Although this approach performs reasonably well, it has the following issues: 1 Learning the error patterns of the target words to generate the ERNs remains a challenging task. Phones or phone errors missing from the ERNs cannot be recognized even if we have well-trained acoustic models; and 2 acoustic models and phonological rules are trained independently, and hence, contextual information is lost. To address these issues, we propose an acoustic-graphemic-phonemic model AGPM using a multidistribution DNN, whose input features include acoustic features, as well as corresponding graphemes and canonical transcriptions encoded as binary vectors. The AGPM can implicitly model both grapheme-to-likely-pronunciation and phoneme-to-likely-pronunciation conversions, which are integrated into acoustic modeling. With the AGPM, we develop a unified MDD framework, which works much like free-phone recognition. Experiments show that our method achieves a phone error rate PER of 11.1%. The false rejection rate FRR, false acceptance rate FAR, and diagnostic error rate DER for MDD are 4.6%, 30.5%, and 13.5%, respectively. It outperforms the ERN approach using DNNs as acoustic models, whose PER, FRR, FAR, and DER are 16.8%, 11.0%, 43.6%, and 32.3%, respectively.

/pdf/mispronunciation-detection-and-diagnosis-in-l2-english-1zr8uf24qg.pdf

Mispronunciation Detection and Diagnosis in L2 English Speech Using Multidistribution Deep Neural Networks

We propose the use of speech attributes, such as voicing and aspiration, to address two key research issues in computer assisted pronunciation training (CAPT) for L2 learners, namely detecting mispronunciation and providing diagnostic feedback. To improve the performance we focus on mispronunciations occurred at the segmental and sub-segmental levels. In this study, speech attributes scores are first used to measure the pronunciation quality at a sub-segmental level, such as manner and place of articulation. These speech attribute scores are integrated by neural network classifiers to generate segmental pronunciation scores. Compared with the conventional phone-based GOP (Goodness of Pronunciation) system we implement with our dataset, the proposed framework reduces the equal error rate by 8.78% relative. Moreover, it attains comparable results to phone-based classifier approach to mispronunciation detection while providing comprehensive feedback, including segmental and sub-segmental diagnostic information, to help L2 learners.

Improving non-native mispronunciation detection and enriching diagnostic feedback with DNN-based speech attribute modeling

rules and representations (such as those found in Chomsky & Halle, 1968). Let us take an example from stress placement. As any learner of English knows, the patterns of English stress placement are complex: robúst, énter, cínema, Appalàchicóla. Chomsky and Halle’s main stress rule was one of the first successful attempts to find patterns in English stress placement. To paraphrase and simplify (ignoring nouns): For verbs or adjectives, assign primary stress to the final vowel of the word if the word ends in a long vowel, or a short vowel followed by a consonant; otherwise stress the penultimate syllable. As Kaye (1990) pointed out: if this is a possible rule, what isn’t? The point to be made here is that, when looking at proposals like the main stress rule, linguists began to realize that grammatical models must be learnable based on the input that the learners are exposed to. It was no longer enough to find the patterns in the data. From this point on, linguistic theory began to offer real insights into the study of both first and second language acquisition. The rules proposed give us descriptions of where the L2 learner starts, where they want to end up and what their intermediate system looks like. J. Archibald

Yong Wang

Papers

Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiers