This paper presents an open-source diarization toolkit which is mostly dedicated to speaker and developed by the LIUM This toolkit includes hierarchical agglomerative clustering methods using well-known measures such as BIC and CLR Two applications for which the toolkit has been used are presented: one is for broadcast news using the ESTER 2 data and the other is for telephone conversations using the MEDIA corpus

https://hal.archives-ouvertes.fr/hal-01433518/document

Lium spkdiarization: an open source toolkit for diarization

The French Technolangue MEDIA-EVALDA project aims to evaluate spoken understanding approaches. This paper describes the semantic annotation scheme of a common dialog corpus which will be used for developing and evaluating spoken understanding models and for linguistic studies. A common semantic representation has been formalized and agreed upon by the consortium. Each utterance is divided into semantic segments and each segment is annotated with a 5-tuplet containing the mode, attribute name representing the underlying concept, normalized form of the attribute, list of related segments, and an optional comment about the annotation. Periodic interannotator agreement studies demonstrate that the annotation are of good quality, with an agreement of almost 90% on mode and attribute identification. An analysis of the semantic content of 12292 annotated client utterances shows that only 14.1% of the observed attributes are domain-dependent and that the semantic dictionary ensures a good coverage of the task.

Semantic annotation of the French media dialog corpus.

Conventional speech recognition systems are based on Gaussian hidden Markov models (HMMs). Discriminative techniques such as log-linear modeling have been investigated in speech recognition only recently. This thesis establishes a log-linear modeling framework in the context of discriminative training criteria, with examples from continuous speech recognition, part-of-speech tagging, and handwriting recognition. The focus will be on the theoretical and experimental comparison of different training algorithms. Equivalence relations for Gaussian and log-linear models in speech recognition are derived. It is shown how to incorporate a margin term into conventional discriminative training criteria like for example minimum phone error (MPE). This permits to evaluate directly the utility of the margin concept for string recognition. The equivalence relations and the margin-based training criteria lead to a unified view of three major training paradigms, namely Gaussian HMMs, log-linear models, and support vector machines (SVMs). Generalized iterative scaling (GIS) is traditionally used for the optimization of log-linear models with the maximum mutual information (MMI) criterion. This thesis suggests an extension of GIS to log-linear models including hidden variables, and to other training criteria (e.g. MPE). Finally, investigations on convex optimization in speech recognition are presented. Experimental results are provided for a variety of tasks, including the European Parliament plenary sessions task and Mandarin broadcasts.

A log-linear discriminative modeling framework for speech recognition.

Conventional speech recognition systems are based on hidden Markov models (HMMs) with Gaussian mixture models (GHMMs). Discriminative log-linear models are an alternative modeling approach and have been investigated recently in speech recognition. GHMMs are directed models with constraints, e.g., positivity of variances and normalization of conditional probabilities, while log-linear models do not use such constraints. This paper compares the posterior form of typical generative models related to speech recognition with their log-linear model counterparts. The key result will be the derivation of the equivalence of these two different approaches under weak assumptions. In particular, we study Gaussian mixture models, part-of-speech bigram tagging models, and eventually, the GHMMs. This result unifies two important but fundamentally different modeling paradigms in speech recognition on the functional level. Furthermore, this paper will present comparative experimental results for various speech tasks of different complexity, including a digit string and large-vocabulary continuous speech recognition tasks.

Equivalence of Generative and Log-Linear Models

The extraction of flat concepts out of a given word sequence is usually one of the first steps in building a spoken language understanding (SLU) or dialogue system. This paper explores five different modelling approaches for this task and presents results on a French state-of-the-art corpus, MEDIA. Additionally, two log-linear modelling approaches could be further improved by adding morphologic knowledge. This paper goes beyond what has been reported in the literature, e.g. in (Raymond & Riccardi 07). We applied the models on the same training and testing data and used the NIST scoring toolkit to evaluate the experimental results to ensure identical conditions for each of the experiments and the comparability of the results. Using a model based on conditional random fields, we achieve a concept error rate of 11.8% on the MEDIA evaluation corpus.

/pdf/a-comparison-of-various-methods-for-concept-tagging-for-3172eeqcbz.pdf

A Comparison of Various Methods for Concept Tagging for Spoken Language Understanding

The aim of the MEDIA project is to design and test a methodology for the evaluat ion of context-dependent and independent spoken dialogue systems. We propose an evaluation paradigm based on the use of test suites from real-world corpora and a common semantic representation and common metrics. This paradigm should allow us to diagnose the context-sensitive understanding capability of dialogue system s. This paradigm will be used within an evaluation campaign involving several si tes all of which will carry out the task of querying information from a database .

/pdf/the-french-media-evalda-project-the-evaluation-of-the-17fje9yn8i.pdf

The French MEDIA/EVALDA Project: the Evaluation of the Understanding Capability of Spoken Language Dialogue Systems.

This paper presents and reports on the progress of the EVALDA/MEDIA project, focusing on the recording protocol of the reference dialogue corpus. The aim of this project is to define and test an evaluation methodology that assess and diagnose the contextsensitive understanding capability of spoken language dialogue systems. Systems from both academic organizations (CLIPS, IRIT, LIA, LIMSI, LORIA, VALORIA) and industrial sites (FRANCE TELECOM R et D, TELIP) will be evaluated. ELDA is the coordinator of the Technolangue/EVALDA multicampaign evaluation project, a national initiative sponsored by the French government, of which MEDIA is a sub-campaign. MEDIA began in January 2003. VECSYS provides the recording platform for the project.

http://www.afcp-parole.org/doc/Archives_JEP/2004_XXVe_JEP_Fes/actes/jep2004/McTait-etal2.pdf

Constitution d'un corpus de dialogue oral pour l'évaluation automatique de la compréhension hors- et en- contexte du dialogue

The problem discussed in this paper concerns how spoken interaction research can give a solution for the universal access for all. This paper will firstly report on the state of the art of speech technology for older people. Secondly, it will describe the case study of the selection of a French dictation system for a fundamental evaluation by elderly people. This session must be considered as a case study. We will place our approach with respect to NIST (http://www.nist.gov/speech/) and EAGLES [Gibbon 1997] evaluation contexts. Finally, we will give preliminary results mainly on the effect of the people age.

Speech technology for universal access in interactive systems

Notre travail &#233;tudie l&#8217;impact du mouvement #BalanceTonPorc aupr&#232;s de femmes ayant subi un harc&#232;lement au travail. Nos r&#233;sultats montrent un int&#233;r&#234;t pour cette d&#233;marche qui peine cependant &#224; se transposer en engagements /actes dans la vie r&#233;elle. Si les victimes interrog&#233;es affirment appr&#233;cier le mouvement, celui-ci n&#8217;a pas eu de traduction particuli&#232;re dans leur vie quotidienne et ne les a pas conduites &#224; une meilleure mobilisation de leurs droits. Nos r&#233;sultats viennent ainsi temp&#233;rer l&#8217;enthousiasme m&#233;diatique qui a consid&#233;r&#233; le &#171;&#160;hashtag activism&#160;&#187; comme l&#8217;alpha et l&#8217;om&#233;ga de la lutte contre le harc&#232;lement. 

Caroline Bousquet

Papers

The French MEDIA/EVALDA Project: the Evaluation of the Understanding Capability of Spoken Language Dialogue Systems.

Constitution d'un corpus de dialogue oral pour l'évaluation automatique de la compréhension hors- et en- contexte du dialogue

Speech technology for universal access in interactive systems

Un # au secours du droit dans le harcèlement sexuel au travail : le cas #BalanceTonPorc