This paper focuses on the tasks of recovering capitalization and punctuation marks from texts without that information, such as spoken transcripts, produced by automatic speech recognition systems. These two practical rich transcription tasks were performed using the same discriminative approach, based on maximum entropy, suitable for on-the-fly usage. Reported experiments were conducted both over Portuguese and English broadcast news data. Both force aligned and automatic transcripts were used, allowing to measure the impact of the speech recognition errors. Capitalized words and named entities are intrinsically related, and are influenced by time variation effects. For that reason, the so-called language dynamics have been addressed for the capitalization task. Language adaptation results indicate, for both languages, that the capitalization performance is affected by the temporal distance between the training and testing data. In what regards the punctuation task, this paper covers the three most frequent punctuation marks: full stop, comma, and question marks. Different methods were explored for improving the baseline results for full stop and comma. The first uses punctuation information extracted from large written corpora. The second applies different levels of linguistic structure, including lexical, prosodic, and speaker related features. The comma detection improved significantly in the first method, thus indicating that it depends more on lexical features. The second method provided even better results, for both languages and both punctuation marks, best results being achieved mainly for full stop. As for question marks, there is a small gain, but differences are not very significant, due to the relatively small number of question marks in the corpora.

Bilingual Experiments on Automatic Recovery of Capitalization and Punctuation of Automatic Speech Transcripts

https://vithea.l2f.inesc-id.pt/wiki/images/1/17/CSLVithea.pdf

Automatic word naming recognition for an on-line aphasia treatment system

Broadcast news play an important role in our lives provid-ing access to news, information and entertainment. The ex-istence of an automatic transcription is an important mediumthat not only can provide subtitles for inclusion of people withspecial needs or be an advantage on noisy and populated envi-ronments, but also because it enables data search and retrievecapabilities over the multimedia streams. In this work we willdescribe and evaluate the automatic speech recognition systemsdeveloped for two Iberian languages, European Portuguese andSpanish and also for Brazilian Portuguese, African Portugueseand English. The developed systems are fully automatic andcapable to subtitling in real-time Broadcast News stream with avery small delay.Index Terms: Speech Recognition, Broadcast News, Iberianlanguages, Accent, Online processing 1. Introduction The Broadcast News (BN) processing system developed at theSpoken Language Systems Lab of INESC-ID integrates sev-eral core technologies, in a pipeline architecture: jingle detec-tion, audio segmentation, automatic speech recognition, punc-tuation, capitalization, topic segmentation/indexation, summa-rization, and translation. The ﬁrst modules of this system wereoptimized for on-line performance, given their deployment inthe fully automatic speech recognition subtitling system that isrunning on the main news shows of the public TV channel inPortugal (RTP), since March 2008.To our knowledge, the majority of subtitling systems de-scribed in the literature rely on speech-to-text alignment ratherthan full automatic speech recognition [1]. Re-speakers alsoare commonly used to simplify the original speech, and speechrecognition engines are adapted to the captioner voice [2].This paper concerns the third module in the pipeline -speech recognition, emphasizing the most recent improvements,and our efforts to port it to other languages (English and Span-ish), and to other varieties of Portuguese, namely those spokenin the South American and African continents.The development of a system for a new language is a chal-lenging task due to the need of new acoustic training data, vo-cabulary deﬁnition, lexicon generation and language model es-timation [3].The paper starts with a description of the main modules ofour recognition engine, emphasizing the two language indepen-dent components - feature extraction and decoder. The nextthree sections are devoted to the three varieties of Portuguesecovered by our system: the original one (European Portuguese,henceforth designated as EP), Brazilian Portuguese (BP), andAfrican Portuguese (AP). The porting efforts for the other twolanguages (European Spanish and American English) are de-scribed in Sections 6 and 7, respectively. For each of these sec-tions, we shall detail the corpora, vocabulary, and lexical andlanguage model generation, ending with performance results.The ﬁnal section discusses the main advantages and shortcom-ings of these systems, namely in what concerns real time closecaptioning applications.

http://www.inesc-id.pt/pt/indicadores/Ficheiros/6729.pdf

The L2F Broadcast News Speech Recognition System

http://www.inesc-id.pt/pt/indicadores/Ficheiros/4830.pdf

Recovering capitalization and punctuation marks for automatic speech recognition: Case study for Portuguese broadcast news

Over the years, screen readers have been an essential tool for assisting blind users in accessing digital information. Yet, its sequential nature undermines blind people's ability to efficiently find relevant information, despite the browsing strategies they have developed. We propose taking advantage of the Cocktail Party Effect, which states that people are able to focus on a single speech source among several conversations, but still identify relevant content in the background. Therefore, oppositely to one sequential speech channel, we hypothesize that blind people can leverage concurrent speech channels to quickly get the gist of digital information. In this paper, we present an experiment with 23 participants, which aims to understand blind people's ability to search for relevant content listening to two, three or four concurrent speech channels. Our results suggest that it is easy to identify the relevant source with two and three concurrent talkers. Moreover, both two and three sources may be used to understand the relevant source content depending on the task intelligibility demands and user characteristics.

Text-to-speeches: evaluating the perception of concurrent speech by blind people

The subtitling of broadcast news programs are starting to become a very interesting application due to the technological advances in automatic speech recognition and associated technologies. However, to build this kind of systems, several advances are necessary both in terms of the technological components and on main blocks integration. In this paper, we are presenting the overall architecture of a subtitling system running daily at RTP (the Portuguese public broadcast company). The goal is to integrate our components in a system for the subtitling of RTP programs. The global system includes the subtitling of recorded and direct programs.

http://www.inesc-id.pt/pt/indicadores/Ficheiros/6396.pdf

Broadcast news subtitling system in Portuguese

This paper describes a new generic text-to-speech synthesis system, developed in the scope of the Tecnovoz Project. Although it was primarily targeted at speech synthesis in European Portuguese, its modular architecture and flexible components allows its use for different languages. We also provide a survey on the development of the language resources needed by the TTS.

DIXI --- A Generic Text-to-Speech System for European Portuguese

In this paper we describe a multi-purpose Spoken Dialogue System platform associated with two distinct applications as an home intelligent environment and remote access to information databases. These applications differ substantially on contents and possible uses but gives us the chance to develop a platform where we were able to represent diverse services to be accessible by a spoken interface. The implemented voice input/output possibilities and the service independence level opens a wide range of possibilities for the development of new applications using the current components of our Spoken Dialogue System.

/pdf/the-development-of-a-multi-purpose-spoken-dialogue-system-46amd6xq0p.pdf

The development of a multi-purpose spoken dialogue system.

Building a generalized platform for having dialogues is a hard problem in the topic of dialogue systems. The problem becomes still more difficult if there is the possibility of having a conversation about more than one domain at the same time. To help solving this problem, we have built a component that deals with everything related to the domains. In this paper, we describe this component, named Service Manager, and explain what and how it passes all the information that a dialogue manager needs to conduct a dialogue.

An Independent Domain Dialogue System Through a Service Manager

In this paper we described our initial work on the development of an embodied conversational agent platform. In the present stage our main focus it is on the development of a multimodal input interface to the system. In this paper we will present an Input and Output Manager block that combines speech, synthetic talking face, text and graphical interfaces. The system support speech input through an ASR and speech output through a TTS, synchronized with an animated face. The graphical and text input are feed through a Text Manger that it is a constituent component of the Input and Output Manager block. All the blocks are tailored for the European Portuguese language. The system is analyzed in the framework of the project Interactive Home of the Future.

Renato Cassaca

Papers

Broadcast news subtitling system in Portuguese

DIXI --- A Generic Text-to-Speech System for European Portuguese

The development of a multi-purpose spoken dialogue system.

An Independent Domain Dialogue System Through a Service Manager

Design of a multimodal input interface for a dialogue system