The understanding of sociolinguistic variation is growing rapidly, but basic gaps still remain. Whether some languages or dialects are spoken faster or slower than others constitutes such a gap. Speech tempo is interconnected with social, physical and psychological markings of speech. This study examines regional variation in articulation rate and its manifestations across speaker age, gender and speaking situations (reading vs. free conversation). The results of an experimental investigation show that articulation rate differs significantly between two regional varieties of American English examined here. A group of Northern speakers (from Wisconsin) spoke significantly faster than a group of Southern speakers (from North Carolina). With regard to age and gender, young adults read faster than older adults in both regions; in free speech, only Northern young adults spoke faster than older adults. Effects of gender were smaller and less consistent; men generally spoke slightly faster than women. As the body of work on the sociophonetics of American English continues to grow in scope and depth, we argue that it is important to include fundamental phonetic information as part of our catalog of regional differences and patterns of change in American English.

/pdf/articulation-rate-across-dialect-age-and-gender-oar45xhmew.pdf

Articulation rate across dialect, age, and gender

Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller models excel. In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q\&A system, two different search engines, a translation system, and a calendar. Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.

Toolformer: Language Models Can Teach Themselves to Use Tools

Language is essentially a complex, intricate system of human expressions governed by grammatical rules. It poses a significant challenge to develop capable AI algorithms for comprehending and grasping a language. As a major approach, language modeling has been widely studied for language understanding and generation in the past two decades, evolving from statistical language models to neural language models. Recently, pre-trained language models (PLMs) have been proposed by pre-training Transformer models over large-scale corpora, showing strong capabilities in solving various NLP tasks. Since researchers have found that model scaling can lead to performance improvement, they further study the scaling effect by increasing the model size to an even larger size. Interestingly, when the parameter scale exceeds a certain level, these enlarged language models not only achieve a significant performance improvement but also show some special abilities that are not present in small-scale language models. To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size. Recently, the research on LLMs has been largely advanced by both academia and industry, and a remarkable progress is the launch of ChatGPT, which has attracted widespread attention from society. The technical evolution of LLMs has been making an important impact on the entire AI community, which would revolutionize the way how we develop and use AI algorithms. In this survey, we review the recent advances of LLMs by introducing the background, key findings, and mainstream techniques. In particular, we focus on four major aspects of LLMs, namely pre-training, adaptation tuning, utilization, and capacity evaluation. Besides, we also summarize the available resources for developing LLMs and discuss the remaining issues for future directions.

A Survey of Large Language Models

—Recent breakthroughs in natural language pro- cessing (NLP) have permitted the synthesis and comprehension of coherent text in an open-ended way, therefore translating the theoretical algorithms into practical applications. The large language-model (LLM) has signiﬁcantly impacted businesses such as report summarization softwares and copywriters. Ob-servations indicate, however, that LLMs may exhibit social prejudice and toxicity, posing ethical and societal dangers of consequences resulting from irresponsibility. Large-scale bench- marks for accountable LLMs should consequently be developed. Although several empirical investigations reveal the existence of a few ethical diﬃculties in advanced LLMs, there is no systematic examination and user study of the ethics of current LLMs use. To further educate future eﬀorts on constructing ethical LLMs responsibly, we perform a qualitative research method on OpenAI’s ChatGPT 1 to better understand the practical features of ethical dangers in recent LLMs. We analyze ChatGPT comprehensively from four perspectives: 1) Bias 2) Reliability 3) Robustness 4) Toxicity. In accordance with our stated viewpoints, we empirically benchmark ChatGPT on multiple sample datasets. We ﬁnd that a signiﬁcant number of ethical risks cannot be addressed by existing benchmarks, and hence illustrate them via additional case studies. In addition, we examine the implications of our ﬁndings on the AI ethics of ChatGPT, as well as future problems and practical design considerations for LLMs. We believe that our ﬁndings may give light on future eﬀorts to determine and mitigate the ethical hazards posed by machines in LLM applications.

Exploring AI Ethics of ChatGPT: A Diagnostic Analysis

Do differences in male versus female /s/ reflect biological or sociophonetic factors?

Driven by the goal of eradicating language barriers on a global scale, machine translation has solidified itself as a key focus of artificial intelligence research today. However, such efforts have coalesced around a small subset of languages, leaving behind the vast majority of mostly low-resource languages. What does it take to break the 200 language barrier while ensuring safe, high quality results, all while keeping ethical considerations in mind? In No Language Left Behind, we took on this challenge by first contextualizing the need for low-resource language translation support through exploratory interviews with native speakers. Then, we created datasets and models aimed at narrowing the performance gap between low and high-resource languages. More specifically, we developed a conditional compute model based on Sparsely Gated Mixture of Experts that is trained on data obtained with novel and effective data mining techniques tailored for low-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Critically, we evaluated the performance of over 40,000 different translation directions using a human-translated benchmark, Flores-200, and combined human evaluation with a novel toxicity benchmark covering all languages in Flores-200 to assess translation safety. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art, laying important groundwork towards realizing a universal translation system. Finally, we open source all contributions described in this work, accessible at https://github.com/facebookresearch/fairseq/tree/nllb.

No Language Left Behind: Scaling Human-Centered Machine Translation

There are many studies demonstrating the phonetic cues to gender found in our speech, but for many of these cues it is not clear if they are a product of our anatomical make-up or a product of our social background. The goal of this study is to demonstrate that one of these cues to gender, the frequency spectrum of the frication produced by the sibilant /s/, contains a socially acquired component. Male and female adult speakers of two languages, Canadian English and Japanese, were recorded producing words containing /s/. The harmonics-to-noise ratio (HNR) of each subject's speech sample was also measured. ANOVA analysis of the spectral moments of /s/ revealed significant differences between the two language groups. As well, the female English speakers showed a significant correlation between the first moment and HNR, suggesting that at least for the female English group, /s/ contains a socially acquired component.

https://twpl.library.utoronto.ca/index.php/twpl/article/download/6208/3197

Evidence from HNR that /s/ is a social marker of gender

Scaling multilingual representation learning beyond the hundred most frequent languages is challenging, in particular to cover the long tail of low-resource languages. A promising approach has been to train one-for-all multilingual models capable of cross-lingual transfer, but these models often suffer from insufficient capacity and interference between unrelated languages. Instead, we move away from this approach and focus on training multiple language (family) specific representations, but most prominently enable all languages to still be encoded in the same representational space. To achieve this, we focus on teacher-student training, allowing all encoders to be mutually compatible for bitext mining, and enabling fast learning of new languages. We introduce a new teacher-student training scheme which combines supervised and self-supervised training, allowing encoders to take advantage of monolingual training data, which is valuable in the low-resource setting. Our approach significantly outperforms the original LASER encoder. We study very low-resource languages and handle 50 African languages, many of which are not covered by any other model. For these languages, we train sentence encoders, mine bitexts, and validate the bitexts by training NMT systems.

Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages

Since it has been realized that loanword phonology is an important source of phonological evidence, people have been debating the exact nature of the input There are two approaches; either the input to the loanword phonology is phonetic, or it is phonemic In this paper I argue that both possibilities exist, with the actual choice depending on the social relationship between the native and the non-native language I then demonstrate that if the input is phonetic, then "phonetic similarity" is emphasized, whereas if the input is phonemic, then "phonemic contrast" is emphasized

https://twpl.library.utoronto.ca/index.php/twpl/article/download/6196/3185

Phonetic similarity and phonemic contrast in loanword adaptation

I begin with a distinction between engendered variation, that is, sex-based phonetic-level variation that shows consistent patterns across communities, and nongendered variation. Much of engendered variation is anatomically-determined, but there is also an important behaviourally-determined component. One example of behaviourally-determined engendered variation is that women speak more clearly, as reflected in their more-dispersed vowel space. I argue that this sex-based difference applies to all types of phonological contrasts, not just vowels, so that the temporal and acoustic correlates of phonological constituents produced by women are differentiated typically more than those produced by men. Furthermore, I demonstrate that “phonetic distinctiveness” indexes other social categories such as social gender and social class, i.e., is a sociolinguistic variable. I corroborate the indexing of social categories in two major applications. First, I compare phonetic correlates in the speech of eight male radio DJs from a range of music genres with ratings of how macho-sounding each DJ is. For each DJ, I examine durational differences between contextual and inherent long and short vowels. As expected, social gender significantly correlates with the vowel duration distinctiveness so that the more macho-sounding DJs produce less distinct vowel length contrasts. Second, I examine dispersion of front and back vowels in the speech of 439 speakers from the Atlas of North American English (Labov et al. 2006). Vowel dispersion is shown to correlate with both sex and social class. That men typically produce less distinct contrasts than women leads to predictions about the role of sex in phonetic-level sound change. Specifically, if the innovative form results in the loss of phonetic distinctiveness (such as two vowels merging), then we predict that men lead the change. An investigation of vowel mergers among the Atlas of North American English speakers reveals that men do lead mergers, and that speakers with a less dispersed vowel system show more instances of mergers, regardless of sex. Thus phonetic distinctiveness as an explanation of sound change accounts for why men lead sound changes that result in loss of phonetic distinctiveness, while women lead sound changes that maintain phonetic distinctiveness.

Kevin Heffernan

Papers

No Language Left Behind: Scaling Human-Centered Machine Translation

Evidence from HNR that /s/ is a social marker of gender

Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages

Phonetic similarity and phonemic contrast in loanword adaptation

Phonetic Distinctiveness as a Sociolinguistic Variable (2007)