An empirical study of smoothing techniques for language modeling

TLDR: This work surveys the most widely-used algorithms for smoothing models for language n -gram modeling, and presents an extensive empirical comparison of several of these smoothing techniques, including those described by Jelinek and Mercer (1980), and introduces methodologies for analyzing smoothing algorithm efficacy in detail.

About: This article is published in Computer Speech & Language.The article was published on 1999-10-01 and is currently open access. It has received 1948 citations till now. The article focuses on the topics: Kneser–Ney smoothing & Smoothing.

Figures

Figure 1: Performance relative to baseline algorithm jelinek-mercer-baseline of algorithms katz and jelinek-mercerwith respect to parameters and c min , respectively, over several training set sizes

Figure 14: Performance relative to baseline of witten-bell-backoff and witten-bell-interp algorithms on WSJ/NAB corpus, bigram and trigram models

Figure 22: Performance relative to baseline of jelinek-mercer and jelinek-mercer-delest algorithms on WSJ/NAB corpus, bigram and trigram models

Table 2: Summary of smoothing algorithms using notation from equation (24); the token \: : : " represents the term (w i 1

Table 3: Parameters that signi cantly a ect perplexity for each smoothing algorithm, and insignificant parameters and their default values

Figure 34: Performance relative to baseline of various algorithms on WSJ/NAB corpus, 4-gram and 5-gram models