Probability mass function

About: Probability mass function is a research topic. Over the lifetime, 2853 publications have been published within this topic receiving 101710 citations. The topic is also known as: pmf.

The Curious Case of Neural Text Degeneration

Abstract: Despite considerable advances in neural language modeling, it remains an open question what the best decoding strategy is for text generation from a language model (e.g. to generate a story). The counter-intuitive empirical observation is that even though the use of likelihood as training objective leads to high quality models for a broad range of language understanding tasks, maximization-based decoding methods such as beam search lead to degeneration — output text that is bland, incoherent, or gets stuck in repetitive loops. To address this we propose Nucleus Sampling, a simple but effective method to draw considerably higher quality text out of neural language models. Our approach avoids text degeneration by truncating the unreliable tail of the probability distribution, sampling from the dynamic nucleus of tokens containing the vast majority of the probability mass. To properly examine current maximization-based and stochastic decoding methods, we compare generations from each of these methods to the distribution of human text along several axes such as likelihood, diversity, and repetition. Our results show that (1) maximization is an inappropriate decoding objective for open-ended text generation, (2) the probability distributions of the best current language models have an unreliable tail which needs to be truncated during generation and (3) Nucleus Sampling is the best decoding strategy for generating long-form text that is both high-quality — as measured by human evaluation — and as diverse as human-written text.

Scenario reduction in stochastic programming: An approach using probability metrics

Abstract: Given a convex stochastic programming problem with a discrete initial probability distribution, the problem of optimal scenario reduction is stated as follows: Determine a scenario subset of prescribed cardinality and a probability measure based on this set that is the closest to the initial distribution in terms of a natural (or canonical) probability metric. Arguments from stability analysis indicate that Fortet-Mourier type probability metrics may serve as such canonical metrics. Efficient algorithms are developed that determine optimal reduced measures approximately. Numerical experience is reported for reductions of electrical load scenario trees for power management under uncertainty. For instance, it turns out that after 50% reduction of the scenario tree the optimal reduced tree still has about 90% relative accuracy.

Non-Normal Dependent Vectors in Structural Safety

Abstract: A general probability distribution transformation is developed with which complex structural reliability problems involving non-normal, dependent uncertainty vectors can be reduced to the standard case of first-order-reliability, i.e. the problem of determining the failure probability or the reliability index isn the space of independent, standard normal variates. The method requires the knowledge of the joint cumulative distribution function or a certain set of conditional distribution functions of the original vector. Some basic properties of the transformation are discussed. Details of the transformation technique are given. Approximations must be introduced for the shape of the safe domain such that its probability content can easily be evaluated which may involve numerical inversion of distribution functions. A suitable algorithm for computing reliability measures is proposed. The field of potential applications is indicated by a number of examples.

On the composition of elementary errors

Abstract: 1. By a variable in the sense of the Theory of Probability we mean a quantity z, which may assume certain real values with certain probahilities. We shall call V(t) the probability function of z if, for every real t, V(t) is equal to the probabiliby that z has a value < t, increased by half the probability that z has exactly the value t.

On the First Passage Time Probability Problem

Abstract: We have derived an exact solution for the first passage time probability of a stationary one-dimensional Markoffian random function from an integral equation. A recursion formula for the moments is given for the case that the conditional probability density describing the random function satisfies a Fokker-Planck equation. Various known solutions for special applications (noise, Brownian motion) are shown to be special cases of our solution. The Wiener-Rice series for the recurrence time probability density is derived from a generalization of Schr\"odinger's integral equation, for the case of a two-dimensional Markoffian random function.