Showing papers on "Probability distribution published in 1990"

Probability: Theory and Examples

Abstract: This book is an introduction to probability theory covering laws of large numbers, central limit theorems, random walks, martingales, Markov chains, ergodic theorems, and Brownian motion. It is a comprehensive treatment concentrating on the results that are the most useful for applications. Its philosophy is that the best way to learn probability is to see it in action, so there are 200 examples and 450 problems.

New criteria for blind deconvolution of nonminimum phase systems (channels)

Abstract: A necessary and sufficient condition for blind deconvolution (without observing the input) of nonminimum-phase linear time-invariant systems (channels) is derived. Based on this condition, several optimization criteria are proposed, and their solution is shown to correspond to the desired response. These criteria involve the computation only of second- and fourth-order moments, implying a simple tap update procedure. The proposed methods are universal in the sense that they do not impose any restrictions on the probability distribution of the (unobserved) input sequence. It is shown that in several important cases (e.g. when the additive noise is Gaussian), the proposed criteria are essentially unaffected. >

Sequential updating of conditional probabilities on directed graphical structures

Abstract: A directed acyclic graph or influence diagram is frequently used as a representation for qualitative knowledge in some domains in which expert system techniques have been applied, and conditional probability tables on appropriate sets of variables form the quantitative part of the accumulated experience. It is shown how one can introduce imprecision into such probabilities as a data base of cases accumulates. By exploiting the graphical structure, the updating can be performed locally, either approximately or exactly, and the setup makes it possible to take advantage of a range of well-established statistical techniques. As examples we discuss discrete models, models based on Dirichlet distributions and models of the logistic regression type.

Boundary detection by constrained optimization

Abstract: A statistical framework is used for finding boundaries and for partitioning scenes into homogeneous regions. The model is a joint probability distribution for the array of pixel gray levels and an array of labels. In boundary finding, the labels are binary, zero, or one, representing the absence or presence of boundary elements. In partitioning, the label values are generic: two labels are the same when the corresponding scene locations are considered to belong to the same region. The distribution incorporates a measure of disparity between certain spatial features of block pairs of pixel gray levels, using the Kolmogorov-Smirnov nonparametric measures of difference between the distributions of these features. The number of model parameters is minimized by forbidding label configurations, which are assigned probability zero. The maximum a posteriori estimator of boundary placements and partitionings is examined. The forbidden states introduce constraints into the calculation of these configurations. Stochastic relaxation methods are extended to accommodate constrained optimization. >

Multiscaling properties of spatial rainfall and river flow distributions

Abstract: Two common properties of empirical moments shared by spatial rainfall, river flows, and turbulent velocities are identified: namely, the log-log linearity of moments with spatial scale and the concavity of corresponding slopes with respect to the order of the moments. A general class of continuous multiplicative processes is constructed to provide a theoretical framework for these observations. Specifically, the class of log-Levy-stable processes, which includes the lognormal as a special case, is analyzed. This analysis builds on some mathematical results for simple scaling processes. The general class of multiplicative processes is shown to be characterized by an invariance property of their probability distributions with respect to rescaling by a positive random function of the scale parameter. It is referred to as (strict sense) multiscaling. This theory provides a foundation for studying spatial variability in a variety of hydrologic processes across a broad range of scales.

Statistical mechanics and phase transitions in clustering.

Abstract: A new approach to clustering based on statistical physics is presented. The problem is formulated as fuzzy clustering and the association probability distribution is obtained by maximizing the entropy at a given average variance. The corresponding Lagrange multiplier is related to the ``temperature'' and motivates a deterministic annealing process where the free energy is minimized at each temperature. Critical temperatures are derived for phase transitions when existing clusters split. It is a hierarchical clustering estimating the most probable cluster parameters at various average variances.

Advanced probabilistic structural analysis method for implicit performance functions

Abstract: In probabilistic structural analysis, the performance or response functions usually are implicitly defined and must be solved by numerical analysis methods such as finite-elemen t methods. In such cases, the commonly used probabilistic analysis tool is the mean-based second-moment method, which provides only the first two statistical moments. This paper presents an advanced mean-based method, which is capable of establishing the full probability distributions to provide additional information for reliability design. The method requires slightly more computations than the mean-based second-moment method but is highly efficient relative to the other alternative methods. Several examples are presented to demonstrate the method. In particular, the examples show that the new mean-based method can be used to solve problems involving nonmonotonic functions that result in truncated distributions.

Links between Markov models and multilayer perceptrons

Abstract: The statistical use of a particular classic form of a connectionist system, the multilayer perceptron (MLP), is described in the context of the recognition of continuous speech. A discriminant hidden Markov model (HMM) is defined, and it is shown how a particular MLP with contextual and extra feedback input units can be considered as a general form of such a Markov model. A link between these discriminant HMMs, trained along the Viterbi algorithm, and any other approach based on least mean square minimization of an error function (LMSE) is established. It is shown theoretically and experimentally that the outputs of the MLP (when trained along the LMSE or the entropy criterion) approximate the probability distribution over output classes conditioned on the input, i.e. the maximum a posteriori probabilities. Results of a series of speech recognition experiments are reported. The possibility of embedding MLP into HMM is described. Relations with other recurrent networks are also explained. >

Incomplete Data in Generalized Linear Models

Abstract: This article examines incomplete data for the class of generalized linear models, in which incompleteness is due to partially missing covariates on some observations. Under the assumption that the missing data are missing at random, it is shown that the E step of the EM algorithm for any generalized linear model can be expressed as a weighted complete data log-likelihood when the unobserved covariates are assumed to come from a discrete distribution with finite range. Expressing the E step in this manner allows for a straightforward maximization in the M step, thus leading to maximum likelihood estimates (MLE's) for the parameters. Asymptotic variances of the MLE's are also derived, and results are illustrated with two examples.

Turbulence and combustion

Abstract: Intermittency and the qualitative form of probability distribution densities in turbulent flows equations for probability distribution densities passive contaminant concentration probability distribution statistical characteristics of small-scale turbulence turbulent combustion of a homogenous mixture.

Simple construction of almost k-wise independent random variables

Abstract: The authors present three alternative simple constructions of small probability spaces on n bits for which any k bits are almost independent. The number of bits used to specify a point in the sample space is O(log log n+k+log 1/ epsilon ), where epsilon is the statistical difference between the distribution induced on any k-bit locations and the uniform distribution. This is asymptotically comparable to the construction recently presented by J. Naor and M. Naor (1990). An advantage of the present constructions is their simplicity. Two of the constructions are based on bit sequences that are widely believed to possess randomness properties, and the results can be viewed as an explanation and establishment of these beliefs. >

Distribution of quadratic forms in normal space-application to structural reliability

Abstract: In second-order reliability methods the failure surface in the standard normal space is approximated by a parabolic surface at the design point. The corresponding probability is computed by asymptotic formulas and by approximation formulas. In this paper the probability content of the parabolic failure domain is computed exactly by inversion of the characteristic function for the parabolic quadratic form. Also, the exact results for the probability content of the failure domain obtained from the full second-order Taylor expansion of the failure function at the design point is presented. The approximating parabola does not depend on the formulation of the failure function as long as this preserves the original failure surface. This invariance characteristic is in general not shared by the approximation obtained using the full second-order Taylor expansion of the failure function at the design point. The exact results for the probability content of the approximating quadratic domains significantly extend the class of problems that can be treated by approximate methods.

Tied mixture continuous parameter modeling for speech recognition

Abstract: The acoustic-modeling problem in automatic speech recognition is examined with the goal of unifying discrete and continuous parameter approaches. To model a sequence of information-bearing acoustic feature vectors which has been extracted from the speech waveform via some appropriate front-end signal processing, a speech recognizer basically faces two alternatives: (1) assign a multivariate probability distribution directly to the stream of vectors, or (2) use a time-synchronous labeling acoustic processor to perform vector quantization on this stream, and assign a multinomial probability distribution to the output of the vector quantizer. With a few exceptions, these two methods have traditionally been given separate treatment. A class of very general hidden Markov models which can accommodate feature vector sequences lying either in a discrete or in a continuous space is considered; the new class allows one to represent the prototypes in an assumption-limited, yet convenient way, as tied mixtures of simple multivariate densities. Speech recognition experiments, reported for two (5000- and 20000-word vocabulary) office correspondence tasks, demonstrate some of the benefits associated with this technique. >

Experimental study of critical-mass fluctuations in an evolving sandpile.

Abstract: We have carried out an experiment in which sandpiles are built up to a critical size and then perturbed by the dropping of individual grains of sand onto the pile. After each grain is added, the size of the resulting avalanche, if any, is recorded. For sufficiently small sandpiles, the observed mass fluctuations are scale invariant and the probability distribution of avalanches shows finite-size scaling. This demonstrates that real, finite-size sandpiles may be described by models of self-organized criticality. However, we also find that this description breaks down in the limit of large sandpiles.

Transforming Neural-Net Output Levels to Probability Distributions

Abstract: (1) The outputs of a typical multi-output classification network do not satisfy the axioms of probability; probabilities should be positive and sum to one. This problem can be solved by treating the trained network as a preprocessor that produces a feature vector that can be further processed, for instance by classical statistical estimation techniques. (2) We present a method for computing the first two moments of the probability distribution indicating the range of outputs that are consistent with the input and the training data. It is particularly useful to combine these two ideas: we implement the ideas of section 1 using Parzen windows, where the shape and relative size of each window is computed using the ideas of section 2. This allows us to make contact between important theoretical ideas (e.g. the ensemble formalism) and practical techniques (e.g. back-prop). Our results also shed new light on and generalize the well-known "softmax" scheme.

On the Behavior of Randomization Tests without a Group Invariance Assumption

Abstract: Fisher's randomization construction of hypothesis tests is a powerful tool to yield tests that are nonparametric in nature in that their level is exactly equal to the nominal level in finite samples over a wide range of distributional assumptions. For example, the usual permutation t test to test equality of means is valid without a normality assumption of the underlying populations. On the other hand, Fisher's randomization construction is not applicable in this example unless the underlying populations differ only in location. In general, the basis for the randomization construction is invariance of the probability distribution of the data under a transformation group. It is the goal of this article to understand the robustness properties of randomization tests by studying their asymptotic validity in situations where the basis for their construction breaks down. Here, asymptotic validity refers to whether the probability of a Type I error tends asymptotically to the nominal level. In particula...

Numerical methods for modeling computer networks under nonstationary conditions

Abstract: Numerical techniques for modeling computer networks under nonstationary conditions are discussed, and two distinct approaches are presented. The first approach uses a queuing theory formulation to develop differential equation models which describe the behavior of the network by time-varying probability distributions. In the second approach, a nonlinear differential equation model is developed for representing the dynamics of the network in terms of time-varying mean quantities. This approach allows multiple classes of traffic to be modeled and establishes a framework for the use of optimal control techniques in the design of network control strategies. Numerical techniques for determining the queue behavior as a function of time for both approaches are discussed and their computational advantages are contrasted with simulation. >

Graded-response neurons and information encodings in autoassociative memories.

Abstract: A general mean-field theory is presented for an attractor neural network in which each elementary unit is described by one input and one output real variable, and whose synaptic strengths are determined by a covariance imprinting rule. In the case of threshold-linear units, a single equation is shown to yield the storage capacity for the retrieval of random activity patterns drawn from any given probability distribution. If this distribution produces binary patterns, the storage capacity is essentially the same as for networks of binary units. To explore the effects of storing more structured patterns, the case of a ternary distribution is studied. It is shown that the number of patterns that can be stored can be much higher than in the binary case, whereas the total amount of retrievable information does not exceed the limit obtained with binary patterns.

Abelian sandpile model on the bethe lattice

Abstract: The authors study Bak, Tang and Wiesenfeld's Abelian sandpile model (1987) of self-organised criticality on the Bethe lattice. Exact expressions for various distribution functions including the height distribution at a site and the joint distribution of heights at two sites separated by an arbitrary distance are obtained. They also determine the probability distribution of the number of distinct sites that topple at least once, the number of toplings at the origin and the total number of toplings in an avalanche. The probability that an avalanche consists of more than n toplings varies as n-1/2 for large n. The probability that its duration exceeds T decreases as 1/T for large T. These exponents are the same as for the critical percolation clusters in mean field theory.

Theoretical Population Genetics

Abstract: Wright-Fisher, Moran and other models on the description of changes in allele frequency survival of new mutations, branching processes probability of fixation, the more general case notes on continuous approximations mean sojourn, absorption and fixation times introduction to probability distributions, probability flux stationary distributions, frequency spectra diffusion methods general comments and conclusions.

A new probabilistic power flow analysis method

Abstract: A method for the simulation of the composite power system is proposed for the purpose of evaluating the probability distribution function of circuit flows and bus voltage magnitudes. The method consists of two steps. First, given the probabilistic electric load model, the probability distribution function of the total generation of generation buses is computed. Second, circuit flows and bus voltage magnitudes are expressed as linear combinations of power injections at generation buses. This relationship allows the computation of the distribution functions of circuit flows and bus voltage magnitudes. The method incorporates major operating practices such as economic dispatch and nonlinearities resulting from the power flow equations. Validation of the method is performed via Monte Carlo simulation. Typical results are presented, showing that the proposed method matches the results obtained with the Monte Carlo simulations very well. Potential applications of the proposed method are: composite power system reliability analysis and transmission loss evaluation. >

Fluctuation analysis: the probability distribution of the number of mutants under different conditions

Abstract: In the 47 years since fluctuation analysis was introduced by Luria and Delbruck, it has been widely used to calculate mutation rates. Up to now, in spite of the importance of such calculations, the probability distribution of the number of mutants that will appear in a fluctuation experiment has been known only under the restrictive, and possibly unrealistic, assumptions: (1) that the mutation rate is exactly proportional to the growth rate and (2) that all mutants grow at a rate that is a constant multiple of the growth rate of the original cells. In this paper, we approach the distribution of the number of mutants from a new point of view that will enable researchers to calculate the distribution to be expected using assumptions that they believe to be closer to biological reality. The new idea is to classify mutations according to the number of observable mutants that derive from the mutation when the culture is selectively plated. This approach also simplifies the calculations in situations where two, or many, kinds of mutation may occur in a single culture.

Dual method for the solution of a one-stage stochastic programming problem with random RHS obeying a discrete probability distribution

Abstract: In this paper we present a method for the solution of a one stage stochastic programming problem, where the underlying problem is an LP and some of the right hand side values are random variables. The stochastic programming problem that we formulate contains probabilistic constraint and penalty, incorporated into the objective function, used to penalize violation of the stochastic constraints. We solve this problem by a dual type algorithm. The special case where only penalty is used while the probabilistic constraint is disregarded, the simple recourse problem, was solved earlier by Wets, using a primal simplex algorithm with individual upper bounds. Our method appears to be simpler. The method has applications to nonstochastic programming problems too, e.g., it solves the constrained minimum absolute deviation problem.

Statistical approach to the geometric structure of thermodynamics.

Abstract: We show how both the contact structure and the metric structure of the thermodynamic phase space arise in a natural way from a generalized canonical probability distribution \ensuremath{\rho} In particular, the metric form and the contact form are found to be derived from the microscopic entropy s=-ln\ensuremath{\rho} Thus the first law and the second law of thermodynamics can be given the geometric interpretation that a thermodynamic system must possess both a contact and a compatible metric structure We proceed to construct explicitly a new nondegenerate bilinear form on the thermodynamic phase space, whose restriction to state space yields the Weinhold-Ruppeiner metric, and whose restriction to Gibbs space can serve as an alternative to the metric proposed by Gilmore

Effective medium theory and network theory applied to the transport properties of rock

Abstract: The effective medium theory (EMT) proposed by Kirkpatrick (1973) is known to be a useful tool to describe the conducting properties of heterogeneous media. We check the validity of the theory on several probability distribution functions. The accuracy of the theory is inferred by comparing the results given by the EMT to those obtained when using an exact method based on the resolution of two-dimensional conducting networks. It turns out that the EMT approximation is quite good for quasi-uniform distributions of conducting elements, whereas large discrepancies appear for highly contrasted distributions such as decreasing exponential functions. The reason is that ensemble averaging and spatial averaging are not equivalent for broad, nonuniform distributions. Therefore the effective medium theory may not be appropriate for the calculation of the transport properties of rocks which exhibit this type of distribution. The nature of the distribution varies, however, depending on the rock type and the length scale of interest. Critical path analysis can be a complementary tool to deal with highly heterogeneous conducting media. The accuracy of such a technique appears to be better than that of the EMT approximation. But it depends also on the representativity of the input data. Finally, an example of data relative to Fontainebleau Sandstone is developed in this paper. It is concluded that for this rock, network theory is the most accurate tool for calculating transport properties from statistically averaged observations on two-dimensional sections.

Structure-factor probabilities for related structures

Abstract: Probability relationships between structure factors from related structures have allowed previously only for either differences in atomic scattering factors (isomorphous replacement case) or differences in atomic positions (coordinate error case). In the coordinate error case, only errors drawn from a single probability distribution have been considered, in spite of the fact that errors vary widely through models of macromolecular structures. It is shown that the probability relationships can be extended to cover more general cases. Either the atomic parameters or the reciprocal-space vectors may be chosen as the random variables to derive probability relationships. However, the relationships turn out to be very similar for either choice. The most intuitive is the expected electron-density formalism, which arises from considering the atomic parameters as random variables. In this case, the centroid of the structure-factor distribution is the Fourier transform of the expected electron-density function, which is obtained by smearing each atom over its possible positions. The centroid estimate has a phase different from, and more accurate than, that obtained from the unweighted atoms. The assumption that there is a sufficient number of independent errors allows the application of the central limit theorem. This gives a one- (centric case) or two-dimensional (non-centric) Gaussian distribution about the centroid estimate. The general probability expression reduces to those derived previously when the appropriate simplifying assumptions are made. The revised theory has implications for calculating more accurate phases and maps, optimizing molecular replacement models, refining structures, estimating coordinate errors and interpreting refined B factors.

Multifractal properties of snapshot attractors of random maps.

Abstract: We consider qualitative and quantitative properties of ``snapshot attractors'' of random maps. By a random map we mean that the parameters that occur in the map vary randomly from iteration to iteration according to some probability distribution. By a ``snapshot attractor'' we mean the measure resulting from many iterations of a cloud of initial conditions viewed at a single instant (i.e., iteration). In this paper we investigate the multifractal properties of these snapshot attractors. In particular, we use the Lyapunov number partition function method to calculate the spectra of generalized dimensions and of scaling indices for these attractors; special attention is devoted to the numerical implementation of the method and the evaluation of statistical errors due to the finite number of sample orbits. This work was motivated by problems in the convection of particles by chaotic fluid flows.

General entropy criteria for inverse problems, with applications to data compression, pattern classification, and cluster analysis

Abstract: Minimum distance approaches are considered for the reconstruction of a real function from finitely many linear functional values. An optimal class of distances satisfying an orthogonality condition analogous to that enjoyed by linear projections in Hilbert space is derived. These optimal distances are related to measures of distances between probability distributions recently introduced by C.R. Rao and T.K. Nayak (1985) and possess the geometric properties of cross entropy useful in speech and image compression, pattern classification, and cluster analysis. Several examples from spectrum estimation and image processing are discussed. >

Entropy splitting for antiblocking corners and -perfect graphs

Abstract: We characterize pairs of convex setsA, B in thek-dimensional space with the property that every probability distribution (p1,...,p k ) has a repsesentationp i =a l .b i , a∃A, b∃B.