Showing papers on "Bayes' theorem published in 1972"

Considerations of sample and feature size

Abstract: In many practical pattern-classification problems the underlying probability distributions are not completely known. Consequently, the classification logic must be determined on the basis of vector samples gathered for each class. Although it is common knowledge that the error rate on the design set is a biased estimate of the true error rate of the classifier, the amount of bias as a function of sample size per class and feature size has been an open question. In this paper, the design-set error rate for a two-class problem with multivariate normal distributions is derived as a function of the sample size per class (N) and dimensionality (L) . The design-set error rate is compared to both the corresponding Bayes error rate and the test-set error rate. It is demonstrated that the design-set error rate is an extremely biased estimate of either the Bayes or test-set error rate if the ratio of samples per class to dimensions (N/L) is less than three. Also the variance of the design-set error rate is approximated by a function that is bounded by 1/8N .

Limiting the Risk of Bayes and Empirical Bayes Estimators—Part II: The Empirical Bayes Case

Abstract: We discuss compromises between Stein's estimator and the MLE which limit the risk to individual components of the estimation problem while sacrificing only a small fraction of the savings in total squared error loss given by Stein's rule. The compromise estimators “limit translation” away from the MLE. The calculations are pursued in an empirical Bayesian manner by considering their performance against an entire family of prior distributions on the unknown parameters.

Convergence Rates for Empirical Bayes Two-Action Problems II. Continuous Case

Abstract: : A sequence of decision problems is considered where for each problem the observation has a probability density function of exponential type with parameter lambda where lambda is selected independently for each problem according to an unknown prior distribution G(lambda). It is supposed that in each of the problems, one of two possible actions (e.g., 'accept' or 'reject') must be taken. Under various assumptions, reasonably sharp upper bounds are found for the rate at which the risk of the nth problem approaches the smallest possible risk for certain refinements of the standard empirical Bayes procedures. For suitably chosen procedures, under situations likely to occur in practice, rates faster than n to the power (-1 + epsilon) may be obtained for arbitrarily small epsilon > 0. Arbitrarily slow rates can occur in pathological situations. (Author)

A Bayesian approach to bioassay.

Abstract: SUMMARY A prior distribution for the class of continuous, non-decreasing potency curves is introduced. The Bayes posterior distribution resulting from an assay experiment with quantal responses is discussed. Several examples are presented where a posterior modal function is used to summarize the posterior distribution. The examples illustrate the value of obtaining smooth estimates of potency and the value of experimental designs using many doses with few observations per dose.

Empirical Bayes methods and the repeated use of a standard

Abstract: SUMMARY The fact that more is generally known about a standard treatment in a clinical trial than about the test treatment is exploited in empirical Bayes estimates based on the results of using the same standard in other trials. Such estimates are proposed for the case of dicho- tomous response, and discussed in terms of an example in cancer research. Methods of design and analysis of experimental trials comparing a test treatment, T, with a standard treatment, S, usually consider the two treatments in a balanced and sym- metric way. However, the description of S as a standard suggests that it has been used many times before, and so prior to the experimental results more may be known about the general effectiveness of S than about T. If data from past trials in which S has been used are syste- matically tabulated, the empirical Bayes method is one way in which this information may be used. We consider this approach for the case of simple trials in which (i) the response is dichotomous, and (ii) an appropriate experimental analysis may be considered to involve separate estimations for the two treatments. Condition (ii) will be a reasonable approxima- tion if sample sizes are not too small. In ? 5, we discuss a set of chemotherapy data published by the Co-operative Breast Cancer Group in which a series of test compounds are each com- pared with testosterone propionate which is considered to be the standard treatment. Suppose that out of n patients in a clinical trial who are given S, x successes are observed, and let p be the corresponding response probability. We also suppose that there exist the results of m - 1 trials of a similar type carried out in the past, in which out of ni patients given

Some Admissible Empirical Bayes Procedures

Abstract: This paper considers some empirical Bayes procedures which have been discussed by H. Robbins, E. Samuel, and M. V. Johns. These procedures are shown to be inadmissible relative to a class of priors and by using some of the results of Rolph [6] admissible procedures are found for two examples. For an introduction to the empirical Bayes approach see Robbins [5].

Bayes, likelihood, or structural'

Abstract: The traditional model of statistics is examined in Section 1. The model, as such, distinguishes response values only by their likelihood functions. Recognition of this restricted identification effectively precludes the need for any theory of sufficient statistics. A probability space that has an attached observer-processor-reporter (OPR) mechanism is examined in Section 2 as a means of assessing the nature of reported information; such information may or may not be observational in character depending on properties of the OPR mechanism. A variation-response model is a probability space and a class of random variables: the probability space describes the sources of variation in the system under investigation and the class of random variables describes the possible presentations of this variation in the response of the system. Section 3 examines how realized values on the probability space are distinguished or identified by the model; Section 4 considers how distributions on the probability space are identified by response variable data. In Sections 5, 6, 7 the essentials of three contemporary approaches to inference are presented and each is accompanied by criticisms that proponents of the other methods might make. The key to these criticisms lies primarily in whether hypothetical information is added to or substantiated information is omitted from the assembled information concerning the system under investigation. In certain contexts the bayesian approach makes an arbitrary but typically consistent choice of input to its analyses; it is not the input suggested by standard invariance analysis. In those cases where a variation-response model is appropriate, the use of this more embracing model presents theoretical support for the bayesian choice (Section 8); but of course with the more embracing model the bayesian premises are not needed to obtain the usual bayesian result. An example in Section 9 illustrates the theoretical simplicity of classical probabilities for certain unknowns other than realized values on a probability space.

Nonparametric Bayes error estimation using unclassified samples

Abstract: The key measure of performance in a pattern recognition problem is the cost of making a decision. For the special case in which the relative cost of a correct decision is zero and the relative cost of an incorrect decision is unity, this cost is equal to the probability of an incorrect decision or error. A pattern recognition system may be viewed as a decision rule which transforms measurements into class assignments. The Bayes error is the minimum achievable error, where the minimization is with respect to all decision rules. The Bayes error is a function of the prior probabilities and the probability density functions of the respective classes. Unfortunately, in many applications, the probability density functions are unknown and therefore the Bayes error is unknown.

The Philosophy and Mathematics of Bayes' Equation

Abstract: The rudiments of Bayesian philosophy are introduced, and the mathematics of its application are surveyed; there is no uniformity of thought concerning either. The more extreme Bayesian philosophy, which allows subjective probabilities, is a means of plausible reasoning, or of making inferences, through inductive logic. Because inferences concerning reliability concepts are important in the decision process, this philosophy has a place in the reliability field. The difficulties of interpreting the probability function and of assigning prior distributions restrict the presentation of a unified philosophy. Thus, only techniques for describing prior probabilities under various circumstances can be given. Application of the Bayesian approach depends on how the reliability decision is conceptualized.

Symptom diagnosis: Bayes's theorem and Bahadur's distribution.

Abstract: The notion of Diagnostic Lexicon, developed by Rinaldo, Scheinok and Rupe (1963), was applied to a data base of 300 gastro-enterological patients having one of six radiologically determined diagnoses (Hiatal Hernia, Duodenal Ulcer, Gastric Ulcer, Cancer, Gallstones, and Functional Disease). Bayes's theorem was used as a diagnostic tool under the assumption that the 11 dichotomously valued symptoms considered had a dependent joint probability distribution proposed by R. R. Bahadur (1961), which contains the sum of all order correlation coefficients. Examination of the Lexicon revealed that the superposition of Bayes's Theorem over Bahadur's Distribution (which was chosen for its utmost generality) led to posterior probabilities, equal to the original frequencies of occurrence of the diagnoses for each individual patient profile. This is testimony to the utility of the Lexicon approach, since it is not likely that such a fact could have come to light otherwise. In conclusion, a mathematical proof of this strange result for the case of two diagnoses and two symptoms is given.

An Engineer's Guide to Building Nonlinear Filters. Volume 1

Abstract: : A comprehensive treatment of numerical approaches to the solution of Bayes Law has been included, describing numerical methods, computational algorithms, two example problems, and extensive numerical results. Bayes Law is an integral equation describing the evolution of the conditional probability distribution, describing the state of a Markov process, conditioned on the past noisy observations. The Bayes Law is, in fact, the general solution to the discrete nonlinear estimation problem. This research represents one of the first successful attempts to approximate the conditional probability densities numerically and evaluate the Bayes integral by quadratures. The methods of density representation studied most thoroughly include orthogonal polynomials, point-masses, gaussian sums, and Fourier series.

Use of Bayes's Theorem in clinical decision. Suicidal risk, differential diagnosis, response to treatment.

Abstract: Methodological similarities between Bayes's Theorem and clinical diagnosis are pointed out. Three `cook-book' examples of the application of the theorem are given and discussed, and comments are made on the use of the method.

Bayesian Concepts for the Estimation of Reliability in the Weibull Life-testing Model

Abstract: Since in many life-testing situations it is not unlikely to note the unpredictable fluctuation of the scale parameter in a failure model, it is justifiable to consider such a parameter as a random variable and, thus, appeal to a Bayesian analysis. In specific, let 0 denote the random variable associated with the scale parameter and 0 its realization. Obviously, a Bayesian analysis depends on the utilization of prior information which, in this case, we assume to exist either in the form of a prior distribution of 0 or a sequence of sufficient statistics from past experiments. For the ordinary Bayes approach, we appeal to a well-known transformation to generalize the results of Bhattacharya (1967) for the one-parameter exponential model so as to include the flexibility provided by the shape parameter ? in the Weibull distribution. In fact, the Weibull failure model has an increasing ( > 1) or decreasing ( < 1) failure rate and, thus, is likely to describe the life-span of items with variable failure rates. The empirical Bayes estimation technique was largely motivated by Robbins (1955), who assumed the existence of a prior distribution for an unknown parameter but not the knowledge of its form. Instead, he substitutes past information which he assumes to exist as a result of the repetitive nature in the problem of estimation. Thus, in the absence of knowledge concerning the form of the prior distribution, we appeal to an empirical Bayes approach to estimate the scale parameter. By using this estimate, an estimate of the reliability function is made possible.

Empirical Bayes Point Estimation in a Family of Probability Distributions

Abstract: The empirical Bayes approach has been described in detail in the literature [1], [2], [3], [6], thus the problem will only be briefly summarized here. Let X be a random variable whose probability distribution depends in a known way on an unknown real parameter 0, with 0 itself being a random variable with unknown a priori distribution function G (0). The aim is to estimate 0 on the basis of the observed value x, which may be vector valued, so that the estimator has small squared error. The problem presents itself repeatedly and independently with the same unknown probability distribution function G (0) and a known family of distribution functions {F (x I 0): 0 e 0), 0 being the set of all possible values of 0. It is well known that for a squared-error loss function the Bayes estimator is the mean of the posterior probability distribution, that is, 0 = E (0 I x). It has been demonstrated by Rutherford and Krutchkoff [4], that if the prior has a known bound for any moment higher than the second, then one can obtain e-asymptotic optimality by a method of truncating a consistent sequence of estimators for the Bayes estimator.

Bayes' controllers with memory for a linear system with jump parameters

Abstract: The Bayes' regulator with perfect memory is derived for a linear stochastic plant with an incomplete probabilistic description. Because of the complexity of the equations characterizing the optimal control, two suboptimal approximations are also given. It is shown that under appropriate conditions, one of the suboptimal controllers is better than the best zero memory regulator.

A Bayes Analysis of Availability for a System Consisting of Several Independent Subsystems

Abstract: A Bayes analysis of system availability A is carried out on the basis of snapshot data obtained on each of the N component subsystems Application of Bayes formula, using a natural conjugate prior probability density function (pdf), yields the posterior pdf of subsystem availability Determination of the posterior pdf of system availability requires the derivation of the pdf of the product of N independent random variables, which is accomplished through the use of the Mellin integral transform From the knowledge of the posterior pdf of A, confidence limits on system availability can be obtained

Simultaneous Estimation of Parameters

Abstract: : The paper provides an elementary introduction to Stein's estimator and its relation to compound Bayes and empirical Bayes estimation methods. (Author)

A Bayesian Approach to Parameter and Reliability Estimation in the Poisson Distribution

Abstract: For life testing procedures, a Bayesian analysis is developed with respect to a random intensity parameter in the Poisson distribution. Bayes estimators are derived for the Poisson parameter and the reliability function based on uniform and gamma prior distributions of that parameter. A Monte Carlo procedure is implemented to make possible an empirical mean-squared error comparison between Bayes and existing minimum variance unbiased, as well as maximum likelihood, estimators. As expected, the Bayes estimators have mean-squared errors that are appreciably smaller than those of the other two.

An Empirical Bayes Approach to Reliability

Abstract: A Bayesian reliability estimation technique known as the ``empirical Bayes approach'' is developed which uses previous experience nce to get a Bayesian point estimator. The techniques require no knowledge of the form of the unknown prior distribution and are robust to assumptions about its form. Empirical Bayes techniques are applicable to situations in which prior, independent observations of the random variable X from the random couple (?, X) are available where ? is the observed parameter of interest distributed in accordance with the unknown prior distribution. Performance comparisons of the empirical Bayes and other well established techniques are developed by examples for the binomial, exponential, Normal, and Poisson situations which often occur in reliability problems. In all cases the empirical Bayes estimator performed better than the classical estimator in minimizing the average squared error.

A Bayes Sampling Allocation Scheme for Stratified Finite Populations With Hyperbinomial Prior Distributions

Abstract: When sampling is carried out independently for the K strata of a finite stratified dichotomous population (defectives vs. standard items), and the number Zi of defectives per stratum sample is observed, the corresponding probability function for X = (Xi , …, xK ) is the product of hypergeometric functions which depend on the sample sizes ni , the stratum sizes Ni , and the number of defectives Mi in the stratum (i = 1, …, K). It is assumed that prior information is available about the Mi 's which can be expressed, by suitable choice of the parameters ai and bi , as the product of independent hyperbinomial functions. In each stratum the cost per observation is a known constant. Using squared error loss function, the prior Bayes risk is found for the linear function of interest, and the optimum allocation of sample sizes is found, the one for which the prior Bayes risk is minimum when the total sampling budget is fixed.

Continuous Sequential Testing of a Poisson Process to Minimize the Bayes Risk

Abstract: A method of testing constant failure rates is presented, where the distribution of failure times is assumed to be exponential. The failure rate is unknown but two alternative failure rates are hypothesized. The objective of the procedure is to minimize the decision cost, and observational costs accumulated during the test. A Bayes procedure is found whereby the prior probability that one of the failure rates is true is updated until it reaches a previously-determined decision point. The exact solutions and approximate solutions for the decision points for the minimum Bayes Risk are provided for the general renewal case.

Bayes adaptive control of two-echelon inventory systems i: development for a special case of one-station lower echelon and monte carlo evaluation

Abstract: Bayes adaptive control policies are developed in the present paper for the special case of a one-station lower echelon: a Poisson distribution of demand, whose mean is assumed to have a prior gamma distribution. The cost structure is of a common type. The ordering policy for the upper echelon, which minimizes expected cost, is replaced by a new type of policy, called Bayes prediction policy. This policy does not require tedious computations, of the sort required by dynamic programming solutions. The characteristics of the policies are studied by Monte Carlo simulation, and supplemented by further theoretical development.