Tests with Repeated Observations

The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization. The treatment focuses on basic unifying themes, and conceptual foundations. It illustrates the versatility, power, and generality of the method with many examples and applications from engineering, operations research, and other fields. It also addresses extensively the practical application of the methodology, possibly through the use of approximations, and provides an extensive treatment of the far-reaching methodology of Neuro-Dynamic Programming/Reinforcement Learning.

/pdf/dynamic-programming-and-optimal-control-21dopm2oo5.pdf

Dynamic Programming and Optimal Control

Information Theory and Statistics

LDP for Finite Dimensional Spaces.- Applications-The Finite Dimensional Case.- General Principles.- Sample Path Large Deviations.- The LDP for Abstract Empirical Measures.- Applications of Empirical Measures LDP.

Large Deviations Techniques and Applications

Foreword.Preface.PART ONE. SURVEY OF PROBABILITY THEORY.Chapter 1. Introduction.Chapter 2. Experiments, Sample Spaces, and Probability.2.1 Experiments and Sample Spaces.2.2 Set Theory.2.3 Events and Probability.2.4 Conditional Probability.2.5 Binomial Coefficients.Exercises.Chapter 3. Random Variables, Random Vectors, and Distributions Functions.3.1 Random Variables and Their Distributions.3.2 Multivariate Distributions.3.3 Sums and Integrals.3.4 Marginal Distributions and Independence.3.5 Vectors and Matrices.3.6 Expectations, Moments, and Characteristic Functions.3.7 Transformations of Random Variables.3.8 Conditional Distributions.Exercises.Chapter 4. Some Special Univariate Distributions.4.1 Introduction.4.2 The Bernoulli Distributions.4.3 The Binomial Distribution.4.4 The Poisson Distribution.4.5 The Negative Binomial Distribution.4.6 The Hypergeometric Distribution.4.7 The Normal Distribution.4.8 The Gamma Distribution.4.9 The Beta Distribution.4.10 The Uniform Distribution.4.11 The Pareto Distribution.4.12 The t Distribution.4.13 The F Distribution.Exercises.Chapter 5. Some Special Multivariate Distributions.5.1 Introduction.5.2 The Multinomial Distribution.5.3 The Dirichlet Distribution.5.4 The Multivariate Normal Distribution.5.5 The Wishart Distribution.5.6 The Multivariate t Distribution.5.7 The Bilateral Bivariate Pareto Distribution.Exercises.PART TWO. SUBJECTIVE PROBABILITY AND UTILITY.Chapter 6. Subjective Probability.6.1 Introduction.6.2 Relative Likelihood.6.3 The Auxiliary Experiment.6.4 Construction of the Probability Distribution.6.5 Verification of the Properties of a Probability Distribution.6.6 Conditional Likelihoods.Exercises.Chapter 7. Utility.7.1 Preferences Among Rewards.7.2 Preferences Among Probability Distributions.7.3 The Definitions of a Utility Function.7.4 Some Properties of Utility Functions.7.5 The Utility of Monetary Rewards.7.6 Convex and Concave Utility Functions.7.7 The Anxiomatic Development of Utility.7.8 Construction of the Utility Function.7.9 Verification of the Properties of a Utility Function.7.10 Extension of the Properties of a Utility Function to the Class ?E.Exercises.PART THREE. STATISTICAL DECISION PROBLEMS.Chapter 8. Decision Problems.8.1 Elements of a Decision Problem.8.2 Bayes Risk and Bayes Decisions.8.3 Nonnegative Loss Functions.8.4 Concavity of the Bayes Risk.8.5 Randomization and Mixed Decisions.8.6 Convex Sets.8.7 Decision Problems in Which ~2 and D Are Finite.8.8 Decision Problems with Observations.8.9 Construction of Bayes Decision Functions.8.10 The Cost of Observation.8.11 Statistical Decision Problems in Which Both ? and D contains Two Points.8.12 Computation of the Posterior Distribution When the Observations Are Made in More Than One Stage.Exercises.Chapter 9. Conjugate Prior Distributions.9.1 Sufficient Statistics.9.2 Conjugate Families of Distributions.9.3 Construction of the Conjugate Family.9.4 Conjugate Families for Samples from Various Standard Distributions.9.5 Conjugate Families for Samples from a Normal Distribution.9.6 Sampling from a Normal Distribution with Unknown Mean and Unknown Precision.9.7 Sampling from a Uniform Distribution.9.8 A Conjugate Family for Multinomial Observations.9.9 Conjugate Families for Samples from a Multivariate Normal Distribution.9.10 Multivariate Normal Distributions with Unknown Mean Vector and Unknown Precision matrix.9.11 The Marginal Distribution of the Mean Vector.9.12 The Distribution of a Correlation.9.13 Precision Matrices Having an Unknown Factor.Exercises.Chapter 10. Limiting Posterior Distributions.10.1 Improper Prior Distributions.10.2 Improper Prior Distributions for Samples from a Normal Distribution.10.3 Improper Prior Distributions for Samples from a Multivariate Normal Distribution.10.4 Precise Measurement.10.5 Convergence of Posterior Distributions.10.6 Supercontinuity.10.7 Solutions of the Likelihood Equation.10.8 Convergence of Supercontinuous Functions.10.9 Limiting Properties of the Likelihood Function.10.10 Normal Approximation to the Posterior Distribution.10.11 Approximation for Vector Parameters.10.12 Posterior Ratios.Exercises.Chapter 11. Estimation, Testing Hypotheses, and linear Statistical Models.11.1 Estimation.11.2 Quadratic Loss.11.3 Loss Proportional to the Absolute Value of the Error.11.4 Estimation of a Vector.11.5 Problems of Testing Hypotheses.11.6 Testing a Simple Hypothesis About the Mean of a Normal Distribution.11.7 Testing Hypotheses about the Mean of a Normal Distribution.11.8 Deciding Whether a Parameter Is Smaller or larger Than a Specific Value.11.9 Deciding Whether the Mean of a Normal Distribution Is Smaller or larger Than a Specific Value.11.10 Linear Models.11.11 Testing Hypotheses in Linear Models.11.12 Investigating the Hypothesis That Certain Regression Coefficients Vanish.11.13 One-Way Analysis of Variance.Exercises.PART FOUR. SEQUENTIAL DECISIONS.Chapter 12. Sequential Sampling.12.1 Gains from Sequential Sampling.12.2 Sequential Decision Procedures.12.3 The Risk of a Sequential Decision Procedure.12.4 Backward Induction.12.5 Optimal Bounded Sequential Decision procedures.12.6 Illustrative Examples.12.7 Unbounded Sequential Decision Procedures.12.8 Regular Sequential Decision Procedures.12.9 Existence of an Optimal Procedure.12.10 Approximating an Optimal Procedure by Bounded Procedures.12.11 Regions for Continuing or Terminating Sampling.12.12 The Functional Equation.12.13 Approximations and Bounds for the Bayes Risk.12.14 The Sequential Probability-ratio Test.12.15 Characteristics of Sequential Probability-ratio Tests.12.16 Approximating the Expected Number of Observations.Exercises.Chapter 13. Optimal Stopping.13.1 Introduction.13.2 The Statistician's Reward.13.3 Choice of the Utility Function.13.4 Sampling Without Recall.13.5 Further Problems of Sampling with Recall and Sampling without Recall.13.6 Sampling without Recall from a Normal Distribution with Unknown Mean.13.7 Sampling with Recall from a Normal Distribution with Unknown Mean.13.8 Existence of Optimal Stopping Rules.13.9 Existence of Optimal Stopping Rules for Problems of Sampling with Recall and Sampling without Recall.13.10 Martingales.13.11 Stopping Rules for Martingales.13.12 Uniformly Integrable Sequences of Random Variables.13.13 Martingales Formed from Sums and Products of Random Variables.13.14 Regular Supermartingales.13.15 Supermartingales and General Problems of Optimal Stopping.13.16 Markov Processes.13.17 Stationary Stopping Rules for Markov Processes.13.18 Entrance-fee Problems.13.19 The Functional Equation for a Markov Process.Exercises.Chapter 14. Sequential Choice of Experiments.14.1 Introduction.14.2 Markovian Decision Processes with a Finite Number of Stages.14.3 Markovian Decision Processes with an Infinite Number of Stages.14.4 Some Betting Problems.14.5 Two-armed-bandit Problems.14.6 Two-armed-bandit Problems When the Value of One Parameter Is Known.14.7 Two-armed-bandit Problems When the Parameters Are Dependent.14.8 Inventory Problems.14.9 Inventory Problems with an Infinite Number of Stages.14.10 Control Problems.14.11 Optimal Control When the Process Cannot Be Observed without Error.14.12 Multidimensional Control Problems.14.13 Control Problems with Actuation Errors.14.14 Search Problems.14.15 Search Problems with Equal Costs.14.16 Uncertainty Functions and Statistical Decision Problems.14.17 Sufficient Experiments.14.18 Examples of Sufficient Experiments.Exercises.References.Supplementary Bibliography.Name Index.Subject Index.

Optimal Statistical Decisions

Preface. Elements of Stochastic Processes. Markov Chains. The Basic Limit Theorem of Markov Chains and Applications. Classical Examples of Continuous Time Markov Chains. Renewal Processes. Martingales. Brownian Motion. Branching Processes. Stationary Processes. Review of Matrix Analysis. Index.

Tests with Repeated Observations

References

Dynamic Programming and Optimal Control

Information Theory and Statistics

Large Deviations Techniques and Applications

Optimal Statistical Decisions

A first course in stochastic processes

Related Papers (5)

A modification of the sequential probability ratio test for testing a normal mean

Sequential detection using higher-order statistics

Sequential rank tests for location

Asymptotic efficiencies of truncated sequential tests

Distribution theory of group sequential t,χ2and F-tests for general linear models