"Parameter Estimation and Inverse Problems, 2/e" provides geoscience students and professionals with answers to common questions like how one can derive a physical model from a finite set of observations containing errors, and how one may determine the quality of such a model. This book takes on these fundamental and challenging problems, introducing students and professionals to the broad range of approaches that lie in the realm of inverse theory. The authors present both the underlying theory and practical algorithms for solving inverse problems. The authors' treatment is appropriate for geoscience graduate students and advanced undergraduates with a basic working knowledge of calculus, linear algebra, and statistics. "Parameter Estimation and Inverse Problems, 2/e" introduces readers to both Classical and Bayesian approaches to linear and nonlinear problems with particular attention paid to computational, mathematical, and statistical issues related to their application to geophysical problems. The textbook includes Appendices covering essential linear algebra, statistics, and notation in the context of the subject. This book includes a companion website that features computational examples (including all examples contained in the textbook) and useful subroutines using MATLAB. It: includes appendices for review of needed concepts in linear, statistics, and vector calculus; features a companion website that contains comprehensive MATLAB code for all examples, which readers can reproduce, experiment with, and modify; offers an online instructor's guide that helps professors teach, customize exercises, and select homework problems; and, is accessible to students and professionals without a highly specialized mathematical background.

/pdf/parameter-estimation-and-inverse-problems-370q4164ci.pdf

Parameter estimation and inverse problems

The problem of finding a root of the multivariate gradient equation that arises in function minimization is considered. When only noisy measurements of the function are available, a stochastic approximation (SA) algorithm for the general Kiefer-Wolfowitz type is appropriate for estimating the root. The paper presents an SA algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite-difference approximation of Keifer-Wolfowitz type procedures. Theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large-dimensional problems. >

/pdf/multivariate-stochastic-approximation-using-a-simultaneous-1nsc6vr7zi.pdf

Multivariate stochastic approximation using a simultaneous perturbation gradient approximation

A new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated. Convergence with probability one is proved for a variety of classical optimization and identification problems. It is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence.

Acceleration of stochastic approximation by averaging

The stochastic multi-armed bandit model is a simple abstraction that has proven useful in many different contexts in statistics and machine learning. Whereas the achievable limit in terms of regret minimization is now well known, our aim is to contribute to a better understanding of the performance in terms of identifying the m best arms. We introduce generic notions of complexity for the two dominant frameworks considered in the literature: fixed-budget and fixed-confidence settings. In the fixed-confidence setting, we provide the first known distribution-dependent lower bound on the complexity that involves information-theoretic quantities and holds when m ≥ 1 under general assumptions. In the specific case of two armed-bandits, we derive refined lower bounds in both the fixedcon fidence and fixed-budget settings, along with matching algorithms for Gaussian and Bernoulli bandit models. These results show in particular that the complexity of the fixed-budget setting may be smaller than the complexity of the fixed-confidence setting, contradicting the familiar behavior observed when testing fully specified alternatives. In addition, we also provide improved sequential stopping rules that have guaranteed error probabilities and shorter average running times. The proofs rely on two technical results that are of independent interest: a deviation lemma for self-normalized sums (Lemma 7) and a novel change of measure inequality for bandit models (Lemma 1).

/pdf/on-the-complexity-of-best-arm-identification-in-multi-armed-7ed20kt1lj.pdf

On the complexity of best-arm identification in multi-armed bandit models

Software system documentation is almost always expressed informally in natural language and free text. Examples include requirement specifications, design documents, manual pages, system development journals, error logs, and related maintenance reports. We propose a method based on information retrieval to recover traceability links between source code and free text documents. A premise of our work is that programmers use meaningful names for program items, such as functions, variables, types, classes, and methods. We believe that the application-domain knowledge that programmers process when writing the code is often captured by the mnemonics for identifiers; therefore, the analysis of these mnemonics can help to associate high-level concepts with program concepts and vice-versa. We apply both a probabilistic and a vector space information retrieval model in two case studies to trace C++ source code onto manual pages and Java code to functional requirements. We compare the results of applying the two models, discuss the benefits and limitations, and describe directions for improvements.

/pdf/recovering-traceability-links-between-code-and-documentation-38oxh8tsjk.pdf

Recovering traceability links between code and documentation

Introduction to Probability with RIntroduction to Probability and StatisticsAn Introduction to Probability and Statistical InferenceIntroduction to ProbabilityIntroduction to Probability, Statistics, and Random ProcessesIntroduction to ProbabilityIntroduction to ProbabilityAn Introduction to Probability Theory and Its Applications, Volume 2Introduction to ProbabilityMathematical StatisticsIntroduction to Probability and Mathematical StatisticsIntroduction to Probability Theory and Statistical InferenceIntroduction to Probability and Statistics for EngineersProbability and GamesIntroduction to Probability and Statistics Using RElementary Probability Theory with Stochastic ProcessesIntroduction to Probability with Statistical ApplicationsMathematical Theory of Probability and StatisticsIntroduction to ProbabilityIntroduction to ProbabilityStochasticsMathematical Statistics and Data AnalysisProbability and StatisticsIntroduction to Probability and MeasureAn Introduction to Measure and ProbabilityIntroduction to Probability and Mathematical StatisticsAn Introduction to Probability and Mathematical StatisticsProbabilityAn Introduction to Probability TheoryA Natural Introduction to Probability TheoryAn Introduction to Probability and StatisticsIntroduction to Mathematical StatisticsIntroduction to ProbabilityIntroduction to ProbabilityIntroduction to ProbabilityProbability and Mathematical StatisticsThe Mathematics of GamesProbability and Mathematical Statistics: Theory, Applications, and Practice in RAn Introduction to Probability Theory and Mathematical StatisticsA Modern Introduction to Probability and Statistics According to a remark attributed to Mark Kac 'Probability Theory is a measure theory with a soul'. This book with its choice of proofs, remarks, examples and exercises has been prepared taking both these aesthetic and practical aspects into account.Developed from celebrated

Introduction to probability and mathematical statistics

Identification and Ranking Procedures (With Special Reference to Koopman-Darmois Populations)

Simple expressions are given for the probability of a correct decision for the Anderson sequential procedure with a triangular boundary, for some values of parameters. This can be used to improve bounds of the probability of correct decision for the Paulson sequential ranking procedures and change slightly the comparison of these with the Bechhofer-Kiefer-Sobel procedures.

https://projecteuclid.org/download/pdf_1/euclid.aos/1176342622

Note on Anderson's Sequential Procedures with Triangular Boundary

Stochastic approximation procedures were shown by Sakrison to become asymptotically efficient estimators when used to minimize the Kullback-Leibler information, if certain conditions hold. Further results in this direction were obtained by Nevel'son and Has'minskij. This paper gives, first, alternative conditions for convergence and, secondly, shows that, under weaker conditions, asymptotic optimality is obtained by a modified stochastic approximation procedure. The modified procedure uses a consistent estimate which leads the approximating sequence to a proper local minimum of the Kullback-Leibler information. The conditions under which the procedure is asymptotically optimal are close to or weaker than those for asymptotic optimality of one-step-correction maximum likelihood methods.

/pdf/on-asymptotically-efficient-recursive-estimation-5cdu06vrm2.pdf

On Asymptotically Efficient Recursive Estimation

Locally asymptotically minimax (LAM) estimates are constructed for locally asymptotically normal (LAN) families under very mild additional assumptions. Adaptive estimation is also considered and a sufficient condition is given for an estimate to be locally asymptotically minimax adaptive. Incidently, it is shown that a well known lower bound due to Hajek (1972) for the local asymptotic minimax risk is not sharp.

Vaclav Fabian

Papers

Introduction to probability and mathematical statistics

Identification and Ranking Procedures (With Special Reference to Koopman-Darmois Populations)

Note on Anderson's Sequential Procedures with Triangular Boundary

On Asymptotically Efficient Recursive Estimation

On Estimation and Adaptive Estimation for Locally Asymptotically Normal Families