THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

In this paper we introduce a variant of temporal logic tailored for specifying desired properties of continuous signals. The logic is based on a bounded subset of the real-time logic mitl, augmented with a static mapping from continuous domains into propositions. From formulae in this logic we create automatically property monitors that can check whether a given signal of bounded length and finite variability satisfies the property. A prototype implementation of this procedure was used to check properties of simulation traces generated by Matlab/Simulink.

/pdf/monitoring-temporal-properties-of-continuous-signals-4691an6o8x.pdf

Monitoring Temporal Properties of Continuous Signals

We first introduce Abstract DPLL, a rule-based formulation of the Davis--Putnam--Logemann--Loveland (DPLL) procedure for propositional satisfiability. This abstract framework allows one to cleanly express practical DPLL algorithms and to formally reason about them in a simple way. Its properties, such as soundness, completeness or termination, immediately carry over to the modern DPLL implementations with features such as backjumping or clause learning.We then extend the framework to Satisfiability Modulo background Theories (SMT) and use it to model several variants of the so-called lazy approach for SMT. In particular, we use it to introduce a few variants of a new, efficient and modular approach for SMT based on a general DPLL(X) engine, whose parameter X can be instantiated with a specialized solver SolverT for a given theory T, thus producing a DPLL(T) system. We describe the high-level design of DPLL(X) and its cooperation with SolverT, discuss the role of theory propagation, and describe different DPLL(T) strategies for some theories arising in industrial applications.Our extensive experimental evidence, summarized in this article, shows that DPLL(T) systems can significantly outperform the other state-of-the-art tools, frequently even in orders of magnitude, and have better scaling properties.

/pdf/solving-sat-and-sat-modulo-theories-from-an-abstract-davis-4x4o75gcy8.pdf

Solving SAT and SAT Modulo Theories: From an abstract Davis--Putnam--Logemann--Loveland procedure to DPLL(T)

Satisfiability Modulo Theories (SMT) refers to the problem of determining whether a first-order formula is satisfiable with respect to some logical theory. Solvers based on SMT are used as back-end engines in model-checking applications such as bounded, interpolation-based, and predicate-abstraction-based model checking. After a brief illustration of these uses, we survey the predominant techniques for solving SMT problems with an emphasis on the lazy approach, in which a propositional satisfiability (SAT) solver is combined with one or more theory solvers. We discuss the architecture of a lazy SMT solver, give examples of theory solvers, show how to combine such solvers modularly, and mention several extensions of the lazy approach. We also briefly describe the eager approach in which the SMT problem is reduced to a SAT problem. Finally, we discuss how the basic framework for determining satisfiability can be extended with additional functionality such as producing models, proofs, unsatisfiable cores, and interpolants.

Satisfiability Modulo Theories

Since the introduction of temporal logic for the specification of computer programs [5], usability has been an issue, because a difficult-to-use formalism is a barrier to the wide adoption of formal methods. Our solution is Sugar, the temporal logic used by the RuleBase formal verification tool [2]. Sugar adds the power of regular expressions to CTL [4], as well as an extensive set of operators which provide syntactic sugar. That is, while these operators do not add expressive power, they allow properties to be expressed more succinctly than in the basic language. Experience shows that Sugar allows hardware engineers to easily and intuitively specify their designs. The full language is used for model checking, and a significant portion can be model checked on-the-fly [3]. The automatic generation of simulation checkers from the same portion of Sugar is described in [1]. While previous papers have described various features of the language, this paper presents the first complete description of Sugar.

/pdf/the-temporal-logic-sugar-z4jdj9md55.pdf

The Temporal Logic Sugar

Prepositional logic formulas containing implications can suffer from antecedent failure, in which the formula is true trivially because the pre-condition of the implication is not satisfiable. In other words, the post-condition of the implication does not affect the truth value of the formula. We call this a vacuous pass, and extend the definition of vacuity to cover other kinds of trivial passes in temporal logic. We define w-ACTL, a subset of CTL and show by construction that for every w-ACTL formula ϕ there is a formula w(ϕ), such that: both ϕ and w(ϕ) are true in some model M iff ϕ passes vacuously. A useful side-effect of w(ϕ) is that if false, any counter-example is also a non-trivial witness of the original formula ϕ.

/pdf/efficient-detection-of-vacuity-in-actl-formulas-8js5fdhuua.pdf

Efficient Detection of Vacuity in ACTL Formulas

The ability to generate a counter-example is an important feature of model checking tools, because a counter-example provides information to the user in the case that the formula being checked is found to be non-valid. In this paper, we turn our attention to providing similar feedback to the user in the case that the formula is found to be valid, because valid formulas can hide real problems in the model. For instance, propositional logic formulas containing implications can suffer from antecedent failure, in which the formula is trivially valid because the pre-condition of the implication is not satisfiable. We call this vacuity, and extend the definition to cover other kinds of trivial validity. For non-vacuously valid formulas, we define an interesting witness as a non-trivial example of the validity of the formula. We formalize the notions of vacuity and interesting witness, and show how to detect vacuity and generate interesting witnesses in temporal model checking. Finally, we provide a practical solution for a useful subset of ACTL formulas.

/pdf/efficient-detection-of-vacuity-in-temporal-model-checking-o2hvw5z4c8.pdf

Efficient Detection of Vacuity in Temporal Model Checking

We introduce an efficient decision procedure for the theory of equality based on finite instantiations. When using the finite instantiations method, it is a common practice to take a range of [1..n] (where n is the number of input non-Boolean variables) as the range for all non-Boolean variables, resulting in a state-space of nn. Although various attempts to minimize this range were made, typically they either required various restrictions on the investigated formulas or were not very effective. In many cases, the nn state-space cannot be handled by BDD-based tools within a reasonable amount of time. In this paper we show that significantly smaller domains can be algorithmically found, by analyzing the structure of the formula. We also show an upper bound for the state-space based on this analysis. This method enabled us to verify formulas containing hundreds of integer and floating point variables.

/pdf/deciding-equality-formulas-by-small-domains-instantiations-1plpbjxnd7.pdf

Deciding Equality Formulas by Small Domains Instantiations

Efficient decision procedures for equality logic (quantifier-free predicate calculus + the equality sign) are of major importance when proving logical equivalence between systems. We introduce an efficient decision procedure for the theory of equality based on finite instantiations. The main idea is to analyze the structure of the formula and compute accordingly a small domain to each variable such that the formula is satisfiable iff it can be satisfied over these domains. We show how the problem of finding these small domains can be reduced to an interesting graph theoretic problem. This method enabled us to verify formulas containing hundreds of integer and floating point variables that could not be efficiently handled with previously known techniques.

/pdf/the-small-model-property-how-small-can-it-be-24ihjcsmq8.pdf

Yoav Rodeh

Papers

The Temporal Logic Sugar

Efficient Detection of Vacuity in ACTL Formulas

Efficient Detection of Vacuity in Temporal Model Checking

Deciding Equality Formulas by Small Domains Instantiations

The small model property: how small can it be?