Showing papers on "Generalization published in 1993"

Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization

Abstract: The paper describes a rank-based fitness assignment method for Multiple Objective Genetic Algorithms (MOGAs). Conventional niche formation methods are extended to this class of multimodal problems and theory for setting the niche size is presented. The fitness assignment method is then modified to allow direct intervention of an external decision maker (DM). Finally, the MOGA is generalised further: the genetic algorithm is seen as the optimizing element of a multiobjective optimization loop, which also comprises the DM. It is the interaction between the two that leads to the determination of a satisfactory solution to the problem. Illustrative results of how the DM can interact with the genetic algorithm are presented. They also show the ability of the MOGA to uniformly sample regions of the trade-off surface.

Alternative Arguments for Generalizing From Data as Applied to Qualitative Research

Abstract: One criticism about qualitative research is that it is difficult to generalize findings to settings not studied. To explore this issue, I examine three broad arguments for generalizing from data: sample-to-population extrapolation, analytic generalization, and case-to-case transfer. Qualitative research often uses the last argument, but some efforts have been made to use the first two. I suggest that analytic generalization can be very helpful for qualitative researchers but that sample-to-population extrapolation is not likely to be.

Improving generalization for temporal difference learning: The successor representation

Abstract: Estimation of returns over time, the focus of temporal difference (TD) algorithms, imposes particular constraints on good function approximators or representations. Appropriate generalization between states is determined by how similar their successors are, and representations should follow suit. This paper shows how TD machinery can be used to learn such representations, and illustrates, using a navigation task, the appropriately distributed nature of the result.

Demonstration of Deductive Meta-Analysis: Ethanol Intake and Risk of Myocardial Infarction

Abstract: Meta-analyses should be analytic and deductive. In a review of the state of the science of meta-analysis in the previous volume of Epidemiologic Reviews, a list of definitions and synonyms of meta-analysis was given: "overview, pooling, data pooling, literature synthesis, data synthesis, quantitative synthesis, and quantitative review" (1, p. 154). Indeed, most metaanalyses are more synthetic than analytic: They produce a summary, such as an aggregate relative risk and 95 percent confidence interval, from a set of individual studies and stop there. Such a "meta-synthesis" has an inductive approach, i.e., generalization from a set of particular observations. By contrast, a deductive approach starts with alternative generalizations (hypotheses) and uses particular observations to discriminate among them. The causal hypothesis of primary interest is considered corroborated if competing hypotheses do not stand up to the evidence (2). The words "inductive" and "deductive" here have the meanings used in logic (3). Induction is the inference "If true for A, then true for B" when A is a part, sample, or special case of B. When epidemiologists infer

On the singular set of stationary harmonic maps.

Abstract: LetM andN be compact riemannian manifolds, andu a stationary harmonic map fromM toN. We prove thatH n−2 (Σ)=0, wheren=dimM and Σ is the singular set ofu. This is a generalization of a result of C. Evans [7], where this is proved in the special caseN is a sphere. We also prove that, ifu is a weakly harmonic map inW 1,n (M, N), thenu is smooth. This extends results of F. Helein for the casen=2, or the caseN is a sphere ([9], [10]).

A generalization of the noisy-or model

Abstract: The Noisy-Or model is convenient for describing a class of uncertain relationships in Bayesian networks [Pearl 1988]. Pearl describes the Noisy-Or model for Boolean variables. Here we generalize the model to nary input and output variables and to arbitrary functions other than the Boolean OR function. This generalization is a useful modeling aid for construction of Bayesian networks. We illustrate with some examples including digital circuit diagnosis and network reliability analysis.

Multiscale representations of Markov random fields

Abstract: Recently, a framework for multiscale stochastic modeling was introduced based on coarse-to-fine scale-recursive dynamics defined on trees. This model class has some attractive characteristics which lead to extremely efficient, statistically optimal signal and image processing algorithms. The authors show that this model class is also quite rich. In particular, they describe how 1-D Markov processes and 2-D Markov random fields (MRFs) can be represented within this framework. The recursive structure of 1-D Markov processes makes them simple to analyze, and generally leads to computationally efficient algorithms for statistical inference. On the other hand, 2-D MRFs are well known to be very difficult to analyze due to their noncausal structure, and thus their use typically leads to computationally intensive algorithms for smoothing and parameter identification. In contrast, their multiscale representations are based on scale-recursive models and thus lead naturally to scale-recursive algorithms, which can be substantially more efficient computationally than those associated with MRF models. In 1-D, the multiscale representation is a generalization of the midpoint deflection construction of Brownian motion. The representation of 2-D MRFs is based on a further generalization to a "midline" deflection construction. The exact representations of 2-D MRFs are used to motivate a class of multiscale approximate MRF models based on one-dimensional wavelet transforms. They demonstrate the use of these latter models in the context of texture representation and, in particular, they show how they can be used as approximations for or alternatives to well-known MRF texture models. >

Optimal Brain Surgeon: Extensions and performance comparisons

Abstract: We extend Optimal Brain Surgeon (OBS) - a second-order method for pruning networks - to allow for general error measures, and explore a reduced computational and storage implementation via a dominant eigenspace decomposition. Simulations on nonlinear, noisy pattern classification problems reveal that OBS does lead to improved generalization, and performs favorably in comparison with Optimal Brain Damage (OBD). We find that the required retraining steps in OBD may lead to inferior generalization, a result that can be interpreted as due to injecting noise back into the system. A common technique is to stop training of a large network at the minimum validation error. We found that the test error could be reduced even further by means of OBS (but not OBD) pruning. Our results justify the t → 0 approximation used in OBS and indicate why retraining in a highly pruned network may lead to inferior performance.

On Overfitting Avoidance as Bias

Abstract: In supervising learning it is commonly believed that penalizing complex functions help one avoid ``overfitting'' functions to data, and therefore improves generalization. It is also commonly believed that cross-validation is an effective way to choose amongst algorithms for fitting functions to data. In a recent paper, Schaffer (1993) presents experimental evidence disputing these claims. The current paper consists of a formal analysis of these contentions of Schaffer's. It proves that his contentions are valid, although some of his experiments must be interpreted with caution.

Variable Precision Rough Sets with Asymmetric Bounds

Abstract: A generalization of the original idea of rough sets as introduced by Pawlak is presented. The generalization, called the Variable Precision Rough Sets Model with Asymmetric Bounds, is aimed at modeling data relationships expressed in terms of frequency distribution rather than a full inclusion relation. The model presented is a direct extension of the previous concept, the Variable Precision Rough Sets Model. The properties of the extended model are investigated and compared to the original model.

Statistical theory of learning curves under entropic loss criterion

Abstract: The present paper elucidates a universal property of learning curves, which shows how the generalization error, training error, and the complexity of the underlying stochastic machine are related and how the behavior of a stochastic machine is improved as the number of training examples increases. The error is measured by the entropic loss. It is proved that the generalization error converges to H0, the entropy of the conditional distribution of the true machine, as H0 + m*/(2t), while the training error converges as H0-m*/(2t), where t is the number of examples and m* shows the complexity of the network. When the model is faithful, implying that the true machine is in the model, m* is reduced to m, the number of modifiable parameters. This is a universal law because it holds for any regular machine irrespective of its structure under the maximum likelihood estimator. Similar relations are obtained for the Bayes and Gibbs learning algorithms. These learning curves show the relation among the accuracy of learning, the complexity of a model, and the number of training examples.

Systematic Generalization, Historical Fate, and the Species Problem

Abstract: The species problem is one of the oldest controversies in natural history. Its persistence suggests that it is something more than a problem of fact or definition. Considerable light is shed on the species problem when it is viewed as a problem in the representation of the natural system (sensu Griffiths, 1974, Acta Biotheor. 23: 85-131; de Queiroz, 1998, Philos. Sci. 55: 238-259). Just as maps are representations of the earth, and are subject to what is called cartographic generalization, so diagrams of the natural system (evolutionary trees) are representations of the evolutionary chronicle, and are subject to a temporal version of cartographic generalization which may be termed systematic generalization. Cartographic generalization is based on judgements of geographical importance, and systematic generalization is based on judgements of historical importance, judgements expressed in narrative sentences (sensu Danto, 1985, Narration and knowledge, Columbia Univ. Press, New York). At higher systematic levels these narrative sentences are conventional and retrospective, but near the “species” level they become prospective, that is, dependent upon expectations of the future. The truth of prospective narrative sentences is logically indeterminable in the present, and since all the common species concepts depend upon prospective narration, it is impossible for any of them to be applied with precision.

Use of Graph Theory to Support Map Generalization

Abstract: In the generalization of a concept, we seek to preserve the essential characteristics and behavior of objects. In map generalization, the appropriate selection and application of procedures (such a...

Some extensions of techniques to handle non-discretionary factors in data envelopment analysis

Abstract: Data Envelopment Analysis (DEA) assumes, in most cases, that all inputs and outputs are controlled by the Decision Making Unit (DMU). Inputs and/or outputs that do not conform to this assumption are denoted in DEA asnon-discretionary (ND) factors. Banker and Morey [1986] formulated several variants of DEA models which incorporated ND with ordinary factors. This article extends the Banker and Morey approach for treating nondiscretionary factors in two ways. First, the model is extended to allow for thesimultaneous presence of ND factors in both the input and the output sets. Second, a generalization is offered which, for the first time, enables a quantitative evaluation ofpartially controlled factors. A numerical example is given to illustrate the different models.

Systematic Generalization, Historical Fate, and the Species Problem

Abstract: The species problem is one of the oldest controversies in natural history. Its persistence suggests that it is something more than a problem of fact or definition. Considerable light is shed on the species problem when it is viewed as a problem in the representation of the natural system (sensu Griffiths, 1974, Acta Biotheor. 23:85-131; de Queiroz, 1988, Philos. Sci. 55:238-259). Just as maps are representations of the earth and are subject to what is called cartographic generalization, so diagrams of the natural system (evolutionary trees) are representations of the evolutionary chronicle and are subject to a temporal version of cartographic generalization, which may be termed systematic generalization

Necessary and sufficient conditions of stability: a multiloop generalization of the circle criterion

Abstract: A linear time-invariant system with a vector output and a vector input is described. This system is closed by an uncertain (nonlinear, time-varying) feedback. The only information on this feedback is given by several integral quadratic inequalities, i.e. the uncertainties under consideration generalize the so-called conic nonlinearities. Necessary and sufficient frequency-domain conditions of stability are obtained. An advanced version of the S-procedure losslessness theorem and some other tools of the absolute stability theory are used. >

Generalization with Componential Attractors: Word and Nonword Reading in an Attractor Network

Abstract: Networks that learn to make familiar activity patterns into stable attractors have proven useful in accounting for many aspects of normal and impaired cognition. However, their ability to generalize is questionable, particularly in quasiregular tasks that involve both regularities and exceptions, such as word reading. We trained an attractor network to pronounce virtually all of a large corpus of monosyllabic words, including both regular and exception words. When tested on the lists of pronounceable nonwords used in several empirical studies, its accuracy was closely comparable to that of human subjects. The network generalizes because the attractors it developed for regular words are componential—they have substructure that reflects common sublexical correspondences between orthography and phonology. This componentiality is faciliated by the use of orthographic and phonological representations that make explicit the structured relationship between written and spoken words. Furthermore, the componential attractors for regular words coexist with much less componential attractors for exception words. These results demonstrate that attractors can support effective generalization, challenging “dual-route” assumptions that multiple, independent mechanisms are required for quasiregular tasks.

Treatment of sentence production deficits in aphasia: A linguistic-specific approach to wh-interrogative training and generalization

Abstract: The present research examines the effects of a linguistic-specific treatment on acquisition and generalization of wh-interrogative structures in two aphasic subjects presenting with deficit patterns consistent with agrammatism. The underlying linguistic representation of sentence structures selected for treatment and generalization was considered based on aspects of Chomsky's (1981) Government Binding (GB) theory, and a linguistic-based, wh-movement treatment strategy was implemented. Using a single-subject multiple-baseline design across behaviours and subjects, the effects of treatment were explored by examining generalization patterns across wh question forms requiring wh-movement (movement of a direct object NP to COMP). Within question form generalization also was evaluated by examining formulation of untrained sentences of varied complexity—with complexity defined in terms of the number of phrasal nodes in the d-structure representation of sentences. Results indicated that for both subjects...

Computer generated Lyapunov functions for a class of nonlinear systems

Abstract: The problem of constructing Lyapunov functions for a class of nonlinear dynamical systems is considered. The problem is reduced to the construction of a polytope satisfying some conditions. A generalization of the concept of sector condition that it makes possible to evaluate a given nonlinear function by using a set of piecewise-linear functions is proposed. This improvement greatly reduces the conservatism in the stability analysis of nonlinear systems. Two algorithms for constructing such polytopes are proposed, and two examples are shown to demonstrate the usefulness of the results. >

Scale-sensitive dimensions, uniform convergence, and learnability

Abstract: Learnability in Valiant's PAC learning model has been shown to be strongly related to the existence of uniform laws of large numbers. These laws define a distribution-free convergence property of means to expectations uniformly over classes of random variables. Classes of real-valued functions enjoying such a property are also known as uniform Gliveako-Cantelli classes. In this paper we prove, through a generalization of Sauer's lemma that may be interesting in its own right, a new characterization of uniform Glivenko-Cantelli classes. Our characterization yields Dudley, Gine, and Zinn's previous characterization as a corollary. Furthermore, it is the first based on a simple combinatorial quantity generalizing the Vapnik-Chervonenkis dimension. We apply this result to characterize PAC learnability in the statistical regression framework of probabilistic concepts, solving an open problem posed by Kearns and Schapire. Our characterization shows that the accuracy parameter plays a crucial role in determining the effective complexity of the learner's hypothesis class. >

A dynamic generalization of the rasch model

Abstract: In the present paper a model for describing dynamic processes is constructed by combining the common Rasch model with the concept of structurally incomplete designs. This is accomplished by mapping each item on a collection of virtual items, one of which is assumed to be presented to the respondent dependent on the preceding responses and/or the feedback obtained. It is shown that, in the case of subject control, no unique conditional maximum likelihood (CML) estimates exist, whereas marginal maximum likelihood (MML) proves a suitable estimation procedure. A hierarchical family of dynamic models is presented, and it is shown how to test special cases against more general ones. Furthermore, it is shown that the model presented is a generalization of a class of mathematical learning models, known as Luce's beta-model.