Showing papers on "Generalization published in 2002"

Choosing Multiple Parameters for Support Vector Machines

Abstract: The problem of automatically tuning multiple parameters for pattern recognition Support Vector Machines (SVMs) is considered. This is done by minimizing some estimates of the generalization error of SVMs using a gradient descent algorithm over the set of parameters. Usual methods for choosing parameters, based on exhaustive search become intractable as soon as the number of parameters exceeds two. Some experimental results assess the feasibility of our approach for a large number of parameters (more than 100) and demonstrate an improvement of generalization performance.

Achieving k -anonymity privacy protection using generalization and suppression

Abstract: Often a data holder, such as a hospital or bank, needs to share person-specific records in such a way that the identities of the individuals who are the subjects of the data cannot be determined. One way to achieve this is to have the released records adhere to k- anonymity, which means each released record has at least (k-1) other records in the release whose values are indistinct over those fields that appear in external data. So, k- anonymity provides privacy protection by guaranteeing that each released record will relate to at least k individuals even if the records are directly linked to external information. This paper provides a formal presentation of combining generalization and suppression to achieve k-anonymity. Generalization involves replacing (or recoding) a value with a less specific but semantically consistent value. Suppression involves not releasing a value at all. The Preferred Minimal Generalization Algorithm (MinGen), which is a theoretical algorithm presented herein, combines these techniques to provide k-anonymity protection with minimal distortion. The real-world algorithms Datafly and µ-Argus are compared to MinGen. Both Datafly and µ-Argus use heuristics to make approximations, and so, they do not always yield optimal results. It is shown that Datafly can over distort data and µ-Argus can additionally fail to provide adequate protection.

Strong converse for identification via quantum channels

Abstract: We present a simple proof of the strong converse for identification via discrete memoryless quantum channels, based on a novel covering lemma. The new method is a generalization to quantum communication channels of Ahlswede's (1979, 1992) approach to classical channels. It involves a development of explicit large deviation estimates to the case of random variables taking values in self-adjoint operators on a Hilbert space. This theory is presented separately in an appendix, and we illustrate it by showing its application to quantum generalizations of classical hypergraph covering problems.

Modal-style operators in qualitative data analysis

Abstract: We explore the usage of the modal possibility operator (and its dual necessity operator) in qualitative data analysis, and show that it-quite literally-complements the derivation operator of formal concept analysis; we also propose a new generalization of the rough set approximation operators. As an example for the applicability of the concepts we investigate the Morse data set which has been frequently studied in multidimensional scaling procedures.

Cranking: Combining Rankings Using Conditional Probability Models on Permutations

Abstract: A new approach to ensemble learning is introduced that takes ranking rather than classification as fundamental, leading to models on the symmetric group and its cosets. The approach uses a generalization of the Mallows model on permutations to combine multiple input rankings. Applications include the task of combining the output of multiple search engines and multiclass or multilabel classification, where a set of input classifiers is viewed as generating a ranking of class labels. Experiments for both types of applications are presented.

Seiberg-Witten prepotential from instanton counting

Abstract: Direct evaluation of the Seiberg-Witten prepotential is accomplished following the localization programme suggested some time ago. Our results agree with all low-instanton calculations available in the literature. We present a two-parameter generalization of the Seiberg-Witten prepotential, which is rather natural from the M-theory/five dimensional perspective, and conjecture its relation to the tau-functions of KP/Toda hierarchy.

Coverage and generalization in an artificial immune system

Abstract: LISYS is an artificial immune system framework which is specialized for the problem of network intrusion detection. LISYS learns to detect abnormal packets by observing normal network traffic. Because LISYS sees only a partial sample of normal traffic, it must generalize from its observations in order to characterize normal behavior correctly. A variation of the r-contiguous bits matching rule is introduced, and its effect on coverage and generalization is studied. The effect of representation diversity on coverage and generalization is also explored by studying permutations in the order of bits in the representation.

Field Algebras

Abstract: A field algebra is a ``non-commutative'' generalization of a vertex algebra In this paper we develop foundations of the theory of field algebras

Generalized Bundle Methods

Abstract: We study a class of generalized bundle methods for which the stabilizing term can be any closed convex function satisfying certain properties. This setting covers several algorithms from the literature that have been so far regarded as distinct. Under a different hypothesis on the stabilizing term and/or the function to be minimized, we prove finite termination, asymptotic convergence, and finite convergence to an optimal point, with or without limits on the number of serious steps and/or requiring the proximal parameter to go to infinity. The convergence proofs leave a high degree of freedom in the crucial implementative features of the algorithm, i.e., the management of the bundle of subgradients ($\beta$-strategy) and of the proximal parameter (t-strategy). We extensively exploit a dual view of bundle methods, which are shown to be a dual ascent approach to one nonlinear problem in an appropriate dual space, where nonlinear subproblems are approximately solved at each step with an inner linearization approach. This allows us to precisely characterize the changes in the subproblems during the serious steps, since the dual problem is not tied to the local concept of $\varepsilon$-subdifferential. For some of the proofs, a generalization of inf-compactness, called *-compactness, is required; this concept is related to that of asymptotically well-behaved functions.

Quasi-Lie Bialgebroids and Twisted Poisson Manifolds

Abstract: We develop a theory of quasi-Lie bialgebroids using a homological approach. This notion is a generalization of quasi-Lie bialgebras, as well as twisted Poisson structures with a 3-form background which have recently appeared in the context of string theory, and were studied by Severa and Weinstein using a different method.

Generalized instrumental variables

Abstract: This paper concerns the assessment of direct causal effects from a combination of: (i) nonexperimental data, and (ii) qualitative domain knowledge. Domain knowledge is encoded in the form of a directed acyclic graph (DAG), in which all interactions are assumed linear, and some variables are presumed to be unobserved. We provide a generalization of the well-known method of Instrumental Variables, which allows its application to models with few conditional independeces.

Cubegrades: Generalizing Association Rules

Abstract: Cubegrades are a generalization of association rules which represent how a set of measures (aggregates) is affected by modifying a cube through specialization (rolldown), generalization (rollup) and mutation (which is a change in one of the cube's dimensions). Cubegrades are significantly more expressive than association rules in capturing trends and patterns in data because they can use other standard aggregate measures, in addition to COUNT. Cubegrades are atoms which can support sophisticated “what if” analysis tasks dealing with behavior of arbitrary aggregates over different database segments. As such, cubegrades can be useful in marketing, sales analysis, and other typical data mining applications in business. In this paper we introduce the concept of cubegrades. We define them and give examples of their usage. We then describe in detail an important task for computing cubegrades: generation of significant cubes which is analogous to generating frequent sets. A novel Grid Based Pruning (GBP) method is employed for this purpose. We experimentally demonstrate the practicality of the method. We conclude with a number of open questions and possible extensions of the work.

Width optimization of the Gaussian kernels in Radial Basis Function Networks

Abstract: Radial basis function networks are usually trained according to a three-stage procedure. In the literature, many papers are devoted to the estimation of the position of Gaussian kernels, as well as the computation of the weights. Meanwhile, very few focus on the estimation of the kernel widths. In this paper, first, we develop a heuristic to optimize the widths in order to improve the generalization process. Subsequently, we validate our approach on several theoretical and real-life approximation problems.

Single-Pair versus Multiple-Pair Answers: Wh -in-Situ and Scope

Abstract: It is a well-accepted generalization that wh-in-situ can take unbounded scope. This squibe examines the diagnostic of paired answers on which this generalization is based and argues that a more refined understanding of the diagnostic leads to a reassessment of the claims about in-situ and scope

Targeted free energy perturbation.

Abstract: In this paper generalization of the free energy perturbation identity is derived, and a computational strategy based on this result is presented. A simple example illustrates the efficiency gains that can be achieved with this method.

New methods for the two-dimensional Schrödinger equation: SUSY-separation of variables and shape invariance

Abstract: Two new methods for the investigation of two-dimensional quantum systems, whose Hamiltonians are not amenable to separation of variables, are proposed. The first one—SUSY-separation of variables—is based on the intertwining relations of higher-order SUSY quantum mechanics (HSUSY QM) with supercharges allowing separation of variables. The second one is a generalization of shape invariance. While in one dimension shape invariance allows us to solve algebraically a class of (exactly solvable) quantum problems, its generalization to higher dimensions has not been explored yet. Here we provide a formal framework in HSUSY QM for two-dimensional quantum mechanical systems for which shape invariance holds. Given the knowledge of one eigenvalue and eigenfunction, shape invariance allows us to construct a chain of new eigenfunctions and eigenvalues. These methods are applied to a two-dimensional quantum system, and partial explicit solvability is achieved in the sense that only part of the spectrum is found analytically and a limited set of eigenfunctions is constructed explicitly.

Characterizing Measurement Error in Scores Across Studies: Some Recommendations for Conducting “Reliability Generalization” Studies

Abstract: T. Vacha-Haase (1998) proposed her “reliability generalization” methodology to characterize (a) typical score reliability for a measure across studies, (b) the variability of score reliabilities, a...

On-line support vector machine regression

Abstract: This paper describes an on-line method for building e-insensitive support vector machines for regression as described in [12]. The method is an extension of the method developed by [1] for building incremental support vector machines for classification. Machines obtained by using this approach are equivalent to the ones obtained by applying exact methods like quadratic programming, but they are obtained more quickly and allow the incremental addition of new points, removal of existing points and update of target values for existing data. This development opens the application of SVM regression to areas such as on-line prediction of temporal series or generalization of value functions in reinforcement learning.

Regular Tree Model Checking

Abstract: In this paper, we present an approach for algorithmic verification of infinite-state systems with a parameterized tree topology. Our work is a generalization of regular model checking, where we extend the work done with strings toward trees. States are represented by trees over a finite alphabet, and transition relations by regular, structure preserving relations on trees. We use an automata theoretic method to compute the transitive closure of such a transition relation. Although the method is incomplete, we present sufficient conditions to ensure termination.We have implemented a prototype for our algorithm and show the result of its application on a number of examples.

Selection of Meta-parameters for Support Vector Regression

Abstract: We propose practical recommendations for selecting metaparameters for SVM regression (that is, ? -insensitive zone and regularization parameter C). The proposed methodology advocates analytic parameter selection directly from the training data, rather than resampling approaches commonly used in SVM applications. Good generalization performance of the proposed parameter selection is demonstrated empirically using several lowdimensional and high-dimensional regression problems. In addition, we compare generalization performance of SVM regression (with proposed choice?) with robust regression using 'least-modulus' loss function (? =0). These comparisons indicate superior generalization performance of SVM regression.

Parameter optimization for machine-learning of word sense disambiguation

Abstract: Various Machine Learning (ML) approaches have been demonstrated to produce relatively successful Word Sense Disambiguation (WSD) systems. There are still unexplained differences among the performance measurements of different algorithms, hence it is warranted to deepen the investigation into which algorithm has the right ‘bias’ for this task. In this paper, we show that this is not easy to accomplish, due to intricate interactions between information sources, parameter settings, and properties of the training data. We investigate the impact of parameter optimization on generalization accuracy in a memory-based learning approach to English and Dutch WSD. A ‘word-expert’ architecture was adopted, yielding a set of classifiers, each specialized in one single wordform. The experts consist of multiple memory-based learning classifiers, each taking different information sources as input, combined in a voting scheme. We optimized the architectural and parametric settings for each individual word-expert by performing cross-validation experiments on the learning material. The results of these experiments show that the variation of both the algorithmic parameters and the information sources available to the classifiers leads to large fluctuations in accuracy. We demonstrate that optimization per word-expert leads to an overall significant improvement in the generalization accuracies of the produced WSD systems.

Convergent adaptive finite elements for the nonlinear Laplacian

Abstract: The numerical solution of the homogeneous Dirichlet problem for the p-Laplacian, $p\in ]1,\infty[$ , is considered. We propose an adaptive algorithm with continuous piecewise affine finite elements and prove that the approximate solutions converge to the exact one. While the algorithm is a rather straight-forward generalization of those for the linear case p=2, the proof of its convergence is different. In particular, it does not rely on a strict error reduction.

Visual evidence: A Seventh Man, the specified generalization, and the work of the reader

Abstract: How do photographs provide evidence for social science arguments? Analysis of A Seventh Man , a book about migrant labor in Europe, by John Berger and Jean Mohr, suggests that they do this by providing specified generalizations, which state a general idea embodied in images of specific people, places, and events.

Beyond NP: Arc-consistency for quantified constraints

Abstract: The generalization of the satisfiability problem with arbitrary quantifiers is a challenging problem of both theoretical and practical relevance. Being PSPACE-complete, it provides a canonical model for solving other PSPACE tasks which naturally arise in AI. Effective SAT-based solvers have been designed very recently for the special case of boolean constraints. We propose to consider the more general problem where constraints are arbitrary relations over finite domains. Adopting the viewpoint of constraint-propagation techniques so successful for CSPs, we provide a theoretical study of this problem. Our main result is to propose quantified arc-consistency as a natural extension of the classical CSP notion.

Marginalizing and conditioning in graphical models

Abstract: A class of graphs is introduced which is closed under marginalizing and conditioning. It is shown that these operations can be executed by performing in arbitrary order a sequence of simple, strictly local operations on the graph at hand. The results are based on a simplification of J. Pearl's notion of $d$-separation. As the simplification does not change the separation properties of graphs for which the original $d$-separation concept is applicable (e.g., directed graphs), it constitutes a true generalization of the latter concept to the present class of graphs.