R. Bharat Rao

Bio: R. Bharat Rao is an academic researcher from University of Illinois at Urbana–Champaign. The author has contributed to research in topics: Generalization & Measure (mathematics). The author has an hindex of 4, co-authored 4 publications receiving 104 citations.

For Every Generalization Action, Is There Really an Equal and Opposite Reaction? Analysis of the Conservation Law for Generalization Performance

Abstract: The “Conservation Law for Generalization Performance” [Schaffer, 1994] states that for any learning algorithm and bias, “generalization is a zero-sum enterprise.” In this paper we study the law and show that while the law is true, the manner in which the Conservation Law adds up generalization performance over all target concepts, without regard to the probability with which each concept occurs, is relevant only in a uniformly random universe. We then introduce a more meaningful measure of generalization, expected generalization performance. Unlike the Conservation Law's measure of generalization performance (which is, in essence, defined to be zero), expected generalization performance is conserved only when certain symmetric properties hold in our universe. There is no reason to believe, a priori, that such symmetries exist; learning algorithms may well exhibit non-zero (expected) generalization performance.

Learning engineering models with the minimum description length principle

Abstract: This paper discusses discovery of mathematical models from engineering data sets. KEDS, a Knowledge-based Equation Discovery System, identifies several potentially overlapping regions in the problem space, each associated with an equation of different complexity and accuracy. The minimum description length principle, together with the KEDS algorithm, is used to guide the partitioning of the problem space. The KEDSMDL algorithm has been tested on discovering models for predicting the performance efficiencies of an internal combustion engine.

For every generalization action, is there really an equal and opposite reaction?

Abstract: An improved vehicle rear-view mirror assembly of the type in which the mirror support arm is carried by means of at least one movable joint from the mounting part of the mirror assembly which is adapted to be secured to the bodywork of the vehicle includes a shroud of a resilient and flexible material mounted on the back of the mirror, and an extension having at least one movable joint with the end region of the extension in sealing relation with the mounting part, the extension enshrouded by a bellows-like tubular gaiter of flexible material. The shroud and the extension reduce the risk of injury to a person who may be accidentally hit by, or hit, the mirror assembly and the bellows-like gaiter also protects the movable joint or joints from dirt and moisture while readily flexing to allow the joint or joints to move or to be adjusted.

Knowledge-based equation discovery in engineering domains

Abstract: Machine learning techniques that can compress datasets into relationships that underlie the data have great utility for engineers. Most model discovery systems are not well suited to discovering comprehensible models in engineering domains. In this paper we present KEDS, a Knowledge-based Equation Discovery System, which uses a model-driven approach to discover equations and thereby build models from engineering data.

A perspective view and survey of meta-learning

Abstract: Different researchers hold different views of what the term meta-learning exactly means. The first part of this paper provides our own perspective view in which the goal is to build self-adaptive learners (i.e. learning algorithms that improve their bias dynamically through experience by accumulating meta-knowledge). The second part provides a survey of meta-learning as reported by the machine-learning literature. We find that, despite different views and research lines, a question remains constant: how can we exploit knowledge about learning (i.e. meta-knowledge) to improve the performance of learning algorithms? Clearly the answer to this question is key to the advancement of the field and continues being the subject of intensive research.

Data mining with an ant colony optimization algorithm

Abstract: The paper proposes an algorithm for data mining called Ant-Miner (ant-colony-based data miner). The goal of Ant-Miner is to extract classification rules from data. The algorithm is inspired by both research on the behavior of real ant colonies and some data mining concepts as well as principles. We compare the performance of Ant-Miner with CN2, a well-known data mining algorithm for classification, in six public domain data sets. The results provide evidence that: 1) Ant-Miner is competitive with CN2 with respect to predictive accuracy, and 2) the rule lists discovered by Ant-Miner are considerably simpler (smaller) than those discovered by CN2.

MultiBoosting: A Technique for Combining Boosting and Wagging

Abstract: MultiBoosting is an extension to the highly successful AdaBoost technique for forming decision committees. MultiBoosting can be viewed as combining AdaBoost with wagging. It is able to harness both AdaBoost's high bias and variance reduction with wagging's superior variance reduction. Using C4.5 as the base learning algorithm, MultiBoosting is demonstrated to produce decision committees with lower error than either AdaBoost or wagging significantly more often than the reverse over a large representative cross-section of UCI data sets. It offers the further advantage over AdaBoost of suiting parallel execution.

Substructure discovery using minimum description length and background knowledge

Abstract: The ability to identify interesting and repetitive substructures is an essential component to discovering knowledge in structural data. We describe a new version of our SUBDUE substructure discovery system based on the minimum description length principle. The SUBDUE system discovers substructures that compress the original data and represent structural concepts in the data. By replacing previously-discovered substructures in the data, multiple passes of SUBDUE produce a hierarchical description of the structural regularities in the data. SUBDUE uses a computationally-bounded inexact graph match that identifies similar, but not identical, instances of a substructure and finds an approximate measure of closeness of two substructures when under computational constraints. In addition to the minimumdescription length principle, other background knowledge can be used by SUBDUE to guide the search towards more appropriate substructures. Experiments in a variety of domains demonstrate SUBDUE's ability to find substructures capable of compressing the original data and to discover structural concepts important to the domain.

The Role of Occam‘s Razor in Knowledge Discovery

Abstract: Many KDD systems incorporate an implicit or explicit preference for simpler models, but this use of “Occam‘s razor” has been strongly criticized by several authors (e.g., Schaffer, 1993s Webb, 1996). This controversy arises partly because Occam‘s razor has been interpreted in two quite different ways. The first interpretation (simplicity is a goal in itself) is essentially correct, but is at heart a preference for more comprehensible models. The second interpretation (simplicity leads to greater accuracy) is much more problematic. A critical review of the theoretical arguments for and against it shows that it is unfounded as a universal principle, and demonstrably false. A review of empirical evidence shows that it also fails as a practical heuristic. This article argues that its continued use in KDD risks causing significant opportunities to be missed, and should therefore be restricted to the comparatively few applications where it is appropriate. The article proposes and reviews the use of domain constraints as an alternative for avoiding overfitting, and examines possible methods for handling the accuracy–comprehensibility trade-off.