Showing papers on "Sequence learning published in 1990"

Learning Logical Definitions from Relations

Abstract: This paper describes FOIL, a system that learns Horn clauses from data expressed as relations. FOIL is based on ideas that have proved effective in attribute-value learning systems, but extends them to a first-order formalism. This new system has been applied successfully to several tasks taken from the machine learning literature.

Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions.

Abstract: Multilayer connectionist models of memory based on the encoder model using the backpropagation learning rule are evaluated. The models are applied to standard recognition memory procedures in which items are studied sequentially and then tested for retention. Sequential learning in these models leads to 2 major problems. First, well-learned information is forgotten rapidly as new information is learned. Second, discrimination between studied items and new items either decreases or is nonmonotonic as a function of learning. To address these problems, manipulations of the network within the multilayer model and several variants of the multilayer model were examined, including a model with prelearned memory and a context model, but none solved the problems. The problems discussed provide limitations on connectionist models applied to human memory and in tasks where information to be learned is not all available during learning. The first stage of the connectionist revolution in psychology is reaching maturity and perhaps drawing to an end. This stage has been concerned with the exploration of classes of models, and the criteria that have been used to evaluate the success of an application have been necessarily loose. In the early stages of development of a new approach, lax acceptability criteria are appropriate because of the large range of models to be examined. However, there comes a second stage when the models serve as competitors to existing models developed within other theoretical frameworks, and they have to be competitively evaluated according to more stringent criteria. A few notable connectionist models have reached these standards, whereas others have not. The second stage of development also requires that the connectionist models be evaluated in areas where their potential for success is not immediately obvious. One such area is recognition memory. The work presented in this article evaluates several variants of the multilayer connectionist model as accounts of empirical results in this area. I mainly discuss multilayer models using the error-correcting backpropagation algorithm and do not address other architectures such as adaptive resonance schemes (Carpenter & Grossberg, 1987). Before launching into the modeling of recognition memory, I need to specify the aims and rules under which this project was carried out. This is important in a new area of inquiry because there are many divergent views about what needs to be

Attention and structure in sequence learning.

Abstract: In this study we investigated the role of attention, sequence structure, and effector specificity in learning a structured sequence of actions

Phases of Meaningful Learning

Abstract: The research literature is examined for evidence suggesting that in complex, meaningful learning the learner passes through a series of stages or phases during which the learning process and the variables influencing it change systematically. After discussing various conceptual and methodological issues, phase theories in both simpler and more meaningful forms of learning are reviewed. Finally, the initial, intermediate, and terminal phases of learning are discussed. It is suggested that during the initial phase of learning the individual typically acquires isolated facts that are interpreted in terms of preexisting schemata and added to existing knowledge structures. Gradually, the learner begins to assemble these pieces into new schemata that provide him or her with more conceptual power until a level of automaticity is achieved.

Aspects of an Educational Learning Theory

Abstract: Contemporary cognitive science has created the possibility of an educational learning theory closely related to existing cognitive theories but operative at a higher level of description. Issues that must be addressed in developing such a theory are: How much of the external world should be included in cognitive descriptions, how to avoid degenerate functionalism, what needs explaining by an educational learning theory, and what its units of analysis should be. It is proposed that a constitutive problem for educational learning theory is the explanation of difficult learning. A unit of analysis called a contextual module is proposed and applied to this problem. An advantage of contextual modules is that they treat environmental situatedness as an emergent property resulting from reasonably well understood processes.

Active and Dynamic Self-Regulation of Learning Processes

Abstract: It is generally assumed that learning is internalization of externally available knowledge and occurs under the active control of one internal source of self-regulation: executive self-regulation. This article argues that these assumptions undermine the creative and multisource nature of learning, limit its domain to incremental learning of facts and definitions, and are largely responsible for achievement and motivational problems children experience in schools. The article defines learning as creative reconceptualization of internal knowledge and proposes two different sources of internal self-regulation: one to regulate largely the sequential aspect of learning and another to coordinate its simultaneous aspect. To extend the domain of learning beyond incremental memorization of facts, both sources of internal self-regulation must be the target of cognitive and metacognitive instruction and research.

Learning: Behavior and cognition

Abstract: An introduction to associative learning. Classical conditioning: Foundations of conditioning Principles and applications Theories of conditioning. Instrumental conditioning: Reinforcement Response suppression Applications Theories of reinforcement: the law of effect revisited. Theoretical processes in associative learning: Learning in an evolutionary context What is learned? Associative versus cognitive theories of learning How is it learned? An information processing model Is associative learning simple or complex?

Complex temporal sequence learning based on short-term memory

Abstract: An approach to storing of temporal sequences that deals with complex temporal sequences directly is presented Short-term memory (STM) is modeled by units comprised of recurrent excitatory connections between two neurons A dual-neuron model is proposed By applying the Hebbian learning rule at each synapse and a normalization rule among all synaptic weights of a neuron, it is shown that a quantity called the input potential increases monotonically with sequence presentation, and that the neuron can only be fired when its input signals are arranged in a specific sequence These sequence-detecting neurons form the basis for a model of complex sequence recognition that can tolerate distortions of the learned sequences A recurrent network of two layers is provided for reproducing complex sequences >

Learning Strategies in Second Language Acquisition: A cognitive theory of learning

Abstract: This chapter describes the rationale for advancing a cognitively based theory in second language acquisition and presents the foundation for the theory as it relates to constructs that will be discussed in later chapters. We suggest that second language acquisition cannot be understood without addressing the interaction between language and cognition, and indicate that at present this interaction is only poorly understood. Second language theorists have not capitalized on the available body of research and theory that has already been worked out in cognitive psychology. The chapter first identifies second language processes as having parallels with the way in which complex cognitive skills are described in cognitive theory. Aspects of cognitive theory are discussed that relate to memory representation and to the process of acquiring complex cognitive skills. The theory on which we rely most extensively is augmented in order to describe more adequately processes that occur in second language acquisition. We go on to discuss the way in which cognitive theory addresses specific language comprehension and language production processes, and conclude by indicating that cognitive theory can extend to describe learning strategies as complex cognitive skills. Finally, we introduce the major types of strategies on which we rely in later chapters. Background The fields of linguistics and cognitive psychology each contain separate paradigms for describing second language acquisition. Linguistic theories assume that language is learned separately from cognitive skills, operating according to different principles from most learned behaviors (e.g., Spolsky 1985).

Role of Cognitive Factors in the Acquisition of Cognitive Skill

Abstract: : This paper reviews recent studies and provides a theoretical framework for examining the relationships between individual differences in basic cognitive abilities and performance on tasks involving the acquisition of cognitive skill. A claim made is that much of cognitive skill can be characterized as (a) knowledge of how operators (symbols denoting an operation to be performed) in a skill domain actually work, and (b) knowledge of when to apply operators to achieve problem-solving goals. Studies of declarative learning, assumed to be the initial stage in skill acquisition, show that background knowledge and working memory capacity are primary determinants of success. Studies of cognitive skill learning per se make use of a distinction between attention capacity, which refers to how much information can be held in working memory at one time, and activation capacity, which refers to how long activation can be maintained. The success of learning by proceduralization is shown to depend on attention capacity, whereas the success of learning by composition is shown to depend additionally on activation capacity. Future research and the benefits of an individual differences approach to analyzing cognitive skill acquisition are discussed. Keywords: Aptitudes; Skills; Cognition; Cognitive ability; Computerized testing; Individual differences; Learning; Learning Abilities Measurement Program (LAMP); Learning ability.

Learning from textbook knowledge: a case study

Abstract: One of the "grand challenges for machine learning" is the problem of learning from textbooks. This paper addresses the problem of learning from texts including omissions and inconsistencies that are clarified by illustrative examples. To avoid problems in natural language understanding, we consider a simplification of this problem in which the text has been manually translated into a logical theory. This learning problem is solvable by a technique that we call analogical abductive explanation based learning (ANA-EBL). Formal evidence and experimental results in the domain of contract bridge show that the learning technique is both efficient and effective.

A hybrid connectionist, symbolic learning system

Abstract: This paper describes the learning part of a system which has been developed to provide expert systems capability augmented with learning. The learning scheme is a hybrid connectionist, symbolic one. A network representation is used. Learning may be done incrementally and requires only one pass through the data set to be learned. Attribute, value pairs are supported as a variable implementation. Variables are represented by groups of connected cells in the network. The learning algorithm is described and an example given. Current results are discussed, which include learning the well-known Iris data set. The results show that the system has promise.

Explanation-Based Learning of Correctness: Towards a Model of the Self-Explanation Effect

Abstract: : Two major techniques in machine learning, explanation-based learning and explanation completion, are both superficially plausible models for Chi's self-explanation effect, wherein the amount of explanation given to examples while studying them correlates with the amount the subject learns from them. We attempted to simulate Chi's protocol data with the simpler of the two learning processes, explanation completion, in order to find out how much of the self- explanation effect it could account for. Although explanation completion did not turn out to be a good model of the data, we discovered a new learning technique, called explanation-based learning of correctness, that combines explanation- based learning and explanation completion and does a much better job of explaining the protocol data. The new learning process is based on the assumption that subjects uses a certain kind of plausible reasoning. Keywords: Self-explanation, Cognitive modelling, Learning, Explanation-based learning, Skill acquisition.

Is learning rate a good performance criterion for learning

Abstract: The most frequently used measure of performance for reinforcement learning algorithms is learning rate. That is, how many learning trials are required before the program is able to perform its task adequately. In this paper, we argue that this is not necessarily the best measure of performance and, in some cases, can even be misleading. In control tasks, such as pole balancing, we have found that a program that learns to balance the pole quickly produces a control strategy that is so specific as to make it impossible to transfer expertise from one related task to another. We examine the reasons for this and suggest ways of obtaining general control strategies. We also make the conjecture that, as a broad principle, there is a trade-off between rapid learning rate and the ability to generalise. We also introduce methods for analysing the results of reinforcement learning algorithms to produce readable control rules.

An information processing approach to the investigation of mathematical problem solving at secondary and university levels

Abstract: This thesis contains ten chapters: three of them are background literature and five have resulted from practical work during the whole period of the research. Chapter 9 is an attempt to extend the idea of the demand of a task, while the last chapter contains conclusions and suggestions for further research. In Chapter 1, the theories of Piaget, Gagne and Ausubel are described and compared with each other. Piaget's stages of intellectual development and how learning processes take place are described and explained. The contribution of the theory in the domains of curriculum, teaching Piagetian tasks as subject matter and matching instruction to development stages is stressed. However, the serious challenges to the theory are (i) horizontal decalage phenomenon, (ii) relating stages with age, (iii) assessing competence and readiness. Gagne's model of an hierarchy of learning comes from theories of transfer. It is built from the top down. The conditions of learning are internal and external and ranged from signal learning to problem solving. The learning process is based on associational chains. The difficulty of the model comes from the nature of a learning hierarchy and its validation. Ausubel's theory of meaningful learning is based on what the learner already knows. It is built up from seven elements which range from meaningful learning to the advance organizer. Meaningful learning occurs as a result of interaction between new and existing knowledge and its variation is due to the growth of differentiation and integration of relevant items in cognitive structure. Failure in learning may occur in situations such as those of conflicting ideas and forgetting. In Chapter 2, Information Processing Theories of Learning are described and the justification of these theories as a fourth paradigm to guide thinking about research is stressed. A model of human memory is given and the components of memory and their features are listed. Stress is placed upon the memory processes and their levels, organization of knowledge, working memory and chunking as a remedy for overload. Two examples of these theories are given namely Neo-Piagetian Theory and the Predictive Model of Holding-Thinking Space. The main goal of the former is to make Piaget's theory functional not just structural. The latter relates performance to the amount of information to be processed in learning and problem solving. This model is applied in both University and Algerian samples. This can be found in Chapter 3. In Chapter 4, the field dependent-independent cognitive style is considered as an important factor affecting performance. The differences between field dependent-independent people may be related to the perceptual field, selected information and the level of guidance. The reason for these differences may be due to the way in which information is both analysed and represented in memory. The practical work has been done with both University and Algerian samples. In Chapter 5, some other factors are described. Most of them are concerned directly with the subject matter. The activities involved in learning mathematics are classified and attention is given to Polya's version of heuristic strategies. The concept of understanding is considered as a basic goal of education and its meaning is given in three different aspects. Most attention is given to the third one, which is known as alternative framework or misconception. The levels of understanding of Skemp are defined and their goals are stressed. The causes of learning difficulties in mathematics are listed, while the different forms of mathematical language are described and their effect on learning is noted. In Chapter 6, the analysis of Paper I (multiple-choice questions) has been done for preliminary Examination of four Scottish schools (a fifth school used only traditional questions). The experimental work is concerned with language, formulation and type of question.

Implicit concept formation

Abstract: This thesis provides a conceptual and empirical analysis of implicit concept formation. A review of concept formation studies highlights the need for improving existing methodology in establish- ing the claim for implicit concept formation. Eight experiments are reported that address this aim. A review of theoretical issues highlights the need for computational modelling to elucidate the nature of implicit learning. Two chapters address the feasibility of different exemplar and Connectionist models in accounting for how subjects perform on tasks typically employed in the implicit learn- ing literature. The first five experiments use a concept formation task that involves classifying "computer people" as belonging to a particular town or income category. A number of manipulations are made of the underlying rule to be learned and of the cover task given subjects. In all cases, the knowledge underlying classification performance can be elicited both by free recall and by forced choice tasks. The final three experiments employ Reber's (e.g., 1989) grammar learning paradigm. More rigorous methods for eliciting the knowledge underlying classification performance are employed than have been used previously by Reber. The knowledge underlying clas- sification performance is not elicited by free recall, but is elicited by a forced-choice measure. The robustness of the learning in this paradigm is investigated by using a secondary task methodol- ogy. Concurrent random number generation interferes with all knowledge measures. A number of parameter-free Connectionist and exemplar models of artificial grammar learning are tested against the experimental data. The importance of different assumptions regarding the coding of features and the learning rule used is investigated by determin- ing the performance of the model with and without each assumption. Only one class of Connectionist model passes all the tests. Fur- ther, this class of model can simulate subject performance in a different task domain. The relevance of these empirical and theoretical results for understanding implicit learning is discussed, and suggestions are made for future research.

A study of the negative transfer problem in artificial neural networks

Abstract: In this study a second sequential learning problem in artificial neural networks is reported. The problem is known as the negative transfer problem. The negative transfer problem is the degradation in the learning ability and performance of artificial neural networks over successive training sessions. Three experiments were designed and conducted to study the effect of negative transfer on the back-propagation model. The effect of negative transfer on the error rate of the network and on the number of sweeps needed for learning was studied. The error rate and the number of sweeps increase as a function of successive training sessions. The two main causes of the negative transfer were found to be: the unbounded growth of the weight values; and the expansion of the search space for subsequent training sessions. Four methods found in the literature were developed to treat the negative transfer problem. The methods were unsuccessful in solving the problem. A new sequential learning framework is introduced. The framework is known as the adaptive memory consolidation. The framework consists of two independent processes. The first process is a weight clipping process which insures that the weights do not attain extreme values. This process is applied to each weight of the network at every sweep. The second process is a memory consolidation process which stabilizes and constrains the search space. This process is applied to the weights between training sessions. The adaptive memory consolidation framework uses past learning to improve future learning. This makes the framework completely adaptive and guided by the network. Two methods were developed based on the adaptive memory consolidation framework. The methods are: the adaptive decay method; and the standard weights method. The methods are computationally simple and inexpensive to use. The methods were successful in eliminating and minimizing the effect of the negative transfer problem. Additionally the methods improved the performance of the network when used in conjunction with shaping schedules.

Adaptive image filter

Abstract: PURPOSE:To realize the optimum filter in a short time by dividing a training pattern into plural partial patterns whose respective part can hold the statistical property of the whole patterns, and repeating learning. CONSTITUTION:A spatial filter is comprised of multilayer perceptrons 6, and a training control part 3 performs sequential learning by using one of partial learning pattern images after dividing an image for training into partial sets whose statistical property included in an original image can be prevented from being missed. Training is performed with the same partial learning pattern image when the error of the output image of the filter goes less than the error at the training last time and a value decided in advance, and a leaning pattern image is replaced by the next partial learning pattern image when the errors are different from each other, then, the learning for all the partial learning pattern images are repeated until the error of the output image goes less than the value decided in advance. Therefore, waste in the training is eliminated, and a remarkably effective filter quickly is realized.

Sequential learning system for group unit of coupling in neural network

Abstract: PURPOSE: To prevent the breakdown of knowledge reflected at the time of setting an initial value of weight by converting a neural network of complete coupling to a network having a certain special structures by setting a group of coupling as a unit, adjusting the degree of learning at every coupling group and determining the sequence of leading. CONSTITUTION: A neural network learning means 4 sets a learning adjustment constant outputted by a learning adjustment constant output means 3 to all couplings in a neural network, and controls learning of the neural network. In accordance with the storage contents of a learning coupling group storage means 5, that is, the designation of a coupling group learned in each learning phase, a learning phase control means 6 allows the learning adjustment constant output means 3 to output, for instance, '1', and '0' to an in-group coupling or an inter-group coupling learned in each learning phase, and to all the coupling except the above coupling, respectively, and also, controls learning of each learning phase. In such a way, it can be prevented that knowledge reflected at the time of setting an initial value of weight is broken down. COPYRIGHT: (C)1992,JPO&Japio

ME—THE SLOW LEARNER and some of its implications

Abstract: The use of simulation ME-THE SLOW LEARNER to promote empathy with learning difficulty or handicap is described briefly together with a more detailed description of the different stages of the debriefing. There is a consideration of the Learning System as it is related to the simulation and to the development of learning handicap together with a brief consideration of the System of the Learner. Aspects of the growing data bank of responses to the simulation are discussed particularly as they relate to the origin and development of learning problems or handicaps. Some of the major learning difficulties identified by past participants in ME-THE SLOW LEARNER are explored. A brief consideration of the complex way in which learning difficulties or handicaps develop is considered and some theoretical perspectives are identified. Finally, the importance of emotion in learning contexts is explored. Full details of the simulation are to be found in Thatcher, D. C. and J. Robinson, 1987, ME-THE SLOW LEARNER-A Manual. Fareham, Hants.: Solent Simulations.