Chunking (psychology)

Abstract: We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.

Abstract: Introduction The Nature of Theories What Are Unified Theories of Cognition? Is Psychology Ready for Unified Theories? The Task of the Book Foundations of Cognitive Science Behaving Systems Knowledge Systems Representation Machines and Computation Symbols Architectures Intelligence Search and Problem Spaces Preparation and Deliberation Summary Human Cognitive Architecture The Human Is a Symbol System System Levels The Time Scale of Human Action The Biological Band The Neural Circuit Level The Real-Time Constraint on Cognition The Cognitive Band The Level of Simple Operations The First Level of Composed Operations The Intendedly Rational Band Higher Bands: Social, Historical, and Evolutionary Summary Symbolic Processing for Intelligence The Central Architecture for Performance Chunking The Total Cognitive System RI-Soar: Knowledge-Intensive and Knowledge-Lean Operation Designer-Soar: Difficult Intellectual Tasks Soar as an Intelligent System Mapping Soar onto Human Cognition Soar and the Shape of Human Cognition Summary Immediate Behavior The Scientific Role of Immediate-Response Data Methodological Preliminaries Functional Analysis of Immediate Responses The Simplest Response Task (SRI) The Two-Choice Response Task (2CRT) Stimulus-Response Compatibility (SRC) Discussion of the Three Analyses Item Recognition Typing Summary Memory, Learning, and Skill The Memory and Learning Hypothesis of Soar The Soar Qualitative Theory of Learning The Distinction between Episodic and Semantic Memory Data Chunking Skill Acquisition Short-Term Memory (STM) Summary Intendedly Rational Behavior Ciyptarithmetic Syllogisms Sentence Verification Summary Along the Frontiers Language Development The Biological Band The Social Band The Role of Applications How to Move toward Unified Theories of Cognition References Name Index Subject Index

Abstract: We describe new algorithms for training tagging models, as an alternative to maximum-entropy models or conditional random fields (CRFs). The algorithms rely on Viterbi decoding of training examples, combined with simple additive updates. We describe theory justifying the algorithms through a modification of the proof of convergence of the perceptron algorithm for classification problems. We give experimental results on part-of-speech tagging and base noun phrase chunking, in both cases showing improvements over results for a maximum-entropy tagger.

Abstract: If we take an existing supervised NLP system, a simple and general way to improve accuracy is to use unsupervised word representations as extra word features. We evaluate Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeddings of words on both NER and chunking. We use near state-of-the-art supervised baselines, and find that each of the three word representations improves the accuracy of these baselines. We find further improvements by combining different word representations. You can download our word features, for off-the-shelf use in existing NLP systems, as well as our code, here: http://metaoptimize.com/projects/wordreprs/

Abstract: Practice, and the performance improvement that it engenders, has long been a major topic in psychology In this paper, both experimental and theoretical approaches are employed in an investigation of the mechanisms underlying this improvement On the experimental side, it is argued that a single law, the power of law of practice, adequately describes all of the practice data On the theoretical side, a model of practice rooted in modern cognitive psychology, the chunking theory of learning, is formulated The paper consists of (1) the presentation of a set of empirical practice curves; (2) mathematical investigations into the nature of power law functions; (3) evaluations of the ability of three different classes of functions to adequately model the empirical curves; (4) a discussion of the existing models of practice; (5) a presentation of the chunking theory of learning