Grammatical Inference and First Language Acquisition
01 Aug 2004-pp 27-34
TL;DR: This paper constructs an appropriate formalisation of the problem using a modern vocabulary drawn from statistical learning theory and grammatical inference and says that a variant of the Probably Approximately Correct (PAC) learning framework with positive samples only, modified so it is not completely distribution free is the appropriate choice.
Abstract: One argument for parametric models of language has been learnability in the context of first language acquisition. The claim is made that “logical” arguments from learnability theory require non-trivial constraints on the class of languages. Initial formalisations of the problem (Gold, 1967) are however inapplicable to this particular situation. In this paper we construct an appropriate formalisation of the problem using a modern vocabulary drawn from statistical learning theory and grammatical inference and looking in detail at the relevant empirical facts. We claim that a variant of the Probably Approximately Correct (PAC) learning framework (Valiant, 1984) with positive samples only, modified so it is not completely distribution free is the appropriate choice. Some negative results derived from cryptographic problems (Kearns et al., 1994) appear to apply in this situation but the existence of algorithms with provably good performance (Ron et al., 1995) and subsequent work, shows how these negative results are not as strong as they initially appear, and that recent algorithms for learning regular languages partially satisfy our criteria. We then discuss the applicability of these results to parametric and non-
Citations
More filters
TL;DR: It is proposed that the study of embodied cognitive agents, such as humanoid robots, can advance the understanding of the cognitive development of complex sensorimotor, linguistic, and social learning skills, which will benefit the design of cognitive robots capable of learning to handle and manipulate objects and tools autonomously.
Abstract: This position paper proposes that the study of embodied cognitive agents, such as humanoid robots, can advance our understanding of the cognitive development of complex sensorimotor, linguistic, and social learning skills. This in turn will benefit the design of cognitive robots capable of learning to handle and manipulate objects and tools autonomously, to cooperate and communicate with other robots and humans, and to adapt their abilities to changing internal, environmental, and social conditions. Four key areas of research challenges are discussed, specifically for the issues related to the understanding of: 1) how agents learn and represent compositional actions; 2) how agents learn and represent compositional lexica; 3) the dynamics of social interaction and learning; and 4) how compositional action and language representations are integrated to bootstrap the cognitive system. The review of specific issues and progress in these areas is then translated into a practical roadmap based on a series of milestones. These milestones provide a possible set of cognitive robotics goals and test scenarios, thus acting as a research roadmap for future work on cognitive developmental robotics.
190 citations
Cites background from "Grammatical Inference and First Lan..."
...…agents should possess a categorical perception ability which allows them to transform continuous signals perceived by sensory organs into internal states or internal dynamics in which members of the same category resemble one another more than they resemble members of other categories [97]....
[...]
30 Aug 2016
TL;DR: The proposed methodology employs Womb Grammars in parsing a subset of noun phrases of the target language Yorùbá, from the grammar of the source language English, which is described as properties between pairs of constituents.
Abstract: We address the problem of inducing the grammar of an under-resourced language, Yoruba, from the grammar of English using an efficient and, linguistically savvy, constraint solving model of grammar induction –Womb Grammars (WG). Our proposed methodology adapts WG for parsing a subset of noun phrases of the target language Yoruba, from the grammar of the source language English, which is described as properties between pairs of constituents. Our model is implemented in CHRG (Constraint Handling Rule Grammar) and, it has been used for inducing the grammar of a useful subset of Yoruba Noun Phrases. Interesting extensions to the original Womb Grammar model are presented, motivated by the specific needs of Yoruba and, similar tone languages.
6 citations
Cites methods from "Grammatical Inference and First Lan..."
...In language acquisition, grammatical inference has been explored in relation to the role of semantics in how children acquire language [14], it has also been used for developing models for empirical analysis of first language acquisition [36] and to answer several other language acquisition questions [11, 36, 12]....
[...]
Book•
30 Dec 2013
6 citations
Cites background from "Grammatical Inference and First Lan..."
...…The Real Problem of Language Acquisition • If all there was to language was an encoding of propositions that the child already has in mind, as in part III, it is not clear why they should bother to learn language at all, as Clark (2004) points out, in defence of a PAC learning model (!)....
[...]
01 Jan 2013
TL;DR: Two simple statistical inference procedures for probabilistic grammars are described and it is shown that they succeed in learning both the phrase structure and lexical entries without explicit negative evidence in a situation that seems highly challenging for any “staged” learner.
Abstract: Probabilistic grammars define a set of well-formed or grammatical linguistic structures, just as all grammars do. But in addition probabilistic grammars also define probability distributions over these structures, which a statistical inference procedure can exploit. This paper describes two simple statistical inference procedures for probabilistic grammars and discusses some of the challenges involved in generalising them to more realistic settings. The first three sections of this paper review important ideas from statistical estimation, namely the Maximum Likelihood Principle and Bayesian estimation, and show how they can be applied to learn the parameters of a Probabilistic Context-Free Grammar from surface strings alone in an idealized “toy” scenario. We show that they succeed in learning both the phrase structure and lexical entries without explicit negative evidence in a situation that seems highly challenging for any “staged” learner, i.e., a learner that learns either lexical entries or phrase structure rules first. Then we discuss attempts to generalize these kinds of methods to more realistic scenarios, and point out some of the challenges that are involved in doing this. One of the main themes of this paper is how little we know about the the linguistic implications of the various approximations made by current statistical inference algorithms. In part because of this, much of the work in this field takes an experimental, “try it and see” approach. The presentation in this paper is deliberately informal: for a more mathematical presentation of much of this material see e.g., Geman and Johnson (2004). The software used to produce the results below is available from http://web.science.mq.edu.au/ mjohnson.
5 citations
Cites background from "Grammatical Inference and First Lan..."
...None of this is specific to PCFGs: see Clark (2004) for a more detailed discussion of the role of negative evidence in statistical learning....
[...]
01 Jan 2014
TL;DR: The goal here is to present the state of the art of the relationship among three different areas: agent technologies, learning models and formal languages and to emphasize the importance of this interdisciplinary research.
Abstract: Considering the important role of interdiciplinarity in current research, this article provides an overview of the interchange of methods among three different areas: agent technologies, learning models and formal languages. The ability to learn is one of the most fundamental attributes of the intelligent behaviour. Therefore, any progress in the theory and computer modelling of learning processes is of great significance to fields concerning with understanding intelligence, and this includes, of course, artificial intelligence and intelligent agent technology. Agent technologies can offer good solutions and alternative frameworks to classic models in the area of computing languages and this can benefit formal models of learning. Formal language theory –considered as the stem of theoretical computer science– provides mathematical tools for the description of linguistic phenomena. This theory is central to grammatical inference, a subfield of machine learning. The interest of the interrelation among these disciplines is based on the idea that the collaboration among researchers in these areas can clearly improve their respective fields. Our goal here is to present the state-of-the art of the relationship among these three areas and to emphasize the importance of this interdisciplinary research.
3 citations
References
More filters
05 Nov 1984
TL;DR: This paper regards learning as the phenomenon of knowledge acquisition in the absence of explicit programming, and gives a precise methodology for studying this phenomenon from a computational viewpoint.
Abstract: Humans appear to be able to learn new concepts without needing to be programmed explicitly in any conventional sense. In this paper we regard learning as the phenomenon of knowledge acquisition in the absence of explicit programming. We give a precise methodology for studying this phenomenon from a computational viewpoint. It consists of choosing an appropriate information gathering mechanism, the learning protocol, and exploring the class of concepts that can be learnt using it in a reasonable (polynomial) number of steps. We find that inherent algorithmic complexity appears to set serious limits to the range of concepts that can be so learnt. The methodology and results suggest concrete principles for designing realistic learning systems.
5,311 citations
"Grammatical Inference and First Lan..." refers background in this paper
...These considerations lead us to some variant of the Probably Approximately Correct (PAC) model of learning (Valiant, 1984)....
[...]
Book•
01 Jan 1986
TL;DR: The best available introduction to Chomsky's current ideas on syntax made accessible to the non-specialist can be found in this article, where Lightfoot, Newmeyer, and Moravcsik present an excellent contribution to the philosophy of language and philosophy of mind.
Abstract: Why do we know so much more than we have evidence for in certain areas, and so much less in others? In tackling these questions--Plato's and Orwell's problem--Chomsky again demonstrates his unequalled capacity to integrate vast amounts of material...A clear introduction to current thinking on grammatical theory. David W. Lightfoot, University of Maryland I feel that it is his most persuasive defense of the idea that the study of linguistic structure provides insight into the human mind. Frederick J. Newmeyer, University of Washington This is an excellent contribution to the philosophy of language and the philosophy of mind...The best available introduction to Chomsky's current ideas on syntax made accessible to the non-specialist. Julius M. Moravcsik, Stanford Unviersity
3,384 citations
Book•
01 Jan 1980
TL;DR: The authors of this book have developed a rigorous and unified theory that opens the study of language learnability to discoveries about the mechanisms of language acquisition in human beings and has important implications for linguistic theory, child language research, and the philosophy of language.
Abstract: The question of language learnability is central to modern linguistics. Yet, despite its importance, research into the problems of language learnability has rarely gone beyond the informal, commonsense intuitions that currently prevail among linguists and psychologists.By focusing their inquiry on formal language learnability theory--the interface of formal mathematical linguistics, linguistic theory and cognitive psychology--the authors of this book have developed a rigorous and unified theory that opens the study of language learnability to discoveries about the mechanisms of language acquisition in human beings. Their research has important implications for linguistic theory, child language research, and the philosophy of language."Formal Principles of Language Acquisition" develops rigorous mathematical methods for demonstrating the learnability of classes of grammars. It adapts the well-developed theories of transformational grammar to establish psychological motivation for a set of formal constraints on grammars sufficient for learnability. In addition, the research deals with such matters as the complex interaction between the mechanism of language learning and the learning environment, the empirical adequacy of the learnability constraints, feasibility and attainability of classes of grammars, the role of semantics in language learnability, and the adequacy of transformational grammars as models of human linguistic competence.This first serious and extended development of a formal and precise theory of language learnability will interest researchers in psychology and linguistics, and is recommended for use in graduate courses in language acquisition, linguistic theory, psycholinguistics, and mathematical linguistics, as well as interdisciplinary courses that deal with language learning, use, and philosophy.Contents: Methodological Considerations; Foundations of a Theory of Learnability; A Learnability Result for Transformational Grammar; Degree-2 Learnability; Linguistic Evidence for the Learnability Constraints; Function, Performance and Explanations; Further Issues: Linguistic Interaction, Invariance Principle, Open Problems; Notes, Bibliography, Index.
1,144 citations
"Grammatical Inference and First Lan..." refers background in this paper
...For some years, the relevance of formal results in grammatical inference to the empirical question of first language acquisition by infant children has been recognised (Wexler and Culicover, 1980)....
[...]
Book•
03 Apr 2012
TL;DR: The AG/PRO Parameter in Early Grammars: Some Empirical Inadequacies and Discontinuous Models of Linguistic Development.
Abstract: 1. Linguistic Theory and Syntactic Development.- 1. Introduction.- 2. A Parameterized Theory of UG.- 3. An Overview.- 3.1 A Note on Methodology.- 4. The Theory of Grammar.- Notes.- 2. The Null Subject Phenomenon.- 1. Introduction.- 2. The Structure of INFL.- 2.1 Rule R.- 3. Null Subjects and the Identity of AG.- 3.1 The Properties of PRO.- 3.1.1 Control of AG/PRO.- 3.1.2 Arbitrary Reference of AG/PRO.- 3.1.3 The Auxiliary Systems of Italian and English.- 4. Summary.- Notes.- 3. The AG/PRO Parameter in Early Grammars.- 1. Introduction.- 2. Null Subjects in Early Language.- 2.1 The Avoid Pronoun Principle.- 3. The Early Grammar of English (G1).- 3.1 The Auxiliaries in Early English.- 3.2 The Filtering Effect of Child Grammars.- 3.2.1 The Semi-Auxiliaries.- 3.2.2 Can't and Don't.- 3.3 G1 and the Syntax of Be.- 4. The Restructuring of G1.- 4.1 The Triggering Data.- 4.2 The Avoid Pronoun Principle in Child Language.- 5. Summary.- Notes.- 4. Some Comparative Data.- 1. Introduction.- 2. The Early Grammars of English and Italian: A Comparison.- 2.1 Postverbal Subjects.- 2.2 Modals in Early Italian.- 2.3 Italian Be.- 3. Early German.- Notes.- 5. Discontinuous Models of Linguistic Development.- 1. Introduction.- 2. Semantically-Based Child Grammars.- 3. Semantically-Based Grammars: Some Empirical Inadequacies.- 3.1 Evidence from Polish and Hebrew.- Notes.- 6. Further Issues in Acquisition Theory.- 1. Summary.- 2. The Initial State.- 2.1 The Subset Principle.- 2.2 The Theory of Markedness.- 2.3 The Isomorphism Principle.- 3. Instantaneous vs. Non-Instantaneous Acquisition 168 Notes.- Index of Names.- Index of Subjects.
781 citations
"Grammatical Inference and First Lan..." refers background in this paper
...This constraint is also necessary for parametric models to make testable empirical predictions both about language universals, developmental evidence and relationships between the two (Hyams, 1986)....
[...]
TL;DR: It is proved that a polynomial-time learning algorithm for Boolean formulae, deterministic finite automata or constant-depth threshold circuits would have dramatic consequences for cryptography and number theory and is applied to obtain strong intractability results for approximating a generalization of graph coloring.
Abstract: In this paper, we prove the intractability of learning several classes of Boolean functions in the distribution-free model (also called the Probably Approximately Correct or PAC model) of learning from examples. These results are representation independent, in that they hold regardless of the syntactic form in which the learner chooses to represent its hypotheses.Our methods reduce the problems of cracking a number of well-known public-key cryptosystems to the learning problems. We prove that a polynomial-time learning algorithm for Boolean formulae, deterministic finite automata or constant-depth threshold circuits would have dramatic consequences for cryptography and number theory. In particular, such an algorithm could be used to break the RSA cryptosystem, factor Blum integers (composite numbers equivalent to 3 modulo 4), and detect quadratic residues. The results hold even if the learning algorithm is only required to obtain a slight advantage in prediction over random guessing. The techniques used demonstrate an interesting duality between learning and cryptography.We also apply our results to obtain strong intractability results for approximating a generalization of graph coloring.
631 citations