Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

We present conditional random fields , a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.

/pdf/conditional-random-fields-probabilistic-models-for-qeo4l25dzg.pdf

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

https://ssli.ee.washington.edu/people/duh/projects/CRFintro.pdf

Probabilistic Models for Segmenting and Labeling Sequence Data

Recent studies of eye movements in reading and other information processing tasks, such as music reading, typing, visual search, and scene perception, are reviewed. The major emphasis of the review is on reading as a specific example of cognitive processing. Basic topics discussed with respect to reading are (a) the characteristics of eye movements, (b) the perceptual span, (c) integration of information across saccades, (d) eye movement control, and (e) individual differences (including dyslexia). Similar topics are discussed with respect to the other tasks examined. The basic theme of the review is that eye movement data reflect moment-to-moment cognitive processes in the various tasks examined. Theoretical and practical considerations concerning the use of eye movement data are also discussed.

/pdf/eye-movements-in-reading-and-information-processing-20-years-3z5ay8hdwc.pdf

Eye movements in reading and information processing: 20 years of research.

Prefactory Note. Acknowledgments. 1. Preliminaries:. Background and Overview. Optimality. Overall Structure of the Argument. Overview of Part I. 2. Optimality in Grammar: Core Syllabification in Imdlawn Tashlhiyt Berber:. The Heart of Dell & Elmedlaoui. Optimality Theory. Summary of discussion to date. 3. Generalization--Forms in Domination Hierarchies IBlocking and Triggering: Profuseness and Economy:. Epenthetic Structure. Do Something Only When: The Failure of Bottom--up Constructionism. 4. Generalization--Forms in Domination Hierarchies IIDo Something Except When: Blocking, or The Theory of Profuseness:. Edge--Oriented Infixation. Interaction of Weight Effects with Extrametricality. Background: Prominence--Driven Stress Systems. The Interaction of Weight and Extrametricality: Kelkar's Hindi/Urdu. Nonfinality and Nonexhaustiveness. Nonfinality and the Laws of Foot Form: Raw Minimality. Nonfinality and the Laws of Foot Form:Extended Minimality Effects. Summary of Discussion of the Except When Effect. Except meets Only: Triggering and Blocking in a Single Grammar. 5. The Construction of Grammar in Optimality Theory:. Construction of Harmonic Orderings from Phonetic and Structural Scales. The Theory of Constraint Interaction. Comparison of Entire Candidates by a Single Constraint. Ons: Binary constraints. Hnuc: Non--binary constraints. Comparison of Entire Candidates by an Entire Constraint Hierarchy. Discussion. Non--locality of interaction. Strictness of domination. Serial vs. Parallel Harmony Evaluation and Gen. Binary vs. Non--binary constraints. Paoini's Theorem on Constraint Ranking. Overview of Part II. 6. Syllable Structure Typology I: the CV Theory:. The Jakobson Typology. The Faithfulness Interactions. Groundwork. Basic CV Syllable Theory. Onsets. Codas. The Theory of Epenthesis Sites. 7. Constraint Interaction in Lardil Phonology:. The Constraints. The Ranking. Some Ranking Logic. Ranking the Constraints. Verification of Forms. Consonant--Final Stems. Vowel Final Stems. Discussion. 8. Universal Syllable Theory II: Ordinal Construction of C/V and Onset/Coda Licensing Asymmetry:. Associational Harmony. Deconstructing Hnuc: Berber, Take 1. Restricting to Binary Marks. Reconstructing the C and V Classes: Emergent Parameter Setting via Constraint Ranking. Harmonic Completeness of Possible Onsets and Peaks. Peak-- and Margin--Affinity. Interactions with Parse. Restricting Deletion and Epenthesis. Further Necessary Conditions on Possible Onsets and Nuclei. Sufficient Conditions on Possible Onsets and Nuclei. The Typology of Onset, Nucleus, and Coda Inventories. The Typology of Onset and Nucleus Inventories. Onset/Coda Licensing Asymmetries. An Example: Berber, Take 2. Simplifying the Theory by Encapsulating Constraint Packages. Encapsulating the Association Hierarchies. An Example: Berber, Take 3. Sufficiency and Richness of the Encapsulated Theory. 9. Inventory Theory and the Lexicon:. Language--Particular Inventories. Harmonic Bounding and Nucleus, Syllable, and Word Inventories. Segmental Inventories. Universal Inventories. Segmental Inventories. Syllabic Inventories. Optimality in the Lexicon. 10. Foundational Issues and Theory--Comparisons:. Thinking about Optimality. Fear of Optimization. The Reassurance. The Connectionism Connection, and other Computation--based Comparisons. Why Optimality Theory has nothing to do with connectionism. Why Optimality Theory is deeply connected to connectionism. Harmony Maximization and Symbolic Cognition. Analysis of 'Phonotactics+Repair' Theories. CV Syllable Structure and Repair. General Structure of the Comparisons: Repair Analysis. Persistent Rule Theory. English Closed Syllable Shortening. Shona Tone Spreading. Summary. The Theory of Constraints and Repair Strategies. Appendix. A.1 The Cancellation and Cancellation/Domination Lemmas. A.2 CV Syllable Structure. A.3 Paoinia s Theorem on Constraint--ranking. References. Index of Constraints. Index of Languages. General Index

http://www.roozbehhormozi.persiangig.com/document/%5bAlan_Prince,_Paul_Smolensky%5d_Optimality_Theory_C(BookFi.org).pdf

Optimality Theory: Constraint Interaction in Generative Grammar

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Linguistics and Philosophy, 1987.

The English Noun Phrase in its Sentential Aspect

I begin with an intuition: when I read a sentence, I read it a chunk at a time. For example, the previous sentence breaks up something like this: 

(1)

[I begin] [with an intuition]: [when I read] [a sentence], [I read it] [a chunk] [at a time]






These chunks correspond in some way to prosodic patterns. It appears, for instance, that the strongest stresses in the sentence fall one to a chunk, and pauses are most likely to fall between chunks. Chunks also represent a grammatical watershed of sorts. The typical chunk consists of a single content word surrounded by a constellation of function words, matching a fixed template. A simple context-free grammar is quite adequate to describe the structure of chunks. By contrast, the relationships between chunks are mediated more by lexical selection than by rigid templates. Co-occurrence of chunks is determined not just by their syntactic categories, but is sensitive to the precise words that head them; and the order in which chunks occur is much more flexible than the order of words within chunks.

Parsing By Chunks

Finite state cascades represent an attractive architecture for parsing unrestricted text. Deterministic parsers specified by finite state cascades are fast and reliable. They can be extended at modest cost to construct parse trees with finite feature structures. Finally, such deterministic parsers do not necessarily involve trading off accuracy against speed — they may in fact be more accurate than exhaustive search stochastic context free parsers.

/pdf/partial-parsing-via-finite-state-cascades-39fg3f6qgz.pdf

Partial parsing via finite-state cascades

The problem of quantitatively comparing the performance of different broad-coverage grammars of English has to date resisted solution. Prima facie, known English grammars appear to disagree strongly with each other as to the elements of even the simplest sentences. For instance, the grammars of Steve Abney (Bellcore), Ezra Black (IBM), Dan Flickinger (Hewlett Packard), Claudia Gdaniec (Logos), Ralph Grishman and Tomek Strzalkowski (NYU), Phil Harrison (Boeing), Don Hindle (AT&T), Bob Ingria (BBN), and Mitch Marcus (U. of Pennsylvania) recognize in common only the following constituents, when each grammarian provides the single parse which he/she would ideally want his/her grammar to specify for three sample Brown Corpus sentences:The famed Yankee Clipper, now retired, has been assisting (as (a batting coach)).One of those capital-gains ventures, in fact, has saddled him (with Gore Court).He said this constituted a (very serious) misuse (of the (Criminal court) processes).

Procedure for quantitatively comparing the syntactic coverage of English grammars

This paper refines the analysis of cotraining, defines and evaluates a new co-training algorithm that has theoretical justification, gives a theoretical justification for the Yarowsky algorithm, and shows that co-training and the Yarowsky algorithm are based on different independence assumptions.

Steven Abney

Papers

The English Noun Phrase in its Sentential Aspect

Parsing By Chunks

Partial parsing via finite-state cascades

Procedure for quantitatively comparing the syntactic coverage of English grammars

Bootstrapping