Developments from enquiries into the learnability of the pattern languages from positive data

A pattern is a finite string of constant and variable symbols. The langauge generated by a pattern is the set of all strings of constant symbols which can be obtained from the pattern by substituting non-empty strings for variables. We study the learnability of one-variable pattern languages in the limit with respect to the update time needed for computing a new single hypothesis and the expected total learning time taken until convergence to a correct hypothesis. Our results are as follows. First, we design a consistent and set-driven learner that, using the concept of descriptive patterns, achieves update time O(n2logn), where n is the size of the input sample. The best previously known algorithm for computing descriptive one-variable patterns requires time O(n4logn) (cf. Angluin, J. Comput. Systems Sci. 21 (1) (1980) 46-62). Second, we give a parallel version of this algorithm that requires time O(logn) and O(n3/logn) processors on an EREW-PRAM. Third, using a modified version of the sequential algorithm as a subroutine, we devise a learning algorithm for one-variable patterns whose expected total learning time is O(l2logl) provided that sample strings are drawn from the target language according to a probability distribution with expected string length l. The probability distribution must be such that strings of equal length have equal probability, but can be arbitrary otherwise. Thus, we establish the first algorithm for learning one-variable pattern languages having an expected total learning time that provably differs from the update time by a constant factor only. Finally, we show how the algorithm for descriptive one-variable patterns can be used for learning one-variable patterns with a polynomial number of superset queries with respect to the one-variable patterns as query language.

/pdf/learning-one-variable-pattern-languages-very-efficiently-on-4n9tivzark.pdf

Learning One-Variable Pattern Languages Very Efficiently on Average, in Parallel, and by Asking Queries

Monotonic and non-monotonic reasoning is introduced into inductive inference In inductive inference, which is a mathematical theory of algorithmic learning from possibly incomplete information, monotonicity means to construct hypotheses somehow incrementally whereas the necessity of non-monotonic reasoning indicates that during hypothesis formation considerable belief revisions may be required Therefore, it is of a particular interest to find areas of inductive inference where monotonic construction of hypotheses is always possible It turned out that in the area of inductive inference of total recursive functions monotonicity can rarely be guaranteed These results are compared to the problem of inductively inferring text patterns from finite samples For this area, there is a universal weakly monotonic inductive inference algorithm The computability of a stronger algorithm which is developed depends on the decidability of the inclusion problem for pattern languages This problems remains open Unfortunately, the latter algorithm turns out to be inconsistent, ie it sometimes generates hypotheses not able to reflect the information they are build upon Consistency and monotonicity can hardly be achieved simultaneously It arises the question under which circumstances an inductive inference algorithm for learning text patterns can be both consistent and monotonic This problem class is characterized by closedness under intersection

Monotonic and Nonmonotonic Inductive Inference of Functions and Patterns

Pattern languages seem to suit case-based reasoning particularly well Therefore, the problem of inductively learning pattern languages is paraphrased in a case-based manner A careful investigation requires a formal semantics for case bases together with similarity measures in terms of formal languages Two basic semantics are introduced and investigated It turns out that representability problems are major obstacles for case-based learnability Restricting the attention to so-called proper patterns avoids these representability problems A couple of learnability results for proper pattern languages are derived both for case-based learning from only positive data and for case-based learning from positive and negative data Under the so-called competing semantics, we show that the learnability result for positive and negative data can be lifted to the general case of arbitrary patterns Learning under the standard semantics from positive data is closely related to monotonic language learning

Case-Based Representation and Learning of Pattern Languages

Patterns are words over an alphabet of constants and variables. New words are created from a pattern as sstrings of constants are substituted for the variables of the pattern. In this paper we investigate the inductive inference of patterns from positive data and quaries, from a complexity-theoretic point of view. Using results from combinatorics on words, we give simple but nontrivial sufficient conditions on the set of the initial examples that guarantee the identification of a unique pattern by making only polynomially many quaries. Counterexamples are also provided to show that the conditions are necessary.

Identification of pattern languages from examples and queries

Assume a finite alphabet of constant symbols and a disjoint infinite alphabet of variable symbols. A pattern p is a non-empty string of constant and variable symbols. The language L(p) is the set of all words over the alphabet of constant symbols generated from p by substituting some non-empty words for the variables in p. A sample S is a finite set of words over the same alphabet. A pattern p is descriptive of a sample S if and only if it is possible to generate all elements of S from p and, moreover, there is no other pattern q also able to generate S such that L(q) is a proper subset of L(p). The problem of finding a pattern being descriptive of a given sample is studied. It is known that the problem of finding a pattern of maximal length is NP-hard. Till now has be known a polynomial-time algorithm only for the special case of patterns containing only one variable symbol. The main result is a polynomial time algorithm constructing descriptive patterns of maximal length for the general case of patterns containing variable symbols from any finite set a priori fixed.

Polynomial Time Inference of General Pattern Languages

Software development is a serious bottleneck of current information processing practice. Consequently, mathematically based attempts to widen this bottleneck form a main stream of theoretical computer science. Among them abstract data type theory plays a leading role, as it is widely accepted that during a complex software design process particular decisions about implementation details should be postponed as long as possible. Instead, properties of and relations between software modules should be specified in an abstract manner emphasizing machine independent features. Algebraic semantics so has set the stage for languages and systems like CLEAR , LARCH , ASL , ACT ONE , and others (of. [04], [I0], [22], and [06], resp.).

Klaus P. Jantke

Papers

Polynomial Time Inference of General Pattern Languages

An algebraic framework for inductive program synthesis

An algebraic framework for inductive program synthesis