Miller (1956) summarized evidence that people can remember about seven chunks in short-term memory (STM) tasks. How- ever, that number was meant more as a rough estimate and a rhetorical device than as a real capacity limit. Others have since suggested that there is a more precise capacity limit, but that it is only three to five chunks. The present target article brings together a wide vari- ety of data on capacity limits suggesting that the smaller capacity limit is real. Capacity limits will be useful in analyses of information processing only if the boundary conditions for observing them can be carefully described. Four basic conditions in which chunks can be identified and capacity limits can accordingly be observed are: (1) when information overload limits chunks to individual stimulus items, (2) when other steps are taken specifically to block the recoding of stimulus items into larger chunks, (3) in performance discontinuities caused by the capacity limit, and (4) in various indirect effects of the capacity limit. Under these conditions, rehearsal and long-term memory cannot be used to combine stimulus items into chunks of an unknown size; nor can storage mechanisms that are not capacity- limited, such as sensory memory, allow the capacity-limited storage mechanism to be refilled during recall. A single, central capacity limit averaging about four chunks is implicated along with other, noncapacity-limited sources. The pure STM capacity limit expressed in chunks is distinguished from compound STM limits obtained when the number of separately held chunks is unclear. Reasons why pure capacity estimates fall within a narrow range are discussed and a capacity limit for the focus of attention is proposed.

/pdf/the-magical-number-4-in-short-term-memory-a-reconsideration-3704b9phuz.pdf

The magical number 4 in short-term memory: a reconsideration of mental storage capacity.

This important text and reference for researchers and students in machine learning, game theory, statistics and information theory offers a comprehensive treatment of the problem of predicting individual sequences. Unlike standard statistical approaches to forecasting, prediction of individual sequences does not impose any probabilistic assumption on the data-generating mechanism. Yet, prediction algorithms can be constructed that work well for all possible sequences, in the sense that their performance is always nearly as good as the best forecasting strategy in a given reference class. The central theme is the model of prediction using expert advice, a general framework within which many related problems can be cast and discussed. Repeated game playing, adaptive data compression, sequential investment in the stock market, sequential pattern analysis, and several other problems are viewed as instances of the experts' framework and analyzed from a common nonstochastic standpoint that often reveals new and intriguing connections.

/pdf/prediction-learning-and-games-31z1lqjpec.pdf

Prediction, learning, and games

We argue that an understanding of the faculty of language requires substantial interdisciplinary cooperation. We suggest how current developments in linguistics can be profitably wedded to work in evolutionary biology, anthropology, psychology, and neuroscience. We submit that a distinction should be made between the faculty of language in the broad sense (FLB)and in the narrow sense (FLN) . FLB includes a sensory-motor system, a conceptual-intentional system, and the computational mechanisms for recursion, providing the capacity to generate an infinite range of expressions from a finite set of elements. We hypothesize that FLN only includes recursion and is the only uniquely human component of the faculty of language. We further argue that FLN may have evolved for reasons other than language, hence comparative studies might look for evidence of such computations outside of the domain of communication (for example, number, navigation, and social relations).

/pdf/the-faculty-of-language-what-is-it-who-has-it-and-how-did-it-2ls6go98wi.pdf

The faculty of language: what is it, who has it, and how did it evolve?

This graduate-level textbook introduces fundamental concepts and methods in machine learning. It describes several important modern algorithms, provides the theoretical underpinnings of these algorithms, and illustrates key aspects for their application. The authors aim to present novel theoretical tools and concepts while giving concise proofs even for relatively advanced topics. Foundations of Machine Learning fills the need for a general textbook that also offers theoretical details and an emphasis on proofs. Certain topics that are often treated with insufficient attention are discussed in more detail here; for example, entire chapters are devoted to regression, multi-class classification, and ranking. The first three chapters lay the theoretical foundation for what follows, but each remaining chapter is mostly self-contained. The appendix offers a concise probability review, a short introduction to convex optimization, tools for concentration bounds, and several basic properties of matrices and norms used in the book. The book is intended for graduate students and researchers in machine learning, statistics, and related areas; it can be used either as a textbook or as a reference text for a research seminar.

Foundations of Machine Learning

The problem of identifying an unknown regular set from examples of its members and nonmembers is addressed. It is assumed that the regular set is presented by a minimaMy adequate Teacher, which can answer membership queries about the set and can also test a conjecture and indicate whether it is equal to the unknown set and provide a counterexample if not. (A counterexample is a string in the symmetric difference of the correct set and the conjectured set.) A learning algorithm L* is described that correctly learns any regular set from any minimally adequate Teacher in time polynomial in the number of states of the minimum dfa for the set and the maximum length of any counterexample provided by the Teacher. It is shown that in a stochastic setting the ability of the Teacher to test conjectures may be replaced by a random sampling oracle, EX( ). A polynomial-time learning algorithm is shown for a particular problem of context-free language identification.

/pdf/learning-regular-sets-from-queries-and-counterexamples-1gn1248tzv.pdf

Learning regular sets from queries and counterexamples

Language learnability has been investigated. This refers to the following situation: A class of possible languages is specified, together with a method of presenting information to the learner about an unknown language, which is to be chosen from the class. The question is now asked, “Is the information sufficient to determine which of the possible languages is the unknown language?” Many definitions of learnability are possible, but only the following is considered here: Time is quantized and has a finite starting time. At each time the learner receives a unit of information and is to make a guess as to the identity of the unknown language on the basis of the information received so far. This process continues forever. The class of languages will be considered learnable with respect to the specified method of information presentation if there is an algorithm that the learner can use to make his guesses, the algorithm having the following property: Given any language of the class, there is some finite time after which the guesses will all be the same and they will be correct. In this preliminary investigation, a language is taken to be a set of strings on some finite alphabet. The alphabet is the same for all languages of the class. Several variations of each of the following two basic methods of information presentation are investigated: A text for a language generates the strings of the language in any order such that every string of the language occurs at least once. An informant for a language tells whether a string is in the language, and chooses the strings in some order such that every string occurs at least once. It was found that the class of context-sensitive languages is learnable from an informant, but that not even the class of regular languages is learnable from a text.

Language identification in the limit

The question of whether there is an automaton with n states which agrees with a finite set D of data is shown to be NP-complete, although identification-in-the-limit of finite automata is possible in polynomial time as a function of the size of D. Necessary and sufficient conditions are given for D to be realizable by an automaton whose states are reachable from the initial state by a given set T of input strings. Although this question is also NP-complete, these conditions suggest heuristic approaches. Even if a solution to this problem were available, it is shown that finding a minimal set T does not necessarily give the smallest possible T.

Complexity of automaton identification from given data

The subject of this paper is the computational complexity of the deadlock prediction problem for resource allocation. This problem is the question “Is deadlock avoidable?” i.e. “Is there a feasible sequence in which to allocate all the resource requests?” given the current status of a resource allocation system. This status is defined by (1) the resource vector held by the banker, i.e. the quantity of resources presently available for allocation, and (2) the resource requests of the processes: Each process is required to make a termination request of the form “Give me resource vector y and I will eventually terminate and return resource vector z.” Also, each process can make any number of partial requests of the form “If you can’t give me y, then give me a smaller resource vector $y'$ and I will be able to reach a point at which I can halt and temporarily return to $z'$, although I will still need need $y - y' + z'$ to terminate.”If (1) the resources are reusable and (2) partial requests are not allowed, ...

Deadlock Prediction: Easy and Difficult Cases

System identification via state characterization

The object of this paper is to clarify Schonemann's unfolding algorithm and, in particular, to make it clear that the equations numbered (3.2) in Schonemann's [1970] article, which define Schonemann's solutions, are not a complete set of restraints for the purpose of defining metric unfoldings. Namely, Schonemann has transformed the original equations which define an unfolding to a set of linear and non-linear equations of which he uses only the linear equations to define his solutions. Given infallible data (solution(s) exist) Schonemann's solutions will include the correct solutions. If enough data are available so that there are enough linear equations to uniquely determine a single solution, then Schonemann's solution will coincide with the correct solution. LetP andQ denote the number of elements in the two sets of points, the interset distances of which are specified by the data in the unfolding problem. Letm denote the dimensionality of the Euclidean space into which these points are to be imbedded. If only the linear equations, numbered (18) herein, are to be used, then Schonemann gives the following data requirement for the solution to be uniquely determined: Max {P − 1,Q − 1} ≥m(m + 3)/2. If the full set of linear and nonlinear equations (18–20) are used, then the amount of data required for a solution to be locally unique is relaxed toP +Q − 1 ≥m(m + 3)/2. Both of these results assume that the equations are independent, which has not been proved.

E. Mark Gold

Papers

Language identification in the limit

Complexity of automaton identification from given data

Deadlock Prediction: Easy and Difficult Cases

System identification via state characterization

Metric unfolding: Data requirement for unique solution & clarification of Schönemann's algorithm