This paper compares text retrieval methods intended for office systems. The operational requirements of the office environment are discussed, and retrieval methods from database systems and from information retrieval systems are examined. We classify these methods and examine the most interesting representatives of each class. Attempts to speed up retrieval with special purpose hardware are also presented, and issues such as approximate string matching and compression are discussed. A qualitative comparison of the examined methods is presented. The signature file method is discussed in more detail.

Access methods for text

This paper provides an introduction to the use of n‐grams in textual information systems, where an n‐gram is a string of n, usually adjacent, characters extracted from a section of continuous text. Applications that can be implemented efficiently and effectively using sets of n‐grams include spelling error detection and correction, query expansion, information retrieval with serial, inverted and signature files, dictionary look‐up, text compression, and language identification.

Applications of n‐grams in textual information systems

A study has been made of the effect of controlled variations in indexing vocabulary size on retrieval performance using the Cranfield 200 and 1400 test collections. The vocabularies considered are sets of variable‐length character strings chosen from the fronts of document and query terms so as to occur with approximate equifrequency. Sets containing between 120 and 720 members were tested both using an application of the Cluster Hypothesis and in a series of linear associative retrieval experiments. The effectiveness of the smaller sets is low but the larger ones exhibit retrieval characteristics comparable to those of words.

Document retrieval experiments using indexing vocabularies of varying size. i. variety generation symbols assigned to the fronts of index terms

Research and development for textual data administration and for formatted database systems have traditionally been done separately. As a consequence, several features have been investigated and realised independently. For example, concurrent access to and update of information by many users is not provided in an information retrieval system (IRS). In fact, update of information during regular operating has long been considered unnecessary in an IRS. On the other hand, a database management system (DBMS) provides a 'multi-user environment', but in essence only for the administration of keyed or numerical (i.e. formatted) data and not for textual data. Textual data fields contain objects such as names, addresses, descriptive information, chemical formulas, etc. Such fields occur very often in database records, where they are administered in the same manner as numerical data. Common functions in an IRS such as retrieving a text via a number of character strings contained in it are supported neither by the available database query languages nor by the internal indexing techniques of a DBMS, and notions such as contents clustering or relevance, which are very important in an IRS, are disregarded in a DBMS. The following observations indicate that, nowadays, there is a trend towards an integrated database management and information retrieval system (DBMIRS): (1) Interactive usage of a DBMS grows. In addition to the regularly running transactions of well-defined application programs we have the spontaneously generated ad hoe transactions generated by interactive users in a problem solving model depending on the analysis of the results they have obtained so far (Blaser and Schauer, 1978; Erbe et al., 1980). This is exactly what we observe as common practice in IR. Query modification-often with a thesaurus-and feedback evaluation are classical fields in IR. In order to support interactive users of a DBMS, one tries to offer multi-attribute retrieval as well as partial range and nearest-neighbour searches for multi-dimensional numerical data. These functions, whose 218

Methods for the administration of textual data in database systems

The storage requirements for retrieval systems utilizing inverted files are calculated assuming different storage modes. Various methods for compression of these large files are analyzed. Binary vectors compressed by run-length coding as well as lists of document numbers were found to be suitable. The problem of minimal storage requirements for the inverted file is solved for different assumptions about index term distributions. A representation combining run-length coded binary vectors with list of document numbers was found to be the most economical. Parameter values for this minimum storage form are calculated and specified in tables as well as displayed graphically.

Compression of large inverted files with hyperbolic term distribution

The structure of a retrospective information retrieval system that uses equifrequent word or text fragments is described, and its advantages over word oriented systems are mentioned briefly. Word fragments are proposed as retrieval elements, and a discussion is given of the changes required in order to process a query. Some necessary modifications in the treatment of logical operators are described, and two conditions are postulated as necessary for the successful operation of the system. Aspects of query processing are illustrated by experimental results obtained from single and two-term queries applied to a portion of the MARC tapes.

H. S. Heaps

Papers

Query processing in a retrospective document retrieval system that uses word fragments as language elements