Information filtering systems are designed for unstructured or semistructured data, as opposed to database applications, which use very structured data. The systems also deal primarily with textual information, but they may also entail images, voice, video or other data types that are part of multimedia information systems. Information filtering systems also involve a large amount of data and streams of incoming data, whether broadcast from a remote source or sent directly by other sources. Filtering is based on descriptions of individual or group information preferences, or profiles, that typically represent long-term interests. Filtering also implies removal of data from an incoming stream rather than finding data in the stream; users see only the data that is extracted. Models of information retrieval and filtering, and lessons for filtering from retrieval research are presented.

/pdf/information-filtering-and-information-retrieval-two-sides-of-zycsi977zk.pdf

Information filtering and information retrieval: two sides of the same coin?

The use of inference networks to support document retrieval is introduced. A network-basead retrieval model is described and compared to conventional probabilistic and Boolean models.

/pdf/inference-networks-for-document-retrieval-1d9pbqy4qn.pdf

Inference Networks for Document Retrieval

The paper highlights the ever increasing complexity in the evaluation of IR systems which has arisen over the last decade. Relevance, cognition, user behaviour, interaction, and a changing view of the boundaries of the system are considered to be contributory factors. Issues such as laboratory versus operational systems, black-box versus diagnostic experiments, and qualitative and quantitative methods are discussed and supported by examples drawn from three groups of evaluative experiments: weighted searching on a front-end system, information-seeking behaviour and the use of OPACs, and the OKAPI experimental retrieval system.

On the evaluation of IR systems

This paper explores the role of paradigms in information retrieval research. The nature of a paradigm is outlined and the fundamental sense of a paradigm as an exemplar is identified. The applicability of the paradigm concept to a multi‐disciplinary field such as information science is discussed and it is concluded that paradigms can be a legitimate feature of information science though they may not be connected with the development of normal science. The features of two paradigms operating in information retrieval research, (1) the physical paradigm and (2) the cognitive paradigm are outlined, and their origins, nature and role examined. It is argued that although most work in information retrieval research takes place within the physical and cognitive paradigms, neither provides the basis for a powerful paradigm directed science. An explanation for the failure to develop a powerful body of theory articulated within a well developed paradigmatic framework is offered with reference to the inherent categorial duality of the field.

The physical and cognitive paradigms in information retrieval research

Machine-learning techniques play the important roles for information filtering. The main objective of machine-learning is to obtain users' profiles. To decrease the burden of on-line learning, it is important to seek suitable structures to represent user information needs. This paper proposes a model for information filtering on the Web. The user information need is described into two levels in this model: profiles on category level, and Boolean queries on document level. To efficiently estimate the relevance between the user information need and documents, the user information need is treated as a rough set on the space of documents. The rough set decision theory is used to classify the new documents according to the user information need. In return for this, the new documents are divided into three parts: positive region, boundary region, and negative region. An experimental system JobAgent is also presented to verify this model, and it shows that the rough set based model can provide an efficient approach to solve the information overload problem.

An information filtering model on the Web and its application in JobAgent

Information retrieval systems have been the subject of experimental testing for some twenty years now. Like any field in this position, a fair amount of know-how has accumulated about the proper conduct of such investigations. The object of this book is to distil this know-how; the object of this chapter is to set the scene. Thus I will be introducing the basic ideas, sketching in some of the main problem areas, and generally preparing the reader for the more specific or concrete chapters that follow. Van Rijsbergen takes up the question of the evaluation of retrieval effectiveness in Chapter 3; in Chapter 4, Belkin considers information retrieval in a wider context; and in Chapter 5, Tague gets down to the detail of conducting experiments.

Stephen Robertson

Papers

The methodology of information retrieval experiment