We introduce a stochastic graph-based method for computing relative importance of textual units for Natural Language Processing. We test the technique on the problem of Text Summarization (TS). Extractive TS relies on the concept of sentence salience to identify the most important sentences in a document or set of documents. Salience is typically defined in terms of the presence of particular important words or in terms of similarity to a centroid pseudo-sentence. We consider a new approach, LexRank, for computing sentence importance based on the concept of eigenvector centrality in a graph representation of sentences. In this model, a connectivity matrix based on intra-sentence cosine similarity is used as the adjacency matrix of the graph representation of sentences. Our system, based on LexRank ranked in first place in more than one task in the recent DUC 2004 evaluation. In this paper we present a detailed analysis of our approach and apply it to a larger data set including data from earlier DUC evaluations. We discuss several methods to compute centrality using the similarity graph. The results show that degree-based methods (including LexRank) outperform both centroid-based methods and other systems participating in DUC in most of the cases. Furthermore, the LexRank with threshold method outperforms the other degree-based techniques including continuous LexRank. We also show that our approach is quite insensitive to the noise in the data that may result from an imperfect topical clustering of documents.

/pdf/lexrank-graph-based-lexical-centrality-as-salience-in-text-10aso8e2ok.pdf

LexRank: graph-based lexical centrality as salience in text summarization

1. Introduction to text mining 2. Core text mining operations 3. Text mining preprocessing techniques 4. Categorization 5. Clustering 6. Information extraction 7. Probabilistic models for Information extraction 8. Preprocessing applications using probabilistic and hybrid approaches 9. Presentation-layer considerations for browsing and query refinement 10. Visualization approaches 11. Link analysis 12. Text mining applications Appendix Bibliography.

/pdf/the-text-mining-handbook-advanced-approaches-in-analyzing-ge55wa29fm.pdf

The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data

(1) Whereas the objectives of the Community, as laid down in the Treaty, as amended by the Treaty on European Union, include creating an ever closer union among the peoples of Europe, fostering closer relations between the States belonging to the Community, ensuring economic and social progress by common action to eliminate the barriers which divide Europe, encouraging the constant improvement of the living conditions of its peoples, preserving and strengthening peace and liberty and promoting democracy on the basis of the fundamental rights recognized in the constitution and laws of the Member States and in the European Convention for the Protection of Human Rights and Fundamental Freedoms;

Some Preliminary Comments on the DIRECTIVE 95/46/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data.

Neural networks are used in business organizations for providing intelligent decision support. The ABI/Inform bibliographic database has over 1 000 publications on business-related research in neur...

A bibliometric analysis

VAN DIJK, TEUN A., News as Discourse Hillsdale, N.J.: Lawrence Erlbaum, 1988. $29.95 cloth. This book attempts the development of a "new, interdisciplinary theory of news in the press" (p. vii). It represents a very ambitious and somewhat speculative effort to weave together a broad range of existing news research approaches into a coherent, heuristic framework. Van Dijk succeeds in providing a useful summary of the literature in news research. Especially valuable is his discussion of recent European research. However, the overall framework is still at an early stage of development. Its utility remains to be demonstrated by future research. For some time now, communication researchers have talked of passing paradigms and ferment in the field. With the decline of past paradigms, our discipline has been left with a hodgepodge of small-scale theories. This is particularly true in the area of news research where various narrative, discourse and information processing theories abound. Van Dijk's book may signal a new era, an era in which efforts will be made to integrate existing conceptual fragments into broader frameworks. His work may be seen as providing a model for others who seek to make sense of our current proliferation of theories. Van Dijk's approach is centered in the tradition of discourse analysis, which evolved out of an integration of literary analysis and linguistics. However, he has aggressively modified earlier forms of discourse analysis in an effort to incorporate insights into the structure and interpretation of discourse derived from cognitive psychology. He is not content to simply apply discourse analysis to the evaluation of news stories. He recognizes the utility of constructing an approach which also considers the production of news by media practitioners and the interpretation of news by audience members. It is these broader concerns which set van Dijk's approach apart from previous analyses of news content. A central concept in van Dijk's theory is the notion of story schemas, which are defined as implicit structures that underlie typical stories. These schemas permit the easy production of news and also facilitate its interpretation by news consumers. The schema concept is at once powerful and ambiguous. There is growing research evidence that demonstrates the utility of positing the existence of cognitive structures (schemas) in people's minds which are activated by content cues and guide interpretion of all forms of communication. The schema concept helps to explain why complex and seemingly ambiguous messages often are easily interpreted by audience members. It also can explain why the same message can be interpreted in highly discrepent ways. If messages contain conflicting cues that lead people to activate different schemas, or if people don't share a homogeneous set of schemas, then it is likely that many contrasting interpretations of story content will be developed. But despite growing consensus concerning the utility of schema as a concept, researchers remain quite divided over both its definition and the type of research that will lead to the most useful findings. …

News as Discourse

(4) Whereas the extension of the scope of Directive 98/34/EC should not prevent Member States from taking account of the different social, societal and cultural implications inherent in the advent of the Information Society; whereas, in particular, the use of the procedural rules laid down in that Directive for Information Society services should not affect cultural policy measures, particularly in the audiovisual field, which Member States might adopt in accordance with Community law, taking account of their linguistic diversity, their specific national and regional characteristics and their cultural heritage; whereas the development of the Information Society should ensure, in any event, proper access of European citizens to the European cultural heritage supplied in a digital environment;

Directive 98/48/EC of the European Parliament and of the Council

ing of Legal Cases: The SALOMON Experience Marie-Francine Moens, Caroline Uyttendaele, Jos Dumortier Interdisciplinary Centre for Law and IT (ICRI) K.U. Leuven Tiensestraat 41 B-3000 Leuven Belgium {marie-france.moens, caroline.uyttendaele, jos.dumortier}@law.kuleuven.ac.be

Abstracting of legal cases: the SALOMON experience

With the worldwide growth of open telecommunication networks and in particular the Internet, the privacy and security concerns of people using these networks have increased. On the one hand, users are concerned about their privacy, and desire to anonymously access the network. On the other hand, some organizations are concerned about how this anonymous access might be abused. This paper intends to bridge these conflicting interests, and proposes a solution for revocable anonymous access to the Internet. Moreover, the paper presents some legal background and motivation for such a solution. However, the paper also indicates some difficulties and disadvantages of the proposed solution, and suggests the need for further debate on the issue of online anonymity.

/pdf/revocable-anonymous-access-to-the-internet-29aimfkgsr.pdf

Revocable Anonymous Access to the Internet

Directive 1999/93/EC on a community framework for electronic signatures

The SALOMON project automatically summarizes Belgian criminal cases in order to improve access to the large number of existing and future court decisions. SALOMON extracts text units from the case text to form a case summary. Such a case summary facilitates the rapid determination of the relevance of the case or may be employed in text search. An important part of the research concerns the development of techniques for automatic recognition of representative text paragraphs (or sentences) in texts of unrestricted domains. These techniques are employed to eliminate redundant material in the case texts, and to identify informative text paragraphs which are relevant to include in the case An evaluation of a test set of 700 criminal cases demonstrates that the algorithms have an application potential for automatic indexing, abstracting, and text linking.

Jos Dumortier

Papers

Directive 98/48/EC of the European Parliament and of the Council

Abstracting of legal cases: the SALOMON experience

Revocable Anonymous Access to the Internet

Directive 1999/93/EC on a community framework for electronic signatures

Abstracting of legal cases: the potential of clustering based on the selection of representative objects