A major challenge in indexing unstructured hypertext databases is to automatically extract meta-data that enables structured search using topic taxonomies, circumvents keyword ambiguity, and improves the quality of search and profile-based routing and filtering. Therefore, an accurate classifier is an essential component of a hypertext database. Hyperlinks pose new problems not addressed in the extensive text classification literature. Links clearly contain high-quality semantic clues that are lost upon a purely term-based classifier, but exploiting link information is non-trivial because it is noisy. Naive use of terms in the link neighborhood of a document can even degrade accuracy. Our contribution is to propose robust statistical models and a relaxation labeling technique for better classification by exploiting link information in a small neighborhood around documents. Our technique also adapts gracefully to the fraction of neighboring documents having known topics. We experimented with pre-classified samples from Yahoo!1 and the US Patent Database2. In previous work, we developed a text classifier that misclassified only 13% of the documents in the well-known Reuters benchmark; this was comparable to the best results ever obtained. This classifier misclassified 36% of the patents, indicating that classifying hypertext can be more difficult than classifying text. Naively using terms in neighboring documents increased error to 38%; our hypertext classifier reduced it to 21%. Results with the Yahoo! sample were more dramatic: the text classifier showed 68% error, whereas our hypertext classifier reduced this to only 21%.

/pdf/enhanced-hypertext-categorization-using-hyperlinks-ge4vw1hbzz.pdf

Enhanced hypertext categorization using hyperlinks

Systems and methods for managing data, such as metadata. In one exemplary method, metadata from files created by several different software applications are captured, and the captured metadata is searched. The type of information in metadata for one type of file differs from the type of information in metadata for another type of file. Other methods are described and data processing systems and machine readable media are also described.

Methods And Systems For Managing Data

A system, method, and computer program product for processing data are described herein. The system maintains first databases of patents, and second databases of non-patent information of interest to a corporate entity. The system also maintains one or more groups. Each of the groups comprises any number of the patents from the first databases. The system, upon receiving appropriate operator commands, automatically processes the patents in one of the groups in conjunction with non-patent information from the second databases. Accordingly, the system performs patent-centric and group-oriented processing of data. A group can also include any number of non-patent documents. The groups may be product based, person based, corporate entity based, or user-defined. Other types of groups are also covered, such as temporary groups. The processing automatically performed by the system relates to (but is not limited to) patent mapping, document mapping, patent citation (both forward and backward), patent aging, patent bracketing/clustering (both forward and backward), inventor patent count, inventor employment information, patent claim tree analysis, and finance. Other functions and capabilities are also covered, including the ability to utilize hyperbolic trees to visualize data generated by the system, method, and computer program product.

Using hyperbolic trees to visualize data generated by patent-centric and group-oriented data processing

A computer-based electronic document and/or paper-based document management application program. The program provides an efficient way to automatically import, index, categorize, store, search, retrieve, manipulate and archive electronic documents. The program is also capable of managing documents regardless of document type or document format.

Computer-based document management system

The secure electronic distribution of research documents over the world wide web to investors who are authorized to receive said research documents. A repository server receives research documents from contributors. Also received are corresponding document profiles with information relating to each research document including authorization information specifying who is permitted to access each research document. The repository server includes a first database for structured query searches and a second database for full text searches. A web server is coupled to the repository server and coupled to the world wide web. The web server receives requests from investors for research documents that satisfy a query. The web server determines whether the first database or the second database should be searched based upon the type of query. The repository server transmits to the web server a list of research documents that satisfy the query and which the investor is authorized to access according to the authorization information. The web server formats the list of documents according to a template form. Optionally, queries can be optimized. The system has a control mechanism to prevent concurrent unauthorized access by two people using the same ID/password combination.

Information delivery system and method

A document storage and retrieval system stores a document body in the form of an image, storing text information in the form of a character code string for retrieval, and executing a retrieval with reference to the text information, followed by displaying a document image relating thereto on a retrieval terminal according to the retrieval result. Such a form of the system is available for retrieving the full contents of a document and also for displaying the document body printed in a format easy to read straight in the form of an image.

Document storage and retrieval system

A document information classification method and apparatus for classifying a document group and arranging a classified result hierarchically on the basis of key words given to the document group and words appearing in documents without dependence on a prescribed classification system. The document group of a document data base (107) and a key word group given to each document of a key word data base (108) are managed by a data management unit (101). A document classification unit (103) classifies documents into folders on the basis of individual key words and stores them. The folders having similar document groups are integrated. Whether the integration is effective or not is judged upon integration. Whether the inside of the integrated folder and the inside of unintegrated folders can be classified in detail or not is judged and a hierarchical classification system is prepared. A classified result is produced in CRT (109) by a classified result output unit (104) to provide environment in which a user can read out the classified result.

Method and apparatus for classifying document information

A document difference extraction method and apparatus which is used for extracting the difference between structured documents properly meeting the sense of a document editor taking the logical meaning and structure of the structured documents into consideration. Structured documents are edited and stored in a memory unit by a document editing program. With reference to a comparison criterion set for the logical structure of each structured document before and after edition, the logical structure of the structural documents before and after edition read from the memory unit is analyzed by a structured document parsing program, and the difference between the structured documents is extracted by a structured document difference extraction program in such a manner as to satisfy the comparison criterion in accordance with the result of parsing. The comparison criterion assumes the form of a table containing a plurality of tags representing logical structures and types of tags for the comparison criterion. The tag types for comparison criterion include tags having contents which are compared only when the particular tags are coincident with each other, tags having contents which are ignored at the time of comparison, a set of tags having the same logical meaning, and a set of tags having contents which are not compared with each other.

Method and apparatus for structured document difference string extraction

An algorithm to normalize the skew of document images is proposed. The skew angle is detected in two stages. In the first stage, connected regions in an image are extracted and some feature parameters are extracted for each region. In the second stage, the Hough transform is calculated for the parameters, and the angle which gives the minimum of the transform is estimated as the skew angle. In experiments using CCITT standard documents, a detection accuracy of less than 0.1 degrees is obtained for printed documents. When graphical elements are included in the documents in addition to printed characters, the accuracy deteriorates to 0.2 degrees . >

An algorithm for the skew normalization of document image

A document storage and retrieval system is provided with means for storing a document body in the form of image, means for storing text information in the form of a character code string for retrieval, means for executing a retrieval with reference to the text information, and means for displaying a document image relating thereto on a retrieval terminal according to the retrieval result. Such a form of the system is available for retrieving the full contents of a document and also for displaying the document body printed in a format easy to read straight in the form of image. Accordingly, users are capable of retrieving documents with arbitrary words and also capable of reading even such a document as is complicated to include mathematical expressions and charts through a terminal in the form of image, the same as on paper. Further, the invention provides a system wherein the text information for retrieval is extracted automatically from the document image through character recognition. Since a precision of the character recognition has not been satisfactory hitherto, a visual retrieval and correction have been carried out without fail by operators. However, there is no necessity for the operators to attend therefor according to the invention. Thus, the text information for retrieval can be generated at the cost of practical time and money even in case of volumes of documents.

Junichi Higashino

Papers

Document storage and retrieval system

Method and apparatus for classifying document information

Method and apparatus for structured document difference string extraction

An algorithm for the skew normalization of document image

Document storage and retrieval system for storing and retrieving document image and full text data