An overview of resource discovery services currently available on the Internet is presented. The authors concentrate on the following discovery tools: the Wide Area Information Servers (WAIS) project, Archie, Prospero, Gopher. The World-Wide Web (WWW), Netfind, the X.500 directory, Indie, the Knowbot Information Service (KIS), Alex, Semantic File Systems, and Nomenclator. The authors summarize the surveyed tools by presenting a taxonomy of their characteristics and design decisions. They also describe where to find and how to access several of the surveyed discovery services. They conclude with a discussion of future directions in the area of resource discovery and retrieval. >

Internet resource discovery services

A Scalable, Non-Hierarchical Resource Discovery Mechanism Based on Probabilistic Protocols ; CU-CS-474-90

The lack of a topology-based information distribution mechanism in wide area networks causes users to waste a large amount of bandwidth on repeat transfers of widely accessed files. As a first step towards designing such a mechanism, in this paper we present measurements of the Internet File Transfer Protocol, gathered over two weeks at the main gateway to the University of Colorado network. We found that nearly 40% of all transfers were duplicate transmissions, accounting for over 54% of the file transmission traffic. A small proportion of files were significantly larger and more frequently transferred, with 6.30% of the files accounting for 19.30% of the transfers. 8.21% of duplicate file transmissions were caused by user errors when transferring binary data, underscoring the need to insulate users from such details. We also found that file transfers occurred in only 44% of the FTP connections. Other connections were probably directory-only requests, underscoring the need for better resource discovery support in the Internet. We present measurements of a number of other statistics as well, including distributions of file sizes, file types, peak transfer times, and sources and destinations.

/pdf/a-measurement-study-of-internet-file-transfer-traffic-3z45q22ily.pdf

A Measurement Study of Internet File Transfer Traffic

Wide area networks offer access to an increasing number and variety of resources. Yet, it is difficult to locate resources of interest, because of the scale and decentralized nature of the environment. The authors are interested in supporting a global confederation of loosely cooperating systems and users that share far more resources than can be completely organized. Therefore, mechanisms are needed to support incremental organization of the resources, based on the efforts of many geographically decentralized individuals, and a range of different information sources of varying degrees of quality. The authors describe a prototype implementation of a set of mechanisms intended to explore this problem in the specific domain of public Internet archives, accessible via the anonymous file transfer protocol. This is an interesting test case, because it encompasses a very large scale, administratively decentralized collection of resources, with considerably practical value. >

https://codeontheroad.com/papers/RD.For.Anon.FTP.pdf

Supporting resource discovery among public Internet archives using a spectrum of information quality

In this report we discuss plans to carry out a longitudinal measurement study of changes in service-level reachability in the global TCP/IP Internet. We overview our experimental design, considerations of network and remote site load, mechanisms used to control the measurement collection process, and network appropriate use and privacy issues, including our efforts to inform sites measured by this study. A list of references and information on how to contact the Principal Investigator are included.

Measurement Study of Changes in Service-Level Reachability in the Global TCP/IP Internet: Goals, Experimental Design, Implementation, and Policy Considerations

Michael F Schwartz

Papers

A Measurement Study of Organizational Properties in the Global Electronic Mail Community ; CU-CS-482-90