Resource Cataloging and Distribution System

Nearly every Web user is familiar with the following problems:

• The file server mentioned in a URL is busy, down, or disconnected from the net.
• A URL becomes ``stale'' or out-of-date.
• Search results often do not reflect the current contents or location of a resource.
• Search services may return duplicate ``hits'' for different locations of the same resource.
• There is no effective way to screen resources based on content.
• When replicas (mirrors) of a resource exist, there is no good way to find out about them.
• There is no way to verify the integrity or authenticity of a replica of a resource.

• Cataloging the characteristics of Internet-accessible resources,
• Efficiently replicating such resources for better accessibility,
• Keeping track of the locations from which replicas may be accessed,
• Allowing web clients to identify nearby locations of a resource,
• Allowing web clients to select and filter resources according to their listed characterists,
• Providing authentication and integrity assurance of characteristics and replicas, and
• Informing interested parties (e.g. mirror sites, search services) of new or updated resources.

• If a file server is busy or unreachable from one location, the client can automatically attempt to retreive the file from a different location. Also, because RCDS allows clients to find a nearby location (using the SONAR proximity estimation service), it minimizes unnecessary use of long-distance bandwidth.
• Since RCDS keeps track of files when they move, the original URL for that file can be used to access the file even if it's not in its original location (as long as no other file uses that same URL)
• Search services can get notification from RCDS of changes to files, allowing them to re-index the file in a timely fashion.
• RCDS maintains an internal document identifier (called a LIFN) for each separate version of a resource. RCDS-aware indexing services receive only one notice (with that LIFN) per new or changed resource; so they need not separately index each replica of a file.
• Clients can query RCDS for the characteristcs of a resource, allowing them to select between alternative representations (say to use Adobe's PDF rather than plain text) even if those representations are in different locations, or to screen out resources which are marked (according to client-specified criteria) as having undesirable content.
• RCDS allows the characteristics of resources, and the resources themselves, to be cryptographically signed. It can also serve as a medium for the distribution of public keys and certificates.

For more information on RCDS:

• Read the overview, which describes the goals and architecture in more detail.
• Here's a description of the resource catalog subsystem, which also describes the protocol.
• View a slide presentation.
• Here's a paper (PostScript format).
• Here's another paper (PostScript) we did for some IEEE publication.
• Here's a technical report in HTML

To Install the RCDS Source Code:

• Download the resource catalog source code.
• View some hints you need to know to get RCDS running.
• View man pages for the RCDS server and client library.

Related Projects:

• SONAR is a system for determining relative proximity of network resources.  RCDS-aware applications can use RCDS to determine locations of network resources, then use SONAR to pick a nearby location at which to access the resource.
• SNIPE (Scalable Network Information Processing Environment) is a highly scalable distributed programming environment which uses the Resource Catalog portion of RCDS as a substrate.
• Slide show comparing Resource Catalog vs. LDAP as a SNIPE substrate
• Netbuild is a system for automatically building complex software programs from components.  It uses RCDS to
• RIB (Repository In a Box) is a toolkit for creating software repositories that can interoperate with other software repositories via the Internet.