CL-HTTP Proxy Service (Alpha Test)

Prism Image CL-HTTP now provides HTTP proxy service. Proxy functionality is integrated into the existing system architecture, so CL-HTTP may act as a proxy without interrupting normal server operations. When CL-HTTP receives a request for a remote resource, it may utilize the client module and relay the information to the client. Proxy access is controlled via a top-level switch as well as a list of allowed subnets. The current release also includes an experimental caching proxy mode. Caching support is optional and is controlled by a separate server preference.

Contents

  1. Proxy Relay
  2. Caching
  3. Continuing Work
  4. Extensions

  1. Proxy Relay

    2. Proxy relay (no caching) service can be provided for all standard HTTP/1.0 and HTTP/1.1 methods. The HTTP/1.0 PUT extension is not supported. Access to basic proxy services is controlled via http:*proxy-service* and http:*proxy-subnets*. The proxy uses the standard CL-HTTP client to retrieve resources, and the default HTTP version used for making such requests is controlled by http:*client-http-version*. The relay methods are intended to be HTTP/1.1 compliant, and discrepancies should be reported (or fixed!) when encountered.

  2. Caching

    3. Construction Proxy access to the experimental caching routines is controlled via http:*proxy-caching-p*. The current version of the cache uses in-memory data structures to store meta-information about cached representations of resources, and the file-system to store their actual entities. The file HTTP:proxy;database.lisp illustrates how to define new database implementations. While the caching functionality is not yet HTTP/1.1 compliant, it interoperates reasonably with HTTP/1.0 clients.

    If the proxy is currently loaded, you can view a list of the cached resources on the server. Note that the cache currently does not automatically expunge itself to make more space. However, methods such as http:cache-size and http:expunge-cache allow such a strategy to be implemented. You may wish to run a seperate process that attempts to keep the cache size within a certain window.

  3. Continuing Work

    4. First Aid As with the rest of the server, we need the help of the CL-HTTP community to push this functionality forward. Most changes to the relay methods should be in the form of bug fixes or efficiency imporovements. The caching methods in HTTP:proxy;proxy-cache.lisp need to be further abstracted and moved closer to HTTP/1.1 compliance.

    As of August 1997, these are some of the issues:

    1. Parsing of search URLs causes conflicts with the proxy, we are developing a new strategy for parsing (or parsing less of) identifiers for remote resources.
    2. Need to avoid parsing most headers on proxying, for efficiency and to avoid a whole class of header-parsing conflicts. This will involve changes to the header and client code.
    3. Support for more conditional methods.
    4. Support for HTTP/1.1 cache control.
    5. Add cache sensitivity to HTTP/1.1 byte-range requests.
    6. Efficient/intelligent methods for computing cache size and expunge order, as well as a standard expunge policy.
    7. Ensure that error handling is correct for caching methods.
    8. Consider resource contention issues more closely (actually implement bimodal locks on the Mac.)
    9. Make sure that chunk decoding is not causing premature exit of some code.

  4. Extensions

    5. This proxy is meant more as a platform for intelligent caching and collaboration research than a competitor for high-load corporate proxies. Several immediate oppurtunities present themselves.

    Cache Prefetch
    The proxy cache can be used along with the W4 Web Walker to implement prefetch operations.
    Collaborative Browsing
    The proxy may be used to infer group browsing patterns and aid in resource discovery.
    Annotation
    The cache may act as a platform for group annotation of resources.
    Intelligent Caching
    CL-HTTP should provide an excellent environment for research in dynamic caching algorithms.

Christopher Vincent (cvince@ai.mit.edu)