Squid ETag header support

This work aims at supporting strong cache validation and Vary matches of objects using the ETag header.

Contents

The ETag header

Description of what the ETag header is

Implementation

How to implement ETag in Squid

Store format

Store format used to support ETag indexing of varying objects

Cache hit processing

How to find and process cache hits

Tests & verifications

Things to test and verify

To do

1. Fix ETag processing of large indexes
2. Fix processing of ETag on aborted requests (http->entry == NULL)
3. Port to new store API
4. Cache validations of tagged objects (ETag) not carrying a "Last-Modified" timestamp.
5. Optimize expired entity matches after If-None-Match to count the If-None-Match as a cache validation.
6. Optimize ETag index updates to not touch the disk store if there is no change.

Progress

2006-06-30 Many bugfixes in 2.6

Many ETag related bugs have been fixed in 2.6 after the merge.

2006-06-07 broken_vary_encoding workaround in 2.6

Workaround for many broken Content-Encoding transforming servers (Apache mod_deflate, mod_gzip, quite likely others) not updating ETag proper.

2006-05-25 Merged into 2.6

ETag support has been merged into Squid-2.6.

2002-08-16 index fixes

Various Vary/ETag index fixes

2002-08-15 memory corruption bugfix

In some conditions the ETag swapin buffer wasn't freed correctly, possibly causing memory corruption.

2002-04-02 memorybuf bugfixes

Some bugfixes in the inherited memorybuf branch

2002-02-23 Bugfix: Large ETag indexes

Fix handling of large ETag indexes

2002-02-13 Bugfix: ETag index processing

Various bugfixes to ETag index processing

2001-06-12 Bugfix: If-Range using LMT

If-Range using LMT timestamps could falsely match is the indicated LMT was newer than a cached copy.

2001-06-01 Bugfix: If-None-Match on cache validations

If-None-Match now also supported on cache validations. Returns the servers 304 reply if the ETag (of the reply) matches.

2001-06-01 More bugfixes for varying objects without ETag

Cache hits on varying objects without ETag also crashed Squid

2001-06-01 Small memory leak on Vary misses

Fixed a memory leak on Vary misses where the ETag index was updated

2001-05-01 Debug log levels

Debug log levels have adjusted to not log anything

2001-06-01 Bugfix for varying objects without a ETag

Crashed if a varying object was received without a ETag

2001-05-31 Squid-2.4 port

The etag changes have been ported to Squid-2.4 patched for Vary support

2001-05-31 If-None-Match and If-Match client side

If-Match and If-None-Match is now supported in client side requests. If the cached etag does not match If-Match then a the request is processed as a cache miss. If it does match If-None-Match then it is processed as a IMS HIT (304 reply).

2001-05-31 Many small bugs hunted down

No longer fails basic tests or leaks huge amounts of memory. There is one known memory leak on cache validations, but it looks like this might be a generic problem not actually caused by the ETag changes..

2001-05-30 If-None-Match etag queries

On cache misses for varying object If-None-Match is now used to query the origin server which (if any) of the caced entities matches the request.

2001-05-30 If-None-Match cache validation

If-None-Match is now used on cache validations, using the entity tag of the exact object being validated. This ensures that even sub-second updates are detected properly, and cache validation is now less time sensitive.

2001-05-29 Entity tag cache hits

The entity-tag/variant index is now consulted on cache hits, making the implementation at least equal to "basic vary" but with a more compressed store (identical variants are only stored once)

2001-05-28 Entity tag and variant index

A index of known entity tags and variants are being kept for varying objects. This index is kept within the internal "vary" marker object.

2001-05-25 Issues identified

ETag implementation issues identified

2001-05-23 ETag documentation

Most ETag relevant documentation collected