Henrik Nordström's Squid work -> Migrating a cache from Squid 1.x to Squid 2.x

Bruce Campbell wrote:

> Since we are 0cost between each other, and I just so happen to have a
> nearly empty cache, I hit upon doing something silly like:
> 
> grab the URLS in the oldcache
> oldcache: cat /cache1dir/log | cut -d ' ' -f 6- >urllist

OK if this log is from a clean shutdown.

> reconfigure the new cache to *only* query the old cache, and get
> the urls into the newcache ( gee, that bitbucket gets very full! ;) ):

Yes this is fully possible, but it may generate a fair amount of traffic
for expired objects that you think are in the cache. A easy way to avoid
this extra traffic is to configure the old cache as a parent to the new
one, and deny miss access. This assumes that you don't have to serve
clients using the new cache while migrating cache contents.

Here is a slightly updated version of my proposal on how to migrate a
1.1.X cache to 1.2, without the need for additional disk space.

1. Set up the new cache on a different port, using the old cache as a
parent. The same disks can be used provided that you use different
directory names.
2. Reconfigure the old cache to deny miss_access to the new cache to
avoid generating extra net traffic for expired objects, and enable the
PURGE method if not enabled.
3. Build a list of cached URLs using a clean cache log (i.e. while the
old cache is down).
4. Build a small program that for each URL
   a) Issues a GET request using the new cache
   b) Issues a PURGE request using the old cache (*1)
5. Change the new squid.conf to a normal one (standard port + peering)
6. Shut down both Squid processes
7. Start the new Squid. It is now running live with most of the old
cache preserved.
8. Wipe the remainings of the old cache directory to reclaim any lost
disc-space (it should be almost empty my now).

*1) No program exists that does this but it can easily be implemented as
a wrapper around tcp-banger2 (available in the test-suite directory.
build with "make tcp-banger2"). Untested example script attached.

---
Henrik Nordström
Sparetime Squid Hacker

[FILE] 1.2_migrate.sh