Henrik Nordström's Squid work -> Squid Patches / Squid-2.2.STABLE

My patches to 2.2.STABLE

• Snapshots
• Individual patches

Squid-2.2 is the previous so called STABLE release of Squid. Due to various stability and performance issues with the Squid-2.3 and Squid-2.4 releases I still maintain my own Squid-2.2 version.

I regard the snapshots truly stable releases, suitable for high load production use. If you are looking for a specific fix then see the individual patches, but I'd recommend using the snapshot in most cases.

Snapshots

Snapshots are jumbo patches containin all the listed patches + other patches mostly from the Squid-2.2 known bugs page or seen on squid-bugs. The Squid version number of these are the version the snapshot patch should be applied to.

These snapshots also contain a cronological changelog documenting what was changed, why and when.

Snapshots can be downloaded from the hno-stable section on the Squid SourceForge page. (go to "All project files" if it is not in the overview)

My individual patches to Squid-2.2.STABLE5

This are my current set of patches to the Squid-2.2 release of Squid. Patches are listed in reverse order with the newest first, so if you have trouble applying a patch it is very likely that it depends on another patch further down in the list.

The Squid version number of the patch does not actually matter. All it indicates is what Squid version I used when I made the patch. All patches listed here applies to the Squid version indicated in the heading above.

If you need more than a few of these then I'd recommend you to grab the snapshot patch above instead.

• Squid-2.2.STABLE5-hno: Fix numerous cross-site scripting issues
  This patch fixes numerous cross-site scripting and HTML encoding errors in error pages, ftp directory listings and gopher results.
• Squid-2.2.STABLE5: Reject unencrypted https requests
  Reject unencrypted https: requests straight away, rather than first try to forward it and then discover than Squid does not know how to talk https...
• Squid-2.2.STABLE5: Handle NULL characters in the server reply headers [also requires two other patches: 1, 2 ]
  Squid failed to detect the end of the servers HTTP headers if the server wronly responds with headers containing a NULL character. This could cause abnormal amount of used cache_mem during the request. (the server in question was mp3 streaming, virtuallu unlimited in size)
• Squid-2.2.STABLE5: Persistent POST's blocking memory
  Persistent POST requests could block quite a bit of memory by not releasing request state data until the client connection was closed.
• Squid-2.3.DEVEL3: The last aclDomainCompare bugs squeezed [requires NLANR squid-2.2.stable5-domain-match.patch and my squid-2.2.STABLE4.isolate_splay.patch.. Note: the isolate_splay patch must be applied with --fuzz=3 due to overlapping regions with the NLANR patch.]
  This patch fixes some remaining aclDomainCompare issues where Squid gave more "is a subdomain of" warnings than it needed to.
• Squid-2.2.STABLE5: Corrections to age calculation
  Some adjustments of how Squid calculates object age.
  * Take any existing Age header into account
  * Adjust properly if the origin servers clock is ahead of us
  * Don't emit a Age header unless it is a cache hit with a age > 0, this is to work around a browser bug in Microsoft IE where IE hangs on certain uncachebale pages.
• Squid-2.3.DEVEL3: Default reference_age is one year
  Minor correction to the comments in squid.conf. The default reference_age is one year, not one month as incorrecly stated in squid.conf comments.
• Squid-2.2.STABLE5: pconn_timeout on client connections / disabling
  Changed pconn_timeout to apply to client connections as well, and added the rule that persistent connections will be disabled if pconn_timeout is < 10 seconds.
• Squid-2.3.DEVEL3: Assertion failure on invalid PASV replies (FTP)
  Squid failed with a assertion failure if a invalid reply to PASV was received.
• Squid-2.3.DEVEL3: FTP log level adjustment for ignored "errors"
  Use log level 3 on ignored read-"errors" like EAGAIN (was 1).
• Squid-2.2.STABLE5: chroot support
  Adds chroot support to Squid
• Squid-2.2.STABLE5: Persistent connections and IMS-HIT
  Squid unintentionally denied persistent connections on IMS-HIT
• Squid-2.2.STABLE5: Ignore Host header
  Recreate Host header from scratch on every request. This is to protect Squid from a malicous user sending requests with inconsistent request URI and Host header, which could teoretically be used to cause cache pollution if the attacker has control of a another web site sharing the same IP address as the site they want to pollute and is allowed to use Squid.
• Squid-2.3.DEVEL3: Range request could cause bandwidth spikes
  Range requests to servers/objects not supporting range requests could cause bandwidth spikes and/or negative hit ratio even if range_offset_limit is set to 0.
• Squid-2.2.STABLE5: ipc hello test fails on some platforms/compilers
  A missing \0 string terminator could on some platforms/compilers cause squid to fail the hello test used when starting child processes.
• Squid-2.2.STABLE3: Fix ARP acl warning
  Fix for a silly bug in the code which dumps ARP acl's, causing a compile time warning and incorrect output in cachemgr.
• Squid-2.2.STABLE5: FreeBSD 3.3 statfs
  FreeBSD requires sys/mount.h for statfs().
• Squid-2.3.DEVEL3: HEAD and ftp://...
  Support HEAD ftp://.. requests without fetching the whole object.
• Squid-2.2.STABLE4: --enable-optimistic-io and fixes
  Configure option --enable-optimistic-io, and a fix for a minor inconsistency when OPTIMISTIC_IO is used.
• Squid-2.3.DEVEL2: Purge ipcache on reload/PURGE
  Purge negatively cached ipcache entries on reload to allow end users to quickly purge sporious errors, and have the PURGE method also purge ipcache to allow the cache administrator to manually purge important entries on DNS updates.
• Squid-2.2.STABLE4: Keep stdio filehandles in daemon mode
  This patch causes Squid to keep stdio filehandles open when starting in daemon mode. If you do not want this them make sure to redirect them to /dev/null
  squid </dev/null >/dev/null 2>&1
  I personally redirect stdout and stderr to /dev/console.
• Squid-2.2.STABLE4: cachemgr object listings improvements
  This patch speeds up object listing generation, and adds two new menu entries: "Lost StoreEntry structures" for listing objects which is neither in memory or on disk, and "Large In-Memory and In-Transit Objects" showing objects taking up an proportionally large portion of the VM cache.
• Squid-2.2.STABLE4: Include request in helper statistics
  Include the request sent in helper statistics to allow the administrator to find out why the helper processes is busy.
• Squid-2.2.STABLE5: Restart helpers when rotating logs
  Restart helper processes when rotating logs, to have cache.log properly rotated.
• Squid-2.2.STABLE4: helpers and open filedescriptors
  Make sure all unneeded filedescriptors is properly closed when starting helper processes.
• Squid-2.2.STABLE4: Delay pid file removal until shutdown
  Delay removal of the pid file until Squid is fully shut down. This is to allow scripts to monitor the pid file waiting for Squid to shut down.
• Squid-2.2.STABLE4: Release unused store entries during rebuild
  Squid locked all "released" store entries in memory during the rebuild procedure, even such entries not needed in the "LateRelease" procedure. Also, during a dirty rebuild a lot of store entries got locked up which never was queued for "LateRelease", causing a huge memory leak.
• Squid-2.2.STABLE4: Free cache_mem objects during cache rebuild
  Squid locked objects in cache_mem to no apparent reason other than causing the cache_mem usage to grow huge during cache rebuilds.
• Squid-2.2.STABLE5: proxy_auth_regex and ident_regex ACL types
  This patch adds proxy_auth_regex and ident_regex ACL types
• Squid-2.2.STABLE5: authenticate_ip_ttl squid.conf option
  With this option you can control how long a proxy authentication will be bound to a specific IP address.
• Squid-2.2.STABLE4: Unexpected 304 replies
  There was a odd HTTP condition that could cause Squid to return "304 Not modified" on plain GET requests without If-Modified-Since. This would happen if the first attempt to retreive an object results in a 5XX error with a Last-Modified header. Squid then automatically tries to find another path for fetching the object, but accidently beleived the retry was a refresh of the error page...
• Squid-2.2.STABLE4: Differentiate cache digest hits on peer type
  Split the CACHE_DIGEST_HIT log tag into CD_PARENT_HIT and CD_SIBLING_HIT
• Squid-2.2.STABLE4: snmp_port disabled by using 0
  Cosmetic change in the documentation on snmp_port, to make it more consistent with icp_port (use "0" to disable, said "-1". In fact any value <= 0 disables SNMP)
• Squid-2.2.STABLE4: dns_restart option to keep dnsserver in bay
  Adds dns_restart option to have dnsservers periodically restart themselves to purge memory. This is to work around apparent leaks in dnsserver (or the resolver librarby). dnsserver processes has seen growing from a few MB up to 50MB in size.
• Squid-2.2.STABLE5: --enable-underscores
  Added --enable-underscores to have Squid not reject hostnames with _ as part of their name. Squid by default rejects such names to conform with internet standards. (this only adds the configure option to define ALLOW_HOSTNAME_UNDERSCORES, the code has been there since long back)
• Squid-2.2.STABLE4: FS statistics patch for Linux
  Added a workaround for broken statvfs output on Linux. (f_frsize == 0)
• Squid-2.2.STABLE4: Higher helper limit to avoid dns queue assertion
  Allow for a larger backlog of requests to helper processes such as dnsserver. Squid will abort with a assertion failure if the backlog grows to large. This patch changes the limit from 2*n_helpers to 5*n_helpers
• Squid-2.2.STABLE4: cachemgr storedir fs available space
  Changed cachemgr filesystem statistics to take into account the amount of reserved (root only) disk space instead of showing raw disk space available.
• Squid-2.2.STABLE4: cache_swap_log name based on cache_dir name
  Allow %s to be used in cache_swap_log to build a log file name based on the cache_dir name instead of numbering the files according to their corresponding cache_dir location in squid.conf. This is very useful if you'd like to be able to add or remove cache directories while using cache_swap_log to have the index files stored outside the cache directories.
• Squid-2.2.STABLE4: Disable pipeline prefetching
  Disable parallell fetches of pipelined requests. There seems to be serious problems if something goes wrong with the second request while the first one is being processed, so the safest bet is to disable this prefetching of pipelined requests for the time being.
• Squid-2.2.STABLE4: Event-queue starvation on high load
  Under high load events in the event queue could experience starvation. This patch fixes a obvious problem where cancelled events would make the queue stall, and boosts the priority of event handling somewhat.
  It also adds more detail to the even queue output in cachemgr.
• Squid-2.2.STABLE4: async-io NUMTHREADS warning threshold
  Use a higher threshold for reporting async-io thread shortage, and a small spell correction in the same message.
• Squid-2.2.STABLE4: Async-IO sync fixes
  Async IO sync operation was somewhat incomplete. There could be some operations pending when the sync completed.
  Also, operations needs to be synced when switching user-id's, or it is uncertain which userid the operations get executed as.
• Squid-2.2.STABLE4: Double slashes on top level FTP directory
  Don't generate a double trailing slashes in BASE HREF if a user opens a FTP server without trailing slash (as in "ftp://squid.nlanr.net"). This only affected the top level directory.
• Squid-2.2.STABLE4: Test for sys_errlist always failing
  Due to a quoting problem in configure.in the sys_errlist test was always failing.
• Squid-2.2.STABLE4: Escape control characters in log files [depends on ftp_password_urls]
  This patch escapes any control characters in the log files, and also fixes a problem with "uri_whitespace encode" where already escaped characters could get doubly escaped.
• Squid-2.2.STABLE4: Async-IO queue info in cachemgr
  Show async-io queue length in cachemgr aio_counters
• Squid-2.2.STABLE4: Async-IO segfaults if USE_PROPER_MUTEX isn't set
  Async-IO on Linux segfaults in condition variables if given high load and USE_PROPER_MUTEX isn't set. This was seen on a Alpha Linux 2.2.10-ac12 box. I knew there was a reason why I made the USE_PROPER_MUTEX code a long time ago..
• Squid-2.2.STABLE4: Some tuning of Async-IO code
  This patch makes some tuning of the Async-IO code to avoid wasting threads on operations which usually does not block. It also reverts to using unlinkd for unlinks even when using async-io due to some load balancing troubles. This can be tuned in include/config.h if you'd like to.
• Squid-2.2.STABLE3: Send configured login to SSL peers
  The cache_peer configured login did not get sent on SSL requests
• Squid-2.2.STABLE3: Switch back to unlink instead of truncate
  Switch back to unlink instead of truncate when releasing cache files. There are some known problems with the use of truncate
  • Very much increased inode usage (slightly more than doubled)
  • Additional race conditions when storing files, which theoretically can cause corrupted cache files if a file is reused while being truncated.
• Squid-2.2.STABLE3: proxy_auth and spaces in username or password
  This patch adds support for spaces in the username or password by encoding unsafe characters before calling the authenticator. Please note that this patch breaks compability with any existing authenticator modules and you need to URL unescape the username and password prior to processing or authentication will fail if unsafe characters is used in the username or password.
• Squid-2.2.STABLE4: Support generic request entities
  Support generic request entities as needed by WebDAV (RFC 2518). Now it is theoretically possible to use WebDAV with Squid, but only if the server does not do strict HTTP/1.1 version checks (Squid still downgrades requests to HTTP/1.0 as required by HTTP standars). You will also need the patch from below adding the new methods to the list of known HTTP methods.
• Squid-2.2.STABLE3: async-io is a bit keen on warn on thread usage
  Async-IO is a bit keen on give a warning about thread usage. This patch increases the burst filter threashold a little bit. Hopefylly I will find time to address this at the real root of the problem (bursty store recycling) in the near future.
• Squid-2.2.STABLE5: Log destination IP on DIRECT
  This patch logs the destination IP as part of the hierarchy tag in access.log when going direct. This has been requested by a number of people from accounting reasons, and logging the hostname is mostly redundant as it is part of the URL as well.
• Squid-2.2.STABLE3: Assertion failure of FTP timeouts
  There was a bug in one of my earlier FTP patches causing an assertion failure on timeouts. [Found by Apiset Tananchai <aet@demo.ksc.co.th>]
• Squid-2.2.STABLE3: use statfs() if statvfs() isn't available
  Some systems (most notably Linux) uses statfs() instead of statvfs(). Squid uses this to print out interesting statistics about real disk usage in it's statistics pages. Not yet used for operational changes.
• Squid-2.2.STABLE3: Minor update to proxy_auth documentation
  The example proxy_auth acl used old (2.0) syntax.
• Squid-2.2.STABLE3: Use O_NONBLOCK instead of O_NDELAY
  The check for O_NONBLOCK was malplaced and O_NDELAY was always selected even if O_NONBLOCK is available.
• Squid-2.2.STABLE3: storeAppend assertion failure on aborted FTP
  Fix for 'assertion failed: store.c:404: "e->store_status == STORE_PENDING"' errors on aborted FTP requests.
• Squid-2.2.STABLE3: Persistent connections request_timeout
  Persistent connections used a hardcoded timeout of 15 seconds instead of request_timeout as documented in squid.conf.
• Squid-2.2.STABLE2: delay pools, large initial level
  It is a bit to easy to get a integer overflow when using delay pools for limiting daily download. This patch changes the initial calculation to use floating point math, allowing a initial pool size of up to 2^31-1.
• Squid-2.2.STABLE4: FTP password URLs [depends on ftp_broken_downloads]
  Changes Squid to preserve any password which was entered in the URL when BASE HREF is used to "correct" directory URLs without a trailing /.
  This patch also fixes a minor issue with URL encoding of filenames. Squid only encoded those characters classified as "unsafe", not those classified as "reserved". What this means is for example if a directory contains a file with a name including "/" then Squid would be confused.
• Squid-2.2.STABLE2: Add new methods defined in RFC2518
  This patch adds forwarding capability for the methods defined in RFC2518 (WebDAV). There is however one important bug left to kill before WebDAV may function through Squid (requests other than PUT/POST containing a requests body are truncated to headers only).
• Squid-2.2.STABLE2: allow-miss cache_peer option
  allow-miss cache peer option to disable the use of "only-if-cached" on requests to siblings. This can be useful in some peering arrangements where icp_hit_stale is enabled.
• Squid-2.2.STABLE2: Don't swap out objects > maximum_object_size
  Don't start swapping out objects with a known size larger than maximum_object_size. Previously Squid would swap out these objects and mark it as private once maximum_object_size was hit.
• Squid-2.2.STABLE2: Configure fix for Solaris/X86
  Configure claimed that it enabled dlmalloc on Solaris/X86, but it didn't actually succeed in doing so.
• Squid-2.2.STABLE3: Verify object meta-data on swap-in
  A additional safeguard to protect Squid from cache pollution/corruption. This patch verifies that the swapped in object matches both the URL and the store key, if not then the object is discarded. This also fixes the potential false-object-hit introduced by Squids hashed store keys.
• Squid-2.2.STABLE3: Don't give ICP/Digest HIT on non-200 objects [not in my snapshot]
  There is a common false hit condition with objects with a HTTP status other than 200 (HTTP_OK). These will cause false hits if a client sends a If-Modified-Since request.
  This patch makes a minor change to the on-disk store, and may or may not be fully compatible with future Squid releases. Don't use this patch unless you are prepared to clean your cache on next upgrade.
• Squid-2.2.STABLE2: UNIX domain IPC communication
  Switches Squid to UNIX domain IPC communication with it's child processes instead of TCP/IP on the loopback interface.
• Squid-2.2.STABLE2: Use estimated RTT for parent selection on timeouts
  Use estimated ICP rtt when selecting parents on ICP timeouts.
• Squid-2.2.STABLE2: ICP timeout selection
  This is an attempt to fix the dynamic ICP timeout selection when one is peering with remote parents and have some close-by siblings with a much lower ICP rtt. This is done by preferring to calculate the ICP timeout based on parents only (based on siblings if there is no alive parents)
• Squid-2.2.STABLE2: comm_poll NULL write handler warning
  This patch is an attempt in fixing/locating those "NULL write handler" warnings which may be seen when poll is enabled.
• Squid-2.2.STABLE4: myport ACL type
  A new ACL type for matching the local port number
• Squid-2.2.STABLE2: Support multiline headers
  HTTP allows request/response headers to be broken up into multiple lines. Squid only parsed the first line, ignoring any continuation lines.
• Squid-2.2.STABLE2: Change log_mime_hrds output to be human readable
  Don't encode more than what is needed to be able to parse the line without ambiguity, and encode \r and \n as "\n" and "\r".
• Squid-2.2.STABLE2: Make the suggested refresh_pattern settings the default
  Make the suggested refresh_pattern settings the default in the distributed squid.conf. They are really needed, or FTP and GOPHER will not be cached at all.
• Squid-2.2.STABLE2: cache_peer connect timeouts [depends on tcp_dead_peer_detection_and_retry]
  This is the last of a series of patches to improve non-ICP peering relations. This patch introduces different timeouts for peer connections. Both a global peer_connect_timeout option, and a cache_peer connect-timeout option. It can be assumed that peers are relatively stable, and that the peer is not working properly when it takes "long" time to connect to one. Default timeout here is 30 seconds, as opposed to the standard connect_timeout of 2 minutes.
• Squid-2.2.STABLE2: Authentication header parsing
  Squid was rather strict about the syntax of authentication headers sent to Squid. This patch extends it to be somewhat forgiving about the syntax used in accordance with HTTP/1.1 guidelines.
• Squid-2.2.DEVEL3: Common log format dates, and end of year
  Common log format dates had a bug which could cause time zone indication to be way off (about a year off) when close to the final hours of the year.
• Squid-2.2.DEVEL3: tcp_incoming_address and visible_hostname
  Make visible_hostname default to tcp_incoming_address if unset
• Squid-2.2.STABLE3: Extended logging of proxy_auth errors
  This patch extends the logging of unexpected proxy_auth information when a access is denied because Squid failed to understand the credentials.
• Squid-2.2.STABLE2: http_port bind address
  Allow one to specify which address each port specified in http_port should be bound to, using address:port syntax.
• Squid-2.2.DEVEL3: Don't allow netdb selection to bypass never_direct
  Netdb selection could bypass never_direct and cause Squid to go direct to an origin site even if never_direct allow was in effect.
• Squid-2.2.DEVEL3: Handle overloaded async-io more gracefully
  Make overloaded async-io a non-fatal error (block instead of terminate), and take some extra actions to limit the risk of overloading.
• Squid-2.2.STABLE3: nonhierarchical_direct squid.conf directive
  Adds a new squid.conf directive: nonhierachical_direct. This controls if requests Squid classifies as non-hierarchical (matches hierarchy_stoplist or non-cachable request type) should go direct if possible, or if parents should be used on such requests.
  Also improved parent selection for never_direct (selects all available parents incase the primary one should fail).
• Squid-2.2.STABLE3: Improved TCP dead peer detection & failover
  A major adjustment of how Squid detects a failing TCP peer and how connections is retried when it fails to connect.
• Squid-2.2.DEVEL3: Improved ICP dead peer detection
  Some minor adjustments of ICP dead peer detection to make it behave well on startup and low load servers. Also lessens the amount of queries sent to dead peers.
• Squid-2.2.STABLE3: Fall back on PORT [depends on ftp_broken_downloads]
  Recover if ftp fails to establish PASV data connection.
• Squid-2.2.STABLE4: Isolate Splay-tree structures
  Cosmetic change to isolate splay-tree structures from the rest of the code.
• Squid-2.2.STABLE5: Don't cache interrupted FTP transfers
  In some circumstances interrupted FTP transfer was cached.
• Squid-2.2.DEVEL3: Assertion failure on FTP PUT to directories
  Squid died with an assertion failure on FTP PUT requests to directories without a filename. This patch changes it to use STOU, or simply MKD if there is no object to store.
• Squid-2.2.DEVEL3: Blank content types logged in store.log
  If the origin server sent a blank content type, then store.log logged a blank field, making it hard to parse. This patch logs "unknown" as is logged when no content type is provided at all.