How to shrink the size of the index

Ideas which I have been tossing around in my head are:

1. Time information is mainly required by ICP and digests. Both only care for if the object is fresh or not. Time information is also required for "Age:" calculation. None of these require all of the information kept today, and can be abstracted down to fewer fields.
   • Use a internal delta-represenation for time. For example based on a one-year window (24 bits).
   • Compress all of the time fields to only two values:
     • request date (24 bits)
     • freshness ttl (16 bits)

     The first should be sufficient for Age: calculation, and both combined for ICP/digest.

     The freshness TTL represents the amount of time from the request date the object is considered fresh. Recalculated each time the full object data is known (refreshes or hits).

     The "last referenced" timestamp is redundant as this information is kept by the removal policy.

2. There is only one removal node per store entry, and the majority of the store entries do have a removal node.
   • Try to join RequestEntry and policy node together again avoid both malloc overhead and numerous pointers. Since the supported removal policies needs to be compiled into the code this should not be too hard. Should only be a matter of making sure there are space for one in the other, and then some simple pointer arithmetic to go the other way around. [DONE] (but there is still a pointer from removal-policy to entry)

     The simple way is to reserve space for the removal node in the store entry, and then use the OFFSETOF macro from cbdata to go back from the node to the StoreEntry.. And similary for memory entries..

     The only criteria is that a fixed amount of space is set aside for the removal policy, and that none of the policies attempts to use more than that (a simple sizeof assert in the policy initialization should make sure of that)

3. MD5 hashes are kept separately
   • Join the MD5 hash directly into the StoreEntry structure to save one pointer and one malloc overhead. [DONE]