---------------------
PatchSet 5011 
Date: 2007/07/12 04:45:40
Author: amosjeffries
Branch: docs
Tag: (none) 
Log:
Pull old StoreageInterface documentation into FileSystems module auto-docs.

Members: 
	src/fs/FileSystems.dox:1.1.2.1->1.1.2.2 

Index: squid3/src/fs/FileSystems.dox
===================================================================
RCS file: /cvsroot/squid-sf//squid3/src/fs/Attic/FileSystems.dox,v
retrieving revision 1.1.2.1
retrieving revision 1.1.2.2
diff -u -r1.1.2.1 -r1.1.2.2
--- squid3/src/fs/FileSystems.dox	12 Jul 2007 01:08:08 -0000	1.1.2.1
+++ squid3/src/fs/FileSystems.dox	12 Jul 2007 04:45:40 -0000	1.1.2.2
@@ -2,8 +2,107 @@
 \defgroup FileSystems	Storage Filesystems
 \ingroup Modules
 
-\todo DOCS: pull generic FS documentation into here
-	individual FS documentation goes in its source files.
-	and the FS gets its own API group which is \ingroup FielSystems
+\section Introduction Introduction
+
+\par
+	Traditionally, Squid has always used the Unix filesystem (\link UFS UFS\endlink)
+	to store cache objects on disk. Over the years, the poor performance of UFS has
+	become very obvious. In most cases, UFS limits Squid to about 30-50 requests per
+	second. Our work indicates that the poor performance is mostly due to the
+	synchronous nature of open() and unlink() system calls, and perhaps thrashing of
+	inode/buffer caches.
+
+\par
+	We want to try out our own, customized filesystems with Squid. In order to do that,
+	we need a well-defined interface for the bits of Squid that access the permanent
+	storage devices. We also require tighter control of the replacement policy by each
+	storage module, rather than a single global replacement policy.
+
+
+\todo DOCS: pull remaining generic FS documentation into here fully.
+
+
+\section Introduction Introduction
+\par
+	Traditionally, Squid has always used the Unix filesystem (\link UFS UFS\endlink)
+	to store cache objects on disk.  Over the years, the
+	poor performance of \link UFS UFS\endlink has become very obvious.  In most
+	cases, \link UFS UFS\endlink limits Squid to about 30-50 requests per second.
+	Our work indicates that the poor performance is mostly
+	due to the synchronous nature of open() and unlink()
+	system calls, and perhaps thrashing of inode/buffer caches.
+
+\par
+	We want to try out our own, customized filesystems with Squid.
+	In order to do that, we need a well-defined interface
+	for the bits of Squid that access the permanent storage
+	devices. We also require tighter control of the replacement
+	policy by each storage module, rather than a single global
+	replacement policy.
+
+\section BuildStructure Build structure
+\par
+	The storage types live in \em src/fs/. Each subdirectory corresponds
+	to the name of the storage type. When a new storage type is implemented
+	configure.in must be updated to autogenerate a Makefile in
+	\em src/fs/foo/ from a Makefile.in file.
+\todo DOCS: add template addition to configure.in for storage module addition.
+\todo DOCS: add template Makefile.am for storage module addition.
+
+\par
+	configure will take a list of storage types through the
+	--enable-store-io parameter. This parameter takes a list of
+	space seperated storage types. For example,
+	--enable-store-io="ufs coss" .
+
+\par
+	Each storage type must create an archive file
+	in \em src/fs/foo/.a . This file is automatically linked into
+	squid at compile time.
+
+\par
+	Each storefs must export a function named \em storeFsSetup_foo().
+	This function is called at runtime to initialise each storage type.
+	The list of storage types is passed through store_modules.sh
+	to generate the initialisation function storeFsSetup(). This
+	function lives in store_modules.c.
+\todo DOCS: find out what has replaced storeFsSetup() and store_modules.c
+\todo DOCS: find out what has replaced storefs
+
+\par Example of the automatically generated file:
+\code
+	// automatically generated by ./store_modules.sh ufs coss do not edit
+
+	#include "squid.h"
+
+	extern STSETUP storeFsSetup_ufs;
+	extern STSETUP storeFsSetup_coss;
+	void storeFsSetup(void)
+	{
+		storeFsAdd("ufs", storeFsSetup_ufs);
+		storeFsAdd("coss", storeFsSetup_coss);
+	}
+\endcode
+\todo DOCS: find out what has replaced storeFSAdd
+\tdo DOCS: find out what has replaced storeFSSetup and storeFSSetup_*
+
+\section OperationOfStorageModules Operation of a Storage Module
+\par
+	Squid understands the concept of multiple diverse storage directories.
+	Each storage directory provides a caching object store, with object
+	storage, retrieval, indexing and replacement.
+
+\par
+	Each open object has associated with it a storeIOState object. The
+	storeIOState object is used to record the state of the current
+	object. Each storeIOState can have a storage module specific data
+	structure containing information private to the storage module.
+
+\par
+	Each SwapDir has the concept of a maximum object size. This is used
+	as a basic hint to the storage layer in first choosing a suitable
+	SwapDir. The checkobj function is then called for suitable
+	candidate SwapDirs to find out whether it wants to store a
+	given StoreEntry. A maxobjsize of -1 means 'any size'.
 
  */