[sldev] Cache speed experiment & results...

[Argent Stonecutter]

On 2008-06-04, at 01:06, Dale Mahalko wrote:
> Since GUID assignments are apparently random, it would be a waste of
> time to create deep subbranches before they are needed since many
> branches would likely remain empty, and empty branches must be
> iterated through when looking for a file.

I didn't pull this scheme out of thin air. I actively use it.

There's no searching or iterating by the client needed. No empty  
directories are created. When you create an object in the cache, you  
do something like this:

sprintf(filename, "%2.2s/%2.2s/%2.2s/%2.2s/%s", uuid, uuid+2, uuid+4,  
uuid+6, uuid);
if(!(fp = fopen(filename, "w"))) {
     filename[2] = 0; success = mkdir(filename); filename[2] = '/';
     if(success != ERR) { filename[4] = 0; success = mkdir(filename);  
filename[4] = '/'; }
     if(success != ERR) { filename[6] = 0; success = mkdir(filename);  
filename[6] = '/'; }
     if(success != ERR) { filename[8] = 0; success = mkdir(filename);  
filename[8] = '/'; }
     if(success != ERR) { fp = fopen(filename, "w"); }
     if(!fp) { fatal(filename, strerror(errno)); }
}

This means that when opening a cached file you have a single system  
call:


sprintf(filename, "%2.2s/%2.2s/%2.2s/%2.2s/%s", uuid, uuid+2, uuid+4,  
uuid+6, uuid);
if(!(fp = fopen(filename, "r"))) {
     fatal(filename, strerror(errno));
}

This avoids populating the tree unnecessarily, while making the  
common operation (reading from cache) MUCH faster than if the file  
name varies depending on depth.