[sldev] Reformatting Textures for the cache

[John Hurliman]

Jason Giglio wrote:
>
> In a directory with 74k+ files there is very little difference in any 
> file system tasks.  You get similar times to these in an empty directory.
>
> "The filesystem is faster than you think." is probably good advice for 
> most programmers these days that jump to some overly complex solution 
> right away.
>
> -Jason 

For the record I think some variation of a direct filesystem cache will 
probably yield the best results, but I wanted to pitch in that a lot of 
the overhead with using a filesystem is more than just finding the file. 
The OS has to check if you have permission to access that file, check if 
any other programs have an access lock on that file, and do some 
bookkeeping work when you open and close the file. Working with a single 
large file containing lots of data (like a VFS) can be as simple as a 
malloc() and copying data in to memory.

To further the discussion about using the uncompressed filesize, there 
are quite a bit of different potential dimensions of images in SL. the 
valid dimensions that I am aware of are any combination of 8, 16, 32, 
64, 128, 256, 512, 1024, and 2048 (ridiculously huge for a real-time 3D 
world). The valid number of layers are 1 (grayscale), 2 (grayscale + 
alpha), 3 (rgb), 4 (rgba), and 5 (rgba + bump, used for avatar bakes). 
It's been a while since I took statistics, but that's quite a few 
different possible sizes of uncompressed image data. An avatar bake 
texture is 512x512x5 (except for the eyes), 1.3MB and there are three 
(or four if they have a skirt) of those for every avatar you see. 
Reading almost 6MB from the disk for every avatar you encounter seems 
like it would get out of control quickly. I can provide some sample SL 
texture data in compressed JP2 and uncompressed TGA format if people 
want to benchmark various compressors against it, let me know.

John Hurliman