[sldev] Texture Cache: A summary of The Plan as it stands

[Jason Giglio]

Buckaroo Mu wrote:
> MY PLAN: When receiving a texture from the region/asset server, it would 
> be first decoded, then store the llRawImage data - the first x number of 
> bytes (however many are currently used for JPG2000 texture headers) in 
> the texture.cache file, with the remaining data stored in the 
> cache/textures/x/uuid files. The index may also include such data as the 

Why continue to break it up?  Just put the raw files whole on the disk.

A reason that JPEG2000 has separate headers is because you can bite off
a fixed chunk at the beginning of the file and use that as a low rez FPO
while the rest is decoding. Raw data is not progressive like wavelet
data is, so breaking the file up makes little sense.

If we are talking about longer-term and larger caches, having a single
centralized header file is probably not the way you want to go for
scalability.



> The immediate impact I can see of these changes is a massive increase in 
> the render speed for previously-cached textures - although there may be 
> some slowdown for new textures, if the JPG2000 format is designed to 
> allow decoding to proceed from a low-resolution to a higher resolution 
> as the file is downloaded the first time. If I'm wrong about that, 
> please correct me (as with anything in this post). In other words, the 
> texture will not progressively decode as it's downloading - it will have 
> to download completely before it is decoded.


Keep in mind the option of enabling re-encoding to S3TC before saving to
disk.  Video cards handle S3TC natively, but it would affect quality.

You should probably do what you want to do here before we throw another
angle into it, but try to leave the door open for saving as S3TC
compressed using the video card hardware to encode and decode.

> This is all very first-stage planning - I presume that the cache discard 
> algorithm will have to be tweaked for larger caches, as well as checking 

We discussed this WAY back in the day.  Right now I believe it's just
using LRU, there are more effective discard algorithms out there.

> the memory footprint required by the larger cache (not sure how much is 
> stored in memory). Changes to the prioritization of downloading may also 

Yes, be careful here.  People perceive a growing in-memory cache as a
"memory leak".

> image format indicator as part of the texture.cache data. This could 
> theoretically pave the way for caching files delivered in other formats 
> - PNG, TGA, etc, should that ever become a potential benefit.

I would not worry about that at this point.  I would turn texture.cache
into XML to store pure metadata, and not the headers, as above.  Then
you can always add more properties if the need arises.

> I'm not sure if I see a benefit from XORing the raw texture data - 
> granted, it's much less processor-intensive than decoding is, but it's 
> still an extra step, and the system described herein would be just as 
> obfuscated as the existing system.

I would benchmark it with no obfuscation first, and then use that as a
baseline when adding any sort of obfuscation.   If you feel like it's
not obfuscated enough once you don't break up the file into two parts,
then the XOR with a constant is another option.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3253 bytes
Desc: S/MIME Cryptographic Signature
Url : http://lists.secondlife.com/pipermail/sldev/attachments/20080613/b874a2d2/smime-0001.bin