[sldev] Cache speed experiment & results...

[Dahlia Trimble]

If the goal were to improve seek time in a cache, how about keeping the
cache on one of those readyboost-capable usb flash drives? They can have
seek times as low as 1 ms or better, although the transfer rate and write
times arent all that fast.

Then again it's probably moot if the bottleneck is the decoding :/

On Tue, Jun 3, 2008 at 5:12 PM, Felix Duesenburg <kfa at gmx.net> wrote:

> Dale Mahalko wrote:
>
>> ...
>> Windows isn't good at dealing with directories containing tens of
>> thousands of files. Ever tried opening a folder containing 20,000
>> files? There is a delay of several seconds trying to do this in
>> Windows. Also the hidden MFT and directory structures can become
>> ridiculously fragmented, also leading to slowdowns.
>>
>>
> Reiser4 is excellent with exactly these requirements. It's not mature
> enough to be considered reliable for critical systems, but for a cache? It
> doesn't help that Reiser himself is removed from the development process,
> but it's still there and the source available. Shouldn't that be possible to
> be included as a virtual filesystem even under Windows?
>
> I also always thought it must be a good way to boost performance if the
> cache lived on its own partition, like a Linux swap partition. For the Linux
> version of SL this would even sound like the most natural way to do it.
> Setting such a thing up on an existing Windows machine has a slight fear
> factor attached and is not for everyone, but if a savvy user can handle it,
> why not offer. I'm speculating, but the difference could be dramatic,
> especially if we had a better file system than NTFS on that partition.
>
> >
> > A proper local database engine would likely deal with these huge
> > numbers of tiny files more efficiently than the local operating
> > system.
>
> Yes, but the usual architecture with the database running as a localhost
> server is not ideal for our situation since everything is squeezed back and
> forth through the network system, plus SQL parsing, unnecessarily. With
> MySQL you could build directly on top of the storage engine though, even
> bypassing the parsing with direct API calls. It would just serve as a
> virtual file system the same way as suggested above. I still believe that
> Reiser4 would be even faster. And according to the description on its (now
> defunct) website, they claimed reaching 94% package density with small
> files.
>
> Felix
>
> _______________________________________________
> Click here to unsubscribe or manage your list subscription:
> /index.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/sldev/attachments/20080603/98809374/attachment-0001.htm