[sldev] Cache speed experiment & results...

Tateru Nino tateru.nino at gmail.com
Tue Jun 3 21:34:39 PDT 2008


Also you get a limited (though large) number of write cycles to any byte 
before it wears out and stops responding.

Dahlia Trimble wrote:
> If the goal were to improve seek time in a cache, how about keeping 
> the cache on one of those readyboost-capable usb flash drives? They 
> can have seek times as low as 1 ms or better, although the transfer 
> rate and write times arent all that fast.
>  
> Then again it's probably moot if the bottleneck is the decoding :/
>
> On Tue, Jun 3, 2008 at 5:12 PM, Felix Duesenburg <kfa at gmx.net 
> <mailto:kfa at gmx.net>> wrote:
>
>     Dale Mahalko wrote:
>
>         ...
>         Windows isn't good at dealing with directories containing tens of
>         thousands of files. Ever tried opening a folder containing 20,000
>         files? There is a delay of several seconds trying to do this in
>         Windows. Also the hidden MFT and directory structures can become
>         ridiculously fragmented, also leading to slowdowns.
>
>
>     Reiser4 is excellent with exactly these requirements. It's not
>     mature enough to be considered reliable for critical systems, but
>     for a cache? It doesn't help that Reiser himself is removed from
>     the development process, but it's still there and the source
>     available. Shouldn't that be possible to be included as a virtual
>     filesystem even under Windows?
>
>     I also always thought it must be a good way to boost performance
>     if the cache lived on its own partition, like a Linux swap
>     partition. For the Linux version of SL this would even sound like
>     the most natural way to do it. Setting such a thing up on an
>     existing Windows machine has a slight fear factor attached and is
>     not for everyone, but if a savvy user can handle it, why not
>     offer. I'm speculating, but the difference could be dramatic,
>     especially if we had a better file system than NTFS on that
>     partition.
>
>
>     >
>     > A proper local database engine would likely deal with these huge
>     > numbers of tiny files more efficiently than the local operating
>     > system.
>
>     Yes, but the usual architecture with the database running as a
>     localhost server is not ideal for our situation since everything
>     is squeezed back and forth through the network system, plus SQL
>     parsing, unnecessarily. With MySQL you could build directly on top
>     of the storage engine though, even bypassing the parsing with
>     direct API calls. It would just serve as a virtual file system the
>     same way as suggested above. I still believe that Reiser4 would be
>     even faster. And according to the description on its (now defunct)
>     website, they claimed reaching 94% package density with small files.
>
>     Felix
>
>     _______________________________________________
>     Click here to unsubscribe or manage your list subscription:
>     /index.html
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Click here to unsubscribe or manage your list subscription:
> /index.html
>   

-- 
Tateru Nino
http://www.massively.com/



More information about the SLDev mailing list