[sldev] Cache speed experiment & results...
Tateru Nino
tateru.nino at gmail.com
Tue Jun 3 21:34:39 PDT 2008
Also you get a limited (though large) number of write cycles to any byte
before it wears out and stops responding.
Dahlia Trimble wrote:
> If the goal were to improve seek time in a cache, how about keeping
> the cache on one of those readyboost-capable usb flash drives? They
> can have seek times as low as 1 ms or better, although the transfer
> rate and write times arent all that fast.
>
> Then again it's probably moot if the bottleneck is the decoding :/
>
> On Tue, Jun 3, 2008 at 5:12 PM, Felix Duesenburg <kfa at gmx.net
> <mailto:kfa at gmx.net>> wrote:
>
> Dale Mahalko wrote:
>
> ...
> Windows isn't good at dealing with directories containing tens of
> thousands of files. Ever tried opening a folder containing 20,000
> files? There is a delay of several seconds trying to do this in
> Windows. Also the hidden MFT and directory structures can become
> ridiculously fragmented, also leading to slowdowns.
>
>
> Reiser4 is excellent with exactly these requirements. It's not
> mature enough to be considered reliable for critical systems, but
> for a cache? It doesn't help that Reiser himself is removed from
> the development process, but it's still there and the source
> available. Shouldn't that be possible to be included as a virtual
> filesystem even under Windows?
>
> I also always thought it must be a good way to boost performance
> if the cache lived on its own partition, like a Linux swap
> partition. For the Linux version of SL this would even sound like
> the most natural way to do it. Setting such a thing up on an
> existing Windows machine has a slight fear factor attached and is
> not for everyone, but if a savvy user can handle it, why not
> offer. I'm speculating, but the difference could be dramatic,
> especially if we had a better file system than NTFS on that
> partition.
>
>
> >
> > A proper local database engine would likely deal with these huge
> > numbers of tiny files more efficiently than the local operating
> > system.
>
> Yes, but the usual architecture with the database running as a
> localhost server is not ideal for our situation since everything
> is squeezed back and forth through the network system, plus SQL
> parsing, unnecessarily. With MySQL you could build directly on top
> of the storage engine though, even bypassing the parsing with
> direct API calls. It would just serve as a virtual file system the
> same way as suggested above. I still believe that Reiser4 would be
> even faster. And according to the description on its (now defunct)
> website, they claimed reaching 94% package density with small files.
>
> Felix
>
> _______________________________________________
> Click here to unsubscribe or manage your list subscription:
> /index.html
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Click here to unsubscribe or manage your list subscription:
> /index.html
>
--
Tateru Nino
http://www.massively.com/
More information about the SLDev
mailing list