[sldev] Cache politics: performance vs obfuscation

Michael Schlenker schlenk at uni-oldenburg.de
Mon Jun 9 23:00:48 PDT 2008


Argent Stonecutter schrieb:
> On 2008-06-09, at 02:22, Dale Mahalko wrote:
>> A move to a local MySQL database
> 
> God no.
> 
> sqlite, please. Without all the MySQL "I'm pretending to be a real 
> database engine" overhead.

SQLites pretty good for some stuff, but caching blob data is not one of 
it due to the required frequent fsync() calls needed to maintain 
transactions. (just see the current discussion around Firefox 3 being 
very slow on Ubuntu).

But MySQL is an even worse idea...

I had to write a heavily parallel (targeting some 1000 parallel writing 
threads) blob storage system for a different application and using 
SQLite, even only for storing the metadata leads to heavy lock 
contention and made the system far less scalable. Current filesystems 
like NTFS or ext3 are pretty okay for storing files and metadata, they 
are made for that kind of usecase..., its even faster to do a stat() on 
disk than to lookup stuff in sqlite in most cases.

Michael




More information about the SLDev mailing list