[sldev] [ANN] certified http development

[Ryan Williams]

Donovan Linden wrote:
> I like the ZODB a lot, and there is also Divmod Axiom, which is 
> glyph's 3rd or 4th object database for Python, and I think this one is 
> pretty solid. It's based on embedding SQLite. However, the information 
> here which requires persistence is so simple that a real object 
> database would be way overkill.
>
That's a good point.  Also, the data is completely opaque to the chttp 
layer, with limited exceptions.  I think that a single method is 
actually enough to interact with the persistent store, no matter how 
it's implemented under the hood.  We shall see!  No need to discuss 
underlying technology libraries for your chickens before they're 
hatched, I guess.

> I think (rdw correct me if I am wrong) the current plan is just to use 
> mysql to store whatever needs to be stored for a Long Time. However, 
> this may be overkill in my mind as well.
>
We're thinking that we'll be doing *everything* in MySQL, because then 
we can wrap an entire do-something-non-idempotent-and-send-a-response 
into one database transaction and hope that those guys implemented ACID 
right.  :-)

We'll also have to store the Long Time data somewhere, MySQL is as good 
a place as any.  Doesn't have to be, though.

> I think the simplest possible solution is just to have the HTTP server 
> write requests as it reads them to a logfile along with the responses 
> after they are written. (It probably couldn't store the body, because 
> that could be of arbitrary size.) If the server crashes, it will read 
> in the logfile, build the appropriate "I've already done that" 
> structures based on what it sees, and then delete the logfile, 
> creating a new one to record all the traffic it successfully handles. 
> But I haven't been the one who has been thinking through the 
> implementation details, so this might not work.
I think this is necessary but not sufficient.  I dunno if this is 
"feature creep", but if you want to perform non-idempotent operations to 
generate a response, you want said operations to not be run multiple 
times even in the case where you've only done half of them and haven't 
written the response to the log before you crash.  Not having this 
guarantee basically means that the exactly-once messaging guarantee is 
worthless, because you can call the handler function an arbitrary number 
of times in the worst case, and that's not really distinguishable from 
receiving the message multiple times.

-RYaN