[sldev] [PROTOCOL] Protocol Documentation

[John Hurliman]

I merged my own documentation skeleton page on the wiki with the 
"Protocol" page at http://wiki.secondlife.com/wiki/Protocol, this is 
still the official home of all protocol documentation for Second Life 
and I assume will be the closest thing there is to an official 
specification of "How Second Life Works".

Community driven documentation is going to be more important then it was 
previously due to the way the protocol is changing. Traditionally we had 
everything going over UDP and the format of things was documented by a 
file called message_template.msg. This allowed clients to either build 
packets and verify them at runtime (such as the SL viewer) or 
pre-generate code to represent packets in memory (such as 
libsecondlife). With the new capabilities system we have no 
message_template.msg, and there is no way to pre-generate code to 
represent message structures or verify the correctness of code at 
runtime. Right now you can't even query what capabilities are available 
to clients although this may change in the future.

The message_template party was fun while it lasted but we are going to 
have to go back to the traditional method of development, meaning good 
documentation that unit tests can be built from and third party 
implementations can work off of, since runtime or compile time 
verification is no longer possible. Linden Lab internally uses the 
viewer and simulator source code for documentation (I'm told), so if we 
want a specification the community will be responsible for generating 
it. Some open questions:

* Is the current structure of the page a good breakdown of the different 
sections of the protocol? Should things be split up or consolidated in 
any places?
* Since all of the capabilities will need to be described here we need a 
standard markup to document them. Should the LLSD notation format be 
used, or some other pseudo-markup?


For some background, this was discussed at Zero Linden's office hour 
today (Zero was out but other Lindens filled in). The conversation went 
something like this:

LL: the capabilities API may not necessarily be the same as the old 
message template
Me: so we need to hand-write new processing code for each capability, 
unlike the old udp system
Others: wouldn't a server-side changelog fix this?
Me: if i understand right, things are transitioning from a system where 
we have a file format detailing the structure of messages, to a system 
where there is no documentation, no way of the client knowing what it is 
supposed to handle or not, and no idea what means what once you figure 
out the capabilities you are supposed to handle
LL: we can't log every change, it would be cluttered and irrelevant
LL: the source code is very clear on how all of this works
Others: the viewer isn't documentation, it's GPL licensed source code 
that is neither a preferred medium for documenting a protocol nor a 
license-compatible method of providing docs
LL: it is very clear documentation
Me: so we can look through the SL viewer source code to learn how things 
work and reimplement it in BSD licensed libsecondlife correct?
[13:31]  Rob Linden: Eddy, as long as you don't steal code, yes
LL: so this whole discussion about documenting the capability API is 
just so people can steal code?
Others: no, we don't want your source code. please keep it
Me: we would like documentation to write independent implementations though
Others: independent implementations... part of that whole architecture 
working group thing


John Hurliman