[sldev] Generic LSL->Client Comms

[Argent Stonecutter]

On Feb 28, 2007, at 6:38 PM, Dale Glass wrote:
> I don't really get why everybody keeps bringing up context switches  
> and
> network stack overhead, when both are constant, not all that big,  
> and getting
> progressively smaller as CPU speeds increase.

Because they're constant, quite big, and getting relatively larger as  
CPU speeds increase. A context switch to another process, whether it  
goes through the network stack or not, at a minimum involves  
switching (if not actually reloading) the page tables, and doesn't do  
any good to your L1 and L2 cache, let alone your pipeline and  
renaming registers - particularly on CPUs like the x86.

For plugins where latency or overhead is not an issue, like command  
and control, sockets are a fine mechanism. There's a lot you can do  
with them. But you don't want to be following linked lists, polling  
internal structures every frame, or tweaking texture maps in real  
time over a socket... the code that does that has to be inside the  
viewer's own address space.

> I think that a much better argument could be made that there's  
> going to be a
> significant impact from converting internal data into some format  
> like XML,
> then parsing it again on the same box, when all that data could be  
> accessed
> for free if the plugin was loaded into the client.

Well, that's not a "performance impact" as much as a design one. You  
have to make sure that you're not effectively building the plugin  
into the viewer (so why have the plugin?) and that you're not putting  
the plugin in a critical path (like, mesh generation). So that comes  
down to "where are the good places to factor the problem", and it  
seems to me that chat is one of them.

> Anybody working
> on something like this can be assumed to have seen at least some  
> HTML, and if
> you get that, you're 90% there regarding XML.

I was a system administrator supporting a large team of developers  
for two decades, and... trust me... it's amazing how many of them  
have no clue about HTML and whose idea of generated HTML or XML is  
completely loco.

And the folks writing LSL scripts who would be taking advantage of  
this are not even that sophisticated. Really.

> What is the "," for?  There you have a weird bit of syntax already.

The "," is for "I'm tossing ideas out on the fly". We could pick  
space, or "/", or ":", or anything else, or even make it optional if  
the message starts with a non-numeric non-space character.

> Will the parser handle a negative channel?

I don't see why not, atoi() does, and it's good enough for indra.l.  
See sample parser below.

> How do you indicate shout or whisper?

Then you use XML.

> Will the parser correctly deal with unicode? Specifically, will it  
> correctly
> detect characters it's looking for, while avoiding the pitfall of  
> that a
> character can be both itself and a part of a multibyte UTF8 character?

No character in the range 00-7F can be part of a multibyte UTF-8  
character, and the only character's I'm looking at in the sample code  
are '@', '#', '<', and ','.

> What if you want to send multiple lines?

Then you use XML.

> I have a quite strong dislike for homebrew protocols for this  
> reason. Are you
> sure that a couple months later you won't have to keep adding weird  
> hacks to
> keep this compatible?

If you limit the scope of the simple case than it's not a problem.

> Such a thing would need to be properly made to deal with eventual  
> expansions.

That's what the XML is for.

> I think PNG has a good idea in this regard: You can add new  
> sections to it,
> and the section's header specifies whether handling it is necessary  
> or not in
> other to display the right result.

I was in the PNG design mailing list. :)

>> Say "throttle up" on channel 32. All multi-line XML messages are
>> preceded by "#" followed by the number of bytes in the message and a
>> newline.
> And now you have the problem of having to implement the same thing  
> twice.

Nah... you'd doit something like this...

// read one command from the socket, send response, and return  
whether the socket should remain open
int handle_command(int sock)
{
   integer status = ERROR_INVALID_LINE;
   char *code = NULL;
   char *line = readline(sock);

   // end of file, the other guy's closed the socket
   if(!line)
     return 0;

   // simple case, look for "@channel,message"
   if(line[0] == '@') {
     int chan;
     char *msg;

     // skip '@'
     line++;
     // and leading space
     while(isspace(line[0])) line++;

     // 'parse' (ha ha) line
     chan = atoi(line);
     msg = strchr(line, ',');

     // override default error status if there's a sane channel &  
message
     if(msg++ && (line[0] == '0' || chan))
       status = chat(chan,msg);
   }

   // single line of xml
   else if(line[0] == '<') code = line;

   // counted lines of xml - simple code, no check for trailing garbage,
   // but an invalid number, or an attempt to send zero (or less)  
bytes of xml, is an error
   else if(line[0] == '#') {
     int len = atoi(&line[1]);
     if(len>1) code = readbytes(sock,len);
   }

   // If there's code, handle the complex case and change the status  
from the default
   if(code) status = handle_xml(code);

   // let the guy know whether it's OK.
   putstatus(status);
   return 1; // socket still open.
}

> That
> almost guarantees that one version will have a bug and another  
> won't, or the
> functionality will slightly differ. That's bad.

See above code. It's C, and written off the top of my head, since the  
comparable code I use is proprietary and in Tcl. I've desk-checked  
it, it should compile even. :)

Things like readbytes, readline, and chat have pretty obvious semantics.