[sldev] message_template.msg ordering and packing : attention LLfolks

[Tleiades]

> I could understand if there was a reason e.g. packing or alignment, but
> take ChatFromSimulator for example, the U8's are together in the template
> description and scattered in the encoding.  I don't know if I would
> characterize the out-of-order-ness as a bug IF the order was given by
> a formula associated with the description rather than being a subtle
> side-effect of the implementation as is the case here.
I have a hard time accepting this as a bug, I do agree that it really isn't 
elegant, and makes things more complicated than it really needs to be, 
without any gains.


> I beg to differ.  Using the ordering in the template file would ensure 
> that.
> The hashfunction/hash table depends on the implemenation of std::map and
> the order and specific entries which are PREHASHED.  The fact that it
> produces a consistent mapping on the current set of platforms is

I'd hate to see an iterator over std::map yielding different sequences, 
depending on the STL library used, IMO that would be a bug in the STL 
implementation.

> more happenstance than design.  Perhaps it was built with the intent
> of making ordering consistent, but if so, it does it in a haphazard and
> brittle way.  I can think of many good ways of determinining an ordering,
> but iterator order of std::map over pointers to hash table entries is not
> anywhere on that list.  Just sorting them against the PREHASH order would
> have been better.  Heck sort them alphabetically.
Yes, I agree, it could have been implemented differently, and the results 
would be easier to understand, but is it really a bug?

>
>> 3. the ordering is only semi-stable
>> I think this is a variation of 2
>>
>> >4. the hash function has 2 bugs in it
>> The codec is inefficient in so many ways, that I don't think the 
>> performace
>> hit on this one is significant in any way, especially since the bucket
>> issue only occurs during initialization.
>
> I don't think there is any question that a hash function which doesn't
> provide a good distribution from the *domain* to the *range* is a buggy
> hash function.  In this case there are 2 simple changes which don't effect
> the performance of the algorithm but dramatically improve the 
> distribution.
> I would chacterize their absence as bugs, and I would suggest that
> a code review would have characterized them likewise.
>
> Also, gMessageStringTable.getString() occurs in such places as
> LLMessageSystem::callHandler(), LLMessageSystem::addVector3(), etc. so
> I don't think it is fair to say that it is only used at initialization.
Yes, getting strings, but do we have any examples of adding strings past the 
point where the PREHASH_xxx are generated?

> LLSD (XML-ness) seems to be a good way to go for the lower frequency 
> messages,
> and for higher frequency messages, the message_template.msg format isn't
> that bad.  If there were fewer entries (because of pulling the lower 
> frequency
> ones out) and the format was stable, one could generate serialization and
> deserialization code from message_template.msg.  The result would be much 
> faster
> and cleaner.
My concern with this mix of sub-protocols is that it is confusing. IMO 
sooner or later the network protocol will need to be documented via some 
sort of standards body, e.g. IETF. by that time the protocol needs to be 
simplified and made more intuitive. My personal pet peewee in that respect 
is that "message_template.msg" really should have been written using ASN.1 
syntax. That way packet formatting would be relegated to the  transport 
layer, i.e. the full range of welldefined codecs could be used, including 
XER if you insist on XML format.

As for generating serialization/deserialization code, I have actually done 
that, in my implementation the message file is converted to some 46kloc. I 
haven't done any actual performace comparisons, but I believe that my 
implementation is more efficient, but cannot back that up in any way. This 
has the added benefit that changes to packets are a lot more likely to 
generate compile time errors rather than run time errors.

To sum it up, I don't think that the current LL implementation is defective, 
i.e. network packets are serialized and deserialized in a consistent manner, 
but I'd hate to try to defend the design choices made in the review process 
of writing an RFC for the protocol; and IMO if SL is to have a long lasting 
future standardizing the protocol is a must.

Tleiades
>
> Does anyone actually use the "dynamic layout" aspect of the current
> binary message_template.msg code?
>
>>
>> Tleiades
>> ps. Having said that the syntax is a little inconsistent, in line 468 the
>> words "sim -> dataserver" isn't marked as a comment. Luckily the viewer
>> parser handles this inconsistency without screwing up :-)
>>
>> _______________________________________________
>> Click here to unsubscribe or manage your list subscription:
>> /index.html
>
> -- 
> John Bradley Plevyak, PhD,   jplevyak at acm.org,   PGP KeyID: 051130BD
>