[opensource-dev] Client-side scripting in Snowglobe

Morgaine morgaine.dinova at googlemail.com
Fri Feb 19 12:05:33 PST 2010

RLVa is a very notable user-defined API that has far transcended its
original purpose, and is now in extensive use wherever enhancements for
accessibility are required.  It really highlights how user-defined
functionality should have been in the viewer from the beginning.

Client-side scripting will undoubtedly take accessibility to the next level,
and I would fully expect RLV itself to become a common binding for many
client-side scripting languages.

Accessibility provides many good examples to illustrate how client-side
scripting would work over a socket-based API in the viewer.  Let's look at a
simple example that is of huge interest and benefit not only to the visually
impaired but to anyone who doesn't particularly enjoy reading scrolling


1) The user either executes the talker script manually (just like any
conventional program), or else the script launches automatically on viewer
startup, or else the script is launched from the viewer's script manager if
there is one.  If there are special accessibility issues to consider, this
step can be complicated, and tailored to the individual needs of the user.

2) The script immediately connects to a predefined port on the same machine
as the viewer, on which the viewer is listening for incoming script
connections.   The viewer's connection listener accepts the connection and
sets up the appropriate TCP channel and messaging command interface for
communicating with the script.

3) The script sends the viewer a message to register a callback for the
NewCommunicateTabOpened event, in order to be notified of new channels of
communication appearing in the viewer.

4) The viewer registers the callback for this script.  (Note that any number
of attached scripts could register interest in the same callback.)

5) Someone starts a personal IM to this user or sends an IM to one of the
user's groups, and a new Communications tab pops up in the viewer.  Since
the script has registered for the corresponding event, a message of the
appropriate type and naming the channel that has just appeared is sent by
the viewer to the script through the socket.

6) The script responds to the incoming event, and sends back a message to
register a callback for the ChatMessageReceived event, parametrized with N,
the number of old chat messages desired (1, if only the message that opened
the chat tab is wanted).

7) The viewer registers the callback and sends the requested N chat lines,
one per message of the appropriate type, parametrized with the channel
identifier and carrying the chat line as payload.  The types could be
different for personal and group IM, vicinity chat, system messages, etc.

8) The script receives each incoming chat event message and sends it to the
local text-to-speech program on the user's machine, which could be a
commercial package or an open source one such as Festival.  Note that the
script could apply transformations to each line to increase listenability,
including assigning gender or tiny voices as desired, or filtering out
undesired types of message such as graphic spam.

9) When no more chat talking is desired, the script can simply be killed.
The viewer will note the TCP socket closing and will remove all registered
callbacks for this script and tidy up appropriately.


It is very important to note that client-side scripts written for
accessibility purposes are very often interfaces to local resources or
applications, and cannot be sandboxed, since those resources or applications
would not be accessible from within a strong-walled sandbox.

This is why a sandboxed CLR environment is not enough for a generic
client-side scripting system --- it doesn't meet a very large set of crucial
use cases.  Both sandboxed and not sandboxed script environments are
required, to cater for untrusted and trusted scripts respectively.



On Fri, Feb 19, 2010 at 3:47 PM, k\o\w <kow at sapinski.com> wrote:

>  RLVa, supports something like this, and can be found in most 3rd party
> viewers:
> http://rlva.catznip.com/blog/
> http://wiki.secondlife.com/wiki/RestrainedLifeAPI
> On 2/19/2010 10:38 AM, Morgaine wrote:
> Not forgetting Erlang, Ruby, LISP, Javascript, and Bourne shell of course.
> :-)
> But here's the fun one, for some value of "fun" ...  Someone would
> undoubtedly write an LSL binding to the socket-based API too.  And however
> much we screw up our noses at LSL, I have no doubt that a large number of SL
> users who know no other language would be very happy to see it. :-)
> Providing a socket-based interface to the viewer would be a hugely
> all-embracing approach to client-side scripting, supporting everyone's
> needs.  I think it deserves consideration.
> Morgaine.
> ===================================
> On Fri, Feb 19, 2010 at 3:23 PM, Morgaine <morgaine.dinova at googlemail.com>wrote:
>> Indeed Rob!
>> Lua is a brilliant language for adding scripting to existing applications
>> --- it's designed expressly for embedding and extending, it has a clean
>> syntax, it is linguistically very powerful, it is very tiny (the whole thing
>> can add under 100KB to your application), it can run sandboxed or not as
>> desired, and it is one of the fastest pure scripting languages currently
>> available, a lot faster than say Python.
>> It is no surprise then that game developers worldwide flocked to it in
>> droves, and now it's one of the most common scripting languages found
>> embedded in games.  WoW fans use it daily as an intrinsic part of their WoW
>> client, and a huge community has grown up around Lua-powered interfaces for
>> that game.
>> So yes, I'm with you on the importance of Lua for client-side scripting of
>> the viewer.
>> However, advocating Lua as the sole means of scripting viewers would be
>> just as bad a mistake as advocating C# or CLR-targetted languages only.  It
>> would support only one part of the scripting community directly, while
>> discriminating against everyone else.  This is not necessary.
>> Instead, defining a socket-based API interface would allow effectively any
>> language to be used for scripting the viewer, since virtually all languages
>> today have socket capabilities.  That would certainly include Lua and C# and
>> Python and Perl and Java and Clojure and C/C++ and ObjectiveC and Smalltalk,
>> to name a few languages that this community uses regularly.
>> The only thing that we would have to agree on would be the set of messages
>> that cross the socket interface, and the set of functions and callbacks that
>> the messages would engage in the viewer.  That's the kind of productive
>> technical discussion we could be having with Lindens, if their design
>> process for client-side scripting were open.  It needs to be.
>> Morgaine.
>> ===========================
>> On Fri, Feb 19, 2010 at 5:24 AM, Rob Nelson <nexisentertainment at gmail.com
>> > wrote:
>>> B-B-But what about Lua, which has already been implemented in FlexLife
>>> (http://flexlife.nexisonline.net)? :(
>>> Fred Rookstown
>>> Lead Developer
>>> On Thu, 2010-02-18 at 12:42 +0000, Morgaine wrote:
>>> > I referred recently to Linden's internal project "Firefly" to add
>>> > client-side scripting to SL viewers.  This has been the topic of open
>>> > discussion at several Office Hours with Lindens in SL, but that
>>> > openness has not extended to many design details --- the Firefly
>>> > design process is not open to the community.  The only technical
>>> > details that are being disclosed about Firefly appear to be:
>>> >
>>> >       * "Scripts" are actually Mono assemblies, so that only languages
>>> >         that compile to Mono will be allowed.
>>> >       * The programs run in a sandbox, which means that most platform
>>> >         resources are not accessible to them.
>>> >
>>> > Please note that I quite like C# as a language, but the following
>>> > remarks are about Mono use in the SL viewer, only, where its tradeoffs
>>> > are poor.
>>> >
>>> > The first known detail about Firefly (mandatory Mono) is problematic
>>> > on several fronts:
>>>  >      1. Only a tiny fraction of the world's applications, libraries
>>> >         and languages work on Mono, so client-side scripting will be
>>> >         unable to benefit from the huge mountain of resources
>>> >         available on the Internet.  This is an extremely severe
>>> >         limitation, and an unnecessary restriction in the context of
>>> >         client-side viewer scripting.  If I want to use a
>>> >         locally-installed package X from within my client-side script,
>>> >         I should be able to.  What runs client-side should always be
>>> >         our individual choice, not someone else's.
>>>  >      2. Programmers want to write client-side scripts in the language
>>> >         that they know best, because that always yields the fastest
>>> >         progress and highest quality results.  There was a good
>>> >         technical reason for forcing everyone to use LSL server-side,
>>> >         but there is no technical reason to impose this requirement on
>>> >         all client-side scripting.  It is counter-productive to force
>>> >         CLR compatibility on client-side script developers when there
>>> >         is a simple alternative:  define a socket-based viewer API for
>>> >         client-side scripts instead, hence usable from any language.
>>>  >      3. Mono runs poorly on Linux, so from being rock-solid on Linux
>>> >         now, the LL-derived viewers will become second-rate on this
>>> >         platform.
>>>  >      4. The viewer is already extremely bloated and a memory hog.
>>> >          Adding a Mono dependency will compound that horribly.
>>>  >      5. There is only one effective supplier of Mono:  Novell.  That
>>> >         is a very bad situation to encourage and to support in the
>>> >         viewer.
>>>  >      6. Some parties identify other reasons for avoiding Mono in
>>> >         general.  Without getting into that subject at all,
>>> >
>>> > The second known detail about Firefly (mandatory sandbox) is
>>> > problematic on two related fronts:
>>>  >      1. Sandboxes by design do not allow most platform resources to be
>>> >         accessed, as a security measure.  This is fine and important
>>> >         when scripts are being downloaded from unknown places (like
>>> >         Javascript in web pages), but that same protection also means
>>> >         that client-side scripts would be powerless to do useful
>>> >         things for us in concert with local applications, files,
>>> >         devices, etc.  Sandboxing client-side scripts effectively
>>> >         hardwires in script weakness for no reason discussed as a
>>> >         requirement.
>>>  >      2. Sandboxed applications cannot be linked with user-chosen
>>> >         native libraries since allowing native code breaks sandbox
>>> >         protection.  This means no accelerators, no extensions, and no
>>> >         interop with other systems since sockets are inaccessible from
>>> >         any strong sandbox.  This also means no evolution or progress
>>> >         outside of what the sandbox designers permit.
>>> >
>>> > This mailing list is concerned with development of open source
>>> > viewers, in particular Snowglobe.  This is heralded as a community
>>> > viewer, embodying community requirements much more directly than the
>>> > LL mainstream viewer.  Client-side scripting will impact on every
>>> > single aspect of Snowglobe bar none, yet the community is being
>>> > excluded from the design of its most powerful infrastructure element.
>>> >  This is entirely wrong, far beyond the normal observation
>>> > that secrecy in design has no place in open source.
>>> >
>>> > It is hard to assess things technically when the design requirements
>>> > are formulated in secret.  The Snowglobe community has design
>>> > requirements too.  Those deserve to be examined here openly, not
>>> > limiting Snowglobe to a design that stems from Linden requirements
>>> > alone.
>>> >
>>> >
>>> > Morgaine.
>>> >
>>>  > _______________________________________________
>>> > Policies and (un)subscribe information available here:
>>> > http://wiki.secondlife.com/wiki/OpenSource-Dev
>>> > Please read the policies before posting to keep unmoderated posting
>>> privileges
> _______________________________________________
> Policies and (un)subscribe information available here:http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting privileges
> _______________________________________________
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/opensource-dev/attachments/20100219/826fa497/attachment-0001.htm 

More information about the opensource-dev mailing list