[opensource-dev] Client Plugin System Design

[Morgaine]

Ricky, I'm not sure even where to begin, because your proposed solution is
simply direct embedding of one or more scripting language systems into the
address space of the viewer.  This ignores every single advantage of
process-separated plugins and replaces them with a catalog of problems that
could fill a book.

Direct embedding is what everyone does when they want a simplistic
sequential scripting facility for configuring or testing a host
application.  We haven't proposed it ourselves not because we've had a brain
lapse and can't see the simplest solution, but because the simplest solution
is a complete disaster when misapplied as a general scripting facility for
user-written extensions.  I regularly embed Lua into host programs your way,
but using that approach here would have so many disadvantages and so few
advantages that I have avoided it intentionally.

I'll try to summarize some key issues for you, although this will leave much
unsaid.  Let's call the conventional approach Direct Language Embedding, or
*DLE* for brevity.


   - Unlike process-based plugins, DLE normally provides no concurrent
   execution.  This means that if you want your plugins to execute concurrently
   you will have to implement either multitasking or threading in the viewer,
   both of which involve shared-state multiprogramming.


   - Shared-state multiprogramming is hard and error-prone because of race
   conditions, non-determinacy, atomicity and locking, transactional integrity,
   as well as deadlock, livelock, and various other highly unfriendly
   critters.  I'm not going to justify this further here as it would take too
   long, but I did my research in this area, and anyone who thinks it's not
   hard needs to study the topic another 10 years.  A reliable solution
   requires operating system quality design, so this job really needs to be
   left to the operating system,


   - Because shared-state multiprogramming is hard, the execution engine for
   client-side scripts would be large, complex, and very hard to get right.  It
   would be literally a multi-year effort to make it work correctly, and it
   would always be hanging by a thread because full-coverage testing is very
   difficult owing to state complexity and timing dependencies.


   - In contrast to DLE, process-based plugins would not require any
   multiprogramming in the viewer at all.  The C++ host side can be completely
   sequential in its processing of messages arriving from plugins, while the
   plugins themselves would run with full concurrency courtesy of the operating
   system.  This makes for simple, reliable, and determinist viewer code.


   - DLE would add the entire code and data space of each embedded language
   runtime into the viewer's address space.  This is bloat with a capital 'B',
   and because virtually every language has a different embedding mechanism
   into the hosting C++ code, there will be very little code reuse.
   Application stability is inversely proportional to application size, so
   bloating the viewer is asking for trouble, not to mention large maintenance
   costs.


   - Because of the preceding issue, in practice DLE will never allow more
   than one or two different language runtimes to be integrated into the
   viewer, and hence a language-agnostic approach to client-side scripting is
   effectively barred.


   - Without language agnosticism, a small proportion of users will be
   favored by the chosen language or runtime while a larger proportion will be
   disadvantaged and their primary skills will not be harnessed.  This is a
   waste of people's skills, time and productivity.


   - Without language agnosticism, you are stuck with the facilities
   provided by the chosen languages or runtime.  If another language has a
   great feature that is highly appropriate for a particular plugin task, too
   bad, you cannot use it.  This is not very empowering, and it's an
   unnecessary restriction.


   - With DLE, the language runtimes would be directly *linked* with the
   viewer code, and hence licensing restrictions would come into play.  This
   would add a whole new rat's nest of problems and effectively make embedding
   of many useful languages impossible, whereas the license barrier provided by
   sockets avoids this issue entirely.


   - Because all plugins run in the address space of the viewer in DLE, the
   opportunity for programs to interfere with each other or with the viewer is
   large.  The viewer's stability drops to that of its least stable plugin.


   - With DLE, security is a nightmare.  Every 3rd party plugin can
   potentially alter something in the viewer unless it runs sandboxed, but if
   it runs sandboxed then it cannot provide much (or any) local interfacing, so
   the power of such plugins is weak.  Fast C/C++ plugins are almost out of the
   question for this reason.


   - Even if low inherent power is accepted so that sandboxing can provide
   security, this sandboxing can only ever be relatively weak *software
   sandboxing* because the plugin code and data lie within the address space
   of the host application.  This contrasts markedly with plugins implemented
   as system processes, which are backed by the much stronger guarantees of
   isolation provided by the system's hardware MMU, even if plugins are written
   in bare-metal C/C++.


   - Because DLE runs plugins as part of the host application, the standard
   system tools for working with processes cannot easily be used.  Even basic
   facilities such as killing a script would have to be implemented from
   scratch within the multiprogramming environment, instead of using the
   existing operating system kill command.  The same applies to profiling,
   performance monitoring, debugging, optimizing, and so on --- all are much
   more difficult within the host application, or not even viable.  (An example
   of this problem is provided by SL's Mono subsystem, which *still* has no
   means of unloading sim-side Mono assemblies after script termination, simply
   because the unload facility was never written.)


   - DLE requires that user programming of plugins be done in an environment
   of special calls to the host application so that plugins can cooperate
   properly with the host's code, and this has to be done very carefully to
   avoid damaging the host.  It is very distant from the simplicity and safety
   of programming external processes, which can use stock textbook styles and
   the normal libraries for the language.  DLE really needs client-side plugin
   programmers to be relative experts, while process-based plugins are safe
   enough for a much larger and less expert audience to program.


This list of negatives could easily grow a lot longer, but to bring it to a
close I'll just mention that DLE has some advantages too, but they are
remarkably few in number.

The most important one is that the interface to viewer facilities can be
very thin and hence fast.  For example, if the purpose of a plugin is to
render fast GL graphics then DLE is a good approach, albeit for single
plugins only.  In addition, as long as no concurrent execution is required,
implementing DLE can be done in under a day (at least for Lua), plus a few
weeks for defining API functions.  It's really that simple.

However, the vast majority of plugin applications do not require bare metal
speed, and what's more, concurrent execution of plugins is a mandatory
requirement.  This makes the tradeoff of pros versus cons weigh massively
against the DLE approach for our application.


Morgaine.


PS. With regards to "Networking code in every plugin just to connect to the
client", networking is made available by the operating system to every
process through system calls or system subroutines, ie. the thinnest
interface possible.  There is no bloat or overhead involved.  Particular
languages sometimes pretty up the system interface a little, but these
bindings do not normally introduce any significant overhead. Throughput and
latency of socket communications is not a significant issue either --- I've
measured them in an environment which emulated this design pattern, and the
level of performance might even suit some rendering tasks.






======================================

On Mon, Mar 8, 2010 at 2:19 AM, Ricky <kf6kjg at gmail.com> wrote:

> So far, barring any LL concepts, we have (as far as I know so far!) two
> designs of plugin system:
> 1: Socket-based plugins - as suggested by Morgaine.
> 2: D-Bus or similar existing IPC tool.
> 3: C++ Dynamically Shared Objects - my suggestion.
>
> Morgaine's design has a couple advantages that I can think of: Namely, a
> distinct lawyer-approved separation of code allowing the plugins to be
> released under different licenses, and a fairly stable target to design
> around, meaning the socket API wouldn't typically change a lot across
> releases, plugins also could be built into other external packages that
> interface with the socket API to provide massive amounts of integration.
> However, it's disadvantages come along: Networking code in every plugin
> just to connect to the client, separate processes that have to be launched
> by something - presumably the client code, but not always...
>
> I'm not too sure about the D-Bus design, there hasn't been much discussion,
> other than it seems a lot like Morgaine's suggestion to me, just an
> available existent re-usable tool.  That can be a good thing, as we wouldn't
> have to worry about the protocol, it would be already done.
>
> The last option is one I've seen references to across the 'net.  All that
> would be written in the client software is one or more Interface Objects
> (those familiar with polymorphic design are very familiar with these,) and
> then a list or two of these for different tasks.  For instance, one list
> could be for those that have registered themselves as Dynamic Script
> interpreters.  These Interfaces would then be implemented by all plugins,
> depending on what tasks the plugin was spec'd to handle.  Plugins would then
> be located in the plugins directory in the main install, and/or in a plugins
> directory under each profile.  They would be loaded automatically upon
> startup of the client, and their functions would be called upon event.  The
> event could be as simple as the last step in the render/update loops, or as
> detailed as when the user clicks or touches a keyboard button.  They also
> are given access to various commands they can call in the client.  The more
> the merrier.
>   The advantages are that these marry right up to the client giving
> full-speed access, along with the ability to get into the nitty-gritty of
> the client's underpinnings.  Also an easier way to code for a lot of
> programmers (like myself) who, while having a passing familiarity with
> networking code, are not terribly interested in writing plugins as clients
> to the client....
> Disadvantages are a potential lock-in to the client's license, although I'm
> not sure about that, and the API could be easily changed while patching some
> other part of the client code.
>
> Each has it's pro's and con's.  And my listing of such is by no means
> comprehensive!  I want to see this hashed out and the best over-all design
> get implemented: Client capabilities would soar fast, and be done in a very
> modular way improving SL massively.
>
> Ricky
> Cron Stardust
>
> _______________________________________________
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/opensource-dev/attachments/20100308/e9ef401b/attachment.htm