[sldev] Plugin work so far

[Soft Noel]

I wish the plugin discussions would continue without people worrying that
no code is being written. I've been working, and the discussions have been
helpful. I'll go over what I've got so far and where I'm going so people
can make comments on the broad strategy, or at least have confidence that
we're not wasting time.

Currently my work is Mac-only, though I hope to get my hands on a Windows
machine this weekend so I can do the DLL work. I'll drop a version on the
list after I get Windows and Mac both working to my satisfaction, but
before Linux. Linux should be easy once Mac is 100%. Sadly, my only Debian
install within viewing distance is on a G4 mini, so it's useless here. :(

I'm not averse to purging or rewriting big chunks of this after I drop my
first release if it does things we don't want, or if others suggest better
ways of doing things.


Here's what a very basic plugin looks like. It's (on mac) a dylib linked
against the static lib libPluginBase:


### start HelloWorld.cpp ###

#include "linden_plugin_common.h"

// Find extra interface types in llpluginterfaceshared.h
PlugInterfaceType interfaces_requested[] =
{
	PI_TYPE_IO,

	PI_TYPE_END
};

// plugin name, your plugin version, SL name, plugin homepage, plugin
flags, interfaces requested
PLUGIN_DECLARE( "Hello Printer", "0.01", "Soft Noel", "http://foo.org/",
PLUGIN_FLAGS_DEFAULT, interfaces_requested );


bool PluginStartup()
{
	pIO->LogPrint( "Hello world." );

	return true;
}


void PluginShutdown( bool crash )
{
	if( !crash )
		pIO->LogPrint( "Goodbye world." );
}

### end HelloWorld.cpp ###

These are the interesting things here:

interfaces_requested is a taglist of enums representing interfaces we
need. These would be things like IO, UI, SYSTEM. END terminates the list.

PLUGIN_DECLARE tells the library that "Hello Printer" is the name of this
plugin, "0.01" is our version number, "Soft Noel" is the author, and
"http://foo.org" is where the user can go browse for updates. All of the
information so far will (later) be displayed in a floater in an SL plugin
manager. For now, I hard-load plugins unconditionally. The flags will be
used for identifying plugins that should default to being disabled in
certain circumstances, for example when an author wants to send out a
plugin he prefers to be used on a beta grid for now.

Everything in PLUGIN_DECLARE gets turned into an accessor function so that
the viewer can inspect a plugin's requirements before deciding whether to
activate it. In addition, PLUGIN_DECLARE creates an accessor with the
version of libPluginBase in use, which is used in negotiating an interface
bundle. (More on that in a bit.)

PluginStartup gets called when a plugin is activated. By this point, all
interfaces have already been acquired. No plugin developer code is ever
called before this point.

PluginShutdown is called when a plugin is terminated, either politely by
(later) the plugin management floater or viewer exit, or rudely by crash
protection. Ideally, interface functions will scrutinize input for
dangerous data/behaviors, print a warning to the log, and terminate a
plugin before it has a chance to crash the viewer. I'm thinking more about
plugin developer iteration time here than I am about end users. I'm
adamant about quick iteration times for casual developers, or they lose
interest fast, so if we can unload/deactivate (Mac doesn't unload dylibs,
just abandons them!) their plugin without booting them from the viewer and
sit ready to load their next plugin build, it's a big win.

Note that I'm far from married to the pINTERFACENAME->Method() calling
convention in the example above. Suggestions about what methods and
accessors should look like to plugin developers are welcome.


On to the viewer...

Currently all new viewer files reside in indra/llplug, and patches outside
this directory are minimal. A plug (as in the directory name) is the
client side of plugin. Here are the major actors and their roles:

LLPlugInterface, base of LLPlugInterfaceIO, LLPlugInterfaceUI, etc

The LLPlugInterfaceIO was that pIO we saw in the plugin. Each
LLPlugInterface derivative implements a subset of the overall available
API. Originally I was going to put it all in one glom, but after a
discussion with Rob Linden about the specialized interfaces in Real's
plugin SDK, I came to realize that parceling this out would be helpful.
The main reason is that some interfaces like a graphic interface will
deprecate more quickly than something like an IO interface. If we try to
maintain backward compatibility, it will be better if it's not an
all-or-nothing affair. (More on that in a bit.)

LLPlugInterfaces come from an LLPlugInterfaceFactory and are retained by
an LLPlugInterfaceManager. LLPlugInterfaceFactory takes a request for an
LLPlugInterface type and a version and returns a compatible interface or
declines the request if an interface version is deprecated or newer than
the viewer. Interface versioning was the source of some contentious
discussion, but it really isn't the headache it sounds like, so I'll prove
that out here:

### start llpluginterfacefactory.cpp ###

#include "linden_common.h"

#include "llplugshared.h"
#include "llpluginterfaceio.h"
#include "llpluginterfacesystem.h"
#include "llpluginterfaceui.h"

#include "llpluginterfacefactory.h"


typedef LLPlugInterface *(*IFMakerFunc)();


struct InterfaceMakerEntry
{
	const char *min_ver, *max_ver;
	PlugInterfaceType type;
	IFMakerFunc maker_func;
	bool nearly_deprecated;
};


// C++ has no type variable or name-based construction, so:
LLPlugInterface *PIFMakeIO_Head() { return new LLPlugInterfaceIO; }
LLPlugInterface *PIFMakeSystem_Head() { return new LLPlugInterfaceSystem; }
LLPlugInterface *PIFMakeUI_Head() { return new LLPlugInterfaceUI; }


static InterfaceMakerEntry InterfaceMakers[]=
{
	// Always put newest interfaces at the top. Engine uses the first fit.
	{ "2007.02.22 00", pluginEngineVersion, PI_TYPE_IO, PIFMakeIO_Head, false },
	{ "2007.02.22 00", pluginEngineVersion, PI_TYPE_SYSTEM,
PIFMakeSystem_Head, false },
	{ "2007.02.22 00", pluginEngineVersion, PI_TYPE_UI, PIFMakeUI_Head, false },

	// Old interfaces on the way out:
};


static S16 InterfaceMakerFindIndex( const char *version, PlugInterfaceType
type )
{
	for( S16 i = 0; i < sizeof(InterfaceMakers)/sizeof(*InterfaceMakers); i++ )
	{
		InterfaceMakerEntry *ime;
		ime = &InterfaceMakers[i];

		if( type != ime->type )
			continue;

		// This is not intuitive. Think of it like this: "v01Bogo", "v00Zippy" -
strcmp
		// will get to the third char and return '1'-'0' = 1, meaning the requested
		// version of v00Zippy is too old, as it's "less" than v01Bogo.
		if( strcmp( ime->min_ver, version ) > 0 )
			continue;

		if( strcmp( ime->max_ver, version ) < 0 )
			continue;

		return i;
	}

	return -1;
}


static IFMakerFunc InterfaceMakerFind( const char *version,
PlugInterfaceType type )
{
	S16 i;
	index = InterfaceMakerFindIndex( version, type );

	if( i >= 0 )
		return InterfaceMakers[i].maker_func;

	return NULL;
}


static bool InterfaceMakerIsNearlyDeprecated( const char *version,
PlugInterfaceType type )
{
	S16 index;
	index = InterfaceMakerFindIndex( version, type );

	if( index >= 0 )
		return InterfaceMakers[index].nearly_deprecated;

	return false;
}


bool LLPlugInterfaceFactory::canCreate( const char *version,
PlugInterfaceType type )
{
	IFMakerFunc ifm;

	ifm = InterfaceMakerFind( version, type );

	return ifm ? true : false;
}


LLPlugInterface *LLPlugInterfaceFactory::create( const char *version,
PlugInterfaceType type )
{
	LLPlugInterface *pi = NULL;
	IFMakerFunc ifm;

	ifm = InterfaceMakerFind( version, type );
	if( ifm )
		pi = ifm();

	if( pi )
	{
		if( InterfaceMakerIsNearlyDeprecated( version, type ) )
			pi->deprecationWarningSet();
	}

	return pi;
}

### end llpluginterfacefactory.cpp ###

The main item of interest is that InterfaceMakerEntry. For a given plug
PlugInterfaceType, it declares the earliest and latest supported version,
and tells how to make that interface. LLPlugInterfaceFactory::create
(through helpers) scans for a compatible interface and builds it. Note
that there can be multiple entries per PlugInterfaceType, so if
PlugInterfaceIO is revised, we can keep an older version of the class
around with a slightly different name and everything where older plugins
expect it to be. This won't mean a bunch of duplicated code thanks to the
PlugInterface compatibility_slave construct. (More on that in a bit.) The
deprecation flag is for (later) informing a user via his plugin management
panel that his plugin is likely to break in a coming release. These would
be set when planning to prune the InterfaceMakers/legacy LLPlugInterfaces.

The compatibility_slave construct: When freezing a version of an interface
for backward compatibility, we create a version of the interface with an
old version number, ie LLPlugInterfaceIOv1 which is derived from
LLPlugInterface (not LLPlugInterfaceIO), and copy the old list of virtual
functions into the new class. The new class would then create the next
NEWER class and stuff it in its compatibility_slave pointer, and all
functions would be direct calls to the compatibility_slave except where we
need to add/remove function arguments, or otherwise make tweaks to make
old-style calls continue to work. We can keep an arbitrary number of
compatibility interfaces without further work, as they all simply chain
upward through their compatibility slaves. This isn't done with class
inheritance as we want to be able to reorder, add, and remove items in
newer versions. We can't inherit older classes from newer classes without
affecting the old structure. We can't inherit new classes from older
classes without losing the ability to remove functions or change data
formats.

The LLPlugInterfaces are passed to the client in an LLPlugBundle. The
bundle is never seen by plugin developers, only libPluginBase. The plugin
developer just sees all those LLPlugInterfaces. LLPlugBundle comes from
LLPlugBundleManager/LLPlugBundleFactory. The LLPlugBundle is an old
fashioned struct, which contains LLPlugInterface pointers for all possible
LLPlugInterface types, ex:

struct LLPlugBundle
{
	LLPlugBundle()
	{
		memset( this, 0, sizeof( *this ) );
	}

	char *version;

    // 1. Don't re-order
	// 2. New interfaces must always be added to the BOTTOM.
	//
	// This struct may get passed to a plugin anticipating an older
	// version of the struct.

	// These are deliberately anonymous/downcast because we'll stuff
	// combinations of versions in to satisfy older interface version
	// requests.

	LLPlugInterface *plug_io;
	LLPlugInterface *plug_sys;
	LLPlugInterface *plug_ui;
};


Jumping back up a bit, we've got our LLPlugBundle with LLPlugInterfaces
getting passed to the plugin and turned into globals. Where this all
happens is in an LLPlug, via LLPlugManager/LLPlugFactory. An LLPlug is
created with the path of the corresponding plugin dylib as an argument.
Straight away, LLPlugin::load() dlopen()s the plugin with RTLD_LAZY
binding, and grabs its version and user displayable information. From this
point on, LLPlugBundleFactory can tell us whether it will be able to
create a compatible set of LLInterfaces, and we can decide if we want to
LLPlugin::enable(), which acquires the interfaces and tells libPluginBase
to set things up and call the plugin developer's PluginStartup().

Of note, each LLPlug has a unique serial number that will be attached to
and used for cleaning up orphaned resources like floaters, menu entries,
and hooks. And this brings us to hooks.

I'm in the middle of a rework on hooks right now after a discussion at Rob
Linden's office. Originally, I was having the plugin author create a class
implementing all possible hooks, just as a hack to get things working.
This isn't maintainable of course, and I'm waffling between two
approaches. One is a class like the above, but specialized for each API
subset. The other is moving to a subscription model similar to what
Bushing proposed. You can see his sample at
https://wiki.secondlife.com/wiki/Hook_example_code ... the main
differences would be that I'd want to work with templated versions based
on parameter structures to avoid anonymous data, and I'd have an instance
of the class per hookable code point rather than trying to keep it generic
like the above example.


I'll hand off the above soon, then everyone can start nitpicking about the
implementation and I'll happily iterate to make this shiny. I'm hoping
people will step in at that point to start filling out the
LLPlugInterfaces with a useful set of API calls and hooks. I want to get
Kelly's suggested data floater example going, then convert Dale's scanner
early on, so my own API calls will revolve around those. If you have a pet
project, now's a good time to start proposing APIs.

There are other useful areas for ongoing discussion if you want to
continue to help me. One that I'm coming up against when we start adding
plugin-driven floaters is the question of where we want to install plugins
and supplemental data. My thinking is something like this:

SLDATADIR = ~/Library/Application Support/SecondLife (or Windows/Linux
equivalent)
SLPLUGDIR = $SLDATADIR/plugins

SLPLUGDIR would contain all the plugin dylibs, DLLs, etc

Optional folders for data would use the the plugin's self-provided name, ex:
  SLPLUGDIR/Hello World/hellotext.xml

For simplicity, we could give plugin developers a function that opened
files with this path.

Similarly, for dynamic data, do we want functions to facilitate creating
and accessing per-user-perplugin data directories like this?
SLPLUGDATADIR = SLDATADIR/$SLNAME/$PLUGINNAME/settings.xml ?

Another coming issue is that I'd like to avoid doing too much UI work
myself. If someone's eager to create the plugin management floater xml and
related dialogs and own future changes, I'd be grateful. What the floater
should look like and what dialogs it needs would be another good area for
discussion.