[opensource-dev] Avatar animation control using gestures and augmented reality visualization.

[Kantonen Tuomas]

Hi.

Earlier  this year I presented our work on using Snowglobe as a platform for mixed reality meetings.
This includes controlling avatars by natural gestures and to be able to view avatars as if they would
be located in the same room as the user.

I opened two "new feature" issues into SL Jira (SNOW-820, SNOW-821) to discuss if some of this work could
be included into Snowglobe. The gesture recognition part might be easiest to integrate, since it does not require
changes in the viewer source, except to be able to initialize and execute the processing thread. However, since all
this image processing requires a lot of processing, and the implementation uses 3rd party libraries, such as
OpenCV, I'm not sure what would be the method of integration that the Snowglobe community would like to
use. Augmented reality visualization needs to touch the rendering pipeline so I'd like to know how eager you
are to include this kind of work.

I have created a couple of web-pages describing how the system works:
 http://virtual.vtt.fi/virtual/proj2/multimedia/MWI/mwi.html.

Currently the system is implemented as a separate DLL + a small set of patches to initialize the DLL and transfer
calls between the viewer and the DLL. I can figure out at least two other possibilities: including all the DLL sources
(and adding all its dependencies) directly into the viewer source, or running the DLL as a separate executable using
some simple IPC interface to communicate with the viewer (not too unlike how the SLVoice works).

You can see all the sources and patches at the project web page. Since the project contains multiple different parts,
I tried to separate the patches into sensible sets. The patches are so small that I think that I could quite easily port
them to the latest viewer trunk but I'd rather not do it until I have a better idea what's the correct integration mechanism.

As a side note, we are currently working on face gestures that was proposed already on 2007
(https://jira.secondlife.com/browse/VWR-2122). If we can figure out a way to include all this client
side video processing, then this work would also be integrated.

--
- Tuomas Kantonen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/opensource-dev/attachments/20100826/21a377a4/attachment-0001.htm