[sldev] Body motion and facial expression tracking, Microsoft did it

[Dahlia Trimble]

Impressive video.. I liked the part where he grabs the beer.
Most of my experience with IK animation is using tools such as 3DS Max and
blender. I have done a small bit of procedural animation in SL, here are a
few examples:

http://www.youtube.com/watch?v=VqdeEIa_Ehs

http://vimeo.com/946012

all calculations are done in real time in both of those videos. And yes,
those are simulated joints ;)

There is a crude form of synchronization of animations in SL now, it's
usually used in "couple animating poseballs". It assumes that animation
sequences have been cached by both viewers and the animations will both
start within a close tolerance. There is no start time designated in the
signal, the viewers respond when the start message is received so any delays
will reduce the quality of the synchronization.


<http://www.youtube.com/watch?v=VqdeEIa_Ehs>

On Sat, Jun 6, 2009 at 9:15 PM, Jan Ciger <jan.ciger at gmail.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Dahlia Trimble wrote:
> > I dont think the math or the complexity of IK is beyond all of the
> > scripters in SL, There are some talented scripters around who could
> > easily solve the problem, some of them in SL may even be PhD level
> > mathematicians ;)
>
> Right :) I didn't say there aren't any, but that they are tiny minority.
> I do happen to have that PhD, but most people I know that are scripting
> (and even very good scripters) don't. Of course, that doesn't mean
> anything, but I somehow do not think that an average SL creator will be
> able to use it meaningfully.
>
> > Rather I think synchronization and lag would prevent a
> > lot of real time on-the-fly animations.
>
> There is no synchronization at the moment. That is one issue I have
> described on my reply to Argent. If we had that, I think we won't need
> IK for the most part. It would be interesting to have it, but 99% of
> things people would want it for could be done by pre-baked but
> synchronized animations.
>
> > I don't think that computing IK on the fly is all that expensive either,
> > it may only be a few hundred floating point operations per keyframe for
> > each skeleton, certainly not expensive for modern day hardware.
>
> Eh, did you try? Do you realize that you are dealing with heavily
> underconstrained problem that even with 15 joints (that is what SL
> avatars have, if I recall right) will generate lots of solutions and you
> need to weed bad ones out somehow. That takes time and computational power.
>
> We were doing semi-real-time IK on decent machines with 70+ joints
> (HAnim skeleton), but you could definitely feel the impact (character
> "thinking" for a few seconds every time IK was triggered). And that was
> only *one* guy. Now imagine a sim with 5 couples dancing (e.g.
> ballroom), some people hugging or whatever - and you are calculating IK
> for every one of them suddenly. The impact would be terrible.
>
> You can see an example of the performance here:
> http://vrlab.epfl.ch/Movies/Movies_index.html
>
> Look at the "Counting Beers" video. The characters look ugly even
> compared to SL, but that was back in 2005 and the design wasn't the
> point of the video. The animations are all procedurally generated except
> the few guys dancing and one guy drinking the beer. The walking is
> procedural, the grasping is IK + special grasping code of my colleague.
> There you can see clearly also the delays due to the IK solver - e.g.
> when the two guys are exchanging the beer. And that was on a single,
> fairly decent machine and heavily tweaked. Moreover, if I recall
> correctly, it wasn't even using all the joints - I think we have
> constrained it to upper body only for performance reasons. It won't be
> too much faster today, because the issue are the algorithms, not the raw
> number crunching power.
>
> The website also shows work on motion re-targeting and adaptation using
> IK which is state of the art. That should give you an idea what is
> doable and at what cost - but many of the IK videos are/were produced
> offline in Maya (calculated using the lab's code and rendered in Maya).
> It would have been way too slow in real time.
>
>
> > I *do* think that if real time on-the-fly animations were to be
> > implemented in any kind of believable sense, there would need to be an
> > improvement made in the speed in which the data could be distributed
> > between clients. Somehow I don't see it as possible with the current
> > load that many SL sims have to deal with already.
>
> Yes, that is the synchronization issue. That would need to be sorted out
> first, before any attempts at things like this.
>
> Regards,
>
> Jan
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org
>
> iD8DBQFKKz7tn11XseNj94gRAp7BAJ99kNbZev+b6+BteuwVW5SDMEot8ACgvVJJ
> 5M6MycOEFfj5oHPB2SiVDDs=
> =Mekj
> -----END PGP SIGNATURE-----
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/sldev/attachments/20090606/267bfd8c/attachment.htm