[sldev] Body motion and facial expression tracking, Microsoft did it

[Argent Stonecutter]

On 2009-06-07, at 10:54, Jan Ciger wrote:
> Argent Stonecutter wrote:
>> Of course, the actual modifications would be small angles applied  
>> to the
>> joints. A few degrees at most. If the motion goes outside the range  
>> it
>> just doesn't happen.

> That would be an incredibly frustrating experience - it would work
> sometimes and sometimes not, only because the av was a centimeter
> farther than last time and the IK couldn't converge any more because  
> of
> your too tight constraints.

It would be no more frustrating than the current situation.

And when you're sitting on a barstool, or running an animation  
involving only your own body, the avatar would never be in the wrong  
place.

> You don't. However you are forgetting that once you are interpolating
> between the original pose and an IK generated pose

I didn't mention any separate "IK generated pose".

> Just imagine the following
> interpolation/blending - hand and lower arm from IK and the upper arm
> from the original keyframe. Suddenly your elbow is stretched and
> twisted, because the upper arm from the keyframe is completely  
> elsewhere
> than where the IK needs it.

The IK would only be applied when the keyframe is almost but not quite  
in the right position. When the keyframe moves it out of range the IK  
cuts out. SL already does this cutting between animations that have  
lead-in and lead-out times, and sometimes it looks funky when you have  
the wrong animations, but usually it looks OK.

>> That's already an issue for multi-animation scripts. It's handled by
>> eye. Does this combination look right to me?

> What do you mean "handled by eye"?

I mean you sit there and run this animation on the legs and hips and  
apply this one that is restricted to the upper body and you look at  
the result and say "yes, that works" or "no, that looks weird".

In this case you'll poke the values into the override list, and run  
the animation, and go "OK, that looks OK", or "no, that doesn't work".

And I still think you're talking about a completely different system  
than the one I'm talking about.

>> But there's no synchronization between clients, just between  
>> avatars on
>> each client.

> There is no synchronization between any avatars neither - btw, client
> and avatar, what do you mean? As if the avatar was not only on the
> client or what ...

Right now, in second life, what I see when I look at a pair of avatars  
standing next to each other may be completely different than what you  
see. They may be as much as 30 or 40 degrees different on out two  
computers. It doesn't matter if on my computer we start hugging a few  
seconds later than we start hugging on your computer. It doesn't  
matter if I see you turn towards me on my computer, and you see me  
turn towards you on your computer. Our computers don't need to be in  
sync with each other, at all.