[sldev] Body motion and facial expression tracking, Microsoft did it

[Jan Ciger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tigro Spottystripes wrote:
> handshaking without haptic feedback and with SL's usual latency
> wouldn't work, I imagine it would be possible to make a handshake
> work in SL  by adapting the IK code used to position the hand and arm
> while selecting an object at arm's reach, and then locally on each
> client play an animation over the IK moving both avatar's hands (or
> the IK target itself) up and down (if it was done server side, or
> just on one client and communicated to the other, or each client
> animating only it's own av, there would be the risk of things not
> matching or lagging too much I guess)
> 

We were trying this back during my time at VRlab. It is pretty hopeless
without the tactile feedback. Human brain works like a servo-control
loop. Without depth/distance perception and tactile sensation that you
have reached the objective you will be only flailing around in the
space. And who has a head mounted display with a force-feedback
exoskeleton at home? To shake hands?

Even the animation is very difficult. IK doesn't work too well for
chains with many joints and few constraints - like a whole avatar body,
not to mention it is terribly slow. Arm IK is not enough, unless you
want to look as if you ate a ruler or even fail to reach the target -
arm reach is not that big and without bending your upper body the task
may be impossible. Furthermore, you are trying to reach a moving target,
even without the usual network-induced latencies it is very tricky.

If someone is interested in this stuff, look at the animation papers
published by Baerlocher et al and Boulic et al. They have developed an
IK system for realistic full body animation - however, it is pretty
"meaty" stuff, consider yourself warned :)

Regards,

Jan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org

iD8DBQFKJtbln11XseNj94gRAoHQAKCv5Ra3l/AG78UVrJw1s88a/Dm36ACg1ld7
XjHEQ18jllbJkcDQfzlpdw4=
=VGKo
-----END PGP SIGNATURE-----