[sldev] Body motion and facial expression tracking, Microsoft did it

[Tigro Spottystripes]

oh, wow, they really tricked me with those vids :/

so the interactions on those videos were all faked then? that sucks :(

do you know of anywhere I can see true proof of concepts/working
prototypes of that system or similar? (I mean, besides that video of the
"segway control system" for SL)

Philippe Bossut (Merov Linden) escreveu:
> Hi,
>
> As Philip said, this is the 3DV camera (MS acquired 3DV end of 2008)  
> and most of those demos have been seen before at Game Developer  
> Conferences and others since a couple of years now. The best one I've  
> seen (and experimented myself) was made by Softkinetic (http://www.softkinetic.net/ 
> ). Awesome feeling! But that needed pretty careful setting and  
> calibration. Also note that the videos by MS are *not* demos (i.e.  
> live effective code) but staged shooting.
>
> One big issue so far with those systems (especially the 3DV camera  
> based on TOF technology, something similar to Lidar) is the ambient  
> noise you get in a real life busy setting. With Lidar in particular  
> the presence of a reflective surface in the field of view screws up  
> the reading big time. If your room is a little crowded, getting the  
> background out is not always easy and mixing several people gets you  
> into interesting tracking problems...
>
> Plus, one very interesting and SL specific problem is "rigging": if  
> you want the movements to be representing yourself (and not just  
> knocking off stuff on screen), meaningful rigging is pretty hard as  
> you need to track every joint (relying on FP/IK for untracked joint  
> like in puppeteering gives weird results... I tried...) and you need  
> to remap positions to keep the respective positions of end points  
> correct (just using the quaternions for each joint for instance won't  
> guarantee that your hands will end up on your head if you take that  
> position in front of the camera, as small bone length differences  
> between your skeleton and the avatar's skeleton will translate in  
> different end positions). Keeping an expressive avatar is actually  
> harder than just using your body as a joystick in a game (which  
> expressivity is limited). This is a challenge they won't even try to  
> stand up for I think (irrelevant for games) but that's very important  
> for SL (assuming we want full body tracking one day).
>
> Well, all that to say that, no, "they" haven't beaten us to it yet.
>
> Cheers,
> - Merov
>
> On Jun 1, 2009, at 8:58 PM, Tigro Spottystripes wrote:
>
>   
>> my main use for it probably would be puppetering of the avatar, at  
>> least
>> above waist since I don't like standing up all that much, but I  
>> imagine
>> this can be expanded to many uses, like the example on one of the  
>> vids,
>> driving vehicles without any special controlers, and with a bit of
>> cooperation of the coders, this could even help to  provide in-world
>> animation RECORDING, you would be able to turn your own room into a
>> mocap studio with just SL and a little gadget, and you would be able  
>> to
>> get live preview on the very avatar you intend to use the anim on (in
>> the cases where a anim is meant for a specific avatar isntead of  
>> being a
>> generic anim)
>>
>>
>> I hope when that thing hits the stores it will cost at most about as
>> much as a Wiimote and got a easy way to interface with regular PCs
>> (isn't the XBox basicly a PC with some purpose built hardware and it's
>> own unique OS partially inspired by Windows and the DirectX tech?)
>>
>> Maya Remblai escreveu:
>>     
>>> Eh. Yeah it's cool, yeah it's great for games made to use it. Me,
>>> though? Ain't no way I could play Phantasy Star Universe, or Katamari
>>> Damacy, or MOTHER3 with that. Besides, I'd collapse from exhaustion
>>> after about ten seconds anyway. So I don't see the relevance to SL,  
>>> I'd
>>> never use it even if I could afford it. How would it handle flying
>>> anyway? ;)
>>>
>>> Also:
>>>
>>> Thomas Grimshaw wrote:
>>>
>>>       
>>>> But they did beat you to it...
>>>>
>>>> What's $100 when you're charging $200 a month for a quarter of a  
>>>> server?
>>>>
>>>> ~T
>>>>
>>>>
>>>>         
>>> Ooh, burn. ;P
>>>
>>> Maya
>>> _______________________________________________
>>> Policies and (un)subscribe information available here:
>>> http://wiki.secondlife.com/wiki/SLDev
>>> Please read the policies before posting to keep unmoderated posting  
>>> privileges
>>>
>>>
>>>       
>> _______________________________________________
>> Policies and (un)subscribe information available here:
>> http://wiki.secondlife.com/wiki/SLDev
>> Please read the policies before posting to keep unmoderated posting  
>> privileges
>>     
>
> _______________________________________________
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/SLDev
> Please read the policies before posting to keep unmoderated posting privileges
>
>