[sldev] Body motion and facial expression tracking, Microsoft did it

Dahlia Trimble dahliatrimble at gmail.com
Mon Jun 8 09:24:19 PDT 2009


A few points about the Natal video that I find difficult to believe from a
technological perspective:
* the actor's arms are often occluded from the sensor by their bodies yet
the avatars react as if the arm positions are tracked

* a few of the actors are wearing flowing clothing that can occlude body
positions such as loose open sweaters

* at the very beginning of one of the videos there is some text along the
bottom left: "Product vision: actual features and functionality may vary"

Given that, I don't think a company like Microsoft would publicize
non-existent technology unless they had some proof of concept working in
their labs. 3D scene recognition via stereo cameras has been around for
quite a few years now and I think the time is ripe for consumer products
like this to come about. I somehow doubt that the first generation products
will work as well as the video suggests.

On Mon, Jun 8, 2009 at 9:08 AM, Ron Blechner <ron at involve3d.com> wrote:

> Oh, p.s.
>
> I met up with someone at the CT Film Festival this weekend who
> confirmed that the whole body interactive demos by Microsoft? STAGED.
> Entirely a mock-up demo. The technology apparently works but they're
> no where near actual product demo.
>
> -Ron / Hiro
>
> On Mon, Jun 8, 2009 at 11:31 AM, Ron Blechner<ron at involve3d.com> wrote:
> > The problem with the handshake:
> >
> > In short - it's 2 seconds worth of interaction. That's a tiny amount
> > of reward for a tremendous amount of work it would take to make it
> > better. And, there are already handshake and bow animations - they
> > don't line up with another avatar, but doing all the work to make
> > handshakes doesn't even create a new function - it just polishes an
> > existing one.
> >
> > Compare that with putting in effort to do better facial animations -
> > that would affect 100% of the time of interaction between two people.
> > And consider that facial animation programming is *client-side* by
> > nature, and doesn't require breaking apart and redoing the avatar code
> > in Second Life to the degree that handshakes / puppeteering does.
> > That's a helluva lot more results for a helluva lot less effort.
> >
> > I'd also like to add that, as Lawson pointed out, there is cultural
> > bias. According to Neilsen, there are 170 million Americans actively
> > using social media. There are twice that in China alone. China has 20%
> > of the world's population. Are we here to make a Western metaverse, or
> > a global one based on ideals not bound by one country, or one culture?
> > The reason we handshake instead of bow is because the West has
> > controlled the vast amounts of wealth in the world. Like it or not,
> > the West is spending its Wealth and a tremendous amount is going to
> > the East. In 20 years - we all may be bowing, like it or not.
> >
> > I don't want to ruffle too many features, but it's an important
> > exercise to escape one's own assumptions and biases when debating
> > functionality with a platform whose stated goal is to be "a new
> > country", as Rosedale would say.
> >
> > And to bring this back - the larger issue is that people tend to get
> > caught up in particular niceties like handshakes and miss
> > bigger-picture items. What good are handshakes if the rest of the
> > conversation is less meaningful because it lacks facial expression?
> >
> > I say, FORGET puppeteering. Forget handshakes. Let's focus our efforts
> > on something much easier to accomplish, and with far more impact on
> > Second Life. Facial expressions from video.
> >
> > *runs off to blog these thoughts*
> >
> >
> > --
> > Ron Blechner
> > Chief Technology Officer
> > Involve, Inc
> > www.involve3d.com
> > SL: Hiro Pendragon
> >
> >
> > p.s. PLUG! http://secondtense.blogspot.com
> >
> > On Fri, Jun 5, 2009 at 1:33 PM, Tateru Nino<tateru.nino at gmail.com>
> wrote:
> >> Maybe it isn't really about handshakes, and more about general
> >> lining-up-of-body-parts between avatars? :)
> >>
> >> However, for most people in first-world western cultures, a handshake is
> the
> >> frequently sole form of socially allowable physical contact between two
> >> people who aren't intimates at some level. That makes it strongly
> symbolic.
> >>
> >> For handshake you can substitute a few variations: Knuckle-bumps,
> high-fives
> >> and such, but they're all basically a handshake with different emotional
> >> flavoring.
> >>
> >> Ron Blechner wrote:
> >>
> >> Question:
> >>
> >> Why are handshakes so important that they are much more of a topic of
> >> discussion of implementation, against facial expressions?
> >>
> >> -Ron / Hiro
> >>
> >> On Fri, Jun 5, 2009 at 7:34 AM, Argent
> >> Stonecutter<secret.argent at gmail.com> wrote:
> >>
> >>
> >> On 2009-06-04, at 08:55, Jan Ciger wrote:
> >>
> >>
> >> Argent, read my comment to Tigro's mail. It wouldn't work. At least
> >> not
> >> in a nice way. For reaching and grasping you need much more IK than
> >> just
> >> the three arm joints and then you are hitting a severely
> >> under-constrained and computationally expensive problem.
> >>
> >>
> >> That's why you don't try and solve it computationally. You don't
> >> replace normal animation, you use this for minor adjustments to the
> >> existing animation, and you limit the strength of the adjustment to
> >> small angles and specific joints.
> >>
> >> So it's down to the person selecting the base animation and providing
> >> the strength and possibly range (either distance or angle).
> >>
> >>
> >>
> >> E.g. in one case I have seen the solver to keep the hands next to the
> >> avatar's waist but stick the waist forward to reach a goal.
> >>
> >>
> >> Wouldn't happen, unless the person selected the waist as the joint
> >> that would move, and unless the waist was already close to the goal.
> >>
> >>
> >>
> >> IK is a nice tool, but extremely hard to use unless you have an
> >> animator
> >> guiding it.
> >>
> >>
> >> Which is the point.
> >>
> >> _______________________________________________
> >> Policies and (un)subscribe information available here:
> >> http://wiki.secondlife.com/wiki/SLDev
> >> Please read the policies before posting to keep unmoderated posting
> >> privileges
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Tateru Nino
> >> http://dwellonit.taterunino.net/
> >>
> >>
> >
>
>
>
> --
> Ron Blechner
> Chief Technology Officer
> Involve, Inc
> www.involve3d.com
> _______________________________________________
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/SLDev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/sldev/attachments/20090608/1515e568/attachment-0001.htm 


More information about the SLDev mailing list