On depth perception in (FL)ARToolKit

[Harry Underwood]

Hi, I was just thinking a few nights ago about the general lack of depth perception by most available AR software frameworks, given that they tend to augment "on" (that is, that the imagery is projected exclusively in front, making it hard or impossible to place real objects in front of the virtual imagery) reality rather than augment "in". I also read Mr. Kawasaki's post about his LiveChromaKey extension, in which he tries to augment still, flat images into the webcam's field of view and behind closer objects.

But I just saw this video, which features the Minoru 3D webcam with this framework called "sentience" to enable the computer to perceive and mark real-world objects by how close or how far they are from the webcam (in other words, "computer stereo vision". 

So I just had this idea, unless this has been discussed on this or the ARToolKit mailing list: Would it be possible to fuse (FL)ARToolKit with "sentience" or some other framework so that the computer could be enabled to dynamically plot virtual objects in a perception of depth that would allow real objects to be visibly passed in front of virtual objects?

I'm thinking that, if (FL)ARToolKit could plot the depth of both virtual-world and real-world objects and allow for real-world objects to pass in front of (and control) virtual objects, then something like ARDesktop could be made available to a very receptive niche audience: http://www.youtube.com/watch?v=V4Yj1Y3Pg20

Harry

[eric socolofsky]

interesting.  hadn't heard of the minoru or sentience, but i'm familiar with (much) more expensive stereo vision options like point grey's bumblebee.  looks like sentience uses openCV, so to use this with flash would require alchemy, and might not be very performant.

also, looks like sentience doesn't currently provide depth information for all pixels in range, but this page:

http://code.google.com/p/sentience/wiki/StereoRanging

shows how to calculate that information.

one of the nice things about the bumblebee is that it provides a depth map -- a greyscale version of the video in which objects are shaded according to their depth.  i've always wanted to work with one of these but last i heard they cost around $1800 USD (ouch).

if you experiment with this, let us know how it goes....

-eric

[eric socolofsky]

btw...that subtractive analysis example is pretty damn nice!  super easy to do this in, say, jitter, but getting those frame rates in flash, that's impressive.  go libspark :D

On Dec 14, 2009, at 11:34 PM, Harry Underwood wrote:

[Makc]

On Tue, Dec 15, 2009 at 10:51 AM, eric socolofsky <er...@transmote.com> wrote:
> i've always wanted to work with one of these but last i heard
> they cost around $1800 USD (ouch).

FLARToolKit price is about 75% of that.

[Harry Underwood]

You mean the commercial license price is 75%? The other license is the GPL, if I'm correct?

Harry

[Makc]

sure

[eric socolofsky]

there's no such thing as an open-source license for the bumblebee.

[Makc]

open-source != free

[eric socolofsky]

yeah, well, it's not supposed to, but considering the incredibly small number of FLARManager sales i've seen so far, it's pretty damn close.

in other words, it's free if you choose to ignore the license.

[Harry Underwood]

Since sentience is written in C#, I think that, if I were to do any experimentation with it, I would first use NyARToolKitCS (a C# port of (Ny)ARToolKit). If it can work with that, then I think that we can look at a possible ActionScript port of sentience (to make it work with (FL)ARToolKit).

To be honest, this may be my first free-time programming experience, since I'm currently in college classes for Java.

Harry