native Flash 3D support

[eric socolofsky]

hello FLARfolks.  i've mentioned this a few times in the past, and numerous people have asked about it...how do we use FLARToolkit with Flash Player 10's native 3D?  it's a problem that i haven't been able to solve on my own, so i'm asking the community if anyone else wants to take up the challenge.

why should you bother?

solving this will allow us to use FLARToolkit with Flash Player's native 3D.  then, any swf could easily be loaded in and manipulated by a user wielding a marker.  this would open up lots of opportunities for AR and unique marker-based interfaces that include the all possibilities of native ActionScript without the application weight or learning curve of platforms like Papervision and Away.  we'd also have the added benefit of enabling Away3DLite support, which natively uses Flash for 3D.

so.  what i know so far:

i've figured out how to convert FLARToolkit's transform matrices into Flash Matrix3Ds; you can see this in FLARGeomUtils.as.

http://transmote.com/codeshare/FLARManager/dev/src/com/transmote/flar/utils/geom/FLARGeomUtils.as

(i'm pretty sure that FLAR_Z_OFFSET is not necessary, so feel free to ignore that.)

however, the projection from 3D to 2D space is not correct.  other 3D platforms (like Papervision and Away3D) use camera objects (in most AS3 frameworks, called Camera3D) to apply perspectival transformations to objects in faux-3D space.  this causes objects to appear larger at lower Z values and smaller at higher Z values, and causes perspectival distortion when closer to the 'camera', similar to that caused by wide-angle lenses.

i believe that Flash 3D uses PerspectiveProjection for this (accessible as DisplayObject.transform.perspectiveProjection).  so, the container that holds all of the 3D content needs to have its perspectiveProjection changed somehow.

FLARToolkit uses a camera parameters file (camera_para.dat out-of-the-box, i renamed it to the somewhat-more-meaningful FLARCameraParams.dat in FLARManager) to define this perspectival distortion.  in Papervision, Away, etc, these camera params are mapped to the perspectival distortion of the 3D container via a FLARCamera3D object.  (the implementation is slightly different for each 3D framework, and these can all be found within org.libspark.flartoolkit.support.)  so, in Flash 3D, these parameters must similarly be mapped to the container's perspectiveProjection object.

i just can't get the math right.  this is the sticking point -- i'm almost positive that it's the only problem left to solve to get at this functionality.

mikael emtinger, developer of the GE Smart Grid AR app, has provided some hints, but i haven't gotten a breakthrough with them yet.  he says:

* * * * * * *

What I think you need to do is extract the FOV-value from the projection matrix in the parameter-file and set it in the perspectiveProjection-property of the "root" of your Flash 3d-scene. You extract the vertical FOV from a projection matrix using:

fov = 2 * atan(1 / m[1][1])

As you can see here Flash calculates the FOV/FocalLength using stageWidth, so the formula above should work. You might need to swap and/or mirror some axis to get it right.

* * * * * * *

if anyone wants to step up to this challenge, the community would be eternally greatful!

thanks all, and enjoy your holidays (if you haven't already).

yours,

-eric

[Makc]

asking someone to step up without even sources that "almost" work is futile :)

btw, regarding fov thing, he's right, and that's what I did with
alternativa, where I could not set camera matrix directly.