performance issues for ConvertProjectiveToRealWorld issues on Mac OS X

[Jacob Fenwick]

I want to draw my points as a point cloud on OS X.
I'm using ConvertProjectiveToRealWorld to change to real world coordinates.
The function is noticeably slower when I compute individual points vs when I convert an entire array of points.

For instance, this is way faster:
XnUInt32 nAllIndex = (dmd.YRes() + dmd.YOffset()) * dmd.XRes();
XnVector3D projective[nAllIndex];
int all = 0;
for (XnUInt16 y = dmd.YOffset(); y < dmd.YRes() + dmd.YOffset(); y++) {
    for (XnUInt16 x = 0; x < dmd.XRes(); x++, depth++, all++){
        projective[all].X = x;
        projective[all].Y = y;
        projective[all].Z = *depth;
    }
}
XnVector3D realworld[nAllIndex];
depth_generator.ConvertProjectiveToRealWorld(nAllIndex, projective, realworld);
all = 0;
glBegin(GL_POINTS)
for (XnUInt16 y = dmd.YOffset(); y < dmd.YRes() + dmd.YOffset(); y++) {
    for (XnUInt16 x = 0; x < dmd.XRes(); x++, depth++){
        glVertex3f(realworld[all].X, realworld[all].Y, realworld[all].Z);
    }
}
glEnd();


Than this:
glBegin(GL_POINTS)
for (XnUInt16 y = dmd.YOffset(); y < dmd.YRes() + dmd.YOffset(); y++) {
    for (XnUInt16 x = 0; x < dmd.XRes(); x++, depth++){
         XnPoint3D point = xnCreatePoint3D(x, y, *depth);
        depth_generator.ConvertProjectiveToRealWorld(1, &point, &point);
        glVertex3f(point.X, point.Y, point.Z);
    }
}
glEnd();


But in the StickFigure example, they clearly convert each point individually rather than converting an array of them.

Is this an OS X specific issue or am I just doing something wrong?

Jacob

[Dan]

We've seen significant slowness when invoking that function repeatedly
on PC as well. I'd suggest batching points as much as possible. Our
guess is that there's overhead in the function call as it goes out to
the DLL and error checks.

A version of that function as an inline would be really nice, or
perhaps something along the lines of GetProjectiveToRealWorldMatrix.
There's probably need to be special code to handle
XN_DEPTH_MAP_NO_DEPTH_VALUE, but I'd wager that inline SSE math code
with a branch around it would be much faster than calling out to the
DLLs every time.

dba

[Ixstala]

You can actually compute this trasnform yourself without OpenNI!

After some digging in the API I found the field of view for the depth
camera:


FovH=1.0144686707507438 (rad)
FovV=0.78980943449644714 (rad)


From there you can find the real world coordinates from the depth
cloud like so:


X_rw=(X_proj/Xres-.5)*Z*XtoZ
Y_rw=(0.5-Y_proj/Yres)*Z*YtoZ,
Z_rw=Z


where XtoZ=tan(FovH/2)*2, and YtoZ=tan(FovV/2)*2, and Z is the value
given by the depth map at each pixel.

I do this for my point cloud generation and it's super fast, no
bothering with the DLL at all.

> > Jacob- Hide quoted text -
>
> - Show quoted text -

[Ixstala]

Also, if you want to use my method above with the RGB information
overlaid make sure you have AltViewPointCapability of the depth
generator set to the image generator so that the depth image is mapped
onto the RGB image's perspective correctly.

> > - Show quoted text -- Hide quoted text -

[Jinhan Lee]

Hi Ixstala,

I am using PrimeSense Dev Kit. Could the FovH and FovV you found be
used for PSDevKit too?
Also, could you tell us in which file those values are defined?

Jinhan

> --
> You received this message because you are subscribed to the Google Groups "OpenNI" group.
> To post to this group, send email to openn...@googlegroups.com.
> To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.
>
>

[Dan]

Thanks for the find Ixstala. Very much appreciated.

Is that FOV hard-coded or dervied at runtime from some variable? I'd
love to write a generic version that would work for any future cameras
and forget about this.

dba

On Feb 10, 6:21 pm, Jinhan Lee <jinhan.le...@gmail.com> wrote:
> Hi Ixstala,
>
> I am using PrimeSense Dev Kit. Could the FovH and FovV you found be
> used for PSDevKit too?
> Also, could you tell us in which file those values are defined?
>
> Jinhan
>

[Jacob Fenwick]

Same here, I'm curious if it's possible to change the FOV.

Thanks a like Ixstala, this is some great detective work.

Jacob

[Eddie Cohen]

FOV can be retrieved at run-time. See DepthGenerator.GetFieldOfView().

Eddie.

[Jacob Fenwick]

Yes but can you change it?

Jacob

[Eddie Cohen]

No. The FOV is determined by the hardware.

[Ixstala]

Yep, I found those values using the getfieldofview method at runtime.
AFAIK they are hardcoded.

-Ixs

On Feb 13, 10:25 am, Eddie Cohen <Eddie.Co...@PrimeSense.com> wrote:
> No. The FOV is determined by the hardware.
>
> From: openn...@googlegroups.com [mailto:openn...@googlegroups.com] On Behalf Of Jacob Fenwick
> Sent: Sunday, February 13, 2011 20:24
> To: openn...@googlegroups.com
> Subject: Re: [OpenNI-dev] Re: performance issues for ConvertProjectiveToRealWorld issues on Mac OS X
>
> Yes but can you change it?
>
> Jacob

> On Sun, Feb 13, 2011 at 1:17 PM, Eddie Cohen <Eddie.Co...@primesense.com<mailto:Eddie.Co...@primesense.com>> wrote:
> FOV can be retrieved at run-time. See DepthGenerator.GetFieldOfView().
>
> Eddie.
>
>
>
> -----Original Message-----
> From: openn...@googlegroups.com<mailto:openn...@googlegroups.com> [mailto:openn...@googlegroups.com<mailto:openn...@googlegroups.com>] On Behalf Of Dan
> Sent: Friday, February 11, 2011 16:07
> To: OpenNI
> Subject: [OpenNI-dev] Re: performance issues for ConvertProjectiveToRealWorld issues on Mac OS X
>
> Thanks for the find Ixstala. Very much appreciated.
>
> Is that FOV hard-coded or dervied at runtime from some variable? I'd
> love to write a generic version that would work for any future cameras
> and forget about this.
>
> dba
>

> On Feb 10, 6:21 pm, Jinhan Lee <jinhan.le...@gmail.com<mailto:jinhan.le...@gmail.com>> wrote:
> > Hi Ixstala,
>
> > I am using PrimeSense Dev Kit. Could the FovH and FovV you found be
> > used for PSDevKit too?
> > Also, could you tell us in which file those values are defined?
>
> > Jinhan
>

> > On Thu, Feb 10, 2011 at 6:02 PM, Ixstala <bdellon9...@gmail.com<mailto:bdellon9...@gmail.com>> wrote:
> > > Also, if you want to use my method above with the RGB information
> > > overlaid make sure you have AltViewPointCapability of the depth
> > > generator set to the image generator so that the depth image is mapped
> > > onto the RGB image's perspective correctly.
>

> > > On Feb 10, 2:59 pm, Ixstala <bdellon9...@gmail.com<mailto:bdellon9...@gmail.com>> wrote:
> > >> You can actually compute this trasnform yourself without OpenNI!
>
> > >> After some digging in the API I found the field of view for the depth
> > >> camera:
>
> > >> FovH=1.0144686707507438 (rad)
> > >> FovV=0.78980943449644714 (rad)
>
> > >> From there you can find the real world coordinates from the depth
> > >> cloud like so:
>
> > >> X_rw=(X_proj/Xres-.5)*Z*XtoZ
> > >> Y_rw=(0.5-Y_proj/Yres)*Z*YtoZ,
> > >> Z_rw=Z
>
> > >> where XtoZ=tan(FovH/2)*2, and YtoZ=tan(FovV/2)*2, and Z is the value
> > >> given by the depth map at each pixel.
>
> > >> I do this for my point cloud generation and it's super fast, no
> > >> bothering with the DLL at all.
>

> > >> On Feb 8, 12:58 pm, Dan <amer...@activate3d.com<mailto:amer...@activate3d.com>> wrote:
>
> > >> > We've seen significant slowness when invoking that function repeatedly
> > >> > on PC as well. I'd suggest batching points as much as possible. Our
> > >> > guess is that there's overhead in the function call as it goes out to
> > >> > the DLL and error checks.
>
> > >> > A version of that function as an inline would be really nice, or
> > >> > perhaps something along the lines of GetProjectiveToRealWorldMatrix.
> > >> > There's probably need to be special code to handle
> > >> > XN_DEPTH_MAP_NO_DEPTH_VALUE, but I'd wager that inline SSE math code
> > >> > with a branch around it would be much faster than calling out to the
> > >> > DLLs every time.
>
> > >> > dba
>

> > > To post to this group, send email to openn...@googlegroups.com<mailto:openn...@googlegroups.com>.
> > > To unsubscribe from this group, send email to openni-dev+...@googlegroups.com<mailto:openni-dev%2Bunsubscribe@goo­glegroups.com>.
> > > For more options, visit this group athttp://groups.google.com/group/openni-dev?hl=en<http://groups.google.com/group/openni-dev?hl=en>.

>
> --
> You received this message because you are subscribed to the Google Groups "OpenNI" group.

> To post to this group, send email to openn...@googlegroups.com<mailto:openn...@googlegroups.com>.
> To unsubscribe from this group, send email to openni-dev+...@googlegroups.com<mailto:openni-dev%2Bunsubscribe@goo­glegroups.com>.

> For more options, visit this group athttp://groups.google.com/group/openni-dev?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups "OpenNI" group.

> To post to this group, send email to openn...@googlegroups.com<mailto:openn...@googlegroups.com>.
> To unsubscribe from this group, send email to openni-dev+...@googlegroups.com<mailto:openni-dev%2Bunsubscribe@goo­glegroups.com>.

> For more options, visit this group athttp://groups.google.com/group/openni-dev?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups "OpenNI" group.
> To post to this group, send email to openn...@googlegroups.com.
> To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.

> For more options, visit this group athttp://groups.google.com/group/openni-dev?hl=en.- Hide quoted text -

[Eddie Cohen]

They are not hardcoded. They are calculated from parameters read from device.