possible bug in OpenNI.NET ?

[Amir Hirsch]

I am unable to get SkeletonJointOrientation data from the OpenNI .NET
wrapper. I believe I've found a bug related to the wrappers for
skeleton capabilities. xnGetSkeletonJoint, xnGetSkeletonJointPosition,
xnGetSkeletonJointOrientation. In SkeletonJointPosition, the pJoint
parameter is passed as a ref. I'm pretty sure it should be a ref in
all the others.

These changes still didn't fix my problem getting the orientations
into Unity which also involve constructing the Quaternion... I copied
the way it was done in the old Nite.cs.

In OpenNIImporter.cs

[DllImport("OpenNI")]
//public static extern XnStatus xnGetSkeletonJoint(XnNodeHandle
hInstance, XnUserID user, SkeletonJoint eJoint, [Out]
SkeletonJointTransformation pJoint);
public static extern XnStatus xnGetSkeletonJoint(XnNodeHandle
hInstance, XnUserID user, SkeletonJoint eJoint, ref
SkeletonJointTransformation pJoint);
[DllImport("OpenNI")]
public static extern XnStatus
xnGetSkeletonJointPosition(XnNodeHandle hInstance, XnUserID user,
SkeletonJoint eJoint, ref SkeletonJointPosition pJoint);
[DllImport("OpenNI")]
//public static extern XnStatus
xnGetSkeletonJointOrientation(XnNodeHandle hInstance, XnUserID user,
SkeletonJoint eJoint, [Out] SkeletonJointOrientation pJoint);
public static extern XnStatus
xnGetSkeletonJointOrientation(XnNodeHandle hInstance, XnUserID user,
SkeletonJoint eJoint, ref SkeletonJointOrientation pJoint);

In SkeletonCapability.cs:

// public void GetSkeletonJoint(UserID user, SkeletonJoint
eJoint, SkeletonJointTransformation joint)
public void GetSkeletonJoint(UserID user, SkeletonJoint
eJoint, ref SkeletonJointTransformation joint)

{
// UInt32 status =
OpenNIImporter.xnGetSkeletonJoint(this.InternalObject, user, eJoint,
joint);
UInt32 status =
OpenNIImporter.xnGetSkeletonJoint(this.InternalObject, user, eJoint,
ref joint);

WrapperUtils.CheckStatus(status);
}

public void GetSkeletonJointPosition(UserID user,
SkeletonJoint eJoint, ref SkeletonJointPosition joint)
{
UInt32 status =
OpenNIImporter.xnGetSkeletonJointPosition(this.InternalObject, user,
eJoint, ref joint);
WrapperUtils.CheckStatus(status);
}

// public void GetSkeletonJointOrientation(UserID user,
SkeletonJoint eJoint, SkeletonJointOrientation joint)
public void GetSkeletonJointOrientation(UserID user,
SkeletonJoint eJoint, ref SkeletonJointOrientation joint)

{
// UInt32 status =
OpenNIImporter.xnGetSkeletonJointOrientation(this.InternalObject,
user, eJoint, joint);
UInt32 status =
OpenNIImporter.xnGetSkeletonJointOrientation(this.InternalObject,
user, eJoint, ref joint);

WrapperUtils.CheckStatus(status);
}

[Eddie Cohen]

Hi Amir,

I'm not really sure what you mean. SkeletonJointPosition is a struct, and so, to be filled by the function it must be passed as ref.
SkeletonJointOrientation on the other hand is a class, which means it can only be passed by ref. It would be wrong to add the ref keyword to the signature.

Can you explain what fails exactly when trying to get orientation?
I changed UserTracker.net sample in function GetJoint to also get orientation, and it worked just fine. I did it like this:

SkeletonJointOrientation orientation = new SkeletonJointOrientation();
this.skeletonCapbility.GetSkeletonJointOrientation(user, joint, orientation);

Eddie.

In OpenNIImporter.cs

In SkeletonCapability.cs:

WrapperUtils.CheckStatus(status);
}

WrapperUtils.CheckStatus(status);
}

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.
To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

[Amir Hirsch]

Have you tried it on OS X? I will switch to windows after this email
and see what happens.

I suspected the skeletonjointtransformation and orientation needed to
take a ref because the old Unity wrapper used a ref to get the
SkeletonJointTransformation:
NiteWrapper.SkeletonJointTransformation trans = new
NiteWrapper.SkeletonJointTransformation();
NiteWrapper.GetJointTransformation(userId, joint, ref trans);

I can't actually get a SkeletonJointTransformation or a
SkeletonJointOrientation without causing the system to freeze. I
modify UserTracker.net to use GetSkeletonJoint to load up a
SkeletonJointTranformation and then try to extract the positions from
that and this freezes on OS X. I haven't tested on windows:

private void GetJoint(uint user, SkeletonJoint joint)
{
SkeletonJointTransformation sjt = new SkeletonJointTransformation();
this.skeletonCapability.GetSkeletonJoint(user, joint, sjt);
SkeletonJointPosition pos = new SkeletonJointPosition();
pos = sjt.position;

here's the dump when it freezes:
http://pastebin.com/e3rFDm2a
....

Trying to read the orientation also freezes the app:
SkeletonJointOrientation ori = new SkeletonJointOrientation();
this.skeletonCapability.GetSkeletonJointOrientation(userId, joint, ori);

Here's the dump from Unity when it freezes from trying to get
orientation: http://pastebin.com/3bhVgqK0

Even when I just get the position data and the skeleton tracking
"works" it still freezes the mono system while exiting. It also
freezes Unity sometimes, but here's what i get from mono
UserTracker.net.exe:
http://pastebin.com/MjqMJm6H

I'm going to try in windows in a little bit

Amir

[Amir Hirsch]

GetSkeletonJointOrientation and GetSkeletonJoint do not work for me in
Unity3D on Windows either.

This code seem to work just fine on Mac and PC:

SkeletonJointPosition pos = new SkeletonJointPosition();

this.skeletonCapability.GetSkeletonJointPosition(userId, joint, ref pos);

But the Orientation and Transform fetches freeze Unity every time:

SkeletonJointOrientation orientation = new SkeletonJointOrientation();

this.skeletonCapability.GetSkeletonJointOrientation(userId, joint, orientation);

SkeletonJointTransformation sjt = new SkeletonJointTransformation();

this.skeletonCapability.GetSkeletonJoint(userId, joint, sjt);

[likeBVH]

The only way I managed to get JointOrientation works is to keep it as
struct, not a class.


[StructLayout(LayoutKind.Sequential)]
- public class Matrix3X3
+ public struct Matrix3X3
{
- [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
- public readonly float[] Elements = new float[9];
+ public float elem1;
+ public float elem2;
+ public float elem3;
+ public float elem4;
+ public float elem5;
+ public float elem6;
+ public float elem7;
+ public float elem8;
+ public float elem9;
};
[StructLayout(LayoutKind.Sequential)]
- public class SkeletonJointOrientation
+ public struct SkeletonJointOrientation
{
/** The actual orientation */
public Matrix3X3 Orientation;

[Amir Hirsch]

Yup, that works. Get rid of the classes for SkeletonJointTransformation and SkeletonJointOrientation... you probably don't need to clobber the matrix into a series of float elements, but i did that too.


here's what i changed my section of OpenNIImporter.cs to:

[StructLayout(LayoutKind.Sequential)]
public struct Matrix3X3 //was a class
{
// [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
// public readonly float[] elements = new float[9];
 public float elem1;
              public float elem2;
              public float elem3;
               public float elem4;
              public float elem5;
              public float elem6;
              public float elem7;
              public float elem8;
              public float elem9;
};

[StructLayout(LayoutKind.Sequential)]
public struct SkeletonJointOrientation //was a class
{

/** The actual orientation */

[MarshalAs(UnmanagedType.Struct)]
public Matrix3X3 Orientation;// = new Matrix3X3();
/** The confidence in the orientation */
public XnConfidence Confidence;
};

[StructLayout(LayoutKind.Sequential)]
public struct SkeletonJointTransformation //was a class
{
/** The position of the joint */
public SkeletonJointPosition position;
/** The orientation of the joint */
public SkeletonJointOrientation orientation;// = new SkeletonJointOrientation();
};







I also added "ref's" to the function imports in the openniimporter.cs:

[DllImport("OpenNI")]
// public static extern XnStatus xnGetSkeletonJoint(XnNodeHandle hInstance, XnUserID user, SkeletonJoint eJoint, [Out] SkeletonJointTransformation pJoint);

public static extern XnStatus xnGetSkeletonJoint(XnNodeHandle hInstance, XnUserID user, SkeletonJoint eJoint, ref SkeletonJointTransformation pJoint);

[DllImport("OpenNI")]
public static extern XnStatus xnGetSkeletonJointPosition(XnNodeHandle hInstance, XnUserID user, SkeletonJoint eJoint, ref SkeletonJointPosition pJoint);
[DllImport("OpenNI")]

// public static extern XnStatus xnGetSkeletonJointOrientation(XnNodeHandle hInstance, XnUserID user, SkeletonJoint eJoint, [Out] SkeletonJointOrientation pJoint);

public static extern XnStatus xnGetSkeletonJointOrientation(XnNodeHandle hInstance, XnUserID user, SkeletonJoint eJoint, ref SkeletonJointOrientation pJoint);

as well as in the SkeletonCapability.cs:


 //public void GetSkeletonJoint(UserID user, SkeletonJoint eJoint, SkeletonJointTransformation joint)

       public void GetSkeletonJoint(UserID user, SkeletonJoint eJoint, ref SkeletonJointTransformation joint)

       {
       
        //   UInt32 status = OpenNIImporter.xnGetSkeletonJoint(this.InternalObject, user, eJoint,   joint);
        UInt32 status = OpenNIImporter.xnGetSkeletonJoint(this.InternalObject, user, eJoint, ref joint);
           WrapperUtils.CheckStatus(status);
       }

       public void GetSkeletonJointPosition(UserID user, SkeletonJoint eJoint, ref SkeletonJointPosition joint)
       {
           UInt32 status = OpenNIImporter.xnGetSkeletonJointPosition(this.InternalObject, user, eJoint, ref joint);
           WrapperUtils.CheckStatus(status);
       }

     //  public void GetSkeletonJointOrientation(UserID user, SkeletonJoint eJoint,  SkeletonJointOrientation joint)
       public void GetSkeletonJointOrientation(UserID user, SkeletonJoint eJoint, ref SkeletonJointOrientation joint)
       
      {
      //     UInt32 status = OpenNIImporter.xnGetSkeletonJointOrientation(this.InternalObject, user, eJoint,  joint);

           UInt32 status = OpenNIImporter.xnGetSkeletonJointOrientation(this.InternalObject, user, eJoint, ref joint);

                 WrapperUtils.CheckStatus(status);
       }

I recompile the .NET wrapper with these changes and my unity wrapper can now pull the orientations.

Awesome.

Amir

[likeBVH]

Good Job. This is one of the inconsistencies in comparison with the c+
+ codes in the existing OpenNI.NET. Hopefully after CES, now there
have time to reveal these issues again.

[likeBVH]

Just for clarification, the codes I put here is from Eddie Cohen.

[Firdaus Muhammad]

Salam,  have u got the answer?
could u please help me,

SkeletonJointPosition pos = new SkeletonJointPosition();
this.skeletonCapability.

GetSkeletonJointPosition(userId, joint, ref pos);

///
how to define/declare userID?
joint? and pos?

could u tell me their data type?

thank you

best regards

[Firdaus Muhammad]

also how to assign/instantiate this :

skeletonCapability