OpenNI c# Wrapper and wpf hand tracking (sample and doubts)

[Rui Marinho]

hello i have manage to get a little sample of hand tracking on wpf
with the new c# wrapper, but i have some issues: first here is the
link to the sample http://dl.dropbox.com/u/1966569/kinect/kinect.rar

Simply run, and wave to get your hand start tracking. it will create a
button that will move with your hand.

My questions are, when does the new hand is created, everytime someone
waves ? Should we uses the NITE api or will this api be as good as
Nite in terms of helping tracking only hands and gestures? what's up
with the SetSmoothing property? can we smooth the movement of the hand
so it doesn't move and every liltte change?

Other thing is i m not being able to get a image from the camera
running in wpf, i get some memory problems, i copy the winforms
sample, build the histogram and all, but in the winforms sample they
use something like overriding the onpaint and draw with e.grraphics,
since i wanted to update a image source in wpf , i need a bitmap
source so i have done a timer that when it ticks, it does somethign
like:

Dispatcher.Invoke(

System.Windows.Threading.DispatcherPriority.Normal,
new Action(
delegate()
{
this.imgteste.Source = loadBitmap(bitmap);

}


public static BitmapSource loadBitmap(System.Drawing.Bitmap source)
{
return
System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(source.GetHbitmap(),
IntPtr.Zero, Int32Rect.Empty,

System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());

}

it works for some seconds but then applciation stops because is out of
memory i think... can someone help me out in this.

My goal is to get some samples working in wpf with the best
performance possible doing simple things and post here in the comunity
to help others.

thanks
Rui

[Davide Orazio Montersino]

Hi Rui and thanks for sharing!

However, I couldn't get your example to work.
This is an error I get whenever I try to use the unmanaged c# wrapper
(which I think your code is using), I hope someone can help:

"Unable to find an entry point named 'xnProductionNodeAddRef' in DLL
'OpenNI'."

On Dec 29, 2:10 pm, Rui Marinho <skim...@gmail.com> wrote:
> hello i have manage to get a little sample of hand tracking on wpf
> with the new c# wrapper, but i have some issues: first here is the

> link to the samplehttp://dl.dropbox.com/u/1966569/kinect/kinect.rar

[Rui Marinho]

Hi Davide, i think that's because u don't have the new openNI. dll i
have uploaded a new file:

http://dl.dropbox.com/u/1966569/kinect/kinect.rar

i have inclueded the 3 assemblys that i think it's need one is mvvm
stuff, and the new openni.net wrapper, that i makie specific reference
on the project, the other is openni.dll, that is the latest build for
the c# wrapper to work, u should copy this one and openi.net and put
in C:\Program Files (x86)\OpenNI\Bin ...

i think the erros is being cause because in that folder u have a older
version of openni.dll , but not sure :)

i m online just post i can help you

On 29 Dez, 13:58, Davide Orazio Montersino

> > Rui- Ocultar texto citado -
>
> - Mostrar texto citado -

[Davide Orazio Montersino]

That's it! Thank you very much Rui.
Actually, I went back on this thread just to write about it, because I
eventually found the solution looking in another thread..
where, it looks like, you had the same problem :)

[Rui Marinho]

yes was that problem, at now i m facing with a major problem that is
rendering in wpf the bitmap, i build a bitmasource every time and that
consumes alot of memory and it stops after a while, anyone can help me
how to figure the best way? I think if i can get a writablebitmap
backbuffer pointint to the bitmap that is coming from the camera it
can work, since in the renderthreat we update the bitmap, but i m
still struggle on how to make it work....
anyone?

thanks

[Rui Marinho]

i think i almost solved, but this is still not the best solution, u can download now the sample and try to see if it doesn't crash.

 

the best solution is not to create a new bitmapsource, but just move the pixes to a writablebitmap that is binded as the imagesource of our brush or image.. trying to make this happening .. :) lol

 

2010/12/29 Rui Marinho <ski...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.
To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/openni-dev?hl=en.

--
Rui Marinho

[Bustillo]

Hi Rui,

Thanks for sharing!
It's been very useful.

Pau

On 29 dic, 20:28, Rui Marinho <skim...@gmail.com> wrote:
> i think i almost solved, but this is still not the best solution, u can
> download now the sample and try to see if it doesn't crash.
>
> the best solution is not to create a new bitmapsource, but just move the
> pixes to a writablebitmap that is binded as the imagesource of our brush or
> image.. trying to make this happening .. :) lol
>

> 2010/12/29 Rui Marinho <skim...@gmail.com>

>
>
>
>
>
> > yes was that problem, at now i m facing with a major problem that is
> > rendering in wpf the bitmap, i build a bitmasource every time and that
> > consumes alot of memory and it stops after a while, anyone can help me
> > how to figure the best way? I think if i can get a writablebitmap
> > backbuffer pointint to the bitmap that is coming from the camera it
> > can work, since in the renderthreat we update the bitmap, but i m
> > still struggle on how to make it work....
> > anyone?
>
> > thanks
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "OpenNI" group.
> > To post to this group, send email to openn...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > openni-dev+...@googlegroups.com<openni-dev%2Bunsubscribe@googlegrou­ps.com>
> > .

> > For more options, visit this group at
> >http://groups.google.com/group/openni-dev?hl=en.
>
> --

> Rui Marinho- Ocultar texto de la cita -
>
> - Mostrar texto de la cita -

[Rui Marinho]

Hi have corrected the memory problembs and improve the performance of
the application, u can now see the image and the histogram and play
with it:

http://dl.dropbox.com/u/1966569/kinect/kinect.rar

One issue i have and didn't know why it's happening is sometimes the
image flicks, and also the histogram and the real image don't overlay
exacly one over the other, and i don't know why.

This new implementation i got rid of the bitmap, and use one
writablebitmap where i change it's buffer.

m_writeableBitmapImage.Lock();
//Just write the pixels to the bitpam
from the image metada, (best way?)
m_writeableBitmapImage.WritePixels(new
Int32Rect(0,0,(int)imgMD.XRes, (int)imgMD.YRes), imgMD.ImageMapPtr,
(int)imgMD.DataSize, m_writeableBitmapImage.BackBufferStride);
m_writeableBitmapImage.Unlock();


If anyone find more issues, performance problems or sugestions to make
this even better i apreciate...

Thanks
Rui

[El Bruno]

Excelent sample Rui !!!

I added a new class named CursorHandler to the kinectNite project to
also move the cursor with the hand. The code for the class is

using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Windows;

namespace kinectNite
{
internal class CursorHandler
{
#region Constants

private const UInt32 MouseEventfLeftDown = 0x0002;
private const UInt32 MouseEventfLeftUp = 0x0004;

//Screen coord conversion factors.
private const double ScreenXConv = 64.0; //65535 / 1024
private const double ScreenYConv = 85.3; //65535 / 768

#endregion

#region DllImports

[DllImport("user32.dll")]
private static extern void mouse_event(UInt32 dwFlags, UInt32
dx, UInt32 dy, UInt32 dwData, IntPtr dwExtraInfo);
[DllImport("user32.dll")]
public static extern IntPtr GetDesktopWindow();
[DllImport("user32.dll")]
public static extern IntPtr GetWindowDC(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern IntPtr ReleaseDC(IntPtr hWnd, IntPtr
hDC);
[DllImport("user32.dll")]
public static extern void mouse_event(uint dwFlags, long dx,
long dy, uint dwData, IntPtr dwExtraInfo);
[DllImport("user32.dll")]
public static extern void keybd_event(byte bVk, byte bScan,
uint dwFlags, IntPtr dwExtraInfo);
[DllImport("user32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool SetCursorPos(int x, int y);

#endregion

/// <summary>
/// Sends the mouse left click.
/// </summary>
/// <param name="location">The location.</param>
public static void SendMouseLeftClick(Point location)
{
MoveMouseTo((int) location.X, (int) location.Y);
mouse_event(MouseEventfLeftDown, 0, 0, 0, new IntPtr());
mouse_event(MouseEventfLeftUp, 0, 0, 0, new IntPtr());
}

/// <summary>
/// Moves the mouse to.
/// </summary>
/// <param name="x">The x.</param>
/// <param name="y">The y.</param>
public static void MoveMouseTo(int x, int y)
{
x = (int) (x*ScreenXConv);
y = (int) (y*ScreenYConv);
Debug.WriteLine(string.Format("mouse to: {0},{1}", x, y));
SetCursorPos(x, y);
}
}
}

And in the MainWindows.xaml.cs I changed

private void DetectPointPrimaryPointUpdate(object sender,
HandPointContextEventArgs e)
{
var x = e.HPC.Position.X;
var y = e.HPC.Position.Y;
var cursorX = 0;
var cursorY = 0;
Dispatcher.Invoke(
DispatcherPriority.Normal,
new Action(
delegate
{
Hand currHand = AvailableHands.First(h =>
h.Id == "1");
currHand.Left = (((lstHands.ActualWidth)/
2) - 30) + x;
currHand.Top = (((lstHands.ActualHeight)/
2) - 30) + (y*-1);
cursorX = (int) currHand.Left;
cursorY = (int) currHand.Top;
}
));
CursorHandler.SetCursorPos(cursorX, cursorY);
}

I'm only missing the click gesture. Any ideas??

Regards

Bruno

[Rui Marinho]

Hi Bruno, i just updated the nite sample with your code, and with the
push event to make the click, also i add a little ripple effect to
show off some cool wpf stuff, if anyone can tweak this to look more
like the surface standby screen. where it looks like u are messing
with the water.

http://dl.dropbox.com/u/1966569/kinect/kinect.rar


Also i have more questions, i change to use the pointdenoiser, how
does this class works? will it help me get more steady capture of the
hands? the thing is every little move is reflected and for tracking
hand i think it coudl help to not handle so many changes, a little bit
less... anyone has ideas?

Also, i m not able to get two hands being tracked?, is because i use
the primary point?, how can i say that is other hand point to be
track ?

thanks.

Rui

[Rui Marinho]

Hi have another sampe where i try to use openi.net.ddl and nite c#
wrapper, but i got erros say is trying to access protected memomry, is
this normal ? can i do this or is not good?!

> > Bruno- Ocultar texto citado -

[El Bruno]

Excelent sample Rui !!!

Happy 2011 for everyone :D

Regards

[Rui Marinho]

here's a little video i made about this two samples, like i said i m
using nite and the primary point to do the hand tracking its in the
main window, in the right corner i m using openi to get image/depth
and also hand tracking. Both are wpf.

http://www.youtube.com/watch?v=sF-uKKvAkD8

[Rikki]

Hi

While run i get an error saying that could not load Nite1 or its
dependencies. Can you help me

On Dec 29 2010, 6:10 pm, Rui Marinho <skim...@gmail.com> wrote:
> hello i have manage to get a little sample of hand tracking on wpf

> with the newc#wrapper, but i have some issues: first here is the
> link to the samplehttp://dl.dropbox.com/u/1966569/kinect/kinect.rar

[Rikki]

HI

When i run the sample i get this errors

Error 1 Unknown build error, 'Could not load file or assembly 'file:///
C:\Users\Dooj\Desktop\kinect\kinect\Dlls\OpenNI.net.dll' or one of its
dependencies. Operation is not supported. (Exception from HRESULT:
0x80131515)' kinect
Error 7 Unknown build error, 'Could not load file or assembly
'file:///C:\Users\Dooj\Desktop\kinect\kinectNite\Dlls\ManagedNite.dll'
or one of its dependencies. Operation is not supported. (Exception
from HRESULT: 0x80131515)' kinectNite

Can you help me on this?

[Ziv Hendel]

Hi Rikki!

A simple Google search on the error code you've received returns this
solution:
http://blog.rthand.com/post/2010/05/30/Getting-HRESULT-0x80131515-when-running-Righthand-DataSet-Visualizer.aspx
You can try to do the suggested unlock on your relevant DLLs... maybe
it will help...

Ziv.

[rikki agarwal]

Hi

Unblock did work but now i am getting this error

Error 4 Unknown build error, 'Could not load type 'System.Runtime.Versioning.TargetFrameworkAttribute' from assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.' kinect

Can you help me on this?

--
You received this message because you are subscribed to the Google Groups "OpenNI" group.
To post to this group, send email to openn...@googlegroups.com.

To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.

[Rui Marinho]

u have have .net 4.0 framework ?

On 16 Jan, 08:00, rikki agarwal <rikki233...@gmail.com> wrote:
> Hi
>
> Unblock did work but now i am getting this error
>
> Error 4 Unknown build error, 'Could not load type
> 'System.Runtime.Versioning.TargetFrameworkAttribute' from assembly
> 'mscorlib, Version=4.0.0.0, Culture=neutral,
> PublicKeyToken=b77a5c561934e089'.' kinect
>
> Can you help me on this?
>
>
>
> On Sat, Jan 15, 2011 at 5:06 PM, Ziv Hendel <z...@openni.org> wrote:
> > Hi Rikki!
>
> > A simple Google search on the error code you've received returns this
> > solution:
>

> >http://blog.rthand.com/post/2010/05/30/Getting-HRESULT-0x80131515-whe...

> > openni-dev+...@googlegroups.com<openni-dev%2Bunsubscribe@googlegrou­ps.com>
> > .

> > For more options, visit this group at

> >http://groups.google.com/group/openni-dev?hl=en.- Ocultar texto citado -

[rikki agarwal]

Yes but which one should i operate on both .net 4.0 and .net 4.0 client profile is not working

To unsubscribe from this group, send email to openni-dev+...@googlegroups.com.

[rikki agarwal]

Can anybody help me solve this?

[Rui Marinho]

I really don t know

Cumprimentos/Regards

Rui Marinho

Enviado do meu iPhone

[Craig]

Hello Rui,

This looks like a great piece of code, very useful. I took a look at
your YouTube video too, and this sample looks like it would really
help me out with a project.

Unfortunately the file on your dropbox seems to be corrupted - could
you please either repost it or send it to me via email?

Regards,
Craig.

On Jan 18, 1:24 pm, Rui Marinho <skim...@gmail.com> wrote:
> I really don t know
>
> Cumprimentos/Regards
>
> Rui Marinho
>
> Enviado do meu iPhone
>

> No dia 17 de Jan de 2011, às 17:34, rikki agarwal <rikki233...@gmail.com>

> escreveu:
>
> Can anybody help me solve this?
>

> On Sun, Jan 16, 2011 at 7:53 PM, rikki agarwal <rikki233...@gmail.com>wrote:
>
>
>
> > Yes but which one should i operate on both .net 4.0 and .net 4.0 client
> > profile is not working
>

> >> > > openni-dev+...@googlegroups.com<openni-dev%2Bunsu...@googlegroups.com>

> >> <openni-dev%2Bunsubscribe@googlegrou­ps.com>
> >> > > .
> >> > > For more options, visit this group at

> >> > >http://groups.google.com/group/openni-dev?hl=en.-Ocultar texto citado

> >> -
>
> >> > - Mostrar texto citado -
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "OpenNI" group.
> >> To post to this group, send email to openn...@googlegroups.com.
> >> To unsubscribe from this group, send email to

> >> openni-dev+...@googlegroups.com<openni-dev%2Bunsu...@googlegroups.com>

> >> .
> >> For more options, visit this group at

[Brian Scherady]

Hi Rui,

 

your video shows a window with a the camera view, is it disabled in your application? If yes how to enable it?

 

Thanks

 

Brian

[Rui Marinho]

It s the two projects running at the sane time one with nite another with openni.

I will check of. The files on depomos are still working.

Cumprimentos/Regards

Rui Marinho

Enviado do meu iPhone

[Brian Scherady]

Thank you Rui for the info.

 

What is "depomos" ?  Where can we get this second project?

 

Thank

 

Brian

---

From: Rui Marinho <ski...@gmail.com>
To: "openn...@googlegroups.com" <openn...@googlegroups.com>
Sent: Mon, January 24, 2011 11:54:04 AM
Subject: Re: [OpenNI-dev] Re: OpenNI c# Wrapper and wpf hand tracking (sample and doubts)

[Rui Marinho]

Sorry my spellchecker was in portuguese, the projects are all in the same solution.

I think the files in the dropbox are still the same, but I will check to see if it s all correct since I don t mess with them for a while now.

Cumprimentos/Regards

Rui Marinho

Enviado do meu iPhone

[Brian Scherady]

OK fine ! So please let me know how to activate the window with the camera view.

 

Thanks.

 

Brian

---

From: Rui Marinho <ski...@gmail.com>
To: "openn...@googlegroups.com" <openn...@googlegroups.com>

Sent: Mon, January 24, 2011 2:09:47 PM

[dario spadoni]

Hi Rui

Will you provide the code you wrote? The link to  dropbox seems to broken...

Thanks

Dario

[Anna Follmann]

Hi Rui!

I saw your video and it looks great. I am just starting with openNI and WPF and it would be great to have a look at you code.

Anyway the dropbox link seems to be broken (same problem that dario has). Could you send it or upload it somewhere again?

Thanks!

Anna

Am Mittwoch, 29. Dezember 2010 14:10:23 UTC+1 schrieb Rui Marinho:

hello i have manage to get a little sample of hand tracking on wpf

with the new c# wrapper, but i have some issues: first here is the
link to the sample http://dl.dropbox.com/u/1966569/kinect/kinect.rar