Depth / RGB Calibration
27 views
Skip to first unread message
Durss
Oct 19, 2011, 10:17:40 AM10/19/11
to AS3OpenNI
Hi :)
Firstly, i'd like to give a huge thank to the auhor(s) of this lib,
it's really sweet to share it with others. Really.!
So, i tried to use it a bit and its works pretty well.
Here are screencaps of what i've done so far if you're interested :
http://www.youtube.com/watch?v=wQG7NuEhLSI
http://www.youtube.com/watch?v=GKu5h6slwZY
So, I'm quite happy to play with this!
BUT (hehe here it comes!) since yesterday i'm experiencing some
troubles...
I don't really know how, but my second test (the han solo like thing)
kind of fucked up the calibration of the kinect :/..
Basically the IR image doesn't fit the RGB one perfectly anymore.
For example on my first video, i'm using the IR depth map to mask the
RGB video to extract only my body (see top right).
But now, the depth map is like 1.5% bigger than the video, so it
doesn't works anymore.
I tried to uninstall the kinect completly and reinstall it, and it
worked. But when i tried to play again with the second test, bad
calibration came back :(..
Is there any way to reset the constructor's calibration from
as3openni?
Do you have any idea where this could come from?
Have a good day!
And thanks again for the job done!
Firstly, i'd like to give a huge thank to the auhor(s) of this lib,
it's really sweet to share it with others. Really.!
So, i tried to use it a bit and its works pretty well.
Here are screencaps of what i've done so far if you're interested :
http://www.youtube.com/watch?v=wQG7NuEhLSI
http://www.youtube.com/watch?v=GKu5h6slwZY
So, I'm quite happy to play with this!
BUT (hehe here it comes!) since yesterday i'm experiencing some
troubles...
I don't really know how, but my second test (the han solo like thing)
kind of fucked up the calibration of the kinect :/..
Basically the IR image doesn't fit the RGB one perfectly anymore.
For example on my first video, i'm using the IR depth map to mask the
RGB video to extract only my body (see top right).
But now, the depth map is like 1.5% bigger than the video, so it
doesn't works anymore.
I tried to uninstall the kinect completly and reinstall it, and it
worked. But when i tried to play again with the second test, bad
calibration came back :(..
Is there any way to reset the constructor's calibration from
as3openni?
Do you have any idea where this could come from?
Have a good day!
And thanks again for the job done!
Message has been deleted
Louis
Oct 21, 2011, 11:39:53 AM10/21/11
to as3o...@googlegroups.com
Hi Durss,
Your example is great, I didn't find the depth map is bigger than video map.
anyway, how could you mask the video map with depth map ?
Louis
Durss
Oct 21, 2011, 1:37:37 PM10/21/11
to AS3OpenNI
Hi :)
Actually in the video examples there is no problem ;). Everything was
working fine at this moment.
The problem came later.
I wanted to put texture on the second test instead of having only a
gray texture. So i worked to do kind of the same thing than in the
first example (except that i had to crop/scale just a part of the
video source). And while playing with parameters to crop correctly i
fugured out that a scaling problem appeared.
I thought i was doing something wrong but discovered that my first
test (the coverflow) wasn't working correctly anymore..
I really don't understand how i broke the calibration, but I did :/.
About the way to mask the video map with the depth map it's quite
easy. A simple BitmapData.copyPixels with the alphaBitmapData
parameter set to the depthmap bitmapdata and the work is done ;)
Durss
Actually in the video examples there is no problem ;). Everything was
working fine at this moment.
The problem came later.
I wanted to put texture on the second test instead of having only a
gray texture. So i worked to do kind of the same thing than in the
first example (except that i had to crop/scale just a part of the
video source). And while playing with parameters to crop correctly i
fugured out that a scaling problem appeared.
I thought i was doing something wrong but discovered that my first
test (the coverflow) wasn't working correctly anymore..
I really don't understand how i broke the calibration, but I did :/.
About the way to mask the video map with the depth map it's quite
easy. A simple BitmapData.copyPixels with the alphaBitmapData
parameter set to the depthmap bitmapdata and the work is done ;)
Durss
Louis
Oct 22, 2011, 1:15:03 AM10/22/11
to as3o...@googlegroups.com
did you try to reinstall the software and driver? are you develop under
window or mac?
window or mac?
for the mask question....
I was try to create one more bitmap for mask like this under onDepth
event handler:
var pt:Point = new Point(0, 0);
maskCanvas.copyPixels(videoCanvas, videoCanvas.rect, pt, depthMapCanvas,
pt, true);
the code just copy all pixel from videoCanvas without alpha data.... I
still thinking how to do it.
Louis :)
Durss
Oct 22, 2011, 8:44:25 AM10/22/11
to AS3OpenNI
"did you try to reinstall the software and driver? are you develop
under
window or mac? "
I'm on windows, and as i said in my first post, uninstalling/
under
window or mac? "
reinstaling is the only way i found to make it work back correctly ;).
But if i start my app it breaks the calibration again..
It's not easy to find where it comes from if i have to uninstall
evrything, reboot, and reinstall after each compilation :/
About the mask normally you should have something like this (forgot
the exact names):
bm = new BitmapData(Definitions.VGA_XXX_WIDTH,
Definitions.VGA_XXX_HEIGHT, true, 0); //Make it transparent!
Then when you extract the pixels from the buffer it should be
transparent (forgot the exact code, but it's in the examples).
By the way, the video bitmapdata should NOT be transparent. I guess
the pixels values of the video are sent without alpha channel so the
the image would be totaly transparent if you do so.
Durss
Louis
Oct 22, 2011, 8:55:49 AM10/22/11
to as3o...@googlegroups.com
Oh ... really trouble on it, I think you may try to reinstall the window
or other PC for test.
or other PC for test.
for the Mask:
yes, I have set it transparent.
private var maskCanvas:BitmapData = new
BitmapData(Definitions.VGA_MAX_WIDTH, Definitions.VGA_MAX_HEIGHT, true,0);
I've got something like my attach (test.png) now, you can see the video
on right side has mask with depth.
but still can see some elements behind on me, not only user on screen.
Louis
Durss
Oct 22, 2011, 9:50:55 AM10/22/11
to AS3OpenNI
Oh ok, you're probably not getting the good depth image.
I didn't tested the others options but you can enable/disable some
options on the AS3OpenNi bridge that relates to the depth image.
You should be able to get only the colored shape without the
background.
> test.PNG
> 405KAfficherTélécharger
I didn't tested the others options but you can enable/disable some
options on the AS3OpenNi bridge that relates to the depth image.
You should be able to get only the colored shape without the
background.
> 405KAfficherTélécharger
Louis
Oct 22, 2011, 9:56:52 AM10/22/11
to as3o...@googlegroups.com
Yes, I also think about the color shape and try to use some function
with copy Channel but seem not work.
with copy Channel but seem not work.
Louis
>> 405KAfficherT�l�charger
Durss
Oct 22, 2011, 10:03:06 AM10/22/11
to AS3OpenNI
You shouldn't have to worry about the channels.
As i said check out the options that relates to depth image on the
AS3OpenNi bridge and enable/disable various of them until you get only
the colored part of the image.
I don't have the code here so i can't tell you exactly what you need.
But for what i remember you should only have one option related to
detph enabled.
Durss
As i said check out the options that relates to depth image on the
AS3OpenNi bridge and enable/disable various of them until you get only
the colored part of the image.
I don't have the code here so i can't tell you exactly what you need.
But for what i remember you should only have one option related to
detph enabled.
Durss
Louis
Oct 22, 2011, 10:12:11 AM10/22/11
to as3o...@googlegroups.com
Oh ...I see. that is depthBackground ! just disable this option is work
now. thanks Durss
and did you try any option to control the depth range ?
now. thanks Durss
and did you try any option to control the depth range ?
Louis
Louis
Oct 22, 2011, 10:23:53 AM10/22/11
to as3o...@googlegroups.com
I found if set this "depthMapSnapOff" option to true, the depthmap will
large 1.5 of the video.
large 1.5 of the video.
On 22/10/2011 22:03, Durss wrote:
Durss
Oct 22, 2011, 10:34:34 AM10/22/11
to AS3OpenNI
Oh! Thanks for this info ahah! If it's only this, i'm so much going to
kick my head on a wall!!!
I hope this is the source of my problem! Thanks ;)
kick my head on a wall!!!
I hope this is the source of my problem! Thanks ;)
Louis
Oct 22, 2011, 10:37:05 AM10/22/11
to as3o...@googlegroups.com
Ha, good ! you should thank for yourself as well.
if you are not solve my problem, I didn't find out that for you :)
if you are not solve my problem, I didn't find out that for you :)
Louis
Durss
Oct 24, 2011, 4:37:46 AM10/24/11
to AS3OpenNI
Thank god!!
The problem was this depthMapSnapOff property that i changed to check
what was its action (and didn't realised it could be the source of my
problem -_-)
Thank you for finding this ;)
@Kerem Iseri :
I'm not used to google groups yet so i don't know where to answer you
as i can't find your answer here (but i received it by mail).
I guess you clicked "answer to the user" on my first message. Maybe i
just have to answer to your mail but as i'm not sure it will appear
here then, i prefer to do it here :D. Or maybe it was just like a
private message but it could interest other people.
Your question was :
"Very impressive menu man! Great job!! Do you have any blog where u
share your stuff or is there anywhere I can test and play around with
it? "
First, thank you for the sweet feedback :)!
I have no blog, well, i do have a really old one (in french) with
mostly stupid things on it so it won't interest you at all ahah. But i
have no "tech blog".
If you wish i could package my examples in .EXE that you could run
them (if you have a kinect installed) ;)
Durss
The problem was this depthMapSnapOff property that i changed to check
what was its action (and didn't realised it could be the source of my
problem -_-)
Thank you for finding this ;)
@Kerem Iseri :
I'm not used to google groups yet so i don't know where to answer you
as i can't find your answer here (but i received it by mail).
I guess you clicked "answer to the user" on my first message. Maybe i
just have to answer to your mail but as i'm not sure it will appear
here then, i prefer to do it here :D. Or maybe it was just like a
private message but it could interest other people.
Your question was :
"Very impressive menu man! Great job!! Do you have any blog where u
share your stuff or is there anywhere I can test and play around with
it? "
First, thank you for the sweet feedback :)!
I have no blog, well, i do have a really old one (in french) with
mostly stupid things on it so it won't interest you at all ahah. But i
have no "tech blog".
If you wish i could package my examples in .EXE that you could run
them (if you have a kinect installed) ;)
Durss
Louis Wong
Oct 24, 2011, 4:58:23 AM10/24/11
to as3o...@googlegroups.com
you're welcome :)
but did you find as3OpenNi always handup and unstable?
Durss
Nov 2, 2011, 5:58:26 AM11/2/11
to AS3OpenNI
@Louis > not sure to understand your question. What do you mean by
"always handup and unstable" ?
@Kerem > As the last time you answered only to me but the answer might
interest other people so i answer you here :)
Your question was :
"OpenNI has the native gestures like wave, push, steady but when it
comes to the swipes, its not working accurately at all. So I'm
wondering how did you achieve to make it so accurate as seen in the
video? I appreciate if you could share some of your experience about
the algorithms of the accurate interactivity."
As you said, the "native" gesture detection is quite unusable. Way too
restrictive for the end user. I first tried with it but quickly
changed to a "custom" way to do the same.
I'm actually just using the point (hand) detection feature. So it's
still a bit restrictive for the end user as he has to wave first to be
detected, but once it's done he can move around as he wants.
The first algo consisted of a virtual activation depth, if the hand
was over this "line", then i considered he was in drag mode. But, it's
not a god solution.. you need to be placed perfectly if you want to be
able to do natural moves.
So I changed for a simple speed average computation to base all my
things on.
So instead of having this virtual limit to go through, i just detect
if the hand goes forward at a minimum speed average (the tricky part
to customize) computed on, for what i remember, something like the 4
or 5 last steps. (that's the values you can see at the top left under
the stats panel).
To detect the "release" i do the same thing but with a lower minimum
average speed. So if the hand goes back at a minimum speed average i
release it.
The tricky part is to define those "minimum" values to get something
natural. And of course it depends on users...
I had another idea i wanted to test but didn't yet. Instead of having
this minimum speed for release, having a kind of hysteresis.
When the user goes in "press" mode, i detect the Z position and
consider that the release will be done at Z - X.
I'm not really sure it would be better but it's hard to say without
testing!
Anyway, it looks magical on my video but just because i'm used to the
gesture. Some collegues tried it and had troubles using it. It's far
from being perfectly natural.
Here are the EXE files if you want to play with it :
http://dl.dropbox.com/u/20758492/flash/kinect/coverflow.exe (86Mo, MP3
inside :D)
http://dl.dropbox.com/u/20758492/flash/kinect/face3d.exe (1,3MO the
Han Solo carbonite thing, you can click to change texture)
http://dl.dropbox.com/u/20758492/flash/kinect/water.exe (1,1Mo just a
useless water toy)
Install AIR 3 first if it's not already the case ;)
http://get.adobe.com/fr/air/
Durss
"always handup and unstable" ?
@Kerem > As the last time you answered only to me but the answer might
interest other people so i answer you here :)
Your question was :
"OpenNI has the native gestures like wave, push, steady but when it
comes to the swipes, its not working accurately at all. So I'm
wondering how did you achieve to make it so accurate as seen in the
video? I appreciate if you could share some of your experience about
the algorithms of the accurate interactivity."
As you said, the "native" gesture detection is quite unusable. Way too
restrictive for the end user. I first tried with it but quickly
changed to a "custom" way to do the same.
I'm actually just using the point (hand) detection feature. So it's
still a bit restrictive for the end user as he has to wave first to be
detected, but once it's done he can move around as he wants.
The first algo consisted of a virtual activation depth, if the hand
was over this "line", then i considered he was in drag mode. But, it's
not a god solution.. you need to be placed perfectly if you want to be
able to do natural moves.
So I changed for a simple speed average computation to base all my
things on.
So instead of having this virtual limit to go through, i just detect
if the hand goes forward at a minimum speed average (the tricky part
to customize) computed on, for what i remember, something like the 4
or 5 last steps. (that's the values you can see at the top left under
the stats panel).
To detect the "release" i do the same thing but with a lower minimum
average speed. So if the hand goes back at a minimum speed average i
release it.
The tricky part is to define those "minimum" values to get something
natural. And of course it depends on users...
I had another idea i wanted to test but didn't yet. Instead of having
this minimum speed for release, having a kind of hysteresis.
When the user goes in "press" mode, i detect the Z position and
consider that the release will be done at Z - X.
I'm not really sure it would be better but it's hard to say without
testing!
Anyway, it looks magical on my video but just because i'm used to the
gesture. Some collegues tried it and had troubles using it. It's far
from being perfectly natural.
Here are the EXE files if you want to play with it :
http://dl.dropbox.com/u/20758492/flash/kinect/coverflow.exe (86Mo, MP3
inside :D)
http://dl.dropbox.com/u/20758492/flash/kinect/face3d.exe (1,3MO the
Han Solo carbonite thing, you can click to change texture)
http://dl.dropbox.com/u/20758492/flash/kinect/water.exe (1,1Mo just a
useless water toy)
Install AIR 3 first if it's not already the case ;)
http://get.adobe.com/fr/air/
Durss
Louis
Nov 2, 2011, 6:00:38 AM11/2/11
to as3o...@googlegroups.com
did you run your demo smooth ?
even not work suddenly and need to restart again ?
even not work suddenly and need to restart again ?
Louis
Durss
Nov 2, 2011, 6:24:53 AM11/2/11
to AS3OpenNI
It' working smoothly yep. Except the 3D thing because of the 3D a bit
heavy but the rest is fine.
I didn't let the applications run long (no more than ~5min) so i can't
tell about the stability of the thing.
Durss
> >http://dl.dropbox.com/u/20758492/flash/kinect/face3d.exe(1,3MO the
> ...
>
> plus de détails »
heavy but the rest is fine.
I didn't let the applications run long (no more than ~5min) so i can't
tell about the stability of the thing.
Durss
> > Han Solo carbonite thing, you can click to change texture)
> >http://dl.dropbox.com/u/20758492/flash/kinect/water.exe(1,1Mo just a
>
> plus de détails »
Louis
Nov 2, 2011, 6:53:10 AM11/2/11
to as3o...@googlegroups.com
icic, maybe try it later to test the stability.
otherwise, did you play with Away3D like this ?
http://www.youtube.com/watch?v=mj34K6Nt8js&feature=BFa&list=PL333B12532CBCD623&lf=mh_lolz
<http://www.youtube.com/watch?v=mj34K6Nt8js&feature=BFa&list=PL333B12532CBCD623&lf=mh_lolz>
he can control the 3D model freely
Louis
>> plus de d�tails �
Durss
Nov 2, 2011, 7:03:06 AM11/2/11
to AS3OpenNI
Nope, not yet :)
That's my next step!
Not easy i think (never played that much with 3D) but must be
interesting to try!
On 2 nov, 11:53, Louis <louis2...@gmail.com> wrote:
> icic, maybe try it later to test the stability.
>
> otherwise, did you play with Away3D like this ?http://www.youtube.com/watch?v=mj34K6Nt8js&feature=BFa&list=PL333B125...
> <http://www.youtube.com/watch?v=mj34K6Nt8js&feature=BFa&list=PL333B125...>
> >>>http://dl.dropbox.com/u/20758492/flash/kinect/water.exe(1,1Mojust a
> ...
>
> plus de détails »
That's my next step!
Not easy i think (never played that much with 3D) but must be
interesting to try!
On 2 nov, 11:53, Louis <louis2...@gmail.com> wrote:
> icic, maybe try it later to test the stability.
>
> <http://www.youtube.com/watch?v=mj34K6Nt8js&feature=BFa&list=PL333B125...>
>
> plus de détails »
Louis
Nov 2, 2011, 7:32:21 AM11/2/11
to as3o...@googlegroups.com
Yup, I still thinking... ha
hope we can solve out how to make this.
hope we can solve out how to make this.
Louis
>> plus de d�tails �
Louis
Nov 2, 2011, 8:21:12 AM11/2/11
to as3o...@googlegroups.com
And I had try add keyframe the Character in Maya for animation. eg,
hand up, jump
then play jump keyframe when I really jump, play hand up when I hand up.
Is funny !
hand up, jump
then play jump keyframe when I really jump, play hand up when I hand up.
Is funny !
On 2/11/2011 19:03, Durss wrote:
>> plus de d�tails �
Reply all
Reply to author
Forward
0 new messages