Optimal Character Recognition With Kinect

[Justin]

Hello,

I am trying to get started with developing an application that will
basically recognize a hand motion that is drawing characters in
space.
I want to start with numbers but want to eventually get to
recognizing
letters, then words.

My question is what is the best algorithm to use. I have to start
from
scratch and don't want to use any pre-existing software or API. I
have
done a little research and seen some work with neural networks and
openCV but I was wondering if this thread would be able to give any
advice. I would preferably like to be able to implement the algorithm
in C#.

Also, another question of mine is where should I get the data points
from. I know there's the sample application that tracks hands but
what
function call do I need to get the point data that is being output by
the hand.

Any advice and help is appreciated.

Thanks

[Jabberwok]

You ask too many questions at a time :) Start with reading OpenNI User
Guide and Nite Controls Programmer's Guide.
The easiest way is to use XnVPointControl. It does what you need -
produce points. Use RegisterPointUpdate to subscribe to hand
movements, save points, then analyze history. PointViewer provides
different concept, it inherits from XnVPointControl and get updates
via protected methods.

About OCR I don't see any points of manually implementing it. (Except
for learning purpose). But all libraries that I know use neural
networks.

[Justin]

Thanks so much for the advice, I will take a look at those functions.
The reason why I am manually implementing it is because it's for a
project. Trust me, if I didn't have to I definitely wouldn't.

Justin

[Mike]

You man want to take a look at the $1 Unistroke Recognizer at
http://depts.washington.edu/aimgroup/proj/dollar/.

I've played with it a little (a long time ago), but I've wondered if
it can be extended to 3D for gesture recognition. You may be able to
teach it a list of alphabetic characters.

It's been ported a bunch of times, including C#, and it's not a lot of
code.

Let us know how it goes.


Mike

[Vangos Pterneas]

I would also suggest using EmguCV (http://www.emgu.com/). It contains
a pre-installed optical character recognition demo in C# (using a
ported version of tesseract library). Download EmguCV and have a look
at "LicensePlateRecognition" demo project.

You could probably combine Kinect and EmguCV for your project.

On Feb 9, 10:37 pm, Mike <mikewar...@psu.edu> wrote:
> You man want to take a look at the $1 Unistroke Recognizer athttp://depts.washington.edu/aimgroup/proj/dollar/.

[FabioZ]

Hi Justin,
you might want to have a look at my demo here: http://www.youtube.com/watch?v=h0dFMrXJi1Y
If it's something that suits your need I'll be happy to share my code
(although it's still a little messy and it needs reworking) ;-)
When it comes to algorithms there are, broadly speaking, quite a few
approaches you can take for hand gesture recognition (all with their
own advantages), be it for numbers, characters or arbitrary signs.
Hidden Markov Models have been very popular as well as neural
networks, and to a minor extent Support Vector Machines and Dynamic
Time Warping.
The approach I used is none of those, because of the specific needs of
my application (which I can elaborate on if you're interested): I used
some sort of template matching algorithm (http://en.wikipedia.org/wiki/
Template_matching) that calculates the distance between a set of
predefined gestures in my database and the gesture the user drew
(albeit after simplification to a "normalised" form).
I am currently working on implementing a variation of the algorithm
that doesn't need scanning sequentially through the database, which is
O(n), but rather manages to be O(log(n) ).
Hope this helps...
PS=I also collected heaps of papers on this matter, if you're
interested I could share those too via Dropbox.

[Justin]

I really appreciate the feedback. I'm going to take a look at all the
options and hopefully be able to implement one of these soon.

@FabioZ

I am interested in learning a little more about what you're working
on. Please send me the valid info and papers whenever you can, I would
be forever thankful. Also, in your youtube video you mentioned a WPF
tutorial that you followed. Would you care sharing that as well
because I'd like to take a look at that too.

Thanks,

Justin

[Peter Kennard]

Hello - been lurking for a bit. Yes have looked, Googled,
etc. First tried OpenKinect and that seemed wrong and got advice to
try the OpenNI based system.

I need to get a Kinect up for a project - and it is cool for much other stuff.

Frustratingly I am having a bear of a time getting started with OpenNI etc.
I went through Vagnos' "How-to: Successfully Install Kinect on
Windows" - spent a couple of days on it so far.

and tried a number of permutations on both an XP Pro box and a
windows & box. (both Dell BTW), unstable, stable, OpenNI drivers, and
the PrimeSense supplied drivers. (Vagnos' "work around"
install) uninstalled drivers / reinstalled re- booted etc
etc. Results were consistsent but failures.

I figure I should get a working install where I can run the samples
before I go further. The target runtime machine is Windows7 Dell box
with an AMD64 in it. My Dev box is XP Pro.

For the "Stable" version install chain (latest as of today from links
provided) I get USB read timeouts, With the unstable builds (latest
as of yesterday) I get "No Registered Modules found" errors.

I have not tried the .net bindings my project I want to use it is
unmanaged C++.

Thanks!