how to detect a waving hand from an non-fixed camera

[hua Fan]

Dear all,

I am wondering is it possible to detect a waving hand from a non-fixed camera? 
By saying non-fixed, I mean the camera can be hold on someone's hand or attached on the top of a moving car etc.

I have seen almost all of examples of motion detection are based on a fixed camera.
So my question emerged and I haven't conceived any good ideas.

Please help if you can. Any possible solutions are welcome.

Best regards,

Winston

[César]

Hi Winston!

Hmmm this is a very though problem. I think it would be needed to apply a good edge detector beforehand, or work with a depth camera (which would make segmentation easier). Otherwise I think it would be necessary to rely on color segmentation. 

If you can assume your users will at least wear long sleeves shirts, you can try color-based segmentation plus something like Camshift to track the hand. If this is possible, perhaps you could take a look at the Camshift sample application: after opening the application, click on Tracking > Select objects, then select your hands. However, this method mostly won't work if the hands come in touch with the head.

Another possibility is to try the Color tracking sample application (unfortunately it is not included in the demo page, but comes with the framework installer under the name Glove Tracking). You can try to tune the color segmentation until only the hands are visible. Then it will do the tracking automatically afterwards.

Hope it helps!

Best regards,

Cesar

[hua Fan]

Hi Cesar,

Thank you for your ideas.

But I can neither use a depth camera nor assume that users wear long sleeves shirts. The color-based segmentation cannot be relied on as the background can be any color.

One possible way is to use Machine Learning : feed in some marked videos and let the Genetic Programming(GP) algorithms to train the classifiers. And I will let them run around 300

generations and pick up the most accurate classifier. 

But do you have a GP trainer in this framework? Or do AForge.net Emgu or OpenCV have it?

Cheers,

Winston

[César]

Hi Winston,

There is support for genetic programming on AForge.NET. However... I am not sure it will be such an straightforward task. It will be necessary to preprocess the images very carefully in order to be able to learn like this, specially using genetic algorithms. I would advise you to first try some edge detection filters using AForge's IPLab (which can be used to test different filters quickly) and see what works best.

The next Accord.NET release is also going to include a few more edge detection filters which perhaps could be useful as well.

Best regards,

Cesar

[hua Fan]

Hi Cesar,

Thank you for your advice, I'm looking forward to your new release.

Cheers,