Frame speed becomes very slow when i use optical flow !!!

[hardik pansuria]

Hi all,

Through optical flow , i can get next points using
calcOpticalFlowPyrLK() and good features using
goodFeaturesToTrack() . But my problem is Frame speed , Its around
0.96 FPS !!! I dont know what should i do??

I am thinking two solutions for this problem .Just tell me Is it right
way to do it or not?? or any other problem is there while using
optical-flow.

(1)First i checked that my one frame processing time is around 20
sec !! i think its too much . i am getting 200 good points , and for
each point , i am doing some calculation. should i do in thread ??

(2) Second , I make one buffer , in which i will store my frames , and
i get it each frame from this buffer rather than taking directly from
camera . so when one frame's process is finished , i will take next
frame from buffer,

I dont know whether i am wrong or right .. ,,,just i want your
suggestion ,if anyone knows what could be the problem ?? Pls
share ,,,,

Thanks

Hardik Pansuria

[Kirill Kornyakov]

1. Try to use FAST instead of goodFeaturesToTrack. Or simple regular grid over image or previous position of the object.
   
2. Try to use smaller (even more smaller) resolution. If you have only one object and you know it previous position an speed, try to run OF on a submat.
3. Try to tune OF parameters (smaller window).
4. Avoid unnecessary memory copies and clones. Prefer to allocate matrices in the very beginning and then reuse them (Mat::copyTo method) instead of adding new matrices to the buffer (Mat::clone). clone can be 3 times slower than copyTo. And copyTo can be very expensive, so better to avoid then too.

Regards,

Kirill

[Hardik]

Hi Kirill,

  Thank you . But I have doubts.

(1) As you wrote "simple regular grid over image or previous position of the object." Can you explain me a little bit ?

(2) In second  point ,you wrote use smaller resolution , I means i have to resize the frame by using resize() function ??  And when i have to use submat function??


Again Thanks a lot ......

--
Regards,

Pansuria Hardik

Android Developer

[Kirill Kornyakov]

1. Let's imagine you want to track position of a human face. In this case you can run goodFeaturesToTrack on the rectangle with face, and then track found points. Alternatively, you can put a regular grid over the face, ie 10x10 = 100 points, which are organized into a simple rectangular grid with equal distance between X and Y coordinates of points. Of course some of them will be weak (and you have to check status returned by OF), but you'll save time which you spend on calculations of goodFeaturesToTrack.
2. Yes, you have to use resize. Submat is an alternative approach, see the example with human face above. If you know that face was in some rectangle and it can't move faster than 10 pixels-per-frame, you can run OF on the same rectangle, plus 10 additional pixels from each side.

You're welcome :)

[Hardik]

got it!! Thank you so much krill, its nice explanation.

[Roman]

Hi Hardik and Kirill, I would like to track position of face, could
you give me a hint how to add grid - extract points in opencv? Thanks
a lot.

Roman

On 20 pro, 05:41, Hardik <har...@sourcebits.com> wrote:
> got it!! Thank you so much krill, its nice explanation.
>
> On Mon, Dec 19, 2011 at 8:55 PM, Kirill Kornyakov <
>
>
>
>
>
>
>
>
>

> kirill.kornya...@itseez.com> wrote:
>
> >    1. Let's imagine you want to track position of a human face. In this

> >    case you can run goodFeaturesToTrack on the rectangle with face, and then
> >    track found points. Alternatively, you can put a regular grid over the
> >    face, ie 10x10 = 100 points, which are organized into a simple rectangular
> >    grid with equal distance between X and Y coordinates of points. Of course
> >    some of them will be weak (and you have to check status returned by OF),
> >    but you'll save time which you spend on calculations of goodFeaturesToTrack.

> >    2. Yes, you have to use resize. Submat is an alternative approach, see

> >    the example with human face above. If you know that face was in some
> >    rectangle and it can't move faster than 10 pixels-per-frame, you can run OF
> >    on the same rectangle, plus 10 additional pixels from each side.
>
> > You're welcome :)
>
> --
> Regards,
>

> *Pansuria Hardik
>
> Android Developer
>
> *

[Hardik]

Hi Roman,

   I have not tried to track using "rectangular grid"  but i have used goodfeautresTotrack () function . But alternatively .. You  put face in the rectangular grid .. and then add some point inside rectangle by  touching on the face .and get that points as input into the opticalflow method and then move face and track that points position using optical flow  and so on... and regularly check to add some new points if some points  become weak ...This is what i understand


Thanks ...

[Kirill Kornyakov]

This is how I do this in C++:

void FormTrackingPointsArray(vector<Point2f>& inPoints, vector<Point2f>& outPoints,

                             int width, int height, int stepX, int stepY)

{

    for( int x = stepX / 2; x < width; x += stepX )

    {

        for( int y = stepY / 2; y < height; y += stepY )

        {

            Point2f pt(x,y);

            inPoints.push_back(pt);

            outPoints.push_back(pt);

        }

    }

}