There might be several reasons of why you don't see the speedup. Let's discuss them all.
- First of all I would ask what you see if you measure optical flow alone. According to OpenCV perf tests, you should get the speedup varying fro, 1.1 to 2.6, and 1.6 on average. Of course the actual performance is greatly affected by your parameters and activity of other threads. If your system is busy, processing time may be too unstable. So, please first of all measure the optical flow only. If you'll see 1.5 speedup, everything is OK. If not, read below.
- Speedup depends on your parameters. Try to use less/more points, smaller/larger search window size. Its possible that if you want to see the actual speedup, you need to give more work to optical flow function.
- But IMO the actual reason why you don't see the speedup, is that you possible include GFTT runtime into measurements, and this function is slow and almost not optimized. Its speedup is 1.1 - 1.8 with 1.3 on average. So, please report perf numbers for these two functions separately. And I think you should use a more lightweight approach for feature points detection. You can try to use FAST feature detector or a regular grid of points. This question was discussed several times here (like this), try to search for "grid" and "optical flow".