OpenCL GPU offloading significantly slower (Titan XP)

[George Nychis]

Hi everyone,

I am trying to use GPU offloading to improve my tesseract performance.

• Tesseract off of 'master' with commit 45a6546
  
• 4 Nvidia Titan XPs
  
• Driver version 390.48
  
• OpenCL 7.5.18-0ubuntu1
  

When running tesseract built with opencl support, I do see it dump a profiler file:

<version>v0.1</version>

<device><type>   </type><name>TITAN Xp</name><driver>390.48</driver><score>▒▒>    </score></device>

<device><type>   </type><name>TITAN Xp</name><driver>390.48</driver><score>▒m>    </score></device>

<device><type>   </type><name>TITAN Xp</name><driver>390.48</driver><score>Q▒▒>    </score></device>

<device><type>   </type><name>TITAN Xp</name><driver>390.48</driver><score>▒sl>    </score></device>

<device><type>    </type><score>-▒▒?   </score></device>

When running with a CPU-based tesseract binary, I see a performance of around 1.766 seconds on a particular image I have:

$ time /usr/bin/tesseract sample.jpg out.txt

Tesseract Open Source OCR Engine v4.0.0-beta.1 with Leptonica

real    0m1.766s

user    0m4.572s

sys     0m0.138s

When running on the GPU I see a pretty significant degradation in performance.  It is picking the GPUs based on the profiling:

$ time local/bin/tesseract sample.jpg out.txt

[DS] Profile read from file (tesseract_opencl_profile_devices.dat).

[DS] Device[1] 1:TITAN Xp score is 0.230911

[DS] Device[2] 1:TITAN Xp score is 0.230911

[DS] Device[3] 1:TITAN Xp score is 0.230911

[DS] Device[4] 1:TITAN Xp score is 0.230911

[DS] Device[5] 0:(null) score is 1.607031

[DS] Selected Device[1]: "TITAN Xp" (OpenCL)

Tesseract Open Source OCR Engine v4.0.0-beta.1-232-g45a6 with Leptonica

real    0m2.410s

user    0m5.001s

sys     0m0.591s

Might anyone have some suggestions as to how I can check what the cause of this is?  I'd greatly appreciate any help.  

I do understand that this is beta and still in develop.  I've read that performance improvements may not be experienced.  Is the degredation due to time uploading to the GPU outweighing the performance benefit?

Thanks!

George

[Janpieter Sollie]

Hi George,

The OpenCL engine of tesseract is currently being renewed for improved accuracy.  The part that you are working with will not make it much faster anymore, as the cost of small kernel launches outwheighs the cost of using AVX2 or related instruction sets.
Be patient, OpenCL for tesseract will come, and it will be much faster, but we need a working engine first

Op woensdag 9 mei 2018 22:42:46 UTC+2 schreef George Nychis:

[nick]

Hi Janpieter

could we speed up the testing process of tesseract with GPU ? is it possible ?