bioidiap/bob.ip.gabor - extract jet without transforming the whole image

[Thai Kien]

Hi,

I have cloned the project bioidiap/bob.ip.gabor and use its c++ api for performing face recognition.

Very impressed by the result, but i found it's quite slow to extract the gabor jet.

It's probably due to the fact that i have to transform the whole image and then extract the gabor jets at only some points on the transformed image.

So, I would like to know if you plan to develop some features helping extract the gabor jets without transforming the whole image.

Thanks and sorry for my so-so english :-)

--

Kien.

[Manuel Günther]

Dear Kien,

the extraction of Gabor jets is an intrinsically expensive operation. Indeed, the current implementation is such that the whole image is transformed, which takes a lot of time and requires a lot of memory, particularly for large images. Since the Gabor wavelet transform is executed in frequency domain, an FFT has to be executed for the image, and 40 inverse transforms for the result. The FFT has a time complexity of O(N^2*log(N)) and is, hence, growing more than linearly with growing image resolutions.

However, usually a large image resolution is not required, face recognition can be achieved with a resolution as small as 64 x 80 pixels (when only the inner part of the face is visible). Hence, we usually scale and crop the face image to a small resolution before we extract Gabor jets. With small image resolutions we have shown that face verification can be done in real-time.

So, there is no plan on my side to implement a Gabor wavelet transform in spatial domain, which would be executed as only at some parts of the image, but which has time complexity of O(N^2*P) (here, N is the resolution of the Gabor wavelet, not of the image, and P is the number of offset positions).

Anyways, if you wish to implement the Gabor wavelet transform by convolution in spatial domain yourself, you can make use of the Gabor wavelet images, which you can obtain using an IFFT of the http://pythonhosted.org/bob.ip.gabor/c_cpp_api.html#bob::ip::gabor::Wavelet::waveletImageC wavelet image.

Best regards

Manuel

P.S. I am also still impressed by the performance of the Gabor jet similarity :-)

[Manuel Günther]

Hi again,

I am a bit curious: Since you are the first user of one of the Bob packages that only uses the C++ part, could you please be so kind to share your experience with compiling your code against it?
- How easy it is to compile the code of the packages of bob? Since bob.ip.gabor requires several other bob packages, did you compile them as well?
- Did you find any issues in compiling and linking your C++ program against the bob packages?
- What is your approach to compile and link the different Bob libraries to your C++ program? Did you need to do a lot of hand-work or was it more or less straightforward?
- Did you include any of the CMakeLists.txt, which are generated during the buildout process?

Thank you very much for your cooperation!
Manuel

[Thai Kien]

Hi,

Thanks for your quick and detailed response.

I think i will give a try to implement the jet extraction by convolution in spatial domain.

As for the compiling bioidiap/bob.ip.gabor, i just cloned the project from github and follow the installation guide:

$ cd <bob.ip.gabor-dir> && python bootstrap-buildout.py && ./bin/buildout

Then, zc.buildout does nearly all the rest: download dependent bob packages, compile them, do python-stuffs... I have encountered only some errors due to missing libraries like blitz. Install them and re-run buildout, the installation finished successfully andi have got all the .so files needed for linking.

For the CMakeLists.txt of my program, i followed the one generated for bob.ip.gabor and copied some stuffs of it into mine: all the  include_directories, link_directories commands and bob library's names for my target_link_libraries command. Then my test program compiles and works like charm.

include_directories(/home/kien/workspace/bob/bob.ip.gabor/src/bob.io.base/bob/io/base/include)

include_directories(/home/kien/workspace/bob/bob.ip.gabor/src/bob.sp/bob/sp/include)

include_directories(/home/kien/workspace/bob/bob.ip.gabor/src/bob.core/bob/core/include)

include_directories(/home/kien/workspace/bob/bob.ip.gabor/bob/ip/gabor/include)

link_directories(/home/kien/workspace/bob/bob.ip.gabor/bob/ip/gabor)

link_directories(/home/kien/workspace/bob/bob.ip.gabor/src/bob.io.base/bob/io/base)

link_directories(/home/kien/workspace/bob/bob.ip.gabor/src/bob.sp/bob/sp)

link_directories(/home/kien/workspace/bob/bob.ip.gabor/src/bob.core/bob/core)

target_link_libraries(testGabor

  bob_io_base bob_sp bob_core bob_ip_gabor

  blitz jpeg boost_system

)

I would say compiling and linking against bob libraries is quite straightforward and required only a minimum knowledge of cmake.

In the other hand, it would still be nicer if we can compile the C++ code of bob without performing all the python stuffs.

I found that bob project is very well documented and have a really high code quality for such an open-source project. And it's a pleasure if my very little contribution could help.

Cheers,

--

Kien

[Manuel Günther]

Dear Kien,

thanks a lot for this insight. The only problematic bit that I might see in your approach is to find the CMakeLists.txt, which are hidden somewhere in the build directory.

I agree that a build procedure independent from python would be nice. Unfortunately, the python installation is already very mature and there would be quite a lot of things to re-implement. Therefore, I don't think that we ever will find the time to do it, particularly since there is no real need for it.

BTW: The missing libraries should be listed in our installation instructions, which are linked from the main (PyPI) page of bob.ip.gabor:
https://github.com/idiap/bob/wiki/Dependencies


Good luck with the implementation of the Gabor wavelet convolution in C++!
Manuel