SWIG, Go and OpenCV 2 (C++)

[Ronoaldo Pereira]

Hello Gophers,

I was using the go-opencv (github.com/lazywei/go-opencv) for face detection on images, but I found it to be a little slow, and in some inputs, I got a detected rectangle with negative values (i.e. rect.X = -311336). Looking at the binding code, I got no clue as why that happened. Also, there is no support for the face detection features in the project's new implementation (go-opencv/gocv) that uses SWIG to interface with the C++ API.

So I decided to attempt doing this directly using SWIG to make the C++ calls for OpenCV API. After a few hours of trial and error, I was able to hack and learn a little about how SWIG can be used to make the OpenCV calls in Go, but it was an error-prone and difficult process. I got the results seen here: https://gist.github.com/ronoaldo/71009b7b91684fecb367 (feedback welcome). I used as starting points the swig tutorial, the swig for Go and C++ documentation, some sample code in the misc/swig directory in the Go source tree and the go-opencv/gocv folder to accomplish that.

I would like to ask a few thinks that I got no information googling on the subject:

1. Constants / Enums: OpenCV defines a set of enums with integer constants. I was unable to make SWIG see them. I have attempted using %include <opencv2/core/core.cpp> for instance, but that failed an "unable to find file" error message. I had to redefine the constants in my .i file with #define. Is that the right approach in these cases?
2. While building the .i file for swig, there was no easy way to "see" the generated Go API. For instance, I forgot to add the public keyword to the class methods in the .i file, and Swig was not generating them. However, doing a go test -v the output was just "undefined method NewSize2i". I discovered inspecting the build flags that with -work flag, I could see the generated output by swig and check the generated methods in Go. Is there an easier way to inspect the swig generated go API? I.e., is there a way to see the godoc output of what swig generates?
3. I am willing to write some tools to parse the Open CV API from the website and generate .i files for each module automatically, but I don't know if this even make sense. Does anyone have done this for other libraries with SWIG and Go? Does it worth the hassle? I see that the Open GL Bindings are generated automatically with tooling, and I liked the approach, but was unable to understand where to begin doing something similar to OpenCV.
4. I would like to use the resulting package in other projects. When using swig, should I first generate the Go bindings manually (swig -go ...), so it is go gettable, or it is enough to have the .swigcxx file to `go get` the package.
5. I would like to contribute back the resulting code but I don't know what is the current maintained OpenCV binding for Go; is it this one github.com/lazywei/go-opencv ?

Thanks!

[Ian Lance Taylor]

On Wed, Sep 16, 2015 at 6:13 PM, Ronoaldo Pereira <rono...@gmail.com> wrote:
>
> Constants / Enums: OpenCV defines a set of enums with integer constants. I
> was unable to make SWIG see them. I have attempted using %include
> <opencv2/core/core.cpp> for instance, but that failed an "unable to find
> file" error message. I had to redefine the constants in my .i file with
> #define. Is that the right approach in these cases?

I see from the gist that you seem to really be using a .i file. But
below you discuss a .swig file. Which approach are you using?

Doing a %include of a .cpp file doesn't make much sense. Are the enum
values define in a .h file that you could %include?

Redefining the constants yourself seems like a last resort.

> While building the .i file for swig, there was no easy way to "see" the
> generated Go API. For instance, I forgot to add the public keyword to the
> class methods in the .i file, and Swig was not generating them. However,
> doing a go test -v the output was just "undefined method NewSize2i". I
> discovered inspecting the build flags that with -work flag, I could see the
> generated output by swig and check the generated methods in Go. Is there an
> easier way to inspect the swig generated go API? I.e., is there a way to see
> the godoc output of what swig generates?

There is no good way to inspect the SWIG generated Go API. Using
-work as you did is the best approach.

> I am willing to write some tools to parse the Open CV API from the website
> and generate .i files for each module automatically, but I don't know if
> this even make sense. Does anyone have done this for other libraries with
> SWIG and Go? Does it worth the hassle? I see that the Open GL Bindings are
> generated automatically with tooling, and I liked the approach, but was
> unable to understand where to begin doing something similar to OpenCV.

SWIG can in effect auto-generate the .swig file for you, in the sense
that SWIG can produce definitions for every function declared in the
.h files that it includes. Just %include the file outside of %{ %}.

> I would like to use the resulting package in other projects. When using
> swig, should I first generate the Go bindings manually (swig -go ...), so it
> is go gettable, or it is enough to have the .swigcxx file to `go get` the
> package.

I think that having the .swigcxx file is sufficient.

Ian

[Ronoaldo Pereira]

Hi Ian,

Thank you for the detailed response!

Em quinta-feira, 17 de setembro de 2015 00:38:16 UTC-3, Ian Lance Taylor escreveu:

On Wed, Sep 16, 2015 at 6:13 PM, Ronoaldo Pereira <rono...@gmail.com> wrote:
>
> Constants / Enums: OpenCV defines a set of enums with integer constants. I
> was unable to make SWIG see them. I have attempted using %include
> <opencv2/core/core.cpp> for instance, but that failed an "unable to find
> file" error message. I had to redefine the constants in my .i file with
> #define. Is that the right approach in these cases?

I see from the gist that you seem to really be using a .i file.  But
below you discuss a .swig file.  Which approach are you using?

Sorry for the confusion. I am using the .swigcxx file; I forgot to include my .swigcxx file in the gist, it was just %including the .i file.

 

Doing a %include of a .cpp file doesn't make much sense.  Are the enum
values define in a .h file that you could %include?

Redefining the constants yourself seems like a last resort.

Indeed, makes no sense. It was a typo. I have tested with %include "opencv2/core/core.hpp" and it was still not working. I found the reason; I should add -I/usr/include to the cgo CXXFLAGS comment, and it was passed to swig call during go build; pkg-config sets it to -I/usr/include/opencv. Fixing the -I parameter made constants available.

 

> While building the .i file for swig, there was no easy way to "see" the
> generated Go API. For instance, I forgot to add the public keyword to the
> class methods in the .i file, and Swig was not generating them. However,
> doing a go test -v the output was just "undefined method NewSize2i". I
> discovered inspecting the build flags that with -work flag, I could see the
> generated output by swig and check the generated methods in Go. Is there an
> easier way to inspect the swig generated go API? I.e., is there a way to see
> the godoc output of what swig generates?

There is no good way to inspect the SWIG generated Go API.  Using
-work as you did is the best approach.

OK. So if I choose the .i aproach and run swig manually, the resulting API could then be inspected, correct?

 

> I am willing to write some tools to parse the Open CV API from the website
> and generate .i files for each module automatically, but I don't know if
> this even make sense. Does anyone have done this for other libraries with
> SWIG and Go? Does it worth the hassle? I see that the Open GL Bindings are
> generated automatically with tooling, and I liked the approach, but was
> unable to understand where to begin doing something similar to OpenCV.

SWIG can in effect auto-generate the .swig file for you, in the sense
that SWIG can produce definitions for every function declared in the
.h files that it includes.  Just %include the file outside of %{ %}. 

Indeed, that worked. It was my misinterpretation of how %include processed the files that was causing some weird errors.

 

> I would like to use the resulting package in other projects. When using
> swig, should I first generate the Go bindings manually (swig -go ...), so it
> is go gettable, or it is enough to have the .swigcxx file to `go get` the
> package.

I think that having the .swigcxx file is sufficient.

Ian

With your clarifications, I was able to better understand how swig works, and how the go tool makes use of it during build. I attempted to expose the core module using the swigcxx file bellow:

opencv.swigcxx:

%module opencv

%{

#include "opencv2/opencv.hpp"

%}

%include "opencv2/core/types_c.h"

%include "opencv2/core/version.hpp"

// This include raises the error

//%include "opencv2/core/core.hpp"

cv.go:

package opencv

// #cgo CXXFLAGS: -std=c++11 -I/usr/include

// #cgo pkg-config: opencv

import "C"

cv_test.go:

package opencv

import "testing"

func TestOpenCVVersion(t *testing.T) {

t.Logf("Version: %v", CV_VERSION)

}

The code bellow works and prints the CV_VERSION value (2.4.8 in my case). It does speel out some warnings, but the test passes. However, If I uncomment the line that includes core.cpp, then it fails with this error:

/usr/include/opencv2/core/core.hpp:457: Error: Syntax error in input(3).

Looking at the offending line: https://github.com/Itseez/opencv/blob/2.4.8/modules/core/include/opencv2/core/core.hpp#L457 I looks like this particular C++ template syntax is not supported by swig. Is there any workaround? Is it a bug on swig or a weird C++ syntax on opencv? Is there a way to skip this during processing?

[Ian Lance Taylor]

On Wed, Sep 16, 2015 at 10:28 PM, Ronoaldo Pereira <rono...@gmail.com> wrote:
>>
>> There is no good way to inspect the SWIG generated Go API. Using
>> -work as you did is the best approach.
>>
>
> OK. So if I choose the .i aproach and run swig manually, the resulting API
> could then be inspected, correct?

Yes.

> The code bellow works and prints the CV_VERSION value (2.4.8 in my case). It
> does speel out some warnings, but the test passes. However, If I uncomment
> the line that includes core.cpp, then it fails with this error:
>
> /usr/include/opencv2/core/core.hpp:457: Error: Syntax error in input(3).
>
> Looking at the offending line:
> https://github.com/Itseez/opencv/blob/2.4.8/modules/core/include/opencv2/core/core.hpp#L457
> I looks like this particular C++ template syntax is not supported by swig.
> Is there any workaround? Is it a bug on swig or a weird C++ syntax on
> opencv? Is there a way to skip this during processing?

My first guess is that this is a bug in the SWIG C++ parser. It's a
fairly unusual construct: a template specialized using a ternary
operator. I don't know the answers to your other questions; you may
want to ask the SWIG developers over at http://github.com/swig/swig.

Ian

[elgazz...@gmail.com]

Hello,

i have the same error (core.hpp:456: Error: Syntax error in input(3) )when trying to wrap  the core opencv module using swig. 

i would like to know if you have an idea to solve this error.

thanks.

Wael

[Ronoaldo José de Lana Pereira]

Hi,

​​

I ended up doing a manual expose of the methods I needed to use in Go, particularly, haars cascade classifier, in a .i file like this:

opencv.i

%module opencv

%{
#include "opencv2/opencv.hpp"
%}

%include "std_string.i"
%include "std_vector.i"

namespace cv {
        #define CV_8U                   0
        #define CV_BGR2GRAY             6
        #define CV_HAAR_SCALE_IMAGE     2
        #define CV_LOAD_IMAGE_GRAYSCALE 0

        #define CV_HAAR_MAGIC_VAL    0x42500000
        #define CV_HAAR_FEATURE_MAX  3
        #define CV_HAAR_DO_CANNY_PRUNING    1
        #define CV_HAAR_SCALE_IMAGE         2
        #define CV_HAAR_FIND_BIGGEST_OBJECT 4
        #define CV_HAAR_DO_ROUGH_SEARCH     8

        class Mat {
        public:
                Mat();
                %rename(MatBytes) Mat(std::vector<char>& vec, bool copyData);
                Mat(std::vector<char>& vec, bool copyData);

                int depth();
                int channels();

                %rename(AtInt) at(int i, int j);

                int flags;
                int dims;
                int rows, cols;
        };

        class Rect {
        public:
                int x, y, width, height;
        };

        class Size2i {
        public:
                Size2i(int width, int height);
                int area();
                int width;
                int height;
        };

        class CascadeClassifier {
        public:
                bool load(
                        const std::string& filename);

                void detectMultiScale(
                        const Mat& image,
                        std::vector<Rect>& objects,
                        double scaleFactor,
                        int minNeightbors,
                        int flags,
                        Size2i minSize,
                        Size2i maxSize);
        };

        Mat imdecode(Mat buf, int flags);
        Mat imread(const std::string filename, int flags);
        void cvtColor(Mat in, Mat out, int code, int destChanels);
}

namespace std {
        %template(ByteSlice) vector<char>;
        %template(RectSlice) vector<cv::Rect>;

}

The close I got with the attempt to build a more generic binding is shared here: https://github.com/ronoaldo/gocv. In the internal package, you will find a .i file that can be used via `go generate` (in gen.go). I did a hack in the core.hpp to make swig happy, but I'm not sure of the consequences of the changes I made. It is labeled experimental for that reason :) - For that go generate to work, you need Swig 3 and the Open CV header files.

My suggestion would be for you to explicitly expose methods you want to use, or use an already made version from here: https://github.com/lazywei/go-opencv

Best,

--

Ronoaldo Pereira

ronoaldo.com