Comments and questions about GSoC OpenCV-Julia binding

[Alexander Sun]

Hi there,

After some research in these days. I have some comments about GSoC  OpenCV-Julia binding proposal. 

Base on the idae of Tim[1]: "return a Mat* to Julia as an opaque type...have ccall-able utility functions that query the size, get the pointer address, etc." I have made get some progress, I will update the OpenCV-Julia prototype later.

So the implementation details about the first part of OpenCV-Julia binding need some changes. There is no requirements of "intermediate struct" in C layer(See my proposal[2]). The better way to do this is return a Mat* to Julia as an opaque type, then use ccall-able utility functions that query the size, get the pointer address, like Tim said. After we can use these data to construct Image type(in Images.jl module) in Julia.

This approach is much like the way that OpenCV-Python binding does. Write pyopencv_to, pyopencv_from functions. The differences is here write jlopencv_to function in Julia level.  

The approach that OpenCV-Python used list below:

Suppose there is a OpenCV-Python function generated by generation tool( Some of these are write by hand):

Python context

*The parameter data type that passed in is ndarray

*convert it to cv::Mat---------pyopencv_to function

*call OpenCV C++ function

*convert the return value to ndarray----------pyopencv_from function

Python context

The data wrapper(like cv2.cpp in OpenCV-Python port), I call it layer. The OpenCV Python function is generated by generation tool(like hdr_parser.py and gen.py in OpenCV-Python port)

The approach that OpenCV-Julia are similar:

Suppose there is a OpenCV-Julia function that generated by generation tool( Some of these need write by hand, like OpenCV-Python does):

Julia context

*Image to cv::Mat-------Here is the question that I want to asked.

*call OpenCV C++ function

*convert the return value to Image----------Already solved

Julia context

I have questions about Image to cv::Mat: If I construct Image type in Julia, how to return it to Mat. It seems we could use Julia C API, such as use jl_array_ptr or something like that pass "primitive" type from Julia to C. But could I access Image type fields and use it to construct Mat in C layer?

After some research, [3], [4] may could help. If this step was overcome, it seems that there was no big technical problems in the progress.

If someone could give some suggestion or sample code, I will be very appreciated!

References:

[1]https://groups.google.com/forum/#!topic/julia-dev/rgm-NluG2KU

[2]http://www.google-melange.com/gsoc/proposal/public/google/gsoc2014/daetalus/5649050225344512 

[3]https://groups.google.com/forum/#!topic/julia-users/7rVpOUohYZg

[4]https://gist.github.com/anonymous/8897943

[Tim Holy]

Hi Alexander,

It looks like Mat has a constructor like this:

Mat(int rows, int cols, int type, void* data, size_t step=AUTO_STEP);

If you had a type defined as

immutable Mat
handle::Ptr{Void}
end

for interoperability with OpenCV (this is your opaque type), then
schematically you could create it from an Image or Array like this:

function Mat(img::AbstractArray)
c = coords_spatial(img)
Mat(ccall((:libopencv, :Mat), Ptr{Void}, (Cint, Cint, Cint, Ptr{Void},
Csize_t), size(img, c[1]), size(img, c[2]), ??, pointer(data(img)), ??)
end

The ?? mean parts you'd have to fill in. I'm not sure what the step parameter
is for, but I suspect it's the stride along the color dimension. (Which must
be first, I guess?) If so, you'd want to check that the image is in color-first
order using colordim(img). While you might permute it if necessary, in my
opinion perhaps it would be better to force the user to do so explicitly, so
you don't encourage silent inefficient conversions.

This constructor would share the same underlying buffer between Julia and
OpenCV, so it is zero-copy.

--Tim

[Jake Bolewski]

Zero copy between contiguous julia arrays and cv::Mat objects is not going to be possible because julia arrays are stored in column major order and cv::Mat object store the underlying data in a row major buffer.  You will have to do a copying transposition of the data when converting between the two.  This is sometimes confusing because opencv uses column major indexing (x,y) while storing the data in a row major format (y, x). 

[Tim Holy]

Images.jl doesn't care what order you store your arrays in. It implements any
storage order you want, and describes the meaning through properties.

--Tim

[Tobi]

See https://github.com/JuliaLang/julia/issues/5932. I think it would be valuable if we would make `Array` support different strides.