ffi examples

[Marco Scoffier]

Hey,

Does anyone have examples using ffi to wrap some custom C/TH functions?

Ideally I would pass a double * or float * of my torch tensor data, from lua to a c/c++ function using clement's torchffi script to expose the pointer on the lua side.  Currently I am a bit lost in the CMakeLists.txt getting the library to compile and install properly without the lua_register() function and getting torch to find and load the lib when required.  I'll figure this out, but if someone had a working example it would save me some time :)

On a higher level, does this sound like a good idea or should I use the old lua C api functions which I already know? I'm going for clarity and simplicity getting new users writing torch + C code. Another concern is the flexibility of getting the lua stack pointer on the C side which makes var args possible in C, and passing lua tables to C relatively straight forward.  Any ideas?

Thanks,

Marco

[The Chemist]

Hey Marco,

Getting back to you a bit late, but I just released a new lib to deal with images, which relies solely on FFI and the raw pointer interface to Torch's Tensors. It's probably exactly what you're looking for: graphicsmagick. In Image.lua, you'll see a full example of an FFI interface to lots of C functions, C structs and C enums. 

It's the first time I use FFI for something real, and I must say it: it's totally amazing. On the Torch side, we're not completely ready to use FFI extensively, since most of our API still uses the Lua-based env, but with the small torchffi package I wrote, you can already do quite a lot.

On a side note, this package is a wrapper to GraphicsMagick, which provides tons of nice building blocks to efficiently manipulate images. I'm using it to pre-process lots of images (resizing, cropping, changing colorspaces, ...), and then export the result into a Tensor. The cool thing about this flow is that all the image transforms leverage the massive amount of work that went into all the ***Magick libs. If anyone's interested in using this new wrapper, let me know, I'd love some help on binding all the functions available in there.

Cheers,

Your nightly chemist.

[Marco Scoffier]

Great lib to have wrapped. Thanks. I'll definitely be using it. It certainly shows the interest of ffi. 

I guess I still long for a solution like the inline C in lush -- where, as you were refining an algorithm you could move from an interpreted loop to an optimized C version in such a straight forward way: just drop the C code right in your lush script with instant variable mapping between the two.  That for me was the ultimate coding paradigm...:)

I was trying to shoe horn my use of ffi to fit this inline, just drop the C in the code model but it's not quite that.

Marco

[The Chemist]

Well, with FFI it should be really straightforward to get there:

    • write your inline C as a multiline string, no need to care about the Lua stack in this code; it's just plain C.

    • call gcc on this string, and generate an .so

    • use ffi.cdef and ffi.load to expose and load your so

and you should be done :-)

Now I think that you should need a lot of C anymore with LuaJIT: for instance looping over pixels of an image, using a cdata pointer, in pure lua, is essentially as fast as C.

Is there a specific case where you think C still makes sense?

[Eugenio Culurciello]

an example of:

"Now I think that you should need a lot of C anymore with LuaJIT: for instance looping over pixels of an image, using a cdata pointer, in pure lua, is essentially as fast as C."

would be nice, do you have one?

[The Chemist]

Yep, install "torchffi", then:

require 'torchffi'

t = torch.randn(100000)

p = torch.data(t)

for i = 0,100000-1 do

   t[i] = t[i] * 2

end

This is as fast as C. Beware, it's 0-based.

Also, Ivo recently overwrote apply() in torchffi, so now if you do:

require 'torchffi'

t:apply(function(x) return x*2 end)

it uses raw ffi pointers as well ...

[Eugenio Culurciello]

small error: p, not t in loop:

require 'torchffi'

t = torch.randn(100000)

p = torch.data(t)

for i = 0,100000-1 do

   p[i] = p[i] * 2

end