Possibility to remove luaJIT dependencies from OpenFace

[Vitalius Parubochyi]

Hello, Brandon,

I interesting in possibility to make OpenFace fully independent from Lua. 

Have you even tried to remove lua dependencies from OpenFace? I mean even on Torch level (for example, use Theano or something else)?

I have found task on the github. But if I understand correctly you only want to remove some your parts of lua implementation, but still keep dependencies from Torch7-LuaJIT. 

Thanks,

Vitalius.

[Brandon Amos]

Hi Vitalius,

At the core, Torch is a C++ library that can be wrapped with Python. I think the best way to remove the Lua dependency from OpenFace will be to use a Python wrapper around Torch's core library. This way we won't have to convert complex Torch models to other formats like Theano, TensorFlow, Caffe, or keras that might not yet support all of the operations we're using, now or in the future. I do not recommend re-writing our training code in another framework since Torch is at a great level of abstraction and always has the latest deep learning tricks and techniques.

You could potentially use Hugh Perkins' pytorch wrappers as a starting point for this: https://github.com/hughperkins/pytorch

-Brandon.

[Vitalius Parubochyi]

You could potentially use Hugh Perkins' pytorch wrappers as a starting point for this: https://github.com/hughperkins/pytorch

Thanks. I will try. 

[Vitalius Parubochyi]

Hi, Brandon,

I spent some time to think about possibility to remove LuaJIT dependencies from OpenFace. I tried to realize full Python implementation without calling Lua code, but it still in progress. By the way, I found interesting solution to implement OpenFace code on Python using lua from Python. It is lutorpy project: https://github.com/imodpasteur/lutorpy. For me, I rewrote your openface_server.lua and torch_neural_net.py files into one torch_neural_net.py file which using lutorpy bridge to run lua code. If you interested I can send you my implementation, but it is very simple. I only merged your code using lutorpy syntax. For me it is useful solution, because now I didn't need create separate LuaJIT process.

Vitalius.

[Brandon Amos]

Hi Vitalius,

 

If you interested I can send you my implementation, but it is very simple.

Interesting! Sure, I'd like to see it. Can you attach a file here or send a link to it somewhere?

-Brandon. 

[Vitalius Parubochyi]

Hi Brandon,

Interesting! Sure, I'd like to see it. Can you attach a file here or send a link to it somewhere?

 

I attached my rewritten version of TorchNeuralNet class. To run it you need install Lutorpy (https://github.com/imodpasteur/lutorpy). Also it is called same as your, because I used it from my own location. 

Vitalius

[Brandon Amos]

I attached my rewritten version of TorchNeuralNet class

Thanks, this is very helpful. Did you try using their `torch.fromNumpyArray` function to avoid the hack of writing an image to disk with Python and reading it with Torch? I'm a little worried about adding another non-standard dependency to OpenFace that needs to be manually installed. Even though it's easy to install, it's an inconvenience for a lot of users.

-Brandon.

[Vitalius Parubochyi]

Thanks, this is very helpful.

Welcome.

 

Did you try using their `torch.fromNumpyArray` function to avoid the hack of writing an image to disk with Python and reading it with Torch?

I tried and their function works great. It converts numpy array into FloatTensor. But I found some differences in Python and Torch image arrays. I'm not familiar with the implementation of the Torch image module, so I'm not sure how image array looks in Torch. But I found three differences: 1) torch image arrays are normalized (0.0-1.0), numpy images are not by default (0-255). 2) I didn't find right order for torch image arrays channels. 3) When I used your hack with image saving and convert RGB or BGR image into torch using `torch.fromNumpyArray` I got different arrays in the both cases.

Maybe if I have a time, I'll explore this more in detail, but for this moment I left your way.

 

I'm a little worried about adding another non-standard dependency to OpenFace that needs to be manually installed. Even though it's easy to install, it's an inconvenience for a lot of users.

I understand. For many OpenFace user this will be additional task. But, for me it is very useful because OpenFace doesn't create separate process anymore (Lutorpy processes Lua code using Lupa (https://github.com/scoder/lupa) written on LuJIT compiler).

It is only my opinion, if this looks interesting for you and other OpenFace users, you can think about separate installation modes (with and without Lutorpy) or make git patch for more experienced users. 

[Brandon Amos]

I've added this file to the repo so it can be obtained and modified if needed. I added a link to this mailing list and noted that you created it. I also added a reference to the file and this mailing list post in the original code when the subprocess fails in case this will help any other users that have the same scenario as you:

https://github.com/cmusatyalab/openface/commit/96743001d38535ffe8ef4e22ad9f674ddcda1480

-Brandon.

[Vitalius Parubochyi]

I've added this file to the repo so it can be obtained and modified if needed. I added a link to this mailing list and noted that you created it. I also added a reference to the file and this mailing list post in the original code when the subprocess fails in case this will help any other users that have the same scenario as you:

https://github.com/cmusatyalab/openface/commit/96743001d38535ffe8ef4e22ad9f674ddcda1480

Great! Thanks. 

[tường đức]

Hello Vitalius Parubochyi,

Could you please tell me how to use your file? Can I delete the torch_neural_net.py file and rename your file to torch_neural_net.py and install openface again?

[Vitalius Parubochyi]

Hi,

On Thursday, 22 September 2016 13:06:44 UTC+3, tường đức wrote:

Hello Vitalius Parubochyi,

Could you please tell me how to use your file? Can I delete the torch_neural_net.py file and rename your file to torch_neural_net.py and install openface again?

Yes. You also don't need the openface/openface_server.lua file anymore. But you must install lutorpy (https://github.com/imodpasteur/lutorpy) manually, because it was not added to requirements. 

[Siddhanta Chakrabarty]

Hi Vitalius and Brandon,

I tried and their function works great. It converts numpy array into FloatTensor. But I found some differences in Python and Torch image arrays. I'm not familiar with the implementation of the Torch image module, so I'm not sure how image array looks in Torch. But I found three differences: 1) torch image arrays are normalized (0.0-1.0), numpy images are not by default (0-255). 2) I didn't find right order for torch image arrays channels. 3) When I used your hack with image saving and convert RGB or BGR image into torch using `torch.fromNumpyArray` I got different arrays in the both cases.
Maybe if I have a time, I'll explore this more in detail, but for this moment I left your way.

I made some modifications to the file torch_neural_net.lutorpy.py so that an opencv image can be converted to torch tensor directly. So the hack of writing image to disk can be avoided. I have attached the file here. Hope this helps!

- Siddhanta

[Siddhanta Chakrabarty]

There were some errors in the previous one. Here's an updated version.