Notes on installing Torch on Windows

[Paul Wagner]

I have been trying to get Torch to run on Windows recently. Inspired by SpaceCowboy850's write-down, I thought that maybe I should share my notes too. Hopefully at least some of this information is useful to others that are trying to get Torch working on Windows!

Current status:

  + luajit-rocks from the main repo is working (https://github.com/torch/luajit-rocks)

  + successfully linked with a BLAS library (LAPACK)

  + Torch works (all tests pass, except for abs; see below)

  + The nn package works (all tests pass)

  + Other packages that at least compile/install: optim, gnuplot, luafilesystem, inspect, image, trepl, sys

  - The nnx package works partially (some tests fail)

  - Packages with non-trivial issues: dataset and ipc (pthreads dependency)

  - GPU acceleration: haven't looked at that yet

Also, ZeroBrane Studio integration works just fine, including debugging.

Ok, here we go!

Prerequisites

====

Git: We need to have a git tool in our path. PortableGit or Git for Windows are some possible options. Cmder comes with a git client, too (didn't try using that one). Make sure that you have the git executable in your PATH.

CMake: Install CMake and make sure that you have it's executable in your PATH.

Visual Studio with the C/C++ module installed: VS 2015 Community Edition works just fine.

As SpaceCowboy850 noted, you should check what development environments and tools you have active in your PATH and in other environment variables already, because some, like cygwin and MinGW, might interfere with the process.

Cmder (optional)

====

I chose to use Cmder to replace the Windows Developer Command Prompt to make things less painful. The Torch REPL can be run through it, too. Skip this section if you wish.

Download and install Cmder, or Cmder Lite if you already have another Git client installed.

Create a new task for VS: Go to Settings -> Startup -> Tasks and create a new task. Name it VS2015 or something, make it the default task, and add the following string as the startup command (replace the project path with whatever you have):

  cmd /k ""%VS140COMNTOOLS%VsDevCmd.bat" & "%ConEmuDir%\..\init.bat"" -new_console:d:"X:\work\torch_projects":t:"VS2015"

Create a new task for Torch: Go to Settings -> Startup -> Tasks and create a new task. Name it Torch or something and add the following string as the startup command (replace the paths with whatever you are going to use):

  X:\torch-luajit-rocks\luajit.exe -new_console:d:"X:\work\torch_projects":t:"Torch"

Aliases can be added as usual (they persist restarts): alias ll=ls -la --show-control-chars -F --color $*

Adding Sublime Text 3's program directory to your path lets you use the command "subl" to quickly open files (for deeper integration, see eg http://goo.gl/yF173L ).

Note: There is the option "Inject ConEmuHk" under Settings -> Features. Enabling it slows down all command execution, especially compilation. Disabling it messes up colored output for second level processes. You might want to switch this temporarily on while trying to make sense of colored make output and otherwise keep it off.

Then some patching: at least for me, command line wrapping was broken when inside git repos. If you have the same issue, then open the file C:\Program Files (x86)\cmder_mini\config\cmder.lua and comment out the line with the os.execute() call. (see https://github.com/cmderdev/cmder/issues/749 )

Install MinGW and MSYS (needed if you want to compile BLAS by yourself)

====

Use the MinGW installer and install all meta-packages from the "Basic Setup" section. As usual with tools from the other side, don't install to Program Files (due to spaces).

Add "C:\MinGW /mingw" to the MSYS etc/fstab file, as instructed at http://www.mingw.org/wiki/getting_started -> After Installing You Should ...

The package manager is at bin/mingw-get.exe.

The MSYS environment can be started via msys\1.0\msys.bat.

Note that we did _not_ add the MinGW bin directory to PATH. (I do not know whether it makes any difference)

BLAS

====

I had the best success with LAPACK, so we are going with that, following the instructions at http://icl.cs.utk.edu/lapack-for-windows/lapack/#build (loosely).

Download and unzip http://www.netlib.org/lapack/lapack-3.6.1.tgz (or newer). Note that the site provides a precompiled version too, but it might be slower. I didn't try it.

Open MSYS and cd into the extracted package, then:

  mkdir build

  cd build

  cmake-gui

Set paths: Build into the current directory, use sources from parent dir

Configure: Select "MSYS Makefiles" and "Use default native compilers"

Set variables:

  BUILD_SHARED_LIBS = TRUE

  CMAKE_GNUtoMS = TRUE

  CMAKE_INSTALL_PREFIX = X:/lapack   (or whatever; can't recall whether the slash really had to be a forward slash)

Configure.

Generate.

Now close CMake and continue in the MSYS prompt:

  make

  make install

Make sure that you have the following DLLs in your path. I chose to copy them all into X:\torch-luajit-rocks, into which I eventually installed Torch, then added that directory to my PATH.

  - All DLLs from X:\lapack\bin\

  - The following DLLs from the bin directory of your MinGW installation: libgcc_s_dw2-1.dll, libgfortran-3.dll, libquadmath-0.dll

LuaJIT + LuaRocks

====

We will install from the main Torch repo while following diz-vara's build instructions (mostly). You get LuaJIT 2.1 if you add -DWITH_LUAJIT21=ON to both of the cmake commands below, otherwise it will be LuaJIT 2.0. I went with 2.0 for now, but maybe it would be wiser to use 2.1, after all? Also, the build commands in the instructions produced a debug build of LuaJIT for me, for some odd reason (Torch ran considerably slower with it). -DCMAKE_BUILD_TYPE=Release makes sure that we get a release version.

Choose a directory into which you want to install LuaJIT, LuaRocks and Torch. For me, it was X:\torch-luajit-rocks. Note, however, that the installation process will install stuff also into ..\share, ie, into X:\share in this case; you might want to add an additional directory level to keep things clean. Now, add the directory you chose to your PATH, while making sure that it comes _before_ CMakes bin directory.

Start Cmder and make sure that the VS2015 task is active (see above). Cd into some temporary directory, then:

  git clone https://github.com/torch/luajit-rocks.git

  cd luajit-rocks

  mkdir build

  cd build

  cmake .. -DCMAKE_INSTALL_PREFIX=X:\torch-luajit-rocks -G "NMake Makefiles" -DWIN32=1 -DCMAKE_BUILD_TYPE=Release

  nmake

  cmake -DCMAKE_INSTALL_PREFIX=X:/torch-luajit-rocks -G "NMake Makefiles" -DWIN32=1 -P cmake_install.cmake -DCMAKE_BUILD_TYPE=Release

Set the following environment variables: (these have been modified a bit from diz-vara's instructions, so as to get some other packages to build)

  LUA_CPATH = X:\torch-luajit-rocks\?.DLL;X:\torch-luajit-rocks\LIB\?.DLL;?.DLL

  LUA_DEV = X:\torch-luajit-rocks

  LUA_PATH = X:\torch-luajit-rocks\?;X:\torch-luajit-rocks\?.lua;X:\torch-luajit-rocks\lua\?;X:\torch-luajit-rocks\lua\?.lua;X:\torch-luajit-rocks\lua\?\init.lua

Test LuaJIT: Completely restart Cmder, then open the Torch task (eg, via the down arrow next to the green plus sign at lower right corner). Try writing some Lua to make sure that the REPL works.

Test LuaRocks: Within Cmder, ctrl-tab back to the VS2015 task and type: luarocks. You should get its help text.

Torch

====

Create the file X:\torch-luajit-rocks\cmake.cmd and add the following content:

if %1 == -E  (

cmake.exe  %* 

) else (

cmake.exe -G "NMake Makefiles"  -DWIN32=1 -dLUA_WIN -DCMAKE_LINK_FLAGS:implib=libluajit.lib -DLUALIB=libluajit %*

)

This is from https://github.com/torch/paths/issues/9 , except -DLUALIB=libluajit has been added (needed to compile the sys package later on). Double-check that X:\torch-luajit-rocks is in your PATH before CMake's bin directory.

Now, in Cmder with VS2015 task active, cd into some temporary directory and write:

  luarocks download torch

For some reason, I didn't manage to have CMake auto-detect LAPACK (nor any other BLAS library, for that matter). I proceeded as follows. Open the downloaded rockspec file and add the following to the configuration command (the one starting with "cmake .. -DCMAKE_BUILD_TYPE=Release -DLUA=$(LUA) [...]"), eg, add right before the "&& $(MAKE)" part:

  -DBLAS_LIBRARIES=X:/lapack/lib/libblas.lib -DBLAS_INFO=generic -DLAPACK_LIBRARIES=X:/lapack/lib/liblapack.lib -DLAPACK_FOUND=TRUE

Back in Cmder's VS2015 task:

  git clone git://github.com/torch/torch7.git

  cd torch7

  luarocks make ..\torch-scm-1.rockspec

Test Torch by launching the Torch task from Cmder and then type:

  require('torch')

  torch.test()

All tests should pass, except maybe the abs() test. (https://github.com/torch/torch7/issues/627 , which seems to have been fixed a few days ago)

The nn package

====

The package depends on luaffi, which in turn fails to build out-of-the-box. Solution is here: https://github.com/facebook/luaffifb/issues/10 ; you need to patch some files.

In Cmder's VS2015 task:

  luarocks download luaffi

  git clone git://github.com/facebook/luaffifb.git

  cd luaffifb

Now open the files ffi.h and test.c, and move the lines

  #include <complex.h>

  #define HAVE_COMPLEX

up into the preceding #else blocks in both files (the #else branch of the #ifdef _WIN32 test).

Back in Cmder's VS2015 task:

  luarocks make ..\luaffi-scm-1.rockspec

Now you should be able to install the nn package. Again in Cmder's VS2015:

  luarocks install nn

At least for me, every compilation unit gives me the following kind of warning: warning C4273: 'THNN_FloatLogSigmoid_updateGradInput': inconsistent dll linkage.

Any ideas what this is about and how to avoid it?

Now test it by launching Cmder's Torch task and typing:

  require('nn')

  nn.test()

All tests should pass.

The trepl package

====

Install readline for Windows from http://gnuwin32.sourceforge.net/packages/readline.htm

Add the GnuWin32 bin directory to your path now and completely restart Cmder. (note that having this in your path all the time will mess up some earlier Torch installation steps)

In Cmder's VS2015 task:

  luarocks download trepl

  git clone git://github.com/torch/trepl

Edit the rock file and modify the build/platforms/windows/modules/readline section:

    incdirs = {"windows","C:/Program Files (x86)/GnuWin32/include"},

    libdirs = {"windows","C:/Program Files (x86)/GnuWin32/lib"},

    libraries = {'readline'}

In Cmder's VS2015 task:

  cd trepl

  luarocks make ../trepl-scm-1.rockspec

Other packages

====

The following packages compiled/installed just fine via luarocks (luarocks install <packagename>). However, I haven't actually tested all of them yet:

  luafilesystem, inspect, image, nnx, optim, gnuplot

The sys package: This installs without issues, as long as you have -DLUALIB=libluajit added to your cmake.cmd (see earlier).

The dataset and ipc packages fail non-trivially, due to pthreads dependency.

Versions

====

The preceding process always installs the most recent version of everything. Just in case that something has changed since writing this and broke something, here are the versions that I used or which were downloaded implicitly via rock installs:

https://github.com/torch/luajit-rocks/commit/4eb4c5b6c6cf94badadebc8d5c39a1d470950036

https://github.com/torch/torch7/commit/7c740d5e8ec7fc10edbc3a75f2667e481eb47180

https://github.com/torch/paths/commit/68d579a2d3b1b0bb03a11637632e6e699b14ad80

https://github.com/torch/cwrap/commit/dbd0a623dc4dfb4b8169d5aecc6dd9aec2f22792

https://github.com/facebook/luaffifb/commit/d1c9712bfaaa73f9bc064227f120320e97fff517

https://github.com/torch/nn/commit/07d3bdd496be72dd132eb70eab96478b96547ffe

ZeroBrane Studio integration

====

The instructions at http://notebook.kulchenko.com/zerobrane/torch-debugging-with-zerobrane-studio worked for me, except that I had to point ZBS to the LuaJIT executable instead of the installation directory: path.torch = [[X:/torch-luajit-rocks/luajit.exe]]

The examples worked fine when executed from a script file, but using Torch from the ZBS's REPL did not work well.

For me, starting a debugging session complained about a missing lua51.dll. This was simply a naming issue: ZBS looks for lua51.dll, while we have libluajit.dll. A quick fix was to copy, in X:\torch-luajit-rocks, libluajit.dll to lua51.dll.

WARNING: This will work here, apparently because the dlls are used from different processes, but in general this is probably a bad solution. Managing to load both dlls in a single application would lead to two different address spaces, if I understand correctly, and then to trouble.

Does anyone have ideas for solving this in a better way?

Remote debugging ( https://studio.zerobrane.com/doc-remote-debugging ) seems to work, too, but I didn't really test this beyond a very basic test. The only thing I had to do, in addition to the steps in the instructions, was to copy the file mobdebug.lua from C:\ZeroBraneStudio\lualibs\mobdebug to X:\torch-luajit-rocks\lua.

Phew. Hope this helps anyone facing the same task!

ps. There is still not much information available about running OpenCL or CUDA versions of Torch/BLAS/nn/... on Windows, so if you have had success with those, please do share!

[Vislab]

Thanks for sharing Paul! It would be nice to make a tutorial out of this for future reference.

[Paul Wagner]

The Torch ecosystem is in constant flux, so the required process is probably going to change over time. Maybe we should keep sharing our findings and notes here (and fixing issues via pull reqs / forks of the problematic repos?), until we get this a bit more solid? Hopefully someone could volunteer at that point in writing a more comprehensive summary tutorial. This might make eventual "official" Windows support more likely, too, which would be nice for those of us who need to have this on Windows.

[jocelyn31...@gmail.com]

I execute the command " luarocks make ..\torch-scm-1.rockspec" , but get errors when building THVector.c.  The compile  indicates Can't open "x86intin.h". Do I miss some configurations? I find this file in MinGW install path. 

I use VS2013, does it matters?

在 2016年8月26日星期五 UTC+8下午9:12:33，Paul Wagner写道：

[jocelyn31...@gmail.com]

I've  fix this problem. The SSE.c file include <x86intrin.h>, which is used by GCC compiler. For vs, include <intrin.h> instead. 

在 2016年8月26日星期五 UTC+8下午9:12:33，Paul Wagner写道：

I have been trying to get Torch to run on Windows recently. Inspired by SpaceCowboy850's write-down, I thought that maybe I should share my notes too. Hopefully at least some of this information is useful to others that are trying to get Torch working on Windows!

[Paul Wagner]

Hi anonymous,

The install notes in the first post, along with additional
information, can now be found at
https://github.com/torch/torch7/wiki/Windows . There is also a related
discussion going on here: https://github.com/torch/torch7/pull/287

THVector.c does not compile out-of-the-box on Windows at the moment
due to this commit:
https://github.com/torch/torch7/commit/df191583ab794dd8e82a101f1d17436902ee5152

For me, VS complains not only about the missing include file, but also
about the inline asm instructions. May I ask, did you need to rename
any 'asm' instructions to '__asm' and was there anything else that you
had to change, besides the #include statement? If you wish, you could
also comment on the aforementioned commit page or in the discussion
thread!

-Paul


On Sun, Sep 11, 2016 at 5:46 PM,
<torch7+APn2wQfCknCljWlciluws6p3T...@googlegroups.com>
wrote:

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "torch7" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/torch7/iQpNfJB_oy0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> torch7+un...@googlegroups.com.
> To post to this group, send email to tor...@googlegroups.com.
> Visit this group at https://groups.google.com/group/torch7.
> For more options, visit https://groups.google.com/d/optout.

[Derrick]

Hi guys,

I've been trying to install torch on windows by following the instructions on https://github.com/torch/distro/tree/master/win-files and have run into problems with the readline.h file.

I was wondering if anyone has had similar problems before and could shed light on the issue. I already have installed the library with the header files of readline from http://gnuwin32.sourceforge.net/packages/readline.htm

and have added the file directory to the path so I'm not sure why it says that the file can't be found. Also, a side note, if this installation succeeds, does it include all the packages (ie. nn, trepl, etc...) or are those separate?

Thanks,

Derrick

On Wednesday, September 14, 2016 at 5:41:05 AM UTC-4, Paul Wagner wrote:

Hi anonymous,

The install notes in the first post, along with additional
information, can now be found at
https://github.com/torch/torch7/wiki/Windows . There is also a related
discussion going on here: https://github.com/torch/torch7/pull/287

THVector.c does not compile out-of-the-box on Windows at the moment
due to this commit:
https://github.com/torch/torch7/commit/df191583ab794dd8e82a101f1d17436902ee5152

For me, VS complains not only about the missing include file, but also
about the inline asm instructions. May I ask, did you need to rename
any 'asm' instructions to '__asm' and was there anything else that you
had to change, besides the #include statement? If you wish, you could
also comment on the aforementioned commit page or in the discussion
thread!

-Paul


On Sun, Sep 11, 2016 at 5:46 PM,

<torch7+APn2wQfCknCljWlciluws6p3Tb_SWGmIv0c-guB_cVBeBw5CBOiiR91sE@googlegroups.com>