How to manually install julia packages on a Windows system

[Able Mashamba]

Dear Informed,

Is there a way to manually install julia packages on a Windows system that has a proxy.pac config system with a paranoid firewall. I have downloaded the packages I need and would want to install them manually as it appears Internet permission settings at my institution are making all Pkg.*() commands fail.

               _

   _       _ _(_)_     |  A fresh approach to technical computing

  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org

   _ _   _| |_  __ _   |  Type "help()" to list help topics

  | | | | | | |/ _` |  |

  | | |_| | | | (_| |  |  Version 0.2.0-rc2

 _/ |\__'_|_|_|\__'_|  |  Commit b372a68 2013-10-26 02:06:56 UTC

|__/                   |  i686-w64-mingw32

julia> Pkg.add("Distributions")

INFO: Initializing package repository C:\Users\amashamba\.julia

INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl

fatal: unable to connect to github.com:

github.com[0: 192.30.252.130]: errno=No error

ERROR: failed process: Process(`git clone -q -b metadata-v2 git://github.com/Jul

iaLang/METADATA.jl METADATA`, ProcessExited(128)) [128]

julia>

[Stefan Karpinski]

If you just make sure that the package source exists in ~/.julia, that should do the trick. In fact, you don't need to mess around with the package manager at all – Pkg commands will fail but loading packages should work fine. Unfortunately, building packages with binary dependencies will likely fail, but if you stick with pure-Julia packages, you should be ok.

[Able Mashamba]

Thanks for the saving response. I was Googling on this ad absurdum.

Cheers

[Yonatan Tekleab]

Hi Stefan,

I'm having the same problem.  Unfortunately the firewall I'm behind is clever enough prevent me from re-configuring git to use https, as many other threads have indicated.

I downloaded the master branch IJulia package from https://github.com/JuliaLang/IJulia.jl, extracted the folder, placed it inside the ~/.julia folder, then removed the ".jl-master" suffix.  This still isn't working for me.  When I try to open IJulia from the command prompt ("ipython notebook --profile julia"), it pulls up the typical IPython notebook.

Any thoughts on what I'm doing wrong?

Thanks in advance.

[Tony Kelman]

Can you do `using IJulia`, and/or `Pkg.build("IJulia")` ? Note also that IJulia depends on several other packages, indicated in the REQUIRE file (and those packages may have other dependencies of their own).

[Yonatan Tekleab]

I think I'm moving in the right direction. I downloaded several packages that IJulia depends on and put them in the ~/.julia/v0.3 directory along with the IJulia package itself. Before I was just sticking them in the ~/.julia directory, and I don't think Julia was seeing the packages.

When trying `using IJulia`, I get "ERROR: ZMQ not properly installed. Please run Pkg.build("ZMQ"). When I run that command, it tries to build Homebrew, WinRPM, and ZMQ, all of which have their own errors.

Homebrew: "could not spawn setenv(`git rev-parse --git-dir`; dir="P:\\.julia\\v0.3\\Homebrew\\deps\\usr"): no such file or directory (ENOENT)"

WinRPM: "update not defined"

ZMQ: "RPM not defined"

When I try `Pkg.build("IJulia")`, it trys to build Homebrew, WinRPM, Nettle, ZMQ, and IJulia.  I get errors for all except IJulia.  The Homebrew, WinRPM and ZMQ errors are the same.  For Nettle, I get: "RPM not defined"

Now I can open an IJulia instance, but the kernel dies shortly after it comes up. The command window states "ERROR: ZMQ not properly installed. Please run Pkg.build("ZMQ")". Then it attempts to restart the kernel and repeats the process.

[Tony Kelman]

So, to explain this a little, IJulia.jl depends on ZMQ.jl for interprocess communication. ZMQ.jl is a Julia wrapper around the C++ libzmq library, so the Julia package needs to download an architecture-specific compiled version of the library (or use a pre-existing installed copy from a package manager, or build from source) before it can work. Homebrew.jl is a Julia wrapper around the homebrew package manager which we use for binary package installation on Macs, and WinRPM.jl is an RPM-metadata parser that downloads cross-compiled Window binaries from the OpenSUSE build service.

If you're on a Windows machine, you should never need to use Homebrew.jl, anywhere that is mentioned in the REQUIRE file should be preceded with the @osx modifier which denotes that it only applies on OS X. On Windows, the zmq library will come from WinRPM.jl. What happens if you just run `using WinRPM` from a freshly-started Julia REPL?

WinRPM itself will need internet access - it doesn't use the same mechanism as Pkg does to download binaries, but it could easily run afoul of a paranoid proxy. Let's see.

[Isaiah Norton]

Are you using the gmail web interface from this same system? If so, then https:// should, in principle, be available and work for git too...

On the other hand, if you are using a separate (windows) system for gmail, then you ought to be able to run Pkg.install/build on the second system, get all requirements you need, and then copy your "C:/Users/USERNAME/.julia/v0.#" directory onto the firewalled system. This is tricky/unreliable on linux, but should be quite simple on windows as long as both systems are same word size -- both 32-bit or 64-bit (because of Microsoft's ABI permanence).

[Yonatan Tekleab]

Okay, at home I have a mac and so I just downloaded the packages that `Pkg.add("IJulia")` installed on my home machine, because I wasn't sure what packages IJulia depended on, and I didn't want to sift through the source code.  So I've now deleted the Homebrew directory.

`using WinRPM` does nothing... no errors.  It just returns the julia prompt again.

If it helps any, i'm attaching a screenshot of the Julia packages I have downloaded and placed into ~/.julia/v0.3/

I appreciate the help.

[Yonatan Tekleab]

yea, i figured the same thing since I am on the same system using https through my browser, but for some reason that I don't understand, Julia won't add/update packages, even when git is configured to use https

[Isaiah Norton]

Well, if https is in fact accessible then the best bet is to troubleshoot git directly first. After configuring the `insteadOf` git setting (per the README) try something simple like `git clone https://github.com/JuliaLang/julia`. There are a lot of guides on the internet for troubleshooting this issue.

[Yonatan]

yes, unfortunately i'm getting a similar error:

fatal: unable to access 'https://github.com/JuliaLang/julia/': Failed connect to github.com:443; No error

ERROR: failed process: Process(`git clone https://github.com/JuliaLang/julia/`, ProcessExited(128)) [128] in error at error.jl:22 (repeats 2 times)

[Isaiah Norton]

I just noticed your sig... in a regulated environment I guess it is not surprising if github is specifically filtered. Various ideas:

- git probably does not pick up Windows proxy server settings automatically (whereas your web browser would). if you access the internet through a proxy server, try configuring git to use it:

http://stackoverflow.com/questions/783811/getting-git-to-work-with-a-proxy-server

- It might be possible to run Windows Julia on your Mac under wine, install the packages, and then copy the resulting folder to the firewalled system. Haven't tried this myself.

- Find an external Windows system on which to run the installations, and then copy everything to the internal system. Possibly EC2 or similar cloud service will let you run a small instance for free for some period of time (I haven't looked recently).

[Jameson Nash]

WinRPM can be run from a mac/linux system directly, if you pass "win32" as the architecture flag to the install methods. I intended it to be fairly friendly and object oriented to use from the command line (and wrote most of it from a Mac).

[Yonatan]

I'll try this.  Thanks for the help