Re: Problem adding packages for fresh install of Julia on OS X Lion

[Jacob Quinn]

From the julia GitHub:

If you are behind a firewall and you need to use the https protocol instead of the git protocol:

git config --global url."https://".insteadOf git://




On Monday, March 4, 2013 3:04:44 PM UTC-6, Todd Small wrote:

Hi,

Last night I downloaded the Julia installer for OS X Lion and Mountain Lion from https://code.google.com/p/julialang/downloads/detail?name=Julia-0.1.dmg&can=2&q=.  There were no hiccups in the installation, but I get the following error message when I try to add Gadfly:

               _

   _       _ _(_)_     |  A fresh approach to technical computing

  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org

   _ _   _| |_  __ _   |  Type "help()" to list help topics

  | | | | | | |/ _` |  |

  | | |_| | | | (_| |  |  Version 0.1.1+111038832.r2e9c.dirty

 _/ |\__'_|_|_|\__'_|  |  Commit 2e9c9bf5fe (2013-02-28 13:16:18)*

|__/                   |

julia> Pkg.add("Gadfly")

MESSAGE: Auto-initializing default package repository /Users/tsmall/.julia.

warning: templates not found /Users/viral/julia/usr/share/git-core/templates

Initialized empty Git repository in /Users/tsmall/.julia/.git/

[master (root-commit) 291e14c] Initial empty commit

git: 'submodule' is not a git command. See 'git --help'.

ERROR: failed process: Process(`git submodule add git://github.com/JuliaLang/METADATA.jl.git METADATA`, ProcessExited(1)) [1]

 in pipeline_error at process.jl:388

 in run at process.jl:378

 in anonymous at no file:86

 in cd at file.jl:26

 in init at pkg.jl:68

 in cd_pkgdir at pkg.jl:31

 in add at pkg.jl:112

 in add at pkg.jl:138

I have git 1.7.12.4 on my machine and use it all the time for work without grief.  My machine is running OS X 10.7.5.  Does anyone have any thoughts about what could be going wrong?

Thanks,

Todd

[Todd Small]

Switching to use the https protocol instead of the git protocol didn't help.  I was able to clone the main Julia repo using the git protocol without trouble.  I want to emphasize that I've installed from the dmg distribution; I'm not trying to compile from source.

I did discover, however, that the Pkg.add("Gadfly") script (?) was not actually creating the .julia directory or initializing the git repo there.  I went ahead did those two steps manually and also added the METADATA.jl.git submodule to .julia.  Now when I try to add Gadfly, I get the following error message:

julia> Pkg.add("Gadfly")

ERROR: could not open file REQUIRE

 in open at io.jl:281

 in open at io.jl:301

 in parse_requires at pkg/metadata.jl:130

 in anonymous at no file:117

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in add at pkg.jl:112

 in add at pkg.jl:138

I gather from skimming the documentation that the REQUIRE file is an essential part of the Package Manager, but I don't know where to find a copy.

In addition, if I try Pkg.update(), I get the following mysterious error message:

git: 'pull' is not a git command. See 'git --help'.

Did you mean this?

shell

ERROR: failed process: Process(`git pull`, ProcessExited(1)) [1]

 in pipeline_error at process.jl:388

 in run at process.jl:378

 in anonymous at no file:465

 in cd at file.jl:26

 in anonymous at no file:464

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in update at pkg.jl:463

Obviously, "pull" is a git command, as is "submodule add" from the original error message!  (I've double-checked that git is on my path.)

Thanks,

Todd

[Patrick O'Leary]

Yeah, you should be building .julia manually; Pkg.init() (which ran automatically, and printed the "Auto-initializing default package repository..." text) should do that.

I wonder if Julia is calling the same git you are normally? Can you run this at the Julia prompt?

run(`git --version`)

[Patrick O'Leary]

Gaah! Vital typographical error!

On Monday, March 4, 2013 7:39:01 PM UTC-6, Patrick O'Leary wrote:

Yeah, you should be building .julia manually

SHOULDN'T. There we go.

[Todd Small]

Julia.app contains its own version of git in /Applications/Julia.app/Contents/Resources/julia/bin.  This path is added to the PATH environment variable by the Julia start-up script and is the one executed by run(`git --version`).

julia> run(`git --version`)

git version 1.8.1.4

julia> run(`which git`)

/Applications/Julia.app/Contents/Resources/julia/bin/git

Something is awry with this git, however.  For example:

[~/.julia]$ which git

/Applications/Julia.app/Contents/Resources/julia/bin/git

[~/.julia]$ git submodule add git://github.com/JuliaLang/METADATA.jl.git METADATA

git: 'submodule' is not a git command. See 'git --help'.

I wonder what would happen if I simple removed this version of git and used version that comes on my Mac?

Todd 

[Stefan Karpinski]

Uh, that's super weird. Viral? Any idea what's up with this git that's in there?

[Todd Small]

And, yes, removing the git in /Applications/Julia.app/Contents/Resources/julia/bin solved my problems.

Thanks for your help!

Todd

[Keno Fischer]

Git is a mess to package correctly, which is why we copy the entire Git portable directory on Windows and add the paths manually, rather than placing it into the Julia directory structure.

[Viral Shah]

Yes, that is the git I am bundling. Clearly, as Keno points out, we need to figure out how to bundle git correctly. Does anyone know what needs to be done? At the very least, I can add the julia bundled git to the end of the PATH rather than in the beginning, so that /usr/bin/git is used if it exists.

I have opened #2469.

-viral

[Viral Shah]

I uploaded new mac binaries, which will prioritize on using system installed git if available. That will at least solve the problem for those users who have git installed. I am still not sure how to configure and install git with relative paths when building the julia distribution.

-viral

On Tuesday, March 5, 2013 2:34:44 AM UTC+5:30, Todd Small wrote:

Hi,

Last night I downloaded the Julia installer for OS X Lion and Mountain Lion from https://code.google.com/p/julialang/downloads/detail?name=Julia-0.1.dmg&can=2&q=.  There were no hiccups in the installation, but I get the following error message when I try to add Gadfly:

               _

   _       _ _(_)_     |  A fresh approach to technical computing

  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org

   _ _   _| |_  __ _   |  Type "help()" to list help topics

  | | | | | | |/ _` |  |

  | | |_| | | | (_| |  |  Version 0.1.1+111038832.r2e9c.dirty

 _/ |\__'_|_|_|\__'_|  |  Commit 2e9c9bf5fe (2013-02-28 13:16:18)*

|__/                   |

julia> Pkg.add("Gadfly")

MESSAGE: Auto-initializing default package repository /Users/tsmall/.julia.

warning: templates not found /Users/viral/julia/usr/share/git-core/templates

Initialized empty Git repository in /Users/tsmall/.julia/.git/

[master (root-commit) 291e14c] Initial empty commit

git: 'submodule' is not a git command. See 'git --help'.

ERROR: failed process: Process(`git submodule add git://github.com/JuliaLang/METADATA.jl.git METADATA`, ProcessExited(1)) [1]

 in pipeline_error at process.jl:388

 in run at process.jl:378

 in anonymous at no file:86

 in cd at file.jl:26

 in init at pkg.jl:68

 in cd_pkgdir at pkg.jl:31

 in add at pkg.jl:112

 in add at pkg.jl:138

[Viral Shah]

I have now uploaded new mac binaries, which I believe should fix this problem. It would be great if someone who does not have git installed on their mac can try it out and report.

Details are documented in this issue.

https://github.com/JuliaLang/julia/issues/2469

-viral

[Viral Shah]

The release-0.1 branch had a bug in the way git was being configured. I will build and upload a new package, which should solve this.

-viral

On Friday, March 29, 2013 11:03:11 PM UTC+5:30, Yuta Takahashi wrote:

Hi,

I've just installed the last julia (Julia-0.1.2.dmg) on 28/3/2013. And I cannot install packages. Here is the message that I got

julia->Pkg.add("Distributions")

fatal: Not a git repository (or any of the parent directories): .git

ERROR: failed process: Process(`git add REQUIRE`, ProcessExited(128)) [128]

 in pipeline_error at process.jl:388

 in run at process.jl:378

 in anonymous at no file:136

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in add at pkg.jl:135

 in add at pkg.jl:167

If I run Pkg.update(), then I got

julia> Pkg.update()

ERROR: chdir: No such file or directory

 in system_error at error.jl:21

 in cd at file.jl:25

 in anonymous at no file:500

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in update at pkg.jl:499

Any thoughts about what could be going wrong?

Thanks,

yuta

2013年3月5日火曜日 7時43分48秒 UTC-6 Viral Shah:

[Viral Shah]

I just uploaded new mac binaries. Perhaps they will fix this problem.

-viral

On Friday, March 29, 2013 11:03:11 PM UTC+5:30, Yuta Takahashi wrote:

Hi,

I've just installed the last julia (Julia-0.1.2.dmg) on 28/3/2013. And I cannot install packages. Here is the message that I got

julia->Pkg.add("Distributions")

fatal: Not a git repository (or any of the parent directories): .git

ERROR: failed process: Process(`git add REQUIRE`, ProcessExited(128)) [128]

 in pipeline_error at process.jl:388

 in run at process.jl:378

 in anonymous at no file:136

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in add at pkg.jl:135

 in add at pkg.jl:167

If I run Pkg.update(), then I got

julia> Pkg.update()

ERROR: chdir: No such file or directory

 in system_error at error.jl:21

 in cd at file.jl:25

 in anonymous at no file:500

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in update at pkg.jl:499

Any thoughts about what could be going wrong?

Thanks,

yuta

2013年3月5日火曜日 7時43分48秒 UTC-6 Viral Shah:

I have now uploaded new mac binaries, which I believe should fix this problem. It would be great if someone who does not have git installed on their mac can try it out and report.

[Connelly Barnes]

FYI, I just now installed the latest Julia binary for Mac OS X from the Julia website [1]. This raised the same error when I first tried to add a package (Pkg.add("PyPlot")):

git: 'submodule' is not a git command. See 'git --help'.

Removing git from the Julia bin directory fixed the problem as it did for Todd.

[1]. Julia Version 0.1.2+113667293.r7252 (Commit 7252bb59ac (2013-03-30 23:23:59)). The latest release version as of Sept 22, 2013.

[Connelly Barnes]

An update to my previous message. I'm not sure if the fix of deleting the Julia git actually worked.

I tried installing the four plotting packages shown on the Julia homepage but none worked. I also tried installing a simpler package, ApproxFun because I hoped that would work but it also did not work. Error messages are below.

I also wiped .julia and reinstalled them again after a 'brew update' since one of the brew packages failed and a Google result suggested this if the Julia packages get in inconsistent state. This gave pretty much the same errors.

I suggest to add a simple package that reliably installs and works to the instructions of [1], as a test for whether the package manager is working.

[1]. http://julialang.org/downloads/

Errors:

julia> using Gadfly

ERROR: syntax: extra token after end of expression

 in include_from_node1 at loading.jl:76

 in reload_path at loading.jl:96

 in require at loading.jl:48

 in include_from_node1 at loading.jl:76

 in reload_path at loading.jl:96

 in require at loading.jl:48

 in include_from_node1 at loading.jl:76

 in reload_path at loading.jl:96

 in require at loading.jl:48

at /Users/connelly/.julia/Iterators/src/Iterators.jl:22

julia> Pkg.add("Winston")

ERROR: in module path: BinDeps not defined

 in include_from_node1 at loading.jl:76

 in anonymous at no file:241

 in cd at file.jl:26

 in runbuildscript at pkg.jl:238

 in _resolve at pkg.jl:314

 in anonymous at no file:155

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in add at pkg.jl:135

 in add at pkg.jl:167

at /Users/connelly/.julia/Tk/deps/build.jl:1

julia> Pkg.add("Gaston")

julia> using Gaston

julia> Gaston.set_terminal("aqua")

julia> x=-pi:.001:pi; y=x.*sin(10./x); plot(x,y) #(plot x*sin(10/x))

gnuplot> set term aqua 1

                  ^

         line 0: unknown or ambiguous terminal type; type just 'set terminal' for a list

0

julia> Pkg.add("PyPlot")

julia> using PyPlot

ERROR: error compiling version: could not load module libzmq: dlopen(libzmq.dylib, 1): image not found

 in include_from_node1 at loading.jl:76

 in reload_path at loading.jl:96

 in require at loading.jl:48

 in include_from_node1 at loading.jl:76

 in include_from_node1 at loading.jl:76

 in reload_path at loading.jl:96

 in require at loading.jl:48

at /Users/connelly/.julia/ZMQ/src/ZMQ.jl:51

julia> Pkg.add("ApproxFun")

ERROR: Path IniFile already exists! Please remove to allow installation.

 in _resolve at pkg.jl:298

 in anonymous at no file:155

 in cd at file.jl:26

 in cd_pkgdir at pkg.jl:34

 in add at pkg.jl:135

 in add at pkg.jl:167

On Monday, March 4, 2013 4:04:44 PM UTC-5, Todd Small wrote:

[Avik Sengupta]

Hi Connelly, 

Unfortunately the 0.1.2 version is so old at this point in time that most packages do not work. In particular, the first two error messages (Gadfly and Winston) are surely due to using a 0.1 version of Julia. Incompatible packages are meant to be marked in METADATA so that they are not installed, but as you can see, there are issues in managing that complexity. 

You may be better off running the 0.2 pre-release versions. 

Regards

-

Avik

[Viral Shah]

I have updated the downloads page to suggest that people should not download v0.1.2 unless they have old code they wish to run with it.

-viral

[Sophie McCoy]

I am also having a problem installing the Distributions package, running the 0.2.0 Pre-release version (after having issues with this same package in the older version).  However, I am able to install other packages, e.g. DataFrames, in both.  I have tried removing the 'git' file, as suggested above, and updating XCode and git.  Any ideas?  

julia> Pkg.add("Distributions")

INFO: Cloning cache of Distributions from git://github.com/JuliaStats/Distributions.jl.git

warning: templates not found /Applications/Julia-0.2.0-pre-bd41367824.app/Contents/Resources/julia/julia/share/git-core

INFO: Cloning cache of NumericExtensions from https://github.com/lindahua/NumericExtensions.jl.git

warning: templates not found /Applications/Julia-0.2.0-pre-bd41367824.app/Contents/Resources/julia/julia/share/git-core

fatal: Unable to find remote helper for 'https'

ERROR: failed process: Process(`git clone -q --mirror https://github.com/lindahua/NumericExtensions.jl.git /Users/sophiemccoy/.julia/.cache/NumericExtensions`, ProcessExited(128)) [128]

 in error at error.jl:22

 in pipeline_error at process.jl:447

 in run at process.jl:424

 in prefetch at pkg/cache.jl:12

 in _resolve at pkg/entry.jl:247

 in _resolve at pkg/entry.jl:223

 in edit at pkg/entry.jl:15

 in add at pkg/entry.jl:22

 in anonymous at pkg/dir.jl:31

 in cd at file.jl:22

 in cd at pkg/dir.jl:31

 in add at pkg.jl:14

Thanks, 

Sophie

[John Myles White]

I think the problem is that NumericExtensions uses https:// in its METADATA.jl entry. I can fix this, but it would be great to make Pkg more robust to this sort of thing.

-- John

[Stefan Karpinski]

This appears to be a git problem, not a Pkg problem. Pkg doesn't really care about the URL protocol one way or the other as long as git can clone it and pull form it.

[John Myles White]

Why can't Pkg pull apart the URLs it finds in METADATA and then try multiple versions until Git succeeds? And why can't Pkg find which URL scheme works for the currently installed version of Git and then do standardization of URLs at run time?

-- John

[Stefan Karpinski]

It could, but for non-GitHub that's a pretty meaningless operation. It's like changing the protocol from HTTPS to FTP or SMTP and hoping it will somehow miraculously work; GitHub happens to make those things work in similar ways but there's no reason to suppose that will work with any non-GitHub URL. The fact that you even have so much choice is entirely GitHub-specific. I'm also unclear about why git would have trouble with any of the standard URL schemes – git, http, ssh and https have all been fully supported by git for years.

[John Myles White]

Fair enough. In that case, I think we need more aggressive patrolling of METADATA that our Travis tests offer. It's just silly that doing installs fail because of minor things like this.

I'm going to change the METADATA entry for NumericExtensions now.

-- John

[Isaiah Norton]

I thought that https:// is preferred because it is less likely to hit corporate firewalls than git://

[Patrick O'Leary]

It appears git has a cURL dependency for HTTPS.

[Elliot Saba]

This error means that the GIT_EXEC_PATH environment variable is not getting set correctly. Sophie, when you do the following from inside of julia, what is the output:

run(`which git`)

println(ENV)

[Mohit Sharma]

I am having the same error on mac:

GIT_EXEC_PATH=/Applications/Julia-0.2.0-pre-bbf8dd8266.app/Contents/Resources/julia/julia/libexec/git-core

GIT_TEMPLATE_DIR=/Applications/Julia-0.2.0-pre-bbf8dd8266.app/Contents/Resources/julia/julia/share/git-core

looks like its adding Julia twice in the path, whereas actual path has only single.

can you guys suggest how to change this path?

[Elliot Saba]

This was an error; corrected in https://github.com/JuliaLang/julia/commit/8af04af83b05ccc4584addeca11b90e3e0d4dd9b

The OSX build should be updated now.  Please redownload and try again, and see if the issue persists.

-E

[Sophie McCoy]

Thanks, Elliot!  It is working!  

Thank you all so much for your help.  

s