Pkg2.add removing packages?

[Andreas Lobinger]

Hello colleagues,

i guess there is a good reason, however, it's not so obvious:

lobi@maroon:~$ julia/julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0-prerelease+3134
 _/ |\__'_|_|_|\__'_|  |  Commit 888ca27 2013-08-09 16:25:25 UTC
|__/                   |  i686-linux-gnu


julia> Pkg2.add("PyCall")
INFO: Cloning cache of PyCall from git://github.com/stevengj/PyCall.jl.git
INFO: Installing PyCall v0.0.0
INFO: Removing IniFile v0.2.1
INFO: Removing Winston v0.3.0
INFO: Removing Tk v0.2.4
INFO: SUCCESS!
INFO: REQUIRE updated.

julia> using Winston
ERROR: could not open file /home/lobi/Winston.jl
 in include_from_node1 at loading.jl:92
 in reload_path at loading.jl:117
 in require at loading.jl:48

?

Wishing a happy day,
        Andreas

[Stefan Karpinski]

It will remove anything that isn't fixed and isn't needed to satisfy your list of required packages.

[Andreas Lobinger]

Hello colleagues,

On Monday, August 12, 2013 6:58:46 PM UTC+2, Stefan Karpinski wrote:

It will remove anything that isn't fixed and isn't needed to satisfy your list of required packages.

that's what i meant with 'not obvious'. What it the meaning of 'not fixed' and 'isn't needed' in this context?

I'm not questioning the functionality of Pkg2, it's just from user perspective: I'm asking for the addition of a package, and three are removed...

[Stefan Karpinski]

A package is "fixed" if you have a branch checked out or have uncommitted content in it. These are installed packages that the version resolution system cannot change or uninstall. In this case you could have run Pkg2.resolve() right before the add and it would have deleted those packages first. That's not a state you could have gotten in just by using the Pkg2 API – maybe you edited your ~/.julia/REQUIRE file by hand at some point and never re-resolved afterwards?

There are other ways that Pkg2.add can remove packages: if a newer version of some required package has fewer requirements, they no-longer-necessary ones will be removed. That's not the situation here, however, because nothing is being upgraded.

[Kevin Squire]

I also recently noticed a situation where it seemed that Pkg2 didn't have correct dependency information, and removed dependent packages of a package it had just installed.  I can't seem to reproduce it, however, so it's likely I did something fishy, like run Pkg in a separate window or modify something.

Is there a timeframe for the Pkg2 -> Pkg rename?

Kevin

[Stefan Karpinski]

This week. I could do a Pkg, Pkg2 => Pkg1, Pkg switch right now.

[J Luis]

Well, for me it doesn't even install anything.
For example

 | | |_| | | | (_| |  |  Version 0.2.0-prerelease+3206
 _/ |\__'_|_|_|\__'_|  |  Commit 4f3276c* 2013-08-12 23:38:46 UTC
|__/                   |  x86_64-w64-mingw32

julia> Pkg2.add("MATLAB")
INFO: Cloning cache of MATLAB from git://github.com/lindahua/MATLAB.jl.git
ERROR: no method convert(Type{SubString{UTF8String}},ASCIIString)
 in setindex! at dict.jl:400
 in gitenv at git.jl:20
 in prefetch at pkg2/cache.jl:12WARNING:
backtraces on your platform are often misleading or partially incorrect

[Jameson Nash]

I can't find the commit that you are on, but #4032 that you are referring to should have been fixed as of 11:50pm UTC

[J Luis]

Ah, I missed that one.
But now, sorry to say it this way, as usual as I think I never managed to install Winston/Cairo

julia> Pkg2.add("Winston")
INFO: Cloning cache of BinDeps from git://github.com/loladiro/BinDeps.jl.git
INFO: Cloning cache of Winston from git://github.com/nolta/Winston.jl.git
INFO: Cloning cache of Cairo from git://github.com/JuliaLang/Cairo.jl.git
INFO: Cloning cache of Color from git://github.com/JuliaLang/Color.jl.git
INFO: Cloning cache of IniFile from git://github.com/JuliaLang/IniFile.jl.git
INFO: Cloning cache of Tk from git://github.com/JuliaLang/Tk.jl.git
INFO: Cloning cache of URIParser from git://github.com/loladiro/URIParser.jl.git
INFO: Installing BinDeps v0.2.2
INFO: Installing Winston v0.3.1
INFO: Installing Cairo v0.2.6
INFO: Installing Color v0.2.4
INFO: Installing IniFile v0.2.1
INFO: Installing Tk v0.2.4
INFO: Installing URIParser v0.0.0
INFO: Running build script for package Cairo
WARNING: An exception occured while building binary dependencies.
You may have to take manual steps to complete the installation, see the error message below.
To reattempt the installation, run Pkg.fixup("Cairo").

WARNING:
backtraces on your platform are often misleading or partially incorrect

ERROR: could not start process `curl --help`: no such file or directory (ENOENT)
 in test_success at process.jl:420
at C:\j\.julia\Cairo\deps\build.jl:222

julia> Pkg.fixup("Cairo")
ERROR: fixup not defined

Runing

Pkg2.fixup("Cairo")

tells me to fix Tk and

julia> Pkg2.fixup("Tk")
INFO: Running build script for package Tk
Trying to load C:\j\.julia\Tk\deps\usr\lib\tcl...
Trying to load C:\j\.julia\Tk\deps\usr\lib\libtcl8.6...
Trying to load C:\j\.julia\Tk\deps\usr\lib\libtcl...
Trying to load C:\j\.julia\Tk\deps\usr\lib\libtcl8.6.so.0...
Trying to load C:\j\.julia\Tk\deps\usr\lib\libtcl8.5...
Trying to load C:\j\.julia\Tk\deps\usr\lib\libtcl8.5.so.0...
Trying to load tcl...
Trying to load libtcl8.6...
Trying to load libtcl...
Trying to load libtcl8.6.so.0...
Trying to load libtcl8.5...
Trying to load libtcl8.5.so.0...
WARNING: An exception occured while building binary dependencies.
You may have to take manual steps to complete the installation, see the error message below.
To reattempt the installation, run Pkg.fixup("Tk").

ERROR: could not start process `curl --help`: no such file or directory (ENOENT)
 in test_success at process.jl:420
at C:\j\.julia\Tk\deps\build.jl:19

[Jameson Nash]

I'm not sure what BinDeps is trying to do on windows, but you can use RPMmd to get all the dependencies manually (I've been meaning to announce this formally, but never really got around to it):

Pkg2.update()

Pkg2.add("RPMmd")

import RPMmd

RPMmd.help()

RPMmd.update()

map(RPMmd.install, ["tk","libcairo2","pango"])

RPMbindir = Pkg.dir("RPMmd","deps","usr","$(Sys.ARCH)-w64-mingw32","sys-root","mingw","bin")

ENV["PATH"]=ENV["PATH"]*";"*RPMbindir

Other packages you may be interested in are:

curl / libcurl (for downloading stuff)

win_iconv (for passing the unicode tests)

gtk2 / gtk3 (for using the Gtk backend)

libxml2 (for heavyweight xml parsing)

zlib / libexpat (these are bundled with julia via `make win-extras`, for bootstrapping RPMmd)

[J Luis]

Pkg2.update()

Pkg2.add("RPMmd")

import RPMmd

julia> import RPMmd
WARNING: skipping repodata/repomd.xml, not in cacheWARNING:

backtraces on your platform are often misleading or partially incorrect

WARNING: skipping repodata/repomd.xml, not in cache
 

RPMmd.help()

After many seconds of cpu at 100% it crashes the console.
 

RPMmd.update()

julia> RPMmd.update()
INFO: Downloading http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_Factory//repodata/repomd.xml
Trying to load urlmon...
Trying to load libexpat-1...
ERROR: error compiling xp_parse: error compiling xp_make_parser: could not load module libexpat-1: %1 is not a valid Win32 application.

 in update at C:\j\.julia\RPMmd\src\RPMmd.jl:105
 in update at C:\j\.julia\RPMmd\src\RPMmd.jl:69


I do have most, if not all, of those dependencies built native with VS. It would be nice if one could just use them

 

[Jameson Nash]

 >I do have most, if not all, of those dependencies built native with VS. It would be nice if one could just use them

If your native VS build works, then great, your packages are already installed and you are all set to just use them -- the BinDeps error doesn't matter.

If not, you will probably need to follow the directions here. I can't support random unknown configurations. Note: The files in the `make win-extras` target are essential for bootstrapping and must be copied to the appropriate places in usr (the `make dist` target does this for you`). I've noticed that git provides some incompatible copies of some of these libraries (zlib and expat, IIRC) which crash julia if they get loaded instead.

[J Luis]

If your native VS build works, then great, your packages are already installed and you are all set to just use them -- the BinDeps error doesn't matter.

If not, you will probably need to follow the directions here. I can't support random unknown configurations.

Well I understand that but neither of Pkg2 or RPMmd worked.

 

[Stefan Karpinski]

On Tue, Aug 13, 2013 at 5:15 PM, Jameson Nash <jam...@mit.edu> wrote:

If your native VS build works, then great, your packages are already installed and you are all set to just use them -- the BinDeps error doesn't matter.

This is not really a great way to indicate success. I realize that Windows is still a bit of an awkward platform for many things, but can't we do a bit better than that? This whole process of installing and then manually calling Pkg2.fixup() and then trying things to see if they work is not exactly smooth.