a few ideas to help beginners use Pkg.X() functions

[Kevin Owens]

1. Alias Pkg.install to Pkg.add, and then give a depreciated warning. When you install a package the message is

INFO: Installing

not

INFO: adding

2. If a package isn't found, check if the last three characters are ".jl", and then prompt the use if they want to install it without the extension. It's very natural to review a github page, think a a package is cool, and then copy the title .. whoops, package not found.

[Leah Hanson]

Those seem like good ideas to me, especially #2.

-- Leah

[Stefan Karpinski]

Just to clarify, "add" refers to adding the requirement for the package to the REQUIRE file. Which may or may not result in installing it. If it's already installed because of a dependency on it from another package, then it will be added without being installed. But this whole business seems to confuse people and probably needs a rethink.

[wil...@gmail.com]

I am in favor of minimalist interface for package management. Keep only: add (install), rm (uninstall), status, update, clone (download), test. The rest of commands (register, tag, submit, publish, checkout, pin, free, generate) should go to separate package for developers.

Create "Pkg.version" for showing list of package versions and switching between them. Try to distance from git syntax for common users.

[Tom Breloff]

I would go one step further and merge "clone" into "add".  Then the primary commands would be: add, rm, status, update, test.

Pkg.add("ABC")  # gets the registered package
Pkg.add("StefanKarpinski/TheNextBigThing.jl")  # similar to clone
Pkg.add("ABC", "master")  # also does checkout (does this already work??)

[Jeffrey Sarnoff]

I go a half-step back, to allow, in addition to rm, rm!, the latter really wiping the package from the pkg directory and the pkg cache directory.

Sometimes it is necessary to force fully reobtaining and regenerating a package, and this process can be derailed by left-overs. 

[Stefan Karpinski]

Honestly, I feel like the "manipulate the REQUIRE file" and let that imply things about installation / removal experiment has kind of failed. People find it unintuitive and confusing. I'm not sure what's better though. I wanted to avoid the kind of situation that happens with a lot of package managers where you have no idea what's installed because you actually want it and what's installed because some version of something you once wanted once needed it.

[Jeffrey Sarnoff]

+1 "avoid .. you have no idea what's .. wanted .."

The underlying intent is still good -- the way that has been constructed focused more on the internal oragami and less on providing semiotic resonance in the use.

(a) request a package (to be provided if not present, to be made current if the present one is not, to be graciously [un]informed if current one is present)

(b) divest a package (go-away without hiding some of yourself, do no harm if not present, divest every dependancy orphaned therewith [at least in the default case])

(c) list subsets of packages they have or subsets of those they do not have (packages that relate to e.g. graphics do they have/exist, find the package[s] that "...") 

(d) make current whatever is present while keeping dependancies and required system libs all matchy-matchy.

(e) clean up after itself .. do not accrete logs of event histories beyond what is needed for proper functioning unless that is demanded

[Jeffrey Sarnoff]

(f) inquire "are there any packages present that have not been used in the past <timespan>"

[Leah Hanson]

I would find it helpful for `rm` to indicate what packages it's removing because they were only here as dependencies of the package being removed, and to give me a chance to say "oops, stop -- I want to go prevent that package from being deleted -- it takes forever to reinstall". And it I do say "no, stop", it should say "here's how to save a package from being deleted this way" (i.e. use Pkg.add).

I have been surprised more than once by having installed Gadfly only as a dependency of something else, which is fine until I rm the something-else and then have to install Gadfly and a small army of its dependencies again.

I haven't removed a package for a while, so this could already have changed without me knowing.

-- Leah

[Stefan Karpinski]

On Sat, Jul 4, 2015 at 6:37 AM, Leah Hanson <astri...@gmail.com> wrote:

I have been surprised more than once by having installed Gadfly only as a dependency of something else, which is fine until I rm the something-else and then have to install Gadfly and a small army of its dependencies again.

Note that the .trash approach means that this should be quite quick since all you need to do is move some directories around. This is one of the reasons I did it that way.

[Tony Kelman]

Though the .trash thing can easily lead to some brokenness with BinDeps. I had a proposal at https://github.com/JuliaLang/julia/issues/11262 for one way to maybe mitigate the "why doesn't Pkg.rm actually delete the package and let me start over from scratch" issue, but haven't gotten around to sketching out a prototype PR.

I like the ease, automation, and uniformity of having package-development features built in from the get go, but I suppose it's true that those features could likely be decoupled from the development cadence of base eventually.

[Carlo Baldassi]

From what I've seen, it seems to me that actually the most common issues are with "rm" rather than with "add".

It seems things mostly go like this: people installs packages with "add" until everything's fine. They may wonder why it's not called "install", and even propose to rename it, but they don't mind too much. Then something goes wrong with a package, at which point the standard immediate reaction is to try the "remove and reinstall" routine, which of course may fail in so many ways because "rm" does not really remove the package (it may actually upgrade it!)

And indeed, people get confused by this because they try stuff expecting things to work in certain way to which they're used (rather than reading the documentation, which is generally a hopeless thing to expect).

So it seems there basically a few main ways to address this:
1) revert to a more "standard" model abandoning any hope of innovation so that commands have standard names, they do what people expect, and nobody needs to understand any new thing
2) keep the current scheme but be clearer in the messages produced by the commands, so that at least people could grasp what's going on; provide convenience commands for the common situation of a broken package (or others)
3) somehow mix the two, i.e. have two layers of API, one with "install" and "delete" and another more fine-grained with "add" and "rm"

You probably can already tell I'm not a fan of the first option. Anyway, the way this is done e.g. by apt-get is to just remember which packages were explicitly installed, and which ones were just installed as dependencies. Then, there is an "autoremove" command which deletes all packages in the latter group which are no longer depended upon; you basically just do a periodic cleanup. Also, when you try to remove a package which is depended upon, you get a message telling you that if you want to remove it it will also uninstall those other packages and are you really sure? So one possibility could be following that model I suppose. It's not terrible after all.

I have no idea how the third option could work, but maybe someone more inventive than me does.

About the second option, the main change would be this: when a user calls Pkg.rm("SomePackage"), one of these things can happen
1) the package was not required and not installed: produce an error
2) the package was required, and nothing depended on it: uninstall it and that's it
3) the package was required, but it's also required by other packages: print a message detailing that the package was not removed because other packages (print which ones) depend on it
4) the package was not required, and it's only installed because of other packages: print a message which says this, and provides details like in the previous case.

Hopefully, this would remove the confusion about "rm" (at least to anyone who bothers to read the messages). Generating the detailed messages would be easy.
More details (e.g. what Leah was proposing) could also be easily added.
Some more details could be provided by Pkg.add as well (e.g. "package XYZ was already installed because...")
The other thing would be to provide some kind of "purge" or "nuke" command, which I believe was already proposed at some point.


Oh and I'm also not a fan of removing stuff from Base just because it's "not minimalist".

[Carlo Baldassi]

I have no idea how the third option could work

well, maybe something like this could  be made to work: we introduce two new commands, "install" and "uninstall". These would have an "apt-get" like semantics, "install" could take URLs as argument etc. "uninstall" would need to be interactive by default, just like "apt-get remove" is. These would be the commands new users are pointed to and are encouraged to use first. Under the hood, these commands would use "add" and "rm" and actually manipulate the REQUIRE file. If desired, a "reinstall" command could be provided to do this: remember the state of REQUIRE, call a forced "uninstall", followed by "install" of everything which was removed (so that the state of REQUIRE is hopefully restored). The current commands could be preserved to provide a more fine-grained approach (kind of like the "plumbing" vs "porcelain" git commands - not that I'm suggesting to look at git as an API inspiration!).

The only part of this which would require some amount of non-trivial implementation is the "uninstall" command, because it would need to analyse the dependency graph, and call "resolve" until the desired result of removing the package is achieved. Not too hard though. The same code could also be reused to produce the more detailed "rm" messages I mentioned above.

In order to clarify the distinction in API level, we could even call the new commands something like "pkginstall" etc. and have them under Base, so that Pkg is reserved for more the middle-level operations.

Thoughts?

[Tomas Lycken]

Regardless of command names and semantics, I think the decision to keep all things package-related in their own module was a genius move; as soon as we can precompile packages and include them in the system image, the Pkg module can be broken out and put in e.g. Pkg.jl, to be included in all distributions of Julia. The main motivation to do this, would be to allow for other, parallel, package managers that might want to handle things differently. In other words, we still provide a built-in, out-of-the-box solution for package management, but if you're the innovative kind and have ideas on how to make things better, the threshold to trying them out is much smaller.

// T