[ANN] conda-zsh-completion v0.4

[Valentin Haenel]

Hi,

after a few more days of tinkering I am happy to announce v0.4 of the
conda-zsh-completion (v0.2 and v0.3 were not announced to minimize list
traffic/spam):

https://github.com/esc/conda-zsh-completion/releases/tag/v0.4

Some examples:

* Rudimentary package version completion has been added:
E.g. if you type:

$ conda install numpy=<TAB>

This will offer all available numpy versions for completion.

* Commands are sensitive to -n,--name,-p,--prefix.
E.g. if you type:

$ conda remove -n demoenv <TAB>

Only packages actually installed in 'demoenv' are completed.

* All configurations are sensitive to the contents of .condarc.
E.g. if you type:

$ conda config --get <TAB>

Only existing keys will be completed>
Likewise if you type:

$ conda config --remove channels <TAB>

Only configured 'channels' will be offered.

Lastly, there is an asciicast showcasing some of the features here:

https://asciinema.org/a/16516

All in all, it has become quite a hack (as was to be expected for shell
completions written in shell) and I fear there are still a
number of glitches (which is normal and made worse by the complete
absence of tests), so please report any issues you may have to:

https://github.com/esc/conda-zsh-completion/issues

best wishes and enjoy!

V-

[Aaron Meurer]

That's great.

Not to be outdone, I have added support for most of this to conda
itself bash with argcomplete (see
http://conda.pydata.org/docs/config.html#tab-completion). I couldn't
add the version completion because of an issue with argcomplete
(https://github.com/kislyuk/argcomplete/issues/94). Completion of
package names, environment names, and channel names for bash
(argcomplete) will be in the next version of conda.

I still contend that someone should add zsh support to argcomplete,
and then both will get improvements automatically. In the meantime,
I'll add a link to this to the conda tab completion docs.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups "conda - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to conda+un...@continuum.io.
> To post to this group, send email to co...@continuum.io.
> Visit this group at http://groups.google.com/a/continuum.io/group/conda/.

[Valentin Haenel]

Hi Aaron,

* Aaron Meurer <aaron....@continuum.io> [2015-02-16]:

> Not to be outdone, I have added support for most of this to conda
> itself bash with argcomplete (see
> http://conda.pydata.org/docs/config.html#tab-completion). I couldn't
> add the version completion because of an issue with argcomplete
> (https://github.com/kislyuk/argcomplete/issues/94). Completion of
> package names, environment names, and channel names for bash
> (argcomplete) will be in the next version of conda.

Thanks for you work on this, having good tab-completion is essential
when using UNIX tools in the 21st century! I just looked at the current
master of conda and it seems to work well. Two nitpicks I discovered
while manually fuzzing the implementation.

$ conda install -c <TAB>

Doesn't seem to offer configured channels?

$ conda config --remove <TAB>

Should include list configuration keys?

I must confess, that I haven't implemented context sensitivity for the
-c,--channel option, because I wasn't sure how this is best done.
After some more deliberation, I figure that only 'install' should be aware
of it? Like:

$ conda install -c esc --override-channels <TAB>

You could do it for update too, but then the list of offered packages
should be the intersection between those already installed in the
environment -- potentially specified by -n or similar -- and those
available from the channel specified.

Should only complete packages from the 'esc' channel on binstar (or
whereever the default is pointing).

Also, I initially tried to build support for more advanced package
specs, e.g. <= and >= and so on, but wasn't sure if conda even supported
this. Even if it did, one would have to auto-quote or auto-escape
certain characters that would otherwise be interpreted by the shell.
E.g. > < for redirection. And that would probably be very, very messy.
So I decided that just supporting '=' is a good trade-off.

> I still contend that someone should add zsh support to argcomplete,
> and then both will get improvements automatically.

Yes, DRYing the code up would be quite beneficial. Having to copy and
paste strings etc.. across is really not that great style. Then again,
having to do two very similar tasks in fairly different languages does
allow us to compare notes.

> In the meantime, I'll add a link to this to the conda tab completion
> docs.

Please do!

V-

[Aaron Meurer]

On Mon, Feb 16, 2015 at 3:04 PM, Valentin Haenel <vale...@haenel.co> wrote:
> Hi Aaron,
>
> * Aaron Meurer <aaron....@continuum.io> [2015-02-16]:
>> Not to be outdone, I have added support for most of this to conda
>> itself bash with argcomplete (see
>> http://conda.pydata.org/docs/config.html#tab-completion). I couldn't
>> add the version completion because of an issue with argcomplete
>> (https://github.com/kislyuk/argcomplete/issues/94). Completion of
>> package names, environment names, and channel names for bash
>> (argcomplete) will be in the next version of conda.
>
> Thanks for you work on this, having good tab-completion is essential
> when using UNIX tools in the 21st century! I just looked at the current
> master of conda and it seems to work well. Two nitpicks I discovered
> while manually fuzzing the implementation.
>
> $ conda install -c <TAB>
>
> Doesn't seem to offer configured channels?

I'm sorry I said channel names above. That was me misremembering. It
isn't supported.

Listing the configured channels isn't very helpful. They are already
included. Adding a configured channel with -c is redundant (unless you
are reordering your channels at the command line, which isn't very
common).

What you'd really want is to complete the channels from binstar, but I
don't know a non-slow way to do this.

>
> $ conda config --remove <TAB>
>
> Should include list configuration keys?

conda config --remove is something that barely works (i.e., you have
to use --force with it, and if you do use it, you'll have to use
--force with every conda config command thereafter). Hence, I have
only minimally supported it, until I can make it work better.

>
> I must confess, that I haven't implemented context sensitivity for the
> -c,--channel option, because I wasn't sure how this is best done.
> After some more deliberation, I figure that only 'install' should be aware
> of it? Like:
>
> $ conda install -c esc --override-channels <TAB>
>
> You could do it for update too, but then the list of offered packages
> should be the intersection between those already installed in the
> environment -- potentially specified by -n or similar -- and those
> available from the channel specified.

Right, there are two sets of package to complete. For install, you
want to list all packages from the channels (taking into account flags
like -c). For update and remove, you want to list only those packages
that are in the environment (taking into account -n and -p).

>
> Should only complete packages from the 'esc' channel on binstar (or
> whereever the default is pointing).
>
> Also, I initially tried to build support for more advanced package
> specs, e.g. <= and >= and so on, but wasn't sure if conda even supported
> this. Even if it did, one would have to auto-quote or auto-escape
> certain characters that would otherwise be interpreted by the shell.
> E.g. > < for redirection. And that would probably be very, very messy.
> So I decided that just supporting '=' is a good trade-off.

It does support them, but you have to quote them on the command line,
like conda install 'python<3'. I'm assuming that works the same in zsh
as bash anyway.

Anyway, I'm pretty happy with the state of tab completion for conda
now that we can complete packages and environment names. The next
thing I would want to see is to extend completion to conda build and
conda env (see https://github.com/kislyuk/argcomplete/issues/76).

>
>> I still contend that someone should add zsh support to argcomplete,
>> and then both will get improvements automatically.
>
> Yes, DRYing the code up would be quite beneficial. Having to copy and
> paste strings etc.. across is really not that great style. Then again,
> having to do two very similar tasks in fairly different languages does
> allow us to compare notes.

Two advantages of argcomplete:

- We can reuse some of the complicated logic inside of conda for
context sensitive completions (like the logic used to complete package
names when -c, --override-channels, --use-local, etc. are used).

- New flags are added to the completer automatically.

>
>> In the meantime, I'll add a link to this to the conda tab completion
>> docs.
>
> Please do!

It's been done. http://conda.pydata.org/docs/config.html#tab-completion

Aaron Meurer

>
> V-

[Valentin Haenel]

Hi,

* Aaron Meurer <aaron....@continuum.io> [2015-02-16]:

> On Mon, Feb 16, 2015 at 3:04 PM, Valentin Haenel <vale...@haenel.co> wrote:

[snip]

> > * Aaron Meurer <aaron....@continuum.io> [2015-02-16]:

> > $ conda install -c <TAB>
> >
> > Doesn't seem to offer configured channels?
>
> I'm sorry I said channel names above. That was me misremembering. It
> isn't supported.
>
> Listing the configured channels isn't very helpful. They are already
> included. Adding a configured channel with -c is redundant (unless you
> are reordering your channels at the command line, which isn't very
> common).

Yes, I agree. I may implement it anyway, but see below.

> What you'd really want is to complete the channels from binstar, but I
> don't know a non-slow way to do this.

> > $ conda config --remove <TAB>
> >
> > Should include list configuration keys?
>
> conda config --remove is something that barely works (i.e., you have
> to use --force with it, and if you do use it, you'll have to use
> --force with every conda config command thereafter). Hence, I have
> only minimally supported it, until I can make it work better.

Oh, I don't really understand how the --force comes about. But OK, I
doubt many people will use this.

> > I must confess, that I haven't implemented context sensitivity for the
> > -c,--channel option, because I wasn't sure how this is best done.
> > After some more deliberation, I figure that only 'install' should be aware
> > of it? Like:
> >
> > $ conda install -c esc --override-channels <TAB>
> >
> > You could do it for update too, but then the list of offered packages
> > should be the intersection between those already installed in the
> > environment -- potentially specified by -n or similar -- and those
> > available from the channel specified.
>
> Right, there are two sets of package to complete. For install, you
> want to list all packages from the channels (taking into account flags
> like -c). For update and remove, you want to list only those packages
> that are in the environment (taking into account -n and -p).

Okay, taking into account additional channels, i.e. those given with -c
but not listed in .condarc raises some issues for the package cache. I
may have to do some more advanced cache management by storing the
channels the packages came from too and invalidate if new channels were
added, for example. That may be a good thing anyway, because any changes
to the .condarc would also be detected.

As for 'conda update', to clarify: imagine I use -c CHANNEL
--overide-channels *and* -n NAME, in that case, only those packages
installed in NAME *and* available from CHANNEL could be updated,
correct?

> > Should only complete packages from the 'esc' channel on binstar (or
> > whereever the default is pointing).
> >
> > Also, I initially tried to build support for more advanced package
> > specs, e.g. <= and >= and so on, but wasn't sure if conda even supported
> > this. Even if it did, one would have to auto-quote or auto-escape
> > certain characters that would otherwise be interpreted by the shell.
> > E.g. > < for redirection. And that would probably be very, very messy.
> > So I decided that just supporting '=' is a good trade-off.
>
> It does support them, but you have to quote them on the command line,
> like conda install 'python<3'. I'm assuming that works the same in zsh
> as bash anyway.

Yes it does. Am I correct in assuming that the following package match
specifications are valid on the command line too:

http://conda.pydata.org/docs/spec.html#package-match-specifications

> Anyway, I'm pretty happy with the state of tab completion for conda
> now that we can complete packages and environment names. The next
> thing I would want to see is to extend completion to conda build and
> conda env (see https://github.com/kislyuk/argcomplete/issues/76).

Yes, 'env' is next on my list, since it is currently the only way to
remove environments? Shouldn't be too hard.

> >> I still contend that someone should add zsh support to argcomplete,
> >> and then both will get improvements automatically.
> >
> > Yes, DRYing the code up would be quite beneficial. Having to copy and
> > paste strings etc.. across is really not that great style. Then again,
> > having to do two very similar tasks in fairly different languages does
> > allow us to compare notes.
>
> Two advantages of argcomplete:
>
> - We can reuse some of the complicated logic inside of conda for
> context sensitive completions (like the logic used to complete package
> names when -c, --override-channels, --use-local, etc. are used).
>
> - New flags are added to the completer automatically.

Yes, I agree 100%, that would be the best solution. For example: I
haven't even taken --use-local into account.

> >> In the meantime, I'll add a link to this to the conda tab completion
> >> docs.
> >
> > Please do!
>
> It's been done. http://conda.pydata.org/docs/config.html#tab-completion

Cool! Let's hope it gets some usage.

best wishes and thanks again!

V-

[Aaron Meurer]

The problem is that pyyaml doesn't have a roundtripping yaml parser,
so if you just use yaml.dump() to write the configuration, it will
stip all comments and formatting from the .condarc (and not even
format it very prettily). To get around this, I wrote a custom but
hacky "parser" that adds the keys and keeps the formatting, but I
never got around to adding support for --remove. So to --remove, you
have to use --force, which uses yaml.dump(), which clears the
formatting and (I believe) formats it in a way that my "parser"
doesn't recognize (hence --force is required for future commands).
Fixing this is on the TODO list.

If you ever want an exercise in writing a parser in Python, write a
roundtripping YAML parser.

>
>> > I must confess, that I haven't implemented context sensitivity for the
>> > -c,--channel option, because I wasn't sure how this is best done.
>> > After some more deliberation, I figure that only 'install' should be aware
>> > of it? Like:
>> >
>> > $ conda install -c esc --override-channels <TAB>
>> >
>> > You could do it for update too, but then the list of offered packages
>> > should be the intersection between those already installed in the
>> > environment -- potentially specified by -n or similar -- and those
>> > available from the channel specified.
>>
>> Right, there are two sets of package to complete. For install, you
>> want to list all packages from the channels (taking into account flags
>> like -c). For update and remove, you want to list only those packages
>> that are in the environment (taking into account -n and -p).
>
> Okay, taking into account additional channels, i.e. those given with -c
> but not listed in .condarc raises some issues for the package cache. I
> may have to do some more advanced cache management by storing the
> channels the packages came from too and invalidate if new channels were
> added, for example. That may be a good thing anyway, because any changes
> to the .condarc would also be detected.
>
> As for 'conda update', to clarify: imagine I use -c CHANNEL
> --overide-channels *and* -n NAME, in that case, only those packages
> installed in NAME *and* available from CHANNEL could be updated,
> correct?

conda update only works if the package is already installed, so it
only needs to care about the list of installed packages (same as
remove). For install, you are correct, though.

>
>
>> > Should only complete packages from the 'esc' channel on binstar (or
>> > whereever the default is pointing).
>> >
>> > Also, I initially tried to build support for more advanced package
>> > specs, e.g. <= and >= and so on, but wasn't sure if conda even supported
>> > this. Even if it did, one would have to auto-quote or auto-escape
>> > certain characters that would otherwise be interpreted by the shell.
>> > E.g. > < for redirection. And that would probably be very, very messy.
>> > So I decided that just supporting '=' is a good trade-off.
>>
>> It does support them, but you have to quote them on the command line,
>> like conda install 'python<3'. I'm assuming that works the same in zsh
>> as bash anyway.
>
> Yes it does. Am I correct in assuming that the following package match
> specifications are valid on the command line too:
>
> http://conda.pydata.org/docs/spec.html#package-match-specifications

No, these are a little different. The command line supports
inequalities, but there are no spaces. The command line options are
like

sympy
sympy=0.7.6
sympy=0.7.6=py27_0
sympy=0.7
sympy==0.7.6
'sympy<0.7.6'
'sympy<=0.7.6'
'sympy>0.7.6'
'sympy>=0.7.6'
'sympy!=0.7.6'

(the quoting is required because < and > are special characters in the shell)

>
>> Anyway, I'm pretty happy with the state of tab completion for conda
>> now that we can complete packages and environment names. The next
>> thing I would want to see is to extend completion to conda build and
>> conda env (see https://github.com/kislyuk/argcomplete/issues/76).
>
> Yes, 'env' is next on my list, since it is currently the only way to
> remove environments? Shouldn't be too hard.

conda remove -n envname --all. Or rm -rf

Aaron Meurer