Re: bash tab variable expansion question?
Clark J. Wang
>
> The attached patch adds a new shell option that, when enabled, is
> intended to restore the bash-4.1 behavior of expanding directory names
> in filenames being completed. I have done some testing, and it seems
> to work the way I intend. This, or some later version, will be part
> of the next bash release. I am soliciting feedback on this iteration.
>
> I'm sending this directly to everyone who's commented negatively about
> the default bash-4.2 behavior, as well as bug-bash. Please try the new
> option (`direxpand') and let me know if it's missing anything. The patch
> includes the original heuristic I sent out back in March, the new shopt
> option, and updates to the documentation and test suite. It should apply
> cleanly to bash-4.2.10.
>
>
Tested with 4.2.10. Overall it works fine for me. But it still has problem
for following scenario:
$ complete -d -o bashdefault cd
$ cd $PWD<TAB>
# it expands to this:
$ cd \$PWD<SPACE>
Bash 4.1 also behaves like that so I'm not sure if it's OK.
> Chet
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
> ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, ITS, CWRU ch...@case.edu
> http://cnswww.cns.cwru.edu/~chet/
>
Martin von Gagern
>> Bash-4.2 tries to leave what the
>> user typed alone, but that leads to an inherently ambiguous situation:
>> when do you quote the `$' in a filename (or, in this case, a directory
>> name)? It could be a shell variable, and it could be a character in
>> the filename.
I'm not sure I'd EVER want completion to escape a $. If I want a
variable, I'll type $, either by itself or inside double quotation
marks. If I want a $ included in a file name, then I'll type \$ or
enclose it in single quotation marks. The above I'd do even for partial
file names. So if bash sees an unquoted $, then I'd call it a variable,
and no ambiguity involved.
> That is the problem, in a nutshell. I posted a partial patch at the
> end of March that applied a heuristic to avoid quoting variable
> expansions in most cases, but there was no way to get the bash-4.1
> behavior back.
Can you explain these heuristics?
> I am soliciting feedback on this iteration.
> Please try the new option (`direxpand') and let me know if it's missing anything.
The option works, as far as I can see.
"ls $HOM" completes to "ls $HOME " instead of "ls $HOME/", though.
The heuristics apparently don't work well with programmatic completion.
With _longopt <http://tinyurl.com/3va3hyz > from the debian bashcomp
project set up for ls, I get "\$HOME/.bash" as the completion for
"$HOME/.bash". And I get no completion at all for "ls $HOM", probably
because there is no dir of that name.
Can we do the following for programmatic completion?
1. If the insertion point is at a variable reference, and the
referenced variable does not exist, then complete variable name
without delegating to programmatic completion
2. Otherwise, store the original String for each word
3. Perform variable expansion for each word
4. Perform completion on the expanded words
5. In the completed words, search for the expanded but uncomleted
strings from 3. and replace them with the unexpanded forms from 2.
I'm not sure what behaviour I'd expect if the variable contains several
words. Don't care too much for that case, so I'll be happy with any
solution.
Greetings,
Martin von Gagern
Roger
>"ls $HOM" completes to "ls $HOME " instead of "ls $HOME/", though.
Weird.
On Gentoo here, "echo $HO" does complete to "echo $HOME", but "ls $HO"
fails to complete here.
(It is enabled here "$ eselect bashcomp list |grep coreutils". Just to
verify, I just wiped $HOME/.bash_completion.d/ and renabled everything and
still no completion using /bin/ls.)
--
Roger
http://rogerx.freeshell.org/
Roger
Here's what I have defined for complete:
$ complete |grep ls
complete -o default -F _longopt ls
--
Roger
http://rogerx.freeshell.org/
Chet Ramey
> Tested with 4.2.10. Overall it works fine for me. But it still has problem
> for following scenario:
>
> $ complete -d -o bashdefault cd
> $ cd $PWD<TAB>
> # it expands to this:
> $ cd \$PWD<SPACE>
>
> Bash 4.1 also behaves like that so I'm not sure if it's OK.
Out of scope. By default, readline always quotes matches as if they were
filenames, and relies on the application to turn that off where
appropriate. The -d option has the effect of telling readline to treat
the matches as filenames, which causes them to be quoted.
This points out a pair of opportunities, though: there is no option for
a progcomp to tell readline to *not* quote the completions as if they
were filenames, and it might be a good idea to save and restore the value
of the `rl_filename_completion_desired' option when `complete -f' or
`complete -d' returns no matches.
Chet Ramey
>> The option works, as far as I can see.
>> "ls $HOM" completes to "ls $HOME " instead of "ls $HOME/", though.
whether or not to append a slash to a directory name, and it doesn't
know that $HOME is a directory name. This has been the case for as
long as readline has existed.
>
> Weird.
>
> On Gentoo here, "echo $HO" does complete to "echo $HOME", but "ls $HO"
> fails to complete here.
completion code to attempt variable completion. You need a
`-o bashdefault' for that.
Chet Ramey
>> On Gentoo here, "echo $HO" does complete to "echo $HOME", but "ls $HO"
>> fails to complete here.
>>
>> (It is enabled here "$ eselect bashcomp list |grep coreutils". Just to
>> verify, I just wiped $HOME/.bash_completion.d/ and renabled everything and
>> still no completion using /bin/ls.)
>
> Here's what I have defined for complete:
>
> $ complete |grep ls
> complete -o default -F _longopt ls
Yep, no `-o bashdefault' here.
Chet Ramey
Hash: SHA1
On 9/5/11 2:05 AM, Martin von Gagern wrote:
> On -10.01.-28163 20:59, Chet Ramey wrote:
>>> Bash-4.2 tries to leave what the
>>> user typed alone, but that leads to an inherently ambiguous situation:
>>> when do you quote the `$' in a filename (or, in this case, a directory
>>> name)? It could be a shell variable, and it could be a character in
>>> the filename.
>
> I'm not sure I'd EVER want completion to escape a $. If I want a
> variable, I'll type $, either by itself or inside double quotation
> marks. If I want a $ included in a file name, then I'll type \$ or
> enclose it in single quotation marks. The above I'd do even for partial
> file names. So if bash sees an unquoted $, then I'd call it a variable,
> and no ambiguity involved.
You might, but `$' has the annoying habit of showing up far too often
in Windows filenames, or in shares mounted from Windows.
>> That is the problem, in a nutshell. I posted a partial patch at the
>> end of March that applied a heuristic to avoid quoting variable
>> expansions in most cases, but there was no way to get the bash-4.1
>> behavior back.
>
> Can you explain these heuristics?
Sure. Bash has to decide when to expand a variable name in the directory
name it's passed. Those names, for simplicity's sake, always start with
`$', so if bash successfully completes a variable name, it removes `$'
from the list of characters that will cause the filename to be quoted.
Similarly for ${...} and $(...), which cause `{' and `}' or `(' and `)'
to be removed also, respectively. This prevents the unwanted quoting of
`$', but is imperfect because it doesn't really handle the case where
other characters in the filename need to be quoted. The `$' will end
up being quoted if, for instance, the filename to be completed in $HOME
contains spaces.
This is at about line 160 of the diff I posted.
> The heuristics apparently don't work well with programmatic completion.
If the compspec is the one posted earlier, the options defeat the
heuristics.
> With _longopt <http://tinyurl.com/3va3hyz > from the debian bashcomp
> project set up for ls, I get "\$HOME/.bash" as the completion for
> "$HOME/.bash". And I get no completion at all for "ls $HOM", probably
> because there is no dir of that name.
>
> Can we do the following for programmatic completion?
> 1. If the insertion point is at a variable reference, and the
> referenced variable does not exist, then complete variable name
> without delegating to programmatic completion
The `-o bashdefault' option mostly handles this case. However, whether
or not to perform programmable completion depends on the command, not
the word to be completed.
> 2. Otherwise, store the original String for each word
> 3. Perform variable expansion for each word
> 4. Perform completion on the expanded words
> 5. In the completed words, search for the expanded but uncomleted
> strings from 3. and replace them with the unexpanded forms from 2.
This is a reasonable approach for a progcomp to take. It's not usually
that complicated -- you really only have one word you're trying to
complete, which may expand to multiple completions.
The other problem is what I explained in my message to Clark Wang: there
is no way for a compspec to tell readline to not quote the completion as
a filename, and no way to selectively tell readline which matches are and
which are not filenames.
Chet
- --
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU ch...@case.edu http://cnswww.cns.cwru.edu/~chet/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk5mxKYACgkQu1hp8GTqdKsuYACdH+UzQVvwOK497UYJdoyDn8Kv
WxYAn24GlCkw6s/GEw8drVlr6HmTqw72
=q0cG
-----END PGP SIGNATURE-----
Roger
>> On Mon, Sep 05, 2011 at 12:40:02AM -0800, Roger wrote:
>>>The option works, as far as I can see.
>>>"ls $HOM" completes to "ls $HOME " instead of "ls $HOME/", though.
>>
>>
>>
>>On Gentoo here, "echo $HO" does complete to "echo $HOME", but "ls $HO"
>>fails to complete here.
>>
>>(It is enabled here "$ eselect bashcomp list |grep coreutils". Just to
>>verify, I just wiped $HOME/.bash_completion.d/ and renabled everything and
>>still no completion using /bin/ls.)
>
>Here's what I have defined for complete:
>
>$ complete |grep ls
>complete -o default -F _longopt ls
Thanks Chet. The following does give me Variable Bash Completion:
complete -o bashdefault -F _longopt ls
--
Roger
http://rogerx.freeshell.org/
Chet Ramey
>
> Thanks Chet. The following does give me Variable Bash Completion:
>
> complete -o bashdefault -F _longopt ls
You might want both -o bashdefault and -o default.
Peter Kruse
On Fri, Sep 2, 2011 at 9:32 PM, Chet Ramey <chet....@case.edu> wrote:
> The attached patch adds a new shell option that, when enabled, is
> intended to restore the bash-4.1 behavior of expanding directory names
> in filenames being completed. I have done some testing, and it seems
> to work the way I intend. This, or some later version, will be part
the users here are happy with this patch, looking forward to see it
in the next release.
Peter
Roger
overwritten with:
$ \$HOME/
Following thread describes the problem for bash-4.2:
http://lists.gnu.org/archive/html/bug-bash/2011-02/msg00296.html
My current version in Gentoo and which are the latest available sources with
latest available patches:
=app-shells/bash-4.2_p10
=sys-libs/readline6.2_p1
--
Roger
http://rogerx.freeshell.org/
Chet Ramey
> When entering "$HOME/" and "TAB KEY" to perform tab completion readline gets
> overwritten with:
>
> $ \$HOME/
http://lists.gnu.org/archive/html/bug-bash/2011-09/msg00012.html
Roger
--
Roger
http://rogerx.freeshell.org/
Michael Kalisz
The shopt "direxpand" feature works as advertised (Thanks!) except that I
noticed it seems to break the name-completion of executables which are not
in you path.
Example:
(colortable16.sh is a script and executable)
$ ll colortable16.sh
-rwxr-xr-x 1 user group 1458 2009-12-03 19:18 colortable16.sh*
$ shopt | grep direxpand
direxpand off
$ ./col<TAB> (will expand the name to colortable16.sh => a correct
behaviour)
$ shopt -s direxpand
$ shopt | grep direxpand
direxpand on
$ ./col<TAB> (will NOT expand the name colortable16.sh => Incorrect
behaviour?)
B.t.w I have the bind 'set show-all-if-ambiguous on' so I only need to
press <TAB> once.
Bug or feature? Any hints?
Thanks in advance,
Michael
> On 2/24/11 5:14 PM, Michael Kalisz wrote:
>
>> Bash Version: 4.2
>> Patch Level: 0
>> Release Status: release
>>
>> Description:
>>
>> Hi!
>>
>> Example:
>>
>> In bash, version 4.1.5(1)-release:
>>
>> $ echo $PWD/<TAB>
>> will expand the $PWD variable to your current directory
>>
>> while in bash, version 4.2.0(1)-release:
>>
>> $ echo $PWD/<TAB>
>> will just escape the $ in front of the $ variable i.e:
>>
>> $ echo \$PWD/
>> The shell-expand-line (Ctrl-Alt-e) works but before I could use just TAB
>>
>> Any hints why? Any way to get the 4.1 behavior in 4.2?
>
> I wrote, in a subsequent message in a related thread:
>
>> The difference is that bash-4.1 expanded $HOME and left the expansion
>> as part of the replacement text. Bash-4.2 tries to leave what the
>> when do you quote the `$' in a filename (or, in this case, a directory
>> name)? It could be a shell variable, and it could be a character in
>> the filename.
>
> end of March that applied a heuristic to avoid quoting variable
> expansions in most cases, but there was no way to get the bash-4.1
> behavior back.
>
> intended to restore the bash-4.1 behavior of expanding directory names
> in filenames being completed. I have done some testing, and it seems
> to work the way I intend. This, or some later version, will be part
> of the next bash release. I am soliciting feedback on this iteration.
>
> includes the original heuristic I sent out back in March, the new shopt
> option, and updates to the documentation and test suite. It should apply
> cleanly to bash-4.2.10.
>
Chet Ramey
> Hi Chet,
>
> The shopt "direxpand" feature works as advertised (Thanks!) except that I
> noticed it seems to break the name-completion of executables which are not
> in you path.
and the differing prefixes make the names not match. I'll have to see
what I can do about that. It might be a nasty fix, since the same function
has to make sure that `.' does expand to $PWD to honor the shell's logical
view of the file system. I'm not sure what negative consequences dropping
that rewrite might bring.
Chet Ramey
> Hi Chet,
>
> The shopt "direxpand" feature works as advertised (Thanks!) except that I
> noticed it seems to break the name-completion of executables which are not
> in you path.
which the directory expansion hook would rewrite the directory name. Only
the dot and dot-dot cases worked in bash-4.1, and I have a fix that
temporarily suppresses the `external' directory rewriting in that case. I
will have to think about the right fix for the rest of the cases.
This worked in bash-4.1 only due to a bug/oversight in the directory
rewrite function logic. It was correcting that bug that uncovered this
problem, not the other changes that went in at the same time.
Chet Ramey
> > Hi Chet,
> >
> > The shopt "direxpand" feature works as advertised (Thanks!) except that I
> > noticed it seems to break the name-completion of executables which are not
> > in you path.
>
> and the differing prefixes make the names not match. I'll have to see
> what I can do about that. It might be a nasty fix, since the same function
> has to make sure that `.' does expand to $PWD to honor the shell's logical
> view of the file system. I'm not sure what negative consequences dropping
> that rewrite might bring.
it fixes the problem for you. It does for my testing.
Chet