Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Bash parameter expansion (remove largest trailing match, remove largest leading match, pattern replacement) does not work

21 views
Skip to first unread message

Bruce Lilly

unread,
Aug 29, 2020, 10:47:11 AM8/29/20
to bug-...@gnu.org
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: netbsd
Compiler: gcc
Compilation CFLAGS: -O2 -D_FORTIFY_SOURCE=2 -I/usr/include
-Wno-parentheses -Wno-format-security
uname output: NetBSD cq60-615dx.blilly.net 9.0 NetBSD 9.0 (GENERIC)
#0: Fri Feb 14 00:06:28 UTC 2020
mkr...@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC
amd64
Machine Type: x86_64--netbsd

Bash Version: 5.0
Patch Level: 17
Release Status: release

Description:
Bash parameter expansion (remove largest trailing match, remove
largest leading match, pattern replacement) does not work
Tested on OpenSUSE Leap 15.2, bash version 4.4.2.3(1)-release
(x86_64-suse-linux-gnu)
OpenBSD 6.7 bash version 5.0.17(1)-release (x86_64-unknown-openbsd6.7)
NetBSD 9.0 bash version 5.0.17(1)-release (x86_64--netbsd)
FreeBSD 12.1-STABLE bash version 5.0.18(2)-release (amd64-portbld-freebsd12.1)

Same results in all cases; this report posted from NetBSD 9.0.

As an example, consider constructing a path from components; the full
path should
have a single separator (slash), so any stray slashes at the tail of
the first part or the
start of the last part should be elided. There are many ways that
that can be done,
the "Repeat By" section demonstrates only a few examples.

Relevant manual sections are:
https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Pattern-Matching

Repeat-By:
$ bash shellbug
Running bash from /usr/pkg/bin/bash
Bash version 5.0.17(1) release GNU
dir=/foo/bar/baz/// separator1=/ separator2(octal escape)=\057
number0=* number1=+ pattern1=${number1}(${separator1})=+(/)
pattern2=${number0}([${separator2}])=*([\057]) base=///grimble/pritz
path=${dir%%${pattern1}}${separator1}${base##${pattern2}}=${dir%%+(/)}/${base##*([\057])}=/foo/bar/baz///////grimble/pritz
path=${dir/%${pattern1}/${separator1}}${base/#${pattern2}/}=${dir/%+(/)//}${base/#*([\057])/}=/foo/bar/baz//////grimble/pritz

Fix:
(demonstrates how a functioning shell handles exactly the same cases):
$ ksh93 shellbug
Running ksh93 from /usr/pkg/bin/ksh93
$Id: Version AJM 93u+ 2012-08-01 $
dir=/foo/bar/baz/// separator1=/ separator2(octal escape)=\057
number0=* number1=+ pattern1=${number1}(${separator1})=+(/)
pattern2=${number0}([${separator2}])=*([\057]) base=///grimble/pritz
path=${dir%%${pattern1}}${separator1}${base##${pattern2}}=${dir%%+(/)}/${base##*([\057])}=/foo/bar/baz/grimble/pritz
path=${dir/%${pattern1}/${separator1}}${base/#${pattern2}/}=${dir/%+(/)//}${base/#*([\057])/}=/foo/bar/baz/grimble/pritz

Robert Elz

unread,
Aug 29, 2020, 1:27:26 PM8/29/20
to Bruce Lilly, bug-...@gnu.org
Date: Sat, 29 Aug 2020 10:22:39 -0400
From: Bruce Lilly <bruce...@gmail.com>
Message-ID: <CAPyES363zNvUZHWo6imXOyC3...@mail.gmail.com>


| Bash parameter expansion (remove largest trailing match, remove
| largest leading match, pattern replacement) does not work

This is no more a bug in bash than it is for your similar report against
the NetBSD sh.

That bash is not ksh93 is not a bug. That bash doesn't implement every
bizarre ksh93 extension is also not a bug.

kre


Bruce Lilly

unread,
Aug 29, 2020, 1:33:53 PM8/29/20
to Robert Elz, bug-...@gnu.org
Robert,
Did you look at the relevant bash manual pages linked in the bug report?


>
>

Robert Elz

unread,
Aug 29, 2020, 1:48:24 PM8/29/20
to Bruce Lilly, bug-...@gnu.org
Date: Sat, 29 Aug 2020 13:33:37 -0400
From: Bruce Lilly <bruce...@gmail.com>
Message-ID: <CAPyES34z6N9+3ir44QHJze=6WBMjX+3mzgBft4m_Y0N_7p=n...@mail.gmail.com>

| Did you look at the relevant bash manual pages linked in the bug report?

Enough to know that [\057] doesn't mean what you seem to believe it should.

kre


Bruce Lilly

unread,
Aug 29, 2020, 1:51:56 PM8/29/20
to Robert Elz, bug-...@gnu.org
Evidently not enough to see the specifications for pattern matching...

>
>

Robert Elz

unread,
Aug 29, 2020, 2:31:01 PM8/29/20
to Bruce Lilly, bug-...@gnu.org
Date: Sat, 29 Aug 2020 13:51:39 -0400
From: Bruce Lilly <bruce...@gmail.com>
Message-ID: <CAPyES344Y8SAXmxCXo8DMJt2...@mail.gmail.com>


| Evidently not enough to see the specifications for pattern matching...

But I did, and I still fail to see anywhere where anything even
suggests that [\057] means anything other than either (which it
should be is less clear) than "a '0', or a '5', or a '7'" or perhaps
the same with the addition of "or a '\'" (that is, in char classes
in glob patterns, it isn't 100% clear whether the \ is a quoting
character, leading to a quoted '0' (which is just a '0') in the
example given, or whether it is just a character. Doesn't matter
here, neither interpretation is what you seem to expect it to mean.

Further, and probably worse here, is that you didn't bother to provide
your "shellbug" test script, so no-one can see what you're actually
doing, and repeat it for themselves. If there was an actual bug here,
perhaps one of the people running a pre-release of bash 5.1 could determine
if the bug still exists there or had already been fixed (but there isn't
likely to be any bug here, so that probably doesn't matter). But without
it we cannot see if you're driving bash correctly to even have an outside
chance of your extended glob patterns working (bash supports some of those
extensions, but not by default).

That lack didn't matter for the NetBSD sh bug report, as it was obvious
you were attempting non-standard extensions, that we simply don't support.
It does matter here.

kre


Koichi Murase

unread,
Aug 29, 2020, 2:52:53 PM8/29/20
to Bruce Lilly, bug-...@gnu.org
2020-08-29 23:47 Bruce Lilly <bruce...@gmail.com>:
> Description:
> Bash parameter expansion (remove largest trailing match,
> remove largest leading match, pattern replacement) does not work

In short, (1) ``octal escape \057'' doesn't have special meaning in
Bash glob patterns. `[\057]' means just one of a literal backslash or
digits 0, 5, or 7. (2) You need to set `shopt -s extglob` to use
extended glob patterns like `+(...)' and `*(...)'. Please look at the
paragraph above the list of extended patterns in the manual.

--
Koichi

Bruce Lilly

unread,
Aug 29, 2020, 2:55:44 PM8/29/20
to Koichi Murase, bug-...@gnu.org
Please don't assume that something output by printf (without quoting) for
clarity is representative of actual expansion by the shell when properly
quoted.

Bruce Lilly

unread,
Aug 29, 2020, 3:02:15 PM8/29/20
to Koichi Murase, bug-...@gnu.org
Thanks for the pointer to "shopt"; I'll check when I get a chance.

Ilkka Virta

unread,
Aug 29, 2020, 3:22:59 PM8/29/20
to Bruce Lilly, Koichi Murase, bug-...@gnu.org
On Sat, Aug 29, 2020 at 9:56 PM Bruce Lilly <bruce...@gmail.com> wrote:

> Please don't assume that something output by printf (without quoting) for
> clarity is representative of actual expansion by the shell when properly
> quoted.
>

If you don't want people to assume (and you shouldn't, if you want them to
help you),
you'd better post the actual script you use, and not just something output
by printf.
Without that, it's impossible to check what quoting you used, or to
reproduce the issue.

Bruce Lilly

unread,
Aug 29, 2020, 3:26:01 PM8/29/20
to Ilkka Virta, Koichi Murase, bug-...@gnu.org
Unfortunately, because bash is GPL, I can't post the copyrighted script
which is covered by a non-GPL license.

Koichi Murase

unread,
Aug 29, 2020, 3:27:54 PM8/29/20
to Bruce Lilly, bug-...@gnu.org
2020年8月30日(日) 3:55 Bruce Lilly <bruce...@gmail.com>:
> Please don't assume that something output by printf (without
> quoting) for clarity is representative of actual expansion by the
> shell when properly quoted.

Please don't assume that I have assumed something strange. To begin
with, you didn't provide the script `shellbug', so something needs to
be assumed anyway. Since it seems you are aware that your original
post would be unclear and confuse others, you could have just included
the contents of the file `shellbug'. Anyway, I did NOT assume that
you have literally written broken quoting like

separator2(octal escape)=\057
pattern1=${number1}(${separator1})

I assumed that you have written like

separator2='\057'
pattern1="${number1}(${separator1})"

because otherwise, it doesn't work with ksh93 either.


To explain it in more detail, first, these parameter expansions work
for me with separator1='/'. The pattern that you constructed from
separator2='\057' is just not supported by Bash. Next, you specify
exactly the same script `shellbug' to both Bash and ksh93, so I could
assume that you are missing the Bash-specific `shopt -s extglob' in
the script because otherwise, ksh93 would have complained it. That is
all what I can guess from your original post.

--
Koichi

Davide Brini

unread,
Aug 29, 2020, 3:38:01 PM8/29/20
to bug-...@gnu.org
On Sat, 29 Aug 2020 15:25:44 -0400, Bruce Lilly <bruce...@gmail.com>
wrote:

> Unfortunately, because bash is GPL, I can't post the copyrighted script
> which is covered by a non-GPL license.

That's ridiculous. You don't have to post the whole script (neither should
you), just a simple code snippet that shows the issue.

--
D.

Koichi Murase

unread,
Aug 29, 2020, 3:40:30 PM8/29/20
to Bruce Lilly, bug-...@gnu.org
2020-08-30 4:25 Bruce Lilly <bruce...@gmail.com>:
> Unfortunately, because bash is GPL, I can't post the copyrighted
> script which is covered by a non-GPL license.

Don't worry. In this case, the GPL doesn't apply. Please read the
following Q&A.

https://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL

Even if your original `shellbug' is licensed under something other,
and it prohibits to make it public, you can still create another
script to reproduce the same issue. Actually, it is considered to be
better practice in reporting bugs to create ``a reduced test case''
which is a minimal script that reproduces the problem rather than
sending the original script.

--
Koichi

Bruce Lilly

unread,
Aug 29, 2020, 3:54:35 PM8/29/20
to Koichi Murase, bug-...@gnu.org
On Sat, Aug 29, 2020, 15:40 Koichi Murase <myoga....@gmail.com> wrote:

> Don't worry. In this case, the GPL doesn't apply. Please read the
> following Q&A.
>
> https://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL
>
> Even if your original `shellbug' is licensed under something other,
> and it prohibits to make it public, you can still create another
> script to reproduce the same issue. [...]
>
It's a bit more complicated than that; if, for example, some excerpt ended
up in regression tests, there would be a question about whether or not
there was a copyright violation. As I understand the GPL (IANAL), it
requires all parts of a "work" to be GPL'd, and that wouldn't be possible
for any parts of the script that ended up in bash regression tests.

Bruce Lilly

unread,
Aug 29, 2020, 4:07:58 PM8/29/20
to Koichi Murase, bug-...@gnu.org
On Sat, Aug 29, 2020, 15:27 Koichi Murase <myoga....@gmail.com> wrote:

> I assumed that you have written like
>
> separator2='\057'
> pattern1="${number1}(${separator1})"
>
> because otherwise, it doesn't work with ksh93 either.
>
You are correct.

To explain it in more detail, first, these parameter expansions work
> for me with separator1='/'. The pattern that you constructed from
> separator2='\057' is just not supported by Bash.

That's surprising, as octal and hexadecimal escapes are fairly common.

Next, you specify
> exactly the same script `shellbug' to both Bash and ksh93, so I could
> assume that you are missing the Bash-specific `shopt -s extglob' in
> the script because otherwise, ksh93 would have complained it. That is
> all what I can guess from your original post.
>
Yes, I'm still looking into that (along with updating a couple of FreeBSD
machines, eating lunch, and monitoring a couple of downloads). I'll
obviously have to wrap the "shopt" bit in a wrapper; is there some minimum
bash version that supports it?

>
>

Koichi Murase

unread,
Aug 29, 2020, 4:53:40 PM8/29/20
to Bruce Lilly, bug-...@gnu.org
2020-08-30 4:54 Bruce Lilly <bruce...@gmail.com>:
> On Sat, Aug 29, 2020, 15:40 Koichi Murase <myoga....@gmail.com> wrote:
>> Don't worry. In this case, the GPL doesn't apply. Please read the
>> following Q&A.
>>
>> https://www.gnu.org/licenses/gpl-faq.en.html#IfInterpreterIsGPL
>>
>> Even if your original `shellbug' is licensed under something other,
>> and it prohibits to make it public, you can still create another
>> script to reproduce the same issue. [...]

I'm sorry if I have confused you, but first, let me clarify that I
meant by `create another script' that you can create a completely new
script from scratch instead of extracting the relevant part of the
original script.

> It's a bit more complicated than that; if, for example, some excerpt
> ended up in regression tests, there would be a question about
> whether or not there was a copyright violation. As I understand the
> GPL (IANAL), it requires all parts of a "work" to be GPL'd, and that
> wouldn't be possible for any parts of the script that ended up in
> bash regression tests.

That's an interesting discussion. I don't know how you define the
"work", but basically GPL only affects the derivative
programs/software but not all the "work" including the output of the
programs or the knowledge obtained in running/developing the code.

How about thinking in this way: You have gotten the knowledge that the
parameter expansions do not behave as you expected, and that knowledge
is not licensed by GPL. Then you create a new script from scratch
based on the knowledge by trying to encode the idea directly and not
to be affected by the style of the original script as much as
possible. I don't know but something like this reduced case:

text=AABBCC patA='*(A)' patC='+(C)'
echo "${text##$patA}, ${text%%$patC}"

I believe this shouldn't be considered GPL'd. Otherwise, anyone who
read a GPL code in the past cannot write any non-GPL programs because
one cannot prove the experience of reading the GPL code doesn't affect
any code that he/she writes thereafter. I sometimes hear that someone
avoids hiring programmers who have read a GPL code in the past for
defensive purposes, but I believe it's a matter of degree.

2020-08-30 5:07 Bruce Lilly <bruce...@gmail.com>:
> That's surprising, as octal and hexadecimal escapes are fairly
> common.

Yes, I know that it is confusing to those who are familiar with modern
Perl-style regular expressions. But historically, POSIX regular
expressions do not support the backslash escape sequences in bracket
expressions `[...]'. The backslash escape sequences in bracket
expressions were the extension historically. As far as I know, in
POSIX, only awk supports backslash sequences in regular expressions.

> Yes, I'm still looking into that (along with updating a couple of
> FreeBSD machines, eating lunch, and monitoring a couple of
> downloads). I'll obviously have to wrap the "shopt" bit in a
> wrapper; is there some minimum bash version that supports it?

Bash 2.02 supports `shopt -s extglob', so you can assume every Bash
has the support. If you are still failing to get an expected
behavior, you can just put the line `shopt -s extglob' in the
beginning of the script. In the case of the above mentioned reduced
case, you can write like this:

shopt -s extglob
text=AABBCC patA='*(A)' patC='+(C)'
echo "${text##$patA}, ${text%%$patC}"

--
Koichi

Bruce Lilly

unread,
Aug 29, 2020, 8:42:40 PM8/29/20
to Koichi Murase, bug-...@gnu.org
On Sat, Aug 29, 2020 at 4:53 PM Koichi Murase <myoga....@gmail.com> wrote:
[...]
> That's an interesting discussion. I don't know how you define the
> "work", but basically GPL only affects the derivative
> programs/software but not all the "work" including the output of the
> programs or the knowledge obtained in running/developing the code.

I'm using it in the sense of the GPL, as in the part of the preamble:
"software and other kinds of works".
It's not clearly defined; vagueness is one of the problems with the GPL.
I'm assuming that for bash it basically means everything that's included
in the distributed package. "Everything" is basically what GPL 3 section
5c says must be covered by the GPL.

[...]
> Otherwise, anyone who
> read a GPL code in the past cannot write any non-GPL programs because
> one cannot prove the experience of reading the GPL code doesn't affect
> any code that he/she writes thereafter. I sometimes hear that someone
> avoids hiring programmers who have read a GPL code in the past for
> defensive purposes, but I believe it's a matter of degree.

That's basically the issue, and it goes both ways (i.e. also people
writing GPL'ed code based on non-GPL source (or slapping a GPL
license on non-GPL code); there's a rather famous case
http://undeadly.org/cgi?action=article&sid=20070913014315 ).

> 2020-08-30 5:07 Bruce Lilly <bruce...@gmail.com>:
> > That's surprising, as octal and hexadecimal escapes are fairly
> > common.
>
> Yes, I know that it is confusing to those who are familiar with modern
> Perl-style regular expressions. But historically, POSIX regular
> expressions do not support the backslash escape sequences in bracket
> expressions `[...]'. The backslash escape sequences in bracket
> expressions were the extension historically. As far as I know, in
> POSIX, only awk supports backslash sequences in regular expressions.

Actually, it works (portably) with
separator2=$'\057'

> Bash 2.02 supports `shopt -s extglob', so you can assume every Bash
> has the support. If you are still failing to get an expected
> behavior, you can just put the line `shopt -s extglob' in the
> beginning of the script. In the case of the above mentioned reduced
> case, you can write like this:

The magic "shopt" incantation was indeed the key: thank you.

I guess this can be considered closed.

It would be nice, however, if there were an environment variable that
could cause bash to default to extglob. My concern is some variants of
make that send recipes line-by-line to a shell, in which case each line
might need to be prefixed by a test for ${BASH_VERSION} and the
shopt builtin.

Ángel

unread,
Aug 29, 2020, 8:55:26 PM8/29/20
to bug-...@gnu.org
On 2020-08-30 at 05:53 +0900, Koichi Murase wrote:
> > It's a bit more complicated than that; if, for example, some
> > excerpt ended up in regression tests, there would be a question
> > about whether or not there was a copyright violation. As I
> > understand the GPL (IANAL), it requires all parts of a "work" to be
> > GPL'd, and that wouldn't be possible for any parts of the script
> > that ended up in bash regression tests.
>
> That's an interesting discussion. I don't know how you define the
> "work", but basically GPL only affects the derivative
> programs/software but not all the "work" including the output of the
> programs or the knowledge obtained in running/developing the code.

I think it would be the other way around.

Suppose someone posted here:
> I found this nice telepathic shell that works better than bash:
> /* Copyrighted by Random Guy. All rights reserved */
> int dwim() {
> ...
> }

and then Chet ripped it copying that code into bash (or readline),
_bash_ would be infringing the copyright of "Random Guy".
Not the other way around.


Second, it is not true that "it wouldn't be possible for any parts of
the script that ended up in bash regression tests" not to be GPL.

A shell script is run by the shell using OS standard functions, and no
part of it is linked with GPL code. As such, you may have a shell
script under any license you wish, even if your shell is GPL. A
regression script is no different to that (and noone brought up that it
should be included in the regression tests, even!).

So you could for instance have a GPL shell and regression tests under a
different license, such as BSD. You could even have a script testing a
regression under a license such as CC-BY-SA 3.0, which is incompatible
with GPL.

Having a mixture of licenses would be undesirable, as it's simpler to
state "everything on this repository is GPL-3.0" or, "the test folder
is under license X and the rest is under license Y". But possible? It's
perfectly possible and legal.


The point that could be made is that bash regression tests are one work
on its own, that it is GPL and that adding your test to the regression
tests links them in a way incompatible with the GPL (I don't think this
would be the case, but ianal). As before, the copyright violation would
be to add such code to the regression tests.

And about test cases, as mentioned in the thread, a good test case will
be short and clear, testing one feature (or bug). As such they are
generally very simple. And often ineligible for copyright.


Best regards


Lawrence Velázquez

unread,
Aug 29, 2020, 9:12:45 PM8/29/20
to Bruce Lilly, Koichi Murase, bug-...@gnu.org
> On Aug 29, 2020, at 8:41 PM, Bruce Lilly <bruce...@gmail.com> wrote:
>
>> On Sat, Aug 29, 2020 at 4:53 PM Koichi Murase <myoga....@gmail.com> wrote:
>>
>> Yes, I know that it is confusing to those who are familiar with modern
>> Perl-style regular expressions. But historically, POSIX regular
>> expressions do not support the backslash escape sequences in bracket
>> expressions `[...]'. The backslash escape sequences in bracket
>> expressions were the extension historically. As far as I know, in
>> POSIX, only awk supports backslash sequences in regular expressions.
>
> Actually, it works (portably) with
> separator2=$'\057'

(a) $'...' is not POSIX. For instance, dash does not recognize it.

% port installed dash
The following ports are currently installed:
dash @0.5.11.1_0 (active)
% dash <<\EOF
heredoc> printf '<%s>\n' $'\057'
heredoc> EOF
<$\057>

(b) If you define separator2 in that way and then substitute it,
the context into which it is substituted sees a slash character,
not the backslash-escaped octal.

$ echo "$BASH_VERSION"
5.0.17(1)-release
$ printf '%s\n' $'\057'
/

vq

Bruce Lilly

unread,
Aug 29, 2020, 10:08:57 PM8/29/20
to Lawrence Velázquez, Koichi Murase, bug-...@gnu.org
On Sat, Aug 29, 2020 at 9:12 PM Lawrence Velázquez <v...@larryv.me> wrote:

> (a) $'...' is not POSIX. For instance, dash does not recognize it.

dash also doesn't have adequate pattern matching for the example
task (building a path while ensuring no empty components); it
has no way to specify one-or-more (or zero-or-more) occurrences
of a pattern such as a slash.

Ilkka Virta

unread,
Aug 30, 2020, 3:47:29 AM8/30/20
to Bruce Lilly, bug-...@gnu.org
On Sat, Aug 29, 2020 at 11:13 PM Bruce Lilly <bruce...@gmail.com> wrote:

> It's a bit more complicated than that; if, for example, some excerpt ended
> up in regression tests, there would be a question about whether or not
> there was a copyright violation. As I understand the GPL (IANAL), it
> requires all parts of a "work" to be GPL'd, and that wouldn't be possible
> for any parts of the script that ended up in bash regression tests.
>

That's hilarious. People post proof-of-concept scripts and code snippets as
part of bug
reports and such every day. If you'd just reduced the problem to a simple
demonstration
(below), you could have explicitly licensed it under the GPL if you were
afraid someone might
want to include it to a GPL'd software. In any case, for a one-liner like
this, it might not even
be copyrightable (at least not everywhere) as pretty much lacks any
creativity. I'd also assume
that test scripts often aren't even compiled with the main program, just
aggregated to the
code distribution. Anyway, it wouldn't be you doing the copyright violation
if someone used
your snippet without license.

Bash and ksh indeed differ in this:

$ bash -c 'str=foo/; sep="\057"; printf %s\\n ${str%%$sep}'
foo/

$ ksh -c 'str=foo/; sep="\057"; printf %s\\n ${str%%$sep}'
foo

And there's nothing in the Bash manuals that says \057 should be taken as
an octal
escape in a pattern match. The workarounds to that are either sep=$'\057'
which is
documented to accept escapes like this, or sep=/ which just works in the
obvious manner.
I do wonder why you'd even bother with trying the octal escape here instead
of just writing
the slash as a slash. Something like \001 would be different, of course.
The fact that $'\057'
does what you seem to want is exactly the part where you might have used a
form of quoting
which would have worked, but there was no way for the reader to check that
because you
hid the code.

$'' is described here:
https://www.gnu.org/software/bash/manual/html_node/ANSI_002dC-Quoting.html
(search for 'octal')

Of course, you also appear to have missed extglob, which I guess is
understandable if you're
coming from ksh. But even so, reducing the problem to smaller, easier to
debug pieces would
have shown the difference there, too, separately from the differences in
handling of octal escapes.
And perhaps led you to read the rest of what the manual says on Pattern
Matching, from the exact
page you linked to. ("If the extglob shell option is enabled using the
shopt builtin, several extended
pattern matching operators are recognized...")

Robert Elz

unread,
Aug 30, 2020, 5:01:21 AM8/30/20
to Bruce Lilly, Lawrence Velázquez, Koichi Murase, bug-...@gnu.org
Date: Sat, 29 Aug 2020 22:08:14 -0400
From: Bruce Lilly <bruce...@gmail.com>
Message-ID: <CAPyES35Ca6yUirzYX71n+9TO...@mail.gmail.com>

| dash also doesn't have adequate pattern matching for the example
| task (building a path while ensuring no empty components); it
| has no way to specify one-or-more (or zero-or-more) occurrences
| of a pattern such as a slash.

It does, you just don't get to do it in one absurdly complex variable
expansion. There is very little that can't be done in sh that is
reasonable to attempt in sh code (floating point matrix inversions are
probably not a candidate for example) - including even the original
7th edition Bourne sh (with no functions, nothing builtin, bugs all over
the place, ...) Sometimes it takes a bunch of code, and sometimes it
might not be extremely fast, but it is possible.

kre


Greg Wooledge

unread,
Aug 31, 2020, 7:49:55 AM8/31/20
to bug-...@gnu.org
On Sat, Aug 29, 2020 at 03:25:44PM -0400, Bruce Lilly wrote:
> Unfortunately, because bash is GPL, I can't post the copyrighted script
> which is covered by a non-GPL license.

On the extremely slim chance that you are genuinely confused, and not
just a troll:

The GPL covers bash. The GPL does NOT cover your script, written by
you, which includes #!/bin/bash or whatever shebang at the top. Your
script is your own work, and you hold the copyright. You get to decide
under what license other people may distribute copies of your script.

However, because I feel the chances of you not being a troll are so
very slim, I'm just going to delete the rest of the unread messages in
this thread immediately.

Chet Ramey

unread,
Aug 31, 2020, 11:44:59 AM8/31/20
to Bruce Lilly, bug-...@gnu.org, chet....@case.edu
On 8/29/20 10:22 AM, Bruce Lilly wrote:

> Bash Version: 5.0
> Patch Level: 17
> Release Status: release
>
> Description:
> Bash parameter expansion (remove largest trailing match, remove
> largest leading match, pattern replacement) does not work
> Tested on OpenSUSE Leap 15.2, bash version 4.4.2.3(1)-release
> (x86_64-suse-linux-gnu)
> OpenBSD 6.7 bash version 5.0.17(1)-release (x86_64-unknown-openbsd6.7)
> NetBSD 9.0 bash version 5.0.17(1)-release (x86_64--netbsd)
> FreeBSD 12.1-STABLE bash version 5.0.18(2)-release (amd64-portbld-freebsd12.1)
>
> Same results in all cases; this report posted from NetBSD 9.0.

There are a number of things wrong with this report; I think later messages
in the thread cover them. In short,

1. BRE bracket expression matching doesn't perform backslash-interpretation
of anything, except to quote the next character, much less octal
constant expansion;
2. You need `shopt -s extglob' to enable the pattern matching you want to
use.

Chet

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU ch...@case.edu http://tiswww.cwru.edu/~chet/

Chet Ramey

unread,
Aug 31, 2020, 11:49:06 AM8/31/20
to Bruce Lilly, Ilkka Virta, chet....@case.edu, Koichi Murase, bug-...@gnu.org
On 8/29/20 3:25 PM, Bruce Lilly wrote:
> Unfortunately, because bash is GPL, I can't post the copyrighted script
> which is covered by a non-GPL license.

This is ridiculous on its face. A script that bash executes doesn't have
any relevance to bash's copyright.

You might have reasons for not wanting to publicly post a copyrighted
script, but bash being GPL is not one of them.

Chet Ramey

unread,
Aug 31, 2020, 11:51:03 AM8/31/20
to Bruce Lilly, Koichi Murase, chet....@case.edu, bug-...@gnu.org
On 8/29/20 3:54 PM, Bruce Lilly wrote:

> It's a bit more complicated than that; if, for example, some excerpt ended
> up in regression tests, there would be a question about whether or not
> there was a copyright violation.

This is not at all the same thing, and could (and would) be handled
separately.

Chet Ramey

unread,
Aug 31, 2020, 12:02:07 PM8/31/20
to Lawrence Velázquez, Bruce Lilly, chet....@case.edu, Koichi Murase, bug-...@gnu.org
On 8/29/20 9:12 PM, Lawrence Velázquez wrote:

>> Actually, it works (portably) with
>> separator2=$'\057'
>
> (a) $'...' is not POSIX. For instance, dash does not recognize it.

I believe it made the cut and will be in the next edition of the standard.
0 new messages