bash -> dash translation

[Seb]

Hi,

I recently got bitten by the change from bash to dash as the default
/bin/sh in Debian. While bash was the default, the following ran fine
(shebang in the script is "#! /bin/sh"):

local projstr=$(${GMT} grdinfo "${raster_prefix}.grd" | \
awk '/Remark: / {sub(/.+Remark: /, ""); sub(/; .+/, ""); print}')

Now that /bin/sh points to dash as the interpreter, the above prints an
error:

./script.sh: 10: ./script.sh: +ellps: bad variable name

where "+ellps" is part of the string passed to awk in the pipe. So this
seems to be a quoting issue, and I can't see anything in the docs that
would clarify what it is. Any advice?

Thanks,

--
Seb

[Seb]

Other than shebang as "#! /bin/bash", of course.

Thanks,
--
Seb

[Kenny McCormack]

In article <87ip36a...@kolob.subpolar.dyndns.org>,
Seb <spl...@gmail.com> wrote:
...

>Other than shebang as "#! /bin/bash", of course.

That, of course, was my first reaction.

So, I have to ask: Why not?

--
This is the GOP's problem. When you're at the beginning of the year
and you've got nine Democrats running for the nomination, maybe one or
two of them are Dennis Kucinich. When you have nine Republicans, seven
or eight of them are Michelle Bachmann.

[Seb]

On Sun, 28 Apr 2013 19:59:10 +0000 (UTC),
gaz...@shell.xmission.com (Kenny McCormack) wrote:

> In article <87ip36a...@kolob.subpolar.dyndns.org>,
> Seb <spl...@gmail.com> wrote:
> ...
>> Other than shebang as "#! /bin/bash", of course.

> That, of course, was my first reaction.

> So, I have to ask: Why not?

Well, I'm getting seduced by this POSIX-compliant dash. It was a pain
debugging this, and it would be nice to have these scripts be as
portable as possible. Having said that, it might too much trouble for
what it's worth. Makes me wonder what most scripts in the system assume
for /bin/sh.


--
Seb

[Seb]

I found the checkbashisms utility in Debian, which returns a lot of
these in my scripts:

possible bashism in script.sh line 951 (local foo=bar):
local var=$1


--
Seb

[Chris F.A. Johnson]

What does $GMT contain?
What is the output of grdinfo?

--
Chris F.A. Johnson, author <http://shell.cfajohnson.com/>
===================================================================
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)

[Alan Curry]

In article <87mwsia...@kolob.subpolar.dyndns.org>,

Put set of quotation marks around the whole $(...), otherwise its output is
word-split and passed to local as separate words, which local tries to do as
separate assignments.

local foo="1 bar=2" # A. makes foo local and sets its value to "1 bar=2"
local foo=1 bar=2 # B. makes foo and bar local and sets their values.
cmd () {
echo 1 bar=2
}
local foo="$(cmd)" # C. same as A.
local foo=$(cmd) # D. same as B.
cmd () {
echo 1 +bar=2
}
local foo="$(cmd)" # E. Like A, but foo's value becomes "1 +bar=2"
local foo=$(cmd) # F. Like B, but tries to create a local called +bar

You're doing something like F, and creating a variable called +ellps, which
isn't allowed. bash seems to be applying a heuristic "you didn't really mean
what you said so I'll do something else" rule.

--
Alan Curry

[Seb]

On Sun, 28 Apr 2013 16:57:01 -0400,
"Chris F.A. Johnson" <cfajo...@gmail.com> wrote:

> On 2013-04-28, Seb wrote:
>> Hi,

>> I recently got bitten by the change from bash to dash as the default
>> /bin/sh in Debian. While bash was the default, the following ran
>> fine (shebang in the script is "#! /bin/sh"):

>> local projstr=$(${GMT} grdinfo "${raster_prefix}.grd" | \ awk
>> '/Remark: / {sub(/.+Remark: /, ""); sub(/; .+/, ""); print}')

>> Now that /bin/sh points to dash as the interpreter, the above prints
>> an error:

>> ./script.sh: 10: ./script.sh: +ellps: bad variable name

>> where "+ellps" is part of the string passed to awk in the pipe. So
>> this seems to be a quoting issue, and I can't see anything in the
>> docs that would clarify what it is. Any advice?

> What does $GMT contain? What is the output of grdinfo?

GMT is a collection of programs for handling geographic and cartesian
data sets. ${GMT} is a variable that identifies which version of the
program following it should be called. grdinfo returns several lines of
metadata on the data set, and the line I'm targetting (/Remark: /) has
this structure:

file.grd: Remark: +proj=aea +units=km +lon_0=-62.82555 +lat_0=58.4784 +lat_1=58.41 +lat_2=58.5468 +ellps=WGS84; GEO ORIG REG= -63.1216 -62.5295 58.3758 58.581

So somehow the single quotes for the awk program failed, and the strings
with "=" are being interpreted by the shell, not by awk. Only with dash
though... Everything is OK with bash.

--
Seb

[Janis Papanagnou]

On 28.04.2013 23:11, Seb wrote:
> On Sun, 28 Apr 2013 16:57:01 -0400,
> "Chris F.A. Johnson" <cfajo...@gmail.com> wrote:
>
>> On 2013-04-28, Seb wrote:
>>> Hi,
>
>>> I recently got bitten by the change from bash to dash as the default
>>> /bin/sh in Debian. While bash was the default, the following ran
>>> fine (shebang in the script is "#! /bin/sh"):
>
>>> local projstr=$(${GMT} grdinfo "${raster_prefix}.grd" | \ awk
>>> '/Remark: / {sub(/.+Remark: /, ""); sub(/; .+/, ""); print}')
>
>>> Now that /bin/sh points to dash as the interpreter, the above prints
>>> an error:
>
>>> ./script.sh: 10: ./script.sh: +ellps: bad variable name
>
>>> where "+ellps" is part of the string passed to awk in the pipe. So
>>> this seems to be a quoting issue, and I can't see anything in the
>>> docs that would clarify what it is. Any advice?
>
>> What does $GMT contain? What is the output of grdinfo?
>
> GMT is a collection of programs for handling geographic and cartesian
> data sets. ${GMT} is a variable that identifies which version of the
> program following it should be called.

One thing to know would be to what string value the variable expands.

> grdinfo returns several lines of
> metadata on the data set, and the line I'm targetting (/Remark: /) has
> this structure:
>
> file.grd: Remark: +proj=aea +units=km +lon_0=-62.82555 +lat_0=58.4784 +lat_1=58.41 +lat_2=58.5468 +ellps=WGS84; GEO ORIG REG= -63.1216 -62.5295 58.3758 58.581
>
> So somehow the single quotes for the awk program failed, and the strings
> with "=" are being interpreted by the shell, not by awk. Only with dash
> though... Everything is OK with bash.
>

Have you tried with other POSIX shells? (Without 'local' of course.)

Janis

[Chris F.A. Johnson]

On 2013-04-28, Alan Curry wrote:
> In article <87mwsia...@kolob.subpolar.dyndns.org>,
> Seb <spl...@gmail.com> wrote:
>>Hi,
>>
>>I recently got bitten by the change from bash to dash as the default
>>/bin/sh in Debian. While bash was the default, the following ran fine
>>(shebang in the script is "#! /bin/sh"):
>>
>> local projstr=$(${GMT} grdinfo "${raster_prefix}.grd" | \
>> awk '/Remark: / {sub(/.+Remark: /, ""); sub(/; .+/, ""); print}')
>>
>>Now that /bin/sh points to dash as the interpreter, the above prints an
>>error:
>>
>>./script.sh: 10: ./script.sh: +ellps: bad variable name
>>
>>where "+ellps" is part of the string passed to awk in the pipe. So this
>>seems to be a quoting issue, and I can't see anything in the docs that
>>would clarify what it is. Any advice?
>
> Put set of quotation marks around the whole $(...), otherwise its output is
> word-split and passed to local as separate words, which local tries to do as
> separate assignments.

That shouldn't be necessary as the output is being assigned to a variable.

It looks like a bug in dash:

x()
{
local x=$(echo q w e r)
y=$(echo q w e r)
echo "x=$x"
echo "y=$y"
}

$ x
x=q
y=q w e r

[Alan Curry]

In article <76h05a-...@cfa.johnson>,

Chris F.A. Johnson <cfajo...@gmail.com> wrote:
>>

>> Put set of quotation marks around the whole $(...), otherwise its output is
>> word-split and passed to local as separate words, which local tries to do as
>> separate assignments.
>
> That shouldn't be necessary as the output is being assigned to a variable.

Are you sure? Consider this:

local $(echo foo=1 bar=2)
local f$(echo oo=1 bar=2)
local foo$(echo =1 bar=2)
local foo=$(echo 1 bar=2)

The first 3 all perform separate assignments to foo and bar. Is it really a
good idea to parse the fourth one according to a different rule just because
the $( happens to come right after the =?

Would you apply this rule to external commands too? Should these:

dd $(echo bs=1 count=1)
dd bs=$(echo 1 count=1)

not be identical?

>
> It looks like a bug in dash:

It looks like removal of an unnecessary special case to me. A feature.

zsh seems to have it both ways, depending on the option KSH_TYPESET.

posh agrees with dash (and not with its close relative pdksh! someone must
have done that on purpose.)

--
Alan Curry

[Seb]

Wow, this is amazing! So it looks like dash has helped expose a mess.
It's even more amazing that Bash somehow managed to avert the mess and
'local' didn't even create any variables from the piped output. Perhaps
Bash's 'local' waits until the end of the pipe to do the assignments?

Thanks,

--
Seb

[Barry Margolin]

In article <klk5et$kui$1...@speranza.aioe.org>,

pac...@kosh.dhis.org (Alan Curry) wrote:

> In article <76h05a-...@cfa.johnson>,
> Chris F.A. Johnson <cfajo...@gmail.com> wrote:
> >>
> >> Put set of quotation marks around the whole $(...), otherwise its output
> >> is
> >> word-split and passed to local as separate words, which local tries to do
> >> as
> >> separate assignments.
> >
> > That shouldn't be necessary as the output is being assigned to a
> > variable.
>
> Are you sure? Consider this:
>
> local $(echo foo=1 bar=2)
> local f$(echo oo=1 bar=2)
> local foo$(echo =1 bar=2)
> local foo=$(echo 1 bar=2)
>
> The first 3 all perform separate assignments to foo and bar. Is it really a
> good idea to parse the fourth one according to a different rule just because
> the $( happens to come right after the =?
>
> Would you apply this rule to external commands too? Should these:
>
> dd $(echo bs=1 count=1)
> dd bs=$(echo 1 count=1)
>
> not be identical?

The difference is that "local" is a built-in, seemingly intended to
modify variable assignment. So it would make sense for the variable
assignments to be parsed as in normal assignments. Normal variable
assignments don't do word splitting, i.e. you can write:

foo=$(echo 1 2)

rather than:

foo="$(echo 1 2)"

Bash obviously interprets it this way.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Chris F.A. Johnson]

Since local is a non-standard command (though I expect it to be
added to the POSIX standard shortly), who's to say what the correct
behaviour is? I think bash has the more logical approach.

[Alan Curry]

In article <barmar-EEE71D....@news.eternal-september.org>,

Barry Margolin <bar...@alum.mit.edu> wrote:
>In article <klk5et$kui$1...@speranza.aioe.org>,
> pac...@kosh.dhis.org (Alan Curry) wrote:
>>
>> Would you apply this rule to external commands too? Should these:
>>
>> dd $(echo bs=1 count=1)
>> dd bs=$(echo 1 count=1)
>>
>> not be identical?
>
>The difference is that "local" is a built-in, seemingly intended to
>modify variable assignment. So it would make sense for the variable
>assignments to be parsed as in normal assignments. Normal variable
>assignments don't do word splitting, i.e. you can write:
>
>foo=$(echo 1 2)
>
>rather than:
>
>foo="$(echo 1 2)"

You *can* write it without explicit quoting, but is there any reason to?
With quotation marks the intent is clear, and you avoid depending on
special treatment by the parser (such a weird corner case that I don't
think it's even documented in any of the shell man pages).

Quoting parameter and command substitutions unless you want the result
to be word-split generally a good habit, and I don't see what's to be
gained by making an exception for assignments.

--
Alan Curry

[Alan Curry]

In article <qis05a-...@cfa.johnson>,

Chris F.A. Johnson <cfajo...@gmail.com> wrote:
>

> Since local is a non-standard command (though I expect it to be
> added to the POSIX standard shortly), who's to say what the correct
> behaviour is? I think bash has the more logical approach.

I hope that at least all the shells with a "local" command have enough
sense to parse it the same way they parse "export". What's your reading
of the POSIX requirements on this:

export foo=$(echo 1 bar=2)

--
Alan Curry

[Chris F.A. Johnson]

On 2013-04-29, Alan Curry wrote:
> In article <barmar-EEE71D....@news.eternal-september.org>,
> Barry Margolin <bar...@alum.mit.edu> wrote:
>>In article <klk5et$kui$1...@speranza.aioe.org>,
>> pac...@kosh.dhis.org (Alan Curry) wrote:
>>>
>>> Would you apply this rule to external commands too? Should these:
>>>
>>> dd $(echo bs=1 count=1)
>>> dd bs=$(echo 1 count=1)
>>>
>>> not be identical?
>>
>>The difference is that "local" is a built-in, seemingly intended to
>>modify variable assignment. So it would make sense for the variable
>>assignments to be parsed as in normal assignments. Normal variable
>>assignments don't do word splitting, i.e. you can write:
>>
>>foo=$(echo 1 2)
>>
>>rather than:
>>
>>foo="$(echo 1 2)"
>
> You *can* write it without explicit quoting, but is there any reason to?

Is there any reason *to* quote it? I don't think so.

> With quotation marks the intent is clear,

No more so than without them.

They are no more necessary than quoting variables in an assignment:

a="$b$c" ## Useless quotes

> and you avoid depending on special treatment by the parser (such a
> weird corner case that I don't think it's even documented in any of
> the shell man pages).
>
> Quoting parameter and command substitutions unless you want the result
> to be word-split generally a good habit,

Unecessary quoting is not a good habit; it shows that you don't know
when they're required and when they're not.

[Barry Margolin]

In article <76015a-...@cfa.johnson>,

"Chris F.A. Johnson" <cfajo...@gmail.com> wrote:

Would you say the same thing about unnecessary parentheses in arithmetic
and logical expressions? I like to write:

foo = (something || something_else) ? bar : baz;

I know the parentheses are redundant, but they make the grouping more
visually obvious.

Often it's more important to make the intent clear to *human* readers,
it doesn't matter if it's unambiguous to the computer.

[Janis Papanagnou]

Parenthesis are a different issue, IMO. Different programming languages
show different operator precedence thus require different grouping. Then
there's also languages even based on inconsistent operator precedence
(like C and successors). So, yes, it makes sense to group expressions in
contexts where people are not only using a single language. (BTW, above
example is not a valid standard shell expression per se. Though it may
be defined in some shells inside arithmetic expressions and expansions.)

>
> Often it's more important to make the intent clear to *human* readers,
> it doesn't matter if it's unambiguous to the computer.

WRT unnecessary quotes I agree with Chris'. I want to add that quoting
(i.e. with its crucial word-split implication) is a very spell specific
issue which, if heavily used, can even obfuscate the code as we see here
occasionally in code. Avoid unnecessary ballast to make code clearer for
the reader; I'd use it only where necessary.

Can you give examples where the "intent [to human readers] is clearer"
with quotes.

Janis