Are there some graceful methods for awk to invoke the shell variables?

[Hongyi Zhao]

Hi all,

I know the following methods to let awk invoke the shell variables:

var="test"
awk 'BEGIN{print "'$var'"}'

or

var="this is a test"
awk 'BEGIN{print "'"$var"'"}'

or

var="this is a test"
awk -v nvar="$var" 'BEGIN {print nvar}'

But, all of the above methods seems cumbersome. Could you please give me
some more graceful methods for awk to invoke the shell variables?

Furthermore, if I want let awk invoke several shell variables instead of
only one, again, are there some graceful methods for this more
complicated case?

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

[Kenny McCormack]

In article <mgts1f$aae$2...@aspen.stu.neva.ru>,
Hongyi Zhao <hongy...@gmail.com> wrote:
...

>Furthermore, if I want let awk invoke several shell variables instead of
>only one, again, are there some graceful methods for this more
>complicated case?

All methods are pretty bad. It's just the nature of the beast.

One way that you might like better is to make sure the variables you need
are all exported (do the appropriate googling if this term is not familiar
to you) and then you can access them from most modern AWKs via the ENVIRON
array (again, google is your friend).

--
"Everything Roy (aka, AU8YOG) touches turns to crap."
--citizens of alt.obituaries--

[Ed Morton]

On 4/18/2015 10:08 AM, Hongyi Zhao wrote:
> Hi all,
>
> I know the following methods to let awk invoke the shell variables:
>
> var="test"
> awk 'BEGIN{print "'$var'"}'

Never do that.

>
> or
>
> var="this is a test"
> awk 'BEGIN{print "'"$var"'"}'

Never do that.

>
> or
>
> var="this is a test"
> awk -v nvar="$var" 'BEGIN {print nvar}'

Always do that unless you need escapes to not be expanded and do not need the
variables to be set in the BEGIN section, then use:

awk '{print nvar}' nvar="$var"

>
> But, all of the above methods seems cumbersome. Could you please give me
> some more graceful methods for awk to invoke the shell variables?
>
> Furthermore, if I want let awk invoke several shell variables instead of
> only one, again, are there some graceful methods for this more
> complicated case?
>
> Regards
>

See http://cfajohnson.com/shell/cus-faq-2.html#Q24.

There's also ENVIRON["var"] but then the variables have to be exported:

export var="whatever"
awk 'BEGIN{print ENVIRON["var"}'

or re-set to themselves on the command line:

var="$var" awk 'BEGIN{print ENVIRON["var"}'

which is just cumbersome and awkward so don't bother with it.

Ed.

[Stephane Chazelas]

2015-04-18 10:39:03 -0500, Ed Morton:
[...]

> >var="this is a test"
> >awk -v nvar="$var" 'BEGIN {print nvar}'
>
> Always do that unless you need escapes to not be expanded and do not
> need the variables to be set in the BEGIN section, then use:
>
> awk '{print nvar}' nvar="$var"

I'd rather say: "never do that unless you can guarantee that
$var will never contain backslash characters".

> >But, all of the above methods seems cumbersome. Could you please give me
> >some more graceful methods for awk to invoke the shell variables?
> >
> >Furthermore, if I want let awk invoke several shell variables instead of
> >only one, again, are there some graceful methods for this more
> >complicated case?
> >
> >Regards
> >
>
> See http://cfajohnson.com/shell/cus-faq-2.html#Q24.
>
> There's also ENVIRON["var"] but then the variables have to be exported:
>
> export var="whatever"
> awk 'BEGIN{print ENVIRON["var"}'
>
> or re-set to themselves on the command line:
>
> var="$var" awk 'BEGIN{print ENVIRON["var"}'
>
> which is just cumbersome and awkward so don't bother with it.

[...]

Hardly any more cumbersome than the -v var="$var" method, and
doesn't suffer from its problem with backslash.

With shells that export all their variables like rc, es or sh
-a, it is less cumbersome.

--
Stephane

[Hongyi Zhao]

On Mon, 20 Apr 2015 21:11:03 +0100, Stephane Chazelas wrote:

> With shells that export all their variables like rc, es or sh -a, it is
> less cumbersome.

Then still need to invoke these vars like the following form?

ENVIRON["var"} # If have spaces in the var's value.

or

ENVIRON[var} # If no space in the var's value.

[Stephane Chazelas]

2015-04-22 12:33:22 +0000, Hongyi Zhao:

> On Mon, 20 Apr 2015 21:11:03 +0100, Stephane Chazelas wrote:
>
> > With shells that export all their variables like rc, es or sh -a, it is
> > less cumbersome.
>
> Then still need to invoke these vars like the following form?
>
> ENVIRON["var"} # If have spaces in the var's value.

ENVIRON["var"]

>
> or
>
> ENVIRON[var} # If no space in the var's value.

No.

ENVIRON[var] returns the value of the *environment* variable
whose name is stored in the *awk* variable called "var".

For instance

awk 'BEGIN {var="PATH"; print ENVIRON[var]}'

would give you the content of the PATH environment variable.

(like

awk 'BEGIN {print ENVIRON["PATH"]}'
)

--
Stephane

[Hongyi Zhao]

On Wed, 22 Apr 2015 13:39:17 +0100, Stephane Chazelas wrote:

>> ENVIRON["var"} # If have spaces in the var's value.

Sorry for this copy-and-paste error from this thread.

>
> ENVIRON["var"]
>
>
>> or
>>
>> ENVIRON[var} # If no space in the var's value.
>
> No.
>
> ENVIRON[var] returns the value of the *environment* variable whose name
> is stored in the *awk* variable called "var".
>
> For instance
>
> awk 'BEGIN {var="PATH"; print ENVIRON[var]}'
>
> would give you the content of the PATH environment variable.
>
> (like
>
> awk 'BEGIN {print ENVIRON["PATH"]}'
> )

Thanks for point this out.

Regards

[Hongyi Zhao]

On Mon, 20 Apr 2015 21:11:03 +0100, Stephane Chazelas wrote:

> With shells that export all their variables like rc, es or sh -a, it is
> less cumbersome.

But I've tried as follows without the export command and still it will
work smoothly:

werner@debian:~$ cat bbb
111
222
333
werner@debian:~$ var=222
werner@debian:~$ awk 'BEGIN {myvar=ENVIRON["var"] } $0 ~ myvar ' bbb
222

[Dave Farrance]

Hongyi Zhao <hongy...@gmail.com> wrote:

>But, all of the above methods seems cumbersome. Could you please give me
>some more graceful methods for awk to invoke the shell variables?

I don't know what the various rules and downsides are because I'm not
very familiar with awk, but these seem to work in bash, and Ed would
probably disapprove of all of them:

$ x=10.0 y=2.5 z=3.0
$ echo $x $y $z | awk '{print $1 * $2 / $3}'
8.33333
$ awk "BEGIN {print $x*$y/$z}"
8.33333
$ awk "BEGIN {a=$x*$y/$z; print \"Result: \" a}"
Result: 8.33333

[Stephane Chazelas]

2015-04-22 14:13:59 +0000, Hongyi Zhao:

> On Mon, 20 Apr 2015 21:11:03 +0100, Stephane Chazelas wrote:
>
> > With shells that export all their variables like rc, es or sh -a, it is
> > less cumbersome.
>
> But I've tried as follows without the export command and still it will
> work smoothly:
>
> werner@debian:~$ cat bbb
> 111
> 222
> 333
> werner@debian:~$ var=222
> werner@debian:~$ awk 'BEGIN {myvar=ENVIRON["var"] } $0 ~ myvar ' bbb
> 222

[...]

It would only work if var had been exported at some point. So
either you did a

export var

before. Or var was already in the environment when your shell
started or you issued a set -a prior to var=222.

--
Stephane

[Ed Morton]

On 4/22/2015 9:17 AM, Dave Farrance wrote:
> Hongyi Zhao <hongy...@gmail.com> wrote:
>
>> But, all of the above methods seems cumbersome. Could you please give me
>> some more graceful methods for awk to invoke the shell variables?
>
> I don't know what the various rules and downsides are because I'm not
> very familiar with awk, but these seem to work in bash, and Ed would
> probably disapprove of all of them:

Correct but you're actually just showing 1 way not multiple.

>
> $ x=10.0 y=2.5 z=3.0
> $ echo $x $y $z | awk '{print $1 * $2 / $3}'
> 8.33333

I assume the above is just showing the expected result as it's not suggesting
anything about how to use shell var values in awk.

> $ awk "BEGIN {print $x*$y/$z}"
> 8.33333
> $ awk "BEGIN {a=$x*$y/$z; print \"Result: \" a}"
> Result: 8.33333

The above are identical wrt expanding shell vars to become part of the body of
an awk script before it is executed.

Notice you now need to escape double quotes and you'd also need to escape $s for
referencing fields so what should be a simple:

echo 7 | awk -v x="$x" '{y=1; print "Result:", $y * x}'

now needs to be cluttered up with escapes and you have to wonder when you come
across something like this what the heck it all means (e.g. why do the variables
y and x both have $s in front of them but one of them has an escape in front of
that):

echo 7 | awk "{y=1; print \"Result:\", \$y * $x}"

Also note that if x contained an undesirable character then you'd get bizarre
errors or other undesirable behavior from the 2nd one but completely reasonable
behavior from the first. e.g. if x was unset:

$ echo 7 | awk -v x="$x" '{y=1; print "Result:", $y * x}'
Result: 0

$ echo 7 | awk "{y=1; print \"Result:\", \$y * $x}"
awk: cmd. line:1: {y=1; print "Result:", $y * }
awk: cmd. line:1: ^ syntax error

Rhetorical questions: Why is that a syntax error? The syntax LOOKS fine, right?

You can use additional quoting around the shell vars to make awk less sensitive
to their contents but you still end up needing to escape quotes and
field-reference $ signs and it's all pointlessly complicated, obfuscating and
error-prone so just don't do it.

Ed.

[Hongyi Zhao]

On Wed, 22 Apr 2015 15:42:04 +0100, Stephane Chazelas wrote:

>> werner@debian:~$ cat bbb 111 222 333 werner@debian:~$ var=222
>> werner@debian:~$ awk 'BEGIN {myvar=ENVIRON["var"] } $0 ~ myvar ' bbb
>> 222
> [...]
>
> It would only work if var had been exported at some point. So either you
> did a
>
> export var
>
> before. Or var was already in the environment when your shell started or
> you issued a set -a prior to var=222.

Not at all, the only done is the above three commands I posted here.

My shell is:

werner@debian:~$ bash --version
GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)

[Hongyi Zhao]

On Thu, 23 Apr 2015 00:01:53 +0000, Hongyi Zhao wrote:

>> It would only work if var had been exported at some point. So either
>> you did a
>>
>> export var
>>
>> before.

After a second thought, I reminded that I really exported that var
before ;-(