Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Pass variable into sed program
9 views
Skip to first unread message
Michael F. Stemper
May 4, 2018, 10:53:34 AM5/4/18
to
I'm in the process of modifying some existing scripts to account for
the fact that the format of their input has changed. The following
piping of multiple sed commands did the trick in one place:
username@hostname$ order=42
username@hostname$ echo 7 | sed "s/,/) ($order,/g" | sed "s/^/($order,/"
| sed "s/$/)/"
(42,7)
username@hostname$ echo 2,3,5,7 | sed "s/,/) ($order,/g" | sed
"s/^/($order,/" | sed "s/$/)/"
(42,2) (42,3) (42,5) (42,7)
username@hostname$
Exactly what I needed. Then, I found another spot in the same script
where I needed the same idiom. I naturally decided to move these three
commands to a sed program:
username@hostname$ cat OldForm.sed
s/,/) ($order,/g
s/^/($order,/
s/$/)/
username@hostname$ echo 7 | sed -f OldForm.sed
($order,7)
username@hostname$
So far, so good.
Then, I tried to actually pass the variable in to the program as a
command-line argument:
username@hostname$ cat OldForm.sed
s/,/) ($1,/g
s/^/($1,/
s/$/)/
username@hostname$ echo 7 | sed -f OldForm.sed $order
sed: can't read 42: No such file or directory
username@hostname$
What's happened is obvious: sed is interpreting the "42" as the
name of a file to read, not as an argument.
I've gone through the O'Reilly book without finding out how to do
this. I've also spent some time searching on-line, and been told
over and over again how to use environment variables in one-liners
(like my opening example had).
What I have been unable to find is any description of how to pass
something into a sed *program*. This seems to be a very basic
action. Is it really true that it's not possible?
--
Michael F. Stemper
What happens if you play John Cage's "4'33" at a slower tempo?
the fact that the format of their input has changed. The following
piping of multiple sed commands did the trick in one place:
username@hostname$ order=42
username@hostname$ echo 7 | sed "s/,/) ($order,/g" | sed "s/^/($order,/"
| sed "s/$/)/"
(42,7)
username@hostname$ echo 2,3,5,7 | sed "s/,/) ($order,/g" | sed
"s/^/($order,/" | sed "s/$/)/"
(42,2) (42,3) (42,5) (42,7)
username@hostname$
Exactly what I needed. Then, I found another spot in the same script
where I needed the same idiom. I naturally decided to move these three
commands to a sed program:
username@hostname$ cat OldForm.sed
s/,/) ($order,/g
s/^/($order,/
s/$/)/
username@hostname$ echo 7 | sed -f OldForm.sed
($order,7)
username@hostname$
So far, so good.
Then, I tried to actually pass the variable in to the program as a
command-line argument:
username@hostname$ cat OldForm.sed
s/,/) ($1,/g
s/^/($1,/
s/$/)/
username@hostname$ echo 7 | sed -f OldForm.sed $order
sed: can't read 42: No such file or directory
username@hostname$
What's happened is obvious: sed is interpreting the "42" as the
name of a file to read, not as an argument.
I've gone through the O'Reilly book without finding out how to do
this. I've also spent some time searching on-line, and been told
over and over again how to use environment variables in one-liners
(like my opening example had).
What I have been unable to find is any description of how to pass
something into a sed *program*. This seems to be a very basic
action. Is it really true that it's not possible?
--
Michael F. Stemper
What happens if you play John Cage's "4'33" at a slower tempo?
Grant Taylor
May 4, 2018, 2:37:08 PM5/4/18
to
On 05/04/2018 08:53 AM, Michael F. Stemper wrote:
> use environment variables in one-liners (like my opening example had).
I'd like to nitpick a moment. You aren't actually changing how sed
> use environment variables in one-liners (like my opening example had).
operates at all. Rather you're relying on the shell to dynamically
change the commands that you're asking sed to execute from the command
line. You aren't actually passing anything to sed. Further, sed isn't
doing anything different on the command line vs the script file.
> What I have been unable to find is any description of how to pass
> something into a sed *program*. This seems to be a very basic action. Is
> it really true that it's not possible?
space with the variable. Then refactor your script file (program as you
called it) to g or G command to "copy / append hold space to pattern
space" in the proper locations in your sed script.
I've not tried this at all, so it may eat your dog. I don't even know
if you can combine -e and -f the way that I'm suggesting.
The only other thing that I can think of is to nest sed(s). Have the
outer execute the commands that the inner sed dynamically modifies and
prints to STDOUT. Or some such shenanigans.
This is also one of the reason that I like to use awk or m4 for some
text processing.
Good luck.
Please share your findings.
--
Grant. . . .
unix || die
Michael F. Stemper
May 5, 2018, 3:52:59 PM5/5/18
to
On 2018-05-04 13:37, Grant Taylor wrote:
> On 05/04/2018 08:53 AM, Michael F. Stemper wrote:
>> use environment variables in one-liners (like my opening example had).
>
> I'd like to nitpick a moment. You aren't actually changing how sed
> operates at all. Rather you're relying on the shell to dynamically
> change the commands that you're asking sed to execute from the command
I know that in the initial one-shot, the shell was doing the expanding.
> On 05/04/2018 08:53 AM, Michael F. Stemper wrote:
>> use environment variables in one-liners (like my opening example had).
>
> I'd like to nitpick a moment. You aren't actually changing how sed
> operates at all. Rather you're relying on the shell to dynamically
> change the commands that you're asking sed to execute from the command
It was only included to show potential respondents what I wanted to do.
>> What I have been unable to find is any description of how to pass
>> something into a sed *program*. This seems to be a very basic action. Is
>> it really true that it's not possible?
>
> Try using the -e option to pass a command to populate the hold pattern
> space with the variable. Then refactor your script file (program as you
> called it) to g or G command to "copy / append hold space to pattern
> space" in the proper locations in your sed script.
led me to:
<https://stackoverflow.com/questions/12833714/the-concept-of-hold-space-and-pattern-space-in-sed>
which says:
"[...] Hold spaces, pattern spaces, and 95% of the sed language [...]
became obsolete as soon as awk was invented in the mid-1970s [...] If
you are using more than s, g, and p (with -n) in sed then you are
almost certainly using the wrong tool."
The bit about it not having changed since the '70s probably explains
why it doesn't have the facility to accept arguments from the command
line, which seems like a no-brainer.
> This is also one of the reason that I like to use awk or m4 for some
> text processing.
python program.
Thanks for the pointer.
--
Michael F. Stemper
Nostalgia just ain't what it used to be.
Joe Rosevear
Jun 18, 2018, 12:17:07 AM6/18/18
to
Michael F. Stemper <michael...@gmail.com> wrote:
[snip]
> Then, I tried to actually pass the variable in to the program as a
> command-line argument:
>
> username@hostname$ cat OldForm.sed
> s/,/) ($1,/g
> s/^/($1,/
> s/$/)/
> username@hostname$ echo 7 | sed -f OldForm.sed $order
> sed: can't read 42: No such file or directory
> username@hostname$
>
> What's happened is obvious: sed is interpreting the "42" as the
> name of a file to read, not as an argument.
>
> I've gone through the O'Reilly book without finding out how to do
> this. I've also spent some time searching on-line, and been told
> over and over again how to use environment variables in one-liners
> (like my opening example had).
>
> What I have been unable to find is any description of how to pass
> something into a sed *program*. This seems to be a very basic
> action. Is it really true that it's not possible?
Hi, I'm a little late in replying, but here is a solution.
I wrote a bash script called OldForm2.sh and I gave it execute
permission:
#!/bin/sh
sed "s/,/) ($1,/g" | \
sed 's/^/('$1',/' | \
sed 's/$/)/'
Here is a sample execution:
joe@zillo:/tmp/joe/scratch$ order=42
joe@zillo:/tmp/joe/scratch$ echo 2,3,5,7 | ./OldForm2.sh $order
(42,2) (42,3) (42,5) (42,7)
What I wrote is very close to what you wrote. The differences are:
1. It is a bash script instead of a sed file.
2. I pass $order not to sed, but to the bash script.
3. I use the continuation symbol "\" to arrange the script in a
format that resembles your sed file.
4. I was careful to use ' and " correctly. Note that for
demonstration I used " in the first line and ' in the second line.
The third line has no $1 so either ' or " could be used in the same
way.
Observe what I did in the second line. $1 is not *in* a string,
because it *is* a string. Bash concatenates strings placed end to
end. The use of " is different because it allows dereferencing of
positional parameters.
I hope this is not too late to be of value. By the way, thanks for the
interesting illustration of use of sed. Although I knew how to solve
the positional parameter problem, I did not know how to pass comma
separated values to sed as you did. Cool.
-Joe
[snip]
> Then, I tried to actually pass the variable in to the program as a
> command-line argument:
>
> username@hostname$ cat OldForm.sed
> s/,/) ($1,/g
> s/^/($1,/
> s/$/)/
> username@hostname$ echo 7 | sed -f OldForm.sed $order
> sed: can't read 42: No such file or directory
> username@hostname$
>
> What's happened is obvious: sed is interpreting the "42" as the
> name of a file to read, not as an argument.
>
> I've gone through the O'Reilly book without finding out how to do
> this. I've also spent some time searching on-line, and been told
> over and over again how to use environment variables in one-liners
> (like my opening example had).
>
> What I have been unable to find is any description of how to pass
> something into a sed *program*. This seems to be a very basic
> action. Is it really true that it's not possible?
I wrote a bash script called OldForm2.sh and I gave it execute
permission:
#!/bin/sh
sed "s/,/) ($1,/g" | \
sed 's/^/('$1',/' | \
sed 's/$/)/'
Here is a sample execution:
joe@zillo:/tmp/joe/scratch$ order=42
joe@zillo:/tmp/joe/scratch$ echo 2,3,5,7 | ./OldForm2.sh $order
(42,2) (42,3) (42,5) (42,7)
What I wrote is very close to what you wrote. The differences are:
1. It is a bash script instead of a sed file.
2. I pass $order not to sed, but to the bash script.
3. I use the continuation symbol "\" to arrange the script in a
format that resembles your sed file.
4. I was careful to use ' and " correctly. Note that for
demonstration I used " in the first line and ' in the second line.
The third line has no $1 so either ' or " could be used in the same
way.
Observe what I did in the second line. $1 is not *in* a string,
because it *is* a string. Bash concatenates strings placed end to
end. The use of " is different because it allows dereferencing of
positional parameters.
I hope this is not too late to be of value. By the way, thanks for the
interesting illustration of use of sed. Although I knew how to solve
the positional parameter problem, I did not know how to pass comma
separated values to sed as you did. Cool.
-Joe
Michael F. Stemper
Aug 12, 2018, 1:16:26 PM8/12/18
to
On 2018-06-17 23:17, Joe Rosevear wrote:
> Michael F. Stemper <michael...@gmail.com> wrote:
>> Then, I tried to actually pass the variable in to the program as a
>> command-line argument:
>>
>> username@hostname$ cat OldForm.sed
>> s/,/) ($1,/g
>> s/^/($1,/
>> s/$/)/
>> username@hostname$ echo 7 | sed -f OldForm.sed $order
>> sed: can't read 42: No such file or directory
>> username@hostname$
>>
>> What's happened is obvious: sed is interpreting the "42" as the
>> name of a file to read, not as an argument.
> Michael F. Stemper <michael...@gmail.com> wrote:
>> Then, I tried to actually pass the variable in to the program as a
>> command-line argument:
>>
>> username@hostname$ cat OldForm.sed
>> s/,/) ($1,/g
>> s/^/($1,/
>> s/$/)/
>> username@hostname$ echo 7 | sed -f OldForm.sed $order
>> sed: can't read 42: No such file or directory
>> username@hostname$
>>
>> What's happened is obvious: sed is interpreting the "42" as the
>> name of a file to read, not as an argument.
>> What I have been unable to find is any description of how to pass
>> something into a sed *program*. This seems to be a very basic
>> action. Is it really true that it's not possible?
>
> Hi, I'm a little late in replying, but here is a solution.
>
> I wrote a bash script called OldForm2.sh and I gave it execute
> permission:
>
> #!/bin/sh
>
> sed "s/,/) ($1,/g" | \
> sed 's/^/('$1',/' | \
> sed 's/$/)/'
>> something into a sed *program*. This seems to be a very basic
>> action. Is it really true that it's not possible?
>
> Hi, I'm a little late in replying, but here is a solution.
>
> I wrote a bash script called OldForm2.sh and I gave it execute
> permission:
>
> #!/bin/sh
>
> sed "s/,/) ($1,/g" | \
> sed 's/^/('$1',/' | \
> sed 's/$/)/'
> What I wrote is very close to what you wrote. The differences are:
>
> 1. It is a bash script instead of a sed file.
>
> 2. I pass $order not to sed, but to the bash script.
Y'know, I usually do my sed-ing as one (or several) part(s) of a long
>
> 1. It is a bash script instead of a sed file.
>
> 2. I pass $order not to sed, but to the bash script.
pipe. I have no idea why I was so bound and determined to make a sed
program this time. Especially since it was for a one-time data format
conversion.
> I hope this is not too late to be of value.
still helps to be reminded that there are multiple ways to skin any
cat.
> By the way, thanks for the
> interesting illustration of use of sed. Although I knew how to solve
> the positional parameter problem, I did not know how to pass comma
> separated values to sed as you did. Cool.
--
Michael F. Stemper
Deuteronomy 24:17
0 new messages