Using sed within a awk statement.
Tibbz
awk '{
system ("sed 's/"$1" /"$2" /g' tempfile0" )
}' ipdata.data
I keep getting this error.
awk: cmd. line:1: system ("sed s/
awk: cmd. line:1: ^ unterminated string
For those who like details read below -------------------
Part of a Korn shell script I am writing needs to pull a IP address
from a reference file and use that IP address to replace a phone
number in another load file. For example.
ipaddressfile
2031230987 10.1.2.3
2033211234 10.2.3.4
2033453214 10.3.4.5
loadfile
2031230987 889 000928 Stop 20312...@abc.abc.com FIL 07012004 1
928 18 BSID
2033453214 576 003133 Stop 20334...@abc.abc.com FIL 07012004 1
928 18 BSID
2031230987 137 012744 Start 20312...@abc.abc.com FIL 07012004 1
155 8 BSID
I need to change the lead phone number to its associated IP. This is
what the results should look like.
10.1.2.3 889 000928 Stop 20312...@abc.abc.com FIL 07012004 1 928
18 BSID
10.3.4.5 576 003133 Stop 20334...@abc.abc.com FIL 07012004 1 928
18 BSID
10.1.2.3 137 012744 Start 20312...@abc.abc.com FIL 07012004 1 155
8 BSID
William Park
> I am having trouble with this awk statement.
>
> awk '{
> system ("sed 's/"$1" /"$2" /g' tempfile0" )
> }' ipdata.data
>
> I keep getting this error.
>
> awk: cmd. line:1: system ("sed s/
> awk: cmd. line:1: ^ unterminated string
Error means what it says. You have
'{
system ("sed '
So, change all your internal single quotes to
'\''
>
>
> For those who like details read below -------------------
>
>
> Part of a Korn shell script I am writing needs to pull a IP address
> from a reference file and use that IP address to replace a phone
> number in another load file. For example.
>
> ipaddressfile
>
> 2031230987 10.1.2.3
> 2033211234 10.2.3.4
> 2033453214 10.3.4.5
>
> loadfile
>
> 2031230987 889 000928 Stop 20312...@abc.abc.com FIL 07012004 1
> 928 18 BSID
> 2033453214 576 003133 Stop 20334...@abc.abc.com FIL 07012004 1
> 928 18 BSID
> 2031230987 137 012744 Start 20312...@abc.abc.com FIL 07012004 1
> 155 8 BSID
>
> I need to change the lead phone number to its associated IP. This is
> what the results should look like.
>
> 10.1.2.3 889 000928 Stop 20312...@abc.abc.com FIL 07012004 1 928
> 18 BSID
> 10.3.4.5 576 003133 Stop 20334...@abc.abc.com FIL 07012004 1 928
> 18 BSID
> 10.1.2.3 137 012744 Start 20312...@abc.abc.com FIL 07012004 1 155
> 8 BSID
Change your 'ipaddressfile' to 'ipaddressfile.sed',
s/^2031230987/10.1.2.3/
s/^2033211234/10.2.3.4/
s/^2033453214/10.3.4.5/
and run 'sed' directly,
sed -f ipaddressfile.sed loadfile
Ref:
man sed
--
William Park <openge...@yahoo.ca>
Open Geometry Consulting, Toronto, Canada
Kenny McCormack
Tibbz <tib...@hotmail.com> wrote:
>I am having trouble with this awk statement.
>
>awk '{
>system ("sed 's/"$1" /"$2" /g' tempfile0" )
>}' ipdata.data
>
>I keep getting this error.
>
>awk: cmd. line:1: system ("sed s/
>awk: cmd. line:1: ^ unterminated string
You should never use sed (or any of the other "lesser tools") when using AWK.
There are basically 2 exceptions to this rule:
1) There is a small set of very obscure things that sed can do that
awk cannot. I am not aware of what any of these are, but from time
to time, sedders will mention this, so I will take their word for it.
2) Performance. If the input file is very large and/or the machine
very slow, sometimes you can get a performance boost by using
grep or sed or something like that (that is more lightweight than
AWK) to do some of the processing. This is rare.
E. Rosten
> You should never use sed (or any of the other "lesser tools") when using AWK.
> There are basically 2 exceptions to this rule:
> 1) There is a small set of very obscure things that sed can do that
> awk cannot. I am not aware of what any of these are, but from time
> to time, sedders will mention this, so I will take their word for it.
Anything with backrefrences, althought gawk suports these by the
non-POSIX extension gensub (do other AWKs?), but only for substitution,
not searching.
> 2) Performance. If the input file is very large and/or the machine
> very slow, sometimes you can get a performance boost by using
> grep or sed or something like that (that is more lightweight than
> AWK) to do some of the processing. This is rare.
It's also probably in most cases premature optimization, which is as we
all know, is evil (the very root of it, no less).
-Ed
--
(You can't go wrong with psycho-rats.) (er258)(@)(eng.cam)(.ac.uk)
/d{def}def/f{/Times findfont s scalefont setfont}d/s{10}d/r{roll}d f 5/m
{moveto}d -1 r 230 350 m 0 1 179{1 index show 88 rotate 4 mul 0 rmoveto}
for /s 15 d f pop 240 420 m 0 1 3 { 4 2 1 r sub -1 r show } for showpage
Kenny McCormack
>Kenny McCormack wrote:
>
>> You should never use sed (or any of the other "lesser tools") when using AWK.
>> There are basically 2 exceptions to this rule:
>> 1) There is a small set of very obscure things that sed can do that
>> awk cannot. I am not aware of what any of these are, but from time
>> to time, sedders will mention this, so I will take their word for it.
>
>Anything with backrefrences, althought gawk suports these by the
>non-POSIX extension gensub (do other AWKs?), but only for substitution,
>not searching.
Well, I only use gawk and tawk, both of which have back-referencing in
substitutions (albeit implemented differently - and, of course, I prefer
the tawk method). I'm not sure what you by "for searching".
>> 2) Performance. If the input file is very large and/or the machine
>> very slow, sometimes you can get a performance boost by using
>> grep or sed or something like that (that is more lightweight than
>> AWK) to do some of the processing. This is rare.
>
>It's also probably in most cases premature optimization, which is as we
>all know, is evil (the very root of it, no less).
Is it "the *love of* premature optimization" ?
BTW, this would make a good question for "Street Smarts" - ask people what
"premature optimization" is - and see what kind of responses you get...
Kenny McCormack
Kenny McCormack <gaz...@interaccess.com> wrote:
...
>the tawk method). I'm not sure what you by "for searching".
This should, of course, read "I'm not sure what you mean by ..."
Brian Inglis
(Tibbz) wrote:
>I am having trouble with this awk statement.
>
>awk '{
>system ("sed 's/"$1" /"$2" /g' tempfile0" )
>}' ipdata.data
>
>I keep getting this error.
>
>awk: cmd. line:1: system ("sed s/
>awk: cmd. line:1: ^ unterminated string
You have a quoting problem; what awk is seeing is:
awk '{ system("sed 's/"$1" /"$2" /g' tempfile0" )}' ipdata.data
^^^^^^^^^^^^^^^^^^^^^^ ^^^^^ ^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^
arg1 arg2 arg3 arg4
try this instead:
awk '{ system("sed '"'"'s/"$1" /"$2" /g'"'"' tempfile0")}' ipdata.data
^^^ ^^^
embedded single quotes
unless you're running under a DOS/Windows shell, in which case just
put the awk code in a file.
ISTM you might be better using join to do the lookup, by having both
input files start with the phone number, sorted in phone number order,
and then use awk to filter the output:
sort -o ipdata.sort ipdata.data ; join -a 1 ipdata.sort tempfile0 |...
--
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada
Brian....@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]ab[dot]ca)
fake address use address above to reply
E. Rosten
> Well, I only use gawk and tawk, both of which have back-referencing in
> substitutions (albeit implemented differently - and, of course, I prefer
> the tawk method). I'm not sure what you by "for searching".
In sed, you can use backrefs in a regex, vis:
/\([0-9][0-9]*\)-\1/p
to print all lines of the form 123-123
In gawk, it would seem that one can only use them in the substitution,
not in the regex itself.
>>It's also probably in most cases premature optimization, which is as we
>>all know, is evil (the very root of it, no less).
>
>
> Is it "the *love of* premature optimization" ?
I don't know. Could be either. Premature optimization is bad enough. But
love of it (*cough* linux *cough*) is just plain awful.
Ed Morton
Tibbz wrote:
> I am having trouble with this awk statement.
>
> awk '{
> system ("sed 's/"$1" /"$2" /g' tempfile0" )
> }' ipdata.data
>
> I keep getting this error.
>
> awk: cmd. line:1: system ("sed s/
> awk: cmd. line:1: ^ unterminated string
Rather than trying to hack a solution to that, the real answer is one of:
a) Don't call sed from awk, just use awk or
b) Don't call sed from awk, call sed from a shell loop
In either case, don't call sed from awk.
Either solution is pretty trivial, but since this is comp.lang.awk:
awk 'NR == FNR {p2i[$1]=$2; next}
{$1 = p2i[$1]; print}' ipaddressfile loadfile
It could be abbreviated to:
awk 'NR==FNR{p2i[$1]=$2;next}$1=p2i[$1]' ipaddressfile loadfile
if you prefer brevity and are sure there will always be an IP address
for every phone number.
If you want to guard against phone numbers without IP addresses, and
handle them explicitly, do something like this:
awk 'NR == FNR {p2i[$1]=$2; next}
$1 in p2i {$1 = p2i[$1]; print; next}
{print "No IP addr: " $0}' ipaddressfile loadfile
Regards,
Ed.
Kenny McCormack
>Kenny McCormack wrote:
>
>> Well, I only use gawk and tawk, both of which have back-referencing in
>> substitutions (albeit implemented differently - and, of course, I prefer
>> the tawk method). I'm not sure what you by "for searching".
>
>
>In sed, you can use backrefs in a regex, vis:
>
>/\([0-9][0-9]*\)-\1/p
>
>to print all lines of the form 123-123
>
>In gawk, it would seem that one can only use them in the substitution,
>not in the regex itself.
I see. As it turns out, I've actually had to do this sort of thing in
tawk, and it has to be done in two steps - where basically you use match(),
and then the backreferencing built into that function (RSTART & RLENGTH) to
figure out what else needs to match. I would imagine that there is nothing
you can do in sed that you can't do in AWK with a little more coding. But
I'm sure some eager sedder will come along with a counter-example.
>>>It's also probably in most cases premature optimization, which is as we
>>>all know, is evil (the very root of it, no less).
>>
>>
>> Is it "the *love of* premature optimization" ?
>
>I don't know. Could be either. Premature optimization is bad enough. But
>love of it (*cough* linux *cough*) is just plain awful.
I was making a joke based on the idea that it is not money that is the root
of all evil, but rather "the love of money". But just out of curiosity,
what is it about Linux that arouses your ire?
E. Rosten
> I see. As it turns out, I've actually had to do this sort of thing in
> tawk, and it has to be done in two steps - where basically you use match(),
> and then the backreferencing built into that function (RSTART & RLENGTH) to
> figure out what else needs to match. I would imagine that there is nothing
> you can do in sed that you can't do in AWK with a little more coding. But
> I'm sure some eager sedder will come along with a counter-example.
To be honest (this is coming from a pretty keen sedder) that the main
use of sed is that it can be quicker to do stuff in sed: suick
one-liners are ususlly quicker.
> I was making a joke based on the idea that it is not money that is the root
> of all evil, but rather "the love of money". But just out of curiosity,
> what is it about Linux that arouses your ire?
The odd bits of the kernel here and there. People seem to love
optimizing driver code before the driver is stable, in particular. Well
one or two people do anyway. I can't remember off hand which bits it is
anyway. Oh, and there's that whole thing with virtual functions, where a
NULL pointer means run the default function, which leads to the whole
test-for-null-and-run-function-or-derference (opposed to just plain old
derefreence) when anyone uses it. But I'm sure that is bourne out of
careful analysis with a profiler, as opposed to any evilness...
But it doens't really arouse my ire much at all, since it's there, it
works and it's pretty good.
Tibbz
> On 10 Sep 2004 08:26:42 -0700 in comp.lang.awk, tib...@hotmail.com
> (Tibbz) wrote:
>
> >I am having trouble with this awk statement.
> >
> >awk '{
> >system ("sed 's/"$1" /"$2" /g' tempfile0" )
> >}' ipdata.data
> >
> >I keep getting this error.
> >
> >awk: cmd. line:1: system ("sed s/
> >awk: cmd. line:1: ^ unterminated string
>
> You have a quoting problem; what awk is seeing is:
>
> awk '{ system("sed 's/"$1" /"$2" /g' tempfile0" )}' ipdata.data
> ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^ ^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^
> arg1 arg2 arg3 arg4
>
> try this instead:
>
> awk '{ system("sed '"'"'s/"$1" /"$2" /g'"'"' tempfile0")}' ipdata.data
> ^^^ ^^^
> embedded single quotes
>
> unless you're running under a DOS/Windows shell, in which case just
> put the awk code in a file.
>
> ISTM you might be better using join to do the lookup, by having both
> input files start with the phone number, sorted in phone number order,
> and then use awk to filter the output:
>
> sort -o ipdata.sort ipdata.data ; join -a 1 ipdata.sort tempfile0 |...
Thanks for all your help guys.
All of your soloutions were very helpful and informative.
guggach
i find, sed in awk ... exotic
sed can allmost what awk can (a little faster)
awk can more then sed can (a little slower)
make a choice, sed ... awk, maybe is
perl the solution ?
--
guggach
------------------------------------------------------------------------
Posted via http://www.codecomments.com
------------------------------------------------------------------------
Jürgen Kahrs
> i find, sed in awk ... exotic
> sed can allmost what awk can (a little faster)
> awk can more then sed can (a little slower)
> make a choice, sed ... awk, maybe is
> perl the solution ?
Speed is not everything. Portability is
sometimes more important. Therefore, Perl
is often not an option because the standard
Unix toolbox has awk and sed, sh and m4,
lex and yacc, but neither Perl nor Python: