Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

sed and bash-script --- help in search and replace content of file.

122 views
Skip to first unread message

raxitsh...@gmail.com

unread,
Jul 24, 2008, 11:04:29 AM7/24/08
to
Hi,

in file_map i do have following entry


$cat file_map
@Somestring@ (at line1)
@some_other_string@ (at line2)
@String3@ (at line3)
@Anyotherstrin@(at line4)
@Yetsomedifferentstring@ (at line5)


$cat data_file
This file content some data, including some content of file_map

What i want to do is

My batcshscripts
read line1 of file_map, replace all occurence of @somestring@ with
@1@ in data file.
read line2 of file_map, replace @some_other_string@ with @2@ in data
file
and likewise....

-----------------This is not working !--------

value=0;
while read line
do
value=`expr $value + 1`;
value1='@'$value'@';
echo $line is replace by $value1 >> file_fin
sed 's/$line/$value1/g' data_file;
done < "file_map"
---------------------------------------

In above content of file_fin comes perfectly fine, but in data file
text is not replaced !

Any idea why ?

Thanks in advance,
Raxit Sheth

Dave B

unread,
Jul 24, 2008, 11:13:53 AM7/24/08
to
Ra...@mykavita.com wrote:

> value=0;
> while read line
> do
> value=`expr $value + 1`;
> value1='@'$value'@';
> echo $line is replace by $value1 >> file_fin
> sed 's/$line/$value1/g' data_file;
> done < "file_map"
> ---------------------------------------
>
> In above content of file_fin comes perfectly fine, but in data file
> text is not replaced !

First, you put variables between single quotes when invoking sed, so the
variables are not interpolated by the shell and thus sed does not see their
values. Use double quotes instead.
Second, the sed command writes to standard output, thus even if you
corrected the first problem the file data_file would still remain unchanged.

There are other problems, but the two above are likely the explanation of
what you're seeing.

--
echo 0|sed 's909=oO#3u)o19;s0#0ooo)].O0;s()(0bu}=(;s#}#.1m"?0^2{#;
s)")9v2@3%"9$);so%op]t(p$e#!o;sz(z^+.z;su+ur!z"au;sxzxd?_{h)cx;:b;
s/\(\(.\).\)\(\(..\)*\)\(\(.\).\)\(\(..\)*#.*\6.*\2.*\)/\5\3\1\7/;
tb'|awk '{while((i+=2)<=length($1)-18)a=a substr($1,i,1);print a}'

raxitsh...@gmail.com

unread,
Jul 24, 2008, 11:31:23 AM7/24/08
to
On Jul 24, 8:13 pm, Dave B <da...@addr.invalid> wrote:
> Ra...@mykavita.com wrote:
> > value=0;
> > while read line
> > do
> > value=`expr $value + 1`;
> > value1='@'$value'@';
> > echo  $line  is replace by $value1  >> file_fin
> > sed   's/$line/$value1/g'  data_file;
> > done < "file_map"
> > ---------------------------------------
>
> > In above content of file_fin comes perfectly fine, but in data file
> > text is not replaced !
>
> First, you put variables between single quotes when invoking sed, so the
> variables are not interpolated by the shell and thus sed does not see their
> values. Use double quotes instead.
> Second, the sed command writes to standard output, thus even if you
> corrected the first problem the file data_file would still remain unchanged.
>
> There are other problems, but the two above are likely the explanation of
> what you're seeing.
Thanks, i did that... now it is only replacing First Value... i am
redirecting to output file.

bash$cat ./script.sh


value=0;
while read line
do
value=`expr $value + 1`;
value1='@'$value'@';
echo $line is replace by $value1 >> file_fin

sed "s/$line/$value1/g" data_file >> output;
done < "file_map"


Tnx.

Dave B

unread,
Jul 24, 2008, 11:46:02 AM7/24/08
to
Ra...@mykavita.com wrote:

> Thanks, i did that... now it is only replacing First Value... i am
> redirecting to output file.
>
> bash$cat ./script.sh
> value=0;
> while read line
> do
> value=`expr $value + 1`;
> value1='@'$value'@';
> echo $line is replace by $value1 >> file_fin
> sed "s/$line/$value1/g" data_file >> output;
> done < "file_map"

This is what happens now: you look into data_file for the first line in
file_map, replace it with @1@, and append the output of this substitution to
"output". Then you look into data_file (still the original one, untouched)
for the second line in file_map, replace it with @2@, and append the output
of this substitution to "output", and so on. If you want all the changes to
go into the same file, you have to use a temporary file, that each time
replaces the data file. An example could be:

value=0;

# for safety, we make a copy of data_file and work on the copy
cp data_file work_data

while IFS= read -r line; do
value=$((value + 1));
value1="@$value@";


echo "$line is replace by $value1" >> file_fin

# look for $line in work_data, replace it with $value1, and write the
# results to "output"
sed "s/$line/$value1/g" work_data > output;

# now rename "output" to "work_data", to incorporate changes. The next
# iteration will do another change, and so on
mv -f output work_data;

done < file_map

If you're happy with the contents of the file "work_data" after the loop,
you can rename it to "data_file".

Note that the algorithm you are using is quite inefficient and can be very
slow if the data file is very large.

raxitsh...@gmail.com

unread,
Jul 24, 2008, 1:24:43 PM7/24/08
to

Dave,
Many Thanks....

[root@localhost BusRoute]# ./script1.sh
sed: -e expression #1, char 33: unknown option to `s'
sed: -e expression #1, char 36: unknown option to `s'
sed: -e expression #1, char 29: unknown option to `s'
sed: -e expression #1, char 36: unknown option to `s'
sed: -e expression #1, char 38: unknown option to `s'
sed: -e expression #1, char 40: unknown option to `s'
sed: -e expression #1, char 37: unknown option to `s'
sed: -e expression #1, char 42: unknown option to `s'
sed: -e expression #1, char 24: unknown option to `s'
sed: -e expression #1, char 33: unknown option to `s'
sed: -e expression #1, char 35: unknown option to `s'
sed: -e expression #1, char 33: unknown option to `s
..
..

Dave B

unread,
Jul 24, 2008, 1:47:32 PM7/24/08
to
Ra...@mykavita.com wrote:

>> echo "$line is replace by $value1" >> file_fin

>> sed "s/$line/$value1/g" work_data > output;

> [root@localhost BusRoute]# ./script1.sh
> sed: -e expression #1, char 33: unknown option to `s'
> sed: -e expression #1, char 36: unknown option to `s'
> sed: -e expression #1, char 29: unknown option to `s'
> sed: -e expression #1, char 36: unknown option to `s'
> sed: -e expression #1, char 38: unknown option to `s'
> sed: -e expression #1, char 40: unknown option to `s'
> sed: -e expression #1, char 37: unknown option to `s'
> sed: -e expression #1, char 42: unknown option to `s'
> sed: -e expression #1, char 24: unknown option to `s'
> sed: -e expression #1, char 33: unknown option to `s'
> sed: -e expression #1, char 35: unknown option to `s'
> sed: -e expression #1, char 33: unknown option to `s

Assuming you did not change the sed line, you probably have strange
characters in the values of "$line" or "$value1".
Inspect the contents of "file_fin" and see if some line contains slashes.
Then try using a different separator (one that does not occur in the
variables), like eg

sed "s_$line_$value1_g" work_data > output;

The "_" is just an example, you can use any character as long as it does not
appear in $line and $value1.

Dave B

unread,
Jul 24, 2008, 1:52:53 PM7/24/08
to
Dave B wrote:

> sed "s_$line_$value1_g" work_data > output;

Oh, and another note: use braces to clearly delimit variables, like eg

sed "s_${line}_${value1}_g"

raxitsh...@gmail.com

unread,
Jul 24, 2008, 2:03:43 PM7/24/08
to
On Jul 24, 10:52 pm, Dave B <da...@addr.invalid> wrote:
> Dave B wrote:
> >  sed "s_$line_$value1_g"  work_data > output;
>
> Oh, and another note: use braces to clearly delimit variables, like eg
>
>  sed "s_${line}_${value1}_g"
Dave,

Thanks, its done. but one more query....
to start with sed, which book (online link preferrable) should i go
for

-Raxit

Dave B

unread,
Jul 24, 2008, 2:42:04 PM7/24/08
to
Ra...@mykavita.com wrote:

> Thanks, its done. but one more query....
> to start with sed, which book (online link preferrable) should i go
> for

There are various documents and resources listed in section 2.3 of the sed FAQ:

http://student.northpark.edu/pemente/sed/sedfaq2.html#s2.3

Bin Chen

unread,
Jul 25, 2008, 1:03:00 AM7/25/08
to

Sed never update the file being replaced, you should redirect the
output of sed to a temp file and then copy the temp file to original
file.

eg)

sed 's/$line/$value1/g' data_file > temp; mv temp data_file;

Bin

James Kanze

unread,
Jul 25, 2008, 8:56:42 AM7/25/08
to
On Jul 25, 7:03 am, Bin Chen <binary.c...@gmail.com> wrote:

[...]


> Sed never update the file being replaced, you should redirect
> the output of sed to a temp file and then copy the temp file
> to original file.

> eg)

> sed 's/$line/$value1/g' data_file > temp; mv temp data_file;

That's:
sed 's/$line/$value1/g' data_file > temp && mv temp data_file;


--
James Kanze (GABI Software) email:james...@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

Ben Bacarisse

unread,
Jul 25, 2008, 9:00:54 AM7/25/08
to
Bin Chen <binar...@gmail.com> writes:

...and the '' should be "" or the variables will not be substituted.

--
Ben.

0 new messages