Help with awk/sed string replacement.

[Matthew Kruer]

What i am trying to accomplish is to build a bash file to run

I have a file that contains a coma delimited fields (package name, build number, uarch) Example below
package1, 1.1.0.24-0, noarch
package2, 1.0.0.8-1, noarch
etc...

I am trying to read the file and generate a bash script based upon is contents
First thing i need to do is generate a bash script to change the version field in a file. This is the area that I am running into issues with because of the quotes and string replacement

awk -F ',' '{print "sed -i 's/version = '*'/version = '$2'/g' /path/$1 "}' inputfile

sed -i 's/version = '*'/version = '$2'/g' /path/$1
should become
sed -i s/version = '*'/version = '1.1.0.24-0'/g /path/package1

Or even better yet just run the sed command to replace the string in the other files.

Any incite of to what i am doing wrong would be helpful. I think i am over thinking the problem.

Thanks

[Ed Morton]

On 5/30/2012 5:22 PM, Matthew Kruer wrote:
> What i am trying to accomplish is to build a bash file to run
>
> I have a file that contains a coma delimited fields (package name, build number, uarch) Example below
> package1, 1.1.0.24-0, noarch
> package2, 1.0.0.8-1, noarch
> etc...
>
> I am trying to read the file and generate a bash script based upon is contents
> First thing i need to do is generate a bash script to change the version field in a file. This is the area that I am running into issues with because of the quotes and string replacement
>

You cannot use 's within an awk script that is using 's to delimit the script.

> awk -F ',' '{print "sed -i 's/version = '*'/version = '$2'/g' /path/$1 "}' inputfile

The above could be written in several ways, I imagine googling "how to use
single quote in awk script" would tell you how. Or put your script in a file and
run it using awk -f.

>
> sed -i 's/version = '*'/version = '$2'/g' /path/$1
> should become
> sed -i s/version = '*'/version = '1.1.0.24-0'/g /path/package1
>
> Or even better yet just run the sed command to replace the string in the other files.
>
> Any incite of to what i am doing wrong would be helpful. I think i am over thinking the problem.

s/incite/insight/

I'd just do it in shell. Something like this (untested):

while IFS=, read -r pkg ver rest; do
sed -i "s/version = '[^']*'/version = '$ver'/g" "/path/$pkg"
done < inputfile

Regards,

Ed.

[Janis Papanagnou]

Am 31.05.2012 01:22, schrieb Matthew Kruer:
> What i am trying to accomplish is to build a bash file to run
>
> I have a file that contains a coma delimited fields (package name,
> build number, uarch) Example below package1, 1.1.0.24-0, noarch
> package2, 1.0.0.8-1, noarch etc...
>
> I am trying to read the file and generate a bash script based upon is
> contents First thing i need to do is generate a bash script to change
> the version field in a file. This is the area that I am running into
> issues with because of the quotes and string replacement
> awk -F ',' '{print "sed -i 's/version = '*'/version = '$2'/g' /path/$1 "}' inputfile

A few hints...

If you put the whole awk program into a file, say mycmds.awk, and
call it as

awk -F, -f mycmds.awk inputfile >outputfile

you won't have the issues with the nested single quotes.

Using printf instead of print will result in code that will make it
clearer visible what output you will get.

You can also do both, the initialisation of the matching rules that
you have in inputfile, and the processing of the data files in one
awk run without need of another shell or a couple of individual sed
processes. For example...

awk -F ', ' -f myscript inputfile

where myscript contains

NR==FNR { f="/path/"$1 ; ARGV[ARGC++]=f ; vers[f]=$2 ; next }
{ gsub(/version = '*'/,"version = '"vers[FILENAME]"'") ; print }

The first line will extend the command line by the files to process,
and will set up a mapping table with the version information. The
second line will be processed for each subsequent files and will do
the substitution.

Janis

[Ed Morton]

Right but it won't modify the /path/* files as the OP wants. To get from what
you have above to that point is more effort than worthwhile compared to just
calling sed -i inside a while loop.

Ed.

[Janis Papanagnou]

Am 31.05.2012 02:52, schrieb Ed Morton:
> On 5/30/2012 6:03 PM, Janis Papanagnou wrote:

>> [...]

>
> Right but it won't modify the /path/* files as the OP wants.

Right, I missed the "-i".

> To get from
> what you have above to that point is more effort than worthwhile
> compared to just calling sed -i inside a while loop.

With the gawk features it's fairly simple to write to a temporary
file and overwrite the current file in the ENDFILE section, as in

NR==FNR { ... }
{ ... ; print >"tmp" }
ENDFILE { system("mv tmp "FILENAME") }

Using sed with -i will also have to create a temporary implicitly.
With other awks it's _a bit_ more overhead, I agree, but not much.
(One should also have a look at a non-conflicting tmp file name.)

A shell solution might be appropriate as well in this context, but
be aware that your proposal upthread needs a couple more tweaks as
well; have a closer look at the IFS issue, for example.

Janis

>
> Ed.

[Janis Papanagnou]

Am 31.05.2012 03:40, schrieb Janis Papanagnou:
> Am 31.05.2012 02:52, schrieb Ed Morton:
>> On 5/30/2012 6:03 PM, Janis Papanagnou wrote:
>>> [...]
>>
>> Right but it won't modify the /path/* files as the OP wants.
>
> Right, I missed the "-i".
>
>> To get from
>> what you have above to that point is more effort than worthwhile
>> compared to just calling sed -i inside a while loop.
>
> With the gawk features it's fairly simple to write to a temporary
> file and overwrite the current file in the ENDFILE section, as in
>
> NR==FNR { ... }
> { ... ; print >"tmp" }
> ENDFILE { system("mv tmp "FILENAME") }

Oops, there's also a close("tmp") necessary before the system(),
in case we don't want to use disjunct names like ".package1.tmp"
(instead of plain "tmp").

[Matthew Kruer]

Thanks to both of you the bash command, it simple and works how I expect it for what I need.

Also Ed

Another Issue that was just thrown on my plate is extracting the version number out of a fields below from an xml file

com.authorization-1.0.4.jar
com.application-2.14.0.737.jar
com.process.runtime_1.0.1.201204171940.jar
com.adaptor_1.0.0.201204171939.jar
com.event_1.0.1.201204171940.jar
com.io_1.0.0.201204171939.jar
com.message_1.0.2.201204171940.jar
com.process_1.0.0.201204171940.jar
com.queue_1.0.0.201204171939.jar
com.resource_1.0.0.201204171940.jar
com.search_1.0.5.201204171940.jar
com.service_1.0.0.201204171940.jar

The logic goes as follows
name (name.type.etc. possibly with - or _) name/build delimiter (- or _) build number (numbers and periods) extension (.jar)
The trick is that I don’t know all the name or length or it the will contain other symbols or numbers the only thing that I do know is that the delimiter should always be the – or_ followed by a number.

I thkn I can now use the “other” logic you posted about another build number logic, https://groups.google.com/forum/?fromgroups#!topic/comp.unix.shell/7rnrQ_Ljiho

gawk '
{ $0 =
gensub(/([[:alnum:]_]+(-next)?)-([[:digit:].-]+)[.]([[:alnum:]_]+).*/,"\\1\t\\3\t\\4","")
}
!a[$1,$3]++
' file

Thank aging for getting this before actually needed it.

[Matthew Kruer]

Another silly question. I am runing this command to replace all file matching the name file1, file2 and file3, and then replace the string NAME string with the FULLNAME

for i in "${!FULLNAME[@]}"
do
find $PROJECTDIR -name "file1" -o -name "file2" -o -name "file3" | xargs sed -i "s|${NAME[$i]}|${FULLNAME[$i]}|g"
done

This works but A feature I would like it for it to tell me what files it is changing and then print/echo the lines in the files that were changed.

find $PROJECTDIR -name "file1" -o -name "file2" -o -name "file3" | xargs sed -n "/${NAME[$i]}/p"

Is there an easy to acomplish this. I did see any good reporting ability within the sed command where it can print/echo out the name of the files and the lines being changed.

Thanks

[Bill Marcum]

You could get the filenames by piping through tee. To get the lines that
are changed, you could either use a temporary file, or use the gnu sed backup
option and then diff the files.

--
Take the henhouse keys away from the foxes.
Put Glass-Steagall in the Constitution!