How to cut and paste to new file

[ismailctest C]

Hi,

From archives.log files, the below matched lines should be cut and paste to newfile.log

Condition1: If available apple & orange in the same line (cut & paste to new file)

Condition2: If available apple, orange & grape  (skip it, dont cut and paste)

Please help with sed,awk etc commands.

I have tried the below command:

grep -Ew "apple.*orange" /home/name/syslog.log | grep -vw "grape" | awk '{print "prefix " $0}' >> /home/name/newfile.log

Update: if matching the condition logs are copying to newfile.log with prefix, result is getting what I am expecting.

Issue: when running cron 1st time, if matching 10 lines, then those lines are copying to newfile.log. Then, 2nd time running time, if matching another new 5 lines, then copying first 10 lines & 2nd 5 lines, its repeating & duplicate logs are coming. Note: How to check and skip duplicate logs when copying logs to newfile.log

Need help:
Duplicate wont come, first copied 10 lines , don't copy while running the cron next time. Should be copied only new lines matched. (Eg. 1st 10 lines keep there in newfile.log & next new 5 lines only append in

Note: Even removing matched lines from syslog.log after moved to newfile.log, this is also okay for me.

Please support, I have tried rsync, cp,sed,awk commands , not getting any idea how to use it.

[Jörg Schin.]

I have some script getting lines i like and write them in another file.

im not sure if there will be any performance issues when doing a cron to often with a cat command on a huge logfile?

i did it with a while loop and read every file itself. 

while read line; do

regex=$(grep "(.*apple.*|.*orange.*)[^grape]")

if [[ $line -eq $regex ]] then

echo "$prefix $line" >> fancylog.log

fi

done

this is just a out of my head example. but the regex should be fine see: https://regex101.com/r/TjKXT9/1

and you should have a look if you dont need to escape the .* to \.*

maybe you could give it a try :) 

[David Correa Rodriguez]

Hello.
About the grep command, it looks good to me. It searches for lines that contain the words "apple" and "orange" but not "grape" in the file "/home/name/syslog.log", and appends them with the prefix "prefix " to the file "/home/name/newfile.log".
To perform what you are trying to achieve, two ideas come to my mind:
- You could clear the newfile.log before running the command by replacing ">>" to ">". This solution is easy to apply but it is not the best, as the command will catch lines that it catched before, but you will not have duplicated lines in the new log.
- Another more complex solution is to track the last line that was read and only append the lines that come after it.

You can do this by storing the last line that was read in a temporary file, and using awk to extract the lines that come after it:

LAST_LINE=$(cat /tmp/last_line)
grep -A9999999 "prefix $LAST_LINE" /home/name/syslog.log | grep -Ew "apple.*orange" | grep -vw "grape" | awk '{if (NR>1) print}' | awk '{print "prefix " $0}' >> /home/name/newfile.log
grep "prefix.*orange" /home/name/newfile.log | tail -1 | awk '{gsub("prefix ",""); print}' > /tmp/last_line

This command assumes that you have a file named /tmp/last_line that contains the last line that was read. 

The first command reads the contents of this file into a variable called LAST_LINE.
The grep -A9999999 "prefix $LAST_LINE" /home/name/syslog.log command searches for the last line that matches the pattern "prefix $LAST_LINE" in "/home/name/syslog.log", and includes all lines that come after it. The -A9999999 option ensures that all lines after the matching line are included.
The first awk command ('{if (NR>1) print}') skips the first line (which will be the line that matches the pattern "prefix $LAST_LINE") so that only the lines that come after it are printed. The second awk command ('{print "prefix " $0}') adds the "prefix " string to the beginning of each line.

Finally, the last command extracts the last line that was appended to "newfile.log" that matches the pattern "prefix.*orange", removes the "prefix " string from it, and writes it to the "/tmp/last_line" file.

Hope it helps.

[ismailctest C]

Hi David,

Thanks , will check this script and update the status.

Need another help also, It searches for lines that contain the words "apple" and "orange" but not "grape" in the file "/home/name/syslog.log"

Then, needs to be removed those lines.

Pl support with any command like sed etc..

[David Correa Rodriguez]

Hello.

You can use the sed command to delete those lines from the file. Here's an example command:

sed -i '/grape/!{/apple.*orange/d}' /home/name/syslog.log

This command uses two expressions inside braces to delete lines that match the pattern "/apple.*orange/" but do not match the pattern "/grape/". The `!` operator before the "/grape/" pattern means "not", so the command will skip any lines that match the "grape" pattern.

The -i option modifies the file "/home/name/syslog.log" in place, so it will remove the lines directly from the file. Be careful when using the `-i` option, as it can modify files irreversibly.

Hope it helps.