Copying files with generated filenames in a directory from inside teh AWKprogram

48 views
Skip to first unread message

Mohsen Owzar

unread,
Sep 20, 2021, 11:33:17 PM9/20/21
to
Hi Guys
I'm working with Cygwin under Windows 10
I have written a program in AWK / Sh to copy or rename some selected files, filtered by a string given by the user, from 01 to the length of the selected files by the filter.
Before you say that this task can be done with another software than AWK, why I used this, OK, I know a bit only AWK and Bash scripting, and if you know another and more effective method than this one, do not hesitate and offer your suggestion.
But the thing is after so many hours struggling with this stuff, I want to know, how can I get these two variables (old and new filenames) out of AWK to copy or move
When I see a video tutorial, I take screenshots sporadically when some actions are done.
The thing is that the screenshots have generic names derived from the date and the name of the video and therefore can be very long, as you can see in the list below:
----------------------------------------------------
2021-09-08 10_10_42-PyQt5 - Apply your own Style using CSS - second method - updated - YouTube.png
2021-09-08 10_10_55-PyQt5 - Apply your own Style using CSS - second method - updated - YouTube.png
2021-09-08 10_13_37-Python PyQt5 -5- Make your first interaction with buttons using Signals Slots - .png
2021-09-08 10_13_46-Python PyQt5 -5- Make your first interaction with buttons using Signals Slots - .png
2021-09-10 07_08_53-Make Button Flashing _ PyQt5 Tutorial - YouTube.png
2021-09-10 07_12_40-Make Button Flashing _ PyQt5 Tutorial - YouTube.png
2021-09-15 09_19_39-Window.Window.Window.Window.png
2021-09-15 09_19_54-Window.png
2021-09-15 09_20_06-Window.png
----------------------------------------------------
Due to his issue, I wanted to give the user the possibility, to run my script with an argument, which is a part of the screenshot name. My AWK program picks this string and takes files containing this string and change their names to names beginning with two digits for 01 to the length of the selected files and concatenated with the given string.
For example:
2021-09-10 07_08_53-Make Button Flashing _ PyQt5 Tutorial - YouTube.png
2021-09-10 07_12_40-Make Button Flashing _ PyQt5 Tutorial - YouTube.png
with the given string as: " Button Flashing" is changed to
01_Button Flashing.png
02_Button Flashing.png
Now my problem:
I have managed to get the string from the user and sort the files in the directory by time and this string, and to get the old_file and new_file names.
I'm struggling since a couple of days to copy or move these files from old names to new names, vain and futile. It brings, all the time, that something is wrong with the destination and I have to check with "cp –help".
I think one problem is due to the fact that I'm using Linux commands (Cygwin) under Windows (CR/LF problem).
Another problem is that the filenames have spaces, and therefore I had to replace them forwards and backwards with some Characters. And the error messages after running the script has to do with this issue.
Because I found after a couple of hours, the given string had one character more than I need (I think CR/LF). Therefore, I had to get rid of the last character, as you can see in my code.
I have tried all combinations with system() command or print, no success. I hope someone can help me with this. I don't see the jungle because of all the trees.
Any help would be very appreciated.
Here is the output of the script:
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
$> Rename_Screenshots_For_String.sh "Button Flashing"

================================================
Date and given arguments
================================================
Generated on: 2021_09_21.05_05
TOP = /cygdrive/c/Users/m.owzar/Desktop/teesst
String = Button Flashing
DATUM = 2021_09_21.05_05
================================================
*** 2021-09-10 07_08_53-Make Button Flashing _ PyQt5 Tutorial - YouTube.png --> 01_Button_Flashing.png
*** 2021-09-10 07_12_40-Make Button Flashing _ PyQt5 Tutorial - YouTube.png --> 02_Button_Flashing.png

2 Files were found!
--------------------------------------------
71 : 2021-09-10 07_08_53-Make Button Flashing _ PyQt5 Tutorial - YouTube.png
22 : 01_Button_Flashing.png
/bin/cp: zusätzlicher Operand 'Button'
„/bin/cp --help“ liefert weitere Informationen.
71 : 2021-09-10 07_12_40-Make Button Flashing _ PyQt5 Tutorial - YouTube.png
22 : 02_Button_Flashing.png
/bin/cp: zusätzlicher Operand 'Button'
„/bin/cp --help“ liefert weitere Informationen.
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
And here is my code:
#! /usr/bin/bash
# Filename: Rename_Screenshots_For_String.sh
#########################################################
String=$1
DATUM=`date +"%Y_%m_%d.%H_%M"`
echo ""
echo "================================================"
echo "Date and given arguments"
echo "================================================"
echo "Generated on: " $DATUM
# =======================================================
ls -rt | grep -i "$1" | awk -v TOP="$(pwd)" -v String="$String" -v DATUM="$DATUM" '
########################################
## Begin Part
########################################
BEGIN {
print "TOP = " TOP
print "String = " String
print "DATUM = " DATUM
print "================================================"
}

########################################
## Main Part
########################################
{
if ($0 ~ / /) {
A = gensub(/ /, "_XXX_", "g", $0)
Arr_A[++N] = A
}

B = gensub(/_XXX_/, " ", "g", A)
Arr_B[++M] = B
}

########################################
## End Part
########################################

END {
Str = gensub(/ /, "_", "g", String)

for (i = 1; i <= M; i++) {
old_file[i] = Arr_B[i]

K = split(old_file[i], ARRAY, ".")
Extension = ARRAY[K]

Str_2 = substr(Str, 1, length(Str)-1)
new_file[i] = sprintf("%02d_%s.%s", i, Str_2, Extension)

printf "*** %s --> %s\n", old_file[i], new_file[i]

}
printf "\n%s Files were found!\n", M
print "--------------------------------------------"

for (i = 1; i <= M; i++) {
print length(old_file[i]), ": ", old_file[i]
print length(new_file[i]), ": ", new_file[i]

system("/bin/cp -T " old_file[i] " " new_file[i])
# print ""old_file[i]"" > new_file[i]
}
}
'

Janis Papanagnou

unread,
Sep 21, 2021, 4:14:55 AM9/21/21
to
A couple comments, pick your options...

Generally it's simpler to do the file manipulation in shell.
(If you provide a terse example and clear description the folks
(here or better in comp.unix.shell) will help you.)

If possible, get rid of punctuation characters and spaces.
Specifically get rid of the quotes as part of the file names.
Keep punctuation in date/time and gsub() all other punctuation
to become spaces. Then gsub() all sequences of spaces to single
underscore (for example).
Since the source file name still has spaces you need to quote the
whole file name that you provide as argument to mv/cp as in
filename = "\"" filename "\""
To do the renaming either construct the shell commands and print
them for execution awk '{printf ... }' | sh or use system(...)
to do it
cmd = "mv -i \"" filename "\" " newname
To prevent name clashes of the transformed names memorize them in
an array and (for example) add numbering if necessary
if (newname in names) names[newname]++
newname = names[newname] newname
For the shell 'mv' command use shell option '-i' to create an
error instead of overwriting existing files (just in case that
even the numbering creates an already existing file name).

There's probably even more hints necessary. A concise post with
a clear example and concrete questions will help you to get your
answers.

HTH.

Janis

PS: And try to keep your line lengths in your posts <78 characters;
this is still Usenet (not a webforum), search for Usenet Netiquette.

Mohsen Owzar

unread,
Sep 21, 2021, 4:43:02 AM9/21/21
to
Oh Thanks alot for your explanation.
I'll try to follow your steps to see if I'm able to manage this task.

Best regards
mohsen

Mohsen Owzar

unread,
Sep 21, 2021, 7:16:26 AM9/21/21
to
Janis Papanagnou schrieb am Dienstag, 21. September 2021 um 10:14:55 UTC+2:
Hi Janis,

After your suggestion I only needed two change in one part
of my code in "system() command".
No more changes were needed.
And I got the correct behavior as I wanted.

::::::::::::::::::::::::::::::::::::::::::
for (i = 1; i <= M; i++) {
print length(old_file[i]), ": ", old_file[i]
print length(new_file[i]), ": ", new_file[i]

cmd = "cp -i \"" old_file[i] "\" " new_file[i]
system(cmd)
::::::::::::::::::::::::::::::::::::::::::
Other question in this regard
If I want to ask user to copy, rename or abort (Cc, Rr or Nn)
before invoking the system() command
I used the following snippet code

::::::::::::::::::::::::::::::::::::::::::
for (i = 1; i <= M; i++) {
print length(old_file[i]), ": ", old_file[i]
print length(new_file[i]), ": ", new_file[i]

printf "Do you want to Copy [Cc] or Rename [Rr], otherwise [Nn]:"
getline answer < "-"

answer = tolower(answer)

if (answer == "c")
cmd = "cp -i \"" old_file[i] "\" " new_file[i]
else if (answer == "r")
cmd = "mv -i \"" old_file[i] "\" " new_file[i]
else
print "You ommited Copying / Renaming files!"

system(cmd)
}
::::::::::::::::::::::::::::::::::::::::::
It doesn't stop at the line containing the "getline" command and waits for user answer.
It goes through all if, else if and else part and gives out
only the message from the else part, e.g.

Do you want to Copy [Cc] or Rename [Rr], otherwise [Nn]:You ommited Copying / Renaming files!

can you also help me out of this dilema?

Regards Mohsen

Janis Papanagnou

unread,
Sep 21, 2021, 9:03:57 AM9/21/21
to
On 21.09.21 13:16, Mohsen Owzar wrote:
> [...]
>
> Other question in this regard
> If I want to ask user to copy, rename or abort (Cc, Rr or Nn)
> before invoking the system() command
> I used the following snippet code
>
> ::::::::::::::::::::::::::::::::::::::::::
> for (i = 1; i <= M; i++) {
> print length(old_file[i]), ": ", old_file[i]
> print length(new_file[i]), ": ", new_file[i]
>
> printf "Do you want to Copy [Cc] or Rename [Rr], otherwise [Nn]:"
> getline answer < "-"
>
> answer = tolower(answer)
>
> if (answer == "c")
> cmd = "cp -i \"" old_file[i] "\" " new_file[i]
> else if (answer == "r")
> cmd = "mv -i \"" old_file[i] "\" " new_file[i]
> else
> print "You ommited Copying / Renaming files!"
>
> system(cmd)
> }
> ::::::::::::::::::::::::::::::::::::::::::
> It doesn't stop at the line containing the "getline" command and waits for user answer.
> It goes through all if, else if and else part and gives out
> only the message from the else part, e.g.
>
> Do you want to Copy [Cc] or Rename [Rr], otherwise [Nn]:You ommited Copying / Renaming files!
>
> can you also help me out of this dilema?

Probably not, because there's not enough information for me to
judge, and 'getline' is a problematic statement anyway. There's
several uncertainties I'd have. - Is your code above part of the
regular standard-input processing loop or executed in the BEGIN
clause? Is "-" a valid standard representation for the standard
input channel in your version of awk? What awk are you using?
There's also no >0 test coded whether getline succedded; it may
provide useful diagnostics. Maybe getline answer < "/dev/tty"
works better? How is variable 'cmd' initialized? What are the
requirements when nothing is entered; executing an empty string,
or the 'cmd' variable's value from the previously looped file?

Janis

Ed Morton

unread,
Sep 21, 2021, 10:17:52 AM9/21/21
to
On 9/20/2021 10:33 PM, Mohsen Owzar wrote:
> Hi Guys
> I'm working with Cygwin under Windows 10
> I have written a program in AWK / Sh to copy or rename some selected files, filtered by a string given by the user, from 01 to the length of the selected files by the filter.
<snip>

Please create and post a minimal example of just the problem you're
asking for help with. Include that minimal code plus concise, testable
sample input and expected output so we can help you.

Keep in mind, though, that awk is a tool to manipulate text while shell
is a tool to manipulate files and processes with a language to sequence
calls to other tools so this approach I see in your code where shell
calls awk to call system to spawn a subshell to call another command
such as cp:

shell { awk { system { shell { cp } } } }

is usually wrong as it's extremely inefficient and usually written in a
fragile way (e.g. in your code you're leaving the arguments to cp
unquoted and so open to globbing, filename expansion, malicious command
execution, etc.), and the correct approach, if some kind of text
manipulation means it's useful to call awk for part of the job, would be
this instead where shell calls awk to parse some text and return some
strings and then shell calls cp using that awk output:

shell { awk; cp }

Regards,

Ed.

Janis Papanagnou

unread,
Sep 21, 2021, 11:54:56 AM9/21/21
to
Reinspecting code from your original post it seems you are feeding
filenames through the standard input channel (i.e. using pipes),
so if you also use the same channel for user's commands selection
input you will have a problem. Using, as proposed, "/dev/tty"
instead of "- might thus already solve the issue with your input
channels, but I haven't analyzed your code so that there might be
yet more issues (like the also mentioned uninitialized variables,
etc.).

> Janis
>

Mohsen Owzar

unread,
Sep 22, 2021, 6:29:47 AM9/22/21
to
Hi Janis

I'm answering to your last post at 15:03 because of your questions.

First, my GAWK version on Windows 10 under Cygwin:

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::
$> gawk --version
GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.0.2-p6, GNU MP 6.2.0)
Copyright © 1989, 1991-2020 Free Software Foundation.
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::

I tried to use getline with GAWK to see if 'getline a < "-"' works.
The following ist the output:

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::
$> gawk ' {printf "Type a number: "; getline a < "-"; print "You typed " a} '

Type a number: 123
You typed 123

Type a number:
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::

With /dev/tty has the same output:

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::
$> gawk ' {printf "number?: "; getline a < "/dev/tty"; print "a = " a} '

number?: 5
a = 5

number?:
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::

It seems to be that both versions work.
And your last question if this "for loop" is placed in the BEGIN part
No

I didn't change anything in this script except the last for loop
in the END part with inserted "getline". I put again this END part below:

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::
END {
Str = gensub(/ /, "_", "g", String)

for (i = 1; i <= M; i++) {
old_file[i] = Arr_B[i]

K = split(old_file[i], ARRAY, ".")
Extension = ARRAY[K]

Str_2 = substr(Str, 1, length(Str)-1)
new_file[i] = sprintf("%02d_%s.%s", i, Str_2, Extension)

printf "*** %s --> %s\n", old_file[i], new_file[i]

}
printf "\n%s Files were found!\n", M
print "--------------------------------------------"

for (i = 1; i <= M; i++) {
print length(old_file[i]), ": ", old_file[i]
print length(new_file[i]), ": ", new_file[i]

printf "Do you want to Copy [Cc] or Rename [Rr], otherwise [Nn]: "
getline answer < "/dev/tty"

answer = tolower(answer)

if (answer == "c")
cmd = "/bin/cp -i \"" old_file[i] "\" " new_file[i]
else if (answer == "r")
cmd = "/bin/mv -i \"" old_file[i] "\" " new_file[i]
else
print "You ommited Copying / Renaming files!"

system(cmd)
}
}
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::

After inserting "/dev/tty" instead of "-", it waits for the user input
and it copies when I type "c" and Enter.

In other words, it means that I can have user inputs from within AWK only with:

getline answer < "/dev/tty"

Other than this method, there is no other possibility to get user input,
not in the BEGIN part for the command arguments?

Nevertheless, I thank you and Ed very much to show me the correct way
and how can I get running my code in spite of my bad coding style.

Mohsen

Janis Papanagnou

unread,
Sep 22, 2021, 7:14:39 AM9/22/21
to
On 22.09.21 12:29, Mohsen Owzar wrote:
> After inserting "/dev/tty" instead of "-", it waits for the user input
> and it copies when I type "c" and Enter.
>
> In other words, it means that I can have user inputs from within AWK
> only with:
>
> getline answer < "/dev/tty"
>
> Other than this method, there is no other possibility to get user
> input, not in the BEGIN part for the command arguments?

Well, you can get user-input from standard input as well, but
then you can't also read the filenames from that channel (as
you seem to do in your code).

Standard input can always be redirected (to pipe, files, etc.).
With /dev/tty you are directly accessing the device; thus this
appears to be the preferred way to get interactive user input.

As I understand your post, it works with that approach. But
your question seems to say that you're uncomfortable with it?

You could of course also use other channels, e.g. passing the
file names as arguments, as in

$ awk 'BEGIN { for (i = 1; i <= ARGC; i++) {
printf "%s> ", ARGV[i]
getline cmd < "-"
printf "Do %s on %s\n", cmd, ARGV[i]
}
}' aaa bbb

aaa> mv
Do mv on aaa
bbb> cp
Do cp on bbb


Janis

Mohsen Owzar

unread,
Sep 22, 2021, 7:37:20 AM9/22/21
to
Oh man,
Yes, a bit uncomfortable, therefore your new suggestion sounds very good.
I've learned again something new.
I'll try to go deeper in this approach.

Best regards
Mohsen

Bruce Horrocks

unread,
Sep 22, 2021, 6:02:51 PM9/22/21
to
On 21/09/2021 04:33, Mohsen Owzar wrote:
> Hi Guys
> I'm working with Cygwin under Windows 10
> I have written a program in AWK / Sh to copy or rename some selected files, filtered by a string given by the user, from 01 to the length of the selected files by the filter.
> Before you say that this task can be done with another software than AWK, why I used this, OK, I know a bit only AWK and Bash scripting, and if you know another and more effective method than this one, do not hesitate and offer your suggestion.

Something like this in bash:

N=1
for X in "*$1*.png" ; do
echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
N = $((N+1))
done

Remove the 'echo' when you've finished testing!

--
Bruce Horrocks
Surrey, England

Janis Papanagnou

unread,
Sep 22, 2021, 7:06:20 PM9/22/21
to
On 23.09.21 00:02, Bruce Horrocks wrote:
> On 21/09/2021 04:33, Mohsen Owzar wrote:
>> Hi Guys
>> I'm working with Cygwin under Windows 10
>> I have written a program in AWK / Sh to copy or rename some selected
>> files, filtered by a string given by the user, from 01 to the length
>> of the selected files by the filter.
>> Before you say that this task can be done with another software than
>> AWK, why I used this, OK, I know a bit only AWK and Bash scripting,
>> and if you know another and more effective method than this one, do
>> not hesitate and offer your suggestion.
>
> Something like this in bash:
>
> N=1
> for X in "*$1*.png" ; do

Isn't it *"$1"*.png

>   echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
>   N = $((N+1))

and N=$((N+1)) ?

Bruce Horrocks

unread,
Sep 22, 2021, 9:33:03 PM9/22/21
to
On 23/09/2021 00:06, Janis Papanagnou wrote:
> On 23.09.21 00:02, Bruce Horrocks wrote:
>> On 21/09/2021 04:33, Mohsen Owzar wrote:
>>> Hi Guys
>>> I'm working with Cygwin under Windows 10
>>> I have written a program in AWK / Sh to copy or rename some selected
>>> files, filtered by a string given by the user, from 01 to the length
>>> of the selected files by the filter.
>>> Before you say that this task can be done with another software than
>>> AWK, why I used this, OK, I know a bit only AWK and Bash scripting,
>>> and if you know another and more effective method than this one, do
>>> not hesitate and offer your suggestion.
>>
>> Something like this in bash:
>>
>> N=1
>> for X in "*$1*.png" ; do
>
> Isn't it *"$1"*.png

Don't know for sure - I think they're both the same or the same under
most conditions, at least.

>
>>   echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
>>   N = $((N+1))
>
> and N=$((N+1)) ?

Yes, no spaces is correct - I so rarely code in bash I forget and want
to pretty-print it!

My quoting on
echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
is wrong as well: the double quotes around $1 need escaping. (Which of
course I noticed about 2 seconds after pressing the send button.) :-(

>
>> done
>>
>> Remove the 'echo' when you've finished testing!
>>
>


Janis Papanagnou

unread,
Sep 22, 2021, 9:55:16 PM9/22/21
to
On 23.09.21 03:33, Bruce Horrocks wrote:
> On 23/09/2021 00:06, Janis Papanagnou wrote:
>>>
>>> Something like this in bash:
>>>
>>> N=1
>>> for X in "*$1*.png" ; do
>>
>> Isn't it  *"$1"*.png
>
> Don't know for sure - I think they're both the same or the same under
> most conditions, at least.

I don't think so. The "*" are not expanded while unquoted * is.
You want them file-globbing expanded in this (and most) case(s).

Janis

Kenny McCormack

unread,
Sep 23, 2021, 2:21:40 AM9/23/21
to
In article <5c9ea403-262b-f93e...@scorecrow.com>,
Bruce Horrocks <07....@scorecrow.com> wrote:
...
>> Isn't it *"$1"*.png
>
>Don't know for sure - I think they're both the same or the same under
>most conditions, at least.

I just did a little testing, to confirm my original thoughts.

Generally, no, it's not the same. Double quoting inhibits filename
expandsion, but it works as expected in a "for i in ..." loop, because it
gets scanned twice in that one, specific case.

>>
>>> echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
>>> N = $((N+1))
>>
>> and N=$((N+1)) ?
>
>Yes, no spaces is correct - I so rarely code in bash I forget and want
>to pretty-print it!

Or you could do:

: $((N++))

and there's probably other ways, as well.

Or just: echo cp "$X" "$(printf '%02d_%s.png' $((N++)) "$1" )"

--
Politics is show business for ugly people.

Sports is politics for stupid people.

Janis Papanagnou

unread,
Sep 23, 2021, 4:52:09 AM9/23/21
to
On 23.09.21 08:21, Kenny McCormack wrote:
> In article <5c9ea403-262b-f93e...@scorecrow.com>,
> Bruce Horrocks <07....@scorecrow.com> wrote:
> ...
>>> Isn't it *"$1"*.png
>>
>> Don't know for sure - I think they're both the same or the same under
>> most conditions, at least.
>
> I just did a little testing, to confirm my original thoughts.
>
> Generally, no, it's not the same. Double quoting inhibits filename
> expandsion, but it works as expected in a "for i in ..." loop, because it
> gets scanned twice in that one, specific case.

No, that's not true. A quoted expression with * in it will keep the *
and the filename in the iteration variable X will have it. If you use
(as in the code shown) that variable again in the correct _quoted_
form "$X" then it still persists and a file and literal * will be
looked for to do the operation.

It would (unreliably) work if in the command $X would be unquoted
cp $X ...
where the * would get expanded, but that wouldn't work with filenames
with spaces. So you need to keep the * unquoted at the right place as
shown above.

>
>>>
>>>> echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
>>>> N = $((N+1))
>>>
>>> and N=$((N+1)) ?
>>
>> Yes, no spaces is correct - I so rarely code in bash I forget and want
>> to pretty-print it!
>
> Or you could do:
>
> : $((N++))

Modern shells (ksh, zsh, and now also bash as I tested) support also
the "arithmetic command" (no ':' and no expansion '$' necessary).

((N++))


Janis

Ed Morton

unread,
Sep 23, 2021, 8:38:26 AM9/23/21
to
On 9/22/2021 8:33 PM, Bruce Horrocks wrote:
> On 23/09/2021 00:06, Janis Papanagnou wrote:
>> On 23.09.21 00:02, Bruce Horrocks wrote:
>>> On 21/09/2021 04:33, Mohsen Owzar wrote:
>>>> Hi Guys
>>>> I'm working with Cygwin under Windows 10
>>>> I have written a program in AWK / Sh to copy or rename some selected
>>>> files, filtered by a string given by the user, from 01 to the length
>>>> of the selected files by the filter.
>>>> Before you say that this task can be done with another software than
>>>> AWK, why I used this, OK, I know a bit only AWK and Bash scripting,
>>>> and if you know another and more effective method than this one, do
>>>> not hesitate and offer your suggestion.
>>>
>>> Something like this in bash:
>>>
>>> N=1
>>> for X in "*$1*.png" ; do
>>
>> Isn't it  *"$1"*.png
>
> Don't know for sure - I think they're both the same or the same under
> most conditions, at least.

No, Janis is correct. The first form will set X to the one literal
string "*$1*.png" while the second will populate it iteratively with the
name of every file in your directory whose name contains whatever value
$1 is and ends in .png.

>
>>
>>>    echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
>>>    N = $((N+1))
>>
>> and  N=$((N+1))  ?
>
> Yes, no spaces is correct - I so rarely code in bash I forget and want
> to pretty-print it!
>
> My quoting on
>   echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
> is wrong as well: the double quotes around $1 need escaping.

No, you can (and should in this case) nest double quotes within command
substitution using $(...). If your intent was to get double quotes in
the output of the printf then you'd write it as

printf '%02d_"%s".png' N "$1"

not:

printf '%02d_%s.png' N \"$1\"

Regards,

Ed.

Bruce Horrocks

unread,
Sep 23, 2021, 2:50:23 PM9/23/21
to
On 23/09/2021 13:38, Ed Morton wrote:
> On 9/22/2021 8:33 PM, Bruce Horrocks wrote:
>> On 23/09/2021 00:06, Janis Papanagnou wrote:
>>> On 23.09.21 00:02, Bruce Horrocks wrote:
>>>> On 21/09/2021 04:33, Mohsen Owzar wrote:
>>>>> Hi Guys
>>>>> I'm working with Cygwin under Windows 10
>>>>> I have written a program in AWK / Sh to copy or rename some selected
>>>>> files, filtered by a string given by the user, from 01 to the length
>>>>> of the selected files by the filter.
>>>>> Before you say that this task can be done with another software than
>>>>> AWK, why I used this, OK, I know a bit only AWK and Bash scripting,
>>>>> and if you know another and more effective method than this one, do
>>>>> not hesitate and offer your suggestion.
>>>>
>>>> Something like this in bash:
>>>>
>>>> N=1
>>>> for X in "*$1*.png" ; do
>>>
>>> Isn't it  *"$1"*.png
>>
>> Don't know for sure - I think they're both the same or the same under
>> most conditions, at least.
>
> No, Janis is correct. The first form will set X to the one literal
> string "*$1*.png" while the second will populate it iteratively with the
> name of every file in your directory whose name contains whatever value
> $1 is and ends in .png.

Hm. The perils of testing on the command line instead of in a script.
You're right but it does look a bit counter-intuitive.
[snip]

>> My quoting on
>>    echo cp "$X" "$(printf '%02d_%s.png' N "$1" )"
>> is wrong as well: the double quotes around $1 need escaping.
>
> No, you can (and should in this case) nest double quotes within command
> substitution using $(...). If your intent was to get double quotes in
> the output of the printf then you'd write it as
>
>     printf '%02d_"%s".png' N "$1"
>
> not:
>
>     printf '%02d_%s.png' N \"$1\"

I meant the \"$1\" ... but quotes in the output of the $() turned out to
be simpler than I thought because "$()" does the right thing.

So I think taking it all together we get:

#! /bin/bash
N=1
for X in *"$1"* ; do
echo cp "$X" "$(printf '%02d_%s.png' "$N" "$1")"
N=$((N+1))
done

which works for a limited test set on my desktop!

Mohsen Owzar

unread,
Sep 24, 2021, 7:50:56 AM9/24/21
to
I'm somehow confused.

As I understood, I can replace my whole code
including Shell, AWK, Shell
with only this following Shell lines:

:::::::::::::::::::::::::::::::::::::::::::::::::
#! /bin/bash
# Filename: Rename_Test.sh
N=1
for X in *"$1"*
do
echo cp "$X" "$(printf '%02d_%s.png' "$N" "$1")"
N=$((N+1))
done
:::::::::::::::::::::::::::::::::::::::::::::::::

When I run this script, I get the following error message:
:::::::::::::::::::::::::::::::::::::::::::::::::
$> ll
/cygdrive/c/Users/m.owzar/Desktop/GUI_View Screenshots/00_09
insgesamt 208
drwx------+ 1 m.owzar Domain Users 0 24. Sep 13:16 .
drwx------+ 1 m.owzar Domain Users 0 24. Sep 09:47 ..
-rwx------+ 1 m.owzar Domain Users 25272 21. Sep 03:47 '2021-09-21 03_47_09-no title.png'
-rwx------+ 1 m.owzar Domain Users 24687 21. Sep 03:47 '2021-09-21 03_47_18-no title.png'
-rwx------+ 1 m.owzar Domain Users 24774 21. Sep 03:47 '2021-09-21 03_47_23-no title.png'
-rwx------+ 1 m.owzar Domain Users 27528 21. Sep 03:47 '2021-09-21 03_47_41-no title.png'
-rwx------+ 1 m.owzar Domain Users 27937 21. Sep 03:47 '2021-09-21 03_47_45-no title.png'
-rwx------+ 1 m.owzar Domain Users 27559 21. Sep 03:47 '2021-09-21 03_47_50-no title.png'
-rwx------+ 1 m.owzar Domain Users 26617 21. Sep 03:48 '2021-09-21 03_48_01-no title.png'

m.owzar@ /cygdrive/c/Users/m.owzar/Desktop/GUI_View Screenshots/00_09
$> Rename_Test.sh "no title"
/home/m.owzar/BinMo/Rename_Test.sh: Zeile 5: Syntaxfehler beim unerwarteten Wort `$'do\r''
'home/m.owzar/BinMo/Rename_Test.sh: Zeile 5: `do
:::::::::::::::::::::::::::::::::::::::::::::::::

I use under Windows 10 the Cygwin, and in there,
my script at the end of each line has CR|LF.

I think for the given string as argument, it is valid also.
Due to this issue in my AWK part of the code,
I had to subtract one character from the given argument.

Does anyone know what this error message mean? (translated in English)

Line 5: Syntax error at unexpected Word `$'do\r''

Regards
Mohsen

Janis Papanagnou

unread,
Sep 24, 2021, 8:12:09 AM9/24/21
to
It means that there's a CR that shouldn't be there. Actually saying
what you mentioned above, that it's a DOS file. But that is illegal.
If you cleanup your script (remove the CRs and keep the LFs only)
that error should vanish.

Janis

>
> Regards
> Mohsen
>

Bruce Horrocks

unread,
Sep 24, 2021, 6:01:39 PM9/24/21
to
Agreed. You could use dos2unix on the script's source file to do this
cleanup as a one-off activity.

Also the script has a bug: if the parameter text doesn't match any file
then the script tries to copy a non-existent file and fails. The
solution is to test that the file exists first before trying to copy it.

:::::::::::::
#! /bin/bash
# Filename: Rename_Test.sh
N=1
for X in *"$1"*
do
if [ -f "$X" ]
then
echo cp "$X" "$(printf '%02d_%s.png' "$N" "$1")"
N=$((N+1))
fi
done
:::::::::::::

Ed Morton

unread,
Sep 24, 2021, 6:05:59 PM9/24/21
to
That is correct. Having shell call awk to call system() to spawn a
subshell to call cp is vastly overcomplicated and inefficient for your
needs as I mentioned in my first comment in this thread.

>
> :::::::::::::::::::::::::::::::::::::::::::::::::
> #! /bin/bash
> # Filename: Rename_Test.sh
> N=1
> for X in *"$1"*
> do
> echo cp "$X" "$(printf '%02d_%s.png' "$N" "$1")"
> N=$((N+1))
> done
> :::::::::::::::::::::::::::::::::::::::::::::::::
>
> When I run this script, I get the following error message:
> :::::::::::::::::::::::::::::::::::::::::::::::::
> $> ll
> /cygdrive/c/Users/m.owzar/Desktop/GUI_View Screenshots/00_09
> insgesamt 208
> drwx------+ 1 m.owzar Domain Users 0 24. Sep 13:16 .
> drwx------+ 1 m.owzar Domain Users 0 24. Sep 09:47 ..
> -rwx------+ 1 m.owzar Domain Users 25272 21. Sep 03:47 '2021-09-21 03_47_09-no title.png'
> -rwx------+ 1 m.owzar Domain Users 24687 21. Sep 03:47 '2021-09-21 03_47_18-no title.png'
> -rwx------+ 1 m.owzar Domain Users 24774 21. Sep 03:47 '2021-09-21 03_47_23-no title.png'
> -rwx------+ 1 m.owzar Domain Users 27528 21. Sep 03:47 '2021-09-21 03_47_41-no title.png'
> -rwx------+ 1 m.owzar Domain Users 27937 21. Sep 03:47 '2021-09-21 03_47_45-no title.png'
> -rwx------+ 1 m.owzar Domain Users 27559 21. Sep 03:47 '2021-09-21 03_47_50-no title.png'
> -rwx------+ 1 m.owzar Domain Users 26617 21. Sep 03:48 '2021-09-21 03_48_01-no title.png'
>
> m.owzar@ /cygdrive/c/Users/m.owzar/Desktop/GUI_View Screenshots/00_09
> $> Rename_Test.sh "no title"
> /home/m.owzar/BinMo/Rename_Test.sh: Zeile 5: Syntaxfehler beim unerwarteten Wort `$'do\r''

As you can see at the end of the above error message your script
contains DOS line endings (CR aka `\r`s). See
https://stackoverflow.com/q/45772525/1745001 for more info and run
`dos2unix` or similar on it to clean it up.

Ed.

Ed Morton

unread,
Sep 24, 2021, 6:15:35 PM9/24/21
to
On 9/24/2021 5:01 PM, Bruce Horrocks wrote:
<snip>
> Also the script has a bug: if the parameter text doesn't match any file
> then the script tries to copy a non-existent file and fails. The
> solution is to test that the file exists first before trying to copy it.

The better solution is to add

shopt -s nullglob

to the top so if the globbing pattern doesn't match anything the result
stored in `X` is null rather than the literal pattern used and so the
loop isn't entered.

But this would all be better discussed over at comp.unix.shell...

Ed.

Kenny McCormack

unread,
Sep 24, 2021, 8:17:58 PM9/24/21
to
In article <d2dfb313-e993-b4a9...@scorecrow.com>,
Bruce Horrocks <07....@scorecrow.com> wrote:
...
>Also the script has a bug: if the parameter text doesn't match any file
>then the script tries to copy a non-existent file and fails. The
>solution is to test that the file exists first before trying to copy it.

Another way is to use bash's nullglob option.

I've always thought the way sh type shells handle "no file" globbing was
wrong (*). That nullglob should have been the default from the get-go.

Of course, the default can't be changed now, but going forward, most bash
code should set nullglob.

(*) Note that csh changed this - which was (and is) a Good Thing, but it is
different from either classic sh or bash-with-nullglob.

P.S. This thread needs to be moved to the shell group. It has nothing to
do with AWK.

--
People sleep peaceably in their beds at night only because rough
men stand ready to do violence on their behalf.

George Orwell

Mohsen Owzar

unread,
Sep 25, 2021, 12:48:42 PM9/25/21
to
Thanks alot to all for their contributions.

@ED, I know now there is a better way to reach my goal.
But nevertheless, I gathered some new perspectives with my bad concept.

Due to what is said above, I have to convert (dos2unix) all my scripts, written with notepad++
under Windows, to be able to run under Cygwin.

Regards
Mohsen

J Naman

unread,
Sep 26, 2021, 12:19:39 PM9/26/21
to
On Saturday, 25 September 2021 at 12:48:42 UTC-4, Mohsen Owzar wrote:

> Due to what is said above, I have to convert (dos2unix) all my scripts, written with notepad++
> under Windows, to be able to run under Cygwin.
> Regards
> Mohsen

Mohsen, Two points. A) Notepad++ offers macro recording to speed up converting and has a command to save
LF=unix, even if the file came in as dos/win and vice versa.
B) dos2unix: I tried Gawk under Cygwin and switched to Win 10 WSL2 running Ubuntu. Win batch files can run unix commands
using unix or windows file paths, e.g. in file from win10, out file to unix. Bash scripts can call Win 10 commands
& programs and access windows and/or unix files. Or pure unix or pure win. I like the flexibility to use what I need.
File i/o BETWEEN unix and win 10 is a little slower, maybe 10%. Gawk unix is 64-bit and runs some .awks 50% faster
for cpu intensive, 15% slower for just huge memory, not much cpu, and I suspect 20%, maybe 50%, slower for lots of
file i/o. So, Gawk under WSL is about the same as under Cygwin. I like both for 64-bit memory for large arrays and
WSL for file system "interoperability"
-just FYIO, john

Mohsen Owzar

unread,
Sep 26, 2021, 12:56:50 PM9/26/21
to
Thanks for your information
Regards
Mohsen
Reply all
Reply to author
Forward
0 new messages