find my_script in multiple directories and execute it

[Harry]

I have my_script.sh in multiple directories.
How could I find it, then change directory there, and execute it, one by one?

./a/b c/d/my_script.sh
./e/f g/h/my_script.sh
...

find . -name 'my_script.sh' -exec (cd dirname of {} ; sh my_script.sh ) \;

I tried just echoing the dirname with no success ...
$ find . -name update_files_b0783.sh -exec echo ${{}%/*} \;
-bash: ${{}%/*}: bad substitution

Any help appreciated.
TIA

[Josef Möllers]

My pragmatic solution: Whip up a script:
#! /bin/bash
dir="${1%/*}"
script="${1##*/}"
cd "$dir" && sh "$script"

Then use the script(name) as the argument to "-exec"

Josef

[Lew Pitcher]

On Thu, 09 Feb 2023 08:49:58 -0800, Harry wrote:

> I have my_script.sh in multiple directories.
> How could I find it, then change directory there, and execute it, one by
> one?
>
> ./a/b c/d/my_script.sh ./e/f g/h/my_script.sh ...
>
> find . -name 'my_script.sh' -exec (cd dirname of {} ; sh my_script.sh )
> \;

How about
find . -name 'my_script.sh' -print | while read SP ; do "$SP" ; done

--
Lew Pitcher
"In Skills We Trust"

[Harry]

On Thursday, February 9, 2023 at 9:13:03 AM UTC-8, Josef Möllers wrote:

> My pragmatic solution: Whip up a script:
> #! /bin/bash
> dir="${1%/*}"
> script="${1##*/}"
> cd "$dir" && sh "$script"
>
> Then use the script(name) as the argument to "-exec"

Josef,
It only went through one of the many directories.

[Harry]

On Thursday, February 9, 2023 at 9:30:51 AM UTC-8, Lew Pitcher wrote:

> How about
> find . -name 'my_script.sh' -print | while read SP ; do "$SP" ; done

Lew
I haven't tried your suggestion yet ...
It has to "cd <the destination directory>" first, because my_script.sh would try to copy files from some_directory to those already existing in <the destination directory>.

[Janis Papanagnou]

What if you expand the loop in shell...?

find -name "my_script.sh" |
while read f ; do ( cd "${f%/*}" && sh "${f##*/}" ) ; done

But note that you need adjustments for pathological filenames.

Also it might be worth a thought in case it's possible that
malicious files in the directory hierarchy might be executed.

Janis

[Lew Pitcher]

That's fair. My suggestion doesn't do that.

[Harry]

On Thursday, February 9, 2023 at 10:21:57 AM UTC-8, Janis Papanagnou wrote:

> What if you expand the loop in shell...?
>
> find -name "my_script.sh" |
> while read f ; do ( cd "${f%/*}" && sh "${f##*/}" ) ; done
>
> But note that you need adjustments for pathological filenames.
>
> Also it might be worth a thought in case it's possible that
> malicious files in the directory hierarchy might be executed.
>
> Janis

Janis

It works.
Thanks a lot for your help.

H.

[Josef Moellers]

Although you seem to have found a solution:
find . -name 'my_script.sh' -exec ./josefs_script {} \;
Is that how you called it?

$ find . -name my_script.sh -exec ./josef_script {} \;
my_script.sh called in /tmp/test/a/b/c/d
my_script.sh called in /tmp/test/e/f/g/h

Josef

[Helmut Waitzmann]

Harry <harryoo...@hotmail.com>:

> I have my_script.sh in multiple directories.
> How could I find it, then change directory there, and execute
> it, one by one?
>
> ./a/b c/d/my_script.sh
> ./e/f g/h/my_script.sh
> ...
>
> find . -name 'my_script.sh' -exec (cd dirname of {} ; sh my_script.sh ) \;
>

find . -name 'my_script.sh' -exec sh -c -- \
'name="${1##*/}" && dir="${1%"$name"} &&
CDPATH= cd -- "${dir:=./}" && sh ./"$name"' sh \{\} \;

[Kenny McCormack]

In article <837cwpb...@helmutwaitzmann.news.arcor.de>,

That's totally unsuitable for the OP. Nobody is going to be able to use
that unless/until they understand it, and, to be frank, *I* don't
understand it (if I took enough time, I'm sure I could work it out, but why
bother?) and if *I* don't understand it, no way is either this OP or any
general OP going to get it.

Bottom line: You really can't post something like that w/o a detailed
explanation of what it is and why you wrote it that way.

--
Alice was something of a handful to her father, Theodore Roosevelt. He was
once asked by a visiting dignitary about parenting his spitfire of a daughter
and he replied, "I can be President of the United States, or I can control
Alice. I cannot possibly do both."

[Harry]

Josef

My apology, I must have done it wrong the first time.
After repeating the test, it works.
Yes, that's the way I called your script, not ./josef_script, but /some/path/josef_script; but it shouldn't matter.

H.

[Oğuz]

On 2/10/23 11:50 PM, Helmut Waitzmann wrote:
>   find . -name 'my_script.sh' -exec sh -c -- \
>     'name="${1##*/}" && dir="${1%"$name"} &&
>      CDPATH= cd -- "${dir:=./}" && sh ./"$name"' sh \{\} \;

That's

find . -name my_script.sh -execdir sh {} \;

with GNU find. Is -execdir not standard yet?

[Keith Thompson]

It's not specified by POSIX, but the GNU find manual says it "was
introduced by the BSD family of operating systems". (It doesn't
say when.)

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */

[Christian Weisgerber]

On 2023-02-11, Keith Thompson <Keith.S.T...@gmail.com> wrote:

>> with GNU find. Is -execdir not standard yet?
>
> It's not specified by POSIX, but the GNU find manual says it "was
> introduced by the BSD family of operating systems". (It doesn't
> say when.)

It wasn't in 4.4BSD. Todd Miller added it on OpenBSD in 1996 and
from there is was copied to FreeBSD and eventually NetBSD.

--
Christian "naddy" Weisgerber na...@mips.inka.de

[Helmut Waitzmann]

gaz...@shell.xmission.com (Kenny McCormack):

> In article <837cwpb...@helmutwaitzmann.news.arcor.de>,
> Helmut Waitzmann <oe.th...@xoxy.net> wrote:
>>
>> find . -name 'my_script.sh' -exec sh -c -- \
>> 'name="${1##*/}" && dir="${1%"$name"} &&
>> CDPATH= cd -- "${dir:=./}" && sh ./"$name"' sh \{\} \;
>
> That's totally unsuitable for the OP.
>

It will solve the OP's problem.  Your post will not.


Josef Möllers solution presented a shell script, which needs be
stored somewhere in a file of its own, that is, the command line
asked by Harry won't be self‐contained.  This might or might not
be a problem.

Janis' solution (as he already stated) won't work with
pathological filenames, but he doesn't show how the adjustments
to correct this would be.

> Nobody is going to be able to use that unless/until they
> understand it, and, to be frank, *I* don't understand it (if I
> took enough time, I'm sure I could work it out, but why bother?)

You are free to refrain from working it out.

> and if *I* don't understand it, no way is either this OP or any
> general OP going to get it.

Do you conclude from what is incomprehensible for you will be
incomprehensible for everybody else?


Do you understand, why Janis' solution fails at "pathological"
filenames?  Do you know how to solve that problem?  Do you know
that on the system API level there are no such pathological
filenames, that is, the pathological effect is introduced alone
by a solution which neglects the existence of such pathological
filenames?

> Bottom line: You really can't post something like that w/o a
> detailed explanation of what it is and why you wrote it that
> way.

I don't know what parts of that solution need to be explained, as
that depends of the knowledge of the reader.  Feel free to ask.

In the bottom line, my solution works like Josef's, but avoids
having a separate shell script:  Instead of using a separate
script file it makes use of the

sh -c -- command_line sh parameters...

invocation mechanism of the shell.  To understand what that is,
consult the explanation of the "-c" invocation option of the
shell in the shell's manual page.  It boils down to invoke the
shell with the option "-c" and a shell script's content supplied
as the first non‐option operand: "command_line".


Compare the contents of the command_line parameter with the
contents of Josef's shell script:  The command_line parameter
resembles the following shell script:

name="${1##*/}" &&
dir="${1%"$name"}" &&
CDPATH= cd -- "${dir:=./}" &&
sh ./"$name"

Compare with Josef's shell script:  Do you see the relationship?

dir="${1%/*}"
script="${1##*/}"
cd "$dir" && sh "$script"

Now, for the differences between the two:  Answer the following
questions:

What will the contents of the "dir" variable be, when Harry
happens to invoke his "find" command with the root directory ("/")
as the starting point?  To what directory will the "cd" command
change when the "find" command finds the "/my_script.sh" file?

What will the "cd" command do, if Harry happens to invoke his
"find" command with a directory the name of starts with a "-"?

What will the "cd" command do, if the CDPATH environment variable
happens to be defined?

For the answers, look into the description of the ${var%pattern}
variable expansion and into the description of the "cd" shell
builtin command in the shell manual page.

You are free of course to present a solution which is easier to
understand.

[Janis Papanagnou]

On 12.02.2023 09:15, Helmut Waitzmann wrote:
> gaz...@shell.xmission.com (Kenny McCormack):
>> [...]

Since my post was referenced I just want to reply on this:

> Janis' solution (as he already stated) won't work with pathological
> filenames, but he doesn't show how the adjustments to correct this
> would be.

My observation is, indeed, that for a specific task a 99% solution
usually helps the asking poster in most cases to 100% fulfill his
task.

Another observation is that when we need a tool for some task that
shall be used over years by many users reliably then any less than
100% coverage may appear to be insufficient, either because at some
time we need to add functionality to be usable for "pathological"
cases, or it may be even a security issue associated with it.

The question is (as far as I am concerned) what effort to spend for
a concrete question. In case of the scripts I use I typically start
with a line or few lines of code that "does the job"; once the state
of usage changes and I want a reliable tool I start adding comments
to the code, use getopts, sometimes signal handling, rework the user
interface to be more flexible or alleviate usage, and whatnot before
that tool goes into my local 'bin' directory. A few operational lines
of code evolves towards a bulletproof, maintainable software tool of
sometimes even many pages of code. (If you think I'm exaggerating,
think e.g. about ^C aborted processes and inconsistencies you leave
e.g. by only partly executed scripts on parts of the directory tree.)

Some improvements are more trivial than others (like using 'cd --'
instead of 'cd') others may be less trivial (line '\n' in filenames).
All sorts of changes for improvements may be added [on demand].

Here I dare to draw the relation to Kenny's quite terse (and probably
rough appearing) comment; simple solutions are easier to grasp than
complex ones, and these 'find' specific syntactical details (together
with all the quoting and escaping orgy) may be considered being too
opaque - personally I don't mind (although I would avoid such in my
own scripts if possible).

To come to an end; if the tools work for the application domain (or
according to specifications, that we rarely see here in Usenet) then
there's nothing to "correct" in a solution. If requirements change
then you have to adjust or change the solution.

In our case I would be more concerned about "script-injections" that
I mentioned in my post. But here as well; it may not be an issue in
the OPs application domain. So a hint suffices and changes can later
be made if the application domain requires it (now or at some time).

Even though the OP already stated "It works." - You're welcome, Harry,
BTW -, I also appreciate to see the completely 'find' based solution
that you additionally provided afterwards. It's certainly good we can
choose (or learn from).

Janis

[Kenny McCormack]

In article <838rh39...@helmutwaitzmann.news.arcor.de>,

Helmut Waitzmann <oe.th...@xoxy.net> wrote:
> gaz...@shell.xmission.com (Kenny McCormack):
>> In article <837cwpb...@helmutwaitzmann.news.arcor.de>,
>> Helmut Waitzmann <oe.th...@xoxy.net> wrote:
>>>
>>> find . -name 'my_script.sh' -exec sh -c -- \
>>> 'name="${1##*/}" && dir="${1%"$name"} &&
>>> CDPATH= cd -- "${dir:=./}" && sh ./"$name"' sh \{\} \;
>>
>> That's totally unsuitable for the OP.
>>
>
> It will solve the OP's problem.

No, it won't. Not really.

>Your post will not.

I did not post any code, so this comment of yours is noise.

--
The difference between communism and capitalism?
In capitalism, man exploits man. In communism, it's the other way around.

- Daniel Bell, The End of Ideology (1960) -

[Helmut Waitzmann]

Helmut Waitzmann <nn.th...@xoxy.net>:

> find . -name 'my_script.sh' -exec sh -c -- \
> 'name="${1##*/}" && dir="${1%"$name"} &&

^
There is a quotation mark missing:  The correct line should be

[hongy...@gmail.com]

Will this subshell-based technique suffer performance bottlenecks under heavy usage?

Regards,
Zhao

[hongy...@gmail.com]

Another cumbersome is: This approach requires a variety of nesting of single and double quotes and requires great care in writing.

> Regards,
> Zhao

[Ben Bacarisse]

I would probably do something like this:

dir="$PWD"
find . -name my_script.sh | while read script; do
cd "$(dirname "$script")"
./"$(basename "$script")"
cd "$dir"
done

but for some uses this could be simplified to something like

find "$PWD" -name my_script.sh | while read script; do
cd "$(dirname "$script")"
"$script"
done

which relies on doing the find with an absolute path.

--
Ben.

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

According to the OP's requirements, for each found
"my_script.sh", the working directory shall be changed to the
directory where the found script lies, then the script executed.

As POSIX‐"find" hasn't got a predicate that changes the working
directory[1], one has to use a program that first changes its
working directory and then invokes the "execve" system call to
start a shell interpreting the script.  Of course one may write
such a program (in a manner like the programs "nohup" or
"nice").  But this goal is easier achieved by using a shell with
the option "-c" like above.

[1] GNU "find" does have such a predicate, though: "-execdir". 
Oğuz already noted that.


A variant would be

find . -name 'my_script.sh' -exec sh -c -- '
for pathname
do
(
name="${pathname##*/}" &&
dir="${pathname%"$name"}" &&

CDPATH= cd -- "${dir:=./}" &&

exec sh ./"$name"
)
done' sh \{\} +

which will cause "find" to invoke the directory changing shell
("sh -c -- ...") on multiple shell scripts found, thus invoke
fewer "sh -c -- ..." shells.

But this advantage implicates, that the shell has to fork
itself – see the parentheses in the "for" loop – before it
changes the working directory and "exec"s the "sh" interpreting
the script.

=> There are fewer "exec"s as before but not fewer "fork"s. 
Also, there is one process more than before running, which might
be a disadvantage when there is few free memory.

Also, there might be another problem, when "find" gathers
multiple found scripts.  If one invocation ot the script removes
or creates any other script that shall resp. should be invoked
later (Harry didn't write anything about that), then the shell
running the "for" loop will try to invoke a non‐existing shell
script resp. omit to invoke a just created shell script.

I won't bother using the "for" loop variant.


(Quoting again:)

> Will this subshell-based technique suffer performance
> bottlenecks under heavy usage?

Maybe.  But if one does

find . -name my_script.sh -print |

while IFS= read -r pathname
do
(
name="${pathname##*/}" &&
dir="${pathname%"$name"}" &&

CDPATH= cd -- "${dir:=./}" &&

exec sh ./"$name"
)
done

one will have almost the same implications as with the "for" loop
above plus the problems with pathological directory names.

[Helmut Waitzmann]

"hongy...@gmail.com" <hongy...@gmail.com>:

> On Monday, February 13, 2023 at 7:28:54 AM UTC+8,
> hongy...@gmail.com wrote:
>> On Saturday, February 11, 2023 at 5:00:27 AM UTC+8,
>> Helmut Waitzmann wrote:
>>> Harry <harryoo...@hotmail.com>:
>>>> I have my_script.sh in multiple directories.
>>>>
>>>> How could I find it, then change directory there, and execute
>>>> it, one by one?
>>>>

>>>> find . -name 'my_script.sh' -exec (cd dirname of {} ; sh my_script.sh ) \;
>>>>
>>> find . -name 'my_script.sh' -exec sh -c -- \
>>> 'name="${1##*/}" && dir="${1%"$name"} &&
>>> CDPATH= cd -- "${dir:=./}" && sh ./"$name"' sh \{\} \;

[…]

> Another cumbersome is: This approach requires a variety of
> nesting of single and double quotes and requires great care in
> writing.

Yes.  The shell language always requires great care in writing. 
That's true.  A part of that language are the quoting rules.  If
one writes a shell command line which invokes a shell giving it a
shell command line as a parameter (which is the case with the
"-c" shell invocation option), then one has to put a command
line, written as one word, into a command line, which may cause
nested quoting and requires great care in writing.

An alternative approach would be, what Josef Möllers did: store
that "inner" command line into a script rather than supply it
quoted with the shell invocation option "-c".  This saves you
from nested quoting but you need to store the script in a file of
its own.

[Helmut Waitzmann]

Helmut Waitzmann <nn.th...@xoxy.net>:

> "hongy...@gmail.com" <hongy...@gmail.com>:
>> On Monday, February 13, 2023 at 7:28:54 AM UTC+8,
>> hongy...@gmail.com wrote:
>>> On Saturday, February 11, 2023 at 5:00:27 AM UTC+8,
>>> Helmut Waitzmann wrote:
>>>> Harry <harryoo...@hotmail.com>:
>>>>> I have my_script.sh in multiple directories.
>>>>>
>>>>> How could I find it, then change directory there, and execute
>>>>> it, one by one?
>>>>>

>>>> find . -name 'my_script.sh' -exec sh -c -- \
>>>> 'name="${1##*/}" && dir="${1%"$name"} &&
>>>> CDPATH= cd -- "${dir:=./}" && sh ./"$name"' sh \{\} \;
>
> […]
>

> The shell language always requires great care in writing. 
> That's true.  A part of that language are the quoting rules.  If
> one writes a shell command line which invokes a shell giving it a
> shell command line as a parameter (which is the case with the
> "-c" shell invocation option), then one has to put a command
> line, written as one word, into a command line, which may cause
> nested quoting and requires great care in writing.

In order to make the command line be one word, one needs to quote
that command line.  This can be achieved as follows:

First, write the command line in an unquoted, "normal" form, like
one would do it when writing a shell script in a file (like Josef
Möllers did).

Then, using the find‐and‐replace capabilities of the text editor,
replace every apostrophe ("'") in the command line by the
sequence of the four characters apostrophe, backslash,
apostrophe, apostrophe ("'\''").

Finally prepend and append an apostrophe to the command line.


In the example above, there are no apostrophes in the unquoted
command line

name="${1##*/}" && dir="${1%"$name"}" &&
CDPATH= cd -- "${dir:=./}" && sh ./"$name"

so there is nothing to be replaced.  Just prepend and append an
apostrophe:

'name="${1##*/}" && dir="${1%"$name"}" &&
CDPATH= cd -- "${dir:=./}" && sh ./"$name"'

That's it.