find/xargs substiution

[Ed Morton]

If I want to remove all files starting with "foo" in my directory I can do:

find . -name foo.* | xargs rm '{}'

or use -exec. What if I want to rename them to bar.* instead of removing them?

I would normally write a loop:

find . -name foo.* |
while IFS= read -r file; do mv "$file" "bar${file#foo}"; done

but is there a way to do that with find -exec or with xargs, something like this
pseudo-code which doesn't work:

find . -name foo.* | xargs mv '{}' "bar${{}#foo}"

The question isn't about how to rename files, it's about how to do substitution
on file names within find -exec or xargs.

Regards,

Ed.

[pk]

The obvious way with -exec is

-exec sh -c 'while [ $# -gt 0 ]; do name=$1; mv "$name" "$name".whatever;
shift; done' sh {} +

which can be easily adapt to work with xargs. Probably a better way would
be to use a tool dedicated to mass renaming of files.

[Ed Morton]

On 2/14/2013 6:22 AM, pk wrote:
> On Thu, 14 Feb 2013 06:00:21 -0600, Ed Morton <morto...@gmail.com> wrote:
>
>> If I want to remove all files starting with "foo" in my directory I can
>> do:
>>
>> find . -name foo.* | xargs rm '{}'
>>
>> or use -exec. What if I want to rename them to bar.* instead of removing
>> them?
>>
>> I would normally write a loop:
>>
>> find . -name foo.* |
>> while IFS= read -r file; do mv "$file" "bar${file#foo}"; done
>>
>> but is there a way to do that with find -exec or with xargs, something
>> like this pseudo-code which doesn't work:
>>
>> find . -name foo.* | xargs mv '{}' "bar${{}#foo}"
>>
>> The question isn't about how to rename files, it's about how to do
>> substitution on file names within find -exec or xargs.
>
> The obvious way with -exec is
>
> -exec sh -c 'while [ $# -gt 0 ]; do name=$1; mv "$name" "$name".whatever;
> shift; done' sh {} +

Sorry, I should have been clearer - I know how to write loops, I'm trying to
learn if/how I can use substitution on the {} current-file-name construct in
find -exec or xargs.

> which can be easily adapt to work with xargs. Probably a better way would
> be to use a tool dedicated to mass renaming of files.
>

As I mentioned above, the question isn't about how to rename files, it's about

how to do substitution on file names within find -exec or xargs.

Thanks,

Ed.

[Octothorpe]

On Thu, 14 Feb 2013 06:00:21 -0600, Ed Morton wrote:

> If I want to remove all files starting with "foo" in my directory I can
> do:
>
> find . -name foo.* | xargs rm '{}'

That doesn't work this will

find . -type f -name foo* -exec unlink {} \;

[Janis Papanagnou]

I don't think that find(1) can do string or file name substitutions
per se; that's the task of other tools. In case of pk's poposal a
shell had been used; I don't think that anything is wrong with that.
Are you expecting that find(1) supports a pattern/substitution option?
Don't count on that. The {} is only a placeholder with a glyph that
resembles the shell variable braces, but that's it AFAICT.

Janis

>
> Thanks,
>
> Ed.

[Janis Papanagnou]

Why not just

find . -name foo.* | xargs rm

Janis

[Janis Papanagnou]

BTW, the shell pattern should be quoted

find . -name "foo.*" | xargs rm

in case there's matching files in the current directory.

Janis

[Ed Morton]

Is one of them xargs?

In case of pk's poposal a
> shell had been used; I don't think that anything is wrong with that.

I agree, I'm just asking about a possible alternative.

> Are you expecting that find(1) supports a pattern/substitution option?

I'm wondering if there is a way to do substitution on the {} file name
representative construct used by find -exec or xargs.

> Don't count on that.

I'm not, I'm just asking.

The {} is only a placeholder with a glyph that
> resembles the shell variable braces, but that's it AFAICT.

That's what it looks like to me too but I never use find -exec or xargs (I
always use a loop) so I'm asking if others with more experience can provide a
definitive answer.

Thanks,

Ed.
>
> Janis
>
>>
>> Thanks,
>>
>> Ed.
>

[Ed Morton]

On 2/14/2013 6:56 AM, Janis Papanagnou wrote:
> Am 14.02.2013 13:54, schrieb Janis Papanagnou:
>> Am 14.02.2013 13:33, schrieb Octothorpe:
>>> On Thu, 14 Feb 2013 06:00:21 -0600, Ed Morton wrote:
>>>
>>>> If I want to remove all files starting with "foo" in my directory I can
>>>> do:
>>>>
>>>> find . -name foo.* | xargs rm '{}'
>>>
>>> That doesn't work this will
>>>
>>> find . -type f -name foo* -exec unlink {} \;

I don't actually care about that. As I said in my original post, this question
is not about how to remove files it's about if/how I can use substitution on {}.

>>
>> Why not just
>>
>> find . -name foo.* | xargs rm

I was trying to highlight the fact that it's the {} construct I'm interested in.

> BTW, the shell pattern should be quoted
>
> find . -name "foo.*" | xargs rm
>
> in case there's matching files in the current directory.

Yes, I rushed through that part to get to the question.

Ed.

[Janis Papanagnou]

Am 14.02.2013 14:11, schrieb Ed Morton:
> On 2/14/2013 6:53 AM, Janis Papanagnou wrote:
>> Am 14.02.2013 13:28, schrieb Ed Morton:
>>> On 2/14/2013 6:22 AM, pk wrote:
>>>> On Thu, 14 Feb 2013 06:00:21 -0600, Ed Morton <morto...@gmail.com>
>>>> wrote:
>>>>
>>>>> If I want to remove all files starting with "foo" in my directory I
>>>>> can
>>>>> do:

[...]

>>>>> The question isn't about how to rename files, it's about how to do
>>>>> substitution on file names within find -exec or xargs.
>>>>
>>>> The obvious way with -exec is
>>>>
>>>> -exec sh -c 'while [ $# -gt 0 ]; do name=$1; mv "$name"
>>>> "$name".whatever;
>>>> shift; done' sh {} +
>>>
>>> Sorry, I should have been clearer - I know how to write loops, I'm
>>> trying to learn if/how I can use substitution on the {}
>>> current-file-name construct in find -exec or xargs.
>>>
>>>> which can be easily adapt to work with xargs. Probably a better way
>>>> would
>>>> be to use a tool dedicated to mass renaming of files.
>>>>
>>>
>>> As I mentioned above, the question isn't about how to rename files, it's
>>> about how to do substitution on file names within find -exec or xargs.
>>
>> I don't think that find(1) can do string or file name substitutions
>> per se; that's the task of other tools.
>
> Is one of them xargs?

Not that I know of; xargs is effectively just composing arguments
from lists. It's also no text processor.

>
> In case of pk's poposal a
>> shell had been used; I don't think that anything is wrong with that.
>
> I agree, I'm just asking about a possible alternative.

An external rename tool that had been proposed would probably result
in the cleanest solution. The mmv(1) command *may* be an appropriate
candidate (haven't used it myself, yet), given its brief description.

>
>> Are you expecting that find(1) supports a pattern/substitution option?
>
> I'm wondering if there is a way to do substitution on the {} file name
> representative construct used by find -exec or xargs.
>
>> Don't count on that.
>
> I'm not, I'm just asking.
>
> The {} is only a placeholder with a glyph that
>> resembles the shell variable braces, but that's it AFAICT.
>
> That's what it looks like to me too but I never use find -exec or xargs
> (I always use a loop) so I'm asking if others with more experience can
> provide a definitive answer.

"yes" (with accompanied example) would be a definite answer;
"no" (naturally without example) cannot have a definite answer,
other than inspecting the man pages. The man pages seem to not
have information on substitutions available with find(1) or
xargs(1).

Janis

[Janis Papanagnou]

The target of this response was "Octothorpe" and the specific
contents of his posting. I felt a reply to what he says very
necessary.

I addressed your question in another posting.

>
> Ed.
>

[Barry Margolin]

In article <kfijj9$i5c$1...@dont-email.me>,

All they can do is replace {} with the found filename. You can't use the
shell's variable interpolation operators on it, since it's not a shell
variable. So if you want to do that, you need to impose an explicit
shell, or pass it as an argument to a command/script that does what you
want.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Octothorpe]

Actually the shell will expand foo.* before find gets the argument.

[Octothorpe]

That won't find foobar or any other file that starts with foo. He gave
an example of finding files that begin with "foo" but then put "foo." in
the example, they are not the same.

Why bring xargs into this when it is not needed?

find . -type f -name 'foo*' -exec rm {} \;

The -name foo* should have been -name 'foo*', needs quoting.

[Janis Papanagnou]

And exactly that is the problem with unquoted patterns!

First problem:
./foo.bar
./dir/foo.baz
The shell will expand foo.bar in the call, and find(1) will find
the file ./foo.bar but not ./dir/foo.baz despite the second does
also match the foo.* pattern.

Second problem:
./foo.bar
./foo.baz
The shell will expand both files in the find(1) call and find will
report a calling syntax error because.

To avoid that you have to quote "foo.*"; then the shell will not
expand the pattern for the find(1) call, but find(1) will expand
it in all visited directories, thus finding all files matching the
given pattern.

Janis

[Janis Papanagnou]

Am 14.02.2013 17:15, schrieb Octothorpe:
> On Thu, 14 Feb 2013 13:54:59 +0100, Janis Papanagnou wrote:
>
>> Am 14.02.2013 13:33, schrieb Octothorpe:
>>> On Thu, 14 Feb 2013 06:00:21 -0600, Ed Morton wrote:
>>>
>>>> If I want to remove all files starting with "foo" in my directory I
>>>> can do:
>>>>
>>>> find . -name foo.* | xargs rm '{}'
>>>
>>> That doesn't work this will
>>>
>>> find . -type f -name foo* -exec unlink {} \;
>>>
>>>
>> Why not just
>>
>> find . -name foo.* | xargs rm
>>
>>
>> Janis
>
> That won't find foobar or any other file that starts with foo. He gave
> an example of finding files that begin with "foo" but then put "foo." in
> the example, they are not the same.

Ed knows very well what he wants to find and will certainly use
the correct pattern.

The question was why use the {} at all if you don't need it.

>
> Why bring xargs into this when it is not needed?

Because it's more efficient than a find(1) call that is terminated
by \; instead of + .

>
> find . -type f -name 'foo*' -exec rm {} \;

BTW, you won't find symbolic links if you use -type f.

Janis

[Octothorpe]

On Thu, 14 Feb 2013 18:21:36 +0100, Janis Papanagnou wrote:

> Am 14.02.2013 17:15, schrieb Octothorpe:
>> On Thu, 14 Feb 2013 13:54:59 +0100, Janis Papanagnou wrote:
>>
>>> Am 14.02.2013 13:33, schrieb Octothorpe:
>>>> On Thu, 14 Feb 2013 06:00:21 -0600, Ed Morton wrote:
>>>>
>>>>> If I want to remove all files starting with "foo" in my directory I
>>>>> can do:
>>>>>
>>>>> find . -name foo.* | xargs rm '{}'
>>>>
>>>> That doesn't work this will
>>>>
>>>> find . -type f -name foo* -exec unlink {} \;
>>>>
>>>>
>>> Why not just
>>>
>>> find . -name foo.* | xargs rm
>>>
>>>
>>> Janis
>>
>> That won't find foobar or any other file that starts with foo. He gave
>> an example of finding files that begin with "foo" but then put "foo."
>> in the example, they are not the same.
>
> Ed knows very well what he wants to find and will certainly use the
> correct pattern.
>
> The question was why use the {} at all if you don't need it.
>
>
>> Why bring xargs into this when it is not needed?
>
> Because it's more efficient than a find(1) call that is terminated by \;
> instead of + .
>

YIKES!!!!!

I did not know that my eight core AMD FX 32G ram with ssd drives had a
efficency problem!

Good golly Miss Molly, I lost a few nano seconds.

>
>> find . -type f -name 'foo*' -exec rm {} \;
>
> BTW, you won't find symbolic links if you use -type f.

I wasn't looking for them. I'm looking for files only.

>
> Janis
>

[Geoff Clare]

Because that will misbehave if any filenames contain whitespace,
backslash, single-quote or double-quote characters, whereas
find -exec does not have that problem.

--
Geoff Clare <net...@gclare.org.uk>

[Kenny McCormack]

In article <php1v9-...@leafnode-msgid.gclare.org.uk>,
Geoff Clare <net...@gclare.org.uk> wrote:
...

>> Why not just
>>
>> find . -name foo.* | xargs rm
>
>Because that will misbehave if any filenames contain whitespace,
>backslash, single-quote or double-quote characters, whereas
>find -exec does not have that problem.

That can be corrected using the "-0" option in both "find" and "xargs".
(Actually, it is "-print0" in "find", but you get the idea...)

And if your versions of "find" and "xargs" don't have these options, then you
need to upgrade.

Also, the reason to use "xargs", rather than "-exec", is to avoid spawning
an "rm" process for each file. You know, efficency and all that. Further,
since removing files is a common operation with "find" (although not, it
should be pointed out, the topic of this thread), it has always seemed to me
that "find" should have it as a built-in. And, indeed, at least one
DOS/Windows port of "find" that I've used, does have a "-rm" built-in.

--
Nov 4, 2008 - the day when everything went
from being Clinton's fault to being Obama's fault.

[pk]

On Fri, 15 Feb 2013 13:59:41 +0000 (UTC), gaz...@shell.xmission.com (Kenny
McCormack) wrote:

> In article <php1v9-...@leafnode-msgid.gclare.org.uk>,
> Geoff Clare <net...@gclare.org.uk> wrote:
> ...
> >> Why not just
> >>
> >> find . -name foo.* | xargs rm
> >
> >Because that will misbehave if any filenames contain whitespace,
> >backslash, single-quote or double-quote characters, whereas
> >find -exec does not have that problem.
>
> That can be corrected using the "-0" option in both "find" and "xargs".
> (Actually, it is "-print0" in "find", but you get the idea...)
>
> And if your versions of "find" and "xargs" don't have these options, then
> you need to upgrade.
>
> Also, the reason to use "xargs", rather than "-exec", is to avoid spawning
> an "rm" process for each file. You know, efficency and all that.

You talk about the need to upgrade, yet don't seem to know about the "+"
termination to -exec, which has surely been around longer than -print0, -0
and all that.

[Casper H.S. Dik]

gaz...@shell.xmission.com (Kenny McCormack) writes:

>That can be corrected using the "-0" option in both "find" and "xargs".
>(Actually, it is "-print0" in "find", but you get the idea...)

>And if your versions of "find" and "xargs" don't have these options, then you
>need to upgrade.

>Also, the reason to use "xargs", rather than "-exec", is to avoid spawning
>an "rm" process for each file. You know, efficency and all that. Further,
>since removing files is a common operation with "find" (although not, it
>should be pointed out, the topic of this thread), it has always seemed to me
>that "find" should have it as a built-in. And, indeed, at least one
>DOS/Windows port of "find" that I've used, does have a "-rm" built-in.

Or just use exec and "+". (Kinda like a build in xargs)

Existed in SVr4 from early on. (But not documented)

Casper

[Janis Papanagnou]

Sure. As already noted upthread, the point here was why
using the {} in above code, as it's unnecessary.

Most issues seem to have been covered anyway, meanwhile.
* Quoting the pattern "foo.*"
* using -print0 and -0 to handle the whitespace-issues
* not restricting to -type f
* using + insread of \; (in case one wants to use -exec)
* quoting/escaping {} to prevent interference with shells

But why are quotes and backslashes a problem? - I'd have
assumed that xargs will just exec the expanded patterns.

Janis

[Janis Papanagnou]

Am 14.02.2013 19:22, schrieb Octothorpe:
> On Thu, 14 Feb 2013 18:21:36 +0100, Janis Papanagnou wrote:
>
>> Am 14.02.2013 17:15, schrieb Octothorpe:

[...]

>>
>>
>>> Why bring xargs into this when it is not needed?
>>
>> Because it's more efficient than a find(1) call that is terminated by \;
>> instead of + .
>>
>
> YIKES!!!!!
>
> I did not know that my eight core AMD FX 32G ram with ssd drives had a
> efficency problem!
>
> Good golly Miss Molly, I lost a few nano seconds.

You consider one process invocation per found file not an issue
to consider? - If all your programming is unnecessarily done that
sloppy there's no need to continue arguing with you. - Good luck.

[...]

[Janis Papanagnou]

Am 15.02.2013 14:59, schrieb Kenny McCormack:
> In article <php1v9-...@leafnode-msgid.gclare.org.uk>,
> Geoff Clare <net...@gclare.org.uk> wrote:
> ...
>>> Why not just
>>>
>>> find . -name foo.* | xargs rm
>>
>> Because that will misbehave if any filenames contain whitespace,
>> backslash, single-quote or double-quote characters, whereas
>> find -exec does not have that problem.
>
> That can be corrected using the "-0" option in both "find" and "xargs".
> (Actually, it is "-print0" in "find", but you get the idea...)
>
> And if your versions of "find" and "xargs" don't have these options, then you
> need to upgrade.
>
> Also, the reason to use "xargs", rather than "-exec", is to avoid spawning
> an "rm" process for each file. You know, efficency and all that. Further,
> since removing files is a common operation with "find" (although not, it
> should be pointed out, the topic of this thread), it has always seemed to me
> that "find" should have it as a built-in.

GNU find has -delete (don't know whether it's standard, though), but
I am reluctant to use it given the warning in the man page:

"Warnings: Don't forget that the find command line is evaluated
as an expression, so putting -delete first will make find try to
delete everything below the starting points you specified."


Janis

[Kenny McCormack]

In article <kfli5n$s3o$1...@speranza.aioe.org>,
Janis Papanagnou <janis_pa...@hotmail.com> wrote:
...

>GNU find has -delete (don't know whether it's standard, though), but
>I am reluctant to use it given the warning in the man page:

Interesting. Obviously, I haven't checked "man find" in a while.

> "Warnings: Don't forget that the find command line is evaluated
> as an expression, so putting -delete first will make find try to
> delete everything below the starting points you specified."

I'm not sure I understand this. I guess they're just saying don't do this
and expect it to work like you (might) expect:

find ... -delete -name ... -type ... -...

I.e., -delete is an expression like everything else, not an option, like you
might expect. But I can't really imagine anyone thinking that - if they are
at all familiar with how "find" works (in particular, say, "-print").

In any case, I'd imagine you'll learn this once you start messing around
with it a bit. As they say, that's what backups are for...

--
"Remember when teachers, public employees, Planned Parenthood, NPR and PBS
crashed the stock market, wiped out half of our 401Ks, took trillions in
TARP money, spilled oil in the Gulf of Mexico, gave themselves billions in
bonuses, and paid no taxes? Yeah, me neither."

[Janis Papanagnou]

Am 15.02.2013 16:16, schrieb Kenny McCormack:
> In article <kfli5n$s3o$1...@speranza.aioe.org>,
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> ...
>> GNU find has -delete (don't know whether it's standard, though), but
>> I am reluctant to use it given the warning in the man page:
>
> Interesting. Obviously, I haven't checked "man find" in a while.
>
>> "Warnings: Don't forget that the find command line is evaluated
>> as an expression, so putting -delete first will make find try to
>> delete everything below the starting points you specified."
>
> I'm not sure I understand this. I guess they're just saying don't do this
> and expect it to work like you (might) expect:
>
> find ... -delete -name ... -type ... -...
>
> I.e., -delete is an expression like everything else, not an option, like you
> might expect. But I can't really imagine anyone thinking that - if they are
> at all familiar with how "find" works (in particular, say, "-print").
>
> In any case, I'd imagine you'll learn this once you start messing around
> with it a bit. As they say, that's what backups are for...

From the wording I expected the worst; and indeed it behaves worst...

While this one will delete all files named bbb in the hierarchy

find . -name "bbb" -delete

this one will delete *all* files in the hierarchy

find . -delete -name "bbb"

That's sick, isn't it?

Janis

>

[Kenny McCormack]

In article <kflkak$3bu$1...@speranza.aioe.org>,

Janis Papanagnou <janis_pa...@hotmail.com> wrote:
...

> From the wording I expected the worst; and indeed it behaves worst...
>
>While this one will delete all files named bbb in the hierarchy
>
> find . -name "bbb" -delete

Right - if you have a directory named "bbb". Presumably, you need to be
familiar enough with your structure so as to know what names are likely to
be files and which ones are directories. Obviously, the safety mavens here
will squawk about this assertion.

In any case, would including "-type f" fix this?

>this one will delete *all* files in the hierarchy
>
> find . -delete -name "bbb"
>
>That's sick, isn't it?

Well, yes, but as I mentioned above, you'd learn not to do that once you'd
been bitten by it once. Live and learn is the Unix motto, isn't it?

--
Faced with the choice between changing one's mind and proving that there is
no need to do so, almost everyone gets busy on the proof.

- John Kenneth Galbraith -

[Barry Margolin]

In article <kflkak$3bu$1...@speranza.aioe.org>,

But no one puts actions before filters when they write find commands.
You wouldn't ever think of writing:

find -exec rm {} \; -name "bbb"

would you?

[Janis Papanagnou]

Am 15.02.2013 16:42, schrieb Kenny McCormack:
> In article <kflkak$3bu$1...@speranza.aioe.org>,
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> ...
>> From the wording I expected the worst; and indeed it behaves worst...
>>
>> While this one will delete all files named bbb in the hierarchy
>>
>> find . -name "bbb" -delete
>
> Right - if you have a directory named "bbb". Presumably, you need to be
> familiar enough with your structure so as to know what names are likely to
> be files and which ones are directories. Obviously, the safety mavens here
> will squawk about this assertion.
>
> In any case, would including "-type f" fix this?
>
>> this one will delete *all* files in the hierarchy
>>
>> find . -delete -name "bbb"
>>
>> That's sick, isn't it?
>
> Well, yes, but as I mentioned above, you'd learn not to do that once you'd
> been bitten by it once. Live and learn is the Unix motto, isn't it?

Yes, sure.

Though given that without a given conjunction it is defined as -a (AND)
I think that an interface like that is dangerous, specifically with an
"action" like -delete. I mean, if you type find -print -name bbb
and you get all files in response it may not be that critical - well,
actually with a subsequent xargs rm it's critical as well - but in
that case I typically just omit the -print anyway.

So instead of getting bitten [even once], I try to take paths where it's
less likely to get bitten at all.

Janis

>

[Janis Papanagnou]

Am 15.02.2013 16:51, schrieb Barry Margolin:
> In article <kflkak$3bu$1...@speranza.aioe.org>,
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
>
>> From the wording I expected the worst; and indeed it behaves worst...
>>
>> While this one will delete all files named bbb in the hierarchy
>>
>> find . -name "bbb" -delete
>>
>> this one will delete *all* files in the hierarchy
>>
>> find . -delete -name "bbb"
>>
>> That's sick, isn't it?
>
> But no one puts actions before filters when they write find commands.
> You wouldn't ever think of writing:
>
> find -exec rm {} \; -name "bbb"
>
> would you?

No, I wouldn't. (Primarily because I prefer using xargs.)

The problem I see is that find's syntax is very bad in that
respect; actions look the same as the other attribute checks,
syntactically.

Janis

>

[Barry Margolin]

In article <kfll1s$64q$3...@news.xmission.com>,

gaz...@shell.xmission.com (Kenny McCormack) wrote:

> In article <kflkak$3bu$1...@speranza.aioe.org>,
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
> ...
> > From the wording I expected the worst; and indeed it behaves worst...
> >
> >While this one will delete all files named bbb in the hierarchy
> >
> > find . -name "bbb" -delete
>
> Right - if you have a directory named "bbb". Presumably, you need to be

No, you don't have to have a directory named "bbb". It will still delete
all files named "bbb" below the current directory. I think he was
actually using this as an example of the safe, expected behavior.

> familiar enough with your structure so as to know what names are likely to
> be files and which ones are directories. Obviously, the safety mavens here
> will squawk about this assertion.

find won't delete non-empty directories, so this isn't too dangerous.
However, the option seems to work depth-first -- if a matching directory
contains only matching files, the files will be deleted, and then the
directory will be deleted as well.

>
> In any case, would including "-type f" fix this?
>
> >this one will delete *all* files in the hierarchy
> >
> > find . -delete -name "bbb"
> >
> >That's sick, isn't it?
>
> Well, yes, but as I mentioned above, you'd learn not to do that once you'd
> been bitten by it once. Live and learn is the Unix motto, isn't it?

--

[Octothorpe]

Im not arguing... I'm retired.

Any way for this problem I don't think you have an argument as I would
care how long it took as this problem looks like a one of problem.

[Barry Margolin]

In article <kfllr4$848$1...@speranza.aioe.org>,

But users don't really think of it that way. They think of it as "if
<attributes> then <actions>", and they enter the arguments that way.

Things do get complicated when you throw in -o and the possibility of
doing different actions in each branch. But find novices are unlikely
to try this (99% of my uses of this are for -prune).

[Janis Papanagnou]

Am 15.02.2013 17:03, schrieb Barry Margolin:
> In article <kfllr4$848$1...@speranza.aioe.org>,
> Janis Papanagnou <janis_pa...@hotmail.com> wrote:
>

[...]

>>
>> The problem I see is that find's syntax is very bad in that
>> respect; actions look the same as the other attribute checks,
>> syntactically.
>
> But users don't really think of it that way. They think of it as "if
> <attributes> then <actions>", and they enter the arguments that way.

Well, I can't tell about other users. My thinking goes along that way;
get data, restrict as necessary. That pattern is also supported by
Unix'es pipelining: data-gerenator | filter-a-bit | filter-some-more .
WRT find I see me start typing find -name bbb -delete and then
adding another restriction like find -name bbb -delete -type f
And I suppose I'd get surprised by the effect.

Don't you think that find's interface is badly designed?

The command tw(1) as a (better?) example differenciates commands from
options.[*]

Janis

[*] http://www2.research.att.com/sw/download/man/man1/tw.html

[Ed Morton]

On 2/14/2013 6:00 AM, Ed Morton wrote:
> If I want to remove all files starting with "foo" in my directory I can do:
>

> find . -name foo.* | xargs rm '{}'
>
> or use -exec. What if I want to rename them to bar.* instead of removing them?
>
> I would normally write a loop:
>
> find . -name foo.* |
> while IFS= read -r file; do mv "$file" "bar${file#foo}"; done
>
> but is there a way to do that with find -exec or with xargs, something like this
> pseudo-code which doesn't work:
>
> find . -name foo.* | xargs mv '{}' "bar${{}#foo}"
>
> The question isn't about how to rename files, it's about how to do substitution
> on file names within find -exec or xargs.
>

> Regards,
>
> Ed.
>

Thanks all for the response, I believe I have my answer - you can't use
substitution on {} in xargs or find -exec so if I want to rename all the files
that find finds using some standard substitution like changing all lower case to
upper case or changing blanks to underscores or replace a string, then I need to
either write a loop:

$ find . -name 'foo.*' |
while IFS= read -r file; do mv "$file" "bar${file#foo}"; done

or write a command and invoke that command with xargs or -exec:

$ cat domv
mv "$1" "bar${1#foo}"

$ find . -name 'foo.*' | xargs domv

Thanks,

Ed.

[Aragorn]

On Friday 15 February 2013 19:15, Michael Vilain conveyed the following
to comp.unix.shell...

> [...] I had to deal with stupidly evil developers like this all the
> time. They create files in /tmp and don't bother do delete them,
> leaving that to me when I end up with a machine that won't boot any
> more after it's been running in production a month.

Install tmpwatch and/or put /tmp on a tmpfs. The latter solution makes
sure your system will always boot [*] because the contents of /tmp will
live in virtual memory and are thus wiped out at every boot. The former
solution is a script that runs from cron and periodically deletes stuff
in /tmp and /var/tmp if the atime is too old.

Some GNU/Linux distributions also automatically wipe out /tmp at every
boot from within the init scripts, albeit that I don't know how other
UNIX variants handle that. I think Solaris wipes /tmp at boot as well,
but I'm not sure.


[*] Unless there's something more serious going on, of course.

--
= Aragorn =
(registered GNU/Linux user #223157)

[Aragorn]

On Friday 15 February 2013 19:25, Aragorn conveyed the following to
comp.unix.shell...

> Install tmpwatch and/or put /tmp on a tmpfs. The latter solution
> makes sure your system will always boot [*] because the contents of
> /tmp will live in virtual memory and are thus wiped out at every boot.

^^^^

That should read "reboot". Or at hardware shutdown, if you will.

[Barry Margolin]

In article <kflnh0$t02$1...@dont-email.me>,

Ed Morton <morto...@gmail.com> wrote:

> Thanks all for the response, I believe I have my answer - you can't use
> substitution on {} in xargs or find -exec

Sorry, all the tangents about other aspects of find syntax have rendered
your original question off-topic.

:)

[Octothorpe]

On Fri, 15 Feb 2013 10:15:20 -0800, Michael Vilain wrote:

> In article <kflgv8$oe5$1...@speranza.aioe.org>,

> Yeah, I had to deal with stupidly evil developers like this all the

> time. They create files in /tmp and don't bother do delete them, leaving
> that to me when I end up with a machine that won't boot any more after
> it's been running in production a month.
>

> Or they don't consider backups. Or that downtime is a way of life
> unless you design redundancy in from the get go.
>
> Or build 'solutions' like this cretin and wonder why it won't scale.

Ok now I am a stupidly evil developer/cretin.
Those things are a figment in your mind.
You should know that developers have had to content with stupidly evil
admins/users/cretins as well. Or don't they exist?

Not to burst your infallible image of your self... I am not presently a
developer and no way what I posted goes expressly into a program or suite
of applications and is not only way to do something. It is up to the OP
to determine if it will suit his/her needs.

Please point me to the post where this was going into an application or
where I am co-developing this with the OP.

[Ed Morton]

On 2/15/2013 1:07 PM, Barry Margolin wrote:
> In article <kflnh0$t02$1...@dont-email.me>,
> Ed Morton <morto...@gmail.com> wrote:
>
>> Thanks all for the response, I believe I have my answer - you can't use
>> substitution on {} in xargs or find -exec
>
> Sorry, all the tangents about other aspects of find syntax have rendered
> your original question off-topic.
>
> :)
>

Yeah, I REALLY thought if I specifically reiterated in my original post "The

question isn't about how to rename files, it's about how to do substitution on

file names within find -exec or xargs." that maybe it'd stay on track but there
y'go... I got my answer anyway, and some extra entertainment too boot so thanks
again to all for contributing :-).

Ed.

[Octothorpe]

On Fri, 15 Feb 2013 17:00:06 -0800, Michael Vilain wrote:

> In article <l5j2v9-...@crazy-horse.bildanet.com>,

> Yes, I know I'm an asshole. It's my job to keep root from people like
> you who just want 'tweek a few things' at 2am, thereby rendering a
> system unbootable. And yes, this has happened to me--the idiot was a
> key developer who later quit when we took root away from him and called
> the head of development 'worthless' in an annual meeting.

Please then take root account from me, yes from all my computers. I bet
you can not, you little twerp. See you lost go back to the cave from
which you came. You are worthless and a pox on the human race.

>
> To often I've had to implement half-assed solutions that didn't consider
> what it would take to run long-term, day-to-day, or if it would scale.
>

Oh my yes you are here to save the world. Do the world a favor and save
it from you.

> If you weren't a stupid, lazy developer, you sure have the attitude of
> one. Now you're just retired. Terrific. Maybe you can fix some Cobol
> code that's running the California state employee's pension fund. And
> I'll get my street corner back from the trash.

May all your apps crash on you at 1:00am, you deserve it and more.

You concoct all these stupid scenarios in your mind, why I do not know.
You have mental problems. You sir are just an egotistical asshole, you
should take your meds, life is too short for your bullshit attitude.

[Octothorpe]

On Sat, 16 Feb 2013 09:25:30 -0800, Michael Vilain wrote:

[putolin]


> Meds, as a rule, do help. But not with dealing with developers who
> think they know more about running the datacenter. No, you can't do
> your upgrade the last week of the month, that's when accounting does
> their month end close and everything has to be up. And yes, life is
> getting to short to deal with people like you. No, you can't have root
> on the production servers. Ask your manager why.

I have not asked for root access to any servers, although I do have root
access to mine own. What fantasies are you playing out?

The system admin should be the one to "upgrade" the app. No wonder you
folks have the issues you do. In case you have not realized a developer
develops the application The developer is not responsible for
installation, it is up to the system admin to install the beast.
Because you don't have the skills to install software apps, Maybe you are
an incompetent system admin?

I posted find -name 'foo*' -exec unlink '{}' \; and now I am a developer?

Actually I am a packager/distribution "developer" as I create custom linux
distributions from scratch. I am almost finished with my latest version
using the rpm package manager. Which will be installed LAMP system for a
leading fortune 500 company.

Thanks for playing.

[Geoff Clare]

Ed Morton wrote:

> Thanks all for the response, I believe I have my answer - you can't use
> substitution on {} in xargs or find -exec so if I want to rename all the files
> that find finds using some standard substitution like changing all lower case to
> upper case or changing blanks to underscores or replace a string, then I need to
> either write a loop:
>
> $ find . -name 'foo.*' |
> while IFS= read -r file; do mv "$file" "bar${file#foo}"; done
>
> or write a command and invoke that command with xargs or -exec:
>
> $ cat domv
> mv "$1" "bar${1#foo}"
>
> $ find . -name 'foo.*' | xargs domv

Or you can use:

find ... -exec sh -c 'for file; do ...; done' sh {} +

as shown by pk early on in this thread. (Actually, pk used a while
loop; I think using for is neater.)

--
Geoff Clare <net...@gclare.org.uk>

[Geoff Clare]

pk wrote:

> On Fri, 15 Feb 2013 13:59:41 +0000 (UTC), gaz...@shell.xmission.com (Kenny
> McCormack) wrote:
>
>> In article <php1v9-...@leafnode-msgid.gclare.org.uk>,
>> Geoff Clare <net...@gclare.org.uk> wrote:
>> ...
>> >> Why not just
>> >>
>> >> find . -name foo.* | xargs rm
>> >
>> >Because that will misbehave if any filenames contain whitespace,
>> >backslash, single-quote or double-quote characters, whereas
>> >find -exec does not have that problem.
>>
>> That can be corrected using the "-0" option in both "find" and "xargs".
>> (Actually, it is "-print0" in "find", but you get the idea...)
>>
>> And if your versions of "find" and "xargs" don't have these options, then
>> you need to upgrade.
>>
>> Also, the reason to use "xargs", rather than "-exec", is to avoid spawning
>> an "rm" process for each file. You know, efficency and all that.
>

> You talk about the need to upgrade, yet don't seem to know about the "+"
> termination to -exec, which has surely been around longer than -print0, -0
> and all that.

As Casper said, -exec ... {} + has been around since SVR4. I don't
know whether that antedates GNU's invention of -print0.

However, POSIX has only required find implementations to support the '+'
terminator since 2001. GNU added support for it not long after that, at
which point it became the more portable of the two solutions (as well as
the more efficient, because it eliminates the pipe I/O between find
and xargs).

--
Geoff Clare <net...@gclare.org.uk>

[Geoff Clare]

Janis Papanagnou wrote:

> Am 15.02.2013 14:24, schrieb Geoff Clare:

>> Janis Papanagnou wrote:
>>
>>> Why not just
>>>
>>> find . -name foo.* | xargs rm
>>
>> Because that will misbehave if any filenames contain whitespace,
>> backslash, single-quote or double-quote characters, whereas
>> find -exec does not have that problem.

> But why are quotes and backslashes a problem? - I'd have
> assumed that xargs will just exec the expanded patterns.

$ ls -1
"c"
'a'
\b
x y
$ find . -type f | xargs ls -1
ls: cannot access ./b: No such file or directory
ls: cannot access ./c: No such file or directory
ls: cannot access ./a: No such file or directory
ls: cannot access ./x: No such file or directory
ls: cannot access y: No such file or directory
$ find . -type f -exec ls -1 {} +
./"c"
./'a'
./\b
./x y

--
Geoff Clare <net...@gclare.org.uk>

[Janis Papanagnou]

Yes, and I think -print0 resp. -0 will fix that.

My question is; why are those supposed to be a problem?
Who does the interpretation? Why will those not just be
passed to exec as individual arguments ny find?

Janis

[Janis Papanagnou]

Am 20.02.2013 17:08, schrieb Janis Papanagnou:
> Am 20.02.2013 15:19, schrieb Geoff Clare:
>> Janis Papanagnou wrote:

>>>[...]

>>> But why are quotes and backslashes a problem? - I'd have
>>> assumed that xargs will just exec the expanded patterns.
>>
>> $ ls -1
>> "c"
>> 'a'
>> \b
>> x y
>> $ find . -type f | xargs ls -1
>> ls: cannot access ./b: No such file or directory
>> ls: cannot access ./c: No such file or directory
>> ls: cannot access ./a: No such file or directory
>> ls: cannot access ./x: No such file or directory
>> ls: cannot access y: No such file or directory

>> [...]

>
> Yes, and I think -print0 resp. -0 will fix that.
>
> My question is; why are those supposed to be a problem?
> Who does the interpretation? Why will those not just be
> passed to exec as individual arguments ny find?

s/ny find/by xargs/

>
> Janis
>

[Geoff Clare]

Janis Papanagnou wrote:

> Am 20.02.2013 15:19, schrieb Geoff Clare:
>> Janis Papanagnou wrote:
>>
>>> But why are quotes and backslashes a problem? - I'd have
>>> assumed that xargs will just exec the expanded patterns.
>>
>> $ ls -1
>> "c"
>> 'a'
>> \b
>> x y
>> $ find . -type f | xargs ls -1
>> ls: cannot access ./b: No such file or directory
>> ls: cannot access ./c: No such file or directory
>> ls: cannot access ./a: No such file or directory
>> ls: cannot access ./x: No such file or directory
>> ls: cannot access y: No such file or directory
>> $ find . -type f -exec ls -1 {} +
>> ./"c"
>> ./'a'
>> ./\b
>> ./x y

> My question is; why are those supposed to be a problem?
> Who does the interpretation? Why will those not just be

> passed to exec as individual arguments by xargs?

The original design of xargs was such that you feed it, on standard
input, the same kinds of things that you would type on the command
line as arguments to the program it executes. Therefore it processes
quoting in much the same way (although the precise rules are slightly
different) as the shell.

However, I doubt if it ever got used much that way. It's main use has
always been to feed it pathnames, one per line (usually from find).
This worked fine back in the day, because everybody only ever used
"sensible" filenames. These days, many people have filenames with
whitespace or quote characters in them, and this behaviour has become
a gotcha.

--
Geoff Clare <net...@gclare.org.uk>

[Kenny McCormack]

In article <mpjhv9-...@leafnode-msgid.gclare.org.uk>,

Geoff Clare <net...@gclare.org.uk> wrote:
...

>However, I doubt if it ever got used much that way. It's main use has
>always been to feed it pathnames, one per line (usually from find).
>This worked fine back in the day, because everybody only ever used
>"sensible" filenames. These days, many people have filenames with
>whitespace or quote characters in them, and this behaviour has become
>a gotcha.

It is a shame that Unix doesn't have the FileNameShort() functionality like
Windows does. In Windows, you can get around most "screwy filename" issues
by applying the FileNameShort() function to the long filename, and then
using the resulting short filename (guaranteed to be 8.3 - i.e., good old
"FAT" - compatible) for all future operations.

P.S. Also, as noted previously, using the "-print0" option in "find" and
"-0" option in xargs solves most of these issues.

--
"The anti-regulation business ethos is based on the charmingly naive notion
that people will not do unspeakable things for money." - Dana Carpender

Quoted by Paul Ciszek (pciszek at panix dot com). But what I want to know
is why is this diet/low-carb food author doing making pithy political/economic
statements?

Nevertheless, the above quote is dead-on, because, the thing is - business
in one breath tells us they don't need to be regulated (which is to say:
that they can morally self-regulate), then in the next breath tells us that
corporations are amoral entities which have no obligations to anyone except
their officers and shareholders, then in the next breath they tell us they
don't need to be regulated (that they can morally self-regulate) ...

[Dan Espen]

gaz...@shell.xmission.com (Kenny McCormack) writes:

> In article <mpjhv9-...@leafnode-msgid.gclare.org.uk>,
> Geoff Clare <net...@gclare.org.uk> wrote:
> ...
>>However, I doubt if it ever got used much that way. It's main use has
>>always been to feed it pathnames, one per line (usually from find).
>>This worked fine back in the day, because everybody only ever used
>>"sensible" filenames. These days, many people have filenames with
>>whitespace or quote characters in them, and this behaviour has become
>>a gotcha.
>
> It is a shame that Unix doesn't have the FileNameShort() functionality like
> Windows does. In Windows, you can get around most "screwy filename" issues
> by applying the FileNameShort() function to the long filename, and then
> using the resulting short filename (guaranteed to be 8.3 - i.e., good old
> "FAT" - compatible) for all future operations.

Good old FAT huh?

I think you need to do some research into the short file name abortion
before you come up with "it is a shame".

Start here:

http://en.wikipedia.org/wiki/8.3_filename

--
Dan Espen

[Octothorpe]

On Thu, 21 Feb 2013 14:04:31 +0000, Kenny McCormack wrote:

>
> It is a shame that Unix doesn't have the FileNameShort() functionality
> like Windows does.

No its not!

[Janis Papanagnou]

Am 21.02.2013 14:24, schrieb Geoff Clare:
>> [...]

>
> The original design of xargs was such that you feed it, on standard
> input, the same kinds of things that you would type on the command
> line as arguments to the program it executes. Therefore it processes
> quoting in much the same way (although the precise rules are slightly
> different) as the shell.

Yeah. And I should have read the man page first; it's clearly stated
as a feature - doh! (Misfeature if you ask me; similar to "read -r"
where the non-special behaviour has to be activated through options.)
So find -print0 and xargs -0 can be considered mandatory. Or using
find's -exec options, as suggested.

>
> However, I doubt if it ever got used much that way. It's main use has
> always been to feed it pathnames, one per line (usually from find).
> This worked fine back in the day, because everybody only ever used
> "sensible" filenames. These days, many people have filenames with
> whitespace or quote characters in them, and this behaviour has become
> a gotcha.

Hmm.. - xargs *could* have been defined to use -0 only in case of
significant newlines in input and don't interpret spaces and quotes
as special case, then even those "modern" file names would be not an
issue and could be handled without -0. We could focus on pathological
file names then (and perhaps consider exclude control characters from
allowed characters in filenames[*]).

Janis

[*] It's really hard for me to abandon that wish. (But I am digressing.)