best way to test for empty dir?

[Marc Herbert]

Hi,

Does anyone know a more elegant way to check for file existence?
Something that does not fork a subshell. And is also more readable
maybe. And is obviously not much longer.

empty_dir()
{
test "x$(echo $1/*$2)" = "x$1"'/*'"$2"
}

Warning: I find neither "noglob" nor "ls" elegant, sorry!

[Greg Wooledge]

On Thu, Dec 10, 2009 at 05:31:20PM +0000, Marc Herbert wrote:
> Does anyone know a more elegant way to check for file existence?
> Something that does not fork a subshell. And is also more readable
> maybe. And is obviously not much longer.

shopt -s nullglob
files=(*)
if (( ${#files[*]} == 0 )); then ...
shopt -u nullglob

> Warning: I find neither "noglob" nor "ls" elegant, sorry!

Well, some people like doing it this way:

files=(*)
if [[ ! -e ${files[0]} ]]; then ...

But:
1) There's a race condition between the initial enumeration and the
check of the first element.
2) It fails to detect dotfiles (do you also consider "dotglob"
inelegant?).
3) I personally find it even more of a hack than enabling nullglob.

[Matias A. Fonzo]

On Thu, 10 Dec 2009 17:31:20 +0000
Marc Herbert <Marc.H...@gmail.com> wrote:

> Hi,
>

Hello

> Does anyone know a more elegant way to check for file existence?
> Something that does not fork a subshell. And is also more readable
> maybe. And is obviously not much longer.
>

> empty_dir()
> {
> test "x$(echo $1/*$2)" = "x$1"'/*'"$2"
> }
>
>

> Warning: I find neither "noglob" nor "ls" elegant, sorry!
>

Maybe you want the Chris F.A Johnson's implementation [1]:

set -- "/tmp/emptydir"/*
[[ -f $1 ]] && echo non-empty || echo empty;

References:
[1] http://www.issociate.de/board/goto/866027/checking_if_a_directory_is_empty.html

--
Matias A. Fonzo <se...@dragora.org>

[Greg Wooledge]

On Thu, Dec 10, 2009 at 05:37:04PM -0200, Matias A. Fonzo wrote:
> Maybe you want the Chris F.A Johnson's implementation [1]:
>
> set -- "/tmp/emptydir"/*
> [[ -f $1 ]] && echo non-empty || echo empty;
>
> References:
> [1] http://www.issociate.de/board/goto/866027/checking_if_a_directory_is_empty.html

The -f in the [[...]] should be -e, or it may give erroneous results if
the first thing matched by the glob happens to be a subdirectory (or
anything other than a plain file).

It's just a positional-parameter variant of:

files=("/tmp/emptydir"/*)
if [[ -e ${files[0]} ]] ...

Has the disadvantage that it clobbers the positional parameters, which
maybe you still want. Has the advantage that it'll work in ksh88,
which doesn't support the array=(...) syntax. They're both non-POSIX
though (due to the [[...]]). Of course, the PP variant is easier to
convert into a working POSIX version, since POSIX has no guarantee of
arrays.

[Chris F.A. Johnson]

On Thu, 10 Dec 2009, Marc Herbert wrote:

> Does anyone know a more elegant way to check for file existence?
> Something that does not fork a subshell. And is also more readable
> maybe. And is obviously not much longer.
>
> empty_dir()
> {
> test "x$(echo $1/*$2)" = "x$1"'/*'"$2"
> }
>
>
> Warning: I find neither "noglob" nor "ls" elegant, sorry!

is_file()
{
for f
do
[ -f "$f" ] && return
done
return 1
}

is_file /path/to/dir/* || echo empty

--
Chris F.A. Johnson, webmaster <http://woodbine-gerrard.com>
===================================================================
Author:
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)

[pk]

Chris F.A. Johnson wrote:

> On Thu, 10 Dec 2009, Marc Herbert wrote:
>
>> Does anyone know a more elegant way to check for file existence?
>> Something that does not fork a subshell. And is also more readable
>> maybe. And is obviously not much longer.
>>
>> empty_dir()
>> {
>> test "x$(echo $1/*$2)" = "x$1"'/*'"$2"
>> }
>>
>>
>> Warning: I find neither "noglob" nor "ls" elegant, sorry!
>
> is_file()
> {
> for f
> do
> [ -f "$f" ] && return
> done
> return 1
> }
>
> is_file /path/to/dir/* || echo empty

This fails if the directory contains a file called "*".

[pk]

pk wrote:

>> is_file()
>> {
>> for f
>> do
>> [ -f "$f" ] && return
>> done
>> return 1
>> }
>>
>> is_file /path/to/dir/* || echo empty
>
> This fails if the directory contains a file called "*".

My bad, it works correctly. The only issue I see is maybe that "-e" would be
more appropriate than "-f" since the first (and perhaps only) element could
be a directory.

[Andreas Schwab]

Greg Wooledge <woo...@eeg.ccf.org> writes:

> It's just a positional-parameter variant of:
>
> files=("/tmp/emptydir"/*)
> if [[ -e ${files[0]} ]] ...

This will still fail if the first file happens to be a dangling symlink.

Andreas.

--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."

[Marc Herbert]

>>> empty_dir()
>>> {
>>> test "x$(echo $1/*$2)" = "x$1"'/*'"$2"
>>> }

pk wrote:
> This fails if the directory contains a file called "*".

Yes. Unlike the ones below, empty_dir() above considers as empty a
directory that has a SINGLE element named '*'. Since I am not so
interested in files named '*', I think I can live with that!

Chris & others wrote:
>> is_file2()
>> {
>> for f
>> do
>> [ -e "$f" ] && return
>> done
>> return 1
>> }

I think I like this one.

Andreas:

> This will still fail if the first file happens to be a dangling symlink.

Yes, but I think it's valuable to refine "fail" again. Dangling symlinkS
(not just the first one) will be ignored, just like they were not
here. Some might find this acceptable.

For purists, does this one works even better?

is_file3()
{
for f
do
[ -e "$f" -o -L "$f" ] && return
done
return 1
}

Thanks to everyone who answered, appreciated.

[Antonio Macchi]

is_file()
{
[ -f "$1" ] && return
return 1
}

is_file /path/to/dir/* || echo empty

you don't need to check more than the first element

[Antonio Macchi]

is_file()
{
[ -f "$1" ]
}

is_file /path/to/dir/* || echo empty

test is redundant too

---

this could be another way to accomplish this

empty_dir() {
eval test \" $1/* \" == \"" $1/* "\";
}

(excluding invisible files...)

[pk]

Marc Herbert wrote:

> For purists, does this one works even better?
>
> is_file3()
> {
> for f
> do
> [ -e "$f" -o -L "$f" ] && return
> done
> return 1
> }

You might also want to enable "dotglob" to catch hidden files...

[Sven Mascheck]

On Fri, Dec 11, 2009 at 12:31:49PM +0000, pk wrote:
> Marc Herbert wrote:

> > is_file3()
> > {
> > for f
> > do
> > [ -e "$f" -o -L "$f" ] && return
> > done
> > return 1
> > }
>
> You might also want to enable "dotglob" to catch hidden files...

empty=yes
for i in .?* *
do
test "$i" = '..' || test -e "$i" && empty=no && break
done

and with test -L added as needed.
Or (beacuse test -e is not strictly traditionally available)
and without special glob options:

empty=no
for i in .* * ; do
case "$i" in
.|..) : ;;
\*) for i in ? ; do
test "$i" = \? && empty=yes ; break
done ; break ;;
*) break ;;
esac
done

or just (if builtins-only is not strictly required)

test -z "`find . ! -name . -prune | sed q`"

comp.unix.shell might match well here and could be entertaining -
IMHO worth to migrate; objections?

[Greg Wooledge]

On Fri, Dec 11, 2009 at 01:09:09PM +0100, Antonio Macchi wrote:
>
> this could be another way to accomplish this
>
> empty_dir() {
> eval test \" $1/* \" == \"" $1/* "\";
> }
>
> (excluding invisible files...)

This one also has the problem of failing if the directory contains a
single file named '*'.

It also blows up if you pass a specially crafted parameter,
e.g. /tmp/'"`date`"' due to lack of sanitizing $1 before calling eval.
Worse, it blows up if the *directory* contains specially crafted files
(such as '"`date`"') and there is *no* workaround for that short of
rewriting the whole thing.

imadev:~$ touch /tmp/sdf/'"`date 1>&2`"'
imadev:~$ empty_dir /tmp/sdf
Fri Dec 11 08:23:27 EST 2009

(Insert generic "now replace date with rm" advice.)

[Chris F.A. Johnson]

On Fri, 11 Dec 2009, Sven Mascheck wrote:
...

> comp.unix.shell might match well here and could be entertaining -
> IMHO worth to migrate; objections?

This has been discussed more than once in c.u.s; check the
archives.

[Chris F.A. Johnson]

On Fri, 11 Dec 2009, Antonio Macchi wrote:

> is_file()
> {
> [ -f "$1" ] && return

> return 1
> }
>
> is_file /path/to/dir/* || echo empty
>
>
>

> you don't need to check more than the first element

You may need to; it depends on the statement of the problem.

[Sven Mascheck]

Chris F.A. Johnson wrote:

> This has been discussed more than once in c.u.s; check the
> archives.

and that's why we better discuss it here now?

[Marc Herbert]

Sven Mascheck a écrit :

I think Chris' message was more like: "let's not discuss it at all and
just read the archives" :-]

In case anyone is interested my winner (so far) is:

exists()
{
[ -e "$1" -o -L "$1" ]
}

if exists foo/*; then
for f in foo/*; do
...
done
fi

[Greg Wooledge]

On Fri, Dec 11, 2009 at 04:16:13PM +0000, Marc Herbert wrote:
> In case anyone is interested my winner (so far) is:
>
> exists()
> {
> [ -e "$1" -o -L "$1" ]
> }
>
> if exists foo/*; then
> for f in foo/*; do
> ...
> done
> fi

What if there's a subdirectory or something and you'd like to skip it?

for f in foo/*; do

test -f "$f" || continue
...
done

Hence my explanations at the bottom of http://mywiki.wooledge.org/BashFAQ/004

[Matias A. Fonzo]

On Fri, 11 Dec 2009 16:16:13 +0000
Marc Herbert <Marc.H...@gmail.com> wrote:

> Sven Mascheck a écrit :
> > Chris F.A. Johnson wrote:
> >
> >> This has been discussed more than once in c.u.s; check the
> >> archives.
> >
> > and that's why we better discuss it here now?
>
> I think Chris' message was more like: "let's not discuss it at all and
> just read the archives" :-]
>
>

> In case anyone is interested my winner (so far) is:
>
> exists()
> {
> [ -e "$1" -o -L "$1" ]
> }
>

The -L is redundant. Because, if the symlink is not broken, the regular file "exists" ( -e ).

A solution to check the broken symlink is:

[ -e "foo" -o -L "foo" -a ! -e "foo" ]

> if exists foo/*; then
> for f in foo/*; do
> ...
> done
> fi
>
>
>

--
Matias A. Fonzo <se...@dragora.org>

[Andreas Schwab]

"Matias A. Fonzo" <se...@dragora.org> writes:

> A solution to check the broken symlink is:
>
> [ -e "foo" -o -L "foo" -a ! -e "foo" ]

In which way is the last check not redundant?

[Stephane CHAZELAS]

2009-12-11, 16:16(+00), Marc Herbert:
> Sven Mascheck a ï¿œcrit :

>> Chris F.A. Johnson wrote:
>>
>>> This has been discussed more than once in c.u.s; check the
>>> archives.
>>
>> and that's why we better discuss it here now?
>
> I think Chris' message was more like: "let's not discuss it at all and
> just read the archives" :-]
>
>
> In case anyone is interested my winner (so far) is:
>
> exists()
> {
> [ -e "$1" -o -L "$1" ]
> }

$ exists =
bash: [: too many arguments

[ -e "$1" ] -o [ -L "$1" ]

(that one would still choke on '=' with the Bourne shell, note).

> if exists foo/*; then
> for f in foo/*; do
> ...
> done
> fi

[...]

Also, if you have the 'r' but not 'x' permission on 'foo', the
wildcard will expand, but the tests will fail. You don't need
the tests, you can do:

(
set -- foo/[*] foo/*
case $1$2 in
("foo/[*]foo/*") echo no non-hidden files or directory not readable;;
(*) echo some non-hidden files in here
esac
)

Or bash specific:

shopt -s nullglob dotglob
files=(foo/*)
(( ${#files[@]} ))

zsh:

files=(foo/*(ND[1]))
(( $#files ))

--
Stï¿œphane

[Stephane CHAZELAS]

2009-12-12, 10:21(+00), Stephane CHAZELAS:
[...]

>> exists()
>> {
>> [ -e "$1" -o -L "$1" ]
>> }
>
> $ exists =
> bash: [: too many arguments
>
> [ -e "$1" ] -o [ -L "$1" ]

[...]

Sorry, I meant

[ -e "$1" ] || [ -L "$1" ]

--
Stï¿œphane

[Matias A. Fonzo]

On Sat, 12 Dec 2009 10:51:31 +0100
Andreas Schwab <sch...@linux-m68k.org> wrote:

> "Matias A. Fonzo" <se...@dragora.org> writes:
>
> > A solution to check the broken symlink is:
> >
> > [ -e "foo" -o -L "foo" -a ! -e "foo" ]
>
> In which way is the last check not redundant?
>

In the sense that you have -e to check existing files and non-broken symlinks -instead of- "-e -o -L".

Regards,
Matías

[Marc Herbert]

Matias A. Fonzo a écrit :

> On Fri, 11 Dec 2009 16:16:13 +0000
> Marc Herbert <Marc.H...@gmail.com> wrote:

>> In case anyone is interested my winner (so far) is:
>>

>> exists()
>> {
>> [ -e "$1" -o -L "$1" ]
>> }
>>
>

> The -L is redundant.

Not for me. I need -L because I want to consider broken symlinks just
like anything else. A broken symlink would be a bug in my code and I want to
detect it ASAP.

> Because, if the symlink is not broken, the regular file "exists" ( -e ).

Please forget about correct symlinks. The -L is here for *broken*
symlinks.

> A solution to check the broken symlink is:
>
> [ -e "foo" -o -L "foo" -a ! -e "foo" ]

For which type of "foo" object does this return a different value than
the above? None.

If common sense is not enough, here is a formal proof that your third
and last test is redundant:

-e or (-L and ! -e) == (-e or -L) and (-e or ! -e) distributivity
(-e or -L) and 1 complements
-e or -L boundedness

<http://en.wikipedia.org/wiki/Boolean_logic#Properties>

[Matias A. Fonzo]

On Mon, 14 Dec 2009 12:21:12 +0000
Marc Herbert <Marc.H...@gmail.com> wrote:

> Matias A. Fonzo a écrit :
> > On Fri, 11 Dec 2009 16:16:13 +0000
> > Marc Herbert <Marc.H...@gmail.com> wrote:
>
> >> In case anyone is interested my winner (so far) is:
> >>
> >> exists()
> >> {
> >> [ -e "$1" -o -L "$1" ]
> >> }
> >>
> >
>
> > The -L is redundant.
>
> Not for me. I need -L because I want to consider broken symlinks just
> like anything else. A broken symlink would be a bug in my code and I want to
> detect it ASAP.
>
>
> > Because, if the symlink is not broken, the regular file "exists" ( -e ).
>
> Please forget about correct symlinks. The -L is here for *broken*
> symlinks.
>

The [ -L "foo" -a ! -e "foo" ] is a specific case to check dangling symlinks.

>
> > A solution to check the broken symlink is:
> >
> > [ -e "foo" -o -L "foo" -a ! -e "foo" ]
>
> For which type of "foo" object does this return a different value than
> the above? None.
>

Is just an example.

> If common sense is not enough, here is a formal proof that your third
> and last test is redundant:
>
> -e or (-L and ! -e) == (-e or -L) and (-e or ! -e) distributivity
> (-e or -L) and 1 complements
> -e or -L boundedness
>
> <http://en.wikipedia.org/wiki/Boolean_logic#Properties>
>

Yeah logic.. I have intuition.

Regards,
Matías

[Andreas Schwab]

"Matias A. Fonzo" <se...@dragora.org> writes:

> On Mon, 14 Dec 2009 12:21:12 +0000
> Marc Herbert <Marc.H...@gmail.com> wrote:
>
>> Matias A. Fonzo a écrit :
>> > On Fri, 11 Dec 2009 16:16:13 +0000
>> > Marc Herbert <Marc.H...@gmail.com> wrote:
>>
>> >> In case anyone is interested my winner (so far) is:
>> >>
>> >> exists()
>> >> {
>> >> [ -e "$1" -o -L "$1" ]
>> >> }
>> >>
>> >
>>
>> > The -L is redundant.
>>
>> Not for me. I need -L because I want to consider broken symlinks just
>> like anything else. A broken symlink would be a bug in my code and I want to
>> detect it ASAP.
>>
>>
>> > Because, if the symlink is not broken, the regular file "exists" ( -e ).
>>
>> Please forget about correct symlinks. The -L is here for *broken*
>> symlinks.
>>
>
> The [ -L "foo" -a ! -e "foo" ] is a specific case to check dangling symlinks.

Combine that with the existence check and you have exactly the
expression above.

[Matias A. Fonzo]

On Tue, 15 Dec 2009 10:37:36 +0100
Andreas Schwab <sch...@linux-m68k.org> wrote:

> "Matias A. Fonzo" <se...@dragora.org> writes:
>
> > On Mon, 14 Dec 2009 12:21:12 +0000
> > Marc Herbert <Marc.H...@gmail.com> wrote:
> >
> >> Matias A. Fonzo a écrit :
> >> > On Fri, 11 Dec 2009 16:16:13 +0000
> >> > Marc Herbert <Marc.H...@gmail.com> wrote:
> >>
> >> >> In case anyone is interested my winner (so far) is:
> >> >>
> >> >> exists()
> >> >> {
> >> >> [ -e "$1" -o -L "$1" ]
> >> >> }
> >> >>
> >> >
> >>
> >> > The -L is redundant.
> >>
> >> Not for me. I need -L because I want to consider broken symlinks just
> >> like anything else. A broken symlink would be a bug in my code and I want to
> >> detect it ASAP.
> >>
> >>
> >> > Because, if the symlink is not broken, the regular file "exists" ( -e ).
> >>
> >> Please forget about correct symlinks. The -L is here for *broken*
> >> symlinks.
> >>
> >
> > The [ -L "foo" -a ! -e "foo" ] is a specific case to check dangling symlinks.
>
> Combine that with the existence check and you have exactly the
> expression above.
>

Not quite.

Here an interesting quote from the Greg's FAQ:

"The -e test (like all other tests besides -L or -h) follows the symbolic link, and therefore it checks on the thing pointed to, not on the link itself. The -L test does not follow the symlink, so it's checking on the link itself. Together, they can indicate the presence of a dangling symlink."

You can see, creating a dangling symlink:

$ ln -sf x y
$ sh -c '[ -e "y" ] && echo true || echo false'
false
$ sh -c '[ -L "a" ] && echo true || echo false'
true

Regards,
Matías

[Andreas Schwab]

Combine the two tests and you have exactly the expression above.

[Matias A. Fonzo]

On Tue, 15 Dec 2009 15:23:33 +0100
Andreas Schwab <sch...@linux-m68k.org> wrote:

Sorry, change the last "a" to "y". :-)

[Antonio Macchi]

> [ -e "foo" -o -L "foo" -a ! -e "foo" ]

it has no sense doing twice the "-e" test

$ ln -s nonexistent foo

$ [ -e "foo" -o -L "foo" -a ! -e "foo" ] && echo ok || echo ko
ok

$ [ -e "foo" -o -L "foo" ] && echo ok || echo ko
ok

as you can see, the first "-e" check imply the second one
(aka, if the first "-e" is false, necessarily the second one will be
true...)

maybe you have "too rougly" join the trick to check a broken link

-L "foo" -a ! -e "foo"

but in this particular case you don't have to check "only" broken links,
but every file, broken links included...

so every file but broken links is "-e"
links and broken links is "-L"

join together -e and -L
every file, included links and broken links

ok?
bye