Read directory in timestamp order?

[Arthur Cohen]

Well, the subject says it all. Is there a way to read a directory in
timestamp order (preferably newest files last)? Or do I need to read the
whole directory into an array, perform a (-M ) on each file and then sort?

Any suggestions?

--Art
(email appreciated)

[Mike Heins]

Arthur Cohen (upse...@cybercom.net) wrote:
: Well, the subject says it all. Is there a way to read a directory in

: timestamp order (preferably newest files last)? Or do I need to read the
: whole directory into an array, perform a (-M ) on each file and then sort?

:

Some program is going to have to do it. It can be as simple as:

$dir = shift || $ENV{HOME};
@dir_time = `ls -rt $dir`;
chop @dir_time;

This offloads the work to the ls program, but it is still going
to do just what you said. You could also do it yourself with
readdir(), grep(), and sort().

--
Regards,
Mike Heins [mailed and posted] http://www.iac.net/~mikeh ___ ___
Internet Robotics |_ _|____ |_ _|
Few blame themselves until they 131 Willow Lane, Floor 2 | || _ \ | |
have exhausted all other Oxford, OH 45056 | || |_) || |
possibilities. |___| _ <|___|
-- anonymous mi...@iac.net 513.523.5028 |_| \_\

[Michael J. Stok]

In article <4l0h8p$l...@orion.cybercom.net>,

Arthur Cohen <upse...@cybercom.net> wrote:
>Well, the subject says it all. Is there a way to read a directory in
>timestamp order (preferably newest files last)? Or do I need to read the
>whole directory into an array, perform a (-M ) on each file and then sort?
>

>Any suggestions?

opendir DIR, '.' or die "$0: can't open . ($!)\n";
@fileList = sort {-M $a <=> -M $b || $a cmp $b} readdir DIR;
closedir DIR;

Hope this helps,

Mike
--
Mike Stok | The "`Stok' disclaimers" apply.
st...@pencom.com | Pencom Systems Administration (work)
st...@cybercom.net | Cyber Access (play)
http://www.cybercom.net/~stok/ | The inevitable WWW page (?)

[Michael J. Stok]

In article <4l2kcu$j...@kalypso.cybercom.net>,

Michael J. Stok <st...@cybercom.net> wrote:
>In article <4l0h8p$l...@orion.cybercom.net>,
>Arthur Cohen <upse...@cybercom.net> wrote:
>>Well, the subject says it all. Is there a way to read a directory in
>>timestamp order (preferably newest files last)? Or do I need to read the
>>whole directory into an array, perform a (-M ) on each file and then sort?
>>
>>Any suggestions?
>
>opendir DIR, '.' or die "$0: can't open . ($!)\n";
>@fileList = sort {-M $a <=> -M $b || $a cmp $b} readdir DIR;
>closedir DIR;

Oops, "preferably newest files last" suggests that the sort code should
be {-M $b <=> -M $a || $a cmp $b}

Also beware that the things returned by readdir are just file names, so
if you're reading a directory that isn't '.' then you may need to prepend
a path to $a and $b before testing them e.g.

$dir = 'some/directory';
opendir (DIR, $dir) || die "$0: can't open $dir ($!)\n";
@fileList = sort {-M "$dir/$b" <=> -M "$dir/$a" || $a cmp $b} readdir DIR;
closedir DIR;

Sorry for following up my own post...

[Tom Christiansen]

[courtesy cc of this posting sent to cited author via email]

In comp.lang.perl.misc,
st...@cybercom.net writes:
:In article <4l2kcu$j...@kalypso.cybercom.net>,

:Michael J. Stok <st...@cybercom.net> wrote:
:>In article <4l0h8p$l...@orion.cybercom.net>,
:>Arthur Cohen <upse...@cybercom.net> wrote:
:>>Well, the subject says it all. Is there a way to read a directory in
:>>timestamp order (preferably newest files last)? Or do I need to read the
:>>whole directory into an array, perform a (-M ) on each file and then sort?
:>>
:>>Any suggestions?
:>
:>opendir DIR, '.' or die "$0: can't open . ($!)\n";
:>@fileList = sort {-M $a <=> -M $b || $a cmp $b} readdir DIR;
:>closedir DIR;
:
:Oops, "preferably newest files last" suggests that the sort code should
:be {-M $b <=> -M $a || $a cmp $b}
:
:Also beware that the things returned by readdir are just file names, so
:if you're reading a directory that isn't '.' then you may need to prepend
:a path to $a and $b before testing them e.g.

CODE 0:

:$dir = 'some/directory';

:opendir (DIR, $dir) || die "$0: can't open $dir ($!)\n";
:@fileList = sort {-M "$dir/$b" <=> -M "$dir/$a" || $a cmp $b} readdir DIR;
:closedir DIR;

Um... we're back to pessimal sorting here again, Mike. While your
approach *works*, it's darned slow. Anytime you put anything in the
sort{} closure, you run the risk of driving up your performance though
the roof. Remember that your routine will be called N * log(N) [base
e] times, and you have to double that to find the number of -M's, for
example. Plus there's the string concat.

Data Size Computations of -M

10 46
100 921
1000 13815
10000 184206

That means that on 100 directory entries, you're doing 821 extra calls,
and on 10,000 entries, you're doing 174,206 extra calls!!

That's pretty wicked.

Here's something that scales a good bit better.

CODE 1:

$dir = shift || '/etc';

opendir (DIR, $dir) || die "$0: can't open $dir ($!)\n";

for (@files = readdir DIR) {
$age{$_} = -M "$dir/$_";
}
@files = sort { $age{$b} <=> $age{$a} || $a cmp $b} @files;
closedir DIR;

Contrary to folklore, it doesn't run faster with a Schwartzian
transform, although it may do something to preserve (or imperil) job
security -- not to mention one's lisp skills:

CODE 2:

$dir = shift || '/etc';

opendir (DIR, $dir) || die "$0: can't open $dir ($!)\n";

@files = map { $_->[0] } sort {
$b->[1] <=> $a->[1] || $a->[0] cmp $b->[0]
} map { [ $_ => -M "$dir/$_" ] } readdir DIR;

A much strategy is to factor out the string concatenation *AND*
the number of times your kernel has to run through namei(),
lookuppn(), or whatever it's called these days.

CODE 3:

$dir = shift || '/etc';
chdir $dir or die "$0: can't chdir $dir ($!)\n";
opendir (DIR, '.') || die "$0: can't opendir $dir ($!)\n";
for (@files = readdir DIR) {
$age{$_} = -M;
}
@files = sort { $age{$b} <=> $age{$a} || $a cmp $b} @files;
closedir DIR;

In fact, this is nearly as fast as the call-the-shell approach:

CODE 4:

$dir = shift || '/etc';
chomp (@files = `ls -at $dir`);

So what does it all mean? Here are timings in user and system time.
Here, ~ means "/homes/tchrist", and was expanded to the same. Watch
the system time in particular. For the little cases, it doesn't matter,
but very quickly it does. (Timings under a second or so should be looked
upon with a fair dose of mistrust.)

Path / /etc ~ ~/scripts
Entries 41 122 236 509

Code 0: The Stok Original .17u .40s .48u 1.14s 1.43u 7.03s 2.21u 13.33s
Code 1: Sort the Indices .10u .13s .20u .21s .43u .70s .64u 1.09s
Code 2: Schwartzian Xform .13u .13s .23u .26s .47u .75s .70u 1.09s
Code 3: Code 1 w/ chdir .07u .17s .17u .29s .43u .61s .72u .81s
Code 4: Backtick `ls` .07u .23s .09u .28s .22u .83s .25u 1.08s

Path ~/Mail/inbox/personal ~/Mail/outbox
Entries 1760 files 18138 files

Code 0: The Stok Original 8.94u 119.19s <KILLED @ 20 mins>
Code 1: Sort the Indices 2.55u 4.03s 45.89u 43.93s
Code 2: Schwartzian Xform 3.09u 4.53s 43.13u 45.33s
Code 3: Code 1 w/ chdir 2.76u 3.10s 45.87u 31.45s
Code 4: Backtick `ls` .78u 4.30s 6.87u 45.00s

See http://www.perl.com/perl/nmanual/pod/perlfunc/sort.html for other
examples.

--tom
--
Tom Christiansen Perl Consultant, Gamer, Hiker tch...@mox.perl.com
"Espousing the eponymous /cgi-bin/perl.exe?FMH.pl execution model is like
reading a suicide note -- three days too late."
--Tom Christiansen <tch...@mox.perl.com>