Scanning through Directory

[Michael Peuser]

There is a portable way of reading directories
os.listdir
in conjunction with
os.stat and a little help from os.path.split and os.path.join
Though using os.stat gives programs a somewhat clumbsy look,
I had not been too annoyed with it until I tried to scan a whole disk.
Reworking the join and split operations already gave a considerable
speed-up

As I need it for Windows I then with very little effort changed to:
win32api.FindFiles
(I already had used win32api.GetDiskFreeSpace)

This was a breakthrough: 5 secs instead of 50+ !

os.listdir seems to do little more than win32api FindFiles
How can it be so slow???
Why is there no portable way of getting "DiskFreeSpace" info
Questions and Questions....

I strongly recommend the win32api calls it you have similar applications.
The interface is extremely simple and the only penalty is non-compatibility
;-)

I also recommend to use the Python profiler from time to time for programms
running more than one second. It might give you interesting insights...

Kindly
Michael Peuser

[Anand Pillai]

Did you try os.path.walk() in your experiments?
If you did, you could post the profile data for it too.
I find it one of the most useful functions defined in
the os.path module.

Thanks!

Anand

"Michael Peuser" <mpe...@web.de> wrote in message news:<bhalk1$qlv$06$1...@news.t-online.com>...

[Michael Peuser]

Hi Anand,

os.path.walk is just a conveniance function around listdir, although a
*very* conveniant one where you can write scanning programs quite readable.

The timing on Windows 2000 is somewhat complicated, as there seems to be
some OS internal caching, so effective reading of the drive is somtimes
suppressed. Even the drive internal cache might influence the situation:

I can give you the following benchmark:
40 GB 2'' Laptop disk, 8 GB in 120.000 files

win32.FindFiles: initial: 43 s follow-up: 3,5s
walk or listdir: initial: 85 s follow-up: 42 s

This means that in one-shot cases the speed-up (2x) does not look so
tremendeous,
it will increase with faster disks (and decrease with faster processors)
However, the potential speed-up in repeated situations is tremendious.
My initial question, how this could be possible is still valid, because the
implementation of listdir will use the API function FindFiles eventually....

Kindly
Michael P

"Anand Pillai" <pyth...@Hotpop.com> schrieb im Newsbeitrag
news:84fc4588.0308...@posting.google.com...