wxDir: A new beginning
Manuel Martin
Yes, you're right. We've been speaking about different things.
Some misunderstanding, mistakes, unexplained and even programming-slang
lacks on my own have derived to my last angry-not-smile post :-(
I promise not to be aggresive.
Please, forgive me if this post is a bit long, for I whish no past
errors will come back again.
I hate to get a 'quick solution', because some time later I'll have
to return to it (to correct or reuse) and 'quick solutions' are
allways partial and bugs-full. So I usually try not to care just about
the moment, but asking myself whatever anybody should ever need.
That's why I put all those 7 parms ...
As I do believe this work (wxDir extended) is really worth, I propose
to begin [almost] from scratch:
1-. What do we actually have?
wxDir is a stop/continue way for enumerating specified files/dirs in
a determinated directory.
2-. What would we need? some ideas:
-traverse subdirs
-stop/continue style. This covers 'stop at first match' feature
-filtering files/dirs to search for: "*.ext" or "any except *.ext" in
'filespec'. User may pass a flag wxDIR_MATCH/wxDIR_MATCH_EXCEPT. Or
user may pass an wxArrayString with several filespecs to use
Filtering on file attributes
Filtering on the contents of the file
-filtering the subdirs to search-in. Same criteria as previus lines
-stop/continue on error
-user must be able to know where we are, what errors have been found
and any relevant information
-storing names and sorting them should be implemented by user in his
own GetNext() iterator. The order we provide is the order OS give us
3-. What will the user use?
-a way to set filters and flags, perphaps in wxDir ctor
-wxDir:GetFirst() and wxDir::GetNext()
-a way to retrieve information wxDir::GetStatus()
4-. How do we implement wxDir?
-To keep backwards compatibily, overload wxDir ctor to acept these
new features, specialy filtering subdirs
-Add more flags: wxDIR_MATCH, wxDIR_MATCH_EXCEPT, wxDIR_STOP_ON_ERROR,
wxDIR_TRAVERSE_SUBDIRS
-Overload GetFirst(filename, filespec=default, flags=default) with
GetFirst(filename, array of filespec, flags=default)
The 2nd way needs explicit complex filter
-We need a private Traverser()
5-. About Traverser()
Recursively calling a search function is IMHO a waste of run-time
if we call it each time user call GetNext().
Better than this, Traverser() stores the last dir (full name) searched.
So, calling OS API (warped) FindNext() works on the right dir.
When FindNext() returns 'no more entries' (FALSE), Traverser() skips
last dir from full name (i.e. "initialdir/subdir1/subdir21" to
"initialdir/subdir1"), calls FindFirst() with wxDIR_DIRS flag,
iterates calling FindNext() until subdir21 is found, calls FindNext()
again to get the new subdir, discard it or not depending on filter
and error (perphaps with ::wxMatchWild(), wxDir::IsOpened() and
wxDir::Exist()) adds it to the last-dir-name member
("initialdir/subdir1/subdir22") and does the search in this dir.
Adding/skipping subdir's name to last-dir-name member we can travel
the whole tree.
This way OS goes over the same dir several times, looking for the
last dir, but recursion would go over more dirs.
Recursion is cool only when getting [filtered] subdirs at once, not
stopping and re-init later on using GetNext()
6-. About GetStatus()
This would be like 'WXDIR_STATUS GetStatus()' where
struct
{
int lastError; //flags as wxDIR_NOT_FOUND, wxDIR_NOT_ACCESS, etc
wxString lastDir; //full name of last currently scanned dir
size_t numFounds;
size_t numDirsScanned;
}WXDIR_STATUS;
I wait for any comments and suggestions.
Regards
Manolo
Vadim Zeitlin
MM> 1-. What do we actually have?
MM> wxDir is a stop/continue way for enumerating specified files/dirs in
MM> a determinated directory.
Or, from a more abstract view point, wxDir wraps a directory object which
already exists in the filesystem.
MM> 2-. What would we need? some ideas:
MM> -traverse subdirs
MM> -stop/continue style. This covers 'stop at first match' feature
MM> -filtering files/dirs to search for: "*.ext" or "any except *.ext" in
MM> 'filespec'. User may pass a flag wxDIR_MATCH/wxDIR_MATCH_EXCEPT. Or
MM> user may pass an wxArrayString with several filespecs to use
This seems too complicated...
MM> Filtering on file attributes
MM> Filtering on the contents of the file
This surely shouldn't be done in the library. Library code should be small
but allow all this and so it's enough to just let the user have filenames in
turn - all the rest is user code's job.
MM> 3-. What will the user use?
MM> -a way to set filters and flags, perphaps in wxDir ctor
MM> -wxDir:GetFirst() and wxDir::GetNext()
No, not in wxDir ctor. You may want to enumerate 2 different kinds of files
in the same dir, for example. So the params should be given to GetFirst(),
just as it is done now.
MM> -a way to retrieve information wxDir::GetStatus()
This function is not independent of GetFirst()/GetNext(), so it doesn't make
a sense to have it separately - this causes all sorts of problems both in the
library (which has to store this info then) and in the user code (which risks
to call it before GetFirst() or after last GetNext() or ...).
Instead, just return any additional info from GetFirst() and GetNext().
MM> 4-. How do we implement wxDir?
Good question. The visitor pattern (what I initially described) is very easy
to implement. If you insist on being able to use GetFirst/Next() to traverse
the files recursively, it gets more complicated because you need to store the
information inside wxDir itself.
This, in turn, is bad because it means you can't have 2 iterations
simulatenously and doing it will result in some very weird errors. Just
imagine:
void DoFooInThisDir(const wxDir& dir)
{
wxString filename;
// find all files.foo in this dir and process them
bool ok = dir.GetFirst(&filename, "*.foo");
while ( ok )
{
ProcessFoo(dir, filename);
ok = dir.GetNext(&filename);
}
}
void ProcessFoo(const wxDir& dir, const wxString& filename)
{
wxString filenameBar;
// if there is a bar near foo, we need to do something special
if ( dir.GetFirst(&filenameBar, "*.bar" )
...
else
...
}
Do you see the problem? Would it be easy to see it in a real program?
MM> -We need a private Traverser()
MM>
MM> 5-. About Traverser()
MM> Recursively calling a search function is IMHO a waste of run-time
MM> if we call it each time user call GetNext().
MM> Better than this, Traverser() stores the last dir (full name) searched.
This is not ideal because of the above. This is why having a callback is
better: like this we don't store anything in wxDir itself and we can have many
simulatenous iterations.
MM> When FindNext() returns 'no more entries' (FALSE), Traverser() skips
MM> last dir from full name (i.e. "initialdir/subdir1/subdir21" to
MM> "initialdir/subdir1"), calls FindFirst() with wxDIR_DIRS flag,
MM> iterates calling FindNext() until subdir21 is found, calls FindNext()
MM> again to get the new subdir, discard it or not depending on filter
MM> and error (perphaps with ::wxMatchWild(), wxDir::IsOpened() and
MM> wxDir::Exist()) adds it to the last-dir-name member
MM> ("initialdir/subdir1/subdir22") and does the search in this dir.
MM> Adding/skipping subdir's name to last-dir-name member we can travel
MM> the whole tree.
Besides, this is much more complicated than implementing the initial proposal
which was already implemented - it was just your code except that you had to
replace _one_ line in it: instead of adding a filename to array, you should
call a callback.
MM> I wait for any comments and suggestions.
I don't see any advantage of this method but I see two problems: non
reentrancy and increased complexity. Just why exactly are you unhappy with
my approach (well, except that it is mine :-)?
Regards,
VZ
Manuel Martin
Even I thought about a horrible way a user could ask wxDir something
like:
'SELECT (files matching "*.ex1" AND "*.ex2" IN "??.sd1" BUT "?a.sb1"
WHERE files_date>mmddyy) or files matching "*.ex3"'
Now light came back. As trivial as:
wxDir::GetFirst(filename, filespec, flags) would mean:
'search for the first file matching filespec.
If (flags & wxDIR_SUBDIRS) and not found in main_dir then search in
all subdirs until finding one.
Then return its fullname in filename'
The traverser subdirs function may be recursive, stopping at first
found. IMO the filename returned should be fullname, so the user can
know where it was found and where next GetNext() will work. But only
if wxDIR_SUBDIRS flag (a new flag), because backwards compatibily of
previus versions of wxDir.
VZ>Just why exactly are you unhappy with my approach
VZ>(well, except that it is mine :-)?
Be sure (VZ) it _doesn't_ matter to me who is approach or code's author.
I just want wxWindows to grow. I find it excellent.
BTW, I read on VC++ help about freeing the data before the dir may be
deleted (i.e. before wxRemoveFile, wxRmdir). I think this should be
advised in the docs.
Regards
Manolo