File Search functionality

[Johann Schmitz]

Hi all,

i've started working on a file search for muc, and hav a few questions about the right/good way to implement this.

Currently there are the following new classes:

com.mucommander.file

FileFinder:
A class to do the searching stuff. It has two public static methods:

public static void findFiles(AbstractFile startDirectory, FileFilter filter, FileFoundListener listener);
and
public static void findFiles(AbstractFile startDirectory, Collection fileFilters, FileFoundListener listener);

It uses exactly one (or a collection of) com.mucommander.file.filter.*. Currently only the
RegexpFilenameFilter has been tested, but the others should work too.

FileFoundLister:
An interface with a single method wich get called if a AbstractFile has passed one of the used filters.

com.mucommander.ui.main.menu.MainMenuBar
New entry in the "file" menu for the search dialog

com.mucommander.ui.action.OpenSearchDialogAction:
An Action to open the SearchDialog (Is it really necessary to create new class for just calling new
SearchDialog().showDialog()?)

com.mucommanderui.dialog.SearchDialog
The search dialog. Up to now the GUI stuff is crappy netbeans generated code - i was too lazy yesterday to
write this by hand ;)
The SearchDialog has two inner classes: The search thread and a ListModel. Both should be moved to an own
class (at least the Thread).
Is there a common way in mucommander to implement a background job? Yes, ive seen the com.mucommander.job
thing, but it seems that all of these jobs are using a ProgressDialog?

The "feature plan":
- "Standard" file search: Search all files containing the search phrase in there path or filename (uses a
RegexpFilenameFilter with ".*" + searchPhrase + ".*". This is a bit dangerous, since a user may enter
blah.zip, and the RegexpFilenameFilter would interpret the dot as "any char".
- RegEx search: Use the regex entered Regex and the RegexpFilenameFilter.
- (later) search in selected dirs (is there a way to get all selected elements in a Panel?)
- (later) search in archives
- (later) search files containing <text>
- (more later) filter files according there size/date

Any suggestions/feedback/wishes for the implementation?

Johann

--
Johann Schmitz
http://www.j-schmitz.net

[A.Yerenkow]

On 17.03.2009 7:38, Johann Schmitz wrote:
> Hi all,
>
> i've started working on a file search for muc, and hav a few questions about the right/good way to implement this.
>
> Currently there are the following new classes:
>
> com.mucommander.file
>
> FileFinder:
> A class to do the searching stuff. It has two public static methods:
>
> public static void findFiles(AbstractFile startDirectory, FileFilter filter, FileFoundListener listener);
> and
> public static void findFiles(AbstractFile startDirectory, Collection fileFilters, FileFoundListener listener);
>
>

And, how about find in found files? Maybe something like that could be :

public static void findFiles(AbstractFileList oldSearch, Collection fileFilters, FileFoundListener listener);

And, if you do that, there will be possibility to "Save Search" result.

[Johann Schmitz]

A.Yerenkow wrote:
> And, how about find in found files? Maybe something like that could be :
>
> public static void findFiles(AbstractFileList oldSearch, Collection fileFilters, FileFoundListener listener);

For "find string x" i have a FileContentFileFilter (extending com.mucommander.file.filter.FileFilter) in mind,
since multiple Filters are combined with "and".

> And, if you do that, there will be possibility to "Save Search" result.

Hm. You mean something like the virtual folders feature of IMAP-Mailboxes?

[A.Yerenkow]

On 17.03.2009 12:53, Johann Schmitz wrote:
> A.Yerenkow wrote:
>
>> And, how about find in found files? Maybe something like that could be :
>>
>> public static void findFiles(AbstractFileList oldSearch, Collection fileFilters, FileFoundListener listener);
>>
> For "find string x" i have a FileContentFileFilter (extending com.mucommander.file.filter.FileFilter) in mind,
> since multiple Filters are combined with "and".
>
>

I mean, I have some criteria in mind, like "all files starts with log* "
and I want search in dir, which have hundreds of thousands files;
I ran search (which could took few minutes, or more), and there are
hundreds of them (search result).
I think, "ola-la, I should narrow my query", and I specify that filen
should have something in content, or date match my criteria, or else.
But, I don't want whole search start over again, I want search run
through current results list.
That what I mean, saying "find in found files" :)
More, I ran very specific search, on whole PC (for example), found what
I need, but if I could'nt save this list in some format - that's not so
good.
So, I should be able to "save search result" in some way (even in plain
txt file with pathes in each line).
And have some way (like button) to quick load any of my saved results,
without triggering whole search-match thing against filesystem.

That for I think

AbstractFileList should fit :)

Anyway, that's only recommendation, you're the boss ;)

[A.Yerenkow]

Hey guys, today I had a few free minutes, to debug muCommander and find
out why is slow Drive list exists.
It exists mainly because of floppy drive, and remote disks.
Here is my solution, which works:

changes in: com.mucommander.file.AbstractFile.java
1326 line
public boolean equals(Object f) {
if(f==null || !(f instanceof AbstractFile))
return false;

return hashCode() == f.hashCode();
}

// Flag, if we already caluclated hashcode
private boolean hashCalced = false;

// Cached Hashcode itself
private int hCode = 0;

/**
* Returns the hashCode of this file's {@link #getURL() URL}.
*
* @return the hashCode of this file's {@link #getURL() URL}.
*/
public int hashCode() {
if(!hashCalced)
{
//not sure if this OK without synchronized
hashCalced = true;
hCode = getCanonicalPath(false).hashCode();
}

return hCode;
}

and in com.mucommander.file.impl.local.LocalFile.java
394 line
public static Vector cachedJavaIoRoots = new Vector();

/**
* Resolves and returns all local volumes:
* <ul>
* <li>On UNIX-based OSes, these are the mount points declared in
<code>/etc/ftab</code>.</li>
* <li>On the Windows platform, these are the drives displayed in
Explorer. Some of the returned volumes may
* correspond to removable drives and thus may not always be
available -- if they aren't, {@link #exists()} will
* return <code>false</code>.</li>
* </ul>
* <p>
* The return list of volumes is purposively not cached so that new
volumes will be returned as soon as they are
* mounted.
* </p>
*
* @return all local volumes
*/
public static AbstractFile[] getVolumes() {
Vector volumesV = new Vector();

// Add Mac OS X's /Volumes subfolders and not file roots ('/')
since Volumes already contains a named link
// (like 'Hard drive' or whatever silly name the user gave his
primary hard disk) to /
if(OsFamilies.MAC_OS_X.isCurrent()) {
addMacOSXVolumes(volumesV);
}
else {
// Add java.io.File's root folders
File[] files = File.listRoots();
boolean needRebuildCachedRoots =
cachedJavaIoRoots.isEmpty() || files.length != cachedJavaIoRoots.size();
if(!needRebuildCachedRoots)
{
//simply checking every exist roots;
for (int i = 0; !needRebuildCachedRoots && i <
files.length; i++)
{
File file = files[i];
boolean found = false;
for (int j = 0; !found && j <
cachedJavaIoRoots.size(); j++)
{
AbstractFile abstractFile = (AbstractFile)
cachedJavaIoRoots.elementAt(j);
if(abstractFile.toString().equals(file.toString()))
{
found = true;
}
}
needRebuildCachedRoots = !found;
}

}

if(needRebuildCachedRoots)
{
cachedJavaIoRoots.clear();
addJavaIoFileRoots(cachedJavaIoRoots);
}

volumesV.addAll(cachedJavaIoRoots);

// Add /etc/fstab folders under UNIX-based systems.
if(OsFamily.getCurrent().isUnixBased())
addFstabEntries(volumesV);
}

// Add home folder, if it is not already present in the list
AbstractFile homeFolder = getUserHome();
if(!(homeFolder==null || volumesV.contains(homeFolder)))
volumesV.add(homeFolder);

AbstractFile volumes[] = new AbstractFile[volumesV.size()];
volumesV.toArray(volumes);

return volumes;
}

[Maxence Bernard]

Hey Alex,
Thanks a lot for looking into this long-standing issue! I'm really
glad you found the root cause for the slowdowns, i.e. calls to
java.io.File#getCanonicalPath().

I will go ahead and modify AbstractFile, though a bit differently from
what you suggested :
- the current #equals implementation will be renamed to
#equalsCanonical -- useful for taking into account case variations on
case-insensitive filesystems (myfile == MYFILE) as well as symbolic
links
- #equals will compare both files' URL for equality -- comparing
absolute paths (myfile != MYFILE) and credentials
=> this will be consistent with the current implementation of
AbstractFile#hashCode which returns the file URL's hash code

With just this modification, the drive popup button appears instantly
under Windows XP, where it took a good second or so before.

I will take a bit more time before committing the change, as I need to
review all existing calls to AbstractFile#equals to determine whether
they are safe or need to be replaced by calls to #equalsCanonical.
I'll let you know as soon as it's in SVN.

Thanks again for your fix!

Cheers,
Maxence

[Maxence Bernard]

Hi Alex,
I just committed the fix I mentionned in my previous email. The drive
button now appears instantly under Windows XP -- drive labels take a
bit more time to appear as they are fetched in a separate thread.
Could you please confirm it's all good on your machine too ?

I added a mention of the fix in the readme file and in :
http://trac.mucommander.com/ticket/122

Thanks again for finding a solution for this annoying issue!

Maxence

[A.Yerenkow]

On 06.06.2009 10:22, Maxence Bernard wrote:
> Hi Alex,
> I just committed the fix I mentionned in my previous email. The drive
> button now appears instantly under Windows XP -- drive labels take a
> bit more time to appear as they are fetched in a separate thread.
> Could you please confirm it's all good on your machine too ?
>
>

Confirm! Drive popup is good and fast, no significant delay noticed :)