Evergreen locking up for ages on accidentally broad Find in Files searches
Martin Dorey
It's quite easy to inadvertently kick off an Evergreen search for a single character.
The real-case log that John sent seems to have been eaten by milkman googlegroups. There's one from a reproduction of a similar case, below.
If you're unfortunate enough to kick of a search for "e" at the root of the BlueArc "Unicorn-1" tree then, even if that's your only work space, it turns into a more severe problem than just being unavailable for a while. Evergreen will become increasingly unusable, eventually starting to throw out-of-memory related exceptions after some minutes and having to be kill -9()d.
Somewhat to my surprise, the patch below seems to fix the case where the user dismisses the dialog. The Cancel case is necessary for Escape, the Accept one for Close. The patch looks a bit hairy to my eye. Would anyone care to suggest improvements, incremental or wholesale? I'm hoping this is a step in the right direction, in keeping with the existing currentSequenceNumber early returns. Clearly, though, it would be better if Evergreen didn't kill itself even if the dialog is left open.
--- /home/martind/software.jessies.org/work/evergreen/src/e/edit/FindInFilesDialog.java (revision 1714)
+++ /home/martind/software.jessies.org/work/evergreen/src/e/edit/FindInFilesDialog.java (working copy)
@@ -543,6 +543,17 @@
FindInFilesDialog.initForm
}
});
form.getFormDialog().setExtraButton(rescanButton);
+ form.getFormDialog().setCancelRunnable(new
Runnable() {
+ public
void run() {
+ currentSequenceNumber.incrementAndGet();
+ }
+ });
+ form.getFormDialog().setAcceptCallable(new
java.util.concurrent.Callable<Boolean>() {
+ public
Boolean call() {
+ currentSequenceNumber.incrementAndGet();
+ return
true;
+ }
+ });
}
private
void initSaveMonitor() {
2010-09-10T15:20:05.70-0700 Evergreen: (hang #1203) event dispatch thread unstuck after 2.31 s.
2010-09-10T15:20:05.352-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/atlas/binding/generated/new/atlasC.cpp" for "e" took 11.97 s!
2010-09-10T15:20:08.232-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/atlas/common/files/FileSystemProvider.wsdl" for "e" took 2.05 s!
2010-09-10T15:20:08.429-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/atlas/binding/generated/new/BAService.h" for "e" took 3.18 s!
2010-09-10T15:20:08.595-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/atlas/common/files/FileSystemProviderDefinitions.xsd" for "e" took 2.20 s!
2010-09-10T15:20:13.922-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/calls/DmsDevType.cpp" for "e" took 2.23 s!
2010-09-10T15:20:13.924-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/calls/DevMatchId.h" for "e" took 2.23 s!
2010-09-10T15:20:13.924-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/calls/DevMatchSender.h" for "e" took 2.23 s!
2010-09-10T15:20:13.925-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/calls/DevMatchSender.cpp" for "e" took 2.23 s!
2010-09-10T15:20:16.921-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/calls/EthernetIntfStats.ostream.cpp" for "e" took 2.34 s!
2010-09-10T15:20:17.13-0700 Evergreen: Searching file "/home/johnh/tiger2050/libs/management/atlas/server/fsfiles.gen.inc" for "e" took 7.40 s!
2010-09-10T15:20:20.159-0700 Evergreen: (hang #1204) event dispatch thread stuck processing event for 1.16 s so far:
javax.swing.tree.TreePath.getPathCount(TreePath.java:148)
javax.swing.tree.TreePath.isDescendant(TreePath.java:238)
javax.swing.JTree.getExpandedDescendants(JTree.java:1917)
javax.swing.plaf.basic.BasicTreeUI.updateExpandedDescendants(BasicTreeUI.java:1650)
javax.swing.plaf.basic.BasicTreeUI$Handler.treeExpanded(BasicTreeUI.java:3721)
javax.swing.JTree.fireTreeExpanded(JTree.java:2656)
javax.swing.JTree.setExpandedState(JTree.java:3427)
javax.swing.JTree.expandPath(JTree.java:2163)
e.gui.ETree.expandOrCollapsePath(ETree.java:40)
e.gui.ETree.expandOrCollapsePath(ETree.java:26)
e.edit.FindInFilesDialog$FileFinder.process(FindInFilesDialog.java:278)
org.jdesktop.swingworker.SwingWorker$3.run(Unknown Source)
org.jdesktop.swingworker.AccumulativeRunnable.run(Unknown Source)
org.jdesktop.swingworker.SwingWorker$DoSubmitAccumulativeRunnable.run(Unknown Source)
org.jdesktop.swingworker.AccumulativeRunnable.run(Unknown Source)
org.jdesktop.swingworker.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(Unknown Source)
javax.swing.Timer.fireActionPerformed(Timer.java:271)
javax.swing.Timer$DoPostEvent.run(Timer.java:201)
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
e.debug.EventDispatchThreadHangMonitor.dispatchEvent(EventDispatchThreadHangMonitor.java:191)
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
....
2010-09-10T15:24:37.335-0700 Evergreen: (hang #1789) event dispatch thread stuck processing event for 4m18s so far:
java.util.Hashtable.get(Hashtable.java:334)
javax.swing.JTree.getExpandedDescendants(JTree.java:1913)
javax.swing.plaf.basic.BasicTreeUI.updateExpandedDescendants(BasicTreeUI.java:1650)
javax.swing.plaf.basic.BasicTreeUI$Handler.treeExpanded(BasicTreeUI.java:3721)
javax.swing.JTree.fireTreeExpanded(JTree.java:2656)
javax.swing.JTree.setExpandedState(JTree.java:3427)
javax.swing.JTree.expandPath(JTree.java:2163)
e.gui.ETree.expandOrCollapsePath(ETree.java:40)
e.gui.ETree.expandOrCollapsePath(ETree.java:26)
e.edit.FindInFilesDialog$FileFinder.process(FindInFilesDialog.java:278)
org.jdesktop.swingworker.SwingWorker$3.run(Unknown Source)
org.jdesktop.swingworker.AccumulativeRunnable.run(Unknown Source)
org.jdesktop.swingworker.SwingWorker$DoSubmitAccumulativeRunnable.run(Unknown Source)
org.jdesktop.swingworker.AccumulativeRunnable.run(Unknown Source)
org.jdesktop.swingworker.SwingWorker$DoSubmitAccumulativeRunnable.actionPerformed(Unknown Source)
javax.swing.Timer.fireActionPerformed(Timer.java:271)
javax.swing.Timer$DoPostEvent.run(Timer.java:201)
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
e.debug.EventDispatchThreadHangMonitor.dispatchEvent(EventDispatchThreadHangMonitor.java:191)
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
2010-09-10T15:24:37.816-0700 Evergreen: (hang #1790) event dispatch thread stuck processing event for 4m18s so far:
e.debug.EventDispatchThreadHangMonitor.postDispatchEvent(EventDispatchThreadHangMonitor.java:227)
e.debug.EventDispatchThreadHangMonitor.dispatchEvent(EventDispatchThreadHangMonitor.java:193)
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
2010-09-10T15:24:37.818-0700 Evergreen: (hang #1790) event dispatch thread unstuck after 4m18s.
Elliott Hughes
> It's quite easy to inadvertently kick off an Evergreen search for a single
> character.
indeed. and once set off, we behave so aggressively that that can be a
problem. it's great when it's actually what you want, but i do find
that it's somewhat self-defeating in that it encourages me to
deliberately type malformed regular expressions that i only fix when
i'm ready. accidentally setting off a search like "peekDouble|" is a
PITA.
i've wondered in the past about treating any regular expression that
matches between characters as an outright mistake. definitely in the
Find in Files context.
i'm also more amenable than usual to adding a hard-coded match limit
here. there just isn't anything useful you can do with 1000 matches in
that UI (except maybe read the summary line; so we might want to just
give up on the tree, which seems to be the most expensive bit anyway).
> The real-case log that John sent seems to have been eaten by milkman
> googlegroups. There's one from a reproduction of a similar case, below.
>
>
>
> If you're unfortunate enough to kick of a search for "e" at the root of the
> BlueArc "Unicorn-1" tree then, even if that's your only work space, it turns
> into a more severe problem than just being unavailable for a while.
> Evergreen will become increasingly unusable, eventually starting to throw
> out-of-memory related exceptions after some minutes and having to be kill
> -9()d.
yeah, that TreeModel's a greedy fucker.
> Somewhat to my surprise, the patch below seems to fix the case where the
> user dismisses the dialog. The Cancel case is necessary for Escape, the
> Accept one for Close. The patch looks a bit hairy to my eye. Would anyone
> care to suggest improvements, incremental or wholesale? I'm hoping this is
> a step in the right direction, in keeping with the existing
> currentSequenceNumber early returns.
seems reasonable. i've used more intention-revealing names, and i've
added the missing check for out-of-date work to DefinitionFinder, but
basically committed this.
> Clearly, though, it would be better if
> Evergreen didn't kill itself even if the dialog is left open.
indeed. especially because, when you bring the dialog back up, it'll
re-start the same search.
--elliott
> --
> You received this message because you are subscribed to the Google Groups
> "evergreen-users" group.
> To post to this group, send email to evergre...@googlegroups.com.
> To unsubscribe from this group, send email to
> evergreen-use...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/evergreen-users?hl=en.
>
--
Elliott Hughes - http://www.jessies.org/~enh/
Martin Dorey
> expressions that i only fix when
> i'm ready
But that bites me more often, and harder, when searching within a non-source file, which I only do occasionally, but where a hard-coded limit would be more controversial. I'm hopeful that your commit will take care of John's pretty frequent bouts of non-responsive Open "Quickly".
Thanks,
Elliott Hughes
>> deliberately type malformed regular
>> expressions that i only fix when
>> i'm ready
>
> I've been tempted to change the code after the event to pre-populate the field with a close parenthesis or some such, or to make the delay before kicking off the search inversely related to the length of the regex,
that's an interesting idea i hadn't thought of, but like i say: one of
my biggest problems is when i accidentally have a subexpression that
matches the gaps between characters. i keep thinking of a preflight
where i test the regex against some constant and detect that specific
case and ignore it (probably with a warning in the UI so we don't just
look like we're broken in cases where the user doesn't realize what
they've done). i can't see any use for a Find in Files or Find with a
pattern like that. Find/Replace is probably the only place where you
might intentionally do that.
> or to do something to make it use less mill after 1000 matches. "I've typed some more letters, stop highlighting 'e' and pay attention to my typing!".
now you're talking about Find rather than Find in Files, right? that's
a related concept, but completely different code. i think Find in
Files (parallel, JTree, ctags), Find/Replace (serial, JList), and Find
(serial, PTextArea) probably all have quite different time/space
characteristics, and probably sufficiently different use cases to need
separate treatment. tbh, Find in Files is the one i have most trouble
with. i can't honestly remember any recent trouble with Find, perhaps
because i mostly find by example (ctrl-f with something already
selected). plus i work exclusively on source; a lot of the complaints
i hear from you relate to large log files. as for Find/Replace i don't
think i use it often enough to have trouble. i do occasionally wish i
could see a diff rather than the two lists, but other than that it
causes me no trouble.
> But that bites me more often, and harder, when searching within a non-source file, which I only do occasionally, but where a hard-coded limit would be more controversial. I'm hopeful that your commit will take care of John's pretty frequent bouts of non-responsive Open "Quickly".
when i've thought about this in the past, i've imagined something like
max(CONSTANT, FUNCTION(document.length())). but arguably that just
makes it twice as hard to arrive at the right heuristic ;-)
Find in Files is probably easier because of the UI limitation. you
just can't do anything reasonable with (say) 1000 matches in that UI.
even there, though, i probably want to see a sample of the matches. i
find that very helpful in realizing why my regex is wrong when it's
not the trivial case of "Evergreen jumped the gun". (having a
hard-coded list of common open source license texts and warning "this
regex matches APL2" would catch a lot of my accidental
match-every-file cases ;-) )
--elliott