Reporting and diagnosing hangs in the developer tools

[Rick Byers]

Hi,

I've recently started helping with a project that has apparently been having troubles with the reliability of Chrome developer tools for a long time (now in Chrome 12 dev channel, but also in previous stable builds).  Mostly they just kill the tab and start debugging over when they run into problems with the dev tools, but I'm finding this a real drag on my productivity - and being on Chromium I'd like to help actually fix these issues to improve the reliability.  

Most of the problem I've seen are with the dev tools tab hanging in some sort of CPU-bound loop (eventually get the wait/kill tab).  We've got an reproducible example right now (on top of a google internal app), but it's not at all obvious to me how to go about diagnosing or reporting such an issue (especially when the problem is intermittent or hard to reproduce).  Are such CPU-bound loops in the dev tools likely to be looping in JavaScript somewhere or in the native code?  Is there a way to attach another javascript debugger instance to the dev tools process?  Should I just attach GDB and try to see what the loop is around?  Assuming there isn't an easy repro - what details can I capture in a bug report to increase the chance it'll get attention and be diagnosable?  Are process dumps useful?  Any logging I can turn on and capture?

If someone (inside google) would like to look at the specific repro we've got now - please let me know (perhaps there's just one underlying bug we're hitting with our use of the console or something).  But since this has been a long-standing problem, mostly with hard-to-reproduce issues, I'm most interested in coming up with a process to follow whenever we hit this sort of issue.

Thanks!

   Rick

[Pavel Feldman]

Hi Nathan,

Sure, lets nail it down. Few hints for you:

- You can open DevTools on DevTools and run JavaScript profiler / Timeline panel (Ctrl+Shift+I in undocked DevTools window)

- If you'd like to set breakpoints in the DevTools on DevTools, best way is to run Chrome with --remote-debugging-port=9222 and open http://localhost:9222 in another Chrome instance (that would make DevTools front-end open as a regular page). You will then open DevTools for that page and debug it reliably

- I you'd like to modify the code / debug / console.log it, you could download latest continuous build, run it with --debug-devtools-frontend=<PATH_TO_EXTRACTED_DEVTOOLS_FRONTEND.ZIP>. You can find this devtools_frontend.zip beside the chrome.zip/dmg in the continuous build folder and extract it. It is just a large web app. There is no need to restart Chrome while working with this flag, reopening DevTools will reload it from disk.

- You can send me the link to your internal app with the steps

Regards

Pavel

[Nathan Bullock]

Adding Rick back into this thread.

[Pavel Feldman]

Hm, I did click reply-to-all and it left me with Nathan and groups.. Probably the @chromium.org glitch. Anyways, moving everyone to @google.com for internal app scenario. Rick, see message below.

Thanks

Pavel

[Rick Byers]

Thanks Pavel,

I'm glad to hear JS debugging of the dev tools is so simple - I didn't even think to try the keyboard shortcut, thanks!

The particular problem we're seeing now (and perhaps ALL the problems we've had with hanging) is definitely related to large amounts of console logging.  Debugging the dev tools and hitting pause when it appears hung, I get this callstack:

WebInspector.ResourceTreeModel.addConsoleMessage() at DevTools.js:9598

WebInspector.ConsoleView.addMessage() at DevTools.js:6178

dispatcher.messageAdded() at DevTools.js:6022

InspectorBackendStub.dispatch() at DevTools.js:3902

If I try any execution control (eg. step in), it gets stuck running (clicking pause greys out the pause button but never actually succeeds in pausing the dev tools again).  Perhaps there's another bug there somewhere?  But anyway this was enough for me.

In this particular case I believe the app we're trying to debug is (incorrectly) stuck in a loop that's outputting a log message every so often (not sure of the exact rate, but it's probably not that high).  The dev tools appear to have non-linear performance characteristics with respect to the number of log messages.  I can repro this with this simple app:

<html>

Log loop

<script>

var i = 0;

var last = new Date().getTime();

function sendlogmsg() 

{

        var newTime = new Date().getTime();

        console.log ('Log message #' + i + ', Delta ms=' + (newTime - last));

        i++;

        last = newTime;

        if (i < 5000)

                window.setTimeout(sendlogmsg, 0);

}

window.setTimeout(sendlogmsg, 0);

</script>

</html>

The first 1500 log messages or so output fairly quickly (probably at the 200/sec rate they are getting generated at), then it hangs for a few seconds - repainting around 2500 and later 3000 messages, then hanging for many seconds until all 5000 are finally handled then the UI updates and becomes responsive again.  It makes no difference if the messages are actually displayed (eg. if I tell dev tools to show me only errors), and it looks like the original app isn't being throttled at all (when I can finally view the full log - I see 4-5ms between each log message).

Should I file a Chromium or WebKit bug for this?  Obviously we can work around this for now by disabling/throttling the logging in our app, but that's a less-than-ideal solution for the debug flavour of our app (logging has been very important for our diagnostics during development - we'd probably want to build some infrastructure to save all the messages and just let us select windows to send over to the JS console).

Some ideas:

• Hopefully there's just some algorithm that's being used that's inefficient on sizes >1000 or so - hopefully something can be changed to just give us a higher ceiling with reasonable perf.
• Regardless, perhaps their is a priority issue here - should a flood of log entries really block the devtools UI from being updated / responding to clicks completely?
• Worst case, perhaps there should be some flood protection - just disable console logging (with a warning) before things get terrible

Thanks,

   Rick