Fixing Internet Explorer specific memory leaks (circular references, etc)

1,258 views
Skip to first unread message

Paul McLachlan

unread,
Nov 7, 2010, 3:10:09 AM11/7/10
to Google Web Toolkit
I’d like to chronicle my experiences fixing a memory leak in our
enterprise GWT application when running on Internet Explorer.

A few facts getting started:

1. Using a click-recording tool, QA could get our application to
leak hundreds of megabytes of memory in Internet Explorer.
2. No leak measured using Java memory analysis tools in hosted mode.
3. Non-trivial application - we have over 100K SLOC just for GWT
(ie, not counting server side or any non .java files)

Reproducibility was handled by QA, but the first problem was working
out what was going on. We didn't see these kinds of problems with
Firefox & this strongly implies some kind of Internet Explorer
circular reference strangeness between DOM elements and javascript.

We spent some time playing with Drip and sIEve, but we were advised
not to use these tools (misleading output) and didn't have wonderful
success with them in any case.

A bit more googling found a javascript memory leak analyzer from
Microsoft at: http://blogs.msdn.com/b/gpde/archive/2009/08/03/javascript-memory-leak-detector-v2.aspx
. That's actually the v2 and most google hits send you to a (now
removed v1), so it's a little difficult to find.

In any case, the results of using Microsoft's javascript memory leak
detector are basically unintelligible in a regular production mode GWT
compile. I had more luck when compiling with -style PRETTY
(obviously), and I also turned on -draftCompile. I think I remember
that -draftCompile did less inlining, which made the generated
javascript closer to our Java code. This was important because the
output of the tool is basically a series of leaks, like:

DIV leaked due to onclick from stack XYZ

where "XYZ" is a javascript stack trace of where the onclick event
handler was set. By clicking back up the stack you can generally get
a reasonable idea of which widget in your application is causing the
problem.

At this point, I didn't actually trust the tool - so from a
methodology perspective my first step was to validate the tools
output.

I commented out code and otherwise configured our application down to
a bare-bones "login, display a couple of things, logout" script that I
could run in a loop. Having done so, I could demonstrate that:

a) that operational loop actually leaked in IE
b) the tool reported about 15 elements as being leaked

Then, I proceeded to ... try to "fix" those leaks.

First attempt was to click the ClickListener (or ClickHandler or
KeyPressHandler or whatever). I mean, calling Handler.remove(), or
removeClickListener() during onDetach(). My theory was that my
ClickHandler was a Java inner class and it somehow just had an inline
reference to the parent object and etc.

No joy. The tool still reported it as a leak. I've since spent a lot
more time going through GWT's implementation of event handling and
it's pretty clear that:

a) the intent is to not have to do this; and
b) it doesn't set element.onclick = null

I understand the argument with b) is that you don't have to. I wasn't
feeling very trusting at this point (I had a memory leak tool from
Microsoft that seemed to disagree with that), so I thought I'd test
it.

Reading http://javascript.crockford.com/memory/leak.html gave me an
idea, so I wrote a little helper method:

public native static void purgeEventHooks( Element elem, boolean
recurse ) /*-{
try {
elem.onclick = null;
elem.ondblclick = null;
elem.onmousedown = null;
elem.onmouseup = null;
elem.onmouseover = null;
elem.onmouseout = null;
elem.onmousemove = null;
elem.onkeydown = null;
elem.onkeypress = null;
elem.onkeyup = null;
elem.onchange = null;
elem.onfocus = null;
elem.onblur = null;
elem.onlosecapture = null;
elem.onscroll = null;
elem.onload = null;
elem.onerror = null;
elem.onmousewheel = null;
elem.oncontextmenu = null;
elem.onpaste = null;

if (recurse) {
var a = elem.childNodes;
if (a) {
l = a.length;
for (i = 0; i < l; i += 1) {
purgeEventHooks(elem.childNodes[i], recurse);
}
}
}
} catch( e ) {
// ignore
}
}-*/;


And then proceeded to call it from onDetach() on the "leaked" element.

Aha - magic -- the Microsoft javascript leak tool no longer reports
that element as a leak!

However, it seems quite possible that all I've managed to do is fool
the leak tool, as opposed to actually fix any leak. So I resolve to
fix all of the leaks on this code path. I eventually managed to do
this (gosh, it was painful -- mostly because it's very difficult from
a javascript stacktrace to where onclick was called to work out which
custom GWT widget is ultimately involved).

Now I have the leak tool reporting no leaks. So, I turn it off and
put the application back into a loop in Internet Explorer --- half
expecting to see it still leaking tens of megabytes and being back
where I started having wasted several days. But... it didn't leak.
IE memory usage varied by a few hundred KB (up and down) throughout
the course of a long test, but no memory leaks.


Soooo - now I'm at a point where I trust the leak tool, and I think
there's some kind of really fundamental problem in this theory that
you don't have to clear DOM element event hooks. So I build a "hello
world" type GWT application intending to, you know, prove this to the
world.

No joy. In that application, (which was just a button added and
removed from a SimplePanel on the RootPanel), it wasn't necessary to
clear the DOM element event hooks to have IE clear everything up. So
there's some "more complicated than trivial" case in which this is
triggered. Unfortunately. Also unfortunately, I have no-where near
the kind of time to go back and trial-and-error work out what that
condition is.


So, now I can't explain why clearing the handlers helps, but I can say
that it does. And I am faced with the really daunting task of trying
to go through our app and explicitly clear event handlers onDetach.

Don't really want to do that -- actually, all I really want to do is
have Widget.onDetach() call, basically DOM.unsinkEvents().

Mucking around in the GWT code, I find a hook that will let me do
that. This is a bit... well, 'distasteful'. But, it worked for me
and got us out of a tight spot / will also let us clear up some of the
other mitigation code we've had in place around this issue.

The observation is that DOM.setElementListener is called on attach
with a value, and on detach with null. And it calls into an abstract
DOMImpl method that I can override with deferred binding (.gwt.xml
<replace-with>).

The implementation of DOMImpl that I use looks like this:

public class NoMemLeaksDOMImplIE8 extends
com.google.gwt.user.client.impl.DOMImplIE8 {

public NoMemLeaksDOMImplIE8() {
super();
}

static native void backupEventBits(Element elem) /*-{
elem.__eventBits_apptio = elem.__eventBits;
}-*/;

static native int getEventBitsBackup(Element elem) /*-{
return (elem.__eventBits_apptio || 0);
}-*/;

static void hookEventListenerChange( DOMImpl impl, Element elem,
EventListener listener ) {
if( listener == null ) {
// Called from onDetach() for Widget (among other places).
//
// Basically, we're going to be detached at this point.
// So.... set all event handlers to null to avoid IE
// circular reference memory leaking badness
backupEventBits( elem );
impl.sinkEvents(elem,0);
} else {
int backup = getEventBitsBackup( elem );
if( backup != 0 ) {
impl.sinkEvents( elem, backup );
}
}
}


public void setEventListener( Element elem, EventListener listener)
{
super.setEventListener( elem, listener );
hookEventListenerChange( this, elem, listener );
}
}

Note that I have to back up the event bits & restore them in case the
element is re-attached back into the DOM later (otherwise it won't get
any of the events it is expecting).

In any case, your actual mileage may vary, but this helps us a LOT.

Lessons learned:

1) Microsoft javascript memory leak detection tool rocks. It is
available from here: http://blogs.msdn.com/b/gpde/archive/2009/08/03/javascript-memory-leak-detector-v2.aspx
It has an automation mode where you could put an "zero leaks"
assertion as part of an automated test.

2) There is some way you can set up a GWT app so that you need to
clear event handlers in order to avoid leaks in IE6, IE7 & IE8. No
plain GWT manipulations I found (removing the clicklisteners, etc)
helped at all. When you get into this state, basically every single
widget you have with an event listener will leak. Microsoft's leak
tool isn't lying, but neither can I explain why or what it is that
pushes the app over this threshold.

3) You can successfully hack the DOM Impl to have Widget.onDetach()
clear the event hooks for you, meaning you don't have to refactor all
your code to follow this pattern.

I hope this helps someone.

- Paul

Slava Lovkiy

unread,
Nov 7, 2010, 4:27:33 PM11/7/10
to Google Web Toolkit
Thanks Paul for sharing your experience with resolving memory leaks, I
think it worth a separate blog post.

Few questions:
1. what version of IE did you use during the testing ?
2. was the performance of the app improved after resolving the memory
leaks ?

Best regards,
Slava Lovkiy

On Nov 7, 7:10 pm, Paul McLachlan <pmclach...@gmail.com> wrote:
> I’d like to chronicle my experiences fixing a memory leak in our
> enterprise GWT application when running on Internet Explorer.
>
> A few facts getting started:
>
>   1. Using a click-recording tool, QA could get our application to
> leak hundreds of megabytes of memory in Internet Explorer.
>   2. No leak measured using Java memory analysis tools in hosted mode.
>   3. Non-trivial application - we have over 100K SLOC just for GWT
> (ie, not counting server side or any non .java files)
>
> Reproducibility was handled by QA, but the first problem was working
> out what was going on.  We didn't see these kinds of problems with
> Firefox & this strongly implies some kind of Internet Explorer
> circular reference strangeness between DOM elements and javascript.
>
> We spent some time playing with Drip and sIEve, but we were advised
> not to use these tools (misleading output) and didn't have wonderful
> success with them in any case.
>
> A bit more googling found a javascript memory leak analyzer from
> Microsoft at:http://blogs.msdn.com/b/gpde/archive/2009/08/03/javascript-memory-lea...
> Readinghttp://javascript.crockford.com/memory/leak.htmlgave me an
> available from here:http://blogs.msdn.com/b/gpde/archive/2009/08/03/javascript-memory-lea...

Paul McLachlan

unread,
Nov 8, 2010, 12:51:19 AM11/8/10
to Google Web Toolkit
I was using IE7 after the first few steps. I didn't observe a
difference in behavior between IE6 and IE7. IE8 leaked as well, but I
didn't subject that version to the same science - our customers are
all IE6 & 7.

I didn't observe any difference in performance just from using the
application - no scientific measurements, although the code I added
into onDetach doesn't look terribly expensive.

Cheers,
Paul

chrisr

unread,
Nov 8, 2010, 3:46:01 PM11/8/10
to Google Web Toolkit
Hi Paul, I'm really interested to see if this works for a project I've
been working on, which has a significant memory leak in IE.

I haven't worked with deferred bindings in GWT before, and I don't
think I'm doing it right, as breakpoints i put in my NoLeaksDOMImpl
class never get hit.

I'm assuming that you aren't supposed to open up the gwt-user jar and
edit the Emulation.gwt.xml file inside, so I added

com.google.gwt.emul.Emulation.gwt.xml to my source w/
<module>
<super-source/>
<replace-with
class="com.google.gwt.user.client.impl.NoMemLeaksDOMImplIE6">
<when-type-is class="com.google.gwt.user.client.impl.DOMImplIE6"/>
</replace-with>
</module>

And I also added
com.google.gwt.user.client.impl.NoMemLeaksDOMImpl....java to my
source..

Am I doing something incorrectly w/ the deferred binding configuration?

jay

unread,
Nov 8, 2010, 4:57:29 PM11/8/10
to Google Web Toolkit
Nope... In your project's .gwt.xml file add this:

<!-- Avoid memory leaks with IE -->
<replace-with class="com.yourco.gwt.client.NoMemLeaksDOMImplIE8">
<when-type-is class="com.google.gwt.user.client.impl.DOMImpl"/>
<when-property-is name="user.agent" value="ie8"/>
</replace-with>

<replace-with class="com.yourco.gwt.client.NoMemLeaksDOMImplIE6">
<when-type-is class="com.google.gwt.user.client.impl.DOMImpl"/>
<when-property-is name="user.agent" value="ie6"/>
</replace-with>


jay

chrisr

unread,
Nov 8, 2010, 7:31:25 PM11/8/10
to Google Web Toolkit
Thanks jay, got that sorted out. I can confirm via breakpoints that
in hosted mode (we're on GWT 1.5) the NoMemLeaks implementation is
getting used, but unfortunately it doesn't fix our memory leak in IE.

Is there a simple way to verify that its actually getting used by IE?

Paul McLachlan

unread,
Nov 9, 2010, 12:58:37 AM11/9/10
to Google Web Toolkit
I'd suggest running Microsoft's memory leak tool and working out which
elements, exactly, are leaking in your app and why.

Once you know that you can purge them by hand -- this is how I
started. I only went into this Widget hack because the list of items
I needed to deal with was... really long. :)

That said - QA has a different use-case that still leaks in our app as
well, so... the story is evolving. I'll update the thread if we find
out more in case it's helpful for others.

Best of luck!

Regards,
Paul

Joel Webber

unread,
Nov 15, 2010, 12:45:16 PM11/15/10
to google-we...@googlegroups.com
Paul,

First off, I'd like to applaud your diligent efforts to track down these leaks. It's often very hard in practice, especially with the paucity of good tools and no access to the browser's source.

That Microsoft leak detector is definitely the right one to be using -- Drip (which I wrote ages ago) was always kind of a hack without access to internal data structures, and I no longer trust it at all. One caveat about the MS tool, though -- IIRC it has two "modes", one of which reports "potential" leaks, and which I've found to be extremely misleading on GWT output. I'd double-check to make sure you're using the strict check. Based on the result that your app *actually* stopped leaking (and the fact that it was leaking in the first place), I'm guessing this isn't the problem, but I thought I'd mention it for completeness.

As far as getting a GWT app to leak is concerned, you are correct that it was carefully designed such that you should never have to remove the event handlers explicitly. We're quite certain the theory is sound, and we've confirmed it on apps up to the complexity of Wave and AdWords. But you *did* see leaks in your app, so how could that happen? The most obvious cases I've seen are:
- Using libraries that wrap external Javascript libraries
  - Either because said library isn't careful enough or because there's some strange interaction making things worse.
- Writing JSNI methods that hook events by hand in Javascript, but not cleaning up after them.
- Overriding Widget.onDetach() without calling super.onDetach() (onUnload() was meant to serve this purpose without the added danger, and we couldn't make onDetach() final, unfortunately). This stops the widgets from calling setEventListener(null), which breaks the reference cycle.

I just posted a wiki page I've been sitting on for a few months with some (hopefully) more useful information and background:

If you uncover any leads in your quest (or a simple repro), I'd love to hear about it. This problem has been my personal Moby Dick for years now, and since I can't eliminate old versions of IE, I'd really like to see it laid to rest!

Cheers,
joel.


--
You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group.
To post to this group, send email to google-we...@googlegroups.com.
To unsubscribe from this group, send email to google-web-tool...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.


Sivasubramanian Thiagarajan

unread,
May 3, 2018, 9:59:55 AM5/3/18
to GWT Users
Hi Paul,

Microsoft memory leak detector tool v2 which you mentioned, the download link does not work any more. Do you know if the same tool or a later version(which can work for newer IE versions like IE11, IE10) available elsewhere? Will really appreciate your help as my team struggles with IE memory leak issues with our GWT app today.

Thanks & regards
Niranjan

Paul McLachlan

unread,
May 3, 2018, 10:47:25 AM5/3/18
to GWT Users
We haven't had any problems with the newer versions of IE.  Is it possible the bug is just in your code as opposed to being browser specific strangeness?  One trick I've used in the past is to run Dev Mode with a normal Java memory tool (such as YourKit).  Obviously you need to look past GWT internals but I've fix problems with this approach before.

Reply all
Reply to author
Forward
0 new messages