Ideas on where xulrunner might leak memory
Win2XUL
with 512Mb, and there is the appearance of a significant memory leak.
We wonder where we should look, or if the various possible memory leaks
in Firefox 1.5 that people seem to talk about mean that the current
xulrunner version must just be assumed to leak (and we must come up
with a work-around, like a periodic restart).
When idle, the content changes every 5 to 10 seconds, cycling through 4
html pages with lots of tables-in-tables, and a little flash. After 72
hours or so, xulrunner goes from 30Mb to 130+Mb, and slows down
dramatically to the point where there is no visible rendered output
before it attempts to navigate to the next page. Currently, eventually
we time it out and restart the machine.
Should we expect to be able to get xulrunner to run with much less
memory leakage (i.e. is it likely to be something specific in our
configuration/implementation), or is it just a developer preview type
issue that may get better with a future xulrunner version (i.e. could
we expect it to get better just waiting for the release corresponding
to Firefox 2, whenever that may be)?
Thank you for any quick pointers, opinions, experience or suggestions!
Boris Zbarsky
> When idle, the content changes every 5 to 10 seconds, cycling through 4
> html pages with lots of tables-in-tables, and a little flash.
As a first step, I'd try eliminating the flash and seeing how that affects
things....
> Should we expect to be able to get xulrunner to run with much less
> memory leakage (i.e. is it likely to be something specific in our
> configuration/implementation), or is it just a developer preview type
> issue that may get better with a future xulrunner version (i.e. could
> we expect it to get better just waiting for the release corresponding
> to Firefox 2, whenever that may be)?
Possibly some of both. Depending on what your content is doing, you may be
triggering known leaks in Gecko 1.8; then the solutions are either a fixed Gecko
or changing the content to avoid the relevant codepaths. ;)
Similarly, you may be triggering unknown leaks... The solutions are the same,
but both are harder without some debugging.
-Boris
Win2XUL
Thank you for the good suggestions... it does seem that the rate may be
slower without any flash on any of the pages. We'll try an accelerated
test with shorter delays between pages as it seems that some memory
leaks are related to page loading. However, the message I'm taking is
that aiming for continuous operation is not realistic at this stage,
especially as we don't have a choice on flash, and we should instead be
looking at automated restarts at a time when it does the least
disruption.
We used to have to do automated periodic restarts on another unnamed
operating system with another unnamed browser, so we know that is a
reliable workaround even if it is ugly. It's a shame here because it's
so close to working without anything ugly... but in time I'm sure the
situation with Xulrunner/Linux will ultimately be better, since it is
very close to working without restarts except for this issue.
Boris Zbarsky
> However, the message I'm taking is
> that aiming for continuous operation is not realistic at this stage
Unless your content is very restricted, yeah. :(
> but in time I'm sure the
> situation with Xulrunner/Linux will ultimately be better, since it is
> very close to working without restarts except for this issue.
That's the hope, but it won't happen by magic... :(
What would help most here is actually if you guys have the cycles to help debug
these leaks (either by making the content in question available to someone else
or by doing a bit of debugging yourselves).
For a start, if you install the extension at
http://dbaron.org/mozilla/leak-monitor/ in Firefox 2 and run it through your
content set, do you see any leaks reported?
-Boris
Win2XUL
Thank you for the suggestions, we've very happy to help with any
debugging, as yes it's not going to happen by magic...
With the extension installed none of the content appears to result in
the extension finding anything (though not sure what the extension
would quite do). Any suggestions on next steps?
Disclaimer: It's not feasible to exactly replicate the conditions on
the target. Because we needed to control the browser navigation
completely, and equivalent of functions like OnNavigate/OnBeforeLoad
either were crashing (under Linux), and/or did not actually intercept
the load prior to the browser having actually started to download page
elements (e.g. it was really more like OnBeforeDisplay), and/or some
events appeared not to bubble up the event stack to be catchable as
implied by documentation, we ultimately had to go for very loose
'embedding' and run our xulrunner browser from a local server. The
local server does make some modifications on-the-fly to the content, so
it's not identical (but it is similar). This results in things like
when the browser does a timeout page refresh, the server never responds
because we intercept the request, and at a later time tell the browser
to go somewhere else (and while it shouldn't result in a memory leak in
the browser, it's certainly not close to standard browser usage for the
equivalent of every hyperlink request being ignored). Then there's the
twist that we have to talk to the browser with sockets, so there's the
possibility that the issues are not in the core of Xulrunner, but in
our little socket-browser application.
In short, it's really ugly and because the ugliness is generally
workarounds to various issues that will be dealt with in xulrunner in
due course, our test bed may be unlikely to be representative of more
useful bugs/leaks to quash. But we're more than happy to try anyway!
(We're also wondering if some of the things that forced the workaround
may be fixed in the 1.9 alpha, but may not be smart to take a chance on
in production). Seems like we are always six or twelve months too
early!
Maybe if we could wrap our heads around the complexity of the overall
project for 1.9 we should somehow be helping with the more fundamental
bugs, but generally embedding issues have been less important than
Firefox issues, which is why we thought we would be better off in the
mainstream. Then there's the minor matter of the client expecting
delivery on a knowable time line... but it's all good, clean fun.
Boris Zbarsky
> With the extension installed none of the content appears to result in
> the extension finding anything (though not sure what the extension
> would quite do). Any suggestions on next steps?
Hmm... If you're compiling your own Gecko you could compile with refcount
logging enabled and see where that gets you... are you?
-Boris
Win2XUL
Thanks... No, the idea was to stay in the mainstream to get the benefit
of all the good work and attention on Firefox, and out of any embedded
backwater, so we don't roll our own. We were tempted with the
embedding API to try and create a version that bypassed the event
bubbling factors that forced us down the local server route, but it was
way beyond our time and knowledge.
However, we were able to create a cycle of 3 html files with a little
flash that is similar to what the local server feeds to the browser.
These accelerated files leak at roughly 1Mb per half an hour on Firefox
2 under Windows. (Note the Flash player needed to be
reinstalled/updated in order to work from local disk on FF2). They
don't trigger the leak detection plug-in.
The zip file is about 38K. Who could they be useful to (and how can we
send them), or how could we use them now that we can reproduce the link
on a more easily accessible platform?
Boris Zbarsky
>> Hmm... If you're compiling your own Gecko you could compile with refcount
>> logging enabled and see where that gets you... are you?
>
> of all the good work and attention on Firefox, and out of any embedded
> backwater, so we don't roll our own.
Sure. You can compile Firefox with refcount logging enabled... ;) Either
compile --enable-debug or explicitly --enable-logrefcnt.
> The zip file is about 38K. Who could they be useful to (and how can we
> send them), or how could we use them now that we can reproduce the link
> on a more easily accessible platform?
If you're willing to make those files public, you should just file a bug and
attach the files to it. Please cc me on the bug if you do. If you're not, you
could just email them to me...
If you're still willing to debug a little yourself, I think
http://www.mozilla.org/performance/leak-tutorial.html and
http://www.mozilla.org/performance/leak-brownbag.html have some information, at
least assuming that what's leaked are XPCOM objects.
-Boris