2.2 Gib :(

[Ludovic Hirlimann]

Hi guys,

Just realized that Thunderbird was eating a wooping 2.2 Gb of memory on
my machine. I've turned gloda off.
I run the latest nightly. Am I the only one seeing such bad perfs ?


Ludo

--
SRE Team
QA Lead for Thunderbird
http://sietch-tabr.tumblr.com/

[Wayne]

There are rare reports

> _______________________________________________
> tb-planning mailing list
> tb-pl...@mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning
_______________________________________________
tb-planning mailing list
tb-pl...@mozilla.org
https://mail.mozilla.org/listinfo/tb-planning

[Ben Bucksch]

On 15.02.2013 15:28, Ludovic Hirlimann wrote:
> Hi guys,
>
> Just realized that Thunderbird was eating a wooping 2.2 Gb of memory on
> my machine. I've turned gloda off.
> I run the latest nightly. Am I the only one seeing such bad perfs ?

No, you are not the only one, I see the same. Thunderbird currently uses
1.2 GB RAM (resident) / 2.2 GB (virtual). This is fairly "normal" for my
Thunderbird.

If you use Lightning, try putting that in a separate profile and
instance. That helped me a lot. (Bug 753343 and 441710)

Ben

[Mike Conley]

The output of about:memory (going to Troubleshooting Information,
about:memory) is sometimes useful for these things.

-Mike

[Ben Bucksch]

On 20.02.2013 18:42, Mike Conley wrote:
> The output of about:memory (going to Troubleshooting Information,
> about:memory) is sometimes useful for these things.

1,179.83 MB (100.0%) -- explicit
├────534.73 MB (45.32%) ── heap-unclassified
├────455.78 MB (38.63%) -- window-objects
│ ├──421.44 MB (35.72%) -- top(none)/detached/window([system])
│ │ ├──420.85 MB (35.67%) -- js/compartment([System Principal], about:blank)
│ │ │ ├──261.13 MB (22.13%) -- gc-heap

[Ludovic Hirlimann]

On 2/20/13 6:42 PM, Mike Conley wrote:
> The output of about:memory (going to Troubleshooting Information,
> about:memory) is sometimes useful for these things.
>

I'm getting 1,780.58 MB (100.0%) -- explicit
├──1,685.53 MB (94.66%) ── heap-unclassified

meaning we probably need a few more probes to be able to analyse , right ?

├─────52.61 MB (02.95%) -- js-non-window
│ ├──34.21 MB (01.92%) -- compartments
│ │ ├──31.35 MB (01.76%) ++ non-window-global
│ │ └───2.86 MB (00.16%) ++ no-global
│ └──18.40 MB (01.03%) ++ (2 tiny)
├─────27.84 MB (01.56%) ++ window-objects
└─────14.60 MB (00.82%) ++ (16 tiny)

[Ludovic Hirlimann]

On 2/21/13 2:24 PM, Florian Quèze wrote:

> On Thu, Feb 21, 2013 at 1:21 PM, Ludovic Hirlimann <lud...@mozilla.com> wrote:
>> On 2/20/13 6:42 PM, Mike Conley wrote:
>>> The output of about:memory (going to Troubleshooting Information,
>>> about:memory) is sometimes useful for these things.
>>>
>> I'm getting 1,780.58 MB (100.0%) -- explicit
>> ├──1,685.53 MB (94.66%) ── heap-unclassified
>>
>> meaning we probably need a few more probes to be able to analyse , right ?

> If you can reliably reproduce, the way to analyse heap-unclassified is
> to run DMD (https://wiki.mozilla.org/Performance/MemShrink/DMD)
>
I'll give it a shot when I get a good hour to look into it. And yes I
can reproduce quite easily.

[acel...@atlas.sk]

Are you guys composing a lot of messages? Can the 2MB leak per compose window open+close explain some of this?
______________________________________________________________
> Od: "Ludovic Hirlimann" <lud...@mozilla.com>
> Komu: "Florian Quèze" <flo...@queze.net>
> Dátum: 21.02.2013 14:30
> Predmet: Re: 2.2 Gib :(
>
> CC: "Mike Conley" <mco...@mozilla.com>, "tb-pl...@mozilla.org" <tb-pl...@mozilla.org>

>----------

[Ludovic Hirlimann]

On 2/21/13 2:32 PM, acel...@atlas.sk wrote:
> Are you guys composing a lot of messages? Can the 2MB leak per compose window open+close explain some of this?
>

I get this maybe 2 or 3 minutes after startup even If I don't do anything.

[acel...@atlas.sk]

____________________________________________________________
> Od: "Ludovic Hirlimann" <lud...@mozilla.com>

> Komu: <acel...@atlas.sk>, "Florian Quèze" <flo...@queze.net>
> Dátum: 21.02.2013 14:35

> Predmet: Re: 2.2 Gib :(
>
> CC: "Mike Conley" <mco...@mozilla.com>, "tb-pl...@mozilla.org" <tb-pl...@mozilla.org>

>On 2/21/13 2:32 PM, acel...@atlas.sk wrote:
>> Are you guys composing a lot of messages? Can the 2MB leak per compose window open+close explain some of this?
>>
>I get this maybe 2 or 3 minutes after startup even If I don't do anything.
>

Then do something already! :-P

Ok, then that is a major problem.

[Ludovic Hirlimann]

On 2/21/13 2:47 PM, acel...@atlas.sk wrote:
> ____________________________________________________________
>> Od: "Ludovic Hirlimann" <lud...@mozilla.com>
>> Komu: <acel...@atlas.sk>, "Florian Quèze" <flo...@queze.net>
>> Dátum: 21.02.2013 14:35
>> Predmet: Re: 2.2 Gib :(
>>
>> CC: "Mike Conley" <mco...@mozilla.com>, "tb-pl...@mozilla.org" <tb-pl...@mozilla.org>
>> On 2/21/13 2:32 PM, acel...@atlas.sk wrote:
>>> Are you guys composing a lot of messages? Can the 2MB leak per compose window open+close explain some of this?
>>>
>> I get this maybe 2 or 3 minutes after startup even If I don't do anything.
>>
> Then do something already! :-P
>
> Ok, then that is a major problem.

currently cleaning my compile directory

[Irving Reid]

My guess is that Ben and Ludo have different problems. Ludo's is the one
I'm used to seeing, with extremely large heap-unclassified. Our current
suspicion is that most of that is in-memory caching of the Mork message DBs.

Ben's is different - more memory tracked in the window-objects than I
usually see. That could be memory leaks or a misbehaving extension.

- irving -

[Wayne Mery (d531)]

while you're at it, you may as well enable msgdb:5 logging to surface
whether dbs are closing

[Ludovic Hirlimann]

On 2/21/13 4:01 PM, Irving Reid wrote:
> My guess is that Ben and Ludo have different problems. Ludo's is the
> one I'm used to seeing, with extremely large heap-unclassified. Our
> current suspicion is that most of that is in-memory caching of the
> Mork message DBs.
>

Would me running Dark matter help you ?

Ludo

--
SRE Team
QA Lead for Thunderbird
http://sietch-tabr.tumblr.com/

[Florian Quèze]

On Thu, Feb 21, 2013 at 1:21 PM, Ludovic Hirlimann <lud...@mozilla.com> wrote:

> On 2/20/13 6:42 PM, Mike Conley wrote:
>> The output of about:memory (going to Troubleshooting Information,
>> about:memory) is sometimes useful for these things.
>>
> I'm getting 1,780.58 MB (100.0%) -- explicit
> ├──1,685.53 MB (94.66%) ── heap-unclassified
>
> meaning we probably need a few more probes to be able to analyse , right ?

If you can reliably reproduce, the way to analyse heap-unclassified is

to run DMD (https://wiki.mozilla.org/Performance/MemShrink/DMD)

--
Florian Quèze

[Ben Bucksch]

On 21.02.2013 14:32, acel...@atlas.sk wrote:
> Can the 2MB leak per compose window open+close explain some of this?

In my case: Some, of course. I write maybe 10 emails/day on average, and
my Thunderbird instance often lives 1-2 months, if no bug (e.g. massive
leaks) prevents that. It's currently running since 37 days.

BTW: I'm running 64bit Linux, so your 2 MB leak may well be 4 MB for me.
That alone could account for 1.5 GB of leaks.

Ben

[acel...@atlas.sk]

> Od: "Ben Bucksch" <ben.b...@beonex.com>
> Komu: <tb-pl...@mozilla.org>
> Dátum: 22.02.2013 15:53

> Predmet: Re: 2.2 Gib :(
>

>On 21.02.2013 14:32, acel...@atlas.sk wrote:
>> Can the 2MB leak per compose window open+close explain some of this?
>
>In my case: Some, of course. I write maybe 10 emails/day on average, and
>my Thunderbird instance often lives 1-2 months, if no bug (e.g. massive
>leaks) prevents that. It's currently running since 37 days.
>
>BTW: I'm running 64bit Linux, so your 2 MB leak may well be 4 MB for me.
>That alone could account for 1.5 GB of leaks.
>

So can you try running with patch from https://bugzilla.mozilla.org/show_bug.cgi?id=765074 ?

[Ben Bucksch]

On 22.02.2013 16:11, acel...@atlas.sk wrote:
> So can you try running with patch fromhttps://bugzilla.mozilla.org/show_bug.cgi?id=765074 ?

Will do.

(My Thunderbird is building right now with --enable-dmd, it's compiling
Gecko, so I added the patch while the build process is running and
assume it will be picked up.)

Ben

[Chris Coulson]

> _______________________________________________
> tb-planning mailing list
> tb-pl...@mozilla.org
> https://mail.mozilla.org/listinfo/tb-planning

Hi,

I checked today and I'm seeing pretty much the same behaviour as Ludo (>
2GB, mostly heap-unclassified). I just made a --disable-jemalloc build
locally and ran it in valgrind, and whilst it's pretty difficult to
separate real leaks from stuff that just isn't cleaned up properly at
shutdown, https://bugzilla.mozilla.org/show_bug.cgi?id=844148 did jump
out to me fairly quickly (although I'm not sure how much it's worth in
the real world yet).

Regards
Chris

[Ben Bucksch]

On 22.02.2013 16:47, Ben Bucksch wrote:
> On 22.02.2013 16:11, acel...@atlas.sk wrote:
>> So can you try running with patch
>> fromhttps://bugzilla.mozilla.org/show_bug.cgi?id=765074 ?
>
> Will do.

I did that, but unfortunately could not use the patch, because some
extensions which I absolutely need (show in/out and Virtual Identity)
don't work in trunk and result in a broken UI.

[Ludovic Hirlimann]

On 2/20/13 6:42 PM, Mike Conley wrote:

> The output of about:memory (going to Troubleshooting Information,
> about:memory) is sometimes useful for these things.
>

Thanks to Jtcranmer's newly reporter I now know what consumes my memory :

2,149.48 MB (100.0%) -- explicit
├──2,075.99 MB (96.58%) -- maildb
│ ├──1,442.06 MB (67.09%) ──
database(imap://lhirlimann%40mozilla.com@xxx/Infra%27s%20Inbox/Cron/YYYYY)

So I think I'm just going to unsubscribe from this folder.

[acel...@atlas.sk]

Shouldn't we first find out why this folder takes so much memory?
Can you tell us anything about it? How many messages/size of them?

I have a test folder of 1 million messages comprising 4GB of mbox data and I do not get such high numbers, when just viewing them.
______________________________________________________________
> Od: "Ludovic Hirlimann" <lud...@mozilla.com>
> Komu: <tb-pl...@mozilla.org>
> Dátum: 02.04.2013 14:02

> Predmet: Re: 2.2 Gib :(
>

>----------

[Ludovic Hirlimann]

3448 messages from 2kb to 10 kb it seems (using My webmail to send and look.

It probably get cycled a lot - eg older messages are removed because they aren't relevant old cron log don't serve you years after.

Ludo

[Ludovic Hirlimann]

On 4/2/13 2:23 PM, Ludovic Hirlimann wrote:
> 3448 messages from 2kb to 10 kb it seems (using My webmail to send and look.
>
> It probably get cycled a lot - eg older messages are removed because they aren't relevant old cron log don't serve you years after.
>
>
>

I'm wondering if I should switch this account to Maildir and see if that
helps ?

Thoughts ?

[acel...@atlas.sk]

If the 'maildb' keeps only metadata about the messages, as I would expect, than maildir may not help anything.
Maybe you could repair, or just re-subscribe to the folder.

______________________________________________________________
> Od: "Ludovic Hirlimann" <lud...@mozilla.com>
> Komu: <tb-pl...@mozilla.org>

> Dátum: 02.04.2013 14:55

> Predmet: Re: 2.2 Gib :(
>

[Wayne Mery (d531)]

maybe we shouldn't mess too much with a good testcase, til we know how
we want to debug it ??

Also, not saying it is the case here, but it is possible the numbers may
be off. ref https://bugzilla.mozilla.org/show_bug.cgi?id=857036

[Wayne]

maybe we shouldn't mess too much with a good testcase, til we know how
we want to debug it ??

in addition, the figure might be off somewhat.
ref https://bugzilla.mozilla.org/show_bug.cgi?id=857036

On 4/2/2013 9:10 AM, acel...@atlas.sk wrote: