Re: Investigating memory bloat in Chromium

291 views
Skip to first unread message

Kentaro Hara

unread,
Jul 25, 2017, 12:43:40 PM7/25/17
to DJ Park, ba...@chromium.org, hajim...@chromium.org, kei...@chromium.org, kou...@chromium.org, pe...@chromium.org, sigb...@opera.com, ta...@chromium.org, tz...@chromium.org, yukis...@chromium.org, yu...@chromium.org, memor...@chromium.org
Hi DJ!

On Tue, Jul 25, 2017 at 5:28 PM, DJ Park <djp...@microsoft.com> wrote:

Hello,

 

I found your contacts on the Blink Memory Team page (https://www.chromium.org/blink/memory-team) and am hoping you can help point me in the right direction. 


Sorry, the team page is out-dated. The new team page is here. (I'll update the old team page to redirect to the new page.)
 

 

I work on performance and memory for a product called Microsoft Teams and have been investigating a memory bloat issue where the app’s memory consumption reaches 1+ gb on Chrome. 


Awesome :D


Our team is working on fixing up leaks in our code base but profiling shows that the majority of the memory is being consumed on the native side.  (e.g. 200mb V8 vs 1+gb native).

 

I’ve collected traces using memory-infra and am trying to make sense of how to make these actionable on our end.  An example trace has mem usage split across blink_gc (100mb), malloc (1gb), and partition_alloc (115mb), etc.  The drill downs provided by //tracing provide a bit more information but I’m not very clear on what to make of the data.  E.g. Of the 1+gb used by malloc, 840mb is listed as <unspecified> and 194mb is listed as metadata_fragmentation_caches. 


Would it be useful to get data with the Native stack profier? It will give you object types allocated on malloc.


 

 

Any guidance or help would be really appreciated. 

 

Thanks in advance,

-DJ




--
Kentaro Hara, Tokyo, Japan

DJ Park

unread,
Jul 25, 2017, 1:35:15 PM7/25/17
to Kentaro Hara, memor...@chromium.org

Thanks for the quick response Kentaro!  Cleaning up cc list.

 

I wasn’t aware of the native stack profiler.  I’m setting up chromium source on my box now and will try it out.  Btw, will symbolize_trace work on a basic memory-infra trace or do I need to run it through TraceOnTap? 

Kentaro Hara

unread,
Jul 25, 2017, 1:38:44 PM7/25/17
to DJ Park, Erik Chen, dsk...@chromium.org, memor...@chromium.org
+Erik +Dmitry for the native stack profiler.

Erik Chen

unread,
Jul 25, 2017, 1:45:11 PM7/25/17
to Kentaro Hara, Etienne Bergeron, DJ Park, dsk...@chromium.org, memor...@chromium.org
+etienneb, who has the most experience with native heap profiling on Windows.

1) Once you've built Chromium locally, native heap profiling can be turned on via about:flags [#enable-heap-profiling]. This requires a restart of Chromium to pick up the changes. Make sure to have built with symbols!

2) Try to capture a trace using chrome://tracing. Make sure it's a short trace.

3) Then use symbolize_trace on the resulting trace to update all its symbols.

4) Try to load the trace in chrome://tracing. 

See this link for images and more detailed instructions [which are a little bit out of date]. 


Wez

unread,
Jul 26, 2017, 8:57:28 PM7/26/17
to Erik Chen, Kentaro Hara, Etienne Bergeron, DJ Park, dsk...@chromium.org, memor...@chromium.org
Erik, I thought we had the symbolize_trace script fixed up so it can work with traces from e.g. Chrome Canary, now?

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.
To post to this group, send email to memor...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAEYHnr1yTfYgO4qPkzV6wg%2Bio-5i-gyo%3D2mTRqvZQ05YuPcnrw%40mail.gmail.com.

Erik Chen

unread,
Jul 26, 2017, 8:59:44 PM7/26/17
to Wez, Kentaro Hara, Etienne Bergeron, DJ Park, dsk...@chromium.org, memor...@chromium.org
On Wed, Jul 26, 2017 at 5:57 PM, Wez <w...@chromium.org> wrote:
Erik, I thought we had the symbolize_trace script fixed up so it can work with traces from e.g. Chrome Canary, now?
That requires symbols, which are not public. [technically the Windows symbols can be accessed via symbol server, which is public, but that isn't hooked up].

DJ Park

unread,
Jul 28, 2017, 3:10:49 PM7/28/17
to Erik Chen, Wez, Kentaro Hara, Etienne Bergeron, dsk...@chromium.org, memor...@chromium.org

Had some issues setting up a local Chromium build so tried another route by running symbolize_trace on a trace I collected from a dev build with heap profiling flag enabled (https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/489682/). 

 

Symbolize_trace ran into an “list index out of range” error on so patched that up with a len check on stdout_data. 

 

File "C:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1202, in _SymbolizeWin

 

Symbolize script runs successfully with following output.

 

Symbolizing...

  Symbolizing 3 PCs from C:\Users\djpark\Temp\chromium\chrome-win32\chrome-win32\chrome.exe...

  Symbolizing 14 PCs from C:\Users\djpark\Temp\chromium\chrome-win32\libglesv2.dll...

  Symbolizing 3 PCs from C:\WINDOWS\SYSTEM32\ntdll.dll...

  Symbolizing 1 PCs from C:\WINDOWS\System32\KERNEL32.DLL...

  Symbolizing 8685 PCs from C:\Users\djpark\Temp\chromium\chrome-win32\chrome_child.dll...

 

If I load the symbolized trace into chrome://tracing, I’m not seeing much of a difference unfortunately.  E.g. There is still a large chunk of memory usage categorized as <unspecified> under malloc category

 

 

Would be great to get some help with a few questions:

  • Am I correct in expecting that a properly symbolized trace should have more details than <unspecified>? 
  • Is symbolize_tracing designed to work with traces captured from prebuilt-dev builds like the one I used or do I need to do a full local build with symbols?  I hit a bunch of compile errors when doing a local build which is why I resorted to using the prebuilt build. 

 

Thank you!

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.


To post to this group, send email to memor...@chromium.org.

Etienne Bergeron

unread,
Jul 28, 2017, 3:14:34 PM7/28/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
Can you share you trace?
I'm gonna check for the symbolisation part.

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 




--
Etienne Bergeron
Chrome

Etienne Bergeron

unread,
Jul 28, 2017, 3:20:12 PM7/28/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

An unsymbolized trace will contains raw PC addresses:


After the symbolisation, addresses are replaced by symbols (when available) or <lib + offset> otherwise.


--
Etienne Bergeron
Chrome

DJ Park

unread,
Jul 31, 2017, 2:52:28 AM7/31/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Thanks Etienne.  Trace attached.  Comparing the output you shared with my output, doesn’t look like symbols are loading in properly. 

 

As I mentioned in my earlier mail, 1) I did have to work around that error in the symbolize_trace script and 2) I used a pre-built version of chrome to collect the initial trace.  Any tips on validating that symbol version and trace version are matching? 

 

From: Etienne Bergeron [mailto:etie...@google.com]
Sent: Friday, July 28, 2017 12:20 PM
To: DJ Park <djp...@microsoft.com>
Cc: Erik Chen <erik...@chromium.org>; Wez <w...@chromium.org>; Kentaro Hara <har...@chromium.org>; dsk...@chromium.org; memor...@chromium.org
Subject: Re: Investigating memory bloat in Chromium

 

 

An unsymbolized trace will contains raw PC addresses:

cid:ii_j5o93c530_15d8aa282c17a0ca

 

 

After the symbolisation, addresses are replaced by symbols (when available) or <lib + offset> otherwise.

https://lh5.googleusercontent.com/_3iIsMIximruvaN8SFykw3GBPeoA78vCOYj2zVY3DEeeXjrMuADfNaYsQd7LKVScV0FFlNyD9XOKbojbia_9MD2_cjOFMdPKGp-QoZ25MHub2XjZxLUBJc12kPW35qEgpx584Kvo

 

On Fri, Jul 28, 2017 at 3:14 PM, Etienne Bergeron <etie...@google.com> wrote:

Can you share you trace?

I'm gonna check for the symbolisation part.

On Fri, Jul 28, 2017 at 3:10 PM, DJ Park <djp...@microsoft.com> wrote:

Had some issues setting up a local Chromium build so tried another route by running symbolize_trace on a trace I collected from a dev build with heap profiling flag enabled (https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/489682/). 

 

Symbolize_trace ran into an “list index out of range” error on so patched that up with a len check on stdout_data. 

 

File "C:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1202, in _SymbolizeWin

 

Symbolize script runs successfully with following output.

 

Symbolizing...

  Symbolizing 3 PCs from C:\Users\djpark\Temp\chromium\chrome-win32\chrome-win32\chrome.exe...

  Symbolizing 14 PCs from C:\Users\djpark\Temp\chromium\chrome-win32\libglesv2.dll...

  Symbolizing 3 PCs from C:\WINDOWS\SYSTEM32\ntdll.dll...

  Symbolizing 1 PCs from C:\WINDOWS\System32\KERNEL32.DLL...

  Symbolizing 8685 PCs from C:\Users\djpark\Temp\chromium\chrome-win32\chrome_child.dll...

 

If I load the symbolized trace into chrome://tracing, I’m not seeing much of a difference unfortunately.  E.g. There is still a large chunk of memory usage categorized as <unspecified> under malloc category

 

cid:image002.jpg@01D3079A.856D7EE0

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

trace_latest_chromium.json.gz

Etienne Bergeron

unread,
Jul 31, 2017, 12:03:00 PM7/31/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
I just realized that you provide me a chromium (local build) trace.

Just to clarify:

1) The symbolisation script is able to fetch and download symbols for official builds only (not local dev build)
    * The script only fetches symbols from GCS. It's a work in progress to fetch from our public symbol servers (windows-only).
2) With these limitations, you are able to symbolize a local dev build on windows with the symbolisation script and the addr2line-pdb.exe executable.
    * building addr2line-pdb executable: ninja -C out\build addr2line-pdb.exe
    * symbolize_trace <your-trace> --addr2line-executable <path-to-addr2line-pdb>

addr2line-pdb.exe is using the standard dbghelp.dll API to resolve symbols.
It needs the appropriate PDB files beside your chrome executable to work.

> Symbolize_trace ran into an “list index out of range” error on so patched that up with a len check on stdout_data. 

How can this happen? I'm curious to see the repro for this case. Can you provide me more info and/or the patch you wrote.
This error and the fix is probably hiding the symbolisation problem you are facing.


To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome




--
Etienne Bergeron
Chrome

DJ Park

unread,
Jul 31, 2017, 12:22:07 PM7/31/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

To help clarify the repro, here are the exact steps I took:

 

  1. Grabbed latest version of Chromium dev build from https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/489682/
  2. Collected trace using above dev build from chrome://tracing after enabling “enable heap snapshot” option under chrome://flags
  3. Grabbed latest version of chromium source using following steps.  Note: Was *not* able to do a successful build due to various compile errors.  https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md
  4. Ran symbolize_trace script specified here: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/heap_profiler.md
  5. Script hit an error saying that it couldn’t find addr2line-pdb.exe
  6. Opened up the addr2line-pdb vcproj in vs2015 and built + added to PATH
  7. Reran symbolize_trace and it hit an error saying “list index out of range” - File "C:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1202, in _SymbolizeWin
  8. Added “if i*2 < len(stdout_data):” check to line 1201 and reran symbolize_trace script
  9. Script succeeded and generated a new trace file and backed up the old one

 

Based on what you’re saying, sounds like the appropriate PDB isn’t getting loaded in.  Any ideas on how I can fix this? 

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

Etienne Bergeron

unread,
Jul 31, 2017, 12:46:29 PM7/31/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
If you are using a build from https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/489682/, you can download the PDBs with the executables.
You need to put the PDBs side by side with the executable, and the dbghelp API will be able to find them automatically.


> Script hit an error saying that it couldn’t find addr2line-pdb.exe

That's normal. You need the executable in your path. It's not required to be from the same build.
Always try to use the latest version if possible: we fixed some bugs a few months ago in that program.
You can also provide the command-line --addr2line-executable <hardcoded-path> is you prefer.

>  Opened up the addr2line-pdb vcproj in vs2015 and built + added to PATH

Perfect. As soon as you have a valid executable (debug/release, x32 or x64) it doesn't really matter.

> Added “if i*2 < len(stdout_data):” check to line 1201 and reran symbolize_trace script

This is probably filtering every addresses conversion. The addr2line-pdb is probably outputting an error message.

> Script succeeded and generated a new trace file and backed up the old one

The script will generate a valid trace as output, even if some addresses were not converted. In your case, I believe all of them were not converted.


To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome




--
Etienne Bergeron
Chrome

Etienne Bergeron

unread,
Jul 31, 2017, 12:55:51 PM7/31/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
> Collected trace using above dev build from chrome://tracing after enabling “enable heap snapshot” option under chrome://flags

"Enable heap snapshot" ? You need to activate heap profiline, with the option "native mode".



note: I'm running a python 64-bit version. Otherwise, the python script is often throwing a MemoryError.
You may face this issue too.

--
Etienne Bergeron
Chrome

Etienne Bergeron

unread,
Jul 31, 2017, 1:12:13 PM7/31/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
I just made a test with the chromium version, and it worked fine for me.

Steps:

2) Extract both archive in the same folder. Executable/DLLs and PDBs should be side by side:
3) Be sure native heap profiling is activate (chrome://flag)
4) Collect a trace (chrome://tracing) with memory-infra (about 2 or 3 seconds)
5) Check that the trace contains a memory dump with unsymbolized stack frames:
6) Save the trace to a temporary file.
7) Symbolize the saved trace

C:\src\traces\test>c:\python_27\files\python.exe c:\src\chromium\src\third_party\catapult\tracing\bin\symbolize_trace --addr2line-executable c:\src\chromium\src\out\build\addr2line-pdb.exe trace_test.json.gz



No errors should happens.

8) Reload the symbolized trace

If any steps is not going the same way than above, tell me.


--
Etienne Bergeron
Chrome

DJ Park

unread,
Jul 31, 2017, 2:29:38 PM7/31/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Tried running through these steps with pdb’s side by side with exe’s.  Also removed the if check I added in symbolize script. 

 

When I run symbolize_trace I still get the previou index out of range error:

 

Trace loaded for Windows NT/62.0.3168.0

Symbolizing...

  Symbolizing 6 PCs from C:\Program Files\Common Files\microsoft shared\ink\tiptsf.dll...

usage: addr2line-pdb [-f|--functions] [-C|--demangle] [-e filename]

(Then list the hex addresses on stdin, one per line)

Traceback (most recent call last):

  File "c:\Sources\chromium\src\third_party\catapult\tracing\bin\symbolize_trace", line 17, in <module>

    sys.exit(symbolize_trace.main(sys.argv[1:]))

  File "c:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1659, in main

    SymbolizeTrace(options, trace, symbolizer)

  File "c:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1407, in SymbolizeTrace

    SymbolizeFiles(symfiles, symbolizer)

  File "c:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1309, in SymbolizeFiles

    symbolizer.SymbolizeSymfile(symfile)

  File "c:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1257, in SymbolizeSymfile

    self._SymbolizeWin(symfile)

  File "c:\Sources\chromium\src\third_party\catapult\tracing\tracing\extras\symbolizer\symbolize_trace.py", line 1201, in _SymbolizeWin

    frame.name = stdout_data[i * 2]

IndexError: list index out of range

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

Etienne Bergeron

unread,
Jul 31, 2017, 2:49:25 PM7/31/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
I think I've got what is going wrong. Cross your fingers.

The default dbghelp.dll is restricted to local debugging only and doesn't allow fetching remote symbols.
Which may explain why you are failing on that specific DLL (which is a system DLL with remote symbols):
  > Symbolizing 6 PCs from C:\Program Files\Common Files\microsoft shared\ink\tiptsf.dll...

I see two possibilities here:
1) You do not have the appropriate env variable. This is telling dbghelp where to fetch symbols.


2) You do not have a dbghelp with remote symbols support. The solution is to install a SDK which is providing a DLL with remote symbols enabled.

The symbolisation script doesn't need the remote symbols to work.
You can use this switch:
  --only-symbolize-chrome-symbols
                        Prevents symbolization of non-Chrome [system] symbols

The system symbols will be resolved to <library.dll + offset> instead of the public symbols. But all chrome symbols will be symbolized correctly.


To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome




--
Etienne Bergeron
Chrome

DJ Park

unread,
Jul 31, 2017, 4:32:55 PM7/31/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Crossed my fingers but still no luck ☹  Just tried running the script with the –only-symbolize-chrome-symbols flag but still getting the same output I shared earlier. 

Just to confirm, it should be fine to use any version of the addr2line-pdb.exe right?  Wondering why the “usage: addr2line-pdb [-f|--functions] [-C|--demangle] [-e filename]” message gets printed in the output. 

Also, here’s how I ran my command.

C:\>c:\Python27\python.exe c:\Sources\chromium\src\third_party\catapult\tracing\bin\symbolize_trace --addr2line-executable c:\Sources\chromium\src\third_party\tcmalloc\vendor\vsprojects\addr2line-pdb\Debug\addr2line-pdb.exe --only-symbolize-chrome-symbols c:\Sources\chromium\src\third_party\catapult\tracing\bin\trace_test.json.gz

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

Etienne Bergeron

unread,
Aug 1, 2017, 10:50:28 AM8/1/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
Crossed my fingers harder!

1) Can you output the command-line executed to symbolize. Try it in a shell.
2) Can you send me a trace with this exact version, I'm gonna try it on my side.

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.


To post to this group, send email to memor...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome




--
Etienne Bergeron
Chrome

DJ Park

unread,
Aug 1, 2017, 11:45:16 AM8/1/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Sorry but I don’t follow. 

 

What do you mean when you say “command-line executed to symbolize”?  I’ve attached the specific version of the symbolize_trace script I’ve been running.  I should mention that I’ve tried running this script in both cmd/shell and still getting the same error as earlier. 

 

As for “trace with this exact version”, are you saying that you want me to sync my chromium source to the exact version of the chrome/symbols I used to collect the trace?  If so, I’ve attached the latest trace I took with this build - https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/489682/

cid:image001.png@01D30AA0.9E521160

3) Be sure native heap profiling is activate (chrome://flag)

cid:image002.png@01D30AA0.9E521160

4) Collect a trace (chrome://tracing) with memory-infra (about 2 or 3 seconds)

cid:image003.png@01D30AA0.9E521160

5) Check that the trace contains a memory dump with unsymbolized stack frames:

cid:image005.png@01D309FF.A657F8D0

6) Save the trace to a temporary file.

7) Symbolize the saved trace

 

C:\src\traces\test>c:\python_27\files\python.exe c:\src\chromium\src\third_party\catapult\tracing\bin\symbolize_trace --addr2line-executable c:\src\chromium\src\out\build\addr2line-pdb.exe trace_test.json.gz

 

cid:image007.png@01D309FF.A657F8D0

No errors should happens.

 

8) Reload the symbolized trace

cid:image012.png@01D309FF.A657F8D0

 

If any steps is not going the same way than above, tell me.

 

 

On Mon, Jul 31, 2017 at 12:55 PM, Etienne Bergeron <etie...@google.com> wrote:

> Collected trace using above dev build from chrome://tracing after enabling “enable heap snapshot” option under chrome://flags

 

"Enable heap snapshot" ? You need to activate heap profiline, with the option "native mode".

 

cid:image002.png@01D30AA0.9E521160

note: I'm running a python 64-bit version. Otherwise, the python script is often throwing a MemoryError.

You may face this issue too.

 

On Mon, Jul 31, 2017 at 12:46 PM, Etienne Bergeron <etie...@google.com> wrote:

If you are using a build from https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/489682/, you can download the PDBs with the executables.

You need to put the PDBs side by side with the executable, and the dbghelp API will be able to find them automatically.

 

cid:image014.png@01D309FF.A657F8D0

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

symbolize_trace
trace_test.json.gz

Etienne Bergeron

unread,
Aug 1, 2017, 1:10:45 PM8/1/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

> As for “trace with this exact version” [...]

No, I was asking for a trace from that specific version:
which is what you did. :)

I was able to symbolize your trace without any problem.


> What do you mean when you say “command-line executed to symbolize”?

The attached version of your symbolization script is only a wrapper to call the real script.
You can see it here: src\third_party\catapult\src\tracing\tracing\extras\symbolizer\symbolize.py

You can add the following print statement to see the running command-line.

If you believe your chromium repo is quite old. You can use the script directly from an recent version of catapult.
You can do a standalone checkout and use the script from it:
  https://github.com/catapult-project/catapult



To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome




--
Etienne Bergeron
Chrome
trace_test.json.gz

DJ Park

unread,
Aug 1, 2017, 5:02:07 PM8/1/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Success!  Looks like the intense crossing of fingers did the trick 😊  That, and an issue I found in symbolize_trace.py where the args being passed into the Popen were incorrect.  The version of addr2line-pdb.exe must be different than the one you have. 

 

Original: cmd = [self.symbolizer_path, '--functions', '--demangle', '--exe', symfile.symbolizable_path]

Fixed: cmd = [self.symbolizer_path, '--functions', '--demangle', '-e', symfile.symbolizable_path]

 

Now that I’ve got a symbolized trace, definitely seeing some more interesting details.  Symbolized trace attached + specific questions called out below.  Would love to get your insights in any of these.  I know there’s a lot of questions so please let me know if it would be easier to set up a quick call instead.  Thanks!

 

Total memory usage for our app in trace: 667mb

  • Blink_GC -  68mb
    • What does an object type of blink::Node correspond to?  Trying to understand why the size/count is so high
    • What does object count represent?  Trying to understand what it means when SVGAnimatePropertyBase has a count of 42,640.
    • The highlight object type looks very complex.  Is there an underlying issue here? 

  • CC - 51mb
    • I’m seeing 14.78mb of discardable images and 37.26mb of tile_memory being used.  Is there something that can be done more efficiently from JS/HTML side to avoid this? 

Machine generated alternative text:

Machine generated alternative text:

 

  • Malloc - 40mb
    • What does media/filters and cc/trees correspond to (they take up 20mb together)?  Again, anything we can do from JS/HTML side to avoid this? 

Machine generated alternative text:

  • Partition_alloc - up 34mb
    • I see that WTF::StringImpl object type is taking up 45.9mb which exceeds the memory usage for partition_alloc.  How can this be? 
    • What does WTF::StringImpl correspond to in JS/HTML terms? 

Machine generated alternative text:

  • Skia – 13mb
    • I see 3 resources listed under GPU Resources that add up to 12mb alone. Any suggestions on how I could track down what these resources are? 

Machine generated alternative text:

 

From: Etienne Bergeron [mailto:etie...@google.com]
Sent: Tuesday, August 1, 2017 10:11 AM
To: DJ Park <djp...@microsoft.com>
Cc: Erik Chen <erik...@chromium.org>; Wez <w...@chromium.org>; Kentaro Hara <har...@chromium.org>; dsk...@chromium.org; memor...@chromium.org
Subject: Re: Investigating memory bloat in Chromium

 

 

> As for “trace with this exact version” [...]

 

No, I was asking for a trace from that specific version:

which is what you did. :)

 

I was able to symbolize your trace without any problem.

cid:ii_j5tu16ur2_15d9ec0638ace75b

 

 

> What do you mean when you say “command-line executed to symbolize”?

 

The attached version of your symbolization script is only a wrapper to call the real script.

You can see it here: src\third_party\catapult\src\tracing\tracing\extras\symbolizer\symbolize.py

 

You can add the following print statement to see the running command-line.

cid:image012.png@01D30ABB.00CB1E30

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

trace_investigation.json.gz

DJ Park

unread,
Aug 3, 2017, 11:06:27 AM8/3/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Quick ping on this to keep the thread alive.  Thanks in advance!

Etienne Bergeron

unread,
Aug 4, 2017, 2:24:18 AM8/4/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
I quickly look to your symbolized trace, and symbolisation didn't worked correctly.
The stackframe are not resolved correctly.



You are probably using an incorrect version of addr2line-pdb.
Take care, there are two version:

If you are using ninja to build chrome, simply use:
  ninja -C out\<build-folder>  addr2line-pdb.c

This is compiling this target:



To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome




--
Etienne Bergeron
Chrome

DJ Park

unread,
Aug 4, 2017, 12:39:17 PM8/4/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Used ninja to generate the addr2-line.pdb and re-ran the script.  Looks like stack frames are being resolved now.  Updated trace attached. 

 

Interesting observations in this particular trace.  CC is taking up 112.3mb.  Majority of this is coming from tile_memory which is chunked into resources of sizes varying from a few kb to 5+mb.  I imagine this is a consequence of our DOM/styles being quite complex.  We’re working on reducing this complexity, but any other tips on reducing this bucket? 

 

As for interpreting the stack frame / object type results, can you help me understand how I should be reading this?  Let’s take partition_alloc for example.  If I sort the Object types by size and picked the largest one (WTF::StringImpl @ 37.5mb) then switch to Stack Frame view, I have a list of stack frames associated with that object type.  How should I be reading this list of stack frames and is there a way to map back to where that 37.5mb of strings were allocated from? 

 

 

From: Etienne Bergeron [mailto:etie...@google.com]
Sent: Thursday, August 3, 2017 11:24 PM
To: DJ Park <djp...@microsoft.com>
Cc: Erik Chen <erik...@chromium.org>; Wez <w...@chromium.org>; Kentaro Hara <har...@chromium.org>; dsk...@chromium.org; memor...@chromium.org
Subject: Re: Investigating memory bloat in Chromium

 

I quickly look to your symbolized trace, and symbolisation didn't worked correctly.

The stackframe are not resolved correctly.

cid:ii_j5xhfs3v0_15dabe82bb0f7ad8

You are probably using an incorrect version of addr2line-pdb.

Take care, there are two version:

On Thu, Aug 3, 2017 at 8:06 AM, DJ Park <djp...@microsoft.com> wrote:

Quick ping on this to keep the thread alive.  Thanks in advance!

 

From: DJ Park
Sent: Tuesday, August 1, 2017 2:01 PM
To: 'Etienne Bergeron' <etie...@google.com>
Cc: Erik Chen <erik...@chromium.org>; Wez <w...@chromium.org>; Kentaro Hara <har...@chromium.org>; dsk...@chromium.org; memor...@chromium.org
Subject: RE: Investigating memory bloat in Chromium

 

Success!  Looks like the intense crossing of fingers did the trick 😊  That, and an issue I found in symbolize_trace.py where the args being passed into the Popen were incorrect.  The version of addr2line-pdb.exe must be different than the one you have. 

 

Original: cmd = [self.symbolizer_path, '--functions', '--demangle', '--exe', symfile.symbolizable_path]

Fixed: cmd = [self.symbolizer_path, '--functions', '--demangle', '-e', symfile.symbolizable_path]

 

Now that I’ve got a symbolized trace, definitely seeing some more interesting details.  Symbolized trace attached + specific questions called out below.  Would love to get your insights in any of these.  I know there’s a lot of questions so please let me know if it would be easier to set up a quick call instead.  Thanks!

 

Total memory usage for our app in trace: 667mb

  • Blink_GC -  68mb
    • What does an object type of blink::Node correspond to?  Trying to understand why the size/count is so high
    • What does object count represent?  Trying to understand what it means when SVGAnimatePropertyBase has a count of 42,640.
    • The highlight object type looks very complex.  Is there an underlying issue here? 

cid:image009.jpg@01D30C2F.30033870

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

DJ Park

unread,
Aug 4, 2017, 12:40:12 PM8/4/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Forgot to attach trace. 

trace_testing.json.gz

Etienne Bergeron

unread,
Aug 4, 2017, 1:13:28 PM8/4/17
to DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org
I didn't look to your trace and I won't have time today. But here is a tool that may help you:

src/third_party/catapult/experimental/tracing/bin/diff_heap_profiler.py

Run your apps, collect a trace, run it for a while, collect a trace... and diff both traces.
You'll get json file of "potential memory leaks".

The output are JSON files (folder output):



You can use an online JSON viewer to format properly these and navigate these files.

You can also run it with only one trace and play with the threshold to get what are the top allocations.

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+unsubscribe@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome




--
Etienne Bergeron
Chrome

DJ Park

unread,
Aug 4, 2017, 1:47:20 PM8/4/17
to Etienne Bergeron, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, memor...@chromium.org

Thanks for the pointers (and for the great help so far).  Definitely owe you a meal/drink if you’re ever in Seattle area. 

 

Detecting leaks is one part of our investigation but the other part is simply understanding why certain allocations are happening and if there are things we can do from HTML/JS land to reduce these.  For that, could use some guidance on how to interpret the trace results. 

 

Would you have some time to hop on a call in the next week or two?  Or perhaps there’s someone else on this alias who could also help? 

 

From: Etienne Bergeron [mailto:etie...@google.com]
Sent: Friday, August 4, 2017 10:13 AM
To: DJ Park <djp...@microsoft.com>
Cc: Erik Chen <erik...@chromium.org>; Wez <w...@chromium.org>; Kentaro Hara <har...@chromium.org>; dsk...@chromium.org; memor...@chromium.org
Subject: Re: Investigating memory bloat in Chromium

 

I didn't look to your trace and I won't have time today. But here is a tool that may help you:

 

src/third_party/catapult/experimental/tracing/bin/diff_heap_profiler.py

 

Run your apps, collect a trace, run it for a while, collect a trace... and diff both traces.

You'll get json file of "potential memory leaks".

 

The output are JSON files (folder output):

 

https://lh3.googleusercontent.com/CBZIOg3ISOXvMWuHdhPELQQ1o00XLyCY7lEUdMr8DF6_KGfE0T2vKJSHJ2OODEAUy54bhdK1xdqo8xBNY7cFMrYbUaI2vkxp-il_KJ-F0N2pL-7akhrZZJmb-m6HWHUP8EAFcwPJ


You can use an online JSON viewer to format properly these and navigate these files.

 

You can also run it with only one trace and play with the threshold to get what are the top allocations.

On Fri, Aug 4, 2017 at 12:39 PM, DJ Park <djp...@microsoft.com> wrote:

Forgot to attach trace. 

 

From: DJ Park
Sent: Friday, August 4, 2017 9:39 AM
To: 'Etienne Bergeron' <etie...@google.com>
Cc: Erik Chen <erik...@chromium.org>; Wez <w...@chromium.org>; Kentaro Hara <har...@chromium.org>; dsk...@chromium.org; memor...@chromium.org
Subject: RE: Investigating memory bloat in Chromium

 

Used ninja to generate the addr2-line.pdb and re-ran the script.  Looks like stack frames are being resolved now.  Updated trace attached. 

 

Interesting observations in this particular trace.  CC is taking up 112.3mb.  Majority of this is coming from tile_memory which is chunked into resources of sizes varying from a few kb to 5+mb.  I imagine this is a consequence of our DOM/styles being quite complex.  We’re working on reducing this complexity, but any other tips on reducing this bucket? 

 

As for interpreting the stack frame / object type results, can you help me understand how I should be reading this?  Let’s take partition_alloc for example.  If I sort the Object types by size and picked the largest one (WTF::StringImpl @ 37.5mb) then switch to Stack Frame view, I have a list of stack frames associated with that object type.  How should I be reading this list of stack frames and is there a way to map back to where that 37.5mb of strings were allocated from? 

 

cid:image001.png@01D30D0E.511FFB30

To unsubscribe from this group and stop receiving emails from it, send an email to memory-dev+...@chromium.org.

 

 



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome



 

--

Etienne Bergeron

Chrome

Daniel Bratell

unread,
Aug 30, 2017, 9:39:01 AM8/30/17
to Etienne Bergeron, 'DJ Park' via memory-dev, DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org
I know it's been a couple of weeks but just a general comment: Your trace had 80,000 Node objects which does indicate a very complex web page or a web page that keeps creating more and more elements.

Some Node elements (<canvas>, <video>, <audio>, <script>, <image>, ....) directly or indirectly own a lot of memory beyond themselves and if that is the case the problem might be bigger than the listed 8 MB.

If it is not a bug in Chromium that keeps them alive, then you should be possible to track them through devtools' memory debugger.  We have had issues (but not recently that I know about) with missing garbage collects but I don't think that is the reason here considering the large object count.

/Daniel
--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

DJ Park

unread,
Sep 5, 2017, 5:40:09 PM9/5/17
to Daniel Bratell, Etienne Bergeron, 'DJ Park' via memory-dev, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org

Thanks for the follow up Daniel. 

 

When you refer to Node objects, are you referring to the object_count listed under “counter” column?  I’m seeing 38,530 in the trace I last sent so want to make sure we’re looking at the same thing. 

 

We’re currently using Angular which makes heavy use of scripts for templates.  We also use inline svg’s (which increase DOM complexity), have a fair amount of images, make use of canvas in several parts of the app, etc.  Is there any documentation on DOM elements that typically end up owning a lot of memory beyond themselves?  Would be great to understand which areas to focus on to get the most bang. 

 

As for using the devtool memory tools (heap snapshot, allocation profile/timeline), my understanding is that this is just giving me a breakdown of JS heap and not the native mem usage related to various components.  Is there a way to attribute native mem usage to the JS objects that I’m missing?  

 

Daniel Bratell

unread,
Sep 6, 2017, 5:26:51 AM9/6/17
to Etienne Bergeron, 'DJ Park' via memory-dev, DJ Park, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org
Yes, it was the Node object count that caught my eye. 

I don't think there is a comprehensive list of indirect memory use per Element type. I know that you need to be careful with video, audio, canvas and maybe img elements.

The connection between what devtools reports and actual memory used is indeed a bit loose, but there is often a correlation. If you can debug from devtools, which has rather easy to use tools compared to the C++ side, then maybe you can figure out if something is wrong in the web page itself.

Sorry for being a bit abstract. I wish I could give some easy suggestions but figuring out the cause of high memory usage remains a complicated problem where few instances are identical.

/Daniel

DJ Park

unread,
Sep 6, 2017, 11:34:18 AM9/6/17
to Daniel Bratell, Etienne Bergeron, 'DJ Park' via memory-dev, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org

No worries. Understand this is a very complex problem and there isn’t a silver bullet. 

 

When you say be careful with video, audio, canvas, img elements, any tips on how we should be careful?  Are there certain usage patterns that cause these elements to retain more memory? 

 

Also, I’d love love love to pick your (or someone else in the know)’s brain about what these different components represent.  For example, what exactly is a node object?  Our app starts off with ~6k elements on page (yes, we’re working on reducing this 😊) but this is nowhere near the ~38k being reported as node objects. 

 

I’d be happy to document whatever we discuss and contribute back.  Current documentation is good for laying out the high level tooling and components, but stops short of diving into the next level of detail which I feel would really help equip devs to fish on their own.  Happy to jump on a call anytime so please let me know. 

 

Thanks

-DJ

 

From: Daniel Bratell [mailto:bra...@opera.com]
Sent: Wednesday, September 6, 2017 2:27 AM
To: Etienne Bergeron <etie...@google.com>; 'DJ Park' via memory-dev <memor...@chromium.org>; DJ Park <djp...@microsoft.com>
Cc: Erik Chen <erik...@chromium.org>; Wez <w...@chromium.org>; Kentaro Hara <har...@chromium.org>; dsk...@chromium.org
Subject: Re: Investigating memory bloat in Chromium

 

Yes, it was the Node object count that caught my eye. 

 

I don't think there is a comprehensive list of indirect memory use per Element type. I know that you need to be careful with video, audio, canvas and maybe img elements.

 

The connection between what devtools reports and actual memory used is indeed a bit loose, but there is often a correlation. If you can debug from devtools, which has rather easy to use tools compared to the C++ side, then maybe you can figure out if something is wrong in the web page itself.

 

Sorry for being a bit abstract. I wish I could give some easy suggestions but figuring out the cause of high memory usage remains a complicated problem where few instances are identical.

 

/Daniel

 

On Tue, 05 Sep 2017 23:39:58 +0200, DJ Park <djp...@microsoft.com> wrote:

 

Thanks for the follow up Daniel. 

 

When you refer to Node objects, are you referring to the object_count listed under “counter” column?  I’m seeing 38,530 in the trace I last sent so want to make sure we’re looking at the same thing. 

 

We’re currently using Angular which makes heavy use of scripts for templates.  We also use inline svg’s (which increase DOM complexity), have a fair amount of images, make use of canvas in several parts of the app, etc.  Is there any documentation on DOM elements that typically end up owning a lot of memory beyond themselves?  Would be great to understand which areas to focus on to get the most bang. 

 

As for using the devtool memory tools (heap snapshot, allocation profile/timeline), my understanding is that this is just giving me a breakdown of JS heap and not the native mem usage related to various components.  Is there a way to attribute native mem usage to the JS objects that I’m missing?  

 

cid:image016.png@01D326E2.73C2D890

DJ Park

unread,
Sep 29, 2017, 11:34:40 AM9/29/17
to Daniel Bratell, Etienne Bergeron, 'DJ Park' via memory-dev, Erik Chen, Wez, Kentaro Hara, dsk...@chromium.org, Niraj Khanchandani

Hi all,

 

Just ran into a case where our app native memory bloated to 600mb+ and noticed some interesting changes from the previous trace I shared.  Note that this trace is taken from the electron version of our app so may have slightly different characteristics from the pure web trace.  I need to figure out how to get a symbolized trace using electron but in the meantime would be great to see if anyone has any insights with the following:

 

1.       Memory under cc category is abnormally high with image cache being at 156mb (previous trace I sent 51mb).  Is there any way to track down what images are being cached? 

 

Machine generated alternative text: Swapped 0.0 MiB 0.0 MiB 0.0 MiB 0.0 MiB 0.0 MiB 0.0 MiB Component details Component v V CC v mage_memory v cache Ox2297BE57080 Overview Private dirty v -1.4 MiB -0.5 MiB 2.0 MiB 0.0 MiB 9.5 MIB -0.5 MiB 0.2 MIB 1. 0.0 MiB -1.1 MiB 0.2 MIB 7.7 MIB 0.0 MiB 0.2 MiB -9.1 MiB 0.3 MiB -12.7 MiB 0.0 KIB 0.0 KIB 0.0 KIB 0.0 KIB 8.0 KIB 0.0 KIB 8.0 KiB 16.0 KIB 0.0 KIB 16.0 KiB 16.0 KIB 0.0 KIB 16.0 KiB 4.0 KIB 0.0 KIB 4.0 KiB 4.0 KIB 0.0 KIB 4.0 KiB 16.0 KIB 0.0 KIB 16.0 KiB 4.0 KIB 0.0 KIB 4.0 KiB 16.0 KIB 0.0 KIB 16.0 KiB 4.0 KIB 0.0 KIB 4.0 KiB 16.0 KIB 0.0 KIB 16.0 KiB 4.0 KIB 0.0 KIB 4.0 KiB 16.0 KIB 0.0 KIB 16.0 KiB blink_gc 1.9 MiB 67.8 MiB 71.7 MiB cc 1.2 MIB 19.6 MiB MiB 187.8 MiB discardable 0.0 MiB 0.0 MiB A 0.0 MiB 0.0 MiB dom storage 0.2 MiB effective font size caches 0.0 MiB 0.0 MiB 1 MiB 1 MiB 0.0 MiB 2.1 MiB 2.0 MiB 2.2 MIB 6.3 MiB 111 leveldb 7.7 MiB malloc 40.4 MiB 13.6 MiB 13.6 MiB 81.6 MiB 233.6 MiB 382.8 MiB net 0.3 MiB 0.3 MiB partition_alloc 1.3 MiB O MiB 59.6 MiB 61.9 MiB size 170,696.0 KIB 156,840.0 KIB 156,840.0 KIB 156,840.0 KIB Skia 0.4 MiB sqlite 0.2 MiB web 10.0 MiB 28.9 MiB 343.4 MiB 391.9 MiB cache 0.0 MiB 0.0 MiB 0.0 MiB 0.0 MiB Help tracing 2.1 MiB 0.8 MiB 0.8 MiB 1.4 MiB 11.1 MiB 16.3 MiB locked size 170,696.0 KIB 156,840.0 KIB 156,840.0 KIB 156,840.0 KIB v cached mage mage mage mage mage mage mage mage mage mage mage mage 101 1 04 106 107 100 id 510 102 id 51 103 id 510 105 id 516 id id 108 id 520 109 id 520 id 512 id 516 514 514 10 id 52

 

  1. Malloc is at 233.6mb total with 133mb of unspecified allocated objects and 106mb of fragmentation.  (previous trace I sent had malloc at 40mb total so this is a huge increase).  Any tips on how we could be avoiding the large amount of fragmentation? 

 

Machine generated alternative text: Component details Component v v malloc v allocated_objects <unspecified> tracing_overhead v suballocations Skia v8 metadata_fragmentation Overview Total resident v 84.2 MiB 47.5 MiB -0.5 MiB 2.0 MiB 53.1 MiB -0.5 MiB 0.2 MIB 165.3 MiB 7.7 MIB 873.7 MiB 1,223.8 MiB 0.0 KIB 0.0 KIB 0.0 KIB 16.0 KIB Peak total resident V PSS v 62.5 MiB 20.2 MiB 24.4 MiB 109.5 MiB 825.4 MiB 1 ,042.o MiB Private dirty 4 MiB 1 MiB 1 MiB Swapped 0.0 MiB 0.0 MiB 0.0 MiB 0.0 MiB 0.0 MiB 0.0 MiB blink_gc 1.9 MiB 67.8 MiB 71.7 MiB cc 1.2 MIB 19.6 MiB 166.7 MiB 187.8 MiB discardable 0.0 MiB 0.0 MiB A 0.0 MiB 0.0 MiB dom storage 0.2 MIB 0.2 MiB size font 147.9 MiB 68.1 MiB 75.1 MiB 419.3 MiB 1,101.1 MiB 1,811. 4 MiB caches -1. -12.7 MiB caches 0.0 MiB 0.0 MiB 1 MiB 1 MiB 0.0 MiB 2.1 MiB 2.0 MiB 2.2 MIB 6.3 MiB leveldb 7.7 MiB malloc 40.4 MiB 13.6 MiB 13.6 MiB 81.6 MiB 382.8 MiB net 0.3 MiB 0.3 MiB partition_alloc 1.3 MiB 1.0 MiB 59.6 MiB 61.9 MiB Help skia v • 0.0 MiB 0.0 MiB 0.0 MiB 0.3 MiB 0.4 MiB effective size 239,202.1 KIB 133,179.3 KIB 133,179.3 KIB 0.0 KiB 106,022.8 KIB virtual size 292,040.0 KIB 250,916.0 KIB 893.2 KIB 133,179.3 KIB 11,377.7 KiB 336.2 KIB 320.2 KIB 106,022.8 KIB object_count v 1 ,929, 100.000 1 ,929, 100.000

 

Thanks

-DJ

Reply all
Reply to author
Forward
0 new messages