how to parse perfetto dump pb file

[Fengjie Chen]

hello google perfetto team

As you know, if enable continuous dump, perfetto will dump more pb files.

I know we can open it on https://www.speedscope.app/. The page is callstack result of one raw-trace snapshort. But I want to find pb file which has biggest heap size usage and then parse it.

Do you know how to parse it and get callstack malloc size by python script?

Is there already has an open source tool/script do this?

====Following is my command=== 

python heap_profile.py -n vendor.qti.camera.provider-service_64 --all-heaps -c 500 -i %sampling_interval%  -o %NOW_TIME%

===perfetto dump file list with continuous dump enabled ===

...

-a---           2024/9/10    17:14          85519 heap_dump.9.11164.libc.malloc.pb
-a---           2024/9/10    17:14         579932 heap_dump.90.11164.libc.malloc.pb
-a---           2024/9/10    17:14         580215 heap_dump.91.11164.libc.malloc.pb
-a---           2024/9/10    17:14         581252 heap_dump.92.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582000 heap_dump.93.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582229 heap_dump.94.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582509 heap_dump.95.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582943 heap_dump.96.11164.libc.malloc.pb
-a---           2024/9/10    17:14         583358 heap_dump.97.11164.libc.malloc.pb
-a---           2024/9/10    17:14         583536 heap_dump.98.11164.libc.malloc.pb
-a---           2024/9/10    17:14         583836 heap_dump.99.11164.libc.malloc.pb
-a---           2024/9/10    17:14        7368166 raw-trace

[Lalit Maganti]

On Tue, 15 Oct 2024, 11:12 'Fengjie Chen' via Perfetto Development - www.perfetto.dev, <perfet...@googlegroups.com> wrote:

hello google perfetto team

As you know, if enable continuous dump, perfetto will dump more pb files.

I know we can open it on https://www.speedscope.app/. The page is callstack result of one raw-trace snapshort. But I want to find pb file which has biggest heap size usage and then parse it.

As an aside not sure why you are using speedscope here, ui.perfetto.dev will also parse these traces just fine. 

Do you know how to parse it and get callstack malloc size by python script?

Is there already has an open source tool/script do this?

Have you considered instead of using continuous dumps to enable the "dump at max" option of heap profiling? If all you care about is the heap profile at the maximum memory use, that will save you all the effort of having to parse the traces and find the maximum. 

But if you do want to parse the traces you can use trace processor (https://perfetto.dev/docs/analysis/trace-processor). There is a Python library for example (https://perfetto.dev/docs/analysis/trace-processor-python).

You can use a query something like: `select ts, sum(size) from heap_profile_allocation group by ts` to get a sense of how big the heap size is.

====Following is my command=== 

python heap_profile.py -n vendor.qti.camera.provider-service_64 --all-heaps -c 500 -i %sampling_interval%  -o %NOW_TIME%

===perfetto dump file list with continuous dump enabled ===

...

-a---           2024/9/10    17:14          85519 heap_dump.9.11164.libc.malloc.pb
-a---           2024/9/10    17:14         579932 heap_dump.90.11164.libc.malloc.pb
-a---           2024/9/10    17:14         580215 heap_dump.91.11164.libc.malloc.pb
-a---           2024/9/10    17:14         581252 heap_dump.92.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582000 heap_dump.93.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582229 heap_dump.94.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582509 heap_dump.95.11164.libc.malloc.pb
-a---           2024/9/10    17:14         582943 heap_dump.96.11164.libc.malloc.pb
-a---           2024/9/10    17:14         583358 heap_dump.97.11164.libc.malloc.pb
-a---           2024/9/10    17:14         583536 heap_dump.98.11164.libc.malloc.pb
-a---           2024/9/10    17:14         583836 heap_dump.99.11164.libc.malloc.pb
-a---           2024/9/10    17:14        7368166 raw-trace

--
You received this message because you are subscribed to the Google Groups "Perfetto Development - www.perfetto.dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to perfetto-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/perfetto-dev/da826634-ae5e-45ab-9337-2abc7d8b26a9n%40googlegroups.com.