Hi, DMPer, Whether this tool is still maintained? Or be replaced with other better one?

[WenSheng He]

Since @Dai has gone, this tool seems not change any more, like 'dmprof cat' still not release yet.

I wonder whether this tool is still maintained? Or be replaced with other better one?

As in my team, this tool is still the best one to find memory consumption and leak location.

Recently we have some patches, especially for the performance & memory for dmprof itself.

[WenSheng He]

Here is one patch:

https://code.google.com/p/chromium/issues/detail?id=489991

[Primiano Tucci]

Don't know if any of the maintainers of DMP is still around.

We in Chrome are now working on a new memory profiling infrastructure integrated with chrome://tracing. That will give the benefit of contextual information (we are instrumenting most of the memory subsystems in chrome), no need of weird rebuild/symbolization steps and the integration with timeline events (in order to correlate which memory component grew up with what was running while it happened).

You can already see some instrumentation being there if you get a bleeding edge ToT build and enable the "memory-infra" category in chrome://tracing. It is very new and a lot of work is in progress.

I will create public wiki pages on chromium.org and make a formal announcement soon (in ~weeks). Stay tuned.

--
You received this message because you are subscribed to the Google Groups "dmprof" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dmprof+un...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/dmprof/3206329e-238e-4863-bb0b-a676b3e1f9ab%40chromium.org.

[Zhenyu Shan]

So will DMP be completely obsolete?

In fact I doubt if the memory-infra can provide deep enough information like stacktrace. I do think rebuild/symbolization is weird and time-consuming, but also effective finding real problem :)

Anyway a new and easy-to-use tool is always welcomed. Waiting for the wiki page and will give it a try.

BTW: I think I've seen something about tools/memory_inspector as a new memory profiling tool. What's this memory_inspector's status and what's the difference between this and memory-infra?

Thank you!

[Primiano Tucci]

So will DMP be completely obsolete?

I can't answer on this. I personally don't have / not aware about any plans on that.

 

In fact I doubt if the memory-infra can provide deep enough information like stacktrace. I do think rebuild/symbolization is weird and time-consuming, but also effective finding real problem :)

This really depends on the problem you are trying to solve.

If you get a chrome process which uses 500M of memory, can you tell where that memory is going? Which subsystem did allocate it? Who is using it? Under which circumstance was allocated (did memory grow while loading a page or just at startup? did memory usage stay up after navigating away from the page?).

Furthermore, most of the big consumers of memory today involve complex cross-process sharing and management (see discardable memory, see graphics tiles jumping across browser <> renderer <> gpu processes). How can you account for all these in DMP?

Furthermore, IIRC DMP is based on TCMalloc instrumentation. As far as I am aware, most of the allocations, especially in the renderer, use ad-hoc allocators (PartAlloc, Oilpan, v8). How do they map in DMP?

The last time I used DMP was ~years ago. Unless very few cases where one has a specific idea about what to look for, from what I remember DMP ends up classifying most of the allocations as "unkooked".

 

BTW: I think I've seen something about tools/memory_inspector as a new memory profiling tool. What's this memory_inspector's status and what's the difference between this and memory-infra?

Memory Inspector was the first attempt, from my side, to build some unified tooling for Android. most of its core functionalities are being migrated to chrome://tracing.

 

Thank you!

Regards,

Primiano

 

On Wednesday, May 20, 2015 at 4:30:41 PM UTC+8, Primiano Tucci wrote:

Don't know if any of the maintainers of DMP is still around.

We in Chrome are now working on a new memory profiling infrastructure integrated with chrome://tracing. That will give the benefit of contextual information (we are instrumenting most of the memory subsystems in chrome), no need of weird rebuild/symbolization steps and the integration with timeline events (in order to correlate which memory component grew up with what was running while it happened).

You can already see some instrumentation being there if you get a bleeding edge ToT build and enable the "memory-infra" category in chrome://tracing. It is very new and a lot of work is in progress.

I will create public wiki pages on chromium.org and make a formal announcement soon (in ~weeks). Stay tuned.

On Tue, May 19, 2015 at 3:24 AM, WenSheng He <wensh...@samsung.com> wrote:

Since @Dai has gone, this tool seems not change any more, like 'dmprof cat' still not release yet.

I wonder whether this tool is still maintained? Or be replaced with other better one?

As in my team, this tool is still the best one to find memory consumption and leak location.

Recently we have some patches, especially for the performance & memory for dmprof itself.

--
You received this message because you are subscribed to the Google Groups "dmprof" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dmprof+un...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/dmprof/3206329e-238e-4863-bb0b-a676b3e1f9ab%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "dmprof" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dmprof+un...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/dmprof/0446f5eb-fc0c-48b3-81f3-0931bc5cdb9d%40chromium.org.

[JungJik Lee]

Hi, Primiano

I have some questions about Memory Inspector.

Are you going to migrate all Memory Inspector features to chrome://tracing?

If so, what will happen to Memory Inspector? going to integrate into chrome:tracing?

Will it (chrome://tracing) also support libheap_profiler in chrome://tracing?

And in your question, was it intent to use stack frame for analyzing n_heap like DMP (nheap-android.py)?

Thanks you~

2015년 5월 20일 수요일 오후 6시 12분 27초 UTC+9, Primiano Tucci 님의 말:

[Primiano Tucci]

Hello

> Are you going to migrate all Memory Inspector features to chrome://tracing?

Yes. Probably the only feature which will not be there is the "task manager mode" where you just see the list of processes and their global stats. That doesn't fit easily in the tracing model. On the other side, it is also the least interesting I think

> If so, what will happen to Memory Inspector? going to integrate into chrome:tracing?

Yes, that is the plan. It will take some time as in this first round we are getting some metrics about all the memory subsystem firsts (Various allocators, various gpu buffers etc) as those are the most difficult to get by other means. Neither DMP nor any other stack-base tool can have an understanding of how, for instance, gpu memory moves between gpu, browser and renderer processes.

Similary more tricky subsystems (E.g. v8) which do a big mmap reservations at the beginning and then manage that memory on their own.

For all these cases we need those subsystems to tell us our details, so we can track their evolution over time.

Once we get that we can go deep down into stack traces. I think the first step there would be pseudo-stack coming from tracing (they are more human friendly and can be get in production) and as a last restort actual stack traces (They are tricky, require to rebuild chrome + have symbols and lot of extra black magic needs to happen)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/dmprof/7ffd2e3d-96f2-4f66-bb35-e046f028e4c1%40chromium.org.

[JungJik Lee]

Hi,

Thanks for your answer.

2015년 6월 8일 월요일 오후 9시 37분 44초 UTC+9, Primiano Tucci 님의 말:

Hello

> Are you going to migrate all Memory Inspector features to chrome://tracing?

Yes. Probably the only feature which will not be there is the "task manager mode" where you just see the list of processes and their global stats. That doesn't fit easily in the tracing model. On the other side, it is also the least interesting I think

> If so, what will happen to Memory Inspector? going to integrate into chrome:tracing?

Yes, that is the plan. It will take some time as in this first round we are getting some metrics about all the memory subsystem firsts (Various allocators, various gpu buffers etc) as those are the most difficult to get by other means. Neither DMP nor any other stack-base tool can have an understanding of how, for instance, gpu memory moves between gpu, browser and renderer processes.

Is it possible to hook the gpu memory by stack-base tool? I don't know well. But especially in android system, surface manager allocates some gpu memory and share the memory with chrome(other apps) process.

so If we don't track in system globally, we couldn't know how large they are.

we analyse them approximately from proc/*/maps (such as kgsl / dma buffer / ion ). so I think we might need to combine the stack frame result with memdump or use both. (stack frame of hooked area and  memdump of unhooked)

And could I ask one more? what is the category for profile_chrome.py / memory_inspector? How could I use it now?

Thank you in advance.

[Primiano Tucci]

Is it possible to hook the gpu memory by stack-base tool? I don't know well. But especially in android system, surface manager allocates some gpu memory and share the memory with chrome(other apps) process.

so If we don't track in system globally, we couldn't know how large they are.

we analyse them approximately from proc/*/maps (such as kgsl / dma buffer / ion ). so I think we might need to combine the stack frame result with memdump or use both. (stack frame of hooked area and  memdump of unhooked)

And could I ask one more? what is the category for profile_chrome.py / memory_inspector? How could I use it now?

It is extremely tricky and I don't see that possible with a non-informed tool (Which looks only at stacks). That's one of the major point of chrome://tracing: creating *informed* profiling tools.

GPU memory is tough: it is allocated on one process, and used on others. Most of the non-informed tools will, in the best cast, double-report the memory, which is not what you want (but is what you get if you look at mmaps).

Very similar problems happen with shared-memory and discardable memory.

This public design doc from chrome://tracing gives an explanation of the challenges and how we are going to deal with that.

The other challenges here are that, on android, account is even more complicated and depends on the specific drivers.

Some drivers don't mmap anything in the process space, but just return handles to stuff that is handled by the kernel space driver. This can be accounted only by looking at the Android memtrack module.

Others, instead, leave gigantic mmaps in the process address space, even when the not using any memory (you might have seen gigantic anon_inode:dmabuf).

I think that the first step forward here is: report the memory that we know / expect to be using from chrome (See crbug.com/483467) and once we have that data we can look at the system and capture the deltas coming from the driver internal behavior.

And could I ask one more? what is the category for profile_chrome.py / memory_inspector? How could I use it now

The category is memory-infra (disabled-by-default) in chrome://tracing (you need a bleeding edge version of chrome, both desktop and mobile). Expect breaking changes until next week. Should become more stable then (I'll make a formal announcement then). We are developing that in these days.

 

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/dmprof/b9b29218-dbf8-4c50-a5d5-910a9ede0fe1%40chromium.org.