Crashpad and browser metrics reports

[tamash...@gmail.com]

Hello,

I am interested in finding more info on how Chrome collects and handles browser metrics and crashes info.
My system is a Windows 10 machine.

A. Browser Metrics:

1. What is the difference between C:\Users\username\AppData\Local\Google\Chrome\User Data\BrowserMetrics\BrowserMetrics-identifier.pma and C:\Users\username\AppData\Local\Google\Chrome\User Data\BrowserMetrics-spare.pma?
2. What is contained in each of them?
3. Which one is it used when navigating to chrome://histograms?
4. Under what circumstances do these files get sent out to Google servers?

B. Crashpad:

1. What data is contained in C:\Users\username\AppData\Local\Google\Chrome\User Data\CrashpadMetrics-active.pma?
2. How can I view the data inside that file?
3. Under what circumstances do these files get sent out to Google servers?

C. Misc:

1. What is the format of .pma files?
2. How can I view the content of these files?
3. When are the crash dump reports from C:\Users\username\AppData\Local\Google\Chrome\User Data\Crashpad\reports\*.dmp gets sent to Chrome servers?
4. If I enable automatic crash sending reports to Google, I see (chrome://crashes) that some reports are not sent and I still have to click the Send Now link there. How is this process automatic then?

[Scott Graham]

Hi Tamas,

On Thu, May 24, 2018 at 2:25 PM <tamash...@gmail.com> wrote:

Hello,

I am interested in finding more info on how Chrome collects and handles browser metrics and crashes info.
My system is a Windows 10 machine.

A. Browser Metrics:

1. What is the difference between C:\Users\username\AppData\Local\Google\Chrome\User Data\BrowserMetrics\BrowserMetrics-identifier.pma and C:\Users\username\AppData\Local\Google\Chrome\User Data\BrowserMetrics-spare.pma?

2. What is contained in each of them?

3. Which one is it used when navigating to chrome://histograms?

These files aren't anything to do with Crashpad, but I believe they correspond to the UMA_ macros in the Chrome, e.g. UMA_HISTOGRAM_COUNTS.

I believe they're double buffered, so one is from the current run (and being written to), and the other is the previous one that's being processed and potentially uploaded. (But that's just a guess, I'm not too familiar with this stuff.)

 

4. Under what circumstances do these files get sent out to Google servers?

The metrics are reported when the user has turned on "Automatically send usage statistics and crash reports to Google" in chrome://settings.

 

B. Crashpad:

1. What data is contained in C:\Users\username\AppData\Local\Google\Chrome\User Data\CrashpadMetrics-active.pma?

2. How can I view the data inside that file?
3. Under what circumstances do these files get sent out to Google servers?

You can see the full list that Crashpad captures at https://cs.chromium.org/chromium/src/third_party/crashpad/crashpad/util/misc/metrics.cc?q=misc/metrics.&sq=package:chromium&g=0&l=1. Generally it's doing things like counting how many times Crashpad gets into a particular state. For example, a crash being encountered, or a minidump written.

 

C. Misc:

1. What is the format of .pma files?

2. How can I view the content of these files?

I don't know; Brian might know if there's a documented format.

 

3. When are the crash dump reports from C:\Users\username\AppData\Local\Google\Chrome\User Data\Crashpad\reports\*.dmp gets sent to Chrome servers?

This is also keyed on the setting in chrome://settings.

 

4. If I enable automatic crash sending reports to Google, I see (chrome://crashes) that some reports are not sent and I still have to click the Send Now link there. How is this process automatic then?

Old reports aren't uploaded when you turn that on, only new reports that are captured after that point will be uploaded. Additionally, there's some throttling applied to not spam the server with crashes, so not every crash will be automatically uploaded, even when enabled.

 

--
You received this message because you are subscribed to the Google Groups "Crashpad-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crashpad-dev...@chromium.org.
To post to this group, send email to crashp...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/crashpad-dev/a5f58a80-7729-416e-be9e-ec87121b7450%40chromium.org.

[Brian White]

I am interested in finding more info on how Chrome collects and handles browser metrics and crashes info.
My system is a Windows 10 machine.

A. Browser Metrics:

1. What is the difference between C:\Users\username\AppData\Local\Google\Chrome\User Data\BrowserMetrics\BrowserMetrics-identifier.pma and C:\Users\username\AppData\Local\Google\Chrome\User Data\BrowserMetrics-spare.pma?

The "BrowserMetrics-identifier.pma" files are the actual written-to files.  One of them is live and any others are from previous runs that we'll be deleted once any final information is retrieved from them.

The "spare" file is an optimization.  it takes time to create and fully realize the file (done to minimize the possibility of disk errors while in active use) so one is created after Chrome has been running a short while.  This empty (all zeros) file will be used on the next run so as to not delay startup trying to create one.

 

2. What is contained in each of them?

3. Which one is it used when navigating to chrome://histograms?

The "live" (i.e. BrowserMetrics-identifier.pma) file has the data shown in chrome://histograms.

 

These files aren't anything to do with Crashpad, but I believe they correspond to the UMA_ macros in the Chrome, e.g. UMA_HISTOGRAM_COUNTS.

I believe they're double buffered, so one is from the current run (and being written to), and the other is the previous one that's being processed and potentially uploaded. (But that's just a guess, I'm not too familiar with this stuff.)

 

4. Under what circumstances do these files get sent out to Google servers?

The metrics are reported when the user has turned on "Automatically send usage statistics and crash reports to Google" in chrome://settings.

Accumulated (live) metrics data is sent periodically, typically about every 30 minutes, and buffered during shutdown.  The PMA files are generally "empty" (not zeros but no "unsent" data) after a normal shutdown procedure.  Only in the event of a program crash do they have unsent data.  Some time after startup, files from previous runs are processed and any final contents sent.

 

 

B. Crashpad:

1. What data is contained in C:\Users\username\AppData\Local\Google\Chrome\User Data\CrashpadMetrics-active.pma?

2. How can I view the data inside that file?
3. Under what circumstances do these files get sent out to Google servers?

You can see the full list that Crashpad captures at https://cs.chromium.org/chromium/src/third_party/crashpad/crashpad/util/misc/metrics.cc?q=misc/metrics.&sq=package:chromium&g=0&l=1. Generally it's doing things like counting how many times Crashpad gets into a particular state. For example, a crash being encountered, or a minidump written.

 

C. Misc:

1. What is the format of .pma files?

2. How can I view the content of these files?

I don't know; Brian might know if there's a documented format.

The top-level format is "PersistentMemoryAllocator":  No formal documentation but pretty simple.

https://cs.chromium.org/chromium/src/base/metrics/persistent_memory_allocator.cc

How histograms are allocated within it is more complicated but amounts to a "metadata" record and a "data" record, actually twice because there is both "logged" and "unlogged" (or "sent" and "unsent" if you prefer) versions.  Plus some optimizations like "delayed allocations".

https://cs.chromium.org/chromium/src/base/metrics/persistent_histogram_allocator.cc

 

 

3. When are the crash dump reports from C:\Users\username\AppData\Local\Google\Chrome\User Data\Crashpad\reports\*.dmp gets sent to Chrome servers?

This is also keyed on the setting in chrome://settings.

 

4. If I enable automatic crash sending reports to Google, I see (chrome://crashes) that some reports are not sent and I still have to click the Send Now link there. How is this process automatic then?

Old reports aren't uploaded when you turn that on, only new reports that are captured after that point will be uploaded. Additionally, there's some throttling applied to not spam the server with crashes, so not every crash will be automatically uploaded, even when enabled.

  Brian
  bcw...@google.com
-----------------------------------------------------------------------------------------
Treat someone as they are and they will remain that way.
Treat someone as they can be and they will become that way.

[Tamas Ionut]

Thanks a lot for the support so far!

I was wondering: is there any tool to read the contents of these .pma files? 

Regards,

Ionut

I was wondering if there is any tool to read the contents of these .pma files. 

[aaron...@viasat.com]

Hello,

I am investigating Chromium's metrics system for the chromium-forked Viasat Browser. I have many of the same questions Tamas did, so this thread was extremely helpful.

I'd like to +1 the question of "is there any tool to read the contents of these .pma files?"

I imagine there must be, but maybe it's closed-source. In tools/metrics/histograms/histograms.xml, there are many references to different views, checkboxes, options, etc. - are these in reference to some closed-source tool that Google employees have access to? Is there an open-source alternative?

Thanks a ton for any help you can give!

-Aaron