App Not Responding (ANR) Reporting

[Jim Chen]

Hi all,

Bug 833990 is tracking a new feature being developed that will report
App Not Responding (ANR) instances in Fennec to Mozilla. Although less
severe than crashes, ANRs are still unpleasant experiences to the user.
And similar to crash reports, these ANR reports will hopefully help
identify potential bugs.

Currently Mozilla receives ANR reports through the Google Play dashboard
which covers Firefox and Firefox Beta only. Separate ANR-reporting will
provide coverage to the Nightly and Aurora channels.

This feature is also being reviewed for security (bug 833991) and
privacy (bug 833995) concerns.

In terms of privacy, ANR reports are similar to crash reports. However,
whereas the user controls crash reporting per-crash, ANR reporting is
controlled by the global enable-telemetry setting.

One precedence in this area is the hang-reporting feature in desktop
Firefox Nightlies. Hang-reporting is also controlled by the global
telemetry setting. However, ANR-reporting provides more stack and log
information than hang-reporting does, so ANR reports could be considered
more sensitive and need finer control.

Feedback welcome!

Cheers,
Jim

[Robert Kaiser]

Jim Chen schrieb:
> Feedback welcome!

So, does that mean this will be another "black box" that our elaborate
existing stability tracking won't see, just like everything else stuffed
into telemetry at this time?
(As a side note, I surely would want this to change but I'm more than
maxed out by doing stability tracking, so I can't drive anything there
or do anything more than either rant on it every now and then or help
someone who is doing the work on how we can work together.)

What speaks against having this actually go into our crash tracking
system that already also tracks e.g. plugin "hangs"? What speaks against
having the users control what they send?

Robert Kaiser

[Taras Glek]

There are a few precedents for doing this sort of thing
a) cleanse the report of potentially sensitive data
b) batch these reports up and then popup a ui every time you send them
asking for confirmation(similar to crash reports)
c) make a new opt-in process for sending sensitive data by dedicated users

I suspect (a) is not viable here.
b) might work.
c) is what we do for shutdown tracking. It requires having a combination
of a special pref and an addon to be installed. See
http://benoitgirard.wordpress.com/2013/01/18/is-your-shutdown-slow-turn-on-the-performance-reporter/
The data is also submitted via a non-telemetry channel. Perhaps
developing a perf-watchdog for mobile is a sensible option. Might be
able to reuse the cleopatra serverside for this.

Taras



>
> Feedback welcome!
>
> Cheers,
> Jim
>

[Jim Chen]

On Wednesday, January 30, 2013 8:50:23 PM UTC-5, Robert Kaiser wrote:
> Jim Chen schrieb:
>
> > Feedback welcome!
>
>
>
> So, does that mean this will be another "black box" that our elaborate
>
> existing stability tracking won't see, just like everything else stuffed
>
> into telemetry at this time?
>
> (As a side note, I surely would want this to change but I'm more than
>
> maxed out by doing stability tracking, so I can't drive anything there
>
> or do anything more than either rant on it every now and then or help
>
> someone who is doing the work on how we can work together.)

Thanks for the feedback! When you say "stability tracking" do you mean just Socorro by itself? I'm not too familiar with how our telemetry data are used, but I think the data go to Metrics and are being tracked by Metrics.

> What speaks against having this actually go into our crash tracking
>
> system that already also tracks e.g. plugin "hangs"?

I think it's mostly for technical reasons. For Android at least, Breakpad is not used for ANR reporting. That means if we want to have the data sent to Socorro, we need to convert the data to something Socorro understands. That would involve significant effort, because Socorro is not made to work with something other than Breakpad, AFAIK. On the other hand, it's relatively easy to save the ANR data and have Telemetry send it to the Metrics server.

> What speaks against
>
> having the users control what they send?

That's up for discussion as part of the privacy review. Currently the ANR reporting feature follows Telemetry settings, which means it can be turned on and off globally.

[Robert Kaiser]

Jim Chen schrieb:

> Thanks for the feedback! When you say "stability tracking" do you mean just Socorro by itself? I'm not too familiar with how our telemetry data are used, but I think the data go to Metrics and are being tracked by Metrics.

What I'd doing is all based on Socorro data, yes. I'm probably the
person on paid staff most well-versed with what we are tracking and
analyzing in that data at this time.
Unfortunately, I have neither any idea what related information we have
within telemetry nor how we would get it linked to Socorro data or
analyzed for stability issues. I'd hope we'd do more of that but
unfortunately I myself am maxed out on time - recently even more than
before, so I'd need help there if we want to become more efficient in
reading and connecting the information we gather.
Oh, and we'll also collect some stability-related data with FHR, I'd
love to get that connected to our other indicators as well.

> I think it's mostly for technical reasons. For Android at least, Breakpad is not used for ANR reporting. That means if we want to have the data sent to Socorro, we need to convert the data to something Socorro understands. That would involve significant effort, because Socorro is not made to work with something other than Breakpad, AFAIK. On the other hand, it's relatively easy to save the ANR data and have Telemetry send it to the Metrics server.

For one thing, good to know. For the other, we also can possibly work
with the Socorro team on making it understand other data.
The latter won't work in the short term, but I'm, pretty sure they'll be
interested to work in that direction in the longer term. And I'd be very
interested in weaving our sources of information on stability together
more tightly (or at all).

>> What speaks against
>> having the users control what they send?
>
> That's up for discussion as part of the privacy review. Currently the ANR reporting feature follows Telemetry settings, which means it can be turned on and off globally.

I'd say it depends on how much privacy-relevant information is in those
reports. We tell people that telemetry data is anonymized, if this data
isn't, it probably needs more confirmation from the user.
That said, on Firefox OS we even default our prompts to let users opt
into always sending crash reports, which definitely can contain private
info.

Robert Kaiser