Improving Talkback topcrash reports
Adam Guthrie
currently using the Talkback crash reporting tool as well as the
Talkback web app. on http://talkback-public.mozilla.org/. However,
Firefox 3 (and hopefully other Mozilla apps.) will be switching to
Breakpad, an new open-source crash reporting tool [0].
In addition to creating a new client for reporting crashes, a new web
app. will be created. There will be a new query tool to search for
crashes, as well as new pages that display topcrash reports [1].
Since we're completely redesigning things, we'll be able to fix and
improve upon everything that people dislike about the current topcrash
reports. There has already been some input from Jesse Ruderman and
Smokey Ardisson on what they'd like to see changed [2] [3], but we'd
love to hear from others, too.
So, if you have anything you'd like to see changed about how these crash
reports are displayed, or have any suggestions for how they could be
improved, here's your chance to let us know!
Regards,
Adam
[0] http://code.google.com/p/google-breakpad/
[1] http://talkback-public.mozilla.org/reports/firefox/
[2]
http://wiki.mozilla.org/Talk:Mozilla_QA_Community:Talkback_Report_Challenge_1
[3] http://wiki.caminobrowser.org/User:Sardisson/Crash_Analysis_UI
RyanVM
> As most of you all know, Firefox (and other Mozilla products) are
> currently using the Talkback crash reporting tool as well as the
> Firefox 3 (and hopefully other Mozilla apps.) will be switching to
> Breakpad, an new open-source crash reporting tool [0].
>
> In addition to creating a new client for reporting crashes, a new web
> app. will be created. There will be a new query tool to search for
> crashes, as well as new pages that display topcrash reports [1].
>
> Since we're completely redesigning things, we'll be able to fix and
> improve upon everything that people dislike about the current topcrash
> reports. There has already been some input from Jesse Ruderman and
> Smokey Ardisson on what they'd like to see changed [2] [3], but we'd
> love to hear from others, too.
>
> So, if you have anything you'd like to see changed about how these crash
> reports are displayed, or have any suggestions for how they could be
> improved, here's your chance to let us know!
>
> Regards,
> Adam
>
> [0]http://code.google.com/p/google-breakpad/
> [1]http://talkback-public.mozilla.org/reports/firefox/
> [3]http://wiki.caminobrowser.org/User:Sardisson/Crash_Analysis_UI
Having the topcrashes sorted by date in descending order instead of
ascending order would be nice. Makes it easier to quickly see what the
last build to experience a crash was.
Smokey Ardisson
thoughts about how I use, or would like to use, crash analysis
webtools (rather than my list of specific issues with Talkback[1]). I
started working on it, but I had some other work on a little thing[2]
that got in my way. Now that the excitement is over, I've spent a
little bit of time polishing up my thoughts on my crash analysis
workflow[3].
I've identified three different patterns/tasks I perform, one a
general "checking on crashes," one a "follow-up on a bug filed based
on crash analysis," and one a "verification of crash bugs" task. For
ease of reference, I'll post below the entirety of these tasks from
the wiki page[3]. I hope this is useful, and please let me know if
you have any questions/comments.
Best,
Smokey
A. On a weekly basis, I want to look at the accumulated crash stats
for that branch and
* Quickly see where the top crashes are occurring (crash
signatures)
* See which crashes are "increasing"
* From the above screen, access crash stacks for each crash
signature
o In some cases, sort the resulting crash stacks by OS/build
ID
o In some cases, weed out known crashes caused by older
versions of plug-ins or haxies
+ Often these stacks have a very generic frame 0
(e.g., libobjc.dylib) and a frame 1 that's unique to the crash; in a
perfect world, it would be great to be able to re-list and examine the
crashes with the generic frame 0 that don't have frame 1 that matches
the known crash (i.e., when looking for patterns among generic frame
0s, don't have to ever [waste time] looking at the ones already
matched to known crashers).
+ In a perfect world, have feedback people email these
people telling them their crash is caused by an old version of a plug-
in or by a haxie
* Possibly "compare" crash signatures to trunk and releases to see
if they're occurring there, too
* Cross-reference with any existing bugs (or recently closed
bugs)
If there are no existing bugs on a major-seeming crash, attempt to
reproduce from URLs and comments and file bug(s).
B. If I file a bug based on crash analysis, particularly one with a
large number of incidents but which I was unable to reproduce, I want
to
* Continue to monitor the crash signature for new incidents which
may provide URLs or comments that allow the bug to become reproducible
o Develop a quick query for this, with an open-ended end
date
* Search across products for incidents if there's reason to
believe it might be in shared code
C. If a bug filed based on crash analysis, or having a crash analysis
component, has been marked FIXED, I want to
* Verify that the crash signature disappears (or, in the case of
overly-generic crash signatures, declines significantly) in builds
following the checkin
* Have easy access to incidents of the crash signature from *only*
builds after the check-in of the fix, in order to check whether the
incidents appear to be the same crash or another crash whose top frame
happens to be the same signature
[1]http://wiki.caminobrowser.org/User:Sardisson/Crash_Analysis_UI
[2]http://www.caminobrowser.org/press/releases/camino1.5/
[3]http://wiki.caminobrowser.org/User:Sardisson/Crash_Analysis_Workflow