[call for status]: May 2021 status update!

[William Kahn-Greene]

Hi!

If you work on crash reporting related things or a crash reporting related project, please reply to this email before Friday, May 7th if you have status updates. Short with outlines is fine. Long is fine. If you already have some kind of status newsletter, feel free to reply with a link to that. I'll take all these replies and compile a newsletter update thing.

/will

Questions:

What's your team/project?

What did you accomplish? (Descriptions, bug numbers, etc)

What are you working on now or think you'll have done this month? (Descriptions, bug numbers, etc)

What do you need help with?

What are you concerned about?

What else do you think is helpful for everyone to know?

[Gabriele Svelto]

Hi Will,
here's my status update. I've added the work that Alexis and Alexandre
also did in the last few weeks since I don't know if they're in the WG
mailing list yet.

* Windows Error Reporting interception landed last month and can
intercept all main process crashes we were previously missing. This
includes __fastfail() crashes, catastrophic OOM crashes, weird DLL
injections and very late shutdown crashes. It significantly increased
nightly crash rate which is good! Content process support is being
worked upon.

* I've augmented Socorro's stack walker to print out richer information
for Windows:
* Unloaded modules are now printed out if they're present (visible in
the raw crash data only for now)
* Authenticode signatures have been fixed and are retrieved for
unloaded modules too
* __fastfail() crashes now print out their underlying reason (like
stack smashing or invalid control flow)
* All NTSTATUS errors are printed out correctly, both if they're the
crash reason or the last error value of a thread

Alexis Beingessner worked on numerous topics regarding Breakpad oxydation:

* She added MISC_INFO_5 to the rust-minidump crate. Which enables it to
inspect special processor status such as Intel CET [1]
* She's adding compact unwinding info support to Symbolic which will
dramatically improve our stack traces on macOS
* She started rewriting Breakpad unwinder in Rust

Alexandre modified the way we handle out-of-memory crash annotations so
that it will never be missing again.

Last but not least Steven Michaud - a former mozillian now contributing
on his own time - added __crash_info support to the stackwalker on
macOS. This allows us to inspect Apple-specific error messages that
might be present in minidumps.

Gabriele

[1]
https://software.intel.com/content/www/us/en/develop/articles/technical-look-control-flow-enforcement-technology.html

Il 30/04/2021 14.43, William Kahn-Greene ha scritto:
> Hi!
>
> If you work on crash reporting related things or a crash reporting

> related project, please reply to this email before *Friday, May 7th* if

> you have status updates. Short with outlines is fine. Long is fine. If
> you already have some kind of status newsletter, feel free to reply with
> a link to that. I'll take all these replies and compile a newsletter
> update thing.
>
> /will
>
>
> Questions:
>
> What's your team/project?
>
> What did you accomplish? (Descriptions, bug numbers, etc)
>
> What are you working on now or think you'll have done this month?
> (Descriptions, bug numbers, etc)
>
> What do you need help with?
>
> What are you concerned about?
>
> What else do you think is helpful for everyone to know?
>

> --
> You received this message because you are subscribed to the Google
> Groups "crash-reporting-wg" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crash-reporting...@mozilla.com
> <mailto:crash-reporting...@mozilla.com>.
> To view this discussion on the web visit
> https://groups.google.com/a/mozilla.com/d/msgid/crash-reporting-wg/CAKnh9qipz7yWo%2BmwxgF_OUU-mpjhV4-fUHEY8wMrpR_4geRzsg%40mail.gmail.com
> <https://groups.google.com/a/mozilla.com/d/msgid/crash-reporting-wg/CAKnh9qipz7yWo%2BmwxgF_OUU-mpjhV4-fUHEY8wMrpR_4geRzsg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/a/mozilla.com/d/optout
> <https://groups.google.com/a/mozilla.com/d/optout>.

[William Kahn-Greene]

I maintain Socorro and Tecken.

In the last month:

• Accomplishments
  
• Added last error value to crash report view.
• https://bugzilla.mozilla.org/show_bug.cgi?id=1692312
• Redid process type support--now "parent" is the value for parent process crash reports and we're phasing out "browser".
• This makes it a lot easier to search for parent crashes and aggregations on process type work now.
• https://bugzilla.mozilla.org/show_bug.cgi?id=1701357
• Improved Extensions tab in report view.
  
• The Extensions tab now shows the extension name, whether it's a system extension, and its signed state.
  
• https://bugzilla.mozilla.org/show_bug.cgi?id=1629943
• https://bugzilla.mozilla.org/show_bug.cgi?id=974968
• Fixed Bugs API to support POST as well as GET.
• https://bugzilla.mozilla.org/show_bug.cgi?id=1282707
• If there are other APIs that would benefit by having POST support, let me know.
  
• Add search by major version for crash reports submitted after April 25th.
• Now you can do searches like "major_version = 88" and "major_version >= 88 and major_version < 90"
  
• https://bugzilla.mozilla.org/show_bug.cgi?id=1111612
• https://bugzilla.mozilla.org/show_bug.cgi?id=1401517
• All Super Search fields now have exists/does-not-exist filter.
  
• https://bugzilla.mozilla.org/show_bug.cgi?id=973894
• Added support for multiple processing pipeline rulesets
• The first non-default ruleset I wrote is "regenerate_signature" which just regenerates the crash signature. It takes 1/10 the time regular processing takes. I'll use this going forward to regenerate crash signatures after signature generation changes.
• We can use this infrastructure for additional processing as well. That's been something we've talked about over the years.
• https://bugzilla.mozilla.org/show_bug.cgi?id=1705469
• Released socorro-siggen 1.0.6 with signature generation changes and some fixes.
• Converted my Socorro Overview presentation into a blog post.
• https://bluesock.org/~willkg/blog/mozilla/socorro_overview_2021.html
• In process
• Finishing up the new symbolication API microservice: https://bugzilla.mozilla.org/show_bug.cgi?id=1636210
  
• Crash Stats fixes
  
• (Back burner): Better Java crash report signature generation: https://bugzilla.mozilla.org/show_bug.cgi?id=1541120
  
• Need help with
• Testing the new symbolication API. If you do anything with symbolication, please let me know.

[Alexis Beingessner]

I'm new to crash-reporting but I guess I work on symbolic and rust-minidump now?

As Gabriele mentioned I've been spearheading an effort I am calling Kill Breakpad,

which I have written up a metabug with a full description here:

https://github.com/getsentry/symbolic/issues/375

The core focus of this effort is

* Ted's rust-minidump (replacing breakpad's minidump parsing)

* Sentry's symbolic crates (replacing all the machinery to parse/manipulate debuginfo, already used by dump_syms).

Our first milestone is attempting to replace minidump-stackwalk with the rust-minidump's minidump-processor binary.

To this end I've done a bit of sporadic work on rust-minidump/symbolic to find my legs and get the ball rolling:

* https://github.com/luser/rust-minidump/pull/137 -- Taught rust-minidump to parse the MISC_INFO_5 format (and wrote tests/printing machinery for all the previous formats)

* https://github.com/luser/rust-minidump/pull/145 -- (in review) Upgraded the minidump-processor unwinder -- can now unwind with frame-pointers and scanning on x86 and x64

* https://github.com/luser/rust-minidump/pull/151 -- (in review) Upgraded the minidump-processor CLI frontend to match dump_syms, and taught it to generate a JSON version of its report (format is "whatever the layout of the current types are", to be iterated on over time)

And while this was not motivated precisely by Kill Breakpad, I have also done:

* https://github.com/getsentry/symbolic/pull/372/ -- (in review) Taught symbolic (and therefore dump_syms) how to dump Apple's Compact Unwinding (.__unwind_info) format into breakpad's format for x86/x64, as well as wrote up a very thorough description of the format (that is otherwise missing from llvm's implementation, which is the only existing documentation of the format). Ideally when this lands it will fix Bug 1691022 (x64 macos missing CFI on socorro).

-- Alexis Beingessner (@Gankra)

--

You received this message because you are subscribed to the Google Groups "crash-reporting-wg" group.

To unsubscribe from this group and stop receiving emails from it, send an email to crash-reporting...@mozilla.com.
To view this discussion on the web visit https://groups.google.com/a/mozilla.com/d/msgid/crash-reporting-wg/CAKnh9qipz7yWo%2BmwxgF_OUU-mpjhV4-fUHEY8wMrpR_4geRzsg%40mail.gmail.com.
For more options, visit https://groups.google.com/a/mozilla.com/d/optout.