Handling of security bugs is flawed

[Rafael Gieschke]

I am sorry for having to write this email. I would have preferred
Mozilla to fix this internally. I will try to hide any bug details as I
still think that there is no use in disclosing these bugs publicly.

I have reported two security bugs (1226977, 1226979) in Firefox 80 days
ago on November 22, 2015 and sent notification emails about them to
secu...@mozilla.org on November 22, 2015, December 2, 2015, and January
6, 2016 (the last one including Chris Beard to make management aware
that something is wrong at Mozilla). Apart from auto-replies ("[...]
we'll investigate and follow up
with your message shortly."), I only received one answer to my second
email stating that some developers would not hopefully have a look at
them. Though, these bugs have not even been confirmed by Mozilla by now!

I think it is not enough to say "We are a foundation, we are good!" and
the Mozilla Mainfesto ("Individuals' security [...] must not be treated
as optional.") becomes total marketing slang if it has no defined and
verifiable practical consequences. One bug I reported is related to
software of another (commercial) software vendor, who has confirmed and
discussed it within 24 hours after reporting (despite the bug most
actually being in Firefox) without having a Mozilla Manifesto.

Neither "Handling Mozilla Security Bugs" policy
(<https://www.mozilla.org/en-US/about/governance/policies/security-group/bugs/>)
nor the "Client Bug Bounty Program"
(<https://www.mozilla.org/en-US/security/client-bug-bounty/>) include
any timeframe but I believe 80 days for even trying to confirm a
security bug is way too much. I guess it would be a good idea to include
specific timeframes in the policy, like a default of 7 days for
confirming and 60 days of fixing security bugs. To make this verifiable
by public, Bugzilla could send reminders (only including bug numbers)
about security-sensitive bugs that are still unconfirmed/unfixed after
these timeframes to a public mailing list, where at least a target date
for their confirmation/resolution could be discussed.

Needless to say, it is also not very satisfying to spend time on writing
a reduced test case and reporting security bugs and to then have to
spend even more time on keeping the issue in your mind and making
Mozilla even look at it but face total ignorance.

Best regards,
Rafael

[Boris Zbarsky]

On 2/10/16 6:08 AM, Rafael Gieschke wrote:
> I am sorry for having to write this email.

Rafael,

I'm also sorry you had to write this...

There should have been someone watching this component, and I'm sorry
there wasn't (largely because the relevant code is not really actively
maintained, unfortunately). This is why we're trying to get explicit
triage rotations set up for all components right now. I believe that if
that were already in place three months ago, this would not have happened.

I don't know why the setup we have in place for explicitly triaging
security bugs failed in this case, or what happened with the mails to
secu...@mozilla.org, since I'm not privy to the former nor on the
latter list. Something clearly failed badly there. :(

In any case, thank you very much for both filing the bugs and for
bringing this issue up. For these two particular bugs, we're going to
figure out who can look at them who somewhat understands the relevant
code and has time; this will likely take a few days to sort out. For
the general problem, I think the regular triage rotations will give us a
much better handle on things and will prevent things from falling
through the cracks like this.

-Boris

[Al Billings]

We do our best to triage all new security bugs in a timely fashion.
These bugs were no exception. They were assigned a sec-moderate rating
as they present a limited risk and were added to our bug-fix queue.

Mozilla has limited engineering resources, and we use these security
ratings to guide which bugs we work on. The queue of sec-moderate bugs
is always being worked on, and unfortunately, we just haven’t gotten to
these yet.

Al Billings
Platform Engineering Security Program Manager

[Rafael Gieschke]

On 02/11/2016 01:28 AM, Al Billings wrote:
> We do our best to triage all new security bugs in a timely fashion.
> These bugs were no exception. They were assigned a sec-moderate rating
> as they present a limited risk and were added to our bug-fix queue.

Thanks for these explanations but I am sorry to disagree.

You only added the sec-moderate rating (which I do not share for reasons
I will give in the bugs to not disclose bug details here) on February 5,
2016 without any further explanation and without even reproducing the
bug. I do not see how you can assign a definite severity rating without
even reproducing and confirming a bug (and these bugs can be reproduced
within 1 minute!).

Apart from that, these bugs were thus 75 days without a security
severity rating and in UNCONFIRMED status. So, it is a "timely fashion"
and "no exception" to let new security bugs stay untriaged for 75 days?

> Mozilla has limited engineering resources, and we use these security
> ratings to guide which bugs we work on. The queue of sec-moderate bugs
> is always being worked on, and unfortunately, we just haven’t gotten to
> these yet.

Again, I am sorry but I think, even for sec-moderate bugs (which can be,
e.g., something like "Disclosure of entire browsing history"), working
on security bugs *as time permits* (and not by setting target dates or
target versions) is the very definition of "treating security as
optional". I do not suggest that all bugs have to be fixed instantly (or
the fix being merged into mozilla-beta or mozilla-aurora) but you have
to set yourself a goal which cannot only be based upon available
resources. With Mozilla engaging into Connected Devices, will it soon
be: "Sorry, your smartwatch does disclose your location history to web
pages, we will fix that as soon as we have enough resources."?

Additionally, this is not really about fixing but but mainly about
confirming private bugs. The whole idea of reporting bugs privately to
vendors is that you share the bug with as few people as possible. This
makes it impossible to discuss the bug publicly or lobby people to look
at it (or, e.g., vote on it). The responsibility of the vendor is, thus,
to at least react to reported bugs (this is true for every information
shared via secu...@mozilla.org) and confirm them as soon as possible. I
did not find any metrics on this by a quick search, but I think 75 days
to confirm a bug should not even be something you aim for for a public bug!

Best regards,
Rafael

[Rafael Gieschke]

On 02/10/2016 09:10 PM, Boris Zbarsky wrote:
> I don't know why the setup we have in place for explicitly triaging
> security bugs failed in this case, or what happened with the mails to
> secu...@mozilla.org, since I'm not privy to the former nor on the
> latter list. Something clearly failed badly there. :(
>
> In any case, thank you very much for both filing the bugs and for
> bringing this issue up. For these two particular bugs, we're going to
> figure out who can look at them who somewhat understands the relevant
> code and has time; this will likely take a few days to sort out. For
> the general problem, I think the regular triage rotations will give us a
> much better handle on things and will prevent things from falling
> through the cracks like this.

Thanks and thanks for looking at the bugs. That does sound great and
yes, I think it will indeed help to handle all bugs in time. Still, I
think at least something about an expected reaction time should be added
to, e.g. the bug bounty document. Right now, you have zero indication in
these documents after which time you have to worry about reported bugs
not being noticed.

Best regards,
Rafael