Intent to implement version 4 of the Safe Browsing protocol

[Francois Marier]

The Safe Browsing service we rely on for protection against malware and
deceptive sites is migrating to a new version of the Safe Browsing
protocol. Version 4 will enable Google to quickly send the most relevant
list entries to clients (based on platform and locale for example) as
well as deal with false positives in a more efficient way.

Meta bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1167038

Implementation plan:
https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation

Link to specification:
https://developers.google.com/safe-browsing/v4/update-api

Platform coverage: Desktop and Android


Estimated or target release:

We'll be rolling it out slowly in the pre-release channels and
monitoring it closely via Telemetry before we switch over. We intend to
have the existing V2 code running in parallel with the V4 code for a
little while. Google has agreed to run the old servers until we have
released an ESR which uses the version 4 servers (most likely ESR 59).


Preference behind which this will be implemented:

urlclassifier.phishTable and urlclassifier.malwareTable
https://wiki.mozilla.org/Security/Safe_Browsing/V4_Implementation#Notes


Do other browser engines implement this?

Chromium is working on it. They have landed large parts of it, but it
hasn't shipped to users yet.


Tests:

Our existing tests for version 2 of the protocol will be extended to
support version 4 too. We are also adding V4-specific tests.

https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/url-classifier/tests
https://hg.mozilla.org/mozilla-central/file/tip/testing/firefox-ui/tests/functional/security
https://hg.mozilla.org/mozilla-central/file/tip/browser/components/safebrowsing/content/test
+ manual smoke tests


Security and Privacy concerns:

The privacy characteristics of the new protocol are essentially the same
as the old protocol.
https://support.mozilla.org/en-US/kb/how-does-phishing-and-malware-protection-work#w_what-information-is-sent-to-mozilla-or-its-partners-when-phishing-and-malware-protection-are-enabled


For more information about Safe Browsing, see the wiki:
https://wiki.mozilla.org/Security/Safe_Browsing

[Mike Hommey]

On Tue, Aug 02, 2016 at 07:28:32AM -0700, Francois Marier wrote:
> The Safe Browsing service we rely on for protection against malware and
> deceptive sites is migrating to a new version of the Safe Browsing
> protocol. Version 4 will enable Google to quickly send the most relevant
> list entries to clients (based on platform and locale for example) as
> well as deal with false positives in a more efficient way.

I thought Intend to implement messages were "limited" to features
exposed to the web. Not that I'd mind for more Intend to implement, but
then don't we need to better define what they apply to?

Mike

[Francois Marier]

After a year's worth of development, bug fixes, and integration testing,
we are now ready to enable the latest version [1] of the Safe Browsing
API in Firefox 56, two releases ahead of schedule and only a few weeks
behind Chrome.

We do not expect any user-visible changes, but will be running an
experiment [2] on beta to compare crash rates between the two versions
of the API. After that, the feature will be rolled out to the release
population in stages [3].

I want to take this opportunity to thank Dimi Lee, Henry Chang and
Thomas Nguyen for spending so much time refactoring and eliminating
ancient code (some of it dating back to when Safe Browsing was a Google
extension for Firefox [4] or part of the Google Toolbar [5]). Thanks to
their hard work, we have not only eliminated crashes and fixed
intermittent test failures that have been around for many years, we now
also have a stable codebase and two new module peers [6] with deep
knowledge of this code.

Big thanks to the other Taipei folks who helped make this happen: Ethan
Tseng, Engineering Manager; Wesly Huang, EPM; and Cynthia Tang, QA
Engineer. Without the work of all of these people, we would not have
been able to complete the migration in time for Google's shutdown of the
old servers.

Francois

[1] https://security.googleblog.com/2016/05/evolving-safe-browsing-api.html
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1377267
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=1387651
[4]
https://web.archive.org/web/20051218171531/http://www.google.com/tools/firefox/safebrowsing/index.html
[5]
https://web.archive.org/web/20060412192055/http://tools.google.com/firefox/toolbar/
[6]
https://groups.google.com/d/topic/mozilla.governance/cF9MwHoQ09M/discussion

[Enrico Weigelt, metux IT consult]

On 16.08.2017 01:46, Francois Marier wrote:
> After a year's worth of development, bug fixes, and integration testing,
> we are now ready to enable the latest version [1] of the Safe Browsing
> API in Firefox 56, two releases ahead of schedule and only a few weeks
> behind Chrome.

How can I get rid of that ?
I don't want my browser to call google servers, nor do I want allow
google to decide what's malware and what's not.

--mtx

[Gijs Kruitbosch]

This and other queries like it are best asked and answered on
https://support.mozilla.org/ . Furthermore, just like with your
next-most-recent posts (about nsString and printf, and about CIDs), the
answer is already documented and easily found using a web search:

https://duckduckgo.com/?q=turn+off+firefox+safebrowsing&t=hb&ia=web
https://duckduckgo.com/?q=mozilla+nsstring+printf&t=hb&ia=web
https://duckduckgo.com/?q=cid+xpcom&t=hb&ia=qa

The first page of results for all these queries all have several
reasonable answers to them. Please use resources like web search, MDN or
support.mozilla.org first, instead of posting here.

I also believe your reflexive responses along the line of "how can I get
a version of Firefox that doesn't have X" for every X (you don't like)
that happens to come up in this group are unproductive. Presumably you
know that we have pretty decent code search tools at
https://dxr.mozilla.org/ and http://searchfox.org/, which can help you
answer questions like "where does this code live", "what prefs govern
it" and "is there a build option to not include it".

The answer to that last question is almost invariably "no". There are
usually prefs while features are being developed, but those will
frequently get removed when features are mature enough that we don't
think turning them off is web-compatible. Firefox and Gecko are
explicitly *not* aiming to have 42,000 build-time defines to remove
every conceivable feature that someone might not want. It's already been
established in previous threads that for your usecase (where you want to
not build lots of features), you basically need to fork Gecko. It helps
nobody to revisit that same subject whenever a feature you don't like
comes up, and effectively derails the relevant threads announcing
features or feature updates. Please stop doing that.

~ Gijs

[Enrico Weigelt, metux IT consult]

On 16.08.2017 12:40, Gijs Kruitbosch wrote:

> This and other queries like it are best asked and answered on > https://support.mozilla.org/ .

Unfortunately, it only tells how to switch some things off, but
not to remove it entirely. Neither does it tell anything about the
security implications of sending meta data to dubious corporations.

> Furthermore, just like with your > next-most-recent posts (about nsString and printf, and about CIDs),
the > answer is already documented and easily found using a web search:

If the docs wouldn't answer all my questions, I wouldn't have asked in
the first place. I'd volunteer to update the docs, but obviously I need
the proper information for that first.

https://developer.mozilla.org/en-US/search?q=printf

--> nothing about using printf() here :(

Regarding CID vs CONTRACTID - still haven't understood why CIDs are
random numbers, instead of human-readable names (similar to hierarchical
class names, eg. "org.mozilla.collections.array) ?

And still I find the naming "CID" (class-ID ?) vs "CONTRACTID" quite
confusing. Why not something like "INTERFACEID" or "PROTOCOLID" vs.
"SERVICEID" ?

The term "contract" isn't entirely obvious (to non-moz folks), it's
often used for interface (the way I can talk to something), instead
of a collection of interfaces / a service that might have multiple
interface.

> I also believe your reflexive responses along the line of "how can I get > a version of Firefox that doesn't have X" for every X (you don't
like) > that happens to come up in this group are unproductive.

This "reflex" comes from the tendency that more and more things are
added - even things that contradict the whole spirit of free software,
like despotic restriction management (even downloads the malware on
its own) - and then we downstreams (packagers, integrators, operators)
have the huge burden of getting these things under control.

Please also consider, that there's more than average John Doe user,
and there're lots of reasons for disabling things (not even compile
them in the first place). Limited resources may be one of them.
Security concerns, managebility, etc, may be others. One's killer
feature can be another one's misfeature. Therefore it's important
that things can be turned off / removed easily (the optimum would
be external components that can be deployed separately).

A bunch of configure options are currently broken, some can't be
easily repaired (w/o touching xpidlgen to add preprocessor).

> The answer to that last question is almost invariably "no". There are > usually prefs while features are being developed, but those will >
frequently get removed when features are mature enough that we don't >
think turning them off is web-compatible.

How exactly is "web-compatible" specified here ?
Does it include $megacorp's servers or automatically download
and execute arbitrary binaries or allow tracking users ?

> Firefox and Gecko are > explicitly *not* aiming to have 42,000 build-time defines to remove >
every conceivable feature that someone might not want.

Doesn't need to be that much. Less than hundred should be sufficient,
and most of them should be orthogonal to the rest.


--mtx

[Daniel Veditz]

On Wed, Aug 16, 2017 at 7:20 AM, Enrico Weigelt, metux IT consult <
enrico....@gr13.net> wrote:

> Regarding CID vs CONTRACTID - still haven't understood why CIDs are
> random numbers, instead of human-readable names

​Someone in 1999 or 2000 thought it was a good idea and set the pattern.​

And still I find the naming "CID" (class-ID ?) vs "CONTRACTID" quite
> confusing. Why not something like "INTERFACEID" or "PROTOCOLID" vs.
>

​It is what it is and no one wants to go through millions of lines of code
renaming everything.​ Just learn it once and move on to things that matter.

Doesn't need to be that much. Less than hundred should be sufficient,
> and most of them should be orthogonal to the rest.
>

​100 options is 4950 configurations to test. More often than not things
that "should be" independent have subtle unexpected linkages. You can't
handwave away the testing through sheer optimism.

-Dan Veditz

[Mike Hommey]

On Wed, Aug 16, 2017 at 12:43:19PM -0700, Daniel Veditz wrote:
> 100 options is 4950 configurations to test.

I think you mean 2^100. That's 1.26 x 10^30.

Mike

[Eric Shepherd (Sheppy)]

Also known as “a boatload.”



Eric Shepherd
Senior Technical Writer, MDN
MDN: https://developer.mozilla.org/
Blog: https://www.bitstampede.com/