SHA1 sunsetting, false positives

[Micah Lee]

Hello Chromium security team,

While using Chrome Version 40.0.2214.111 (64-bit) in Debian Jessie, I'm
receiving the Chrome SHA1 deprecation warning when visiting
https://firstlook.org/. However, this is not only a false positive, but
we can't figure out how to reproduce this in Chrome running on other
systems. I think this might be some sort of obscure, hard to reproduce,
bug in the SHA1 detection code.

Here are some screenshots from my Chrome (one from firstlook.org, and
one from our CDN at prod01-cdn03.cdn.firstlook.org):
https://imgur.com/a/EHShW

I also get the same warning when viewing our domain in an incognito window.

This warning doesn't appear in Chrome on other platforms, or in Debian
VMs we test on, but it reliably happens on my computer. At first we
thought this was some sort of fluke (like maybe my Chrome profile is
corrupt), but it turns out that other users are in fact seeing this too:

https://twitter.com/ageis/status/563647806334717953

As far as we can tell, everything on our end is configured properly. The
Qualys SSL test gives us an A+:
https://www.ssllabs.com/ssltest/analyze.html?d=firstlook.org

Is there a way to gather more information about why my browser is giving
this warning? Is there a log that I can look at that might contain more
information, or a way to get my browser to reload the domain in some
sort of verbose debug mode?

Thanks.

--
Micah Lee
The Intercept - https://theintercept.com/

[Lucas Garron]

There have been similar issues where users had locally cached intermediate certificates using SHA-1.

Could you click on "Certificate information" and see if you get a SHA-1 cert that other users don't?

»Lucas

[Chris Palmer]

So, it could be that on your Debian system, certificate chain path
building builds through a SHA-1-signed intermediary certificate. Path
building is (a) insane and (b) dependent on the state of your client's
trust anchor store and set of cached intermediate certificates.

Examine the entire chain in the certificate viewer, not just the
end-entity certificate. I bet you'll find a SHA-1-signed intermediate
cert.

You can also get a full log by following these instructions:
https://dev.chromium.org/for-testers/providing-network-details.

[Micah Lee]

I just tested, and when I view the certificate information I get the same SHA-256 and SHA-1 fingerprints that others see who don't get the warning.

Though I did notice a difference. In my browser that gets the SHA-1 warning, the certificate chain looks like this:

Builtin Object Token:AddTrust External Root > UTN - DATACorp SGC > AddTrust External CA Root > COMODO RSA Certificate Authority > COMODO RSA Domain Validation Secure Server CA > *.firstlook.org

However in a Chrome that doesn't have this issue the certificate chain looks like this:

Builtin Object Token:AddTrust External Root > COMODO RSA Certificate Authority > COMODO RSA Domain Validation Secure Server CA > *.firstlook.org

So two intermediates, UTN - DATACorp SGC and AddTrust External CA Root, are present in my browser, but not in others. That's odd. However all intermediates list of a SHA-256 fingerprint.

[Ryan Sleevi]

On Fri, Feb 6, 2015 at 4:12 PM, Micah Lee <mica...@theintercept.com> wrote:
> I just tested, and when I view the certificate information I get the same
> SHA-256 and SHA-1 fingerprints that others see who don't get the warning.
>
> Though I did notice a difference. In my browser that gets the SHA-1 warning,
> the certificate chain looks like this:
>
> Builtin Object Token:AddTrust External Root > UTN - DATACorp SGC > AddTrust
> External CA Root > COMODO RSA Certificate Authority > COMODO RSA Domain
> Validation Secure Server CA > *.firstlook.org
>
> However in a Chrome that doesn't have this issue the certificate chain looks
> like this:
>
> Builtin Object Token:AddTrust External Root > COMODO RSA Certificate
> Authority > COMODO RSA Domain Validation Secure Server CA > *.firstlook.org
>
> So two intermediates, UTN - DATACorp SGC and AddTrust External CA Root, are
> present in my browser, but not in others. That's odd. However all
> intermediates list of a SHA-256 fingerprint.

Welcome to the wonderful world of PKI.

This issue was fixed in NSS 3.17.4 (
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/NSS_3.17.4_release_notes
) with the weaker roots no longer being preferred (cached or
otherwise).

As Chris mentioned, providing a net-internals log would help, but from
memory, I believe the External CA Root cert you mentioned, as well as
UTN - DataCorp, are both SHA-1 _signed_. Note that we display a SHA-1
and SHA-2 fingerprint for all of these in the UI, but it is the
*signature* that matters (and the signature section in the UI will
describe the algorithm used)

[Ryan Sleevi]

I should also add that if you're running anything earlier than NSS
3.17.3, your distro is being somewhat lax in security updates. Any
update to the root store should be treated as an important security
upgrade.

In this case, several roots were removed in the 3.17.3 timeframe that
had not at all been audited for _years_ (by the time they were
removed). Hopefully that should send chills down your spine.

[Chris Palmer]

On Fri, Feb 6, 2015 at 4:17 PM, Ryan Sleevi <rsl...@chromium.org> wrote:

> This issue was fixed in NSS 3.17.4 (
> https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/NSS_3.17.4_release_notes
> ) with the weaker roots no longer being preferred (cached or
> otherwise).

We should hope Debian picks up this update ASAP. However, we should
not hold our breath. :)

> As Chris mentioned, providing a net-internals log would help, but from
> memory, I believe the External CA Root cert you mentioned, as well as
> UTN - DataCorp, are both SHA-1 _signed_. Note that we display a SHA-1
> and SHA-2 fingerprint for all of these in the UI, but it is the
> *signature* that matters (and the signature section in the UI will
> describe the algorithm used)

Right: certificate fingerprint != certificate signing algorithm. See
attached screenshots.

[Micah Lee]

Yup, that's exactly it. Everything before COMODO RSA Certification
Authority uses SHA-1 for its signature algorithm on my system. But in my
Debian Jessie VM I don't see "UTN - DATACorp SGC" or "AddTrust External
CA Root" at all in the certificate chain.

I wonder why, or how to fix it. Is "COMODO RSA Certificate Authority"
signed by both "Builtin Object Token:AddTrust External Root" and
"AddTrust External CA Root"? Does this mean as soon as I visit a website
in my VM that caches the "AddTrust External CA Root" intermediate, with
it's SHA-1 signature, that all websites signed by "COMODO RSA
Certificate Authority" will start showing the SHA-1 warning?

But thank you, this has been very helpful.

[Ryan Sleevi]

Note that though this turned out to be a system issue, you should feel
free to file false-positive reports in our bug tracker.

https://code.google.com/p/chromium/issues/detail?id=437733 is/was the
tracking bug for the NSS issue, which is what prompted discovery of
the NSS logic issue (bools and ints, not the same :D)

[Ryan Sleevi]

On Fri, Feb 6, 2015 at 4:35 PM, Micah Lee <mica...@theintercept.com> wrote:
> Yup, that's exactly it. Everything before COMODO RSA Certification
> Authority uses SHA-1 for its signature algorithm on my system. But in my
> Debian Jessie VM I don't see "UTN - DATACorp SGC" or "AddTrust External
> CA Root" at all in the certificate chain.
>
> I wonder why, or how to fix it. Is "COMODO RSA Certificate Authority"
> signed by both "Builtin Object Token:AddTrust External Root" and
> "AddTrust External CA Root"? Does this mean as soon as I visit a website
> in my VM that caches the "AddTrust External CA Root" intermediate, with
> it's SHA-1 signature, that all websites signed by "COMODO RSA
> Certificate Authority" will start showing the SHA-1 warning?

Unfortunately, until distros ship NSS 3.17.4, yes. The moment you
visit any site where one of the *older* certs as necessary for chain
building, it will corrupt the process state (and disk cache, by
transitive virtue) into automagically preferring the weak chain.

Installing NSS 3.17.4 will fix this. Jessie is still on 3.17.2 (e.g.
before all of the root program security updates, and before this fix),
while wheezy and squeeze are both running (irresponsibly?) old
versions (with respect to how trustworthy the root store is) of 3.14.5
/ 3.12.8, respectively.