Re: Script to generate CAs (issue 5323075)
xt...@google.com
> This patch addresses issue170 in the issue tracker. It also fixes 157
and 156.
Hi jaraco,
There are a couple of concerns regarding an approach that bundles a
large root CA file with httplib2, as well as with the patch as is in
particular:
* As a practical matter, this approach might have significant
performance issues. The python 2.x SSL API is severely lacking, and one
cannot create and get a handle for a reusable SSL context (in 3.x this
is available afaik). Which means that openssl will read in and parse the
provided CA file on *every connection*, which is a good reason to keep
the CA file small.
* The mozilla source contains certs that are marked as untrusted; IIUC
these are used to black-list CAs in mozilla. The import script would
have to skip those. Which is what for example, debian/ubuntu does (see
scripts in the source of the ca-certificates package). It looks like
that in addition, debian explicitly blacklists/excludes some CAs.
* In many uses, pulling in all these roots adds unnecessary risk
exposure due to potentially compromised CAs. In particular since there
is no automated update mechanism which would propagate the deletion or
blacklisting of a compromised CA in the Mozilla source. There have been
a number of events in recent past where CAs have been black-listed (see,
e.g.,
http://en.wikipedia.org/wiki/DigiNotar#Issuance_of_fraudulent_certificates),
so this is not a theoretical concern.
After conferring with jcgregorio, we'd like to propose the following
approach:
* By default, install with the existing, minimal root CA cert.
* On systems (e.g. unix) where a system provided CA cert file is
available, provide an install-time option to instead use a symlink to
(not a copy of) this file. Since the CA certs file will be updated by
the distribution, httplib won't be responsible for updates. As far as
security considerations are concerned, it would be reasonable to use the
symlink by default; however the performance concern still stands.
Cert file locations include (from
http://weekly.golang.org/src/pkg/crypto/tls/root_unix.go):
"/etc/ssl/certs/ca-certificates.crt", // Debian Linux
"/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
"/etc/ssl/ca-bundle.pem", // OpenSUSE
* Otherwise, provide an option to fetch and parse the mozilla cert
source at install time, and print a warning that the user will be
responsible for tracking changes to the mozilla source, and
re-installing if necessary. The parsing script would have to be modified
to take untrusted/blacklisted certs into account. If its license
permits, perhaps the debian script could be used.
Does this sound workable?
Thanks,
--xtof
Jason R. Coombs
I see now that it's not just a technical matter of including the appropriate
certs. By reading the source code history and bug reports, my impression was
that CAs were being added incrementally as they were needed, which seemed
like a terrible approach. Furthermore, the CA file that was included
suggested that it was generated from the Mozilla source, even though it
clearly did not have all of the certificates.
I agree that parsing the Mozilla source at any time is probably undesirable,
if it does indeed include certificates which are known to not be
trustworthy.
However, I believe there are some issues with the proposed approach as well.
The proposal is to keep a minimal subset of trusted certificate authorities
bundled with httplib2, but it doesn't define which authorities or how an
authority is included or excluded from the list. It would be wise to have a
standard metric which an authority must meet to be included, rather than
deferring to the potentially biased whimsy of some developers. Otherwise, it
becomes unclear which authorities should or shouldn't be included.
Furthermore, technical limitations of the underlying libraries should not be
a factor affecting inclusion or exclusion of an authority's certificate. If
there are performance limitations, those should be addressed directly, not
by excluding legitimate authorities.
Additionally, it seems unreasonable to exclude certificate authorities based
on the premise that they might at some point become compromised. Each of
these authorities has undergone the same verification process through the
Mozilla organization. I don't think it's in the best interest of the
httplib2 project to second-guess those audits and instead establish their
own arbitrary criteria for a trustable authority.
If httplib2 is going to be in the business of verifying the validity of
certificates by default (and failing if they fail to validate), then in my
opinion, it should be able to effectively validate most certificates issued
by legitimate, public CAs.
While you suggest that adding additional CAs adds unnecessary exposure,
omitting those CAs has the potential of adding even greater exposure. If the
library cannot cover a similar security space as common browsers, then
consumers of the application will find themselves forced for practical
reasons to disable certificate validation, in which case httplib2 has
encouraged diminished security.
I suggest that httplib2 implement a fair and objective measure for inclusion
of CAs bundled with the library (whether this be all, none, some
objectively-determined subset, or an industry-standard such as Mozilla's
set), but then implement a solution similar to the one you described:
1) httplib2 first attempts to load trusted certificates from the system. I
don't believe there's any need to copy/install/symlink the certs; just load
them from the system on demand. Avoiding a post-install step dramatically
increases the simplicity of packaging. This approach might work well with
the Windows approach where system certificates are stored in the registry.
2) If system certificates are not available (or are disabled by a user
preference), certificates will be loaded from the library.
3) (optional) The application could have the option to downselect a subset
of the certificates to be used in validation. This option could be used by a
security-sensitive application to restrict the available certificates or to
improve performance (if necessary).
This approach has several advantages.
1) It prefers system certificates, which defers the security and updates to
the user's system, which is much more likely to be regularly updated and
gets updates without changes to httplib2.
2) It provides a decent fallback which can provide a best-effort
verification of certificates expected to be found in the wild.
3) It addresses the needs of the bugs as filed in the issue tracker.
4) It does not add post-install steps or other user configuration for the
vast majority of deployments.
5) It better leverages the capability already present in the
community/industry.
6) It provides a fair and equitable metric for CA inclusion.
7) It is more platform friendly.
8) It puts a priority of security and capability and defers optimization.
I look forward to your thoughts on my suggestions.
Regards,
Jason
Joe Gregorio
how many platforms
for which we can consistently find and use local CA files on. If we can cover
Windows, Mac and Ubuntu then I'll be fine with it, if we can't then
I'd like to have
the fallback position be the small CA file we ship today as opposed to leaving
then with an empty CA file.
Here I would say that #3 should be #1 and that it is not an optional
feature, that
a user must be able to pass in a cert filename to be used instead of the system
certs.
--
Joe Gregorio http://bitworking.org