Certificate Transparency in a web or proxy server – notes for possible implementation

[tra...@gmail.com]

[I'd love to get some help clearing up misconceptions in the below text (unfortunately scraped from a somewhat easier-to-read .odt). TIA! The purpose of this is to work out what changes over the long term would be appropriate for Apache httpd, though most aspects of the text would apply to other web or proxy servers as well.]

This describes possible implementations of Certificate Transparency in web and/or proxy servers. Different levels of compliance with Certificate Transparency are described in terms of configurable modes of operation.

“Web server” in this document serves the role of “TLS server” while “proxy server” in this document serves the role as “TLS client.”

This document refers to the following additional applications used as part of administering the web/proxy server which are not necessarily part of the web or proxy server:

1. log submission application
The web/proxy server administrator uses this tool to submit a server certificate to a log and obtain an SCT in a suitable format. This tool can maintain a set of SCTs in a suitable format as certificates are submitted to multiple logs.

2. Log auditing application
A web proxy when communicating with untrusted TLS servers will queue data for auditing, to be processed off-line by the log auditing application; the proxy should make a best effort attempt to avoid queuing duplicate information. Log auditing failures should be reflected in the set of logs known to the proxy server, though that may require manual intervention by the administrator.

Several "SSL variables" are referred to in the description. Such variables are useful as information for web applications, for custom logging, and in some cases specific server processing can be triggered by specific values.

Part I, Web server

Supporting TLS communication from a client to a server (possibly upgraded after initial, unencrypted communication); the OCSP Stapling feature referred to below may be missing from some web server implementations or purposefully disabled

Web server is configured in “SCT-required” mode

* The administrator may need to use the log submission tool to submit a server certificate and obtain an SCT.
** This must be used if the SCT is not in a certificate extension and the SCT is not returned by the OCSP server.
** This may be used to add responses from one or more additional logs that a client may require.

* The server software will ensure that at least one SCT is available in the certificate extension, or the SCT TLS extension is maintained separately (such as in a .pem file), or the SCT is part of the OCSP stapling response.

** Processing should fail at the earliest point practical if this requirement is not met, which may be at the time of server startup or during the handshake.
*** The server can check for the extension in the certificate or the TLS extension in foo.pem at startup.
*** The earliest (practical) point that the OCSP stapling response may be examined will vary according to the server implementation.
*** If a particular ClientHello does not include the Certificate Status extension, SCTs can only be provided to that client via a certificate extension or a configured TLS extension.
*** Does the SCT-available status need to be stored in the session cache? No, validation will occur on the initial handshake.

* The server will perform basic sanity checking on all SCTs at the earliest point practical if a check fails, since the administrator can diagnose a configuration problem much more easily when the server software identifies the issue.

* The server will represent the source(s) of SCT in SSL variable SSL_SCT_SOURCES; this will be a comma-delimited list of source types, with these types represented by “certext” (certificate extension), “tlsextfile” (TLS extension in ServerInfoFile), and “ocsp” (part of OCSP stapling response).

Web server is configured in “opportunistic SCT” mode

* The possible lack of one or more SCTs to provide to the client will not result in failures triggered by the server (though the client may refuse interoperation).
* If an SCT is available, the server will perform basic sanity checking on it and fail at the earliest point practical if a check fails.
* If an SCT is available, the server will represent the source(s) of SCT in SSL variable SSL_SCT_SOURCES as described previously. Otherwise it will be set to “none”.

Web server is configured in SCT-unaware mode

* This non-default mode might be required to temporarily work around known configuration or software implementation problems.
* There is no requirement that an SCT will be delivered, but it is not specifically prevented.
* Any of the three mechanisms (certificate extension, TLS extension provided by the administrator as part of the server configuration, part of OCSP stapling response) can be used, but no sanity checks will be performed and SSL_SCT_SOURCES will be set to “unknown” or unset.

Part II, proxy initiates TLS connections to back-end server

Typical scenarios include

* proxy initiates TLS connection to statically configured back-end servers, often under the same administrative and network control as the proxy (i.e., often “trusted”)
* proxy initiates TLS connection to arbitrary back-end servers (i.e., “untrusted”)
* CONNECT over TLS to untrusted back-end servers (i.e., client has TLS connection to server and issues CONNECT request to connect to backend)
* forward proxy over SSL to untrusted back-end servers (BIG-IP has this particular feature)

The proxy is a TLS client in these scenarios, initiating the handshake with a back-end server. Depending on the configuration, it must decide what action to take if no SCT is received during the handshake or if it is obviously invalid, or if the log which generated the SCT is known to be untrusted.

In the case of multiple SCTs provide in a handshake, a single valid SCT is sufficient, in order to bypass additional processing. (This is an obvious area for variation; some failures to accept a particular SCT may not be unusual, including receiving SCTs from logs which are unknown to the proxy or which are known to have failed audit.)

An SCT cannot be validated unless the public key of the log which generated it is available; thus, logs are expected to be statically configured to the proxy, possibly in a manner that can be updated dynamically when a log fails an audit.

When an SCT is provided during the handshake, the proxy can determine if the SCT provided is associated with the server certificate if the log which generated the SCT is trusted.

A separate OCSP request could potentially be used to query the OCSP server if an SCT is not otherwise provided, but this is not necessary due to the availability of sufficient mechanisms for delivering the SCT at the time of the handshake. (The proxy could be configured to perform OCSP queries for other reasons.)

The CertificateStatus extension should always be included in the ClientHello when SCT processing is enabled; it may of course be enabled for other reasons.

Log auditing is an asynchronous operation, so the server certificate and SCT(s) must be stored, presumably without duplicates, for use by auditing.

SCT processing for trusted back-end servers (typical for reverse proxy) should be easy to disable, even in “SCT-required” mode. Presumably if certificate validation is explicitly disabled for a back-end server then SCT processing would not be desired either.

Proxy is configured in “SCT-required” mode

The handshake will be aborted if an SCT is not provided by the server or if immediate validation fails for all received SCTs.

SSL_PROXY_SCT_SOURCES – this will be a comma-delimited list of source types, with these types represented by “certext” (SCT is in certificate extension), “tlsext” (SCT is in TLS extension), and “ocsp” (SCT is in OCSP stapling response).

SSL_PROXY_SCT_STATUS – “valid” or “invalid” or “unknown-log”

Proxy is configured in “opportunistic SCT” mode

In “opportunistic SCT” mode, the lack of an SCT from a back-end server is not considered an error; however, if one is available then it will be processed in the same manner as in “SCT-required” mode.

SSL_PROXY_SCT_SOURCES – same as above, or “none” if no SCT is available

SSL_PROXY_SCT_STATUS – same as above, or “none” if no SCT is available

Proxy is configured in “SCT-unaware” mode

All SCT processing by the proxy is omitted.

SSL_PROXY_SCT_SOURCES – “unknown” or unset
SSL_PROXY_SCT_STATUS – “unknown” or unset

[Ben Laurie]

Everything I haven't commented on I agree with :-)

Thanks for this!

On 8 January 2014 23:31, <tra...@gmail.com> wrote:
> [I'd love to get some help clearing up misconceptions in the below text (unfortunately scraped from a somewhat easier-to-read .odt). TIA! The purpose of this is to work out what changes over the long term would be appropriate for Apache httpd, though most aspects of the text would apply to other web or proxy servers as well.]
>
> This describes possible implementations of Certificate Transparency in web and/or proxy servers. Different levels of compliance with Certificate Transparency are described in terms of configurable modes of operation.
>
> “Web server” in this document serves the role of “TLS server” while “proxy server” in this document serves the role as “TLS client.”
>
> This document refers to the following additional applications used as part of administering the web/proxy server which are not necessarily part of the web or proxy server:
>
> 1. log submission application
> The web/proxy server administrator uses this tool to submit a server certificate to a log and obtain an SCT in a suitable format. This tool can maintain a set of SCTs in a suitable format as certificates are submitted to multiple logs.

In the long run, it'd be nice if the server could do this bit
automatically (some configuration required!):

1. First fetch from some (configured) URL a list of "good" logs.

2. Fetch SCTs from some (configured) number of those logs.

3. When serving, periodically refresh the good log list, and ensure
there's a supply of SCTs from good logs.

4. Send a configured number of SCTs from different good logs in a TLS
extension. Note that the current spec allows this number to be 1.

>
> 2. Log auditing application
> A web proxy when communicating with untrusted TLS servers will queue data for auditing, to be processed off-line by the log auditing application; the proxy should make a best effort attempt to avoid queuing duplicate information. Log auditing failures should be reflected in the set of logs known to the proxy server, though that may require manual intervention by the administrator.

Presumably the auditor can keep a database of already-audited certs?
Perhaps with LRU to limit size...

> Several "SSL variables" are referred to in the description. Such variables are useful as information for web applications, for custom logging, and in some cases specific server processing can be triggered by specific values.
>
> Part I, Web server
>
> Supporting TLS communication from a client to a server (possibly upgraded after initial, unencrypted communication); the OCSP Stapling feature referred to below may be missing from some web server implementations or purposefully disabled
>
> Web server is configured in “SCT-required” mode
>
> * The administrator may need to use the log submission tool to submit a server certificate and obtain an SCT.
> ** This must be used if the SCT is not in a certificate extension and the SCT is not returned by the OCSP server.
> ** This may be used to add responses from one or more additional logs that a client may require.
>
> * The server software will ensure that at least one SCT is available in the certificate extension, or the SCT TLS extension is maintained separately (such as in a .pem file), or the SCT is part of the OCSP stapling response.
>
> ** Processing should fail at the earliest point practical if this requirement is not met, which may be at the time of server startup or during the handshake.
> *** The server can check for the extension in the certificate or the TLS extension in foo.pem at startup.
> *** The earliest (practical) point that the OCSP stapling response may be examined will vary according to the server implementation.
> *** If a particular ClientHello does not include the Certificate Status extension, SCTs can only be provided to that client via a certificate extension or a configured TLS extension.

If the client doesn't include the extension, it presumably can't
process CT anyway, so its OK to send it no SCTs.

> *** Does the SCT-available status need to be stored in the session cache? No, validation will occur on the initial handshake.
>
> * The server will perform basic sanity checking on all SCTs at the earliest point practical if a check fails, since the administrator can diagnose a configuration problem much more easily when the server software identifies the issue.
>
> * The server will represent the source(s) of SCT in SSL variable SSL_SCT_SOURCES; this will be a comma-delimited list of source types, with these types represented by “certext” (certificate extension), “tlsextfile” (TLS extension in ServerInfoFile), and “ocsp” (part of OCSP stapling response).

What is an "SSL variable"? I'm guessing you mean environment variables for CGIs?

>
> Web server is configured in “opportunistic SCT” mode
>
> * The possible lack of one or more SCTs to provide to the client will not result in failures triggered by the server (though the client may refuse interoperation).
> * If an SCT is available, the server will perform basic sanity checking on it and fail at the earliest point practical if a check fails.
> * If an SCT is available, the server will represent the source(s) of SCT in SSL variable SSL_SCT_SOURCES as described previously. Otherwise it will be set to “none”.
>
> Web server is configured in SCT-unaware mode
>
> * This non-default mode might be required to temporarily work around known configuration or software implementation problems.
> * There is no requirement that an SCT will be delivered, but it is not specifically prevented.
> * Any of the three mechanisms (certificate extension, TLS extension provided by the administrator as part of the server configuration, part of OCSP stapling response) can be used, but no sanity checks will be performed and SSL_SCT_SOURCES will be set to “unknown” or unset.
>
> Part II, proxy initiates TLS connections to back-end server
>
> Typical scenarios include
>
> * proxy initiates TLS connection to statically configured back-end servers, often under the same administrative and network control as the proxy (i.e., often “trusted”)
> * proxy initiates TLS connection to arbitrary back-end servers (i.e., “untrusted”)
> * CONNECT over TLS to untrusted back-end servers (i.e., client has TLS connection to server and issues CONNECT request to connect to backend)
> * forward proxy over SSL to untrusted back-end servers (BIG-IP has this particular feature)
>
> The proxy is a TLS client in these scenarios, initiating the handshake with a back-end server. Depending on the configuration, it must decide what action to take if no SCT is received during the handshake or if it is obviously invalid, or if the log which generated the SCT is known to be untrusted.
>
> In the case of multiple SCTs provide in a handshake, a single valid SCT is sufficient, in order to bypass additional processing. (This is an obvious area for variation; some failures to accept a particular SCT may not be unusual, including receiving SCTs from logs which are unknown to the proxy or which are known to have failed audit.)
>
> An SCT cannot be validated unless the public key of the log which generated it is available; thus, logs are expected to be statically configured to the proxy, possibly in a manner that can be updated dynamically when a log fails an audit.

Right - I think relying on server operators to keep configuration in
line with reality is a bad idea - instead they should use some
mechanism TBD to get the current list (note that this mechanism,
unlike the one for servers, needs some kind of transparency of its
own).

> be configured to fetch "known good"

>
> When an SCT is provided during the handshake, the proxy can determine if the SCT provided is associated with the server certificate if the log which generated the SCT is trusted.
>
> A separate OCSP request could potentially be used to query the OCSP server if an SCT is not otherwise provided, but this is not necessary due to the availability of sufficient mechanisms for delivering the SCT at the time of the handshake. (The proxy could be configured to perform OCSP queries for other reasons.)

In general, we do not expect clients to ever have to make a separate
request to validate CT.

> The CertificateStatus extension should always be included in the ClientHello when SCT processing is enabled; it may of course be enabled for other reasons.
>
> Log auditing is an asynchronous operation, so the server certificate and SCT(s) must be stored, presumably without duplicates, for use by auditing.

Again, the auditor seems the natural place for de-duping.

> SCT processing for trusted back-end servers (typical for reverse proxy) should be easy to disable, even in “SCT-required” mode. Presumably if certificate validation is explicitly disabled for a back-end server then SCT processing would not be desired either.
>
> Proxy is configured in “SCT-required” mode
>
> The handshake will be aborted if an SCT is not provided by the server or if immediate validation fails for all received SCTs.
>
> SSL_PROXY_SCT_SOURCES – this will be a comma-delimited list of source types, with these types represented by “certext” (SCT is in certificate extension), “tlsext” (SCT is in TLS extension), and “ocsp” (SCT is in OCSP stapling response).
>
> SSL_PROXY_SCT_STATUS – “valid” or “invalid” or “unknown-log”
>
> Proxy is configured in “opportunistic SCT” mode
>
> In “opportunistic SCT” mode, the lack of an SCT from a back-end server is not considered an error; however, if one is available then it will be processed in the same manner as in “SCT-required” mode.
>
> SSL_PROXY_SCT_SOURCES – same as above, or “none” if no SCT is available
>
> SSL_PROXY_SCT_STATUS – same as above, or “none” if no SCT is available
>
> Proxy is configured in “SCT-unaware” mode
>
> All SCT processing by the proxy is omitted.
>
> SSL_PROXY_SCT_SOURCES – “unknown” or unset
> SSL_PROXY_SCT_STATUS – “unknown” or unset
>

> --
> You received this message because you are subscribed to the Google Groups "certificate-transparency" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to certificate-transp...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Jeff Trawick]

On Friday, January 10, 2014 8:12:33 AM UTC-5, Ben Laurie wrote:

Everything I haven't commented on I agree with :-)

Thanks for this!

Just a few questions/comments below...  Thanks!

 

On 8 January 2014 23:31,  <tra...@gmail.com> wrote:
> [I'd love to get some help clearing up misconceptions in the below text (unfortunately scraped from a somewhat easier-to-read .odt).  TIA!  The purpose of this is to work out what changes over the long term would be appropriate for Apache httpd, though most aspects of the text would apply to other web or proxy servers as well.]
>
> This describes possible implementations of Certificate Transparency in web and/or proxy servers.  Different levels of compliance with Certificate Transparency are described in terms of configurable modes of operation.
>
> “Web server” in this document serves the role of “TLS server” while “proxy server” in this document serves the role as “TLS client.”
>
> This document refers to the following additional applications used as part of administering the web/proxy server which are not necessarily part of the web or proxy server:
>
> 1. log submission application
> The web/proxy server administrator uses this tool to submit a server certificate to a log and obtain an SCT in a suitable format.  This tool can maintain a set of SCTs in a suitable format as certificates are submitted to multiple logs.

In the long run, it'd be nice if the server could do this bit
automatically (some configuration required!):

1. First fetch from some (configured) URL a list of "good" logs.

 Just to confirm: There's no existing description of this service, right?

2. Fetch SCTs from some (configured) number of those logs.

I think I was subconsciously routing around an issue I was confused about --  there's no "fetch SCT" command in the simplest sense; you submit or resubmit the certificate to the log and will possibly get the same SCT back as last time.  The server doesn't care.  Right?

yes

 

> To unsubscribe from this group and stop receiving emails from it, send an email to certificate-transparency+unsub...@googlegroups.com.

[Jeff Trawick]

I think your comments are resolved well enough for now with these updates:

https://github.com/trawick/ct-httpd/commits/master/README.md

[Ben Laurie]

Right.

>
>>
>> 2. Fetch SCTs from some (configured) number of those logs.
>
>
> I think I was subconsciously routing around an issue I was confused about --
> there's no "fetch SCT" command in the simplest sense; you submit or resubmit
> the certificate to the log and will possibly get the same SCT back as last
> time. The server doesn't care. Right?

Right.

>> > an email to certificate-transp...@googlegroups.com.

>> > For more options, visit https://groups.google.com/groups/opt_out.
>

> --
> You received this message because you are subscribed to the Google Groups
> "certificate-transparency" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to certificate-transp...@googlegroups.com.