TLS cert verification without sending SNI

[David Fifield]

I wonder if it's possible to support this use case? I'd like to be able
to give tls.clientHandshake a domain name to use for verification,
without sending the server_name extension (SNI). Currently, whatever
name you have in Config.ServerName is used both as the serverName in a
clientHelloMsg, and as the DNSName in an x509.VerifyOptions:

https://code.google.com/p/go/source/browse/src/pkg/crypto/tls/handshake_client.go?r=b3759654d42d1d5ca0ac38a2d400f047ff1f7bec#57
https://code.google.com/p/go/source/browse/src/pkg/crypto/tls/handshake_client.go?r=b3759654d42d1d5ca0ac38a2d400f047ff1f7bec#252

I want this ability for a specific use case having to do with Internet
censorship circumvention. In some circumstances it's possible to reach a
censored domain name over HTTPS by putting the censored domain in the
HTTP Host header and a different name in the SNI. (A censor gets to see
the SNI but not the Host header.) In some cases sending a different SNI
doesn't suffice; you have to omit SNI entirely. That's what this request
is about. Here is a short summary of the issue as it affects us:

https://trac.torproject.org/projects/tor/ticket/12208

At least three circumvention systems (Tor, Psiphon, and Lantern) are
using this domain-name technique and have run into this issue. Lantern
and Psiphon made their own minor forks of the crypto/tls library in
order to support this use case, SNI-less handshakes with verification:

https://github.com/getlantern/tls/commit/c26f6d9c4b17213b1bd78a734465f2e33be3d12f
https://bitbucket.org/psiphon/psiphon-circumvention-system/commits/c93134904eb67b5c36e80ee2b1cdf2e264113c22

David Fifield

[Brad Fitzpatrick]

I think your needs would be satisified after https://code.google.com/p/go/issues/detail?id=8522 is fixed, combined with supplying your own http.Transport.Dial func for your http.Client's Transport.

David Fifield

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[o...@getlantern.org]

I would actually vote for resolving this in the crypto/tls package instead of net/http, as some non-HTTP clients may want to do the same sort of thing and the functionality seems to cohere most closely with crypto/tls anyway.

I tweaked our tls fork to show one way that this might be done and would be happy to submit a code review with that change if the approach seems reasonable:

https://github.com/getlantern/tls/compare/c7f47000f376de40e121f93968452af3edce0dbc...master

Cheers,

Ox

[Brad Fitzpatrick]

I sent out something very similar just now (https://codereview.appspot.com/137940043) before seeing this so I'm glad ours are both so close... it suggests that this might work for you. Please confirm.

[David Fifield]

On Tue, Sep 02, 2014 at 09:42:20AM -0700, Brad Fitzpatrick wrote:
> I sent out something very similar just now (https://codereview.appspot.com/
> 137940043) before seeing this so I'm glad ours are both so close... it suggests
> that this might work for you. Please confirm.

I think it works for me as far as name verification goes. I can specify
a VerifyCertificate function that alters x509.VerifyOptions.DNSName
before calling x509.Certificate.Verify.

If I understand correctly, we still need something else to prevent the
sending of SNI while still allowing verification. I.e., something like
https://github.com/getlantern/tls/commit/c26f6d9c4b17213b1bd78a734465f2e33be3d12f
Currently, in order to omit SNI, you have to set config.ServerName = "",
and that's not allowed without config.InsecureSkipVerify:
https://code.google.com/p/go/source/browse/src/pkg/crypto/tls/handshake_client.go?r=40906c63a04eeb1fc292cd559e57bfcdeff64e7f#36
I tried using an https URL with an IP address, like
http.Get("https://93.184.216.119/"), but that causes a string
representation of the IP to be sent as the SNI, rather than causing it
to be omitted.

David Fifield

[Ox Cart]

Works for me, and I like that the default VerifyOptions are always supplied.  The comment on VerifyCertificate could use some explanation of what the certs and opts parameters are and where they came from.

As David says, we still need something that allows us to not send the ServerName extension in the client handshake.  Two options would be a new parameter as in the commit that David referenced, or having the optional VerifyCertificate function get called even if InsecureSkipVerify is true, in which case we could just set InsecureSkipVerify and supply our custom verification function.  I think having an explicit parameter is probably clearer.

Cheers,

Ox

-----------------------------------------------------------------------------------------

"I love people who harness themselves, an ox to a heavy cart,

who pull like water buffalo, with massive patience,

who strain in the mud and the muck to move things forward,

who do what has to be done, again and again."

- Marge Piercy

[agl]

On Tuesday, September 2, 2014 3:54:16 PM UTC-7, Ox Cart wrote:

Works for me, and I like that the default VerifyOptions are always supplied.  The comment on VerifyCertificate could use some explanation of what the certs and opts parameters are and where they came from.

As David says, we still need something that allows us to not send the ServerName extension in the client handshake.  Two options would be a new parameter as in the commit that David referenced, or having the optional VerifyCertificate function get called even if InsecureSkipVerify is true, in which case we could just set InsecureSkipVerify and supply our custom verification function.  I think having an explicit parameter is probably clearer.

I think that this use case broke in review. However, you can set InsecureSkipVerify and then verify the certificates yourself once the handshake has completed.

Cheers

AGL 

[Brad Fitzpatrick]

Not with the hooks we have in net/http or this CL as far as I can see.

[Ox Cart]

agl, I think I get it - something like this: https://gist.github.com/oxtoacart/5e78d25a7f9a9cda10cd

That should do it.

Cheers,

Ox

-----------------------------------------------------------------------------------------

"I love people who harness themselves, an ox to a heavy cart,

who pull like water buffalo, with massive patience,

who strain in the mud and the muck to move things forward,

who do what has to be done, again and again."

- Marge Piercy

[Brad Fitzpatrick]

That trick of fetching http when you really want https only gets you so far: once a redirect is involved, you can't fake it anymore and it fails.

This is why we need either an http.Transport.DialTLS hook or support for this policy in *tls.Config.

[Ox Cart]

Brad,

I agree that for the general case of using HttpClient we'd need the proposed change(s).  I was just saying that for my purposes I'll be okay without that, and I think that the same workaround may give David what he needs as well.

Thanks everyone for your help!

Cheers,

Ox

-----------------------------------------------------------------------------------------

"I love people who harness themselves, an ox to a heavy cart,

who pull like water buffalo, with massive patience,

who strain in the mud and the muck to move things forward,

who do what has to be done, again and again."

- Marge Piercy

[Ox Cart]

Drats.  Even if InsecureSkipVerify is true, the client hello still includes the ServerName.  So I guess we still need some way to suppress that.

Sorry for the noise.

Cheers,

Ox

-----------------------------------------------------------------------------------------

"I love people who harness themselves, an ox to a heavy cart,

who pull like water buffalo, with massive patience,

who strain in the mud and the muck to move things forward,

who do what has to be done, again and again."

- Marge Piercy

[Ox Cart]

Okay, I realized that ServerName is only included because I was using tls.Dial().  I can work around this by using net.Dial() and then tls.Client(), as shown here - https://gist.github.com/oxtoacart/5e78d25a7f9a9cda10cd#file-domainfront-go-L44

Cheers,

Ox

-----------------------------------------------------------------------------------------

"I love people who harness themselves, an ox to a heavy cart,

who pull like water buffalo, with massive patience,

who strain in the mud and the muck to move things forward,

who do what has to be done, again and again."

- Marge Piercy

[Brad Fitzpatrick]

To close the loop on this: this should be fixed by Transport.DialTLS in https://code.google.com/p/go/source/detail?r=455042166f13

Just supply a DialTLS hook which doesn't send SNI information.