x509: failed to load system roots and no roots provided on Darwin with Go 1.1

[Owen]

Hi,

I am trying to build my project using cross-compile tool goxc (https://github.com/laher/goxc) on a Linux box. But the Darwin builds are having problems "x509: failed to load system roots and no roots provided" when trying to make http GET call to a https url. That problem doesn't exist if I build my project locally (OSX 10.8.4). The Go version is 1.1 both on the Linux box and my Mac. Here is the build log: https://drone.io/github.com/jingweno/gh/184. 

Any thoughts?

Thanks,

Owen

[minux]

On Thu, Jun 13, 2013 at 11:47 AM, Owen <jing...@gmail.com> wrote:

I am trying to build my project using cross-compile tool goxc (https://github.com/laher/goxc) on a Linux box. But the Darwin builds are having problems "x509: failed to load system roots and no roots provided" when trying to make http GET call to a https url. That problem doesn't exist if I build my project locally (OSX 10.8.4). The Go version is 1.1 both on the Linux box and my Mac. Here is the build log: https://drone.io/github.com/jingweno/gh/184. 

Cross compiling will disable cgo, but accessing system root certificates on Mac OS X

requires cgo.

so you can't cross compile non-trivial networking Mac OS X program from Linux (if you

can't set up a Mac OS X C cross compiling environment on Linux).

[Am Laher]

Hi Minux, thanks for the heads-up. Do you know if there are any other limitations of the cross-compiler? 

It would be good to get some definitive info on this.

I'd like to document goxc with this kind of limitation, and probably log warning messages when cross-compiling to darwin.

Also, your 'if ...' statement suggests it might be possible to set up a cgo cross-compiler - using a x-compiled gcc toolchain?

That would be interesting. More work-intensive I'm sure though.

Cheers

[Dave Cheney]

> Hi Minux, thanks for the heads-up. Do you know if there are any other
> limitations of the cross-compiler?
> It would be good to get some definitive info on this.

Cross compilation disabled cgo, that is the only restriction.

> I'd like to document goxc with this kind of limitation, and probably log
> warning messages when cross-compiling to darwin.
>
> Also, your 'if ...' statement suggests it might be possible to set up a cgo
> cross-compiler - using a x-compiled gcc toolchain?
> That would be interesting. More work-intensive I'm sure though.

It is possible to do, but the amount of work (not to mention the CI
load) required, is substantial. For example, in your linux -> darwin
example, the compilation is pretty straight forward, but the linking
including having access to the darwin .framework libraries is non
trivial and hugely error prone.

[Am Laher]

> Cross compilation disabled cgo, that is the only restriction.

Sorry I should rephrase the question: do you know of any other parts of the standard library which depend on cgo, in the same way as this 'Darwin root certificates' example?

>  ... the linking including having access to the darwin .framework libraries is non trivial and hugely error prone.

OK thanks, good to know in advance. I won't go there unless I really need to! 

[Dave Cheney]

From memory

os/user
net (falls back to a native go dns resolver)
crypto/tls (cgo is used to bind to the system ssl keystore)

[Owen Ou]

Would it be possible that more and more of the following codes are implemented in Go so that cross compile can be more useful?

Cheers,
Owen

[Dave Cheney]

I cannot answer that. I know that adding a cgo requirement to a package in that standard library is not taken lightly.

[minux]

On Sat, Jun 15, 2013 at 11:39 AM, Am Laher <a...@laher.net.nz> wrote:

Also, your 'if ...' statement suggests it might be possible to set up a cgo cross-compiler - using a x-compiled gcc toolchain?

That would be interesting. More work-intensive I'm sure though.

In fact, it is very hard to do.

On Sat, Jun 15, 2013 at 12:37 PM, Owen Ou <jing...@gmail.com> wrote:

Would it be possible that more and more of the following codes are implemented in Go so that cross compile can be more useful?

There are things that are almost impossible to do with pure Go.

As another example, net package uses DNS lookup in libc, as it's impossible for Go to implement every host lookup
possibilities provided by all libc in all supported platforms.
(For example, some libc has special caching layer, and without intensive work, Go can't access the cached result)

os/user, which access user databases, shouldn't be made to understand LDAP, which is better leaved for system
libc to do.

[Am Laher]

I think the lack of a certificate store would be by far the most commonplace issue of those, particularly because DNS already falls back to a pure-go alternative (I couldn't say what disadvantages the pure-go solution has compared with the cgo solution, but presumably it can support the rudimentary lookups required by most programs).

Fortunately, it seems feasible to roll your own os-specific 'crypto.tls.Config' (and host-lookup, if the standard pure-go solution doesn't work for you). 

It's not ideal but it might help bridge the gap.

Here's an example of pure-go implementation for certificate resolution (and DNS lookups) for Android:

https://github.com/bradfitz/camlistore/commit/51e88c7434c1f46e7aaa985de03659f2afd28998

It seems to have disappeared from the HEAD of this repository - not sure why - but it's interesting anyway. Here's Brad's post about it: https://plus.google.com/115863474911002159675/posts/DWmyygSrvt7

I wonder if there's an existing Darwin implementation kicking about anywhere. 

I'm not an OSX user so I don't know how easy it would be to hack something together. 

Any ideas?

[Am Laher]

Suggestion: bundle the github.com certificate chain as part of your app, and then load the chain into your tls.Config.

See this gist as proof of concept.
I loaded up the github certificate chain (root & intermediate) into an empty CertPool, then it (i) successfully connects to github, and (ii) fails to connect facebook:
https://gist.github.com/laher/5795578

In this case I added the certificate to a new 'CertPool', but you could add it to the existing CertPool, so it doesn't lose the existing CAs.
You could even do this in a file with build flags +build darwin,!cgo, so that it only affects the cross-compiled binary.
Provided github don't change certificate providers, you wouldn't have to update these bundled CA certs until 2018 :)

Do you think that it would be acceptable (legal?) to distribute the certificates along with your binary?

(Note that I cross-posted this for Owen on https://github.com/laher/goxc/issues/10)