SSL and letsencrypt

[Jeff Abrahamson]

I'm a bit confused about setting up mongodb with SSL.

I use letsencrypt to get SSL certificates.  With quite a bit of reading and fiddling around, it seems that I need to create the pem file thus:

    # This script must run as root, both to see letsencrypt SSL
    # certificates and to write the certificates for mongo.
    #
    # Mongodb wants a slightly modified setup than what letsencrypt
    # provides.  So we need to do this every time we renew certificates.
    # We'll therefore use this script for certificate renewal to avoid
    # race conditions beyond the very short time between calling certbot
    # and the reload.

    # Cf. https://stackoverflow.com/a/39189581/833300

    /usr/bin/certbot renew --quiet

    cd /etc/letsencrypt/live/{{ server_name }}

    ssl_dir=/etc/ssl/mongo
    cat privkey.pem fullchain.pem > $ssl_dir/mongo.pem
    chmod 600 $ssl_dir/mongo.pem
    chown mongodb:mongodb $ssl_dir/mongo.pem

The documentation about a ca.pem file I'm finding even more confusing.  Some sources suggested I should use the letsencrypt fullchain.pem, but that leads to errors from mongod.

Other sources suggested I should download the identtust cert thus:

    # For ca.crt, I created ./TrustID.root.pem, which I just copied from here
    # https://www.identrust.com/certificates/trustid/root-download-x3.html
    # as explained here:  https://letsencrypt.org/certificates/ .

and that was accepted by mongod.  But then when I want to connect with the mongo commandline client, I kept getting errors about the CA file I was providing until I finally commented out the CAFile directive in mongod.conf.  And then it was fine.

It also seems that I need to provide that mongo.pem file to every client that wants to connect.  (At least, I haven't gotten it working any other way.)  Our fleet is relatively small and tightly controlled, so I can do that, but it seems odd to me that everyone needs a copy of the private key.

Given how common letsencrypt has become, I assume I've missed something fundamental that I've not found a simple example or tutorial.  So, my apologies if the answer is just a pointer to such a thing.

Many thanks.

[Jeff Abrahamson]

As I continue, this is seeming more and more strange.  When certbot decides to upgrade the SSL certificate, all client access will fail.

Something is clearly not right here.

Can anyone here offer any enlightenment?

Thanks.

Jeff Abrahamson

http://www.p27.eu/jeff/

[Jeff Abrahamson]

It turns out that I can simply remove the ssl cert from the commandline.  This is odd, because at the end of last week it appeared to be necessary.

So it seems this might be closed.  I'll be re-imaging this host at some point soon and I'll provide an update when I do that so that there' s a clean explanation of what I needed to do to use letsencrypt and mongo together.

Jeff Abrahamson
https://www.p27.eu/jeff/

[Jorge Sanchez]

Hi Jeff, I somewhat confused as well, I'll be running some related experiments, just wanted to check first if you've found out something else.

[Jeff Abrahamson]

HI, Jorge.

I still don't have a clear idea what the right thing to do is.  I ended up omitting the net:ssl:CAFile directive in the config, which works but means that some bit of SSL-ness isn't being verified correctly.

I'm not happy with the current situation on our mongo servers, and I'd like to sort this out, but, of course, my life is about juggling which things I'll ignore when...

Jeff Abrahamson
+33 6 24 40 01 57
+44 7920 594 255

http://p27.eu/jeff/

[jonatha...@10gen.com]

Hi Jeff,

I think what you had in your initial email of July 6th was mostly correct. I was able to use letsencrypt by manually getting a certificate, concatenating the full chain and key into a single file and starting mongod with that file as the sslPEMKeyFile

$ cat privkey.pem fullchain.pem > mongo.pem

$ mongod --sslMode=requireSSL --sslPEMKeyFile=/path/to/mongo.pem

The --sslCAFile option to mongod is somewhat tricky. Technically, if the certificate/key file contains the entire chain (and it does in this case), the --sslCAFile  option is not required. The --sslCAFile option has a double meaning of use the certificates in this file to negotiate a connection, and require all clients to have a client certificate signed by this CA. That's why it worked as long as the client also had mongo.pem. You could use the --sslCAFile option with letsencrypt by doing:

$ cat privkey.pem cert.pem > mongo.pem

$ mongod --sslMode=requireSSL \

         --sslPEMKeyFile=/path/to/mongo.pem \

         --sslCAFile=/path/to/chain.pem \

         --sslAllowConnectionsWithoutCertificates

The difference between these two configurations is that the latter will allow client certificates and the former won't. You probably don't want to setup client certificates, so the first configuration is probably what you want. When you renew the certificate, you'll have to restart mongod with the new key/cert, but you shouldn't have to do anything else. On the client side, you may have to specify --sslCAFile=/path/to/chain.pem (a file containing just the CA certs) so that the client can verify the server's certificate, but since letsencrypt is trusted by most browsers you shouldn't have to even do that. I was able to connect just by running mongo --ssl hostname.

Hope this helps, and let me know if you have any more questions.

Jonathan

[Jeff Abrahamson]

Thanks, Jonathan.

What made me suspicious was this line in the logs as mongod starts:

2018-08-23T14:56:41.615Z I CONTROL  [initandlisten] ** WARNING: No SSL certificate validation can be performed since no CA file has been provided
2018-08-23T14:56:41.615Z I CONTROL  [initandlisten] **          Please specify an sslCAFile parameter.

It sounds like you're saying that that should be read "can't certify clients certificates" rather than "can't certify certificates", and so, really, SSL encryption is working just fine.

(If that's true, I'd argue that the log message is a bit misleading at best.)

[jonatha...@10gen.com]

Agreed. SSL encryption is working just fine, it's just that the client can verify the server's identity and not the other way around. I've filed https://jira.mongodb.org/browse/DOCS-11993 to improve the documentation on this and https://jira.mongodb.org/browse/SERVER-36827 to improve the warning message.

JBR

[Jeff Abrahamson]

Awesome, thanks!

Jeff

p27.eu/jeff

[Jorge Sanchez]

Thanks Jeff and Jonathan, this information will help me a lot with what I'll be doing in the next week or so, I know who to reach out if I find any problem down the road!