DigiCert certificate not working with MongoDB
Irfan Sayed
Here is the setup: The environment :
what we have
Mongo server : Ubuntu 16.04 , mongo version : 4.0.10
have configured the /etc/mongod.conf file as well. mentioned the path of the Pem file and CA file correctly. the CN name matches the hostname and is exactly the same.
what is the issue
but now when i run the following mongo command , it fails
root@m1:/home/administrator# mongo --port 27017 --ssl --host m1.com –-sslPEMKeyFile /etc/pemfile.pem –-sslCAFile /etc/cafile.pemMongoDB shell version v4.0.13 connecting to: mongodb://m1.com:27017/%E2%80%93-sslPEMKeyFile?gssapiServiceName=mongodb 2019-12-05T06:50:31.195-0500 I NETWORK [js] DBClientConnection failed to receive message from m1.com:27017 - SocketException: short read 2019-12-05T06:50:31.195-0500 E QUERY [js] Error: network error while attempting to run command 'isMaster' on host 'm1.com:27017' : connect@src/mongo/shell/mongo.js:344:17 @(connect):2:6 exception: connect failed root@m1:/home/administrator#
the current server is primary/master server i guess.
now, i am not getting whether it is cert issue OR mongoDB... how i can confirm that ?
Kevin Adistambha
Hi,
The message SocketException seems to imply that the issue is network rather than SSL.
Is there anything printed in the mongod logs at this time? Is the mongod bound to the correct IP since it’s bound to only 127.0.0.1 by default? Is the connection refused by the firewall?
Best regards,
Kevin
Irfan Sayed
ertificate. Ending connection from 192.168.43.2:40944 (connection id: 64)
2019-12-09T00:51:40.371-0500 I NETWORK [worker-2] end connection 192.168.43.2:40944 (0 connections now open)
2019-12-09T00:51:42.430-0500 I NETWORK [listener] connection accepted from 192.168.43.2:40952 #65 (1 connection now open)
2019-12-09T00:51:42.434-0500 E NETWORK [worker-2] SSL peer certificate validation failed: unable to get issuer certificate
2019-12-09T00:51:42.435-0500 I NETWORK [conn65] Error receiving request from client: SSLHandshakeFailed: SSL peer certificate validation failed: unable to get issuer certificate. Ending connection from 192.168.43.2:40952 (connection id: 65)
2019-12-09T00:51:42.435-0500 I NETWORK [worker-2] end connection 192.168.43.2:40952 (0 connections now open)
2019-12-09T00:51:42.494-0500 I NETWORK [listener] connection accepted from 192.168.43.2:40954 #66 (1 connection now open)
2019-12-09T00:51:42.498-0500 E NETWORK [worker-2] SSL peer certificate validation failed: unable to get issuer certificate
2019-12-09T00:51:42.498-0500 I NETWORK [conn66] Error receiving request from client: SSLHandshakeFailed: SSL peer certificate validation failed: unable to get issuer certificate. Ending connection from 192.168.43.2:40954 (connection id: 66)
2019-12-09T00:51:42.498-0500 I NETWORK [worker-2] end connection 192.168.43.2:40954 (0 connections now open)
2019-12-09T00:51:42.556-0500 I NETWORK [listener] connection accepted from 192.168.43.2:40956 #67 (1 connection now open)
2019-12-09T00:51:42.564-0500 E NETWORK [worker-2] SSL peer certificate validation failed: unable to get issuer certificate
2019-12-09T00:51:42.564-0500 I NETWORK [conn67] Error receiving request from client: SSLHandshakeFailed: SSL peer certificate validation failed: unable to get issuer certificate. Ending connection from 192.168.43.2:40956 (connection id: 67)
2019-12-09T00:51:42.564-0500 I NETWORK [worker-2] end connection 192.168.43.2:40956 (0 connections now open)
net:
port: 27017
bindIpAll: true
maxIncomingConnections: 65536 #default
wireObjectCheck: true
ssl:
mode: requireSSL
PEMKeyFile: /etc/ssl/private_mongodb/pemfile.pem
CAFile: /etc/ssl/private_mongodb/cafile.pem
allowConnectionsWithoutCertificates: false
serviceExecutor: adaptivemongo --port 27017 --ssl --host m1.com –-sslPEMKeyFile /etc/ssl/private_mongodb/pemfile.pem –-sslCAFile /etc/ssl/private_mongodb/cafile.pem) is being executed on the same mongodb machine where mongod daemon is running.
Kevin Adistambha
Hi,
I believe the error unable to get issuer certificate originated in OpenSSL and not MongoDB, so I’m thinking it’s due to the CA certificate. I don’t see any problem with your MongoDB config nor your mongo shell parameters.
So a quick search on that exact phrase led me to to this page, which coincidentally resides in Digicert’s site: https://knowledge.digicert.com/solution/SO12792.html
Could you try to do the troubleshooting steps listed in that page? You might also want to ask Digicert to help provide you with a CA certificate with the right chain on it.
Best regards,
Kevin
Irfan Sayed
2019-12-10T07:34:00.917+0000 I EXECUTOR [initandlisten] No thread count configured for executor. Using number of cores / 2: 2
2019-12-10T07:34:00.918+0000 I NETWORK [initandlisten] waiting for connections on port 27017 ssl
2019-12-10T07:34:00.918+0000 I EXECUTOR [worker-1] Started new database worker thread 1
2019-12-10T07:34:00.918+0000 I EXECUTOR [worker-2] Started new database worker thread 2
2019-12-10T07:34:29.805+0000 I NETWORK [listener] connection accepted from 127.0.0.1:38338 #1 (1 connection now open)
2019-12-10T07:34:29.810+0000 E NETWORK [worker-2] no SSL certificate provided by peer; connection rejected
2019-12-10T07:34:29.810+0000 I NETWORK [conn1] Error receiving request from client: SSLHandshakeFailed: no SSL certificate provided by peer; connection rejected. Ending connection from 127.0.0.1:38338 (connection id: 1)
2019-12-10T07:34:29.810+0000 I NETWORK [worker-2] end connection 127.0.0.1:38338 (0 connections now open)
SERVER CERTIFICATE FOR M1.COM
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
[ROOT CERTIFICATE]
[CA CERTIFICATE]
Kevin Adistambha
Hi,
SSLHandshakeFailed: no SSL certificate provided by peer
Did you supply a certificate on the mongo --sslPEMKeyFile <client certificate here> ... ?
Best regards,
Kevin
Irfan Sayed
- CA file should have the CA + Trusted Root certificate
- Pem file should have private key and server certificate
Irfan Sayed
root@mongo3:/home/serveradmin# openssl pkcs12 -in hcp-champ-user.pfx -out pem-certificate-and-key-file.pem
Enter Import Password:
MAC verified OK
Enter PEM pass phrase: “I did not put any password here”
root@mongo3:/home/serveradmin# mongo --port 27017 --ssl --host mongo3.ct.abc.com
MongoDB shell version v4.0.10
connecting to: mongodb://mongo3.ct.abc.com:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("594eff5a-80f3-4849-9326-895f4b5332e2") }
MongoDB server version: 4.0.10
Server has startup warnings:
2019-12-18T06:37:50.502+0000 I STORAGE [initandlisten]
2019-12-18T06:37:50.502+0000 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-12-18T06:37:50.502+0000 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten]
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten]
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten]
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2019-12-18T06:37:51.839+0000 I CONTROL [initandlisten]
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
> show databases
admin 0.000GB
config 0.000GB
local 0.000GB
> db.getSiblingDB("$external").runCommand(
... {
... createUser: "emailAddress=a...@abc.com,CN=hp-user,OU=OMongo,O=abc International Inc.,L=Morris Plains,ST=New Jersey,C=US",
... roles: [
... { role: "readWrite", db: "test" },
... { role: "userAdminAnyDatabase", db: "admin" }
... ],
... writeConcern: { w: "majority" , wtimeout: 5000 }
... }
... )
{ "ok" : 1 }
root@mongo3:/home/serveradmin# mongo --ssl --sslPEMKeyFile /home/serveradmin/pem-certificate-and-key-file.pem --sslCAFile /etc/ssl/private_mongodb/server.pem --authenticationDatabase '$external' --authenticationMechanism MONGODB-X509
2019-12-18T08:01:31.725+0000 E NETWORK [main] cannot read PEM key file: /home/serveradmin/pem-certificate-and-key-file.pem error:0906D06C:PEM routines:PEM_read_bio:no start line
Failed global initialization: InvalidSSLConfiguration Can not set up PEM key file.
root@mongo3:/home/serveradmin# openssl verify pem-certificate-and-key-file.pem
pem-certificate-and-key-file.pem: C = US, ST = New Jersey, L = Morris Plains, O = abc International Inc., OU = OMongo, CN = hp-user, emailAddress = a...@abc.com
error 20 at 0 depth lookup:unable to get local issuer certificate
Irfan Sayed
Kevin Adistambha
Hi,
I see the error during your “ authenticate the mongo client using SSL option “ step:
PEM_read_bio:no start line
This is not MongoDB’s error message. It’s from the underlying OpenSSL code. I would think that this is because your PEM file is malformed. Exactly how, I’m not really sure. Please check if your PEM file contains a structure similar to:
-----BEGIN PRIVATE KEY-----
....
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
....
-----END CERTIFICATE-----
If it’s not in this format, then it’s not in PEM format.
Also your OpenSSL verification is missing the CAFile parameter, so it’s saying that it’s unable to get local issuer certificate. You should supply the --CAFile parameter for this command containing the CA’s certificate in PEM format.
Best regards,
Kevin
Irfan Sayed
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/60112c28-be8a-4d3b-bc38-0e0555bef992%40googlegroups.com.
Kevin Adistambha
Hi,
Do you have any valid steps/guide/url for Digicert CA certificates to be working with MongoDB ? please suggest
Unfortunately I’m not familiar with Digicert and how they distribute their certificates. You might want to ask Digicert support to convert their certificate to the correct PEM format for you. You can probably ask them to provide the right certificates with the correct chain for use with MongoDB, since MongoDB uses standard PEM key file format.
Best regards,
Kevin
Irfan Sayed
2020-01-14T12:06:04.851+0000 I CONTROL [main] ***** SERVER RESTARTED *****
2020-01-14T12:06:04.853+0000 E NETWORK [main] cannot read certificate file: /root/cert/keycert.pem error:0200100D:system library:fopen:Permission denied
2020-01-14T12:06:04.853+0000 F CONTROL [main] Failed global initialization: InvalidSSLConfiguration: Can not set up PEM key file.
2020-01-14T12:15:41.882+0000 I CONTROL [main] ***** SERVER RESTARTED *****
2020-01-14T12:15:41.884+0000 E NETWORK [main] cannot read certificate file: /root/cert/keycert.pem error:0200100D:system library:fopen:Permission denied
2020-01-14T12:15:41.885+0000 F CONTROL [main] Failed global initialization: InvalidSSLConfiguration: Can not set up PEM key file.
2020-01-14T12:19:21.965+0000 I CONTROL [main] ***** SERVER RESTARTED *****
2020-01-14T12:19:21.971+0000 E NETWORK [main] cannot read certificate file: /root/cert/keycert.pem error:0200100D:system library:fopen:Permission denied
2020-01-14T12:19:22.004+0000 F CONTROL [main] Failed global initialization: InvalidSSLConfiguration: Can not set up PEM key file.
root@mongo1:~/cert# ll
total 20
drwxr-xr-x 2 mongodb mongodb 4096 Jan 14 10:47 ./
drwxrw-rw- 4 root root 4096 Jan 14 12:23 ../
-rw-r--r-- 1 root root 3055 Jan 14 10:43 caroot.pem
-rw-r--r-- 1 root root 4167 Jan 14 10:42 keycert.pem
root@mongo1:~/cert#
root@mongo1:/var/lib/mongodb# lltotal 200
drwxr-xr-x 4 mongodb mongodb 4096 Jan 14 10:37 ./
drwxr-xr-x 46 root root 4096 Jan 14 09:04 ../
-rw------- 1 mongodb mongodb 16384 Jan 14 10:37 collection-0-2281808182628794107.wt
-rw------- 1 mongodb mongodb 16384 Jan 14 10:37 collection-2-2281808182628794107.wt
-rw------- 1 mongodb mongodb 4096 Jan 14 10:37 collection-4-2281808182628794107.wt
drwx------ 2 mongodb mongodb 4096 Jan 14 10:37 diagnostic.data/
-rw------- 1 mongodb mongodb 16384 Jan 14 10:37 index-1-2281808182628794107.wt
-rw------- 1 mongodb mongodb 16384 Jan 14 10:37 index-3-2281808182628794107.wt
-rw------- 1 mongodb mongodb 4096 Jan 14 10:37 index-5-2281808182628794107.wt
-rw------- 1 mongodb mongodb 4096 Jan 14 10:37 index-6-2281808182628794107.wt
drwx------ 2 mongodb mongodb 4096 Jan 14 09:31 journal/
-rw------- 1 mongodb mongodb 16384 Jan 14 10:37 _mdb_catalog.wt
-rw------- 1 mongodb mongodb 0 Jan 14 10:37 mongod.lock
-rw------- 1 mongodb mongodb 16384 Jan 14 10:37 sizeStorer.wt
-rw------- 1 mongodb mongodb 114 Jan 14 09:31 storage.bson
-rw------- 1 mongodb mongodb 45 Jan 14 09:31 WiredTiger
-rw------- 1 mongodb mongodb 4096 Jan 14 10:37 WiredTigerLAS.wt
-rw------- 1 mongodb mongodb 21 Jan 14 09:31 WiredTiger.lock
-rw------- 1 mongodb mongodb 1065 Jan 14 10:37 WiredTiger.turtle
-rw------- 1 mongodb mongodb 57344 Jan 14 10:37 WiredTiger.wt
root@mongo1:/var/lib/mongodb#
Kevin Adistambha
Hi,
The error is: fopen:Permission denied and the key is owned by root:
-rw-r--r-- 1 root root 4167 Jan 14 10:42 keycert.pem
In contrast, MongoDB is using the mongodb user & group.
Unfortunately at this point I can only point out the cause. SSL keys are very sensitive information that should be handled with extreme care and policy, since anyone having access to the keys can impersonate you. If this key is used as part of a larger website deployment, then it’s even riskier since if the key inadvertently got leaked, someone can pretend to be your website, and anyone connected to the fake website wouldn’t be able to tell the difference.
It appears that you have so much trouble setting this up. Have you considered using MongoDB Atlas instead where all this setup was done for you, and you’ll have access to a deployment already setup with SSL and with security best practices?
Best regards,
Kevin