RavenDB 4 - certificates

[Andrej Krivulčík]

I'm being thick here or missing something obvious, but is there any documentation on how to configure certificates for RavenDB 4? How to generate the certificates, what are requirements for them etc. This is not obvious for me as someone not familiar with this topic so I would appreciate any pointers here.

[Oren Eini (Ayende Rahien)]

There are two types of certificates in play here.

The first is the certificate you need to run HTTPS, this is a standard SSL certificate, with all the usual rules.

Next, there are the client certificates, which can be generated by RavenDB directly without anything you need to do.

The easiest way to handle that is via:

https://github.com/ravendb/ravendb/tree/v4.0/scripts/certificates

We'll have full docs for the next milestone release.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Fri, Aug 25, 2017 at 5:23 PM, Andrej Krivulčík <kriv...@gmail.com> wrote:

I'm being thick here or missing something obvious, but is there any documentation on how to configure certificates for RavenDB 4? How to generate the certificates, what are requirements for them etc. This is not obvious for me as someone not familiar with this topic so I would appreciate any pointers here.

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrej Krivulčík]

Thanks Oren for the explanation and the link to the scripts, that helped. I could get the server certificate running, I'm still struggling with the client certificates, will continue next week.

I'll describe issues I encountered, it might help you write the documentation.

1. Not sure which certificate generation script should I use so let's go with the Windows script (generate-server-cert.ps1).

• Edit the values to match my needs and successfully generate the certificate.
  
• Server startup fails with the following:
  
• System.InvalidOperationException: Unable to start the server due to  invalid certificate configuration! Admin assistance required. ---> System.InvalidOperationException: Could not load certificate file C:\[...].pfx ---> Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException: The specified network password is not correct
  

2. I honestly have no idea what this means so I fall back to the Linux script (generate-selfsigned-cert.sh), as I'm a bit familiar with openssl.

• Edit the values, generate the key and certificates.
  
• Same error, The specified network password is not correct.
  
• Realize that the certificate file is encrypted, no idea if is there a way to specify the certificate passphrase in the config.
  

3. -nodes to the rescue, generate the certificate unencrypted, server starts up successfully.

• Unable to do anything in the studio as I don't have a client certificate.
  
• When looking around in the studio (with SSL disabled), I can't find any way to manage client certificates.
  

4. Handy script obtain-client-cert.ps1 is available, let's give it a try: changed the URL and certificate file (to point to the server certificate).

• Output:
  

...

Sending client cert request: {

    "SecurityClearance":  "ClusterAdmin",

    "Name":  "client-test",

    "Password":  null,

    "Permissions":  {

                    }

}

VERBOSE: POST https://localhost/admin/certificates with -1-byte payload

Invoke-WebRequest : The remote server returned an error: (500) Internal Server Error.

...

• -1-byte payload - I'm not very familiar with Invoke-WebRequest so I just assume this is not a problem.
  
• Fire up Fiddler and the output changes (no idea why - when I run the script directly it always returns 500):
  

...

Sending client cert request: {

    "SecurityClearance":  "ClusterAdmin",

    "Name":  "client-test",

    "Password":  null,

    "Permissions":  {

                    }

}

VERBOSE: POST https://localhost.fiddler/admin/certificates with -1-byte payload

Invoke-WebRequest : The remote server returned an error: (403) Forbidden.

...

• Response in Fiddler says: {"Type":"InvalidAuth","Message":"This server requires client certificate for authentication, but none was provided by the client"}
• Ok, so I need to bootstrap the first client certificate without SSL enabled.

5. Disable SSL in the server, change the URL in the script, run the script, 500 Internal Server Error again, no way to find out why without Fiddler.

•  - Response in Fiddler: {"Error":"System.FormatException: Could not convert Sparrow.Json.BlittableJsonReaderObject ('{}') to Sparrow.Json.BlittableJsonReaderArray ---> System.InvalidCastException: Object must implement IConvertible.\r\n... ...
  

6. Change the permissions (https://github.com/ravendb/ravendb/blob/v4.0/scripts/certificates/obtain-client-cert.ps1#L24) to be array instead of a dictionary, remove lines 25-27 as I don't have DatabaseNames and DatabaseAccess.

• {"Error":"System.NullReferenceException: Object reference not set to an instance of an object.\r\n   at Raven.Server.Utils.CertificateUtils.CreateSelfSignedClientCertificate(String subjectName, CertificateHolder certificateHolder) in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Utils\\CertificateUtils.cs:line 30\r\n   at Raven.Server.Web.Authentication.AdminCertificatesHandler.<Generate>d__0.MoveNext() in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Web\\Authentication\\AdminCertificatesHandler.cs:line 34\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Raven.Server.Routing.RequestRouter.<HandlePath>d__5.MoveNext() in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Routing\\RequestRouter.cs:line 108\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()\r\n   at Raven.Server.RavenServerStartup.<RequestHandler>d__12.MoveNext() in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\RavenServerStartup.cs:line 154","Message":"Object reference not set to an instance of an object.","Type":"System.NullReferenceException","Url":"/admin/certificates"}
  

It's getting really late, I'll tackle this next week, but I'm starting to get lost. Not sure if I should have generated an authority certificate (generate-authority-cert.ps1) and use it for generating the server certificate or I'm missing some server configuration option or there is another problem somewhere. Also, I have several questions:

•  - What is the recommended sequence of SSL certificate generation? Is my process okay (generate the certificate in a Linux machine, nothing else)? Or when running the server on a Windows system, I need to generate the authority certificate?
  
•  - Is there a way to configure the certificate passphrase in the server config?
  
•  - Is there a way to manage client certificates using the studio?
  
•  - Do I need to bootstrap the first client certificate before enabling SSL?
  
•  - What is the correct usage of obtain-client-cert.ps1?
  
•  - Is $CertPath supposed to be configured to the server certificate (it's used in a variable $serverCert later, which implies yes, but is sent in -Certificate parameter in Invoke-WebRequest, which specifies the client certificate so I'm confused here).
  
•  - Is $permissions supposed to be a dictionary?
  
•  - What values can be used in $DatabaseAccess?
  

Sorry for the wall of text and lots of questions but although I try to follow the RavenDB 4 development and the news on the blog and in this group, it's difficult to figure out certain details. Also, I guess that documentation for all this is not (yet) available so I might be missing the understanding of crucial concepts behind the system. All of this would be very useful if included in official documentation.

Thanks

Andrej

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

Thank you for the detailed post.

The UI for managing certs has just landed in the code, so it is likely not on your build yet.

The rough logic we follow with the certs:

- generate the HTTPS certificate  (using OpenSSL, the pass phrase is test in the scripts you have), but I have no idea what the ps1 is using.

- Once you have that sorted out and the server started, you'll typically not be able to do anything, because no certificate was installed as trusted.

- At that point, you can either use the insecure mode to register, certificate and switch, or you can use the same server certificate (it already knows it and trust it, after all) as the initial command to register the first client certificate.

All of which will be in the docs around the same time as our next release, and I'll make sure that this email is front and center when this is done to make sure we write it clearly.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Andrej Krivulčík]

Oren, thanks again for your clarification.

When I try to use the server certificate as the client certificate as suggested, I receive a different errors when using Invoke-WebRequest (from Windows) and curl (from Linux). The Windows error message is in my previous email under point #6 (System.NullReferenceException: Object reference not set to an instance of an object.\r\n   at Raven.Server.Utils.CertificateUtils.CreateSelfSignedClientCertificate(String subjectName, CertificateHolder certificateHolder) in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Utils\\CertificateUtils.cs:line 30).

When using curl under Linux (curl -vvvv --insecure -X POST -d '{ "SecurityClearance": "ClusterAdmin", "Name": "client-test", "Password": null, "Permissions": [] }' --cert 'db-1-keycert.crt:test' -H 'Content-Type: application/json'  'https://[...]/admin/certificates'), I get the following error in response:

{"Error":"System.ArgumentException: Unknown object id - S - passed to distinguished name\r\n   at Org.BouncyCastle.Asn1.X509.X509Name.DecodeOid(String name, IDictionary lookUp)\r\n   at Org.BouncyCastle.Asn1.X509.X509Name..ctor(Boolean reverse, IDictionary lookUp, String dirName, X509NameEntryConverter converter)\r\n   at Raven.Server.Utils.CertificateUtils.CreateSelfSignedCertificateBasedOnPrivateKey(String subjectName, String issuerName, AsymmetricKeyParameter issuerPrivKey, Boolean isClientCertificate, Int32 yearsUntilExpiration) in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Utils\\CertificateUtils.cs:line 73\r\n   at Raven.Server.Utils.CertificateUtils.CreateSelfSignedClientCertificate(String subjectName, CertificateHolder certificateHolder) in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Utils\\CertificateUtils.cs:line 30\r\n   at Raven.Server.Web.Authentication.AdminCertificatesHandler.<Generate>d__0.MoveNext() in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Web\\Authentication\\AdminCertificatesHandler.cs:line 34\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Raven.Server.Routing.RequestRouter.<HandlePath>d__5.MoveNext() in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\Routing\\RequestRouter.cs:line 108\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()\r\n   at Raven.Server.RavenServerStartup.<RequestHandler>d__12.MoveNext() in C:\\Builds\\RavenDB-4.0-Beta\\src\\Raven.Server\\RavenServerStartup.cs:line 154","Message":"Unknown object id - S - passed to distinguished name","Type":"System.ArgumentException* Connection #0 to host 104.45.15.249 left intact

","Url":"/admin/certificates"}

I'm not familiar with BouncyCastle so I'm not sure what is missing. The requests *should* be identical in Windows and Linux, but I'm struggling to figure out how to preserve the client certificate being sent when proxying the request using Fiddler.

Could you briefly describe steps how to initialize a fresh database with first client certificate from stock unconfigured installation to the certificate available?

My steps which ended with the error messages above:

1. Generate the server certificate using the Linux script with -nodes option in the last step.
   
2. Configure the server to use the certificate ("ServerUrl": "https://0.0.0.0:443", "PublicServerUrl": "https://IPADDRESS:443", "Security.Certificate.Path": "D:\\RavenDB\\db-1.pfx" in settings.json).
3. Call the /admin/certificates endpoint using the scripts above, with the following payload: { "SecurityClearance": "ClusterAdmin", "Name": "client-test", "Password": null, "Permissions": [] }.

I did not NOT generate the CA certificate using https://github.com/ravendb/ravendb/blob/v4.0/scripts/certificates/generate-authority-cert.ps1 - is this necessary? Should the CA certificate be imported to trusted certificate store for some reason?

I just found a way to configure the certificate password in server configuration (Security.Certificate.Password), which was the reason of the error message in my first point. Please note the double space in the error message (Unable to start the server due to [DOUBLE SPACE HERE] invalid certificate configuration!), which led me to believe that there is a missing text there. After using the certificate generated in Windows, the obtain-client-cert.ps1 obtains the certificate successfully.

I'm leaving the info above here, maybe it will be useful for polishing the documentation, and to summarize, the following items could be improved:

• Server doesn't generate client certificates when configured to use a certificate without password generated using openssl.
• obtain-client-cert.ps1 should use an array for Permissions.

Andrej

[iftah]

Hi Andrej,

It seems that you are working with an outdated build. The certificate generation process was refined since the last beta release (40018).
For example, we fixed the null reference exception and the permissions are a dictionary and not an array.

So the first step is to pull the latest changes and rebuild. Or wait a few days for the next release.

I'm writing a quick getting-started guide. It is not complete yet but it can definitely help you with these first steps. Please take a look:
https://docs.google.com/document/d/1dS-bnp5bhbXqbsclEylZyVuC8MmF9W0BVyUuBWmzNx8/edit?usp=sharing

If you have any issues after reading the guide, or something in the guide is not clear enough, please let me know.

Best regards,

 

Hibernating Rhinos Ltd  

Iftah Ben Zaken l RavenDB Core Team 

[Andrej Krivulčík]

Iftah, thanks for the information, I'm using the latest beta build 40018 so that's probably the reason of the mismatches. I'll be out of office until next week so I'll give the next build a try.

From a quick look, the guide you are working on seems to be very informative, I'll check it and let you know if I'll have any suggestions.

Andrej