[grpc][c++] pem_root_certs & GRPC_SSL_REQUEST_CLIENT_CERTIFICATE_BUT_DONT_VERIFY

[natha...@skeyecode.com]

Hi,

I am currently putting in place an internal CA, and I have encountered an issue with the SSL behavior.

## case 1

- GetServerCredentials: pem_root_certs = "";
- GetClientSslCredentials: ssl_opts.pem_root_certs =
      ca_cert;

-> GetPeerIdentity empty: call fails
expected result: should work

## case 2

- GetServerCredentials: ssl_opts.pem_root_certs = ca_cert;
- GetClientSslCredentials: ssl_opts.pem_root_certs = "";

-> "Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed"
Logical: server's cert is not trusted by the client

expected result: logical b/c server's cert if not trusted by the system

## case 3

NOTE: this is the "standard case"

- GetServerCredentials: ssl_opts.pem_root_certs = ca_cert;
- GetClientSslCredentials: ssl_opts.pem_root_certs =
      ca_cert;

-> Ok [GetPeerIdentity returns the client's cert]

## case 4

- GetServerCredentials: ssl_opts.pem_root_certs = FAKE_ca_cert;
- GetClientSslCredentials: ssl_opts.pem_root_certs =
      ca_cert;

->
I1128 16:45:54.518586236    8769 subchannel.cc:656]          New connected subchannel at 0x60400001d550 for subchannel 0x616000031280
E1128 16:45:54.518711293    8789 ssl_transport_security.cc:472] Corruption detected.
E1128 16:45:54.518737349    8789 ssl_transport_security.cc:448] error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding
E1128 16:45:54.518749675    8789 ssl_transport_security.cc:448] error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed
E1128 16:45:54.518760139    8789 ssl_transport_security.cc:448] error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP lib
E1128 16:45:54.518768124    8789 secure_endpoint.cc:181]     Decryption error: TSI_DATA_CORRUPTED
D1128 16:45:54.519019924    8769 dns_resolver.cc:259]        In cooldown from last resolution (from 13 ms ago). Will resolve again in 987 ms
../tests/test_route_device_autolink.cpp:79: Failure
Value of: res.ok()
  Actual: false
Expected: true

My question is: why does the server need to have the CA's crt?
It makes sense that it needs to be present on the client when using a self-signed server cert(or any non system-trusted ones).

Is this a bug or the expected behavior?

My problem is that I am using(or trying to use) cfssl as CA, and I would rather avoid copying around the root crt(even if its not at all a security issue).
That is because cfssl API does not allow me to access the CA's .crt, so it would have to copy it out-of-band.

NOTE: it could be related to https://github.com/grpc/grpc/issues/12146

[jian...@google.com]

I would like to understand your setting better. Does your client present a certificate? 

If client does not have a certificate, you could use GRPC_SSL_DONT_REQUEST_CLIENT_CERTIFICATE.

When client presents a certificate to server, server needs to verify the certificate that is chained back to root certificates. That is reason server needs CA's cert (as root pem) so that it could verify client certificate. So it is expected behavior.

Does this answer your question?

[natha...@skeyecode.com]

Hi,

Indeed the clients present certificates.
The short question is: why when the server is set to GRPC_SSL_REQUEST_CLIENT_CERTIFICATE_BUT_DONT_VERIFY does it need the root cert?
In this case it's specifically not verifying the client, so there is no point in needing it? Or am I missing something?
What I would expect is both "case 1" and "case 3" above to work, b/c it should make no difference if the server has a root cert to check against in my setup.

Also this not consistent with eg gRPC-java(ie the prod client), they are working fine without the root cert(although ssl verification must obliviously be disabled b/c I am using self-signed certs).

[Jiangtao Li]

Nathan,

If server side sets GRPC_SSL_REQUEST_CLIENT_CERTIFICATE_BUT_DONT_VERIFY, then server does not need to set client's CA cert as root cert. 

On your case (1), did you see handshake succeed? Where did you see the failure, during GetPeerIdentity? It may be a bug, let me reproduce.

Thanks,

Jiangtao

--
You received this message because you are subscribed to a topic in the Google Groups "grpc.io" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/grpc-io/KukId711HTE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/20e05913-4042-4f4f-bde3-2a42abd7515e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jiangtao Li]

I just ran helloworld greeter client and greeter server, where server side does not have any root pem. Server side uses GRPC_SSL_REQUEST_CLIENT_CERTIFICATE_BUT_DONT_VERIFY. Client presents its certificate to server. Client and server can talk. GetPeerIdentity() did not crash. I may need more details before I can reproduce.

[Jiangtao Li]

Nathan,

On server side, you can use the following to get the client' certificate

  std::vector<grpc::string_ref> cert_list =

      server_context.auth_context().FindPropertyValues(GRPC_X509_PEM_CERT_PROPERTY_NAME);

Thanks,

Jiangtao

[natha...@skeyecode.com]

The problem is that this property is not there when the server has no root cert(I'm pretty sure GetPeerIdentity uses GRPC_X509_

PEM_CERT_PROPERTY_NAME)

:

//   ssl_opts.pem_root_certs = ca_cert;

I1213 10:55:11.344137202   19144 subchannel.cc:656]          New connected subchannel at 0xad4830 for subchannel 0x7fffd0008400
D1213 10:55:11.344300801   19153 grpc_auth.h:45]             DebugPrintAuthContext: transport_security_type = ssl
D1213 10:55:11.344309721   19153 grpc_auth.h:45]             DebugPrintAuthContext: ssl_session_reused = false
D1213 10:55:11.344314347   19153 grpc_auth.h:33]             DebugPrintInputMetadata: :authority = localhost:11043 [hex: 6c6f63616c686f73743a3131303433]
D1213 10:55:11.344320586   19153 grpc_auth.h:33]             DebugPrintInputMetadata: :path = /skcapipb.SkcApi/DeviceIdentityAutolink[hex: 2f736b6361706970622e536b634170692f4465766963654964656e746974794175746f6c696e6b]
D1213 10:55:11.344323637   19153 grpc_auth.h:33]             DebugPrintInputMetadata: accept-encoding = identity,gzip [hex: 6964656e746974792c677a6970]
D1213 10:55:11.344326047   19153 grpc_auth.h:33]             DebugPrintInputMetadata: app_id = TODO_test_app_id [hex: 544f444f5f746573745f6170705f6964]
D1213 10:55:11.344328340   19153 grpc_auth.h:33]             DebugPrintInputMetadata: app_key = TODO_test_app_key [hex: 544f444f5f746573745f6170705f6b6579]
D1213 10:55:11.344330996   19153 grpc_auth.h:33]             DebugPrintInputMetadata: grpc-accept-encoding = identity,deflate,gzip [hex: 6964656e746974792c6465666c6174652c677a6970]
D1213 10:55:11.344333631   19153 grpc_auth.h:33]             DebugPrintInputMetadata: grpc-trace-bin =  [hex: 000085c7a4e05140a4080a09f1925b739819016b27e1e513b773ed0200]
D1213 10:55:11.344336142   19153 grpc_auth.h:33]             DebugPrintInputMetadata: user-agent = grpc-c++/1.16.0 grpc-c/6.0.0 (linux; chttp2; gao) [hex: 677270632d632b2b2f312e31362e3020677270632d632f362e302e3020286c696e75783b206368747470323b2067616f29]

VS

ssl_opts.pem_root_certs = ca_cert;

I1213 10:54:27.822408055   18834 subchannel.cc:656]          New connected subchannel at 0xad35d0 for subchannel 0x7fffd4008400
D1213 10:54:27.822553761   18854 grpc_auth.h:45]             DebugPrintAuthContext: transport_security_type = ssl
D1213 10:54:27.822562749   18854 grpc_auth.h:45]             DebugPrintAuthContext: x509_common_name = localhost
D1213 10:54:27.822568911   18854 grpc_auth.h:45]             DebugPrintAuthContext: x509_pem_cert = -----BEGIN CERTIFICATE-----
MIIFNDCCAxwCAQEwDQYJKoZIhvcNAQELBQAwXjELMAkGA1UEBhMCRlIxDzANBgNV
BAgMBkZyYW5jZTEOMAwGA1UEBwwFUGFyaXMxDTALBgNVBAoMBFRlc3QxDTALBgNV
BAsMBFRlc3QxEDAOBgNVBAMMB1Jvb3QgQ0EwHhcNMTgxMTI5MTQ1NzI4WhcNMTkx
MTI5MTQ1NzI4WjBiMQswCQYDVQQGEwJGUjEPMA0GA1UECAwGRnJhbmNlMQ4wDAYD
VQQHDAVQYXJpczENMAsGA1UECgwEVGVzdDEPMA0GA1UECwwGQ2xpZW50MRIwEAYD
VQQDDAlsb2NhbGhvc3QwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQD3
TojD/WAZvn17s3PKXKQFwB2FQ3SEkM06UUJXe3W7Lto4yBdu6gWK7BGrJCqN1jkA
cQKqoWQ1DgnlZQXaVeX/RiJTRb6jIPd8Kk620sYjcAQ/F3EI0yhlH+qszJzYeMSh
nCyvmqWNdHSKB7X8R/r+WikhOr22oJskDwG+skE0ehBzXc/0c4euwK+POI89yv1B
YIk9EgWHr2puU8wGFFVj3DHo+p//BXf9SH3kTIYidiX7z/zN4vTAwU10SgUxFXJX
DU5X0mOv1AqF5o+ELJ7lQClX0gpZ5zuHrVMPwlgJyaWiO8MPoBP7ozNZdJihu5La
9v/O9RfCKpNxO/wvCj80igY+ymIw9n0+ouNkkrQOVUCxThZVD9rIRWd9FajZeiji
wYi7P8keU+UdlQBa0Y/OcP6hst1b+2Pm5i7HJz5ZXgsEAhywGvuKw//+xILlfoIn
UCFVafgBaMdLdauTlUu+Zcw6kPepBBt79p9nanKo3W2xlq1tESjCS8XIN9YUWYsA
IFLMhW3HYIvnrhNHomh6adCYGTBiLxn5r0jB9N3Xf0kx4pTI6GiTYMhGVJcejlyc
KwEqy6vHWQj47rYuyLOWnBtwjJ+itp3NYDaMxz1QveIiE0ECveQEjcVYU1gJTgwo
0ieoVDub0YCTeo40Lir67JdSOeInHByqoO6ptOXfPwIDAQABMA0GCSqGSIb3DQEB
CwUAA4ICAQCx4HGvc9vNXRn76IjMKl9Jj/84T5rYumNPRPu7GgL5EgvdPGRO0baq
ETL1+iOlNwkNM91K8ggDeUutj+od6RD9lgBMuzeFsY/mCkKNxzn8D7/vjUIDkar9
C7BUABh8V0CqhMcBJ0kOramjtM/Vzb0DGlU8W8rqd9rCHBg+X/Ut6mf0yofD1aAx
oBZfSDzDWgMs6tLpFMEnA+4cQqcyThbcTwRGi2uthk0sYmZm5mMgcRpDEARC4wah
9WiDeioiw5fmhq8i6F2PWzYelSkY3uynPITtQljStqziseV5ZsVOUPqmCffDIUuh
opluaLKVINPxRgn6ksxY526cebn/RSYLBDGPGhrORF1YGr9UYOL8DtTqYbnG7w4l
0EviO/R+OUYsNpt94cso3CnqvsN3Tnu+aEMX7d4/KJTTnfP5NlGPegN/UxQqufGo
R/jHUzOUd07186LqDMaz+em5Mj8lte9AjeY4WNSJCq2KK6tRac+efrUHXBiPFyI9
5GQCIpf0ZSFw1B+DPYfpOcLsGy1ETtddIMhDMY1gOyKBHNKW5mGZz5pdUmSgLSOW
Ka2CdLK1JNcLNaB2gM7Yrch58wVhoWaPTrL+IBqwth+5ATYY7fKv51tiPm+bRq8R
OD1YFpvISiZhcefhm0e1IQXB5wPorZMPnBt9LJhc9r73aVjcxUlMvw==
-----END CERTIFICATE-----

D1213 10:54:27.822593407   18854 grpc_auth.h:45]             DebugPrintAuthContext: ssl_session_reused = false
D1213 10:54:27.822597844   18854 grpc_auth.h:33]             DebugPrintInputMetadata: :authority = localhost:11043 [hex: 6c6f63616c686f73743a3131303433]
D1213 10:54:27.822603860   18854 grpc_auth.h:33]             DebugPrintInputMetadata: :path = /skcapipb.SkcApi/DeviceIdentityAutolink[hex: 2f736b6361706970622e536b634170692f4465766963654964656e746974794175746f6c696e6b]
D1213 10:54:27.822606896   18854 grpc_auth.h:33]             DebugPrintInputMetadata: accept-encoding = identity,gzip [hex: 6964656e746974792c677a6970]
D1213 10:54:27.822610022   18854 grpc_auth.h:33]             DebugPrintInputMetadata: app_id = TODO_test_app_id [hex: 544f444f5f746573745f6170705f6964]
D1213 10:54:27.822613058   18854 grpc_auth.h:33]             DebugPrintInputMetadata: app_key = TODO_test_app_key [hex: 544f444f5f746573745f6170705f6b6579]
D1213 10:54:27.822616576   18854 grpc_auth.h:33]             DebugPrintInputMetadata: grpc-accept-encoding = identity,deflate,gzip [hex: 6964656e746974792c6465666c6174652c677a6970]
D1213 10:54:27.822619776   18854 grpc_auth.h:33]             DebugPrintInputMetadata: grpc-trace-bin =  [hex: 0000ec265091bca686986b2eec96af55698301a5fcdc00770c12e10200]
D1213 10:54:27.822623045   18854 grpc_auth.h:33]             DebugPrintInputMetadata: user-agent = grpc-c++/1.16.0 grpc-c/6.0.0 (linux; chttp2; gao) [hex: 677270632d632b2b2f312e31362e3020677270632d632f362e302e3020286c696e75783b206368747470323b2067616f29]

Only diff is the commented out "ssl_opts.pem_root_certs = ca_cert;".
Client is grpc++(unittest client); it is is set to use a client certificate, and the same root cert as the server.

Does it work in your code? If yes I could have messed up when setting up the grpc::AuthMetadataProcessor, or something else.

[Jiangtao Li]

You are right. If server does not put client's CA cert as root cert, FindPropertyValues(GRPC_X509_PEM_CERT_PROPERTY_NAME) return nothing back. It seems a bug to me. Let me debug and get back to you. Thanks much for reporting this.

Thanks,

Jiangtao

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/f5307d2d-1d3d-4e23-b0c0-66a8e28128c6%40googlegroups.com.

[Jiangtao Li]

+David Benjamin +Julien Boeuf 

David, 

Here is scenario we have, we want to check whether it is expected behavior in boringssl. Client presents a certificate to server. However, server does not put client's CA certificate in the root pem. 

On server side, before handshake

SSL_CTX_set_verify(SSL_VERIFY_PEER, AlwaysAcceptCallback); // AlwaysAcceptCallback is customized X509_STORE_CTX_set_verify_cb and always returns 1, which means server does not really verify the client's certificate chain.

After client/server handshake successfully,

SSL_get_peer_certificate() returns NULL. Is this intended behavior of SSL? I expect boringssl would return client certificate.

If server puts client's CA cert in its root pem, then SSL_get_peer_certificate will return the client's cert correctly.

Thanks,

Jiangtao

[David Benjamin]

SSL_get_peer_certificate should return the certificate, yeah. Are you sure the client sent one? Returning NULL should mean it didn't. Maybe configure your verify callback to print something and see if it prints anything.

D1213 10:54:27.822610022   18854 grpc_auth.h<span style="color:rgb

[David Benjamin]

Re "If server puts client's CA cert in its root pem, then SSL_get_peer_certificate will return the client's cert correctly", servers send, in the CertificateRequest message, the CAs they accept. A lot of clients will use that list to filter candidate client certificates and not even consider certificates which don't match.

opluaLKVINPxRgn6ksxY526cebn/<span style="color:rgb(102,0,102)" class="m_8444382121985872722m_6520396557009402669m_3633894517602340111gmail-m_1529790313407667673gmail-m_-3830410836943948300styled

[Jiangtao Li]

Thank you David for helping me debug. It is indeed a bug introduced in https://github.com/grpc/grpc/commit/7ae3733cab31c9c8b06dc5961984e063685de261.

Fixing coming.

Thanks,

Jiangtao

[Jiangtao Li]

Nathan,

https://github.com/grpc/grpc/pull/17500 shall fix your problem. I verified locally. The code was this way in the very beginning.

Thanks,

Jiangtao