Talking to backends with private CA signed certificates

Jörg Erdmenger

unread,

Apr 3, 2023, 11:10:17 AM4/3/23

to comm...@krakend.io

Hi,

I'm trying to run krakend on kubernetes talking to TLS encrypted kubernetes services that are signed by a cluster internal CA. I have configured TLS like so for krakend:

"tls": { 
  "private_key": "/etc/krakend-src/tls/tls.key", 
  "public_key": "/etc/krakend-src/tls/tls.crt", 
  "ca_certs": [ "/etc/krakend-src/tlsCA/service-ca.crt" ]
}
and kubernets now happily talks to the __health endpoint via tls. 
If I now call one of the endpoints that is configured I get an error like this:

2023/04/03 14:50:17 KRAKEND ERROR: [ENDPOINT: /test/hello] Get 
"https://k5-apismpl.banking-solutions-dev.svc/apismpl/api/hello/hello": 
tls: failed to verify certificate: x509: certificate signed by unknown 
authority
[GIN] 2023/04/03 - 14:50:17 | 500 |    6.631086ms |  192.168.101.10 | 
GET      "/test/hello"
Error #01: Get 
"https://k5-apismpl.banking-solutions-dev.svc/apismpl/api/hello/hello": 
tls: failed to verify certificate: x509: certificate signed by unknown 
authority

I understand that the ca_certs property is apparently only used for mTLS cases - at least 
this is how I read https://www.krakend.io/docs/authorization/mutual-authentication/#mtls-example

But I'm wondering if I miss out on anything - is that now a fairly common usecase, that I 
want to talk to private endpoints that are secured by private CAs? And if I want to achieve this, is 
then really the only way to rebuild the krakend image https://hub.docker.com/r/devopsfaith/krakend 
and add my private CA there? It feels like a bit too static for kubernetes deployments,
if I need to rebuild my container images when my cluster CA is rolled over.

Any help is appreciated - because other than that krakend looks really promising and lightweight

jörg

David Hontecillas

unread,

Apr 4, 2023, 4:10:39 AM4/4/23

to KrakenD Community, jo...@woerd.org

Hi,

You are right, that is the approach.

In the soon to be released next krakend CE version, there will be additional client_tls options and you will be able to add your self signed certificate in a more convenient way (you can take a look at this lura PR https://github.com/luraproject/lura/pull/652 to have a preview or the new config settings)

Meanwhile you have two options:

- what you said: create your own image from the krakend one, and add you self signed certificate to the image and do the `update-ca-certificates` in order to be able to connect to self signed endpoints.

Should be something like this:

```

FROM devopsfaith/krakend:2.2.1

ADD your_ca_root.crt /usr/local/share/ca-certificates/foo.crt

RUN chmod 644 /usr/local/share/ca-certificates/foo.crt && update-ca-certificates

```

- If you are not working in a production environment, you can use the allow_insecure_connections flag to skip certificate validation when making requests to backends: https://www.krakend.io/docs/service-settings/http-transport-settings/#allow_insecure_connections

Hope that helps.

Regards.

Max Yakovenko

unread,

Jul 15, 2023, 5:14:24 AM7/15/23

to KrakenD Community, David Hontecillas, jo...@woerd.org

Hi there!

I experience almost the same issue, but with keycloak. I am getting the following error:

```

2023/07/15 09:01:13 http: TLS handshake error from 127.0.0.1:52308: remote error: tls: bad certificate
[KRAKEND] 2023/07/15 - 09:01:13.901 ▶ ERROR [ENDPOINT: /debug/oidc][JWTValidator] Unable to validate the token: Get "https://keycloak.cs.traefik.me/realms/cs/protocol/openid-connect/certs": tls: failed to verify certificate: x509: certificate signed by unknown authority
[GIN] 2023/07/15 - 09:01:13 | 401 | 12.628069ms | 172.31.0.4 | GET "/debug/oidc"

```

I made self-signed certificates which I pass to keycloak and krakend as well. They are wildcard ones. This is my configuration file for krakend:

```

"version": 3,
"$schema": "https://www.krakend.io/schema/v2.4/krakend.json",
"name": "CS API Gateway",
"port": 443,
"debug_endpoint": true,
"echo_endpoint": true,
"timeout": "3000ms",
"cache_ttl": "300s",
"tls": {
"public_key": "/etc/x509/public.cert.pem",
"private_key": "/etc/x509/private.key.pem",
"min_version": "TLS11"
},
"client_tls": {
"allow_insecure_connections": true
},
"endpoints": [
{
"endpoint": "/debug/oidc",
"extra_config": {
"auth/validator": {
"operation_debug": true,
"alg": "RS256",
"cache": true,
"jwk_url": "https://keycloak.cs.traefik.me/realms/cs/protocol/openid-connect/certs"
}
},
"backend": [
{
"@comment": "An endpoint for authentication testing",
"host": ["https://krakend.cs.traefik.me"],
"url_pattern": "/__debug/auth"
}
]

}
],

```

Here is the response from keycloak (https://keycloak.cs.traefik.me/realms/cs/protocol/openid-connect/certs):

```

{
"keys": [
{
"kid": "S-xahxHPaedmY_CJNDrzkwxmU4kP-jSs0ebUQVE16BE",
"kty": "RSA",
"alg": "RS256",
"use": "sig",
"n": "pilMRFy-I2q_1GamXAPaa4toRDspHiLSLnSSxIKBU_iF0Tgtzk6nfOIQtb6QPGqvUuOepl_oVEuhjdMAnsig1FG9Kwr5K8b3PdX5QdlpeU0Sy0jiQ5-NnzeLUMMnwu0N0rUUFY4jXde9874mhAZ6pox1p1ZU980RXjW6wLgRTBAjNn0ktfiXh5NamdOF_A3aS4oQnu0dgd3z0f6ZagFYLDYuQkOmvKalNwy7QuAJhhoC4BBzAYrCBUU1p8IeYZRQ3_Yi3Qx1w5TG61KIL-NhGlZCRPfCeFr9w62goZUA9Cg3ostVlXbZK8_liMJJcVZxdS32f027KNv-OfChKHJ7Tw",
"e": "AQAB",
"x5c": [
"MIICyzCCAbMCBgGE4eSebjANBgkqhkiG9w0BAQsFADApMScwJQYDVQQDDB53YXRzb24tYXNzaXN0YW50LW5vZGVqcy1zZXJ2ZXIwHhcNMjIxMjA1MTA0NDI3WhcNMzIxMjA1MTA0NjA3WjApMScwJQYDVQQDDB53YXRzb24tYXNzaXN0YW50LW5vZGVqcy1zZXJ2ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCmKUxEXL4jar/UZqZcA9pri2hEOykeItIudJLEgoFT+IXROC3OTqd84hC1vpA8aq9S456mX+hUS6GN0wCeyKDUUb0rCvkrxvc91flB2Wl5TRLLSOJDn42fN4tQwyfC7Q3StRQVjiNd173zviaEBnqmjHWnVlT3zRFeNbrAuBFMECM2fSS1+JeHk1qZ04X8DdpLihCe7R2B3fPR/plqAVgsNi5CQ6a8pqU3DLtC4AmGGgLgEHMBisIFRTWnwh5hlFDf9iLdDHXDlMbrUogv42EaVkJE98J4Wv3DraChlQD0KDeiy1WVdtkrz+WIwklxVnF1LfZ/Tbso2/458KEocntPAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAC+kasu34iPkkY9fodKNeD9CNUG5d19PU9aylyYnmOUJ6fUYYsW/j9bYhiJ8FHlzczSNKtJmD3zOfsM3afUYzjsVEhaimh66qZU+esLK2pJDb6hpnwYTLgYHL/6+yhKVxQUEtsYV/kee0Ewmh3lCmMltZPgfs3HbK/JB6TldTqtnxuBiRE9XKK+IxEUupdjY/zNTU4/fSefzqIY7IRyIiaIiKutLjVpV3vweEb8HJfohNA8m7hrM4trFrlvGob8JFCaGnjjcKJ5NphQnMrXgROMQaqWtT5W0iSJeAUYEHNzXxb5nMO6+Usk+Hx8gpq30uEhb6L3p5CHCAXzknH7HrIs="
],
"x5t": "uPIy6e0VNHpI7lNIVFOu4YXxRuo",
"x5t#S256": "EBQYKxpVvntlVvyS8KSaCuPhpbini9EUAnkdzU7mZIg"
},
{
"kid": "3Jsa5RpbyK9SNcLFQUM4jxeAI40bhWsOUS_iSNTjKmA",
"kty": "RSA",
"alg": "RSA-OAEP",
"use": "enc",
"n": "qNtXt84el7c33-uiw0z76Lho6uX2gyEO4mDyOXn0qWOyztUcme0uyI1Vs8Z9ay5ic4OXPjkgUyhuEdAxaT7j56Rnp-v-6CDzbcKtGFeGc7gaqYtlIoORYzNDKjd-EGZiDylXCbkZu4w4I-9iiycxbmCS1if2PrViGhCs_4tvswyY71u-v3krz69H1upgKPfkjWbASuewX0zcaRRf76ufIjRudCTsF9bywt2yoWVNUKkvMnrNHAV9CUULF5RRUNmziWFwMbFVbyVcjFQYvygwL8YYZbs_DNw0HVyhFop1b31nlA5xoO8n2O8_TrITsldGcQy40qF8v9s-z8pSSdKM-Q",
"e": "AQAB",
"x5c": [
"MIICyzCCAbMCBgGE4eSfbzANBgkqhkiG9w0BAQsFADApMScwJQYDVQQDDB53YXRzb24tYXNzaXN0YW50LW5vZGVqcy1zZXJ2ZXIwHhcNMjIxMjA1MTA0NDI3WhcNMzIxMjA1MTA0NjA3WjApMScwJQYDVQQDDB53YXRzb24tYXNzaXN0YW50LW5vZGVqcy1zZXJ2ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCo21e3zh6Xtzff66LDTPvouGjq5faDIQ7iYPI5efSpY7LO1RyZ7S7IjVWzxn1rLmJzg5c+OSBTKG4R0DFpPuPnpGen6/7oIPNtwq0YV4ZzuBqpi2Uig5FjM0MqN34QZmIPKVcJuRm7jDgj72KLJzFuYJLWJ/Y+tWIaEKz/i2+zDJjvW76/eSvPr0fW6mAo9+SNZsBK57BfTNxpFF/vq58iNG50JOwX1vLC3bKhZU1QqS8yes0cBX0JRQsXlFFQ2bOJYXAxsVVvJVyMVBi/KDAvxhhluz8M3DQdXKEWinVvfWeUDnGg7yfY7z9OshOyV0ZxDLjSoXy/2z7PylJJ0oz5AgMBAAEwDQYJKoZIhvcNAQELBQADggEBAJOappeo8EDOvs10iwpOZJh+1SGj8eZH9i+xPBThCaQG1UZ4AJ9NEipgg/YmM5cjgGdzaAx2W9eIKO3qnp5Q+D68V/SNPOBqpvdVZ+OIEtoeCM4XMQN34amJbem9jxokie9Il0SzzSYSzwf8bh+4avahWRb13n5ZwKEPFG6S6XJ7xjQpJchUpo6MrhD6O3R3k17zTPIJ/wzlDRiFm13zaJ8WVRSM/f/5f+JeXWk3TVHqoWZSLhYM/odZoG/2/M901ElibQhQgfKGZmQvkRuKwVvW2JZhZ+ZHKCvStIkCfAvfMPbaal5NncIhnWJ5VsAHOsGmsqMX6A/reuF5e7M3n6M="
],
"x5t": "xVuprC2qZQwLtMmgugDl6HYq07o",
"x5t#S256": "tk_W1z9tt75J5-scDx-35obQ78JQaKMO17iOPr0oyjo"
}
]
}

```

Reply all

Reply to author

Forward