SSL Errors in endpoints

[t...@growkudos.com]

Hi all, 

I have been getting some odd SSL errors from one of our endpoints:

2018/10/29 16:35:17 [alert] 14#14: *687399 ignoring stale global SSL error (SSL: error:0400006b:RSA routines:OPENSSL_internal:BLOCK_TYPE_IS_NOT_01 error:04000088:RSA routines:OPENSSL_internal:PADDING_CHECK_FAILED error:0400006b:RSA routines:OPENSSL_internal:BLOCK_TYPE_IS_NOT_01 error:04000088:RSA routines:OPENSSL_internal:PADDING_CHECK_FAILED) while sending to client, client: 10.154.0.3, server: , request: "POST /check HTTP/1.1", upstream: "http://127.0.0.1:8080/check", host: "service.name"

It looks like the request is going through to the service running on 127.0.0.1:8080 without a problem. Though i am unsure what the cause of the issue is.

I have not been able to find any reference to the errors online.

Can anyone shed any light onto what their errors are referring to and how to overcome then?

Thanks

Tim

[Andrew Gunsch]

+Wayne

What Endpoints runtime are you using?

Is this log coming from the Endpoints proxy?

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/16c1e1c1-eb9c-41ca-8fe4-d4280fc8cbc0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Little]

Hi,

We are using the Extensible Service Proxy in within GKE.

Thanks

--

Tim Little

[Wayne Zhang]

Hi Piotr,  this SSL error is from ESP, (nginx),  could you help?  Thanks

-Wayne

[t...@growkudos.com]

Hi guys,

Thanks for looking into this.

I have added a log exclusion in my logs to limit how many of these we are seeing but it would be good to get the bottom of what is causing them.

Tim

On Thursday, November 1, 2018 at 4:45:50 PM UTC, Wayne Zhang wrote:

Hi Piotr,  this SSL error is from ESP, (nginx),  could you help?  Thanks

-Wayne

On Thu, Nov 1, 2018 at 9:39 AM Tim Little <t...@growkudos.com> wrote:

Hi,

We are using the Extensible Service Proxy in within GKE.

Thanks

On Thu, Nov 1, 2018 at 4:33 PM Andrew Gunsch <gun...@google.com> wrote:

+Wayne

What Endpoints runtime are you using?

Is this log coming from the Endpoints proxy?

On Thu, Nov 1, 2018 at 9:31 AM <t...@growkudos.com> wrote:

Hi all, 

I have been getting some odd SSL errors from one of our endpoints:

2018/10/29 16:35:17 [alert] 14#14: *687399 ignoring stale global SSL error (SSL: error:0400006b:RSA routines:OPENSSL_internal:BLOCK_TYPE_IS_NOT_01 error:04000088:RSA routines:OPENSSL_internal:PADDING_CHECK_FAILED error:0400006b:RSA routines:OPENSSL_internal:BLOCK_TYPE_IS_NOT_01 error:04000088:RSA routines:OPENSSL_internal:PADDING_CHECK_FAILED) while sending to client, client: 10.154.0.3, server: , request: "POST /check HTTP/1.1", upstream: "http://127.0.0.1:8080/check", host: "service.name"

It looks like the request is going through to the service running on 127.0.0.1:8080 without a problem. Though i am unsure what the cause of the issue is.

I have not been able to find any reference to the errors online.

Can anyone shed any light onto what their errors are referring to and how to overcome then?

Thanks

Tim

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/16c1e1c1-eb9c-41ca-8fe4-d4280fc8cbc0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tim Little

[t...@growkudos.com]

Hi guys,

Was anyone able to help out with this?

Thanks

Tim

[qiwz...@google.com]

Most likely these SSL errors are from JWT signature verification failures.   ESP is using OpenSSL code to verify JWT signature.  If verification fails, the errors were not cleared, somehow showed up in Nginx ssl code path.

ESP needs to call ERR_clear_error(); to clear these errors.

Thanks

-Wayne

[qiwz...@google.com]

It is a bug in ESP.  It will be fixed in https://github.com/envoyproxy/envoy/pull/5170