ESPv2 error with custom domain name Upstream connect error or disconnect/reset before headers

[Nyah Check]

Hi,

I am trying to set up a grpc service deployed in a MIG behind a custom domain name accessed via IAP. I've followed these tutorials:

1. https://cloud.google.com/endpoints/docs/grpc/get-started-mig-lb-espv2

2. https://cloud.google.com/endpoints/docs/grpc/troubleshoot-gce-deployment

3. https://cloud.google.com/endpoints/docs/openapi/troubleshoot-response-errors#reset_reason_connection_failure

I've successfully setup a load balancer behind the custom domain after defining a global IP address added as an A record for the domain name. The MIGs seems to report a healthy state of the endpoints containers running in the container. However, I'm not able to hit the endpoint service.

I try hitting the backend service by running the client with the call:

client --address `https://domain_name/{GRPC_Path}/GET` 

I get this error:

"Error fetching list: status: Unavailable, message: \"upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER\", details: [], metadata: MetadataMap { headers: {\"content-type\": \"application/grpc\", \"alt-svc\": \"h3=\\\":443\\\"; ma=2592000,h3-29=\\\":443\\\"; ma=2592000\", \"content-length\": \"0\", \"date\": \" XXXXGMT\", \"via\": \"1.1 google\"} }"

this is from outside the MIG on my work station and not from GCP. 

The next thing I did was login to the GCP instance to see the status of the endpoint runtime container.

$ curl http://localhost:8090/endpoints_status

curl: (7) Failed to connect to localhost port 8090 after 0 ms: Connection refused

I published port 9000 and tried the same thing and it didn't work. At this point I"m confused since I've looked at link 3 above to view the error but I don't really know how to resolve it. It's hard to tell where I'm getting these configs wrong. It took me more than a week just to get to this point. I"m not very familiar with grpc and how these endpoints are best configured behind an https domain on GCP. 

Any ideas will be greatly appreciated.

Thanks,

[Shuo Yang]

I read https://cloud.google.com/endpoints/docs/grpc/get-started-mig-lb-espv2 and it doesn't mention TLS or IAP. How did you set those up? The TLS error may come from there.

When you login to your GCP instance (VM), you have many tools to debug

• `docker ps` shows the running container and their status, the port they expose
• curl localhost may not work directly because the container doesn't expose to the host network. `docker exec -it <container name> /bin/bash` will help you login to the container instead.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/b6bb37b6-7a72-4bce-ba08-4f7f6b186cf9n%40googlegroups.com.

--

Thanks,

Shuo Yang