Path Translation Error

[Tyler Reeves]

Just followed the how-to on using ESP in front of Cloud Run and I am getting an error in path translation.  No matter where I put the x-google-backend rule (top level or operation level) or what I put in the path_translation field in my OpenAPI yaml definition, the APPEND_PATH_TO_ADDRESS rule is always being applied. I am trying to make a post call to another cloud run endpoint with no defined path other than the base path.  

[Wayne Zhang]

Could you post your OpenApi spec. and the output of "gcloud endpoints configs describe" here?  thanks

On Sun, Dec 8, 2019 at 10:58 PM Tyler Reeves <ad...@sivlabs.com> wrote:

Just followed the how-to on using ESP in front of Cloud Run and I am getting an error in path translation.  No matter where I put the x-google-backend rule (top level or operation level) or what I put in the path_translation field in my OpenAPI yaml definition, the APPEND_PATH_TO_ADDRESS rule is always being applied. I am trying to make a post call to another cloud run endpoint with no defined path other than the base path.  

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/efb7c166-0b19-4188-bfd7-d17eef58609c%40googlegroups.com.

[Tyler Reeves]

Sure thing, I put them in files since both are a little lengthy (gcloud describe command is in Describe.txt)

On Monday, December 9, 2019 at 10:03:37 AM UTC-7, Wayne Zhang wrote:

Could you post your OpenApi spec. and the output of "gcloud endpoints configs describe" here?  thanks

On Sun, Dec 8, 2019 at 10:58 PM Tyler Reeves <ad...@sivlabs.com> wrote:

Just followed the how-to on using ESP in front of Cloud Run and I am getting an error in path translation.  No matter where I put the x-google-backend rule (top level or operation level) or what I put in the path_translation field in my OpenAPI yaml definition, the APPEND_PATH_TO_ADDRESS rule is always being applied. I am trying to make a post call to another cloud run endpoint with no defined path other than the base path.  

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

[Tyler Reeves]

I also wanted to add this which is the stackdriver log I keep getting (notice it is trying to append the request path to the base url):

2019-12-09 06:20:26.554 MST

POST404273 B55 msPostmanRuntime/7.20.1 https://transaction-create-dev-h2mzpr56tq-uc.a.run.app/transaction/create

[Wayne Zhang]

From the output of gcloud CLI, it is using CONSTANT_ADDRESS.  

backend:
  rules:
  - address: https://transaction-create-dev-h2mzpr56tq-uc.a.run.app
    jwtAudience: https://transaction-create-dev-h2mzpr56tq-uc.a.run.app
    pathTranslation: CONSTANT_ADDRESS
    selector: v1.myra_gateway_h2mzpr56tq_uc_a_run_app.MyraTransactionCreate

Since your "address=https://transaction-create-dev-h2mzpr56tq-uc.a.run.app" doesn't have path portion, and your are using constant_address path translation if If your incoming path to ESP is /transaction-create, it should not append any path, the outgoing path to the backend should be "/". 

I don't understand why the path to your function is /transaction/create from your stack driver log. Are you sure you look at the right request log?

Could you post Cloud Run logs of the request in ESP service and your backend service?

[Tyler Reeves]

Sry I posted the the openapi spec for my first attempt (which had path transaction-create) and stackdriver of my second deployment which only difference was the path name changed to /transaction/create.  To be clear, I just now deployed the openAPI attached to this email and tried to do a post request to this endpoint (the esp gateway endpoint), https://myra-gateway-h2mzpr56tq-uc.a.run.app/transaction/create. The stackdriver

Stackdriver log for backend service:

2019-12-09 12:42:13.252 MST

POST404272 B33 msHTTPBot-iOS/2.2 https://transaction-create-dev-h2mzpr56tq-uc.a.run.app/transaction/create

Stackdriver log for esp service:

2019-12-09 12:42:13.265 MST

POST404811 B2.5 sHTTPBot-iOS/2.2 https://myra-gateway-h2mzpr56tq-uc.a.run.app/transaction/create

2019-12-09 12:42:17.029 MST

2019/12/09 19:42:17[error]5#5: Failed to call https://servicecontrol.googleapis.com/v1/services/myra-gateway-h2mzpr56tq-uc.a.run.app:report, Error: FORBIDDEN: server response status code: 403, Response body: #The caller does not have permission

2019-12-09 12:42:17.229 MST

[libprotobuf ERROR external/servicecontrol_client_git/src/service_control_client_impl.cc:182] Failed in Report call: Service control request failed with HTTP response code 403

[Wayne Zhang]

the 403 is a different problem.  Do we still have problem with path_translation?   If you "address" field in x-google-backend has /path_prefix,  ESP will use that /path_prefix regardless the incoming request path /foo/bar to ESP since you are using constant_address.  Is this working as you expected?  

For 403 issue, it could be a latency issue:  if you just used "gcloud endpoints service deploy" to deploy endpoint service "myra-gateway-h2mzpr56tq-uc.a.run.app",  it will take up to 30 minutes to propagate.  if you get 403 on "report", please wait for 30 minutes.

If have been more than 30 minutes,  then it could be permission issue.  are you using the same project to deploy ESP Cloud Run service as the project for "gcloud endponts service deploy",   if so, is the service account used for deploying ESP cloud run service has the Project editor role ?

[Tyler Reeves]

I posted the 403 message in case it had anything to do with the path translation problem.  Since it does not, we can leave it alone for the time being.  However, for the path translation problem, this is what I'm expecting.  With the current OpenAPI configuration, I want to make a post request to the cloud run esp gateway endpoint https://myra-gateway-h2mzpr56tq-uc.a.run.app/transaction/create which should forward my post request to my backend service endpoint https://transaction-create-dev-h2mzpr56tq-uc.a.run.app .  Based on the stackdriver logs, what appears to be happening is when I make a post call to https://myra-gateway-h2mzpr56tq-uc.a.run.app/transaction/create, https://transaction-create-dev-h2mzpr56tq-uc.a.run.app /transaction/create is being called instead.  It appears to be taking the request path and appending to the backend base url which is why I keep getting a 404 Resource not found status (because i never defined that request path in my backend cloud run function).  It is my understanding if x-google-backend address field in the open api spec is defined as  https://transaction-create-dev-h2mzpr56tq-uc.a.run.app  and the path_translation field is defined as CONSTANT_ADDRESS (by default), then the esp gateway should be routing to the exact address defined in x-google-backend.  Is my understanding incorrect?

[Wayne Zhang]

It looks like ESP has a bug with CONSTANT_ADDRESS and "address" field has only HOST portion, without any path.   You can work around this by adding a path_prefix in the address field.  We are debugging it, will update the result here.

Thanks

-Wayne

[Tyler Reeves]

Sounds good, thank you for your help.

[Wayne Zhang]

https://github.com/cloudendpoints/esp/issues/735

Thanks

-Wayne