Enable CORS

[Alan Cabrera]

I'm trying to enable CORS on my endpoint by adding the following to my service configuration file:

endpoints:

  - name: bookstore.endpoints.foo-project.cloud.goog

    allow_cors: true

I try to hit the endpoint and I get:

< HTTP/2 403 

< server: nginx

< date: Wed, 16 May 2018 20:58:12 GMT

< content-type: application/json

< www-authenticate: Bearer

< 

{

 "code": 7,

 "message": "The service does not allow CORS traffic.",

 "details": [

  {

   "@type": "type.googleapis.com/google.rpc.DebugInfo",

   "stackEntries": [],

   "detail": "service_control"

  }

 ]

}

Does anyone have an suggestions as to how to get this working via a change in the service configuration file?  I'm trying to avoid futzing with my ESP's nginx configuration file to get this working.

Regards,

Alan

[Alan Cabrera]

Also, I do not want to use the --cors_preset option since I want my backend server performing the CORS pre-flight decisions.

[Alan Cabrera]

I'm going to assume that there's no obvious answer to this and file a ticket with Google support. 

On Wednesday, May 16, 2018 at 2:06:44 PM UTC-7, Alan Cabrera wrote:

[Christophe Taton]

Hi Alan,

On Wednesday, May 16, 2018 at 2:06:44 PM UTC-7, Alan Cabrera wrote:

I'm trying to enable CORS on my endpoint by adding the following to my service configuration file:

endpoints:

  - name: bookstore.endpoints.foo-project.cloud.goog

    allow_cors: true

Did you mean `x-google-endpoints` instead of `endpoints` ?

I just verified and the `x-google-endpoints.allow_cors` feature appears to be working correctly.

Once set to true, OPTIONS requests for all paths configured in the OpenAPI spec are accepted and forwarded to the backend.

Note that as an alternative, you can also manually declare OPTIONS operations for individual paths.

[Alan Cabrera]

The x-google-endpoints property is not allowed in a gcloud service configuration file.  The name of the attribute is allow_cors which is documented in 

https://cloud.google.com/endpoints/docs/grpc-service-config/reference/rpc/google.api#google.api.Endpoint

Anything else results in an endpoints services deployment failure, e.g.

ERROR: (gcloud.endpoints.services.deploy) INVALID_ARGUMENT: Cannot convert to service config. 

'ERROR: api-config.yaml:16: Found field 'x-google-endpoints' which is unknown in 'google.api.Service'.'

[Christophe Taton]

On Friday, May 18, 2018 at 11:35:02 AM UTC-7, Alan Cabrera wrote:

The x-google-endpoints property is not allowed in a gcloud service configuration file.  The name of the attribute is allow_cors which is documented in 

https://cloud.google.com/endpoints/docs/grpc-service-config/reference/rpc/google.api#google.api.Endpoint

Anything else results in an endpoints services deployment failure, e.g.

ERROR: (gcloud.endpoints.services.deploy) INVALID_ARGUMENT: Cannot convert to service config. 

'ERROR: api-config.yaml:16: Found field 'x-google-endpoints' which is unknown in 'google.api.Service'.'

My mistake, I had assumed you were using OpenAPI.

Is your backend using gRPC with HTTP/JSON transcoding then?

Can you describe your use-case a little bit more, maybe?

[Alan Cabrera]

Yes, it's using gRPC.  I understand that vanilla gRPC does not have CORS support, but that is *not* the problem.  The ESP does not allow CORS requests to flow through despite it being configured to allow_cores as per the documentation.

There are workarounds, but the ones discussed on stackoverflow, etc., are a way to get CORS to "work" with vanilla gRPC; it basically returns a canned reply to pre-flight checks with no security checks what so ever.  I am not interested in using that kludgy workaround.

I have augmented my gRPC server to handle CORS pre-flight requests and have confirmed that it is properly working.  My final, hopefully, impediment is getting the ESP to simply let CORS pre-flight requests through.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/187de3cb-f1e0-4e26-8e47-bc12fd3b892d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[lei...@google.com]

Hi Alan,

For diagnosing this issue, can you share the logs (e.g., nginx logs) and your configurations? 

Thanks,

Lei

On Wednesday, May 16, 2018 at 2:06:44 PM UTC-7, Alan Cabrera wrote:

[Christophe Taton]

I see.

The allow_cors feature of ESP automatically creates OPTIONS operations for all operations defined in the service configuration.

If your service config doesn't declare any operation, setting allow_cors is essentially a no-op.

I believe you have 2 options.

1. Explicitly create operations for which ESP is expected to forward OPTIONS requests.
2. Create a single catch-all operation.

I attached an example for 2, which I verified works.

Note that the format of the service config file is particularly fragile. I had to leave some unused annotations such as sourceContext, even though these aren't useful here.

Let me know if that helps!

C.

On Fri, May 18, 2018 at 1:31 PM Alan Cabrera <acab...@nianticlabs.com> wrote:

Yes, it's using gRPC.  I understand that vanilla gRPC does not have CORS support, but that is *not* the problem.  The ESP does not allow CORS requests to flow through despite it being configured to allow_cores as per the documentation.

There are workarounds, but the ones discussed on stackoverflow, etc., are a way to get CORS to "work" with vanilla gRPC; it basically returns a canned reply to pre-flight checks with no security checks what so ever.  I am not interested in using that kludgy workaround.

I have augmented my gRPC server to handle CORS pre-flight requests and have confirmed that it is properly working.  My final, hopefully, impediment is getting the ESP to simply let CORS pre-flight requests through.

On Fri, May 18, 2018 at 11:52 AM, 'Christophe Taton' via Google Cloud Endpoints <google-clou...@googlegroups.com> wrote:

On Friday, May 18, 2018 at 11:35:02 AM UTC-7, Alan Cabrera wrote:

The x-google-endpoints property is not allowed in a gcloud service configuration file.  The name of the attribute is allow_cors which is documented in 

https://cloud.google.com/endpoints/docs/grpc-service-config/reference/rpc/google.api#google.api.Endpoint

Anything else results in an endpoints services deployment failure, e.g.

ERROR: (gcloud.endpoints.services.deploy) INVALID_ARGUMENT: Cannot convert to service config. 

'ERROR: api-config.yaml:16: Found field 'x-google-endpoints' which is unknown in 'google.api.Service'.'

My mistake, I had assumed you were using OpenAPI.

Is your backend using gRPC with HTTP/JSON transcoding then?

Can you describe your use-case a little bit more, maybe?

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.

[Alan Cabrera]

My endpoints service configuration entry is below.  The log entry from the esp is

10.1.0.2 - - [18/May/2018:21:42:14 +0000] "OPTIONS /endpoints.examples.bookstore.Bookstore/GetShelf HTTP/2.0" 403 207 "http://localhost:4200/server" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36"

[Alan Cabrera]

For

http:

  rules:

  - selector: "*"

    custom:

      kind: "*"

      path: "/*"

I'm getting the error

[error]9#9: Invalid HTTP template: /*

when I deploy the API.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

[lei...@google.com]

Hi Alan,

From the ESP code https://github.com/cloudendpoints/esp/blob/master/src/api_manager/check_service_control.cc#L41, the error message ("message": "The service does not allow CORS traffic.") is due to that your service configuration does not configure the methods for your api. The example configuration provided by Christophe configures the methods for the api and thus works. To get your service configuration working, a configuration of the methods of the api should be added.

Regards,

Lei

[Alan Cabrera]

Happy to add whatever additional configuration is needed.  

I'm now at the point where I've followed Christophe's suggestion but am getting a "Invalid HTTP template: /*" logging error from the ESP.

BTW, thanks for everyone's help here.  When this is all done, I'll write a blog post for future gRPC/Endpoints developers. 

Regards,

Alan

[Christophe Taton]

It turns out the `allow_cors` feature only works on HTTP/JSON transcoded operations.

From what I see, ESP won't forward OPTIONS requests for plain gRPC method calls, when using `allow_cors`.

Does your use-case require sending plain gRPC requests from a browser, or would you be able to use HTTP/JSON transcoded operations?

[Alan Cabrera]

i could transcode the gRPC requests I guess.  I'd rather not go through the effort.

Is there a way to simply let all OPTIONS requests flow through to the backend?

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/b3b1b3e6-f139-4e10-be95-a295df382e22%40googlegroups.com.

[Christophe Taton]

On Monday, May 21, 2018 at 6:03:16 PM UTC-7, Alan Cabrera wrote:

i could transcode the gRPC requests I guess.  I'd rather not go through the effort.

Is there a way to simply let all OPTIONS requests flow through to the backend?

I was trying more options yesterday and realized I had made a mistake earlier : I had left ESP configured to use an HTTP1 backend.

It appears ESP doesn't provide any way to forward any pre-flight requests to a gRPC backend (whether transcoded or not).

I have a couple more ideas I'd like to test.

[Christophe Taton]

One option that appears to work is to customize the nginx.conf template (https://github.com/cloudendpoints/endpoints-tools/blob/master/start_esp/nginx-auto.conf.template#L109) and add a statement like the following:

      # End Endpoints v2 Support

      if ($request_method = OPTIONS) {

        proxy_pass https://123.132.213.231:9000;

      }

You can mount the customized template as a volume in the ESP container, then add the following command-line argument to ESP:

    --nginx_config=/path/to/nginx.conf.template

I tested this with a Python HTTP2 backend, and I could have it handle OPTIONS requests this way.

I'm not sure exactly how your gRPC backend has been modified to support CORS requests, so you might need to adapt a few things.

[Alan Cabrera]

The IP address looks hardcoded.  Is there something else I should use?

--

You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/d50e341b-a07a-45e0-9d3f-e9d75533c89d%40googlegroups.com.

[Christophe Taton]

For the record:

The solution I mentioned in my previous reply works for transcoded HTTP/JSON paths but not for plain gRPC HTTP paths (eg. "/package.Service/Method").

The last resort, fat finger option is to insert another proxy in front of ESP, to intercept all OPTIONS request and forward them directly to the backend, while forwarding all other requests to ESP.

[lei...@google.com]

There is a way to direct the http OPTIONS requests to a grpc method. The following is an example I created as a proof of concept, in which the http OPTIONS requests on the path /v1/shelves are mapped to the grpc method ListShelves(). By following this example, you can direct the http OPTIONS requests to a grpc method defined by you.

In the example, the annotation added to the grpc proto file is as follows, which maps the http OPTIONS method on the path /v1/shelves to the grpc method ListShelves().

service Bookstore {
  rpc ListShelves(google.protobuf.Empty) returns (ListShelvesResponse) {
    // Define HTTP mapping to the OPTIONS http method

    // The following maps the http OPTIONS method on the path /v1/shelves to the grpc method ListShelves().
    option (google.api.http) = { custom: 
        { kind: "options"
          path: "/v1/shelves"
        };
    };
  }

The following result of running the curl command with the http OPTIONS method shows that the mapping works (the http OPTIONS request reaches the grpc ListShelves() method).

~$ curl -v -X OPTIONS http://$SERVER_IP:8080/v1/shelves

*   Trying 104.154.72.254...

* TCP_NODELAY set

* Connected to 104.154.72.254 (104.154.72.254) port 8080 (#0)

> OPTIONS /v1/shelves HTTP/1.1

> Host: 104.154.72.254:8080

> User-Agent: curl/7.58.0

> Accept: */*

> 

< HTTP/1.1 200 OK

< Server: nginx

< Date: Mon, 28 May 2018 19:50:40 GMT

< Content-Type: application/json

< Transfer-Encoding: chunked

< Connection: keep-alive

< 

* Connection #0 to host 104.154.72.254 left intact

{"shelves":[{"id":"1","theme":"Fiction"},{"id":"2","theme":"Fantasy"}]}

[Christophe Taton]

Yes, this approach might work for HTTP transcoded requests, although I wonder if it's possible to set the headers returned in the response.

That being said, I believe that Alan is trying to call the gRPC methods directly from the browser, without transcoding (ie. /endpoints.examples.bookstore.Bookstore/ListShelves).

[Alan Cabrera]

That is correct.

Regards,

Alan

On Tue, May 29, 2018 at 1:05 PM, 'Christophe Taton' via Google Cloud Endpoints <google-clou...@googlegroups.com> wrote:

Yes, this approach might work for HTTP transcoded requests, although I wonder if it's possible to set the headers returned in the response.

That being said, I believe that Alan is trying to call the gRPC methods directly from the browser, without transcoding (ie. /endpoints.examples.bookstore.Bookstore/ListShelves).

--

You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/a3312417-401e-447d-968f-4a5a91ea776b%40googlegroups.com.

[lei...@google.com]

Since a browser is using the HTTP protocols (instead of the gRPC protocol), I think a transcoding between the HTTP protocols and the gRPC protocol is needed before the browser is able to call the gRPC methods.