Cloud Endpoints questions - in the context of GKE
853 views
Skip to first unread message
mark.j.chilvers
Jul 20, 2016, 4:44:01 PM7/20/16
to Google Cloud Endpoints
I've been exploring the alpha Cloud Endpoints for a couple of days now and have a fairly long list of questions. Hopefully someone can point me in the right direction! I'm attempting to use Cloud Endpoints to expose a number of services that are deployed in GKE.
1) Is there documentation around all of the arguments that can be passed into the ESP image? (i.e. -n, -s, -v and so on)
2) I have the need to do some mapping from the exposed service endpoint (say "/foo") to the path the underlying services are expecting (sometimes just "/"). I have tried using the nginx.conf from the sample here: the https://github.com/GoogleCloudPlatform/endpoints-samples/tree/master/gke . When using this config, along with a YAML based upon esp_echo_custom_config.yaml, I can get the mapping to work, but validation of (say) API keys then doesn't occur. When I switch back to the canned nginx config, along with a config based upon esp_echo.yaml API keys are now checked, but I can't perform URL mapping. Is something missing from the sample nginx.conf ?
3) I was hopeful that defining the service in swagger / OpenAPI would mean my API would be exposed via the API Explorer. I guess that is not the case? Or perhaps this is just something I'm unable to get working?
4) Is it possible to deploy multiple (different) swagger files to the same project (i.e. same host)?
5) In the case of GKE it's somewhat confusing that the swagger points to <PROJECT_ID>.appspot.com as the host, but my services are not available via this URL. Is it envisioned this will always be the case? If a GKE service COULD be exposed via the appspot URL then this could simplify use of SSL (using the appspot cert instead of having to create our own).
6) Are there any examples of steps necessary to set up a GKE service secured via known client IDs? I did make a (brief) attempt to test using the OAuth playground. I switched out the Auth endpoint for the one mentioned in the sample swagger files (https://www.googleapis.com/oauth2/v1/certs), but using this endpoint returns 3 certificates, and unsure how to proceed to test the flow. Using the standard Google Oauth endpoint & setting the Authorization: Bearer header appears not to work (Endpoints complains about invalid token).
Martin Maly
Jul 21, 2016, 6:47:25 PM7/21/16
to mark.j.chilvers, Google Cloud Endpoints
Hi Mark,
thanks for reaching out with your questions. I'll do my best to answer inline.
On Wed, Jul 20, 2016 at 1:44 PM, mark.j.chilvers <mark.j....@pwc.com> wrote:
I've been exploring the alpha Cloud Endpoints for a couple of days now and have a fairly long list of questions. Hopefully someone can point me in the right direction! I'm attempting to use Cloud Endpoints to expose a number of services that are deployed in GKE.1) Is there documentation around all of the arguments that can be passed into the ESP image? (i.e. -n, -s, -v and so on)
Great question. This is the full list of options the ESP Docker image accepts:
Options:
-h
Shows this message.
-s ENDPOINTS_SERVICE_NAME
Required. The name of the Endpoints Service.
-v ENDPOINTS_SERVICE_VERSION
Required. The version of the Endpoints Service which is assigned
when deploying the service API specification.
e.g. 2016-04-20R662
-a HOST:PORT
The application server address to which nginx will proxy the requests.
Default is 127.0.0.1:8081.
-n NGINX_CONF_PATH
Specifies the path to a custom nginx.conf. If this option is specified,
all other options will be ignored.
-N PORT
Port to expose /nginx_status. Default is 8090.
-p PORT
The nginx http port. Default is 8080.
-S PORT
Enables the nginx ssl port to serve https request.
By default, the ssl port is disabled.
Examples:
(1) Starts nginx at http port 80 and https port 443 that proxies the requests to
the application at 127.0.0.1:8000
$(command basename $0) -a 127.0.0.1:8000 -p 80 -S 443 -s SERVICE_NAME -v SERVICE_VERSION
(2) Starts nginx with a custom nginx config mounted at /app/nginx/
$(command basename $0) -n /app/nginx/nginx.conf -s SERVICE_NAME -v SERVICE_VERSION
2) I have the need to do some mapping from the exposed service endpoint (say "/foo") to the path the underlying services are expecting (sometimes just "/"). I have tried using the nginx.conf from the sample here: the https://github.com/GoogleCloudPlatform/endpoints-samples/tree/master/gke . When using this config, along with a YAML based upon esp_echo_custom_config.yaml, I can get the mapping to work, but validation of (say) API keys then doesn't occur. When I switch back to the canned nginx config, along with a config based upon esp_echo.yaml API keys are now checked, but I can't perform URL mapping. Is something missing from the sample nginx.conf ?
The example you used should be a good place to start. To do the URL mapping you may be able to try the following in your nginx.conf:
...
location /foo/ {
proxy_pass <your backend service> ;
}
...
For this case, an incoming URL path: /foo/some/path will get forwarded to backend as /some/path
However, the Endpoints configuration will be applying API management features to the "/foo/some/path" URL and that URL namespace needs to match your Open API (Swagger) specification for Endpoints to manage the API traffic.
3) I was hopeful that defining the service in swagger / OpenAPI would mean my API would be exposed via the API Explorer. I guess that is not the case? Or perhaps this is just something I'm unable to get working?
At the moment, API Explorer is not integrated with Endpoints. However, Open API ecosystem has a rather wonderful Swagger UI which can be used with Endpoints applications and other APIs that use Swagger / Open API.
This link may be helpful: https://cloud.google.com/endpoints/docs/adding-swagger
4) Is it possible to deploy multiple (different) swagger files to the same project (i.e. same host)?
It is possible to deploy multiple different Swagger API definitions to the same project, but they each need to use different Swagger host property. For example, if your project name is "my-project", you could have two Swagger files with host properties:
We also support custom domains (https://cloud.google.com/endpoints/docs/custom-domain) where upon ownership verification you can use your custom domain with an Endpoints API.
5) In the case of GKE it's somewhat confusing that the swagger points to <PROJECT_ID>.appspot.com as the host, but my services are not available via this URL. Is it envisioned this will always be the case? If a GKE service COULD be exposed via the appspot URL then this could simplify use of SSL (using the appspot cert instead of having to create our own).
We are actively working on the service names and URLs via which the services are accessible. The custom domain support I mentioned above may help (https://cloud.google.com/endpoints/docs/custom-domain) but there are some rough edges as you point out. We'd love to hear more feedback as you continue experimenting with Endpoints.
6) Are there any examples of steps necessary to set up a GKE service secured via known client IDs? I did make a (brief) attempt to test using the OAuth playground. I switched out the Auth endpoint for the one mentioned in the sample swagger files (https://www.googleapis.com/oauth2/v1/certs), but using this endpoint returns 3 certificates, and unsure how to proceed to test the flow. Using the standard Google Oauth endpoint & setting the Authorization: Bearer header appears not to work (Endpoints complains about invalid token).
We have several examples on how to use Google Client IDs for auth:
The tokens that you obtain from OAuth playground are different tokens than those used with Endpoints. Endpoints uses JWT tokens. whereas OAuth playground works with Google access tokens. Please let us know if the readmes in the examples above don't lead you in the right direction.
Hope this helps, do let us know if you have more questions or if the answers above do not clarify things sufficiently. We'll be happy to help.
Martin (and several other Endpoints engineers who provided details on some of your questions).
The information transmitted, including any attachments, is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you received this in error, please contact the sender and delete the material from any computer. PricewaterhouseCoopers LLP is a Delaware limited liability partnership. This communication may come from PricewaterhouseCoopers LLP or one of its subsidiaries.
--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/ddb54ff6-113b-4093-8832-161fe5084226%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
mark.j.chilvers
Jul 21, 2016, 7:02:38 PM7/21/16
to Google Cloud Endpoints, mark.j....@pwc.com
Thanks Martin. This is really helpful!!
In terms of my second question - your suggestion was exactly as I had tried previously - but endpoints would never enforce the traffic. Anyway - I'll give it another try.
Thanks again!
Martin Maly
Jul 21, 2016, 7:06:52 PM7/21/16
to mark.j.chilvers, Google Cloud Endpoints
Hi Mark,
if you have difficulty with the second question, feel free to send me your nginx.conf (or snippet of it). Maybe I'll spot something. If the techniques we try still don't work we can set up a video conference and look deeper under the hood etc.
Just let me know.
Martin
The information transmitted, including any attachments, is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited, and all liability arising therefrom is disclaimed. If you received this in error, please contact the sender and delete the material from any computer. PricewaterhouseCoopers LLP is a Delaware limited liability partnership. This communication may come from PricewaterhouseCoopers LLP or one of its subsidiaries.
--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/0fe48ff1-37d6-42b0-8cf0-1712f9005709%40googlegroups.com.
matthew.r...@blue-green-group.com
Jan 16, 2017, 11:36:16 AM1/16/17
to Google Cloud Endpoints, mark.j....@pwc.com
Hi Martin and Mark. Those answers were indeed really helpful. Also note that, as installed by the GCE .deb package, ESP is started (and its environment/context configured) by /usr/sbin/start_esp which has its own commandline options, some of which are used to supply commandline options to ESP (custom nginx) itself:
$ ./start_esp -h
usage: start_esp [-h] [-k SERVICE_ACCOUNT_KEY] [-s SERVICE] [-v VERSION]
[-n NGINX_CONFIG] [-p HTTP_PORT] [-P HTTP2_PORT]
[-S SSL_PORT] [-N STATUS_PORT] [-a BACKEND]
[-c SERVICE_CONFIG_URL] [-z HEALTHZ]
ESP start-up script. This script fetches the service configuration from the
service management service and configures ESP to expose the specified ports and
proxy requests to the specified backend.
The service name and config ID are optional. If not supplied, the script
fetches the service name and the config ID from the metadata service as
attributes "endpoints-service-name" and "endpoints-service-config-id".
ESP relies on the metadata service to fetch access tokens for Google services.
If you deploy ESP outside of Google Cloud environment, you need to provide a
service account credentials file by setting GOOGLE_APPLICATION_CREDENTIALS
environment variable or by passing "-k" flag to this script.
If a custom nginx config file is provided ("-n" flag), the script launches ESP
with the provided config file. Otherwise, the script uses the exposed ports
("-p", "-P", "-S", "-N" flags) and the backend ("-a" flag) to generate an nginx
config file.
optional arguments:
-h, --help show this help message and exit
-k SERVICE_ACCOUNT_KEY, --service_account_key SERVICE_ACCOUNT_KEY
Use the service account key JSON file to access the
service control and the service management. You can
also set GOOGLE_APPLICATION_CREDENTIALS environment
variable to the location of the service account
credentials JSON file. If the option is omitted, ESP
contacts the metadata service to fetch an access
token.
-s SERVICE, --service SERVICE
Set the name of the Endpoints service. If omitted and
-c not specified, ESP contacts the metadata service to
fetch the service name.
-v VERSION, --version VERSION
Set the service config ID of the Endpoints service. If
omitted and -c not specified, ESP contacts the
metadata service to fetch the service config ID.
-n NGINX_CONFIG, --nginx_config NGINX_CONFIG
Use a custom nginx config file instead of the config
template /etc/nginx/nginx-auto.conf.template. If you
specify this option, then all the port options are
ignored.
-p HTTP_PORT, --http_port HTTP_PORT
Expose a port to accept HTTP/1.x connections. By
default, if you do not specify any of the port options
(-p, -P, and -S), then port 8080 is exposed as
HTTP/1.x port. However, if you specify any of the port
options, then only the ports you specified are
exposed, which may or may not include HTTP/1.x port.
-P HTTP2_PORT, --http2_port HTTP2_PORT
Expose a port to accept HTTP/2 connections. Note that
this cannot be the same port as HTTP/1.x port.
-S SSL_PORT, --ssl_port SSL_PORT
Expose a port for HTTPS requests. Accepts both
HTTP/1.x and HTTP/2 secure connections. Requires the
certificate and key files /etc/nginx/ssl/nginx.crt and
/etc/nginx/ssl/nginx.key
-N STATUS_PORT, --status_port STATUS_PORT
Change the ESP status port. Status information is
available at /endpoints_status location over HTTP/1.x.
Default value: 8090.
-a BACKEND, --backend BACKEND
Change the application server address to which ESP
proxies the requests. Default value: 127.0.0.1:8081.
For HTTPS backends, please use "https://" prefix, e.g.
https://127.0.0.1:8081 and provide the certificate and
key files /etc/nginx/ssl/backend.crt and
/etc/nginx/ssl/backend.key. For HTTP/1.x backends,
prefix "http://" is optional. For GRPC backends,
please use "grpc://" prefix, e.g.
grpc://127.0.0.1:8081.
-c SERVICE_CONFIG_URL, --service_config_url SERVICE_CONFIG_URL
Use the specified URL to fetch the service
configuration instead of using the default URL
template https://servicemanagement.googleapis.com/v1/s
ervices/{}/config?configId={}.
-z HEALTHZ, --healthz HEALTHZ
Define a health checking endpoint on the same ports as
the application backend. For example, "-z healthz"
makes ESP return code 200 for location "/healthz",
instead of forwarding the request to the backend.
Default: not used.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
Pallavi Srivastava
Apr 2, 2019, 2:56:21 PM4/2/19
to Google Cloud Endpoints
Hi Mark,
These links are not working
On Friday, 22 July 2016 04:17:25 UTC+5:30, Martin Maly wrote:
Hi Mark,thanks for reaching out with your questions. I'll do my best to answer inline.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
Andres Varela Koda
Apr 3, 2019, 6:38:55 PM4/3/19
to Google Cloud Endpoints
It looks like those were some old docs for authentication. Those have been moved to the following links, depending on the flavor of Endpoints you use:
For an example of how make a call from javascript:
Note that much of the information from that post is outdated, we now have an intergrated API Explorer with Endpoints in the Cloud Endpoints Portal.
We also have versioning if you want to use the same host for a new version of your API:
Reply all
Reply to author
Forward
0 new messages