Distinguishing between empty lists and null

[Andrew Paugh]

In json payloads, there's a huge difference between null and empty lists. Despite that, there seems to be no way to distinguish the two in Cloud Endpoints. (Likely because of protorpc?)

If I'm doing a PATCH operation with a repeated StringField called 'phone_numbers', then the following payload should remove all phone numbers:
{
  "phone_numbers": []
}

Whereas the following payload should do nothing to the phone numbers:
{
}

However, for both scenarios I see an empty list. Is there anything I can do to identify the second scenario vs the first?

[Rose Davidson]

Hi Andrew,

I assume from your mention of protorpc that you are using the Python Framework for Endpoints. Protocol buffer messages remove all values which are equal to the default value, on the assumption that the receiving party has a copy of the schema and can put those default values back in. Protorpc does the same thing when parsing and generating JSON.

Unfortunately protorpc does not provide any means to distinguish between an explicit default and an implicit default, since as the name suggests it was intended for RPC-style APIs rather than REST-style.

There are a number of ways you can work around this limitation, but they would require either adding additional fields to your messages or forking the framework code. Alternatively, you could write your application for App Engine Flex and deploy it using the normal OpenAPI Endpoints support.

I hope this helps!

Rose Davidson

Cloud Endpoints

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/a7193e5d-c061-4550-9aa0-e3f7292274e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Paugh]

Thanks Rose.

I'm curious about the code forking approach that you mentioned. Are there any examples of how that could be done? Wondering if it'd be a relatively simple change.

[Rose Davidson]

It's not particularly elegant, but you could patch endpoints_dispatcher.py in the transform_rest_request method: 

https://github.com/cloudendpoints/endpoints-python/blob/d3fe5f504d8b58714c455d8d42ead1fbedad0bc9/endpoints/endpoints_dispatcher.py#L559

The structure of the framework wraps the EndpointsDispatcherMiddleware around a WSGI app created by protorpc. The middleware has to rewrite the request so that protorpc will know how to handle it. You could patch the transform_rest_request method to store a copy of the original input JSON in an environment variable and access it later in your method code.

Once you've made this modification, there's a number of ways you can use your modified framework in your app; you should read about git urls in requirements.txt files, for instance: https://stackoverflow.com/a/16584935

Hope this helps!

On Fri, May 18, 2018 at 12:48 PM, Andrew Paugh <andrew...@dialpad.com> wrote:

Thanks Rose.

I'm curious about the code forking approach that you mentioned. Are there any examples of how that could be done? Wondering if it'd be a relatively simple change.

On Friday, 18 May 2018 11:09:15 UTC-7, Rose Davidson wrote:

Hi Andrew,

I assume from your mention of protorpc that you are using the Python Framework for Endpoints. Protocol buffer messages remove all values which are equal to the default value, on the assumption that the receiving party has a copy of the schema and can put those default values back in. Protorpc does the same thing when parsing and generating JSON.

Unfortunately protorpc does not provide any means to distinguish between an explicit default and an implicit default, since as the name suggests it was intended for RPC-style APIs rather than REST-style.

There are a number of ways you can work around this limitation, but they would require either adding additional fields to your messages or forking the framework code. Alternatively, you could write your application for App Engine Flex and deploy it using the normal OpenAPI Endpoints support.

I hope this helps!

Rose Davidson

Cloud Endpoints

On Thu, May 17, 2018 at 2:33 PM, Andrew Paugh <andrew...@dialpad.com> wrote:

In json payloads, there's a huge difference between null and empty lists. Despite that, there seems to be no way to distinguish the two in Cloud Endpoints. (Likely because of protorpc?)

If I'm doing a PATCH operation with a repeated StringField called 'phone_numbers', then the following payload should remove all phone numbers:
{
  "phone_numbers": []
}

Whereas the following payload should do nothing to the phone numbers:
{
}

However, for both scenarios I see an empty list. Is there anything I can do to identify the second scenario vs the first?

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/a7193e5d-c061-4550-9aa0-e3f7292274e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/df60158f-becf-4ce0-833b-f31fcefaa298%40googlegroups.com.

[Andrew Paugh]

Thanks Rose. That's great information.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/a7193e5d-c061-4550-9aa0-e3f7292274e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.

[Andrew Paugh]

Hi Rose,

I've implemented this on my side and it seems to work, so thank you for the suggestion.

In the spirit of a cleaner implementation, can you think of any way to attach it to a request rather than an environment variable? I suspect the answer is no, but it'd be awesome if there was a way to store it on the request or request_state on the API handler.

Thanks,
Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/a7193e5d-c061-4550-9aa0-e3f7292274e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rose Davidson]

Hi Andrew,

Unfortunately I don't think there's any good way to do that. A lot of the limitations of the current Framework are due to the way protorpc works, and this is one of those limitations.

(If I were to write the Framework from scratch today, I would not base it on protorpc.)

Good luck, and I'm glad you made it work!

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/a7193e5d-c061-4550-9aa0-e3f7292274e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/df60158f-becf-4ce0-833b-f31fcefaa298%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/a9e16d11-23b7-4269-a4f7-bbf6495092da%40googlegroups.com.