NDB Query hangs for one set of criteria

[Myles Bostwick]

I have a datastore query I make as an integral part of my system. It's an equality filter on four properties. When I make this query for one set of criteria, the query just hangs. It does not hang for this set of criteria in the datastore viewer, so I can see that there are only two results to return. I can fetch both of these entities by id without problem, but the query always hangs, both in remote shell and in production. The query is running fine for a multitude of different criteria, it appears to just be this one set. I have attempted replacing both entities so that the criteria still matches, but they have different ids and I get the same result.

For some more specific information, the four properties are defined as follows:

    company = ndb.KeyProperty(indexed=True)

    aggregator_serial = ndb.StringProperty(indexed=True)

    instrument_serial = ndb.StringProperty(indexed=True)

    time_slices = ndb.IntegerProperty(indexed=True, repeated=True)

I'm running gcloud versions

Google Cloud SDK 207.0.0

app-engine-java 1.9.64

app-engine-python 1.9.71

app-engine-python-extras 1.9.69

beta 2018.06.22

bq 2.0.34

core 2018.06.22

gcloud

gsutil 4.32

Any help on what I should try for next steps would be greatly appreciated.

Thanks in advance,

Myles

[Jordan (Cloud Platform Support)]

This may be due to your repeated property. Depending on your query on this property, you may not have the required indexes for a fast result. Therefore the Datastore would attempt a zigzag merge join on existing indexes in attempts to satisfy the query. As per the documentation, this could indeed lead to poor performance, and it is recommended to plan out your indexes according to your queries. 

You can test the above by removing the repeated property from your query to see if it improves performance. If it is still slow, than this may simply be an issue with your App Engine code (since you mentioned that this query works in the Datastore Console). To investigate your application, you can use the Stackdriver Trace tool to see why your requests are hanging and where all of the time is being spent. 

- Note that Google Groups is reserved for general product discussions and not technical support. For further support in improving your Datastore queries it is recommended to post your detailed questions (along with the actual queries) to Stack Exchange using the supported Cloud tags.