Huge Variation in datastore_v3.Get() time
56 views
Skip to first unread message
Rajesh Gupta
Mar 22, 2017, 7:31:03 AM3/22/17
to google-a...@googlegroups.com
Hello,
I am analyzing the trace of the request. Please find attached.
The datastore_v3.Get() api call duration varies from 11ms to 131ms.
If there are more datastore_v3.Get(), then the response time is highly unpredictable.
Regards,
Rajesh
Accounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and Mobile
Ivo Bellin Salarin
Mar 22, 2017, 7:56:24 AM3/22/17
to google-a...@googlegroups.com
The datastore.Get implementation can use cache (memcached). But I can't see that kind of calls on your graph.
Perhaps the size of entities you're trying to get is the reason for this gap?
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CA%2BS7ijaD-HfM5FZbVWDvc76J%3DmAcpiyXszEsFyzp1HBtap%2Bz9g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Nicholas (Google Cloud Support)
Mar 22, 2017, 10:46:58 AM3/22/17
to Google App Engine
The 130ms request seems entirely reasonable. There's also only one request listed there so it may at the very least be an irregular occurrence though you do mention that more causes more latency but that cannot be seen from the Stackdriver Trace graph screenshot you've attached.
More importantly, not every request is the same and different factors may play a role in the latency you describe.
- Networks may experience heavier latency at different times depending on congestion. Internal Google networks are no exception. We have not however experienced any heavy latency with the Cloud Datastore in some time so this is not likely the cause. Additionally, one would have to show consistent latency over a given period of time for network congestion to be a more likely cause.
- Are your keys properly sharded? Though Cloud Datastore can generate sharded keys upon committing entities, it is possible for users to use their own keys. In these cases, one must take particular care to shard keys as this empowers the Datastore to locate entities more quickly while reducing the risks of data contention. The article Avoiding datastore contention though somewhat dated, still holds valid precepts.
- What are the fetched entities? Entities may vary in size greatly. Currently, the maximum size of an entity is 1,048,572 bytes. Entities of this size may take more time to transmit along the wire.
It's difficult to assess the performance expectation you present without more detailed information such as key distribution, entity size/contents, consistency in latency and parameters of load testing you might have done to confirm this is a platform issue.
Based on the information you've provided, I would generally recommend the following:
- Splitting up requests if you know that they will require serial interdependent Datastore fetches
- Using memcache whenever possible to keep frequently accessed information closer
- Consider using the form of Get that retrieves multiple entities at once to reduce time costs
Note also that the Cloud Datastore does not have any latency/performance-related SLAs at this time. Nevertheless, if after going through the above recommendations, you load test and find there to be consistent latency issues, I'd recommend filing an issue on the Issue Tracker. We frequently that forum as well.
Hope this addressees your concerns.
Nicholas
On Wednesday, March 22, 2017 at 7:56:24 AM UTC-4, Ivo Bellin Salarin wrote:
The datastore.Get implementation can use cache (memcached). But I can't see that kind of calls on your graph.
Perhaps the size of entities you're trying to get is the reason for this gap?
Le mer. 22 mars 2017 12:30, Rajesh Gupta <rajesh.gupta@veersoftsolutions.com> a écrit :
Hello,--I am analyzing the trace of the request. Please find attached.The datastore_v3.Get() api call duration varies from 11ms to 131ms.If there are more datastore_v3.Get(), then the response time is highly unpredictable.Regards,RajeshAccounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and Mobile
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Reply all
Reply to author
Forward
Message has been deleted
0 new messages