Latency graph

130 views
Skip to first unread message

Deepak Singh

unread,
Sep 18, 2016, 12:03:07 PM9/18/16
to google-a...@googlegroups.com
PFA the attached graph for our latencyon dashboard (Java flex env).
What should we conclude based on this data ? 

--
Tech team
Latency.PNG

Nick (Cloud Platform Support)

unread,
Sep 21, 2016, 7:52:17 PM9/21/16
to Google App Engine
Hey Deepak,

Since latency is the amount of time requests take to process and return their response, it appears that you have high latency occurring in short time periods fairly regularly. You can check the logs in those time periods (also, observe the latency graph over a smaller time-frame to get a more accurate determination of what that time period is), and see what kind of requests they are. The latency could depend on many factors, but most likely it depends on the code which is running inside the request handler. Is it waiting synchronously on a network call which itself has high latency? Is it fetching a lot of data? Is it running a very intensive computation? Etc.

I hope this helps get you started on looking into the latency. Let me know if you have any further questions, as I'll be happy to help!

Cheers,

Nick
Cloud Platform Community Support

Deepak Singh

unread,
Sep 30, 2016, 2:01:51 PM9/30/16
to google-a...@googlegroups.com
Hi Nick,

We have done a rigorous analysis for the latency issue here and we came to the following conclusion -

Latency was analysed for the same url/request with the same source code on flex env.
Latency is at irregular interval but most frequently and changes multiple times in an hour.
Latency is so high (e.g. 140308) that it results with response code  502. Here 99% of the latency time is UNTRACED time on platform.
At the moment the problem starts, latency / untraced time goes on increasing with each request until the platform comes to the normal behaviour.


Now since it is common behaviour - 

Very high latency and most frequently.
502 response code.
Untraced time is close to latency.

So we conclude that there is nothing wrong with the code since the full latency is UNTRACED time and response code is 502. This is the common behaviour for all the requests that suffered.

It proves to be a critical platform issue, at least with our app. 
Pls look into it.

Here are few request/log for your reference - 

Gae/request_log_id

57eea31500ff0d8f062fcbf95d0001737e62616c616a69796174726166696e6465720001362d342d36370001012f

57eea30900ff0e2c2c75100fa00001737e62616c616a69796174726166696e6465720001362d342d363700010138

57eea2f500ff04af2e506cdb530001737e62616c616a69796174726166696e6465720001362d342d36370001011d

57eea44700ff072044473595e20001737e62616c616a69796174726166696e6465720001362d342d363700010145

57ee7a3000ff08a4e1abebdb3b0001737e62616c616a69796174726166696e6465720001362d342d363700010104

57ee7a0d00ff055faed4696a470001737e62616c616a69796174726166696e6465720001362d342d363700010140



Regards
Tech team

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a04aa159-97e3-4005-b39c-3d5b39b670cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Deepak Singh

Deepak Singh

unread,
Oct 3, 2016, 11:19:06 AM10/3/16
to google-a...@googlegroups.com
Hi Nick,

Did you get a chance to look into it as the problem still persists ? Let me know we can send our app id privately to you if required.

Regards
Tech team

On Fri, Sep 30, 2016 at 11:30 PM, Deepak Singh <deepaks...@gmail.com> wrote:
Hi Nick,

We have done a rigorous analysis for the latency issue here and we came to the following conclusion -

Latency was analysed for the same url/request with the same source code on flex env.
Latency is at irregular interval but most frequently and changes multiple times in an hour.
Latency is so high (e.g. 140308) that it results with response code  502. Here 99% of the latency time is UNTRACED time on platform.
At the moment the problem starts, latency / untraced time goes on increasing with each request until the platform comes to the normal behaviour.


Now since it is common behaviour - 

Very high latency and most frequently.
502 response code.
Untraced time is close to latency.

So we conclude that there is nothing wrong with the code since the full latency is UNTRACED time and response code is 502. This is the common behaviour for all the requests that suffered.

It proves to be a critical platform issue, at least with our app. 
Pls look into it.

Here are few request/log for your reference - 

Gae/request_log_id

57eea31500ff0d8f062fcbf95d0001737e62616c616a69796174726166696e6465720001362d342d36370001012f

57eea30900ff0e2c2c75100fa00001737e62616c616a69796174726166696e6465720001362d342d363700010138

57eea2f500ff04af2e506cdb530001737e62616c616a69796174726166696e6465720001362d342d36370001011d

57eea44700ff072044473595e20001737e62616c616a69796174726166696e6465720001362d342d363700010145

57ee7a3000ff08a4e1abebdb3b0001737e62616c616a69796174726166696e6465720001362d342d363700010104

57ee7a0d00ff055faed4696a470001737e62616c616a69796174726166696e6465720001362d342d363700010140



Regards
Tech team

On Thu, Sep 22, 2016 at 5:22 AM, 'Nick (Cloud Platform Support)' via Google App Engine <google-appengine@googlegroups.com> wrote:
Hey Deepak,

Since latency is the amount of time requests take to process and return their response, it appears that you have high latency occurring in short time periods fairly regularly. You can check the logs in those time periods (also, observe the latency graph over a smaller time-frame to get a more accurate determination of what that time period is), and see what kind of requests they are. The latency could depend on many factors, but most likely it depends on the code which is running inside the request handler. Is it waiting synchronously on a network call which itself has high latency? Is it fetching a lot of data? Is it running a very intensive computation? Etc.

I hope this helps get you started on looking into the latency. Let me know if you have any further questions, as I'll be happy to help!

Cheers,

Nick
Cloud Platform Community Support


On Sunday, September 18, 2016 at 12:03:07 PM UTC-4, Deepak Singh wrote:
PFA the attached graph for our latencyon dashboard (Java flex env).
What should we conclude based on this data ? 

--
Tech team

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a04aa159-97e3-4005-b39c-3d5b39b670cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Deepak Singh



--
Deepak Singh
Reply all
Reply to author
Forward
0 new messages