GAE time sync across instances

[Sam Hill]

Since I haven't been able to find a definite answer to this I thought I'd ask here. 

We currently use a go app deployed using GAE which can potentially cause us problems if we are not receiving a consistent time value.

Does anyone know how Google sync the time for their app instances and if anyone has any experience with time drift ?

There are several posts from a few years back on stackoverflow etc saying that either NTP is used or to not trust the time between instances.

Any help greatly appreciated.

Sam

[Nick (Cloud Platform Support)]

Hey Sam,

As the old threads on this topic did not yield a particularly decisive answer, I will see if I can manage to produce such a more decisive answer, or to see about getting this into the documentation. I can't make any promises other than that I will report back to this thread with any findings. In the meantime, feel free to let me know more details about any possible use-case which would have made this bit of information particularly important. I'll be glad to help advise on that as well.

Sincerely,

Nick
Cloud Platform Community Support

[Nick (Cloud Platform Support)]

Hey Sam, 

Thanks for your patience. I can now report based on my investigation of the matter:

1. There aren't any guarantees for time synchronization for App Engine, and this is reflected in the material in the docs. In general, there should only be a few milliseconds of drift at the most, but this is just an estimate and should not be taken as a guarantee, which isn't given. 

2. Like all of Google's infrastructure, the instances will be kept in sync with one another. The Google-internal NTP is called the TrueTime API and is described in a blog post and an article: [1] [2]. Tolerances information is not available for this system, but suffice to say it works well enough for all our purposes.

3. Google syncs time with Mountain View, which is is essentially the US Pacific time zone.

As far as documentation is concerned, I don't think there is anything we could add right now since there are no guarantees or tolerances. 

If you have a specific use case that absolutely must use a highly-accurate standard of time, creating a Compute Engine instance / cluster to handle this task in a centralized manner is probably the best option. Few applications have need of such accuracy, though, and I hope this post reassures you about time-sync for instances. Let me know if you have any questions, I'll be happy to help!

Regards,

Nick

Cloud Platform Community Support


On Monday, July 25, 2016 at 5:40:39 AM UTC-4, Sam Hill wrote:

[Alex Martelli]

On Fri, Jul 29, 2016 at 10:51 AM, 'Nick (Cloud Platform Support)' via Google App Engine <google-a...@googlegroups.com> wrote:

Hey Sam, 

Thanks for your patience. I can now report based on my investigation of the matter:

1. There aren't any guarantees for time synchronization for App Engine, and this is reflected in the material in the docs. In general, there should only be a few milliseconds of drift at the most, but this is just an estimate and should not be taken as a guarantee, which isn't given. 

2. Like all of Google's infrastructure, the instances will be kept in sync with one another. The Google-internal NTP is called the TrueTime API and is described in a blog post and an article: [1] [2]. Tolerances information is not available for this system, but suffice to say it works well enough for all our purposes.

3. Google syncs time with Mountain View, which is is essentially the US Pacific time zone.

As far as documentation is concerned, I don't think there is anything we could add right now since there are no guarantees or tolerances. 

If you have a specific use case that absolutely must use a highly-accurate standard of time, creating a Compute Engine instance / cluster to handle this task in a centralized manner is probably the best option. Few applications have need of such accuracy, though,

Sorry to interject, but this may not be necessarily the case for applications dealing with SEC-regulated fintech -- I recommend https://www.youtube.com/watch?v=c5Fx9fZqoo4 (bias warning: I'm an ACM member and this excellent talk has as its core motivation that of convincing viewers to join the ACM!-).

TL;DR: the SEC, NYSE, IEX, and other exchanges, are fiercely fighting over proposed regulations regarding millisecond (and FRACTIONS thereof!) synchronization of security trades -- see e.g http://www.bloomberg.com/news/articles/2016-04-25/iex-foes-allies-unite-on-one-point-a-millisecond-really-counts .  The ACM talk takes no position on the market politics of the issue (neither the speaker nor the intended audience are qualified professionally to speak to it, after all), but tears into the technical difficulties in ANY distributed system to achieve sub-millisecond synchronization (in securely auditable ways, as any SEC regulation would of course require).

App Engine -- as you correctly note, Nick -- cannot offer any guarantees in the matter; one would have to roll their own, just as on any other distributed system (if one's subject to potential future SEC regulations for fintech -- I cannot think of any other field in which this would matter, though I encourage everybody to watch the ACM video anyway and decide for themselves if THEIR applications might be impacted). I'm chiming in only to point out that, fintech being hardly a tiny niche, the problem should not be minimized. A Feature Request for better on-demand time sync might be warranted... even though the ACM video strongly suggests it is simply unfeasible to comply with such tight constraints (in ANY distributed system, including on-premises data centers).

Alex

 

and I hope this post reassures you about time-sync for instances. Let me know if you have any questions, I'll be happy to help!

Regards,

Nick

Cloud Platform Community Support


On Monday, July 25, 2016 at 5:40:39 AM UTC-4, Sam Hill wrote:

Since I haven't been able to find a definite answer to this I thought I'd ask here. 

We currently use a go app deployed using GAE which can potentially cause us problems if we are not receiving a consistent time value.

Does anyone know how Google sync the time for their app instances and if anyone has any experience with time drift ?

There are several posts from a few years back on stackoverflow etc saying that either NTP is used or to not trust the time between instances.

Any help greatly appreciated.

Sam

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/b1060c5c-94e0-4c8f-9b92-afc1dfe7eff6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nick (Cloud Platform Support)]

As usual, Alex is intervening with some very interesting and relevant truth! Thanks for pointing that out, I had an inkling that financial applications might be concerned, but no idea the depths it went to. Now after checking out the links I can see of course it would be relevant in that field!

Agreed that if such a Feature Request is desired by anybody, a Public Issue Tracker post will be in order. We'll be happy to take the request!

Cheers,

Nick
Cloud Platform Community Support

On Friday, July 29, 2016 at 2:36:30 PM UTC-4, Alex Martelli wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

Visit this group at https://groups.google.com/group/google-appengine.