another wrinkle with GAE/Python Managed VMs: timezone set to local deploy location, not UTC like in GAE

[Adam Sah]

e.g. os.environ['TZ'] changed to America/Chicago when before it was UTC.

The docs don't mention this difference anywhere.

If anyone has a workaround, pls reply - I'm working on a workaround now, will post if successful.

hope this helps,

adam

[Adam Sah]

Here's what I've tried so far...

 - set os.environ['TZ'] to 'UTC' on startup - failed because GAE overwrites this.

 - set os.environ['TZ'] to 'UTC' on each call by creating a child class to webapp.WSGIApplication and adding it to __call__() - failed because GAE overwrites this.

 - monkeypatching datetime.datetime.now() to use a specially hacked function - failed because GAE disallows modifying now()

I'm currently trying a hack to rewrite all calls (100+) to datetime.datetime.now() to instead use an app-specific implementation which returns the UTC time - this won't work for third party libraries, but that maybe OK for my app, which uses pylint custom rules to migrate engineers to using app-specific replacement functions (e.g. we disallow direct calls to gc.collect() so we can track stats better).  If this works, I'll post.

adam

[Adam Sah]

oops, to be specific here's what I'm seeing:

   datetime.datetime.now().strftime("%c %Z") ==> Thu Mar 10 14:57:31 2016    (i.e. timestamp is in CST with no timezone info)

   os.environ['TZ'] ==> America/Chicago

where dev_appserver and standard GAE return:

   datetime.datetime.now().strftime("%c %Z") ==> Thu Mar 10 21:00:14 2016    (i.e. timestamp is in UTC with no timezone info)

   os.environ['TZ'] ==> 'UTC'

adam

[Adam Sah]

OK, horrible workaround deployed.  Roughly:

main.py:

PATHS=[...

  (r'/_ah/warmup', WarmupView),

  (r'/_ah/start', StartView),

class WarmupView(webapp.RequestHandler):

  def get(self, *args):  # pylint:disable=W0613,R0201

    base.SERVER_TZ = None  # allow base.set_now() to reset server timezone

    base.now()

class StartView(webapp.RequestHandler):

  def get(self, *args):  # pylint:disable=W0613,R0201

    base.SERVER_TZ = None  # allow base.set_now() to reset server timezone

    base.now()

base.py:

import pytz

...

SERVER_TZ = None

...

def now(nowval=None):

  global SERVER_TZ

  ...

  if SERVER_TZ is None:

      try:

        # HACK of the year award!  os.environ['TZ'] isn't available during GAE/Managed VMs startup

        # and date +%Z and /etc/timezone didn't have useful information either

        # even env | grep TZ failed!!

        import subprocess

        localtz = subprocess.check_output("date +%Z", shell=True).strip()

        # SUPER HACK: I couldn't find pytz-compatible timezone strings, so I hacked it.

        # Yes, I know I'm going to hell for this.

        SERVER_TZ = 'America/Chicago' if localtz == 'CST' else 'UTC'

      except:

        SERVER_TZ = 'UTC'

    nowval = datetime.datetime.now()  # pylint:disable=W9914

    if SERVER_TZ != 'UTC':

      # set the tz to the server tz, convert to UTC, then strip the tz info

      nowval = nowval.replace(tzinfo=pytz.timezone(SERVER_TZ)).astimezone(

        pytz.timezone('UTC')).replace(tzinfo=None)

  return nowval

adam

[Nick (Cloud Platform Support)]

Hey Adam,

Glad to see you found a workaround, I was going to some lengths to verbally describe what you basically ended up doing. You should post a quick description of this issue to the Public Issue Tracker for App Engine and we'll be happy to take a look. It seems rational to have UTC as the default.

Cheers,

Nick
Cloud Platform Community Support

[Michael Sander]

I've seen this too, but I just started working on Managed VM. Has this been around for a while? It seems like a pretty major bug for it not to be fixed. Also, the workaround is totally nuts. Is this going to be fixed or is the hack our only recourse?

[Nick (Cloud Platform Support)]

Hey Michael,

We're currently working on getting this resolved, although the best way to track its resolution is by posting and following a Public Issue Tracker issue. At any rate, we're currently working on it and it should be resolved soon. If your application is sensitive to the system time-zone, the natural recourse is to set it explicitly, rather than assume a default. Nonetheless, UTC is a sensible default I believe and should be implemented. 

I'll update this thread with any progress on the issue.

Best wishes,

Nick
Cloud Platform Community Support

[Jon Parrott]

To further Nick's point here, Python provides datetime.datetime.utcnow() which allows you to explicitly get the current time in UTC. It's almost always preferable to use that unless you explicitly want to deal with system-local time.

[Nick (Cloud Platform Support)]

Hey Folks,

Here's the PIT thread to track this. Expect any updates to be posted there.

[Adam Sah]

ah... utcnow() works correctly and indeed that's the ideal solution for apps that can update their code.

thanks!

adam

[Michael Sander]

Yes, datetime.utcnow() does work correctly, however, many libraries that are out of my direct control call datetime.now().  This is especially apparent when using django, which uses datetime.now() all throughout its code-base. For example, the auto_now feature in django automatically sets the time of the database item you are saving to be datetime.now(), which can result in items in your database being set incorrectly. Further, the popular humanize django plugin (which humanizes dates and time deltas) may also behave differently from expected.

This is blocking using managed VM with the very popular django web framework.