date and timezone quickness

[tin h]

Hello everyone, Happy New Year!

I noticed some quirkiness with time inside the singularity container.  

When I run the date command inside the container, it would be 8 hours ahead.  It is as if the timezone data has cancelled between the host and the container (I am at GMT-8).

Inside the singularity container TZ defaults to "PST".

If I set TZ to "America/Los_Angeles" then date reports correct time.

File time-stamp are similarly affected.

Oh, I remember there were some weird issue with "PST".  Maybe folks at other timezone don't have this problem?

Below I have some command session that show more details of the issue.  

Singularity: 2.2.1

Host: Ubuntu 14.04

Image: CentOS 6.8 

sn@backbay:/opt/singularity_repo$ date

Tue Jan  3 15:49:08 PST 2017

sn@backbay:/opt/singularity_repo$ /opt/tmp/bin/singularity shell centos6.img

Singularity: Invoking an interactive shell within container...

Singularity.centos6.img> date

Tue Jan  3 23:49:11 PST 2017

Singularity.centos6.img> echo $TZ

PST

Singularity.centos6.img> export TZ="America/Los_Angeles"

Singularity.centos6.img> date

Tue Jan  3 15:49:44 PST 2017

Singularity.centos6.img> exit

exit

sn@backbay:/opt/singularity_repo$ date; touch /tmp/time_test_host; ls -l /tmp/time_test*

Tue Jan  3 15:52:58 PST 2017

-rw-rw-r-- 1 sn sn 0 Jan  3 15:52 /tmp/time_test_host

sn@backbay:/opt/singularity_repo$ /opt/tmp/bin/singularity shell centos6.img

Singularity: Invoking an interactive shell within container...

Singularity.centos6.img> 

Singularity.centos6.img> touch /tmp/time_test_container

Singularity.centos6.img> date; ls -l /tmp/time_test*

Tue Jan  3 23:54:05 PST 2017

-rw-rw-r-- 1 sn sn 0 Jan  3 23:54 /tmp/time_test_container

-rw-rw-r-- 1 sn sn 0 Jan  3 23:52 /tmp/time_test_host

Singularity.centos6.img> exit

exit

sn@backbay:/opt/singularity_repo$ ls -l /tmp/time_test*

-rw-rw-r-- 1 sn sn 0 Jan  3 15:54 /tmp/time_test_container

-rw-rw-r-- 1 sn sn 0 Jan  3 15:52 /tmp/time_test_host

sn@backbay:/opt/singularity_repo$ 

[tin h]

FYI, 

I dug a bit deeper into this problem, the east coaster are in luck, as EST works fine, but west coast need to use PST8PDT or America/Los_Angeles.

I filed a bug with Red Hat, hopefully they will look into a fix.

https://bugzilla.redhat.com/show_bug.cgi?id=1411150

# below is the correct, expected time:

sn@backbay:/opt/singularity_repo$ date

Sun Jan  8 12:38:49 PST 2017

sn@backbay:/opt/singularity_repo$ echo $TZ

sn@backbay:/opt/singularity_repo$ 

# but inside the container, the default time, even stated as PST, is wrong:

sn@backbay:/opt/singularity_repo$ /opt/tmp/bin/singularity exec centos7c.img date

Sun Jan  8 20:39:08 PST 2017

# explicitly setting TZ to PST still produce wrong result:

sn@backbay:/opt/singularity_repo$ export TZ=PST; /opt/tmp/bin/singularity exec centos7c.img date

Sun Jan  8 20:39:24 PST 2017

# setting TZ to Los Angeles works:

sn@backbay:/opt/singularity_repo$ export TZ=America/Los_Angeles; /opt/tmp/bin/singularity exec centos7c.img date

Sun Jan  8 12:39:39 PST 2017

# EST works, so how come PST doesn't? :

sn@backbay:/opt/singularity_repo$ export TZ=EST; /opt/tmp/bin/singularity exec centos7c.img date

Sun Jan  8 15:40:12 EST 2017

sn@backbay:/opt/singularity_repo$ export TZ=America/New_York; /opt/tmp/bin/singularity exec centos7c.img date

Sun Jan  8 15:40:22 EST 2017

# PST8PDT works as well.  

sn@backbay:/opt/singularity_repo$ export TZ=PST8PDT; /opt/tmp/bin/singularity exec centos7c.img date

Sun Jan  8 12:41:04 PST 2017

 

[Jason Stover]

EST is the default timezone when installed isn't it? So, maybe taking
/etc/localtime from the build system to put in the container image as
/etc/localtime?

Or have a timezone option in the build definition that takes the TZ
data from /usr/share/zoneinfo and puts it in as /etc/localtime, or
just uses the default EST if not set?

From a couple different servers:
$ strings /etc/localtime | tail -n1
UTC0
$ strings /etc/localtime | tail -n1
CST6CDT,M3.2.0,M11.1.0

-J

> --
> You received this message because you are subscribed to the Google Groups
> "singularity" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to singularity...@lbl.gov.

[Jason Stover]

On Sun, Jan 8, 2017 at 3:24 PM, tin h <tin...@gmail.com> wrote:
>
> # PST8PDT works as well.
> sn@backbay:/opt/singularity_repo$ export TZ=PST8PDT;
> /opt/tmp/bin/singularity exec centos7c.img date
> Sun Jan 8 12:41:04 PST 2017
>

Poking around on the servers here, it looks like the separate PST/PDT
timezone files no longer exist. If you look at America/Los_Angeles
it's doing:

$ strings /usr/share/zoneinfo/America/Los_Angeles | tail -n1
PST8PDT,M3.2.0,M11.1.0

So the PST8PDT entry wins:
$ strings /usr/share/zoneinfo/PST8PDT | tail -n1
PST8PDT,M3.2.0,M11.1.0

-J

[tin h]

Thanks for the pointers in deciphering the timezone data.

I found a fix to the problem.

Singularity's etc/init has this:

    # Set the timezone if it is not already set

    if [ -z "${TZ:-}" ]; then

        TZ=`date +%Z`

    fi

My host does not have TZ set (likely Red Hat default), and:

    > tail -1 /etc/localtime

    PST8PDT,M3.2.0,M11.1.0

    

In this condition, date +%Z returns PST.

However, as pointed out, PST is not defined in /usr/share/zoneinfo.

A Red Hat engineer explained that any TZ not defined in zoneinfo would be treated as a user defined timezone, and without further specification, would be treated to have an offset of 0 from GMT.

This resulted in perceived incorrect time when I ran the date command inside the container.

I am arguing with Red Hat that any time zone info returnable by date +%Z should have a corresponding timezone file from the OS, though I am not sure if they would act on it.

In the mean time, I updated the init script with the following checks

        if [[ "${TZ}"  = "PST" ]] || [[ "${TZ}" = "PDT" ]]; then TZ="PST8PDT"; fi

        if [[ "${TZ}"  = "CST" ]] || [[ "${TZ}" = "CDT" ]]; then TZ="CST6CDT"; fi

and the container returns the correct time now.

I can generate a PR and submit it to github if others would find this useful.

Best,

Tin

--
You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity+unsubscribe@lbl.gov.

[Gregory M. Kurtzer]

Great work on digging through this! Yes, please submit a PR if you can figure out a generalized way to solve this (e.g. For all time zones). 

Thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.

--


You received this message because you are subscribed to the Google Groups "singularity" group.

To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.