Jenkins problem with clocks going back?

[Matthew...@diamond.ac.uk]

This morning, checking the "Manage Jenkins" page, it said:

"Some projects have builds whose timestamps are inconsistent. These will confuse Jenkins when it tries to look up build records."
There is a link to http://jenkins-ci.org/issue/18289.

What happened is that the clocks went back (end of summer time), which means jobs scheduled in the "duplicated" hour ran twice, and some of the later runs were at an earlier wall-clock time than the previous. Hence Jenkins gets confused.

It's not a big problem for us (it might be more of a problem for others), but any suggestions as to how to handle this in future? Should this be considered a Jenkins bug?

I added a comment about this case to https://issues.jenkins-ci.org/browse/JENKINS-22106 "@midnight is vulnerable to DST witching hours"

Matthew


--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

[James Nord (jnord)]

> It's not a big problem for us (it might be more of a problem for others), but
> any suggestions as to how to handle this in future

My preffered way of handling this is to always run your infrastructure in Etc/UTC.

It means you never hit the time changing issue - and when you need to integrate with servers in a different timezone[1] all the logs will tie up perfectly.

Now - an enhancement would be for the Jenkins UI to convert from Server time to user local time to display timestamps in the UI...

/James

(you may have zero now - but you may end up using cloud or acquiring a foreign company in the future)

[Dirk Heinrichs]

Am 27.10.2014 um 11:23 schrieb James Nord (jnord):

My preffered way of handling this is to always run your infrastructure in Etc/UTC.

Fine if it's Unix/Linux. Doesn't work on Windows, because hardware clock is always on localtime there.

Bye...

    Dirk

--

Dirk Heinrichs, Senior Systems Engineer, Engineering Solutions
Recommind GmbH, Von-Liebig-Straße 1, 53359 Rheinbach
Tel: +49 2226 1596666 (Ansage) 1149
Email: d...@recommind.com
Skype: dirk.heinrichs.recommind
www.recommind.com

[James Nord (jnord)]

Hi Dirk,

 

I really don’t follow – unless you are on an out of date windows verison it does now have an option for UTC as a proper UTC unline the old “Grenwich Mean Time” which was actually Europe/London.

 

 

 

No daylight saving options are there

 

/James

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dirk Heinrichs]

Am 27.10.2014 um 11:47 schrieb James Nord (jnord):

I really don’t follow – unless you are on an out of date windows verison it does now have an option for UTC as a proper UTC unline the old “Grenwich Mean Time” which was actually Europe/London.

OK, I need to elaborate. On Unix/Linux, one usually sets the hardware clock to UTC, and determines the actual time that is displayed to the user by setting a time zone. That means that no matter what DST settings that timezone has, it doesn't have any impact on any running process, because hardware clock always stays the same, time is just displayed differently.

On Windows, OTOH, HW clock is always set to the time (and timezone) set in the OS, which means that DST changes actually change the HW clock and thus result in time jumps which don't happen on Linux.

HTH...

[James Nord]

Hi Dirk,

If you set the timezone to Etc/UTC even on windows the hardware clock will never jump - as there is no DST to adjust the clock by.

/James

[Matthew...@diamond.ac.uk]

As the originator of this thread, thanks for your responses. We're on Linux, so the Windows concerns thankfully don’t apply.

As stated, the usual way to handle this kind of thing is for the server to keep time in UTC, and the application converts to local time for display etc. purposes. The application, in this case, is Jenkins, and it doesn’t look like it really handles this properly - it does everything in local time, and gets confused with the summer time change.

Generally, local time in the Jenkins interface is what _I_ want. When I write a job schedule, I want the times to be local -I think of a job as running at 10pm local time, and not the UTC equivalent.

Other Jenkins installations may have different requirements as well, of course. Users and slaves can be in a different time zone to the master.

[Daniel Beck]

On 27.10.2014, at 17:16, Matthew...@Diamond.ac.uk wrote:

> As stated, the usual way to handle this kind of thing is for the server to keep time in UTC, and the application converts to local time for display etc. purposes. The application, in this case, is Jenkins, and it doesn’t look like it really handles this properly - it does everything in local time, and gets confused with the summer time change.

There are two pull requests that may help here:

https://github.com/jenkinsci/jenkins/pull/1376
https://github.com/jenkinsci/jenkins/pull/1379

----

Regarding configuration options:

https://wiki.jenkins-ci.org/display/JENKINS/Change+time+zone

The Jelly date format property only applies to the UI, so it seems straightforward to run Jenkins on UTC (incl. build IDs) and display local time on the UI.

I'm currently trying to run a test instance with the following:

-Duser.timezone=Universal -Dorg.apache.commons.jelly.tags.fmt.timeZone=Europe/Berlin

So far it seems to behave correctly (Server is in CET, Build IDs are in UTC, and the UI -- except things like timer triggers -- is in CET again).

----

FWIW I had the problem where builds were scheduled to run hourly, and the 2:xx AM CET builds appear to have overridden the 2:xx AM CEST builds with identical ID, resulting in a SEVERE log message when they were supposed to be log rotated ("Failed to rotate log"). For me, deleting the obsolete build symlink did the trick. No idea how to handle if the builds aren't auto deleted. I'd be extra careful with builds created during those two hours.

[Dirk Heinrichs]

Am 27.10.2014 um 16:49 schrieb James Nord:

If you set the timezone to Etc/UTC even on windows the hardware clock will never jump - as there is no DST to adjust the clock by.

Yes, that's clear.  But this time will also be displayed to the user, which is not the nicest thing to have for most people. On Linux, the HW clock will never jump AND the user will get the correct time displayed, according to her timezone setting.

Bye...