Changes to timezone-js

[Jan Niehusmann]

Hi Matthew,

as you suggested to discuss this on the mailing list, I hope it's ok
that I just answer your mail through the list.

> I wish we'd talked about tests from the very beginning -- the removal of the after-September hack breaks the tests for Jerusalem and Sao Paulo. We should talk about how to fix this best for everybody.

Unit tests are indeed very important for such a library as it's really
easy to break some corner cases while fixing some other ones.

Therefore I had a look at the JSUnit tests today. But I had to do
some changes to make them work at all:

- test.html doesn't set timezoneJS.timezone.defaultZoneFile which breaks
all tests for non north-american time zones.

- asynchronous loading of time zone data made the tests fail sometimes.
With async:false it works reliably

- Brazil changed time zone rules in 2008 (the change was announced only
a few weeks in advance). One test case for Sao Paulo is still using
the old rules and therefore fails with recent rule definitions.

I pushed these changes to github on branch 'jsunit', so you can apply
them easily and independently from the other changes.

Making this work took some time, so I didn't finish testing and updating
my changes to the timezone code.

But I'll probably have some spare time during the next few days so can
keep working on that.

Comments on my patches are always welcome, but at the moment it's
probably better to consider them as experimental.

Jan

[Jan Niehusmann]

On Tue, Mar 30, 2010 at 11:06:23PM +0200, Jan Niehusmann wrote:
> Making this work took some time, so I didn't finish testing and updating
> my changes to the timezone code.

I just ran the tests on my updates. A minor fix made them pass. (It's
always a bad sign if a comment doesn't match the code...)

Just for reference, I pushed that change to a branch named
'tmp_fix_dst_switching_bug'. The ugly name is chosen on purpose, at this
is only a temporary fix, as I'm not yet satisfied by this: The code
still behaves differently than standard javascript for the local time zone.

This differences are only minor (like the handling of ambiguous local
time representations), nevertheless I'm planning to update that patch.

Jan

[mde]

Jan,

I've applied your changes to the JSUnit test page. Very nice work;
thanks very much.

I should have some time tomorrow to check out your changes in the
tmp_fix_dst_switching_bug branch.

Thanks very much for your work on this!

Matthew

[Ricky Romero]

I'm doing a bit of an evil thing. :-P I'm scraping timeanddate.com
for a ton of time zone changes that we can test against. Not all of
the cases will map to the TZ database, and more recent rules will
probably not match up right, but anything from 1930 to 2000 should
make for a pretty good batch. There are so many different time
changes that I think we would be remiss if we didn't have a generous
test suite.

I'm thinking about breaking the data into one file per TZ name, so
"America/Chicago" would be one test suite, and "Africa/Tripoli" would
be another.

This will probably take some time to grab and massage into something
we can test against, so I'll see about reporting back later this week.

Ricky

[mde]

Ricky,

A little bit of evil is a good thing! Nice work.

As y'all know, date-related stuff is always a gold mine for corner
cases, so having lots of real data to throw at the code is the only
real way to make sure it's sane.

Be sure to check out Jan's recent changes to the tests. Separating
date and timezone algo testing looks like a good approach.

Matthew

[Ricky Romero]

In creating my unit tests, I've come across a stumbling block. The
Fleegix Date object assumes that you already know what time it is in
your target time zone. From the documentation:

// Pre-DST-leap
var dt = new fleegix.date.Date(2006, 9, 29, 1, 59, 'America/
Los_Angeles');
dt.getTimezoneOffset(); => 420
// Post-DST-leap
var dt = new fleegix.date.Date(2006, 9, 29, 2, 0, 'America/
Los_Angeles');
dt.getTimezoneOffset(); => 480

So, if you give it some time which is covered by a Daylight Saving
Time delta, like so:

Asia/Jerusalem: Sun Sep 27 01:59:59 2009 IDT // 60 minute DST in
force
var dt = new fleegix.date.Date(2009, 8, 27, 1, 30, 'Asia/Jerusalem');
Asia/Jerusalem: Sun Sep 27 01:00:00 2009 IST // After "falling back"

We can't accurately figure out whether the user meant for it to be
before or after the DST jump, because 1:30 AM happens "twice."

I think it would work better if we could assume that the user is
giving us UTC instead (or just take a Date object we get from them and
take off the system offset ourselves), and automatically return the
time with their requested timezone's offset applied. If they change
TZIDs, we return to UTC, then calculate the new offset and return the
date with new TZID's offset applied. It's more intuitive that way.

That logic would transform us to this:

// IDT
var dt = new fleegix.date.Date(2009, 8, 26, 22, 59, 59, 'Asia/
Jerusalem');
dt.toString(); => Sun Sep 27 2009 01:59:59 GMT+0300 (IDT)
dt.getTimezoneOffset(); => 180
// IST fall-back
var dt = new fleegix.date.Date(2009, 8, 26, 23, 0, 0, 'Asia/
Jerusalem');
dt.toString(); => Sun Sep 27 2009 01:00:00 GMT+0200 (IDT)
dt.getTimezoneOffset(); => 180

Thoughts?

Ricky

[Ricky Romero]

On another note, zdump (on Mac OS X) actually gives us all the testing
data we need without having to be evil and hit poor timeanddate.com
over and over. :-)

umiumi:~ ricky$ zdump -v /usr/share/zoneinfo/Asia/Jerusalem
/usr/share/zoneinfo/Asia/Jerusalem Fri Dec 13 20:45:52 1901 UTC = Fri
Dec 13 23:06:32 1901 JMT isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Sat Dec 14 20:45:52 1901 UTC = Sat
Dec 14 23:06:32 1901 JMT isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Mon Dec 31 21:39:19 1917 UTC = Mon
Dec 31 23:59:59 1917 JMT isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Mon Dec 31 21:39:20 1917 UTC = Mon
Dec 31 23:39:20 1917 IST isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Fri May 31 21:59:59 1940 UTC = Fri
May 31 23:59:59 1940 IST isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Fri May 31 22:00:00 1940 UTC = Sat
Jun 1 01:00:00 1940 IDT isdst=1
/usr/share/zoneinfo/Asia/Jerusalem Sat Oct 31 20:59:59 1942 UTC = Sat
Oct 31 23:59:59 1942 IDT isdst=1
/usr/share/zoneinfo/Asia/Jerusalem Sat Oct 31 21:00:00 1942 UTC = Sat
Oct 31 23:00:00 1942 IST isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Wed Mar 31 23:59:59 1943 UTC = Thu
Apr 1 01:59:59 1943 IST isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Thu Apr 1 00:00:00 1943 UTC = Thu
Apr 1 03:00:00 1943 IDT isdst=1
/usr/share/zoneinfo/Asia/Jerusalem Sun Oct 31 20:59:59 1943 UTC = Sun
Oct 31 23:59:59 1943 IDT isdst=1
/usr/share/zoneinfo/Asia/Jerusalem Sun Oct 31 21:00:00 1943 UTC = Sun
Oct 31 23:00:00 1943 IST isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Fri Mar 31 21:59:59 1944 UTC = Fri
Mar 31 23:59:59 1944 IST isdst=0
/usr/share/zoneinfo/Asia/Jerusalem Fri Mar 31 22:00:00 1944 UTC = Sat
Apr 1 01:00:00 1944 IDT isdst=1
/usr/share/zoneinfo/Asia/Jerusalem Tue Oct 31 20:59:59 1944 UTC = Tue
Oct 31 23:59:59 1944 IDT isdst=1
/usr/share/zoneinfo/Asia/Jerusalem Tue Oct 31 21:00:00 1944 UTC = Tue
Oct 31 23:00:00 1944 IST isdst=0
[etc.]

[Ricky Romero]

Sorry, I meant this:

// IDT
var dt = new fleegix.date.Date(2009, 8, 26, 22, 59, 59, 'Asia/
Jerusalem');
dt.toString(); => Sun Sep 27 2009 01:59:59 GMT+0300 (IDT)

dt.getTimezoneOffset(); => -180

// IST fall-back
var dt = new fleegix.date.Date(2009, 8, 26, 23, 0, 0, 'Asia/
Jerusalem');

dt.toString(); => Sun Sep 27 2009 01:00:00 GMT+0200 (IST)
dt.getTimezoneOffset(); => -120

(Note the corrected timezone offsets and abbreviations)

[Jan Niehusmann]

Hi Ricky,

On Fri, Apr 02, 2010 at 09:23:56PM -0700, Ricky Romero wrote:
> I think it would work better if we could assume that the user is
> giving us UTC instead (or just take a Date object we get from them and
> take off the system offset ourselves), and automatically return the
> time with their requested timezone's offset applied. If they change
> TZIDs, we return to UTC, then calculate the new offset and return the
> date with new TZID's offset applied. It's more intuitive that way.

With the latest changes, it was easy to implement the necessary support
in timezoneJS.timezone: I added an (optional) third parameter isUTC to
getTzInfo. If it's true, the date object is assumed to contain a true
UTC timestamp instead of local time.

http://github.com/jannic/timezone-js/commit/11c416df920f909da436293e58f6d2cee1657e41

Jan

[Jan Niehusmann]

On Fri, Apr 02, 2010 at 09:29:27PM -0700, Ricky Romero wrote:
> On another note, zdump (on Mac OS X) actually gives us all the testing
> data we need without having to be evil and hit poor timeanddate.com
> over and over. :-)

Great hint, I didn't know that command. (It's available on Linux, as
well)

Thank you!

[Ricky Romero]

Cool, I'll get it into a nice large test case then. :-)

Ricky

> http://github.com/jannic/timezone-js/commit/11c416df920f909da436293e5...
>
> Jan

[mde]

Nice work, guys. I just merged these changes. The only modification I
made was to put your names in the Contributions section at the bottom.
I guess it needed some reformatting to be more obvious -- y'all didn't
seem to notice it. :)

I've put y'all at the top of the list -- please add your e-mail
addresses if you'd like. The other contributors have theirs listed.

Looks like it was a good idea to get this discussion out in the
open. :) I'm really happy to see this discussion and potential
improvements.

BTW, in case you weren't aware -- zic is the zone info compiler on
Unixy OSes. You can do "man zic" to see a bunch of info about the
Olson data format.

M.