What tool generates TimeZoneConstants.properties?

681 views
Skip to first unread message

Ahmad Bawaneh

unread,
Dec 7, 2018, 7:54:09 AM12/7/18
to GWT Contributors
Anyone know what is generating TimeZoneConstants.properties, it says it is generated from CLDR version 25 .. but i cant find any thing that generates this gile in gwt code nor in gwt-tools code, was it generated externally?

Thomas Broyer

unread,
Dec 7, 2018, 9:32:40 AM12/7/18
to GWT Contributors


On Friday, December 7, 2018 at 1:54:09 PM UTC+1, Ahmad Bawaneh wrote:
Anyone know what is generating TimeZoneConstants.properties, it says it is generated from CLDR version 25 .. but i cant find any thing that generates this gile in gwt code nor in gwt-tools code, was it generated externally?

Have you tried pinging Manolo? He's the last one to have updated the file, maybe he remembers the details. Either him or John Tamplin. 

Colin Alworth

unread,
Dec 7, 2018, 9:55:50 AM12/7/18
to google-web-tool...@googlegroups.com
I messaged Manolo this morning after seeing this, and in gitter where we're talking about this.

From some digging in history, an old copy of the properties file referenced pytz2011n, which looks like it might be a specific release of pytz. The commit message Manolo made had a reference to some internal Google python script, which I'm guessing takes that pytz library/database (looks like they also just reference iana) and makes the .properties file, and updates the interface that points at the file.

The commits for previous changes went through google code and were imported to git, so I'm not sure we can find much more public commentary.

--
  Colin Alworth

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thomas Broyer

unread,
Dec 7, 2018, 10:17:20 AM12/7/18
to GWT Contributors
We may want to generate that file using java.time.ZoneId then; WDYT?


On Friday, December 7, 2018 at 3:55:50 PM UTC+1, Colin Alworth wrote:
I messaged Manolo this morning after seeing this, and in gitter where we're talking about this.

From some digging in history, an old copy of the properties file referenced pytz2011n, which looks like it might be a specific release of pytz. The commit message Manolo made had a reference to some internal Google python script, which I'm guessing takes that pytz library/database (looks like they also just reference iana) and makes the .properties file, and updates the interface that points at the file.

The commits for previous changes went through google code and were imported to git, so I'm not sure we can find much more public commentary.

--
  Colin Alworth



On Fri, Dec 7, 2018, at 8:32 AM, Thomas Broyer wrote:


On Friday, December 7, 2018 at 1:54:09 PM UTC+1, Ahmad Bawaneh wrote:
Anyone know what is generating TimeZoneConstants.properties, it says it is generated from CLDR version 25 .. but i cant find any thing that generates this gile in gwt code nor in gwt-tools code, was it generated externally?

Have you tried pinging Manolo? He's the last one to have updated the file, maybe he remembers the details. Either him or John Tamplin. 


--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.

Ahmad Bawaneh

unread,
Dec 8, 2018, 6:50:14 AM12/8/18
to GWT Contributors
I think in the GWT2 context we can just generate a new file with the same old tool with updated data, but in the GWT3 context i think we need to think of another approach.
few weeks ago i was working on a JodaTime emulation for GWT, one part of that work was the generation of time zone info, i managed to build a tool to generate them based on JodaTime runtime time zone info provider which instead of creating run time instances i made the code emit source code and it worked, i still think it could be made better by implementing a wrapper instead of changing the code itself, anyway .. the time zones info in joda time was based on iana database, and joda time is a will maintained and regularly updated and reliable, so we can use that as a base for a generator instead if inventing the whole wheel.

in my approach i was generating a class per time zone which a factory class to get a time zone by id, but if we still think that the JSON constants is a better approach we still can generate the JSONs from those files.

this is just a thought that i had as i was working on emulating joda time.

Manuel Carrasco Moñino

unread,
Dec 17, 2018, 5:22:29 PM12/17/18
to google-web-tool...@googlegroups.com, John A. Tamplin
Sorry for the late reply. 
In theory the process is: google modifies original cldr distribution per version, and gives a patch to us, then we run our generator and commit modified files.
We don't run any python script, hence TimeZoneConstants.properties should have been always provided by google.
I don't remember how that file in my commit was generated, apparently we should have some utility file for that, I have checked out to previous commit snapshot, patch cldr with goole patch, and run tools to generate files, though TimeZoneConstants.properties is not generated. It's so long ago that I don't conserve original tree with some program I could have utilised for that.  CC-ed J.A. Tamplin, he might remember about if there was some parallel utility.
I'll try to investigate further though

- Manolo


To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/e48d1ef7-5f72-4802-8ffa-9358e194f523%40googlegroups.com.

Colin Alworth

unread,
Dec 17, 2018, 7:11:01 PM12/17/18
to google-web-tool...@googlegroups.com
Thanks Manolo - in the course of his updates, Ahmad went over the patch file used previously and found the changes to either be already merged or entirely incompatible - no one objected to simply switching to the stock version of CLDR, so that's what was done instead.

Goktug did some quick checking internally but didn't find anything right away, I owe him a more in-depth writeup of what we think will point at the tool which had been used in the past.

--
  Colin Alworth

Ahmad Bawaneh

unread,
May 21, 2019, 6:33:16 PM5/21/19
to GWT Contributors
In order to overcome this i was trying an experiment based on some work i did before, my original work was about generating TimeZoneInfo classes based in joda-time which uses iana tables. i was trying to make use of this work to regenerate the TimeZoneConstans.properties file, even though i was able to generate the file i notice that i have different values for the transitions and offsets, which made me doubts my results, because now i think its either i am passing the wrong values or the values in the original file have some sort of formula, so i am posting both files looking for help to seek what i am doing wrong or to confirm my results.

Thanks
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.


--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.


--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.
generated-from-my-tool-TimeZoneConstants.properties
original-TimeZoneConstants.properties

Jens

unread,
Jun 5, 2019, 5:01:59 AM6/5/19
to GWT Contributors
I looked at the file and the GWT code using it and the format seems pretty clear to me. Writing a new tool looks pretty simple, we just have to decide which implementation strategy to choose.


1.) use JDK java.time package to read the tzdata information packaged with Java.

Given that JDK is now released every 6 month, this might also be a nice minimum time frame for possibly automated GWT patch releases with new tzdata.

Otherwise if we want to update tzdata in GWT outside of JDK releases, the process is a little more complex because we first have to generate a tzdb.dat file to update a JDK installation. To generate that file, we would also need to use the source from the JDK version we use to execute the GWT tool that generates the GWT json files because the tzdb.dat file format might change between JDK versions.


2.) Use Joda Time and their api to read the tzdata information

You would need to checkout Joda Time sources, put the new tzdata files into a special package and then build Joda Time. A tool would then depend on that custom build Joda Time and use the Joda Time API to produces the GWT JSON files.



Personally I kind of like option 1.) because we could then align automated GWT releases to JDK releases, e.g. one month after JDK release, and the GWT tool could just use standard JDK API without any dependencies.



-- J.

Jens

unread,
Jun 5, 2019, 10:53:33 AM6/5/19
to GWT Contributors

I looked at the file and the GWT code using it and the format seems pretty clear to me. 

Totally forgot to write the format down for documentation:

europeBerlin = {

 // Time Zone Id
 "id": "Europe/Berlin",

 // List of <hours since epoch, DST offset in minutes> pairs. The DST offset is added on top of std_offset if DST is active. Thus it is 0 when switching to normal time.
 "transitions": [89953, 60, 94153, 0, ... ],

 // name of time as pair of <short name, full name>. If transitions is empty, size must be 2, otherwise 4. The first pair is normal time, the second pair is DST.
 "names": ["CET", "Central European Time", "CEST", "Central European Summer Time"],

  // The offset during normal time.
 "std_offset": 60

}



The current file shipped with GWT has no negative transition points, so it does not contain any data before 1970 which is kind of sad. Using IANA db we would have a lot more historic data. GWT also has future transition points which can usually be calculated using the DST rule defined in tzdata (java.time has an API for that and JodaTime probably as well). So a new tool would need to do that as well.


-- J.

Thomas Broyer

unread,
Jun 6, 2019, 7:26:17 AM6/6/19
to GWT Contributors

On Wednesday, June 5, 2019 at 11:01:59 AM UTC+2, Jens wrote:
I looked at the file and the GWT code using it and the format seems pretty clear to me. Writing a new tool looks pretty simple, we just have to decide which implementation strategy to choose.


1.) use JDK java.time package to read the tzdata information packaged with Java.

Given that JDK is now released every 6 month, this might also be a nice minimum time frame for possibly automated GWT patch releases with new tzdata.

Fwiw, the tzdata on Linux (and possibly any non-Windows OS) is generally updated independently from the JDK, and is shared between all JDK versions on the machine.
 
Otherwise if we want to update tzdata in GWT outside of JDK releases, the process is a little more complex because we first have to generate a tzdb.dat file to update a JDK installation. To generate that file, we would also need to use the source from the JDK version we use to execute the GWT tool that generates the GWT json files because the tzdb.dat file format might change between JDK versions.


2.) Use Joda Time and their api to read the tzdata information

You would need to checkout Joda Time sources, put the new tzdata files into a special package and then build Joda Time. A tool would then depend on that custom build Joda Time and use the Joda Time API to produces the GWT JSON files.

There's also ICU4J, which we already use for other i18n related code generation. Its API is similar to the JDK one, but it comes with its own tzdata bundle (afaict).

Personally I kind of like option 1.) because we could then align automated GWT releases to JDK releases, e.g. one month after JDK release, and the GWT tool could just use standard JDK API without any dependencies.

The upside of using the JDK is that it uses the "system" (though java-specific) tzdata (at least on Linux, at least with OpenJDK packaged by the distros), so it's almost guaranteed to be up-to-date. The downside is that is machine/environment-dependent.
The upside of using ICU4J (or possibly JodaTime) is that it's not system/environment dependent. The downside is that it requires updating ICU4J, but that's probably a good idea anyway (but it'd be easier to do it within a separate project using "managed dependencies" –Maven, Gradle, etc.–; we probably wouldn't want to upload every new version of ICU4J to the "tools" repository; this could possibly be done in the new gwt-i18n project for example, with a special task to generate the files needed for GWT2).

I think using the JDK will be "good enough" (given the status quo of having outdated data that we don't even know how to update), and we could still migrate to ICU4J later if we feel it's better.

Thomas Broyer

unread,
Jun 6, 2019, 7:28:36 AM6/6/19
to GWT Contributors
Is bundling data before 1970 really worth the additional bloat? If you ask me, I wouldn't even go down to 1970.

Learner Evermore

unread,
Jun 6, 2019, 7:40:53 AM6/6/19
to GWT Contributors
One more thing to consider is being in synchronization with the Java-based server, if it is used - i.e. using the same data as the server that hosts the GWT client code and serves the API for it.

This is important when scheduling events whose times must/should be stored in their specific / local time zones - i.e. not converting them ahead of time because things may change (as they do). A disagreement between the server and the client would not be welcome in that case. There may be more examples.

Learner Evermore

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/31eeb824-4f40-4fe0-8135-e21f5988166b%40googlegroups.com.

Jens

unread,
Jun 6, 2019, 9:10:11 AM6/6/19
to GWT Contributors

Is bundling data before 1970 really worth the additional bloat? If you ask me, I wouldn't even go down to 1970.

Well ideally the developer should decide how much data the application needs and not GWT SDK. So we should either include all or make it configurable.

If you look at moment-timezone js library the gzipped size of timezone data is about 40kb. Their format is different, but I would expect GWT's json to be in the same region if it would contain all data + future data.

I already thought if GWT could maybe use that moment-timezone js library internally (keeping GWTs property file for API compatibility but make it empty). Yes the JS it is not very well optimizable in GWT 2.x but the reason to use it is that once you have older historic data the current GWT Json format can not express certain situations. For example the standard offset for certain time zones changes multiple times over the past years, but GWT only knows a single standard offset (likely the most recent one at the time the file as generated). Also some DST changes are down to second granularity, however GWT only stores hours as transition points and offsets in minutes which would result in some dates being treated wrong for these timezones.

Of course we could also update the GWT Json format to cover these situations, but it also feels like reinventing the wheel.

-- J.


Reply all
Reply to author
Forward
0 new messages