New Time/Date API

[Evan Czaplicki]

Context

Part of getting 0.19 ready is resolving a bunch of "small" issues, like recursive value code-gen. One of these issues is that the Date library does not really do what anyone wants, and I have been looking into this for quite a while. A few days ago I worked with Stoeffel (one of my colleagues) to crystalize these thoughts into an actual API, and it went pretty well!

Overview

After looking at a bunch of Date APIs, I was always confused by libraries that actually use year/month/day as part of the internal representation of a date.

If you instead only track a POSIX time and time zone, you can always derive the date. This representation is simpler and easier to work with. If you want the date in another time zone, use the same POSIX time! What is the wire format for a date? A POSIX time! It seems like a very clean separation.

Based on that intuition, Stoeffel and I made this draft API.

If there are things you think are missing, please state the concrete usage scenario as concisely as possible.

Next Steps: Part 1

I need to coordinate with the maintainer of elm-community/elm-time to figure out how it should fit with the 0.19 version of things. I have reached out to folks on Slack to get that going.

Next Steps: Part 2

A crucial part of this API is that you create a Time.Zone based on the IANA time zone database. This allows the (POSIX + zone) representation to give you correct years, months, etc. (The moment.js JSON representation of the database is 23kb when minified and gzipped, and that data needs to get to browsers somehow.)

Folks will have different needs, so I do not want core to commit to in any particular loading strategy. Some people may want it hard coded into the generated JS. Some folks may want to load it as needed as JSON. Etc. So I made an escape hatch such that library authors can provide alternate ways of loading that information. 

Ultimately, I want my Time Zone proposal to get through TC39 so no one needs to download this stuff anymore. This is another reason I do not want to commit core to a particular loading strategy.

Point is, I think there should be a library that helps people load time zones. Such a library would need to hook data up to something like this:

customZone : String -> List Era -> Maybe Zone

type alias Era =

  { start : Int

  , offset : Int

  , abbreviation : String

  }

So based on a wrong subset of this data, you could say:

newYork =

  customZone "America/New_York"

[ Era 2120108400000 240 "EDT"

, Era 2140668000000 300 "EST"

]

If you make a library for this, let's talk about it before you publish, and I'd recommend delaying publish until 0.19 anyway. I suspect it should be published under the author's name, but this could vary depending on the API details.

The elm-community/elm-time library has this data in Elm code, and I want to coordinate about how to improve that. I.e. parsing from strings seems like the wrong call.

Note 1: I am not convinced that the representations used by moment.js are great or correct (see the nulls in there) so I think it'd be good to keep an open mind. Maybe it'd be better to derive this data from the IANA database directly in whatever format makes sense.

Note 2: With per-declaration DCE in 0.19 it is actually fine to have lots of data in a library. Folks will only get what they need in their generated code.

[W. Brian Gourlie]

The obvious shortcoming in using POSIX as an internal representation is that you can't represent dates prior to 1970. Perhaps the internal representation should be the offset from the UNIX epoch, meaning the internal representation could be negative. This would allow the Date API to accommodate dates before and after the epoch with a range of 2^52 seconds in either direction.

Brian

[Evan Czaplicki]

There were a couple questions in other emails. I will try to summarize the questions with answers here:

• Will Posix time work outside of 1970 to the 2^32 cap of 2038? Yes. The algorithms for year, month, etc. work extremely far into the past and future. By keeping the Posix type opaque, we can move to 64-bit ints internally (or whatever else).
• Should core support parsing ISO 8601 times? Perhaps. I didn't implement it yet though. That function could also live in a time-extra or a localization library. TBD.
• Should core support parsing "fuzzy" times? Like times that are formatted by some locale or missing info? No, I think that's better in a library for localization.
• There needs to be a smooth way to get time zone data. How does this API do that? The majority of my email is devoted to outlining my plan for that happening outside of core. I am communicating about this well in advance of the 0.19 release such that the necessary work on libraries that support a variety of time zone data loading approaches is not blocked on me. I am confused how that was not clear.
• How do I do "four hours later" with this API? Add (4 * 60 * 60 * 1000) to the Posix time.
• How do I do "two days later" but keep the time the same across daylight-saving moves? This API had a travel function at first for this exact case, but as I thought it through, I could not figure out a situation when this was exactly what I wanted. I suspect folks will be using a custom representation in any realistic scenario. For example, in a calendar, I would have a representation that makes sense for describing recurring events, and the logic of "two days later" needs to be bespoke for that case because if any of the dates cross a leap year, they should all behave the same way. As another example where it's not quite right, Google Calendar does not have a "two days later" feature AFAIK. You move everything through the UI. So when I say "concrete scenario" I mean an actual situation, not the generic description of situations that may exist like "business rules require operations on DateTime like shifting time..."

Some folks also suggested that this live outside core at first. That seems reasonable. I'm not sure what is correct yet.

--
You received this message because you are subscribed to the Google Groups "elm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/413c958a-032a-4b0c-9d1b-089498048e8e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Brauner]

I'm not sure it applies to Elm as it is for now a client-side language but I know of a reason to not store dates as posix time but instead as year-month-day-hour-minute: some country might decide to change timezones or to change the rules of daylight saving time. This has happened in Turkey recently so this is not hypothetical. Let's say you're writing a calendar application and a user specified that some event should happen at 4pm. Then you want the event to happen at 4pm regardless of the country's policy. You can always alter your database to amend all the dates after a certain point in time. But storing them in a format that records the original user intent date saves you from doing this.

[Rupert Smith]

On Tuesday, October 31, 2017 at 1:57:47 PM UTC, Paul Brauner wrote:

I'm not sure it applies to Elm as it is for now a client-side language but I know of a reason to not store dates as posix time but instead as year-month-day-hour-minute: some country might decide to change timezones or to change the rules of daylight saving time. This has happened in Turkey recently so this is not hypothetical. Let's say you're writing a calendar application and a user specified that some event should happen at 4pm. Then you want the event to happen at 4pm regardless of the country's policy. You can always alter your database to amend all the dates after a certain point in time. But storing them in a format that records the original user intent date saves you from doing this.

If I was writing a booking application for say a world-wide car rental business, I would not use posix timestamps to record customers preferences about drop-off and pick-up times for the kinds of reason that you outline above. I would instead create my own custom type that capture year-month-day or year-month-day-hour-minute, or find another library that provided such a type; in order to faithfully capture the intention.

However, I don't think that is the job of the core time library.

It is a small point, and perhaps convention has already overruled it, but I would not call the posix timestamp type 'Date'. It would seem more accurate to call it 'TimeStamp' or even just 'PosixTime'. It isn't a 'date'. It can be converted to a date with understanding and care. It also feels a bit weird to be timestamping things down to millisecond accuracy with something called Date.

But I know what it really is, so perhaps it doesn't matter?

[Cdeszaq]

For analytics applications, particularly ones focused around time series data, there's a significant mismatch between the realities of time (ie. "it's complicated(tm)") and the user's mental model of time. The core challenge is that most users think in terms of "local time", but the analytics tools and systems need to work within a more precise time framework. Bridging this gap consumes a great deal of mental effort for developers, and is also a source of a significant number of hard-to-find correctness bugs, based on my experience. Here are a few of the areas related to closing that gap that I didn't see well supported in the proposed API:

Instant Offsets

A specific example, which it looks like someone may have already commented on, is how to get an instant that is X hours back in time from some other instant. Since most users will think about this in local time, and often in their own time zone, computing this accurately across DST or leap boundaries is non-trivial. Based on how you answered that challenge, it seems like it's up to the caller to take the Posix time, add the requisite number of milliseconds, and then convert back into the desired Zone. While a simple transformation, that seems like exactly the sort of detail a Time abstraction should hide from me.

This same question gets even harder, however, when dealing with irregular time units, like Months or Years; getting an instant X months or years back in time, for example. In that case, you can't simply calculate the number of milliseconds to use for the offset because months and years are irregularly sized. At this point, I think it's no longer a good idea to push the responsibility of doing the calculation onto the user of the API, since the calculation requires knowledge of things encoded in the TZ database.

Partial Instants

Another use-case I didn't notice as being supported is the ability to partially specify instants. The toCivil function is along those lines, but in the reverse. The place this comes up is that humans (especially in an analytics app) often want to only specify the Year, or Month, or Week that they are interested in. Fully specifying the instant would be painful. 2017-01-01T00:00:00.000Z, for example, only really cares about the year in the 1st "digit". The rest of it is just noise.

A different way to present this is that humans often think about time at different granularities depending on what they are doing with it, while the API as written doesn't seem to support that very well, requiring consumers to close that gap themselves each time it comes up.

Weeks

While not super common, analytics apps often think in terms of weeks. eg. Week over Week reporting, weekly aggregations, etc.  I didn't see any treatment of weeks in any significant way, yet it is a common human abstraction. Additionally, ISO 8601 also has a special section devoted to week representation, so supporting ISO 8601 via this API will be less rich if it doesn't have weeks as a core concept.

Calendars

A final concern that came to mind is representing entirely different calendars, likeHebrew, for instance. I am less familiar with the needs for this, but alternate calendar representation is a core capabilities of well regarded time libraries like JodaTime. I have not dealt with those issues myself, so I cannot speak to their relative merits of inclusion in a core time abstraction, but I had not seen them mentioned yet.

It's been a bit since I've delved into the murky mental depths of this, but I believe the core mismatch is that humans generally think of time in an ISO 8601 way, in terms of years, months, days, hours, etc. while time itself is a continuous dimension. Humans bucket time, turning it into a discrete space (or many discrete spaces of differing granularity), but we often represent it as a continuous thing (ie. a count of milliseconds since some epoch). Bridging that gap, from the human mental model into the reality of continuous time while dealing with the complexities of Zones, etc., is the need a Time API is trying to fill.

Given the complexities and difficulties of dates and times, and how widely the complexity can vary based on the use-cases, the right choice may very well be to keep a very simple representation and API. Since the goal of the new API is to "do what people want", it may not make sense for it to try to do what only advanced analytics applications want. But if that is the chosen path, I think having some disclaimers or "anti-docs" about what the abstraction is known not to do well would go a long way towards setting expectations.

Sorry for the book-length feedback, and for being a bit late to the party, but Time has caused me such pain in the past that I can't help but share my experiences so that others may hopefully benefit. I'm perhaps overly familiar with time-based analytics, and I'm very new to Elm, so please forgive me if I simply misunderstood the API, but those are the concerns and ill-supported use-cases that jumped out at me at 1st glance. Hopefully some of the concerns are mitigated by other aspects of Elm, but I thought it best to call them out since they were non-obvious to me. Admittedly, analytics is not what the majority of applications use Date/Time for, but it is a significant category of interactive applications, so I'd hate to see them stymied due to insufficient Time capabilities.

- Rick

On Sunday, October 29, 2017 at 8:59:30 PM UTC-5, Evan Czaplicki wrote: