[Proposal] Add `shift/2-3` to calendar types

[Theo Fiedler]

Preface

We currently have `add/2-3` to manipulate calendar types in the standard library. These functions allow adding a specified amount of time of given unit to a date/time. The standard library currently misses means to apply more complex, or logical durations to calendar types. e.g. adding a month, a week, or one month and 10 days to a date.

Reasons for it

While similar functionality exists in libraries, such as CLDR, Timex, Tox, adding this functionality to the standard library has already been requested and discussed at multiple occasions over the past years. To list a few examples:

- https://github.com/elixir-lang/elixir/pull/10199

- https://elixirforum.com/t/get-date-n-months-years-in-the-past/48346/3

- https://elixir-lang.slack.com/archives/C0HEX82NR/p1709581478427009?thread_ts=1709368588.334759&cid=C0HEX82NR

Furthermore the shift behaviour in the extremely popular library Timex changed in Elixir >= 1.14.3 which may have complicated the mostly lean and non-breaking language upgrade Elixir has to offer.

Elixir has a great set of modules and functions that deal with date and time, the APIs are consistent and `shift/2-3` should fit right in, solving many standard needs of various industries, be it for reporting, appointments, events, finance... the list goes on, engineers probably face the need to shift time logically more often than not in their careers.

Technical details

Duration

A date or time must be shifted by a duration. There is an ISO8601 for durations, which the initial implementation is loosely following. The structure of a Duration lives in its own module with its own set of functions to create and manipulate durations. One example of where it diverts from the ISO standard, is that it implements microseconds. Microseconds in a duration are stored in the same format as in the time calendar types, meaning they integrate well and provide consistency.

Shift

The shift behaviour is implemented as a callback on Calendar and supported by all calendar types: Date, DateTime, NaiveDateTime and Time. Date, Time and NaiveDateTime each have their own implementation of a "shift", while DateTime gets converted to a NaiveDateTime before applying the shift, and is then rebuilt to a DateTime in its original timezone. `shift/2-3` also has guaranteed output types (which isn't a given in many libraries) and follows the consistent API which is established in the calendar modules.

Find the current state of the implementation here: 
https://github.com/elixir-lang/elixir/pull/13385

Benchmarks

There are some benchmarks + StreamData tests in the PR description.

Outlook

After  adding the Duration type and shift behaviour to the standard library, the following things could be explored and derived from the initial work:

• Implementing a protocol that allows Duration to be applied to any data type, not just dates and times.
• A range-like data type that allows us to do recurring constructs on any data type. For example, Duration.interval(~D[2000-01-01], month: 1), when iterated, would emit {:ok, date} | {:error, start, duration, reason} entries
• A sigil for easy creation of durations: ~P[3 hours and 10 minutes]
• Making it so add/2-3 reuses the shift_* functions

Reasons against it

While I am convinced that adding `shift/2-3` to the standard library would be very beneficial, nothing really speaks against the points mentioned above to be implemented in a library instead. However, something as crucial and central as date/time manipulation should still be part of the standard library, negating the risk of breaking changes, inconsistent behaviour and outdated or too unique ergonomics which aren't widely applicable, unlike what should be part of the standard library.

Many thanks to @jose & @kip for the initial reviews and everyone in advance taking the time to read the proposal!

Looking forward to hear other peoples ideas and opinions on the subject!

[José Valim]

The main argument for having it in core is:

  * It integrates directly with the Calendar behaviour
  * We could provide built-in sigils in the future to create readable durations, such as ~P[3 hours and 10 minutes]
  * Postgrex, Explorer, CLDR, etc all implement their own version of durations

Arguments for not having it in core: it happens that all of the arguments above can also be solved without adding Duration to Elixir and, instead, by creating a custom library:

  * A separate library could extend the calendar behaviour with shift_* functions
  * Third-party sigils can also be provided by libraries

  * Postgrex, Explorer, and CLDR could create or use a package with a duratio type shared across them all

I would love to hear the community thoughts.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/cb0ed628-3848-4de0-aa13-c0f4761e4d99n%40googlegroups.com.

[Panagiotis Nezis]

+1 for this, awesome work Theo. Shifting dates/timestamps is such a common operation and a standard implementation would be beneficial for everybody.

PS. I would expect plural in the duration fields. 

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BNmFsMhbkRubMjnmM8c_Amq8DgmKCJtzJ1GEuM4-sVgw%40mail.gmail.com.

[José Valim]

We discussed plural vs singular and settled on singular so it mirrors the calendar types.  Thoughts?

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAPxxbtjvFwnMXe134RR8wjnYk%2Bm-%2BF%2BO_79dWKk3G-bt99Ln%2Bw%40mail.gmail.com.

[Kip Cole]

I went with plurals in my ex_cldr_calendars implementation and I wish I didn’t.  Consistency across APIs reduces cognitive load in my opinion, even if grammatically (in English) dissonant.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BNWO687y8hOWhGFLLAWWF_bwJ8bJ1%3Dd%3DPOWgH9a%3DfdYQ%40mail.gmail.com.

[José Valim]

After a quick glance on other programming languages, it seems Python, Java, Rust, and C# all have plural names. Erlang also uses plural in its helper functions in the timer module. So we might want to follow suit.

[Theo Fiedler]

While i was strongly leaning towards singular, i understand why one would expect plural. Given that seems to be pretty standard in wild, i am fine changing it as well.

What mostly put me off about was that we'd end up with `Time.add(t, 3, :minute)` vs `Time.shift(t, minutes: 3)`, which after all, maybe isn't too bad, as we can keep the plural keys exclusive to durations. Another reason for going with plurals is that it _should_ make migrating from some libraries to the standard library relatively straight forward (with the exception of microseconds).

[José Valim]

Compatibility with the other time units is an important point. My mind is back on singular again. :)

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/3a82f501-cc62-409c-a653-d4b6e5943407n%40googlegroups.com.

[Theo Fiedler]

Right, it would make using a Duration in combination with the `add/2-3` functions much harder than it needs to be. So far all time units in Elixir are singular, and I think we do gain something from consistently sticking to that, regardless of the context of durations, calendar types and what not.

I've seen some libraries allowing both, singular and plural, which i dont want to have anything to do with, except for mentioning it though haha.

What i currently see is:

Reasons for plural:

- Known standard across various libraries and programming languages

- Sounds natural, to shift a date by "3 months" instead of "3 month".

Reasons for singular:

- Compatible with time units already defined in Elixir (also relevant for extending the use of Duration later on)

- Reduced cognitive load as the time units are always spelled the same regardless of the context

The reasons for singular do outweigh the reasons for plural, so unless we're making some very strong points for diverging from that, let's keep it singular!

[Kip Cole]

> I've seen some libraries allowing both, singular and plural, which i dont want to have anything to do with, except for mentioning it though haha.

I very very nearly did this in ex_cldr_calendars but I’m glad I didn’t.  I think it looks good on the surface but creates lots of confusion and ambiguity late.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/786f6323-7f47-44dc-8438-8a6fc94737afn%40googlegroups.com.

[Sabiwara Yukichi]

I'm personally leaning more towards the plural, because it feels semantically more correct to treat a point of time and a duration as separate.

d.year means the same thing if d is either a date or a datetime, but for a duration calling it d.years emphasizes the difference.

It could perhaps help catch errors as well, both for the human and the compiler.

One (arguably contrived) example would be structurally typed code which doesn't enforce any type in particular but uses the dot access or partial pattern matches like %{year: ..., month: ...} in order to support both dates or datetimes. Passing in a duration wouldn't make sense semantically, having different names would make it fail properly.

I also agree with other reasons mentioned, the known standard one especially.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/786f6323-7f47-44dc-8438-8a6fc94737afn%40googlegroups.com.

[Kip Cole]

In my head, a Date.t is semantically a duration. So it’s completely valid to pass it as a duration to Date.shift as I see it. Which argues for singular names.

This conversation is a bit like “is a date a point in time or an interval”. And the answer is yes, depending.

Sent from my iPhone

On 8 Mar 2024, at 14:39, Sabiwara Yukichi <sabi...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CANnyohYGR832XTg9SawCtat3NKxJVMu%3D7KW%3D8VJ7fTCjQro8Eg%40mail.gmail.com.

[José Valim]

It is worth noting that Date and friends in Elixir require a calendar field, which is not present in Duration, and therefore Duration won't be usable as Date (and friends).

The biggest question is if we consider the fields in Duration a unit or not. If they are units, then the most consistent choice is to keep them singular, to mirror System.time_unit and friends.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/5C8C67FE-2A5D-4375-B316-B49727955084%40gmail.com.

[Bruce Tate]

The biggest question is if we consider the fields in Duration a unit or not. If they are units, then the most consistent choice is to keep them singular, to mirror System.time_unit and friends.

This is the API I prefer: units. IMHO, it is more important to keep consistency with Elixir libraries. 

-bt

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4K00prM7KsJ47mcc2MwL2ttz%2BNOpHp6WUt9PEPQa2gK4A%40mail.gmail.com.

--

Regards,

Bruce Tate

CEO

Groxio, LLC.

512.799.9366

br...@grox.io

grox.io

[Kip]

I'm just now implementing the new callbacks for `ex_cldr_calendars` and in reviewing the implementation for `Calendar.ISO` is strikes me that the whole implementation, except for one line (see below) depends only on other calendar callbacks. And therefore could be moved into the `Calendar` module and used as an implementation for any calendar that implements `Calendar` behaviour.

This line would need to change to use `calendar.months_in_year/1` and a few other calls would need to change to be calendar-referenced to. Moving the code might also allow centralising the options handling and exceptions - it's a little unusual for me to see callbacks handling the options validation rather than the public API.

I am more than happy to submit a PR for this very small refactor (thanks to the very clean implementation) if this idea is considered to have merit.

[José Valim]

Hi Kip, please send a PR. I think this will be easier to see in code but what you said makes sense on paper. :)

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/ef317fb5-5b37-4bbd-9cb3-a290919468b9n%40googlegroups.com.

[Kip Cole]

> but what you said makes sense on paper

Thats been an issue my whole life. Works well on paper, but …..  :-)

Will work up a PR ASAP.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JvYKuoCPBa7-sak3rUJ%2B0dsStVrEF2bZ6FXGkU6o2szQ%40mail.gmail.com.