[Proposal] latest and earliest functions for DateTime and NaiveDateTime

[Tyson Buzza]

I have been using DateTime and NaiveDateTime alot over the last year and have found that using the compare function to involves some mental gymnastics. When using compare you often just want the most recent or oldest timestamp, or else you want to sort a list of timestamps.

I am proposing four new functions for each module: earliest/2, latest/2, sort_earliest/2, and sort_latest/2.  earliest/2 and latest/2 each take two DateTime structs and return the first/last time respectively. The sort_earliest/2 and sort_latest/2 functions take a list and an optional mapper, similar to the Enum.sort_by/2 function. sort_earliest/2 and sort_latest/2 will return the list in ascending/descending chronological order respectively.

Example usage:

```
start_time = DateTime.earliest(datetime1, datetime2)
start_time = Enum.reduce(datetime_list, &DateTime.earliest/2)
sorted_times = DateTime.sort_earliest(datetime_list)

sorted_records = DateTime.sort_earliest(records, &(&1.timestamp))
```

This example is a function that uses sort_earliest to check if a time is inside a given time range:

```

  def in_range(time_stamp, {start_time, end_time} = _range) do

    case sort_earliest([time_stamp, start_time, end_time]) do

      [^start_time, ^time_stamp, ^end_time] ->

        true

      _ ->

        false

    end

  end

```

The implementation would be something simple like this:

```

  def sort_earliest(list, mapper \\ fn x -> x end) do

    Enum.sort_by(list, mapper, fn x, y ->

      :lt == DateTime.compare(x, y)

    end)

  end

  def earliest(%DateTime{} = datetime1, %DateTime{} = datetime2) do

    case DateTime.compare(datetime1, datetime2) do

      :gt ->

        datetime2

      _ ->

        datetime1

    end

  end

```

Might need to make sure sort_earliest is an in place sort.

[José Valim]

I love this. My suggestion is to add earliest(date, date) and earliest(list_of_dates) and the same for latest. We need to add it to Time, Date, NaiveDateTime and DateTime.

A PR is very appreciated.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/47178019-74cd-44bd-b26e-4b092c5c04a2%40googlegroups.com.

[José Valim]

Oh, I see why you called it sort_earliest, because you also want to allow a function to be given, in the style of sort_by. I am not a big fan of sort_earliest though.

What if we add only: DateTime.earliest(list_of_dates) and DateTime.earliest(list_of_dates, fun)? You can always get the earliest of two dates by passing it a list with two elements.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[YongHao Hu]

If Tyson Buzza don't mind, I would love to write this as my first PR as I had used Elixir in work for two years and want to do a contribution. : P

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KYAzPQDYUdyWie3pkamFERh5uHVS-8FJEs%2BsAUDZTRug%40mail.gmail.com.

--

--
Regards,
YongHao Hu

[Tyson Buzza]

I'm in a similar situation to you YongHao. Sorry, but would you mind if I did it?

To be clear the functionality would be:

DateTime.earliest(enumerable, mapper \\ fn x -> x end)

with a typespec like

@spec earliest(Enum.t(), (any() -> t())) :: list()

right?

On Thursday, October 10, 2019 at 4:42:45 PM UTC+8, YongHao Hu wrote:

If Tyson Buzza don't mind, I would love to write this as my first PR as I had used Elixir in work for two years and want to do a contribution. : P

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-l...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/47178019-74cd-44bd-b26e-4b092c5c04a2%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-l...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KYAzPQDYUdyWie3pkamFERh5uHVS-8FJEs%2BsAUDZTRug%40mail.gmail.com.

[Wojtek Mach]

I think earliest and latest are good because they are descriptive but wondering if they should be called min and max instead. The advantage of the latter is they’d be similar to Kernel.min/2 and Enum.min/1,2. I guess a disadvantage is someone reading `min` could think it returns the minimum allowed date (-9999-12-31 etc)

[Wojtek Mach]

Should Date.earliest operate on `Date.t` or `Calendar.date`? That is, should we allow the following?


iex> Date.earliest([~N[2019-01-01 09:00:00], ~D[2019-01-02])
~D[2019-01-01]

Seems like this could be useful.

[José Valim]

To be clear the functionality would be:

DateTime.earliest(enumerable, mapper \\ fn x -> x end)

with a typespec like

@spec earliest(Enum.t(), (any() -> t())) :: list()

right?

Correct.

--

[José Valim]

Good catch. It should operate on Calendar.date, as all other functions in the Date module.

Btw, the reason I prefer earliest/latest is exactly because it has semantic meaning. It also avoids any confusion from people thinking the regular min/max work with calendar types (as you said).

[Wiebe-Marten Wijnja]

Playing devil's advocate here for a minute: The wanted functionality could also be reached using

Enum.sort(enumerable, &(DateTime.compare(&1, &2) in [:lt, :eq]))

for `DateTime.earliest`

and similarly:

Enum.sort(enumerable, &(DateTime.compare(&1, &2) in [:eq, :gt]))

for `DateTime.latest`.

Arguably this does require some mental gymnastics. However, is that enough reason to introduce eight new functions? (two for each of Time, Date, NaiveDateTime and DateTime)?

Maybe these mental gymnastics are not 'too bad'?

Or maybe we do want to make it somewhat easier for the users, but there exists a simpler, more fundamental alternative we could implement, which could then be re-usable in multiple contexts?

For instance, if we only had `DateTime.earliest(dt1, dt2)` then we could write:

Enum.sort(enumerable, &(DateTime.earliest(&1, &2) == &1))

for the first case, and

Enum.sort(enumerable, &(DateTime.earliest(&1, &2) == &2))

for the second.

~Qqwy/Marten

[Ben Wilson]

FWIW We have an earliest_date and latest_date helper in almost every Elixir app we've built. The Enum.sort solution requires requires too many leaps to be at a glance readable if it's been a bit since you used DateTime.compare.

Definitely a fan of including this.

[José Valim]

Yes, I am with Ben. I have considered adding similar APIs already but I could not come up with a good proposal. Now I am quite happy with the current one.

[Fernando Tapia Rico]

I would drop the `mapper` function, and have a similar interface to `Enum.min/2`: `DateTime.earliest(list_of_dates, empty_fallback // fn -> raise(...) end)`.

[Tyson Buzza]

The earliest function as proposed by Jose above is more like the sort_by function than the min function. It returns a sorted list.

[Fernando Tapia Rico]

Right.

I meant to return a single element, the "earliest" one. Otherwise I would find using a superlative for the function name confusing.

The Decimal library could be another example. It provides a Decimal.compare/2 and a more semantic Decimal.min/2 and Decimal.max/2.

[José Valim]

Apologies, I think my previous replies on the topic were confusing/incomplete.

I proposed Date.earliest/2 meaning that it would return a single date, but of course, the sort variants are by far more useful. So we would need Date.sort_earliest/2 or similar but I am bit conflicted because if it is called sort, it should rather be in Enum as a general purpose function?

Therefore, here are other ideas:

1. We can change Enum.sort/2 and friends so that, besides the sorting function allowing true/false, it also accepts :gt/:eq/:lt. This means sorting by earliest is as simple as: Enum.sort(dates, &Date.compare/2). But we still need a way to do descending sort .

2. Add a new sorting function that when given two elements, must return the first, instead of true/false today. This means that we could do Enum.pairsort(dates, &Date.earliest/2) and Enum.pairsort(dates, &Date.latest/2). "pairsort" is obviously a horrible name, suggestions are welcome.

Thoughts?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/bce11106-9f29-457e-b63f-e6eef0633d74%40googlegroups.com.

[Tyson Buzza]

My preference would be for the second of those options, but I don't have a better name for pairsort.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KUZ2c54nfWi6oeU%3DbXfs2k_pbOz20Fv4%3DdMNfPUBmPcg%40mail.gmail.com.

[Bruce Tate]

I strongly prefer rolling this all into enum. As you say, José, turtles all the way down!

This is a brainstorm for naming options, not a proposal. I'm not attached to any of these. 

Pairsort options: 

- order(..., ordering \\ :asc)

- order_asc, order_desc

- compare

optionally, a variant on sort: 

sort_elements

or, translate elements to other things that can be sorted, like this: 

rank(dates, ordering \\ :asc)

That could take

[date1, date2]

and return rankings that can be sorted, like this: 

[2, 1]

That could be as easy as converting to unix date. 

Do any of these ideas help?

-bt

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KUZ2c54nfWi6oeU%3DbXfs2k_pbOz20Fv4%3DdMNfPUBmPcg%40mail.gmail.com.

--

Regards,

Bruce Tate

CEO

Groxio, LLC.

512.799.9366

br...@grox.io

grox.io

[José Valim]

I have put more thoughts into this.

If our goal is to have something that can be easily sortable, computed a max and a min, etc, then the best option is to introduce a sortable function:

sortable(date) :: term()

The goal of this function is to return a sortable term, according to Erlang Term Ordering.

Then we can do:

Enum.sort_by(datetimes, &Date.sortable/1)

Enum.min_by(datetimes, &Date.sortable/1)

Enum.max_by(datetimes, &Date.sortable/1)

If you want to sort users by their update date:

Enum.sort_by(users, & &1.updated_at |> Date.sortable())

Enum.min_by(users, & &1.updated_at |> Date.sortable())

Enum.max_by(users, & &1.updated_at |> Date.sortable())

The only downside of this implementation is that it may not be as efficient because we will have to convert to iso days so they are sortable across calendars.

===

There is a longer discussion here about introducing a new protocol, called Ordered, and an Enum.order/1 function. Then Enum.order(dates) would automatically work, because it would dispatch to a protocol which is always correct. This is all good on paper but I can foresee two downsides:

1. We would need to introduce an "Ordered" version of min/max. Perhaps something like order_first/order_last or order_min/order_max.

2. I am not aware of any prior art on this. Most of the protocols/typeclasses/interfaces I know for ordering build them based on comparisons. Is anyone aware of a reference on the topic?

Thoughts?

[José Valim]

To clarify, the Ordered protocol would be like this:

defprotocol Ordered do

  def ordered(data)

end

Basically, it would work like Date.sortable, but implemented with a protocol so it works across data-types.

José Valim

www.plataformatec.com.br

Founder and 

Director of R&D

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/56a992a6-0ae1-4d67-9145-dbee40935bae%40googlegroups.com.

[José Valim]

Nevermind, the Ordered protocol still wouldn't work for Integer/Float/Decimal, as they can't converted to a single ordered/sortable term.

I think a complete proposal would need to contemplate both calendar types and the latest reverse_sort proposal. Stay tuned.

José Valim

www.plataformatec.com.br

Founder and 

Director of R&D

[Wiebe-Marten Wijnja]

On 17-10-2019 22:34, José Valim wrote:

I have put more thoughts into this.

If our goal is to have something that can be easily sortable, computed a max and a min, etc, then the best option is to introduce a sortable function:

sortable(date) :: term()

The goal of this function is to return a sortable term, according to Erlang Term Ordering.

I believe that having something that would work with Erlang's Term Ordering was why in the distant past there was a proposal to return `:<`, `:=` or `:>` as return type of Date.compare (and variants).
However, if I remember correctly, it was deemed less readable than `:lt`, `:eq` :gt` and that was considered more important.

The only downside of this implementation is that it may not be as efficient because we will have to convert to iso days so they are sortable across calendars.

Yes. And of course for incompatible calendars it will fail at some point. To properly work with calendars that might or might be incompatible, we'd need to not only remember the ISO day (and time) but also e.g. the offset since midnight.

===

There is a longer discussion here about introducing a new protocol, called Ordered, and an Enum.order/1 function. Then Enum.order(dates) would automatically work, because it would dispatch to a protocol which is always correct. This is all good on paper but I can foresee two downsides:

1. We would need to introduce an "Ordered" version of min/max. Perhaps something like order_first/order_last or order_min/order_max.

2. I am not aware of any prior art on this. Most of the protocols/typeclasses/interfaces I know for ordering build them based on comparisons. Is anyone aware of a reference on the topic?

I believe this technique (which is essentially what `Enum.sort_by` already does) is frequently called a 'Schwartzian Transform':

-Take a collection of things

- For each collection transform it into a pair of {sortable_key, element} where `sortable_key` is part of a total ordering with all other `sortable_keys` in the collection (in our case: the Erlang Term Ordering).

- Sort this collection of pairs.

- Drop all the first elements of the pairs again.

Thoughts?

Not a bad idea.

I do wonder about your follow-up: Why would it not work for integers? (and float and decimals)? At least as long as we are comparing collections of only integers or only floats or only decimals, the result would be sensible.

If we want to make Ordered work between them, well, then it becomes a bit more difficult: In that case we probably would need to 'upcast' all of them into some kind of more general tuples-of-integers-and-strings form.
Hmm, interesting problem!

[José Valim]

 

I do wonder about your follow-up: Why would it not work for integers? (and float and decimals)? At least as long as we are comparing collections of only integers or only floats or only decimals, the result would be sensible.

If we want to make Ordered work between them, well, then it becomes a bit more difficult: In that case we probably would need to 'upcast' all of them into some kind of more general tuples-of-integers-and-strings form.

Exactly. That's the problem and at a quick glance I could not find a solution. I am glad to be proven wrong though!

 

[Dmitry Belyaev]

I suppose it's reasonable to have a defined ordering within a particular class e.g. number class includes integers and floats, and have some (possibly undefined) ordering between classes. This basically is the same as Erlang term order, but for every struct we could define its own class or in some special cases a few structures would map to the same ordering class.

Then to make Erlang term ordering to work for us we can build a tuple {{class, class_ordering_term}, original_term}, e.g {{Number, 1}, 1}, {{DateTime, epoch_integer}, %DateTime{...}}, {{Atom, nil}, nil}.

--
Kind regards,
Dmitry Belyaev

[José Valim]

Here is a proposal I am satisfied with: https://github.com/elixir-lang/elixir/pull/9426

It solves sorting but it doesn't solve getting the earliest/latest. There will be another PR to address those. The idea is to provide a general framework to address these limitations, as it will also be useful for libraries like Decimal.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[José Valim]

The second part is: https://github.com/elixir-lang/elixir/pull/9427

The PRs together should make it possible everything proposed in this thread, but in a generic way so it also works with Decimal and other comparison algorithms.