DateTime.from_iso8601 function is missing

[Joe McIlvain]

We have `Date.to_iso8601`, `Date.from_iso8601`, `Time.to_iso8601`, `Time.from_iso8601`, and `DateTime.to_iso8601` functions, but we do not have a `DateTime.from_iso8601` function.

Congruence led me to believe that this function *had to* exist somewhere, so I spent a significant amount of time searching and confirming that it was indeed missing.

I assumed that there must be a good reason for it to be missing, so I searched the issue tracker, this mailing list, and eventually asked in IRC. Here's a link to the transcript of that discussion, for context: https://botbot.me/freenode/elixir-lang/2016-11-28/?msg=77118328&page=6

In short, it was suggested that conversion from ISO8601 to DateTime format was ambiguous, since there is no concrete time zone encoded in the ISO8601 format, and the DateTime struct, expects one.

However, we do have "generic" time zone codes that could be used to convey the situation of "I know the UTC offset, but I don't know anything else about the time zone". According to wikipedia, the IANA tz database defines the "Etc" area, and codes like "Etc/GMT+3" for this kind of situation.  This seems to be the approach that the `timex` library uses when parsing formats that only know the UTC offset.

For whatever it's worth, Ruby's `Time.parse` method uses `nil` for the time zone code when it is not known, and relies on only having `utc_offset` defined, so we could also do something similar to that.

In general, I'd argue that having bidirectional conversions for `DateTime` (or no conversion at all) is the best way to violate the "principle of least surprise", especially when bidirectional transformations are implemented for `Date` and `Time` to/from ISO8601.

Is there another reason to exclude this function?  If so, we should at the very least have the reasoning on record so that users who run into the same question can answer it by searching instead of asking again.

Thanks,

[José Valim]

You have quoted the rationale correctly: ISO8601 does include the proper timezone.

I will have to discuss with maintainers what is the best way to go from here. Using "Etc/GMT+3" may potentially defeat the purpose of using DateTime because we will be unable to properly account for time changes. On the other hand, we may add the proposed function with extensive disclaimers about its behaviour.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Joe McIlvain]

For whatever it's worth, it seems to me that if you're using ISO8601
as the canonical representation of time in your application (whether
for the user-facing representation, or for database-facing
representation), it means that you're already okay with losing this
locale information.

Maybe it's a lack of imagination on my part, but it's hard for me to
imagine a real-world application where the ISO8601 format is used to
represent time but the locale/daylight-savings-time information is
important - I'd think that if this information was important to you,
you wouldn't use a machine-oriented format that throws the information
away. That is, for human facing display formats we throw information
like this away quite often (e.g. human-friendly sentence formats like
"5 days ago"), but ISO8601 is a mostly machine-friendly format (that
happens to be somewhat human-friendly), so it seems unlikely to be
part of an application unless it encodes everything the application
needs.

To think about it in another way, you lose the information when you do
`DateTime.to_iso8601`, not in `DateTime.from_iso8601`. If anything,
it seems like `DateTime.to_iso8601` should be the controversial,
disclaimered transformation because this is the lossy point where
information about the specific locale is lost.

I definitely agree that we want to prioritize correctness and avoid
surprising behaviour, but I think this tradeoff (applications that
store/accept/parse data in a lossy format can't make use of the lost
information) is pretty reasonable and would do a good job of making
the standard library maximally useful for the widest variety of
applications, provided that the caveats/limitations are
well-documented. Particularly in this specific case, it seems to be a
common tradeoff made with respect to ISO8601-parsing libraries (see
the `timex` library and Ruby standard library examples I quoted in the
previous email).

Also, I recognize that I can simply use `timex` for my application and
be done with it, as it already behaves exactly like I'm talking about,
but 1) I think this would be a reasonable and congruent thing to
expect from the standard library, and 2) the `timex` application does
a ton of things that I don't need it to, and even requires to be added
to the mix `applications` so it can run supervised backend processes -
definitely more than I expect for just wanting to parse a string to a
DateTime.

Thanks for your consideration!

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elixir-lang-core" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elixir-lang-core/WbvuNovvB0U/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elixir-lang-co...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2B3F0U03svEfAy85_qscW3Ay6Eb2YLx4UqGNAbyYNTaVQ%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.

[José Valim]

To think about it in another way, you lose the information when you do
`DateTime.to_iso8601`, not in `DateTime.from_iso8601`.  If anything,
it seems like `DateTime.to_iso8601` should be the controversial,
disclaimered transformation because this is the lossy point where
information about the specific locale is lost.

The issue is that DateTime.from_iso8601 does not contain enough information to build a DateTime in the first place. The time_zone field is not currently allowed to be nil (solution 1) and Etc/GMT+X does not support all offsets, such as 5.5h, which is a valid offset (solution 2).

So I agree that DateTime.to_iso8601 is controversial (and we have warnings on its documentation) but at the moment DateTime.from_iso8601 is not possible. I believe it would only be possible if we convert it to UTC but I am not sure if it is a valid trade-off in our case.

[José Valim]

We have added DateTime.from_iso8601 that can parse a ISO8601 as long as it has an offset. It will convert the datetime to UTC (a valid timezone) and also return the parsed offset.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Allen Madsen]

As another data point, I've built 5 apps that only use UTC and iso8601
for transport. This feature is much appreciated since it's pretty much
the only reason I use Timex.

Allen Madsen


On Mon, Nov 28, 2016 at 6:20 PM, José Valim

<jose....@plataformatec.com.br> wrote:
> We have added DateTime.from_iso8601 that can parse a ISO8601 as long as it
> has an offset. It will convert the datetime to UTC (a valid timezone) and
> also return the parsed offset.
>
>
>
> José Valim
> www.plataformatec.com.br
> Skype: jv.ptec
> Founder and Director of R&D
>
> On Mon, Nov 28, 2016 at 10:39 PM, José Valim
> <jose....@plataformatec.com.br> wrote:
>>>
>>> To think about it in another way, you lose the information when you do
>>> `DateTime.to_iso8601`, not in `DateTime.from_iso8601`. If anything,
>>> it seems like `DateTime.to_iso8601` should be the controversial,
>>> disclaimered transformation because this is the lossy point where
>>> information about the specific locale is lost.
>>
>>
>>
>> The issue is that DateTime.from_iso8601 does not contain enough
>> information to build a DateTime in the first place. The time_zone field is
>> not currently allowed to be nil (solution 1) and Etc/GMT+X does not support
>> all offsets, such as 5.5h, which is a valid offset (solution 2).
>>
>> So I agree that DateTime.to_iso8601 is controversial (and we have warnings
>> on its documentation) but at the moment DateTime.from_iso8601 is not
>> possible. I believe it would only be possible if we convert it to UTC but I
>> am not sure if it is a valid trade-off in our case.
>>
>

> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to elixir-lang-co...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Jp0dx5x%2BH6im3iho%2BXE2rB2Yg-%3Dh72nLTwHDgygH8ttA%40mail.gmail.com.

[OvermindDL1]

What about a DateTime.from_iso8601 where you *have* to pass in a timezone or whatever else missing information in to it as well?  That way the onus falls on the user to figure out what it should be?

[Joe McIlvain]

That works great for me.  The next step in this process for me is always convert-to-UTC anyway, and it seems like the return value gives enough information for someone with a different use case to piece together a makeshift timezone from the UTC offset if they need to.  Seems like a good solution.

Thanks for the quick response, and I look forward to the 1.4.0 release!

[Allen Madsen]

That works for me as well.

Allen Madsen

> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to elixir-lang-co...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-core/6d2cbe95-6dae-4347-a66a-4e031cfa2c65%40googlegroups.com.