feedback wanted: instant literal

434 views
Skip to first unread message

Stuart Halloway

unread,
Nov 4, 2011, 2:04:31 PM11/4/11
to cloju...@googlegroups.com
Instant literal for Clojure:
http://dev.clojure.org/jira/browse/CLJ-871 and
http://dev.clojure.org/pages/viewpage.action?pageId=950382

This is work in progress. I will add tests and update the patch for
screening based on/depending on feedback here.

If this flies, will be looking for volunteers to do equivalent in
ClojureScript and ClojureCLR.

Stu

Sean Devlin

unread,
Nov 4, 2011, 2:08:54 PM11/4/11
to cloju...@googlegroups.com
Would you like my datable protocol stuff?

Sean


--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To post to this group, send email to cloju...@googlegroups.com.
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.


Chas Emerick

unread,
Nov 4, 2011, 2:51:07 PM11/4/11
to cloju...@googlegroups.com
I've been absent for the prior discussions, and time and date stuff is not really in my wheelhouse, so perhaps I'm just blind to the pressing need. But, I'd like to suggest that perhaps this is a data type that doesn't necessarily need to be elevated to be supported in the reader (and potentially be convinced otherwise, if anyone's willing to take a whack).

That is, why date/time information, and not URLs, or IP addresses, or any number of other scalars that might be handy? For these other examples, I'd say strings are proper and reasonable because they don't presuppose any particular handling of the data involved — those decisions can be made locally, or in canonical, widely-used libraries, and so on. Dates strike me as being in the same category.

Are there any other languages that have date/time literals that we might be able to learn lessons from?

From a purely tactical perspective, seeing joda slip in (as an optional dependency, first of its kind I think!) gives me pause. Now, if I don't have joda around, there are cases where some code simply won't load. Also, now I need to go learn the Joda API if I'm going to work with date data in Clojure? Are we prepared to cope with upstream issues with joda, API changes, releasing patched builds of Clojure as necessary because of it? Not necessarily dealbreakers IMO, but definitely a point of concern.

Finally, totally aside from any actual technical considerations, it seems to me that date/time support is a third rail of language / platform / library development, leaving no one who touches it unscarred. I'd hate to see Clojure's support get it wrong in some subtle way that will either cause breakage or pain down the road.

As I said, date/time stuff is not generally my bag, so I may be suffering from various misconceptions and such.

Cheers,

- Chas

Alex Miller

unread,
Nov 4, 2011, 2:56:10 PM11/4/11
to cloju...@googlegroups.com
My personal reaction is that while Joda is great and building a lib on
top of Joda seems highly useful, the idea of making Clojure core
depend on Joda and return Joda objects seems undesirable.

More specifically (but not too organized):

1) Clojure core then has to deal with dependencies on different
versions of Joda, which may vary in its own Java-level dependency at a
rate different than Clojure. That seems like a complecting of what
Clojure core is. :) You also increase the size of a typical Clojure
install (joda-time is ~500k). If you make it optional, that has it
whole own set of complexities. My god man, the Maven work alone drove
you to drink! :)

2) Now that we have Java literal constructors, you can fairly easily
create Joda objects with 8601 strings:
- #org.joda.time.DateTime("2004-12-13T21:39:45.618-08:00")
- vs #@2004-12-13T21:39:45.618-08:00 (if I understand the proposal)

3) How often do you actually use literal instants in source? I use
them a fair bit in writing test code, but usually you are reading
timestamps as values from somewhere else, not putting this as literals
in your code. Seems like the bigger win is really in having a
standard way to *print* instants to make outputs cleaner. And there's
no reason that can't be done in a lib.

4) If JSR 310 is added in JDK 8 then you have a reader that doesn't
return the now-standard JDK objects which have all of the properties
that are desirable about Joda. Would the reader change at that point?

5) Maybe a moot point but an idle question: is it possible to get
permission to use the portions of Joda that are useful (parser)
without including all of Joda?

In all, I'm unconvinced that the added complexity is worth the value.
(But having a core lib that wrapped Joda and provided everything else
on that page would be great.)

Alex

Sean Devlin

unread,
Nov 4, 2011, 3:06:07 PM11/4/11
to cloju...@googlegroups.com
FYI, parsing ISO8601 is really easy.  It's rendering that's hard.  You could always delegate that to java.util.GregorianCalendar


If you stick to UTC instants and avoid timezones, you don't need Joda.

Sean

Stuart Halloway

unread,
Nov 4, 2011, 3:18:56 PM11/4/11
to cloju...@googlegroups.com
> That is, why date/time information, and not URLs, or IP addresses, or any number of other scalars that might be handy? For these other examples, I'd say strings are proper and reasonable because they don't presuppose any particular handling of the data involved — those decisions can be made locally, or in canonical, widely-used libraries, and so on. Dates strike me as being in the same category.

Funny, I would like to do URLs next. I would call making these decisions locally the "JSON mistake" -- creating a cottage industry of conversions that could be avoided with a common approach.

> Are there any other languages that have date/time literals that we might be able to learn lessons from?

I take the lesson to be stick to standards, i.e. ISO8601.

> From a purely tactical perspective, seeing joda slip in (as an optional dependency, first of its kind I think!) gives me pause. Now, if I don't have joda around, there are cases where some code simply won't load.

This is more about data than about code, but point taken.

> Also, now I need to go learn the Joda API if I'm going to work with date data in Clojure?

No. Rich has already corrected me, pointing out that the API should return java.util.Date. Raises question of an *option* to return Joda types for those who want it.

> Are we prepared to cope with upstream issues with joda, API changes, releasing patched builds of Clojure as necessary because of it? Not necessarily dealbreakers IMO, but definitely a point of concern.

Implementation did not be begin before reviewing Joda's track recording on API compatibility. Answer: excellent.

> Finally, totally aside from any actual technical considerations, it seems to me that date/time support is a third rail of language / platform / library development, leaving no one who touches it unscarred. I'd hate to see Clojure's support get it wrong in some subtle way that will either cause breakage or pain down the road.

Yep.


Stu

Stuart Halloway

unread,
Nov 4, 2011, 3:26:53 PM11/4/11
to cloju...@googlegroups.com
> My personal reaction is that while Joda is great and building a lib on
> top of Joda seems highly useful, the idea of making Clojure core
> depend on Joda and return Joda objects seems undesirable.

Clojure will not return Joda objects (excep as an option) and will not depend on Joda (except at build time).

> 1) Clojure core then has to deal with dependencies on different
> versions of Joda, which may vary in its own Java-level dependency at a
> rate different than Clojure. That seems like a complecting of what
> Clojure core is. :) You also increase the size of a typical Clojure
> install (joda-time is ~500k). If you make it optional, that has it
> whole own set of complexities. My god man, the Maven work alone drove
> you to drink! :)

Our first thought (which may revive based on this feedback) was to provide two implementations

(1) default impl based on Java 6 XML date stuff
(2) autodiscover Joda and upgrade if present

No Joda requirement, but your parsing gets faster if you have Joda.

> 2) Now that we have Java literal constructors, you can fairly easily
> create Joda objects with 8601 strings:
> - #org.joda.time.DateTime("2004-12-13T21:39:45.618-08:00")
> - vs #@2004-12-13T21:39:45.618-08:00 (if I understand the proposal)

This is a non-starter for me, committing to Joda in a much deeper way than a literal that might be implemented with any library.

> 3) How often do you actually use literal instants in source? I use
> them a fair bit in writing test code, but usually you are reading
> timestamps as values from somewhere else, not putting this as literals
> in your code. Seems like the bigger win is really in having a
> standard way to *print* instants to make outputs cleaner. And there's
> no reason that can't be done in a lib.

I have ~zero interest in using instant literals in source. For me this is about CLJ vs. e.g. JSON as a data format, not source code.

> 4) If JSR 310 is added in JDK 8 then you have a reader that doesn't
> return the now-standard JDK objects which have all of the properties
> that are desirable about Joda. Would the reader change at that point?

Current proposal (contra my patch, which is wrong) is to return java.util.Date right now.

> 5) Maybe a moot point but an idle question: is it possible to get
> permission to use the portions of Joda that are useful (parser)
> without including all of Joda?

I would rather include the Java 6 stuff than have to bundle a special Joda.

> In all, I'm unconvinced that the added complexity is worth the value.
> (But having a core lib that wrapped Joda and provided everything else
> on that page would be great.)

Do the clarifications and options here provide a path that addresses this concern?

Stu

Sean Devlin

unread,
Nov 4, 2011, 3:27:25 PM11/4/11
to cloju...@googlegroups.com
java.util.Date doesn't handle timezone offsets.  I remember this was a problem when I tried to develop my own library, because it made it impossible to convert things to a time zone other than local.  That's why I had my own Date type.



Stu

Chris Granger

unread,
Nov 4, 2011, 3:36:50 PM11/4/11
to cloju...@googlegroups.com
Dates suck. Date operations also suck. I think it'd be great to have a standard lib for date functions...

But I don't really understand the motivation for adding a literal. Based on the notes page, it seems that the main motivation is storage, but I can already store time as millis, like I do in every other language/system. This seems like a very dangerous trend to me, with little overall benefit. We don't solve any usability problems by adding a whole host of syntax for people to learn.

Cheers,
Chris.

Chas Emerick

unread,
Nov 4, 2011, 3:49:04 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 3:18 PM, Stuart Halloway wrote:

>> That is, why date/time information, and not URLs, or IP addresses, or any number of other scalars that might be handy? For these other examples, I'd say strings are proper and reasonable because they don't presuppose any particular handling of the data involved — those decisions can be made locally, or in canonical, widely-used libraries, and so on. Dates strike me as being in the same category.
>
> Funny, I would like to do URLs next. I would call making these decisions locally the "JSON mistake" -- creating a cottage industry of conversions that could be avoided with a common approach.

I appreciate the sentiment, but the advantage of a cottage industry is that you can switch vendors if necessary. Towards something I'm more familiar with: I'm painfully aware of java.net.URL being insufficient in many cases and broken in others, so that also means inventing one's own representation or using someone else's that's better.

Making many such decisions and then hitching the language's syntax to them doesn't hold a lot of appeal IMO, and doesn't seem to provide much bang for the buck.

>> Are there any other languages that have date/time literals that we might be able to learn lessons from?
>
> I take the lesson to be stick to standards, i.e. ISO8601.

Sure, 8601 is great, but that only speaks to the string representation; the topic is the semantics and ongoing maintenance of syntactic changes to support that representation directly. Any known examples of other languages that have tread this path before we could learn from?

>> From a purely tactical perspective, seeing joda slip in (as an optional dependency, first of its kind I think!) gives me pause. Now, if I don't have joda around, there are cases where some code simply won't load.
>
> This is more about data than about code, but point taken.

Chris has a good point elsewhere: why not ship around millis as longs then? Seems better than strings, pretty unambiguously date data, and requires no changes anywhere.

Cheers,

- Chas

Alex Miller

unread,
Nov 4, 2011, 3:55:53 PM11/4/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 2:26 PM, Stuart Halloway
<stuart....@gmail.com> wrote:
>> My personal reaction is that while Joda is great and building a lib on
>> top of Joda seems highly useful, the idea of making Clojure core
>> depend on Joda and return Joda objects seems undesirable.
>
> Clojure will not return Joda objects (excep as an option) and will not depend on Joda (except at build time).

That sounds better. Based on below you expect to return
java.util.Date. Note that Date has only ms precision, not nano
precision and many XML and other iso dates you see in the wild DO
require support for nanos (for example, stuff coming back from Oracle,
XSD, etc). Similarly they can't encode timezones or offsets. This
all very narrowly scopes what you *can* read with full fidelity
(certainly less than the full ISO spec).

Would this be the first mutable object the reader can return? Is that
philosophically or practically objectionable? The Joda objects (and
JSR 310) objects are immutable of course.

>> 1) Clojure core then has to deal with dependencies on different
>> versions of Joda, which may vary in its own Java-level dependency at a
>> rate different than Clojure.  That seems like a complecting of what
>> Clojure core is. :)  You also increase the size of a typical Clojure
>> install (joda-time is ~500k).  If you make it optional, that has it
>> whole own set of complexities.  My god man, the Maven work alone drove
>> you to drink!  :)
>
> Our first thought (which may revive based on this feedback) was to provide two implementations
>
> (1) default impl based on Java 6 XML date stuff
> (2) autodiscover Joda and upgrade if present
>
> No Joda requirement, but your parsing gets faster if you have Joda.

Can you be specific? I assume you mean:
http://download.oracle.com/javase/6/docs/api/javax/xml/datatype/DatatypeFactory.html

The Java XML libs are designed to be replaced via the ESOM classloader
and there is a long history of issues with incompatibilities in that
mechanism. This seems like a source of possible subtle differences in
reader support varying across JVMs, OSes, app servers that use wacky
libs, etc. I've been bitten by all of those at one time or another in
Java XML land. While I have great trust in Joda and Stephen
Colebourne, it's still a dependency outside core control that is
*inside the reader*.

>> 2) Now that we have Java literal constructors, you can fairly easily
>> create Joda objects with 8601 strings:
>> - #org.joda.time.DateTime("2004-12-13T21:39:45.618-08:00")
>> - vs #@2004-12-13T21:39:45.618-08:00   (if I understand the proposal)
>
> This is a non-starter for me, committing to Joda in a much deeper way than a literal that might be implemented with any library.

Sure, but it's a choice in the hands of the user, not part of core.
However, I think the point is unimportant given #3.

>> 3) How often do you actually use literal instants in source?  I use
>> them a fair bit in writing test code, but usually you are reading
>> timestamps as values from somewhere else, not putting this as literals
>> in your code.  Seems like the bigger win is really in having a
>> standard way to *print* instants to make outputs cleaner.  And there's
>> no reason that can't be done in a lib.
>
> I have ~zero interest in using instant literals in source. For me this is about CLJ vs. e.g. JSON as a data format, not source code.

This was tremendously helpful in me in understanding your goals. I
understand that totally.

>> 4) If JSR 310 is added in JDK 8 then you have a reader that doesn't
>> return the now-standard JDK objects which have all of the properties
>> that are desirable about Joda.  Would the reader change at that point?
>
> Current proposal (contra my patch, which is wrong) is to return java.util.Date right now.
>
>> 5) Maybe a moot point but an idle question: is it possible to get
>> permission to use the portions of Joda that are useful (parser)
>> without including all of Joda?
>
> I would rather include the Java 6 stuff than have to bundle a special Joda.
>
>> In all, I'm unconvinced that the added complexity is worth the value.
>> (But having a core lib that wrapped Joda and provided everything else
>> on that page would be great.)
>
> Do the clarifications and options here provide a path that addresses this concern?

The reader comment greatly increases my perceived value. I still
think the costs are high.

Alex Miller

unread,
Nov 4, 2011, 4:06:44 PM11/4/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 2:18 PM, Stuart Halloway
<stuart....@gmail.com> wrote:
>> That is, why date/time information, and not URLs, or IP addresses, or any number of other scalars that might be handy?  For these other examples, I'd say strings are proper and reasonable because they don't presuppose any particular handling of the data involved — those decisions can be made locally, or in canonical, widely-used libraries, and so on.  Dates strike me as being in the same category.
>
> Funny, I would like to do URLs next. I would call making these decisions locally the "JSON mistake" -- creating a cottage industry of conversions that could be avoided with a common approach.

Given that we do sem web stuff, we deal with uris everywhere (you do
mean URI not URL right? And do you mean the version that Java
implements or the actual spec? And what about IRIs? Do you want to
validate?). I think if possible these are even more treacherous to do
"right" than date/time!

The way we deal with this for round-trippable data holding URIs is to
define a custom print-dup for URI that outputs a form that can be
evaled back into the same java.net.URI object: something like
(java.net.URI. "http://foo"). Kind of gross, but has worked fine in
practice for us.

Sean Corfield

unread,
Nov 4, 2011, 4:18:24 PM11/4/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 12:18 PM, Stuart Halloway
<stuart....@gmail.com> wrote:
> No. Rich has already corrected me, pointing out that the API should return java.util.Date. Raises question of an *option* to return Joda types for those who want it.

Then ClojureCLR will perforce return a different date type with a
(potentially / likely) different API and so will ClojureScript. That
means that code based on these literals would be non-portable which
doesn't seem a good idea to me.

Why not leave this to libraries - at the most pick an existing library
and promote it to contrib? Then each platform can at least provide a
compatible library API.

I guess I don't see a solid enough justification for making this a
language level feature in the ticket / design page...

(I certainly have nothing against Joda Time - I just took over
clj-time from Mark McGranahan because I need a Clojure 1.3.0
compatible version with additional API coverage at World Singles!)
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

Rich Hickey

unread,
Nov 4, 2011, 4:28:16 PM11/4/11
to cloju...@googlegroups.com

Since you are not to be a consumer of this by your own admission, what is the point of your arguing? Make a proposal, implement it or give it a rest please. Let's avoid another opinion-fest.

Everyone should note the proposal is one that makes the resulting datatype pluggable (your fn gets the ISO8601 string and can return whatever it wants). All that will be dictated by the feature is ISO8601. If you want to capture nanos, retain the offset etc, have at it. The default representation is the platform one, no other possible arguable choice and the msecs are readily obtained from it. j.u.Date sucks, no interesting conversation to be had about it.

This is a toe in the water experiment for support for more data types (like URIs, IPs, UUIDs etc) where the representation will inevitably differ across applications. The idea is that the reader support conveys the semantics (this is a point in time, not a string or number) and printed representation, without dictating the programmatic type. This will allow people to use Clojure data as a richer transport format without locking anyone (or even two ends of the same application) into particular libraries etc.

Rich

Kevin Downey

unread,
Nov 4, 2011, 4:31:00 PM11/4/11
to cloju...@googlegroups.com
"
using a bindable var for a read time setting is kind of a drag stuff like:

(binding [*instant-reader* some-other-instant-reader]
#@2011-11-04T14:42Z)

will not have the desired effect, so then what are the use cases for
the bindable var? should the repl bind it? should the compiler?
"

http://dev.clojure.org/jira/browse/CLJ-871?focusedCommentId=27196&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-27196

> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>

--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

Rich Hickey

unread,
Nov 4, 2011, 4:33:59 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 4:18 PM, Sean Corfield wrote:

> On Fri, Nov 4, 2011 at 12:18 PM, Stuart Halloway
> <stuart....@gmail.com> wrote:
>> No. Rich has already corrected me, pointing out that the API should return java.util.Date. Raises question of an *option* to return Joda types for those who want it.
>
> Then ClojureCLR will perforce return a different date type with a
> (potentially / likely) different API and so will ClojureScript. That
> means that code based on these literals would be non-portable which
> doesn't seem a good idea to me.
>

It most certainly does not. A lot of code that needs to read and transmit dates never needs to manipulate them.

> Why not leave this to libraries - at the most pick an existing library
> and promote it to contrib? Then each platform can at least provide a
> compatible library API.
>

It does leave it (the representation) to libraries. Nothing about this proposal stops anyone from writing portable libraries.

Rich

Kevin Downey

unread,
Nov 4, 2011, 4:39:32 PM11/4/11
to cloju...@googlegroups.com

I don follow how people can write portable libraries? is the compiler
going to bind *instant-reader* and your code can just set! it? seems
like a similar issue to reader macros.

Stu mentions using this for data mostly, which sort of side steps the
issue with the bindable var (turns into something similar to
*print-dup*) but it limits extensibility to the data case, with the
code case being second class.

Chas Emerick

unread,
Nov 4, 2011, 4:55:29 PM11/4/11
to cloju...@googlegroups.com
On Nov 4, 2011, at 4:28 PM, Rich Hickey wrote:

> Since you are not to be a consumer of this by your own admission, what is the point of your arguing? Make a proposal, implement it or give it a rest please. Let's avoid another opinion-fest.

I jumped in because this is, as you say, a toe in the water / camel's nose under the tent that has repercussions and future implications far beyond dates. Also, as I said at the outset, I was interested in learning what the motivations and benefits were; certainly didn't think I was arguing.

Anyways, this is design stuff, so it's got at least one foot in the realm of opinion and taste by definition. Saying "feedback wanted" won't get you what you want if only strictly technical review is desired/welcome.

Surprised,

- Chas

Stuart Sierra

unread,
Nov 4, 2011, 5:12:38 PM11/4/11
to Clojure Dev
This discussion, and the presence of *instant-reader* in Stu's patch,
suggest the beginnings of an extensible Reader. In the interest of
using the Clojure Reader as a data format, I'm all for it. But I feel
like this thread is a proxy for a discussion about WHERE those
extension points should live, and who should make the decisions.

Right now, the Reader has NO extension points. The lack of
extensibility is deliberate, to ensure code portability. But with a
rebindable parsing function, you've almost let reader macros in the
back door. It's not hard to imagine people abusing *instant-reader* to
parse things that are not dates. (Maybe the reader would enforce
ISO-8601 syntax before calling *instant-reader*, but one could still
abuse that format.)

The addition of new literals won't have much impact on the programmer
writing Clojure code. But it increases the utility of the Clojure
Reader as the basis for a rich, portable data format, something I
support.

I also detect an undercurrent of resentment here: Rich Hickey (with
Stuart Halloway) gets to define reader macros, but nobody else does.
Yes, as the language designer, that is his privilege. But there's an
advantage to that kind of control too: with one person making all the
critical decisions, the rest of us can be assured of compatibility.

What this feature suggests is the carefully-curated addition of new
types to the Reader. These new types should have precise semantics
specified outside of Clojure: ISO time, UUIDs, and URIs would all
qualify. The semantics of what these literals mean can be independent
of the code/types used to represent them on any particular platform.

On the other hand, this reminds me of XML Schema datatypes [1],
something I'm not eager to get back into. But Clojure isn't designed
by a committee, so I think we're safe.

- Stuart Sierra
Clojure/core: http://clojure.com

[1] http://www.w3.org/TR/xmlschema-2/#built-in-datatypes

Stuart Halloway

unread,
Nov 4, 2011, 5:16:39 PM11/4/11
to cloju...@googlegroups.com
> Would you like my datable protocol stuff?
>
> Sean

It or something like it, in a contrib. Will you be the Conj?

Stu

Stuart Halloway

unread,
Nov 4, 2011, 5:18:54 PM11/4/11
to cloju...@googlegroups.com
>> Funny, I would like to do URLs next. I would call making these decisions locally the "JSON mistake" -- creating a cottage industry of conversions that could be avoided with a common approach.
>
> I appreciate the sentiment, but the advantage of a cottage industry is that you can switch vendors if necessary. Towards something I'm more familiar with: I'm painfully aware of java.net.URL being insufficient in many cases and broken in others, so that also means inventing one's own representation or using someone else's that's better.
>
> Making many such decisions and then hitching the language's syntax to them doesn't hold a lot of appeal IMO, and doesn't seem to provide much bang for the buck.


Chas,

This is the one place where I think we are not on the same page. If there is a standard format, you can switch vendors of that format. If there isn't, you can't do anything but switch representation end-to-end to target the next vendors bespoke thing.

Stu

Kevin Downey

unread,
Nov 4, 2011, 5:19:43 PM11/4/11
to cloju...@googlegroups.com
why are there no examples of how this would be used? my guess is it is

(binding [*instant-reader* …] (read something))

but it would be nice to see intended uses a long side the code changes

> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>

--

Stuart Halloway

unread,
Nov 4, 2011, 5:19:44 PM11/4/11
to cloju...@googlegroups.com

Chas,

For my part: thanks for the prompt and well-articulated feedback. Look forward to seeing you next week!

Stu

Rich Hickey

unread,
Nov 4, 2011, 5:20:03 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 2:51 PM, Chas Emerick wrote:

> I've been absent for the prior discussions, and time and date stuff is not really in my wheelhouse, so perhaps I'm just blind to the pressing need. But, I'd like to suggest that perhaps this is a data type that doesn't necessarily need to be elevated to be supported in the reader (and potentially be convinced otherwise, if anyone's willing to take a whack).
>
> That is, why date/time information, and not URLs, or IP addresses, or any number of other scalars that might be handy?

Why not? You have to start somewhere, and there was a proposal submitted in this area.

> For these other examples, I'd say strings are proper and reasonable because they don't presuppose any particular handling of the data involved — those decisions can be made locally, or in canonical, widely-used libraries, and so on.

"Proper" and "reasonable" - more rhetoric from you. Stop stealing the good adjectives for your opinion.

A string that represents a date/uri/ip etc fails to convey that it is doing so. All this proposal does is say - this string represents a point in time.

> Are there any other languages that have date/time literals that we might be able to learn lessons from?

Are you volunteering to do that research? Or just implying we haven't?

>
> From a purely tactical perspective, seeing joda slip in (as an optional dependency, first of its kind I think!) gives me pause. Now, if I don't have joda around, there are cases where some code simply won't load.

No code that loads now will fail to load. New features, new requirements. If you know of solid ISO8601 code we can include in Clojure speak up.

> Also, now I need to go learn the Joda API if I'm going to work with date data in Clojure?

No.

> Finally, totally aside from any actual technical considerations, it seems to me that date/time support is a third rail of language / platform / library development, leaving no one who touches it unscarred. I'd hate to see Clojure's support get it wrong in some subtle way that will either cause breakage or pain down the road.

Thus the minimal nature of the proposed support.

> As I said, date/time stuff is not generally my bag, so I may be suffering from various misconceptions and such.
>


But you are quick with opinions nonetheless.

Rich

Rich Hickey

unread,
Nov 4, 2011, 5:20:06 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 3:49 PM, Chas Emerick wrote:

>
> On Nov 4, 2011, at 3:18 PM, Stuart Halloway wrote:
>
>>> That is, why date/time information, and not URLs, or IP addresses, or any number of other scalars that might be handy? For these other examples, I'd say strings are proper and reasonable because they don't presuppose any particular handling of the data involved — those decisions can be made locally, or in canonical, widely-used libraries, and so on. Dates strike me as being in the same category.
>>
>> Funny, I would like to do URLs next. I would call making these decisions locally the "JSON mistake" -- creating a cottage industry of conversions that could be avoided with a common approach.
>
> I appreciate the sentiment, but the advantage of a cottage industry is that you can switch vendors if necessary. Towards something I'm more familiar with: I'm painfully aware of java.net.URL being insufficient in many cases and broken in others, so that also means inventing one's own representation or using someone else's that's better.
>

The representation is pluggable.

> Making many such decisions and then hitching the language's syntax to them doesn't hold a lot of appeal IMO, and doesn't seem to provide much bang for the buck.
>

Who's buck is it anyway?

>>> Are there any other languages that have date/time literals that we might be able to learn lessons from?
>>
>> I take the lesson to be stick to standards, i.e. ISO8601.
>
> Sure, 8601 is great, but that only speaks to the string representation; the topic is the semantics and ongoing maintenance of syntactic changes to support that representation directly.

Huh? Nothing more is need to support that string format than reading and printing it.

> Any known examples of other languages that have tread this path before we could learn from?

What is the point of this question? Are you asking someone else to do the research, or volunteering?

>>> From a purely tactical perspective, seeing joda slip in (as an optional dependency, first of its kind I think!) gives me pause. Now, if I don't have joda around, there are cases where some code simply won't load.
>>
>> This is more about data than about code, but point taken.
>
> Chris has a good point elsewhere: why not ship around millis as longs then? Seems better than strings, pretty unambiguously date data, and requires no changes anywhere.

There are things conveyed in ISO8601 that are lost when longs. Also, longs are not readable by humans as ISO8601 is.

Why don't you consider the problem a bit more before wasting our time with these knee-jerk questions and bike-shedding? This is not the IRC.

Rich

Stuart Halloway

unread,
Nov 4, 2011, 5:24:01 PM11/4/11
to cloju...@googlegroups.com
> This discussion, and the presence of *instant-reader* in Stu's patch,
> suggest the beginnings of an extensible Reader. In the interest of
> using the Clojure Reader as a data format, I'm all for it. But I feel
> like this thread is a proxy for a discussion about WHERE those
> extension points should live, and who should make the decisions.
>
> Right now, the Reader has NO extension points.

This isn't true. #= is a gaping hole.

> The lack of
> extensibility is deliberate, to ensure code portability. But with a
> rebindable parsing function, you've almost let reader macros in the
> back door. It's not hard to imagine people abusing *instant-reader* to
> parse things that are not dates. (Maybe the reader would enforce
> ISO-8601 syntax before calling *instant-reader*, but one could still
> abuse that format.)

Surely better than #=

> The addition of new literals won't have much impact on the programmer
> writing Clojure code. But it increases the utility of the Clojure
> Reader as the basis for a rich, portable data format, something I
> support.
>
> I also detect an undercurrent of resentment here: Rich Hickey (with
> Stuart Halloway) gets to define reader macros, but nobody else does.
> Yes, as the language designer, that is his privilege. But there's an
> advantage to that kind of control too: with one person making all the
> critical decisions, the rest of us can be assured of compatibility.

I am happy to offer my opinion, but AFAICT I am the most-shot-down guy on this list. Which is fine. I will keep coming back, hopefully with increasingly useful opinions.

Stu

Rich Hickey

unread,
Nov 4, 2011, 5:24:01 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 5:19 PM, Kevin Downey wrote:

> why are there no examples of how this would be used? my guess is it is
>
> (binding [*instant-reader* …] (read something))
>
> but it would be nice to see intended uses a long side the code changes
>

The pluggability is still under consideration, but most simply the handler can be set!. Making it bindable lets you dynamically plug something specific in when required, but will probably not be used other than that.

Rich

Rich Hickey

unread,
Nov 4, 2011, 5:33:29 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 3:55 PM, Alex Miller wrote:

> On Fri, Nov 4, 2011 at 2:26 PM, Stuart Halloway
> <stuart....@gmail.com> wrote:
>>> My personal reaction is that while Joda is great and building a lib on
>>> top of Joda seems highly useful, the idea of making Clojure core
>>> depend on Joda and return Joda objects seems undesirable.
>>
>> Clojure will not return Joda objects (excep as an option) and will not depend on Joda (except at build time).
>
> That sounds better. Based on below you expect to return
> java.util.Date. Note that Date has only ms precision, not nano
> precision and many XML and other iso dates you see in the wild DO
> require support for nanos (for example, stuff coming back from Oracle,
> XSD, etc). Similarly they can't encode timezones or offsets. This
> all very narrowly scopes what you *can* read with full fidelity
> (certainly less than the full ISO spec).

It only limits what the default representation provides. You can plug in to retain any of that.

>
> Would this be the first mutable object the reader can return? Is that
> philosophically or practically objectionable?

It's quite objectionable. This should in no way be construed as advocacy for j.u.Date, a poster child of insufficiency :) But, j.u.Date still retains the semantics and every lib can convert from it. We would certainly encourage people to use Joda, and want to swap in a handler that returns that when you choose to use it, but can't force that.

> The Joda objects (and
> JSR 310) objects are immutable of course.
>

Yes, much better.

>>> 1) Clojure core then has to deal with dependencies on different
>>> versions of Joda, which may vary in its own Java-level dependency at a
>>> rate different than Clojure. That seems like a complecting of what
>>> Clojure core is. :) You also increase the size of a typical Clojure
>>> install (joda-time is ~500k). If you make it optional, that has it
>>> whole own set of complexities. My god man, the Maven work alone drove
>>> you to drink! :)
>>
>> Our first thought (which may revive based on this feedback) was to provide two implementations
>>
>> (1) default impl based on Java 6 XML date stuff
>> (2) autodiscover Joda and upgrade if present
>>
>> No Joda requirement, but your parsing gets faster if you have Joda.
>
> Can you be specific? I assume you mean:
> http://download.oracle.com/javase/6/docs/api/javax/xml/datatype/DatatypeFactory.html
>
> The Java XML libs are designed to be replaced via the ESOM classloader
> and there is a long history of issues with incompatibilities in that
> mechanism. This seems like a source of possible subtle differences in
> reader support varying across JVMs, OSes, app servers that use wacky
> libs, etc. I've been bitten by all of those at one time or another in
> Java XML land. While I have great trust in Joda and Stephen
> Colebourne, it's still a dependency outside core control that is
> *inside the reader*.

Other suggestions welcome here. It seems like all experts in this area have applied themselves to Joda's implementation.

>
>>> 2) Now that we have Java literal constructors, you can fairly easily
>>> create Joda objects with 8601 strings:
>>> - #org.joda.time.DateTime("2004-12-13T21:39:45.618-08:00")
>>> - vs #@2004-12-13T21:39:45.618-08:00 (if I understand the proposal)
>>
>> This is a non-starter for me, committing to Joda in a much deeper way than a literal that might be implemented with any library.
>
> Sure, but it's a choice in the hands of the user, not part of core.
> However, I think the point is unimportant given #3.
>

The key is the marriage to a representation here, actually making the data less portable.

>
>>> 4) If JSR 310 is added in JDK 8 then you have a reader that doesn't
>>> return the now-standard JDK objects which have all of the properties
>>> that are desirable about Joda. Would the reader change at that point?
>>
>> Current proposal (contra my patch, which is wrong) is to return java.util.Date right now.
>>
>>> 5) Maybe a moot point but an idle question: is it possible to get
>>> permission to use the portions of Joda that are useful (parser)
>>> without including all of Joda?
>>
>> I would rather include the Java 6 stuff than have to bundle a special Joda.
>>
>>> In all, I'm unconvinced that the added complexity is worth the value.
>>> (But having a core lib that wrapped Joda and provided everything else
>>> on that page would be great.)
>>
>> Do the clarifications and options here provide a path that addresses this concern?
>
> The reader comment greatly increases my perceived value. I still
> think the costs are high.

Which costs? People that don't use this pay nothing.

Rich

Rich Hickey

unread,
Nov 4, 2011, 5:47:31 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 5:12 PM, Stuart Sierra wrote:

> This discussion, and the presence of *instant-reader* in Stu's patch,
> suggest the beginnings of an extensible Reader. In the interest of
> using the Clojure Reader as a data format, I'm all for it. But I feel
> like this thread is a proxy for a discussion about WHERE those
> extension points should live, and who should make the decisions.
>
> Right now, the Reader has NO extension points. The lack of
> extensibility is deliberate, to ensure code portability. But with a
> rebindable parsing function, you've almost let reader macros in the
> back door. It's not hard to imagine people abusing *instant-reader* to
> parse things that are not dates. (Maybe the reader would enforce
> ISO-8601 syntax before calling *instant-reader*, but one could still
> abuse that format.)
>

One should presume that code that enforces it will be added at any point ;)

> The addition of new literals won't have much impact on the programmer
> writing Clojure code. But it increases the utility of the Clojure
> Reader as the basis for a rich, portable data format, something I
> support.
>
> I also detect an undercurrent of resentment here: Rich Hickey (with
> Stuart Halloway) gets to define reader macros, but nobody else does.
> Yes, as the language designer, that is his privilege. But there's an
> advantage to that kind of control too: with one person making all the
> critical decisions, the rest of us can be assured of compatibility.
>

Any such resentment would be bewildering. Is that what we should expect for picking up a user proposal and trying to deliver on it?

> What this feature suggests is the carefully-curated addition of new
> types to the Reader. These new types should have precise semantics
> specified outside of Clojure: ISO time, UUIDs, and URIs would all
> qualify. The semantics of what these literals mean can be independent
> of the code/types used to represent them on any particular platform.
>

Precisely. And it is in this area I have the most interest in feedback. I don't want to dictate programmatic types, due to the lack of consensus (many people opt out of the candidates in the Java libs). But an open reader, or constructor-driven reader expressions, don't deliver on portable data conveyance. So, this semantics + printed representation, independent of programmatic representation, seems promising. While it doesn't do many things (like provide portable libraries around the types), it doesn't preclude them, and by taking that out of scope avoids taking on too much and doing a poor job, especially when, e.g. Joda is perfectly good.

> On the other hand, this reminds me of XML Schema datatypes [1],
> something I'm not eager to get back into. But Clojure isn't designed
> by a committee, so I think we're safe.

Yes, well, more stuff implies more stuff unfortunately.

Rich

Alex Miller

unread,
Nov 4, 2011, 5:49:25 PM11/4/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 3:28 PM, Rich Hickey <richh...@gmail.com> wrote:
> Everyone should note the proposal is one that makes the resulting datatype pluggable (your fn gets the ISO8601
> string and can return whatever it wants). All that will be dictated by the feature is ISO8601. If you want to capture
> nanos, retain the offset etc, have at it. The default representation is the platform one, no other possible arguable
> choice and the msecs are readily obtained from it. j.u.Date sucks, no interesting conversation to be had about it.

I understand the desire to read and print a standard representation of
(semantically) a point in time. Broader data reader semantics could
be very useful in using Clojure data in more places such as
configuration data.

I don't understand how the intermediate data representation can vary
in it's ability to carry the information in the lexical form. That
is, if there are multiple implementations (juDate, Joda's DateTime,
etc) and if Joda's DateTime representation prints in ISO8601 reader
format including say timezone, then is read by another system using
juDate, the object produced has *the wrong point in time* and can't
round-trip back to the same printed form. This seems like a problem
to me as it decreases the portability of the lexical form.

My second question is whether there will be a standard protocol to
interrogate the abstract representation of the ISO8601 in a
type-agnostic way. For instance, to get the "year" of the abstract
point in time. Without that, it seems hard to avoid tying yourself to
a particular concrete class.

Alex Miller

unread,
Nov 4, 2011, 5:55:19 PM11/4/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 4:33 PM, Rich Hickey <richh...@gmail.com> wrote:
> Other suggestions welcome here. It seems like all experts in this area have applied themselves to Joda's implementation.

Actually, the JSR 310 group has all of the Joda experts plus several
additional folks (key people in the area of database and XSD
integration in particular) and they have departed from Joda at several
points to address issues in Joda. I would suggest another alternative
would be to tie into JSR 310 as an optional lib and when JSR 310
becomes part of the JDK, Clojure has a natural migration path back to
the platform.

Sean Corfield

unread,
Nov 4, 2011, 6:03:28 PM11/4/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 1:28 PM, Rich Hickey <richh...@gmail.com> wrote:
> Everyone should note the proposal is one that makes the resulting datatype pluggable (your fn gets the ISO8601 string and can return whatever it wants).

OK, that satisfies me. I can see the option then for 3rd party
time/date libraries to implement a reader hook so users can choose to
rely on, say, clj-time (wrapped around Joda Time) for not only their
basic date manipulations but also the reader / printer implementation,
yes?

In the Old Notes section, it says "A collection of very common
comparators and predicates." - is the updated proposal just the
literal format / reader hook and no longer a proposal to incorporate
such comparators / predicates? If so, again, I'm satisfied (since I'd
prefer to defer to 3rd party libraries on that).

> This is a toe in the water experiment for support for more data types (like URIs, IPs, UUIDs etc) where the representation will inevitably differ across applications.

Interesting. Any thoughts yet on how this might be generalized?
#@somestring is fine for instant but as the number of data types
grows, is there likely to be a general, extensible reader syntax that
allows consistent hooks for new literal types? Along the lines of
#[instant@2011-11-04T14:42Z] for something bound to *instant-reader*
and #[foo@something] read by *foo-reader*...?

Stuart Halloway

unread,
Nov 4, 2011, 6:07:08 PM11/4/11
to cloju...@googlegroups.com
I just updated http://dev.clojure.org/pages/viewpage.action?pageId=950382 with corrections and to include a Q &A section that covers some of todays' thread, if anyone wants a summary without having to wade through all this.

Stu

> Instant literal for Clojure:
> http://dev.clojure.org/jira/browse/CLJ-871 and
> http://dev.clojure.org/pages/viewpage.action?pageId=950382
>
> This is work in progress. I will add tests and update the patch for
> screening based on/depending on feedback here.
>
> If this flies, will be looking for volunteers to do equivalent in
> ClojureScript and ClojureCLR.
>
> Stu

Stuart Halloway

unread,
Nov 4, 2011, 6:10:51 PM11/4/11
to cloju...@googlegroups.com
> I don't understand how the intermediate data representation can vary
> in it's ability to carry the information in the lexical form. That
> is, if there are multiple implementations (juDate, Joda's DateTime,
> etc) and if Joda's DateTime representation prints in ISO8601 reader
> format including say timezone, then is read by another system using
> juDate, the object produced has *the wrong point in time* and can't
> round-trip back to the same printed form. This seems like a problem
> to me as it decreases the portability of the lexical form.

I think this divides into two cases:

(1) You choose a local rep that doesn't have enough information to cover ISO8601 semantics, e.g. loses submillis. Then you lose information, sorry. Get a better class.

(2) You choose a local rep that (maybe) has extra non-instant stuff in it, like timezone. This stuff won't be part of the print/read contract, so no portability problem.

If #2 cannot be made true that's a problem, but I don't see any implementation challenge there. Maybe I am missing something.

Stu

Rich Hickey

unread,
Nov 4, 2011, 8:21:17 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 5:49 PM, Alex Miller wrote:

> On Fri, Nov 4, 2011 at 3:28 PM, Rich Hickey <richh...@gmail.com> wrote:
>> Everyone should note the proposal is one that makes the resulting datatype pluggable (your fn gets the ISO8601
>> string and can return whatever it wants). All that will be dictated by the feature is ISO8601. If you want to capture
>> nanos, retain the offset etc, have at it. The default representation is the platform one, no other possible arguable
>> choice and the msecs are readily obtained from it. j.u.Date sucks, no interesting conversation to be had about it.
>
> I understand the desire to read and print a standard representation of
> (semantically) a point in time. Broader data reader semantics could
> be very useful in using Clojure data in more places such as
> configuration data.
>
> I don't understand how the intermediate data representation can vary
> in it's ability to carry the information in the lexical form.
> That
> is, if there are multiple implementations (juDate, Joda's DateTime,
> etc) and if Joda's DateTime representation prints in ISO8601 reader
> format including say timezone, then is read by another system using
> juDate, the object produced has *the wrong point in time* and can't
> round-trip back to the same printed form.

Those are 2 separate things. It will not have the "wrong point in time". 4+ 2 = 6. But it may not convey everything the original string did. The intention is you can run all code (including libraries not under your control) with the date representation of your choice, and thus can choose one with the properties you desire.

> This seems like a problem
> to me as it decreases the portability of the lexical form.
>

People who care will choose more precise representations.

> My second question is whether there will be a standard protocol to
> interrogate the abstract representation of the ISO8601 in a
> type-agnostic way. For instance, to get the "year" of the abstract
> point in time. Without that, it seems hard to avoid tying yourself to
> a particular concrete class.

It is out of scope to assist with that. Protocol based libraries might help. Many people won't care one bit about that orthogonal concern. Remember, that problem exists *right now*, in the absence of this proposal. People are choosing concrete representations incompatible with others, marrying them etc.

Everyone considering this proposal needs to compare it to *nothing*, which is what we have now. It will not be a panacea, but in spite of that can have utility. Trying to cover every need will lead us back to nothing, since there are no widely acceptable solutions here.

Rich

Rich Hickey

unread,
Nov 4, 2011, 8:22:16 PM11/4/11
to cloju...@googlegroups.com

Is that code usable right now? (Capability and license-wise?)

Rich

Rich Hickey

unread,
Nov 4, 2011, 8:27:36 PM11/4/11
to cloju...@googlegroups.com

On Nov 4, 2011, at 6:03 PM, Sean Corfield wrote:

> On Fri, Nov 4, 2011 at 1:28 PM, Rich Hickey <richh...@gmail.com> wrote:
>> Everyone should note the proposal is one that makes the resulting datatype pluggable (your fn gets the ISO8601 string and can return whatever it wants).
>
> OK, that satisfies me. I can see the option then for 3rd party
> time/date libraries to implement a reader hook so users can choose to
> rely on, say, clj-time (wrapped around Joda Time) for not only their
> basic date manipulations but also the reader / printer implementation,
> yes?
>
> In the Old Notes section, it says "A collection of very common
> comparators and predicates." - is the updated proposal just the
> literal format / reader hook and no longer a proposal to incorporate
> such comparators / predicates? If so, again, I'm satisfied (since I'd
> prefer to defer to 3rd party libraries on that).

The old notes section is dead. It was an over-reaching proposal without a problem statement.

>
>> This is a toe in the water experiment for support for more data types (like URIs, IPs, UUIDs etc) where the representation will inevitably differ across applications.
>
> Interesting. Any thoughts yet on how this might be generalized?

Yes. For some reason I find the prospect of starting to talk about that here right now exhausting :)

> #@somestring is fine for instant but as the number of data types
> grows, is there likely to be a general, extensible reader syntax that
> allows consistent hooks for new literal types? Along the lines of
> #[instant@2011-11-04T14:42Z] for something bound to *instant-reader*
> and #[foo@something] read by *foo-reader*...?
>

I have ideas for comprehensive (standard and open, both conflict-free) reader data extensibility along those lines. It is partially predicated on the same programmatic-representation independence as we are now exploring for dates.

Rich

Sean Corfield

unread,
Nov 4, 2011, 8:42:09 PM11/4/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 5:27 PM, Rich Hickey <richh...@gmail.com> wrote:
> The old notes section is dead. It was an over-reaching proposal without a problem statement.

Thanx for that clarification.

>> Interesting. Any thoughts yet on how this might be generalized?
> Yes. For some reason I find the prospect of starting to talk about that here right now exhausting :)

Understood. Maybe I get some _discussion-free_ time at the Conj to
hear those thoughts :)

> I have ideas for comprehensive (standard and open, both conflict-free) reader data extensibility along those lines. It is partially predicated on the same programmatic-representation independence as we are now exploring for dates.

Cool.

Rich Hickey

unread,
Nov 4, 2011, 8:50:43 PM11/4/11
to cloju...@googlegroups.com

Also, do you know if j.u.Date survives in JSR 310? And where is the code these days? The java.net links one finds are mostly dead. If anyone is up to speed on JSR 310, a status summary would be welcome.

Rich

Luc Prefontaine

unread,
Nov 4, 2011, 9:27:29 PM11/4/11
to cloju...@googlegroups.com
Hi all,

I read the whole thread before issuing the following feed back:

a) If I look back at the date manipulations I had to do in many projects, they seldom involve
operations like adding/subtracting dates. There's much more representation/display involved code than
anything else or setting timers. Not necessarily in the user local time zone btwy. Look at airline reservation systems...

b) Milliseconds cannot do the job for a very simple reason, the year 2038 problem which will
force designers to adopt solutions that will impact applications. This will not
be implemented on a dime so you can expect some turmoil here trying to exchange time values using
this representation well before 2038. For those who think that 2038 is far away, I thought the same about 2000
in the mid 1980s...

c) How you display dates is a local decision. Choose whatever is appropriate to do so. That may well depend on your
context, here we have to exchange values with medical system using the HL7 format. Elsewhere we are stuck
with Java.sql.Date and co. We would love to have a unified representation in Clojure and stop coping with
the Date class until it's time to do so (like db persistence time)

d) Having a common representation to exchange date/time values between the various Clojure implementations
is IMHO of immediate value.

e) No need to fiddle now with how this representation can be manipulated. That can be thought through over time
and confined to libs instead of having the underlying representation (java.util.Date and co) spreading
everywhere in our apps.

f) It would impact how other libraries present date/time values to callers in a beneficial way.
As an example, when I read a database table, most of the time I do not want to end up with a Java Date object...
but presently I have no choice.

I vote for UTC all the way. ISO 8601 seems fine to me:

a) ISO has been there for a long time
b) we are only talking about using a representation
c) they will deal with the 2038 tsunami
d) software will most likely stick at least remotely to it.

I do not have yet anything intelligent to say about other entities like URLs. So I'll stop here :))))

Luc

b) I vote for ISO8601 representation

On Fri, 4 Nov 2011 14:04:31 -0400
Stuart Halloway <stuart....@gmail.com> wrote:

> Instant literal for Clojure:
> http://dev.clojure.org/jira/browse/CLJ-871 and
> http://dev.clojure.org/pages/viewpage.action?pageId=950382
>
> This is work in progress. I will add tests and update the patch for
> screening based on/depending on feedback here.
>
> If this flies, will be looking for volunteers to do equivalent in
> ClojureScript and ClojureCLR.
>
> Stu
>

--
Luc P.

================
The rabid Muppet

Luc Prefontaine

unread,
Nov 4, 2011, 9:30:34 PM11/4/11
to cloju...@googlegroups.com

No, I did not imply with my last remark that my feedback on the date literal
thing was intelligent or even well structured :)

Luc Prefontaine

unread,
Nov 4, 2011, 10:05:16 PM11/4/11
to cloju...@googlegroups.com
Forgot to add that milliseconds representation is not cross platform consistent even
if it's represented using 64 bits long.

On Fri, 4 Nov 2011 21:27:29 -0400
Luc Prefontaine <lprefo...@softaddicts.ca> wrote:

Hugo Duncan

unread,
Nov 4, 2011, 10:07:48 PM11/4/11
to cloju...@googlegroups.com
On Fri, 04 Nov 2011 20:27:36 -0400, Rich Hickey <richh...@gmail.com>
wrote:

> On Nov 4, 2011, at 6:03 PM, Sean Corfield wrote:
>
>> On Fri, Nov 4, 2011 at 1:28 PM, Rich Hickey <richh...@gmail.com>
>> wrote:
>>> Everyone should note the proposal is one that makes the resulting
>>> datatype pluggable (your fn gets the ISO8601 string and can return
>>> whatever it wants).

> I have ideas for comprehensive (standard and open, both conflict-free)

> reader data extensibility along those lines. It is partially predicated
> on the same programmatic-representation independence as we are now
> exploring for dates.

I would be interested in hearing about the choice of plugin interface
here. It seems to me that the problem is composed of defining a string
format, a parser for that format that converts to a (plain) data
structure, and a choice of interface (or protocol) to impose on the data
structure via a concrete type.

The current proposal is to allow a plugin to convert from string to an
instance of a type.

An alternative might be to standardise a parser and a map based data
format that it produces, allowing a plugin to convert from standard data
structure to final type, or removing the need for a plugin at all (though
I suppose that might require a defrecord to make round-tripping possible).

This would standardise the parsing of the string format, removing the
liklihood of different plugins parsing things differently, no doubt at
some performance cost.

I can see this might be possible for some data types, but also see issues
for things like URI's with custom schemes (i.e. with open formats).

--
Hugo Duncan

Alex Miller

unread,
Nov 4, 2011, 10:14:16 PM11/4/11
to cloju...@googlegroups.com
310 is now here:

http://sourceforge.net/apps/mediawiki/threeten/index.php?title=ThreeTen

My understanding is that the large core is stable but there are certain areas that are incomplete.  I think the parts needed for this discussion are stable but I haven't played with the code in quite a while.  Because the code is for a jsr the ip situation is quite complicated.  I think Stephen understands those issues better than anyone I know though and could answer them best.

Alex Miller

unread,
Nov 4, 2011, 10:22:15 PM11/4/11
to cloju...@googlegroups.com
Also, re Date, yes it survives for compatibility and because we never take our Java toys away.  But as with File and jsr 203 in java 7 it is retrofitted into new interfaces and conversions are available.  I would be happy to be more specific when I get off a phone and onto a computer.


On Friday, November 4, 2011, Rich Hickey <richh...@gmail.com> wrote:
>

Alex Miller

unread,
Nov 5, 2011, 12:32:30 AM11/5/11
to cloju...@googlegroups.com
Since you're talking solely about UTC and no offsets or timezones,
simple usages of 310 for parsing/formatting look like:

user=> (import 'javax.time.calendar.LocalDateTime)
javax.time.calendar.LocalDateTime
user=> (def t (LocalDateTime/parse "2007-12-03T10:15:30.123456789"))
#'user/t
user=> t
#<LocalDateTime 2007-12-03T10:15:30.123456789>
user=> (str t)
"2007-12-03T10:15:30.123456789"

Or perhaps you really want an Instant (the prior LocalDateTime example
represents the "human-scale" timestamp, Instant really represents a
"machine" timeline of UTC with a particular means of dealing with
leap-seconds, etc etc. In that case you want:

user=> (def i (Instant/parse "2007-12-03T10:15:30.123456789Z"))
UnsupportedOperationException javax.time.Instant.parse (Instant.java:323)

but as you can see that method is not yet implemented (there is a TODO
in the code and this is listed in the todo file). I can't say I
understand why exactly given that this is virtually identical to the
LocalDateTime parser above.

All of the pre-built formatters are defined in
javax.time.calendar.format.DateTimeFormatters class (LocalDateTime's
formatter is DateTimeFormatters.isoLocalDateTime().

java.util.Date is now an InstantProvider and has a toInstant() method.
You can also create a Date with an InstantProvider.

Ben Smith-Mannschott

unread,
Nov 5, 2011, 3:48:58 AM11/5/11
to cloju...@googlegroups.com
On Sat, Nov 5, 2011 at 02:27, Luc Prefontaine
<lprefo...@softaddicts.ca> wrote:
> b) Milliseconds cannot do the job for a very simple reason, the year 2038 problem which will
>   force designers to adopt solutions that will impact applications. This will not
>   be implemented on a dime so you can expect some turmoil here trying to exchange time values using
>   this representation well before 2038. For those who think that 2038 is far away, I thought the same about 2000
>   in the mid 1980s...

This is not the problem with milliseconds as they are used in Java.

Java uses a signed 64-bit integer (long) to store milliseconds since
the epoch. A 64-bit signed millisecond counter is enough to get us to
about 292'471'208 years beyond the epoch.

The 2038 problem refers to the wrap-around caused by the use of a
signed 32-bit integer to store the number of *seconds* since the
epoch. This is an issue for Oracle (or whoever provides the JVM). I
fail to see this causing problems for software running on the JVM.

// ben

Luc Prefontaine

unread,
Nov 5, 2011, 10:10:16 AM11/5/11
to cloju...@googlegroups.com

This is not only a Clojure/Java centric issue, when I pass 1 to an external system,
the semantic is clear, it's still 1 outside Clojure. It could suffer from limited
representation by the other party (byte,short or int) but it still 1.

Epoch times are not meeting this criteria:

http://www.epochconverter.com/

The time origin of U*X and several other languages is the same has Java, Jan 1, 1970.
This representation is also used in data in many legacy systems. But it's not the same
everywhere.

More against this, if Clojure is to implement a data type, it should be consistent across
implementations when in comes to representation. CLR does not have the same epoch representation
even if it will cross that 2038 wall without a scratch. A bit odd to have the same data type
on two implementations with incompatible representations.

Not adhering to some universal format that conveys the semantic properly for
a type as common as date time is a mistake.

And now the impacts of the most important human factor, procrastination....

If milliseconds were to be the official time representation in
Clojure, how do you prevent people from operating directly with 32 bit values with external systems that
have the 2038 wall coming by ?

Dividing the time by 1000 to obtain a compatible value is too easy if your time origin is the same (Jan 1, 1970) which is
fairly common).
That would only last until the deadline or until a system decides to slip the time window before the 2038 deadline by
changing it's time origin by lets say 20 years.... The milliseconds meaning has just changed for this system.
Don't laugh, I have seen this in some apps to get some relief from the year 2000 wall. That's a recurrent pattern,
procrastinate until it's too late and then find a way out, any way.

The epoch representation is subject to interpretation and changes in the not so far future,
if you rely on this in Clojure, you put the burden of dealing with the issue on the programmer's shoulders.

Of course if you code in Clojure never dealing with the outside world, it
may look sufficient. But life is not like that, legacy stuff is out there and you have to interact with it.

At least with a UTC format you can send it to the remote system/app and get it to adapt it to its own constraints.
You are not in the loop of date/time conversion and if it ever changes, it's hidden in that system.

If you ever decide to jump in these conversion details, at least you will have to think about the issue throughly,
it will not be as simple as a dive by 1000 answer...

That was the long answer :)

Luc

--

Ben Smith-Mannschott

unread,
Nov 5, 2011, 2:16:09 PM11/5/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 19:04, Stuart Halloway <stuart....@gmail.com> wrote:
> Instant literal for Clojure:
> http://dev.clojure.org/jira/browse/CLJ-871 and
> http://dev.clojure.org/pages/viewpage.action?pageId=950382
>
> This is work in progress. I will add tests and update the patch for
> screening based on/depending on feedback here.
>
> If this flies, will be looking for volunteers to do equivalent in
> ClojureScript and ClojureCLR.

I think it's a good choice to require UTC, so as "not to get into the
localization business". I'm less sure about returning java.util.Date
since it's only really intended for local time, broken, and mutable. I
understand the reluctance to return a type here from Joda-Time by
default since that would force a dependency we probably don't want in
core by default.

Have you considered having the default behavior be to return a custom
type provided by Clojure paired with a function in core to easily
convert this type to java.util.Date? Such a type would not need to
incorporate much ugliness about date mechanics, as it can be just a
immutable container for the fields parsed out of the ISO8601
representation.

Alternately, have you considered just having the *default* parser
implementation "parse" the date to a String? As long as we're limiting
ourselves to YYYY-MM-DDTHH:MM:SS.xxxxxx format, this would be lossless
and would sort correctly lexicographically. This would work out fine
for applications that just need to pass the data through. Others could
plugin in a custom parser for Joda-Time or what-have-you.

// Ben

Ben Smith-Mannschott

unread,
Nov 5, 2011, 5:07:26 PM11/5/11
to cloju...@googlegroups.com
On Fri, Nov 4, 2011 at 19:04, Stuart Halloway <stuart....@gmail.com> wrote:
> Instant literal for Clojure:
> http://dev.clojure.org/jira/browse/CLJ-871 and
> http://dev.clojure.org/pages/viewpage.action?pageId=950382
>
> This is work in progress. I will add tests and update the patch for
> screening based on/depending on feedback here.
>
> If this flies, will be looking for volunteers to do equivalent in
> ClojureScript and ClojureCLR.

A few more questions:

Have you considered supporting the notion of precision in these
literal instants? 8601 has this notion, such that one can write:

- 2011-04Z to mean April 2011 (UTC), but nothing more precise.
- 2011-04-01Z to mean April 1, 2011, but no information about hours/mins/...

Or are the #@ instants always fully specified to some precision? What
is that precision? Milliseconds? Microseconds? Nanoseconds?

I hope we're not going to try to support negative years or years
greater than 9999, right? (YAGNI)

If one were to decide that the default representation of a parsed
instant will be a type supplied by Clojure, would it be better to
implement that in Java or (thinking of CiC), using deftype/defrecord?

I had another question when I started writing this, but I've forgotten
it. I guess it's bedtime.

// Ben

> Stu

Rich Hickey

unread,
Nov 6, 2011, 8:23:37 AM11/6/11
to cloju...@googlegroups.com

On Nov 5, 2011, at 2:16 PM, Ben Smith-Mannschott wrote:

> On Fri, Nov 4, 2011 at 19:04, Stuart Halloway <stuart....@gmail.com> wrote:
>> Instant literal for Clojure:
>> http://dev.clojure.org/jira/browse/CLJ-871 and
>> http://dev.clojure.org/pages/viewpage.action?pageId=950382
>>
>> This is work in progress. I will add tests and update the patch for
>> screening based on/depending on feedback here.
>>
>> If this flies, will be looking for volunteers to do equivalent in
>> ClojureScript and ClojureCLR.
>
> I think it's a good choice to require UTC, so as "not to get into the
> localization business". I'm less sure about returning java.util.Date
> since it's only really intended for local time, broken, and mutable.

The mutable nature of j.u.Date is non-threatening, People who mutate dates get what they deserve. If what we've heard about JSR-310 is true, some more useful facilities may arise around it eventually. Those who don't want it can replace it.

> Have you considered having the default behavior be to return a custom
> type provided by Clojure paired with a function in core to easily
> convert this type to java.util.Date? Such a type would not need to
> incorporate much ugliness about date mechanics, as it can be just a
> immutable container for the fields parsed out of the ISO8601
> representation.
>

I don't see what it adds over Date. There will be no existing downstream consumers of that, where there are plenty for Date. Ditto on any other host on which Clojure runs.

A custom type (without a library) means that everything you read that way needs to be post-processed. If that's the case, why not just plug in a parser that returns what you want up front?

I'd be very concerned about the pressure for a library that would surround any such custom type. It is specifically an anti-requirement of this to engage in such a wheel-reinventing endeavor. It's very difficult to argue against the host date unless you are going to provide a complete, portable date abstraction. My guess is that, even if we did that, the most frequently called function in the library would be the one that converted back into the host format :(

> Alternately, have you considered just having the *default* parser
> implementation "parse" the date to a String? As long as we're limiting
> ourselves to YYYY-MM-DDTHH:MM:SS.xxxxxx format, this would be lossless
> and would sort correctly lexicographically. This would work out fine
> for applications that just need to pass the data through. Others could
> plugin in a custom parser for Joda-Time or what-have-you.

It don't think it's fine, as they no longer can distinguish dates from strings. If people want to use strings for dates, they can do that now.

Rich

Rich Hickey

unread,
Nov 6, 2011, 8:54:33 AM11/6/11
to cloju...@googlegroups.com

On Nov 5, 2011, at 5:07 PM, Ben Smith-Mannschott wrote:

> On Fri, Nov 4, 2011 at 19:04, Stuart Halloway <stuart....@gmail.com> wrote:
>> Instant literal for Clojure:
>> http://dev.clojure.org/jira/browse/CLJ-871 and
>> http://dev.clojure.org/pages/viewpage.action?pageId=950382
>>
>> This is work in progress. I will add tests and update the patch for
>> screening based on/depending on feedback here.
>>
>> If this flies, will be looking for volunteers to do equivalent in
>> ClojureScript and ClojureCLR.
>
> A few more questions:
>
> Have you considered supporting the notion of precision in these
> literal instants? 8601 has this notion, such that one can write:
>
> - 2011-04Z to mean April 2011 (UTC), but nothing more precise.
> - 2011-04-01Z to mean April 1, 2011, but no information about hours/mins/...
>
> Or are the #@ instants always fully specified to some precision? What
> is that precision? Milliseconds? Microseconds? Nanoseconds?
>

That isn't exactly about precision. I.e. the phrase "nothing more precise" goes beyond "I have no more information" to "I want to retain the fact that there isn't more information" (i.e. to imply the range of time covered by the precision interval). The semantics we want to represent by this literal are those of a point in time. April 2011 isn't a point in time, to get one there will be imputed rounding (i.e. a presumption of zero for trailing components). On the other end, how much precision gets preserved is a programmatic representation issue. j.u.Dates do msecs.

That said, ISO8601 can be used for e.g. calendar dates and times-of-day, and I'd like to support as least the latter of these, but AFAICT, you can't detect that use via parsing rules only. Thus, we'd need a second literal for time-of-day. Ditto durations, intervals etc. Instant is the most important, and thus first up.

> I hope we're not going to try to support negative years or years
> greater than 9999, right? (YAGNI)
>

Their use is only supported by mutual agreement between sender and receiver. I don't see a reason to get in the way of that. Obviously, a compatible programmatic representation must be chosen.

> If one were to decide that the default representation of a parsed
> instant will be a type supplied by Clojure, would it be better to
> implement that in Java or (thinking of CiC), using deftype/defrecord?
>

I'm far from deciding to use a custom type.

Rich

David Powell

unread,
Nov 6, 2011, 8:57:24 AM11/6/11
to cloju...@googlegroups.com
Some comments...

Generality

There have been suggestions that this might become a more general extensibility point for data types.  I am more interested in seeing this area explored, than seeing a specific reader macro for dates.  I've wrote these comments with that in mind.


Default Implementation

It has been suggested that the default implementation should be String.  I like this idea, especially in the general case, where we might decide that Java's URL classes or whatever are too lossy and broken, so a fallback that preserves everything, and doesn't break code if an implementation is unavailable.

The problem is it is lossy.  If another implemenation calls read and hands you some data, then the datatype tagging has been stripped.
Another alternative would be a DateStamp deftype holding the string.  This would preserve both the lexical data value, and the fact that the thing is a date-stamp, and could then be used with protocol-based abstractions.  It could also be str-ified by people who need to read data, but don't need to perform operations on the type.  It also saves on the performance cost of parsing dates in data when the consumer might not even be interested in the dates in that data.

A Joda adapter should then be provided as an optional library.  3rd parties could provide alternative adapters for other implementations such as the JSR, Date, Calendar, or the icu4j stuff.

This idea could be extended to other custom datatypes.  A function could be supplied to convert the default wrapped lexical value type to the currently installed provider for cases when data has been read by outside code and passed to code that handles dates - this could then be mapped over the data.


Time Zones

I have a couple of Clojure applications that do date computations (using JodaTime), that typically have times recorded as midnight.  In England, we use +00:00 time in the winter, and +01:00 time in the summer.

This means that if I have some dates encoded as UTC, representing the first day of each month, they will look something like:

#@2011-01-01T00:00:00Z
#@2011-02-01T00:00:00Z
#@2011-03-01T00:00:00Z
#@2011-03-31T23:00:00Z
#@2011-04-30T23:00:00Z
#@2011-05-31T23:00:00Z
#@2011-06-30T23:00:00Z
#@2011-07-31T23:00:00Z
#@2011-08-31T23:00:00Z
#@2011-09-30T23:00:00Z
#@2011-11-01T00:00:00Z
#@2011-12-01T00:00:00Z

It makes the dates barely human readable, as I have to mentally figure out whether a date is before or after the last sunday in March in order to know what the date component is.  This would be a big compromize for me, with bascially zero gain over mapping Joda over date strings.  I don't think I'd use it.  You guys would have this new toy, and I'd be stuck in with my strings and the cottage industry of parsers.

I'd imagine that the people at ISO, the IETF, and W3C probably knew what they were talking about when they wrote specifications allowing for a timezone component.  What is the justification for disallowing them in Clojure's lexical form?


Date Standards

In this thread, I keep seeing people use terms like 'standards', 'ISO8601', and even a demo parser showing how easy ISO8601 is to parse.
It seems as if most people havent actually read ISO8601 and are just guessing as to its contents.

Are our implementations going to support these valid ISO8601 dates?

20111010
2009-W53-7
2010-03

I presume not.

The Atom syndication feed format chose to use the RFC3339 profile of ISO8601.  I'd suggest reading section 3.3 of RFC4287 to understand this choice, as it explains its compatability with other profiles.  I'd also suggest reading RFC3339 which explains ISO8601 and the IETFs prefered profile.


Plugin Mechanism

Please don't do classpath based auto-discovery of Joda.  Have we learned nothing from the commons-logging disaster?

I don't want my clojure subcomponent failing because another subcomponent has introduced a depenency on Joda, or because some J2EE classloader intricacies, OSGi, or Google App Engine have outsmarted your auto-discovery mechanism.  Step away from the classloaders...

I am a bit concerned that data readers will have the option to install sane date implementations, but plain code won't, so applications will potentially have to deal with two implementations - one of which might have poor fidelity or capabilities.

It is tempting to say that 'load' binds the provider to #(throw Exception. "Instants only supported for data"), until we have thought more about why and how we would handle these things in code.  Though that wouldn't be so nice for defmacro-based DSLs.

I think that the lexical date type solution above fixes most of these problems though.

-- 
Dave

Rich Hickey

unread,
Nov 6, 2011, 10:18:40 AM11/6/11
to cloju...@googlegroups.com

On Nov 6, 2011, at 8:57 AM, David Powell wrote:

> Some comments...
>
> Generality
>
> There have been suggestions that this might become a more general extensibility point for data types. I am more interested in seeing this area explored, than seeing a specific reader macro for dates. I've wrote these comments with that in mind.
>
>
> Default Implementation
>
> It has been suggested that the default implementation should be String. I like this idea, especially in the general case, where we might decide that Java's URL classes or whatever are too lossy and broken, so a fallback that preserves everything, and doesn't break code if an implementation is unavailable.
>
> The problem is it is lossy. If another implemenation calls read and hands you some data, then the datatype tagging has been stripped.
> Another alternative would be a DateStamp deftype holding the string. This would preserve both the lexical data value, and the fact that the thing is a date-stamp, and could then be used with protocol-based abstractions. It could also be str-ified by people who need to read data, but don't need to perform operations on the type. It also saves on the performance cost of parsing dates in data when the consumer might not even be interested in the dates in that data.
>
> A Joda adapter should then be provided as an optional library. 3rd parties could provide alternative adapters for other implementations such as the JSR, Date, Calendar, or the icu4j stuff.
>
> This idea could be extended to other custom datatypes. A function could be supplied to convert the default wrapped lexical value type to the currently installed provider for cases when data has been read by outside code and passed to code that handles dates - this could then be mapped over the data.

Would you do this for numbers too? If not, why not?


>
>
> Time Zones
>
> I have a couple of Clojure applications that do date computations (using JodaTime), that typically have times recorded as midnight. In England, we use +00:00 time in the winter, and +01:00 time in the summer.
>
> This means that if I have some dates encoded as UTC, representing the first day of each month, they will look something like:
>
> #@2011-01-01T00:00:00Z
> #@2011-02-01T00:00:00Z
> #@2011-03-01T00:00:00Z
> #@2011-03-31T23:00:00Z
> #@2011-04-30T23:00:00Z
> #@2011-05-31T23:00:00Z
> #@2011-06-30T23:00:00Z
> #@2011-07-31T23:00:00Z
> #@2011-08-31T23:00:00Z
> #@2011-09-30T23:00:00Z
> #@2011-11-01T00:00:00Z
> #@2011-12-01T00:00:00Z
>
> It makes the dates barely human readable, as I have to mentally figure out whether a date is before or after the last sunday in March in order to know what the date component is. This would be a big compromize for me, with bascially zero gain over mapping Joda over date strings. I don't think I'd use it. You guys would have this new toy, and I'd be stuck in with my strings and the cottage industry of parsers.
>
> I'd imagine that the people at ISO, the IETF, and W3C probably knew what they were talking about when they wrote specifications allowing for a timezone component. What is the justification for disallowing them in Clojure's lexical form?
>

Who's disallowed them? Someone might have suggested UTC-only in this discussion, but not I.

Also, ISO doesn't really have timezones, only offsets. We certainly intend to support offsets in the literal. They might or might not be retained by a programmatic representation. A program might or might not print with a local or dynamically supplied offset reference.

You seem confused here - we intend to support ISO8601, with interpretation as a point in time. If you want to interpret points in time with midnight times as calendar dates, who's stopping you? There is no requirement to specify the Z offset.

>
> Date Standards
>
> In this thread, I keep seeing people use terms like 'standards', 'ISO8601', and even a demo parser showing how easy ISO8601 is to parse.

I've never contended ISO8601 was easy. Thus the dependence on someone who's already undertaken the job.

> It seems as if most people havent actually read ISO8601 and are just guessing as to its contents.
>

Um, I've got it open right here.

> Are our implementations going to support these valid ISO8601 dates?
>
> 20111010
> 2009-W53-7
> 2010-03
>
> I presume not.

Why not? If Joda can parse them for us, some of those seem quite convenient to write.

>
> The Atom syndication feed format chose to use the RFC3339 profile of ISO8601. I'd suggest reading section 3.3 of RFC4287 to understand this choice, as it explains its compatability with other profiles. I'd also suggest reading RFC3339 which explains ISO8601 and the IETFs prefered profile.
>

Yes, and I'd consider arguments for the RFC3339 subset, but one significant negative is this:

"Simplicity is achieved by making most fields and punctuation mandatory."

This is less desirable for any reader format that might ever be written by hand. E.g. people can happily pretend they are UTC Z and ignore times when talking about (calendar) dates. With RFC3339 they cannot. Consider e.g. a DSL for talking to databases.

Further, if we are to ever support time-related types other than timestamps (e.g. time-of-day), we will be forced off of RFC3339.

>
> Plugin Mechanism
>
> Please don't do classpath based auto-discovery of Joda. Have we learned nothing from the commons-logging disaster?
>
> I don't want my clojure subcomponent failing because another subcomponent has introduced a depenency on Joda, or because some J2EE classloader intricacies, OSGi, or Google App Engine have outsmarted your auto-discovery mechanism. Step away from the classloaders...
>

The auto-discovery is just for the (default, internal) parsing support, which requires it or won't work. It won't be used to drive the programmatic representation.

> I am a bit concerned that data readers will have the option to install sane date implementations, but plain code won't, so applications will potentially have to deal with two implementations - one of which might have poor fidelity or capabilities.
>

Agreed, and that won't be the case. There will likely be explicit config-based setup as well (TBD, and suggestions welcome). I think there has been too much focus on the binding/dynamic aspect.

> It is tempting to say that 'load' binds the provider to #(throw Exception. "Instants only supported for data"), until we have thought more about why and how we would handle these things in code. Though that wouldn't be so nice for defmacro-based DSLs.
>

Instants will not be supported only for data. Obviously if you write an instant literal in your code it will have to turn into an object when loaded.

> I think that the lexical date type solution above fixes most of these problems though.

Not without tradeoffs, esp. re: human use.

Rich

Rich Hickey

unread,
Nov 6, 2011, 11:06:20 AM11/6/11
to cloju...@googlegroups.com

On Nov 6, 2011, at 8:57 AM, David Powell wrote:

>
> The Atom syndication feed format chose to use the RFC3339 profile of ISO8601. I'd suggest reading section 3.3 of RFC4287 to understand this choice, as it explains its compatability with other profiles. I'd also suggest reading RFC3339 which explains ISO8601 and the IETFs prefered profile.
>

I like the idea of an ISO8601-compliant, yet simplifying 'profile' like RFC3339 [1]. I am concerned about not having good parsing support like Joda's in JS.

As I said, RFC3339, in making almost everything mandatory, is not a good fit for human use in several domains. But something like it, that allowed for the elision of trailing components, might be perfect, and support simple(r) parsers.

Anyone want to take a crack at modifying the RFC3339 ABNF (section 5.6) to incorporate elision of trailing components and making a Clojure parser for it?

No time offset means Z.

T and Z must be uppercase when supplied.

Rich

[1] http://www.ietf.org/rfc/rfc3339.txt

David Powell

unread,
Nov 6, 2011, 11:19:31 AM11/6/11
to cloju...@googlegroups.com
On Sun, Nov 6, 2011 at 3:18 PM, Rich Hickey <richh...@gmail.com> wrote:

> I'd imagine that the people at ISO, the IETF, and W3C probably knew what they were talking about when they wrote specifications allowing for a timezone component.  What is the justification for disallowing them in Clojure's lexical form?
>
 
Who's disallowed them? Someone might have suggested UTC-only in this discussion, but not I.

Oh ok, I was mistaken then.  I'd assumed that the text in the Q&A section of the proposal implied that timezones would not be allowed.  Together with the implementation that seems to specify a UTC offset - though I'm not sure exactly what that does...

(It is hard work debating this when the proposal is rather terse, the implementation doesn't match the proposal, and there seem to be assumptions in the thread that match neither; so bear with me).


You seem confused here - we intend to support ISO8601, with interpretation as a point in time. If you want to interpret points in time with midnight times as calendar dates, who's stopping you? There is no requirement to specify the Z offset.

I was confused as to whether offsets were allowed by the proposal.  If they weren't then it would lead to a suboptimal readable form for midnight dates.  As you explain - that is not the intention, so I'm happy with that.
 
> It seems as if most people havent actually read ISO8601 and are just guessing as to its contents.

Um, I've got it open right here.

That's cool.  I wasn't talking about you.  (Actually I've never read it - only second hand via RFC3339).  It was just a warning for others to be a bit more precise in what they are talking about if we are proposing a syntax.  I've never seen a data format or application that uses the whole of ISO8601 rather than the full thing.  If we don't use a profile then we may find that Joda is the only practical implementation.  I presume that the proposed java.xml.bindDatatypeConverter doesn't support ISO8601 formats outside of the XSD profile.

We need to discuss whether we want to support things like ISO8601's unqualified local time - I'm not even sure how well Joda supports mixing these with absolute times.  It has a high risk of being misinterpreted and causing interop issues.

-- 
Dave

Rich Hickey

unread,
Nov 6, 2011, 11:29:30 AM11/6/11
to cloju...@googlegroups.com

On Nov 6, 2011, at 11:19 AM, David Powell wrote:

>
>
> On Sun, Nov 6, 2011 at 3:18 PM, Rich Hickey <richh...@gmail.com> wrote:
>
> > I'd imagine that the people at ISO, the IETF, and W3C probably knew what they were talking about when they wrote specifications allowing for a timezone component. What is the justification for disallowing them in Clojure's lexical form?
> >
>
> Who's disallowed them? Someone might have suggested UTC-only in this discussion, but not I.
>
> Oh ok, I was mistaken then. I'd assumed that the text in the Q&A section of the proposal implied that timezones would not be allowed. Together with the implementation that seems to specify a UTC offset - though I'm not sure exactly what that does...
>
> (It is hard work debating this when the proposal is rather terse, the implementation doesn't match the proposal, and there seem to be assumptions in the thread that match neither; so bear with me).
>

Understood, and sorry for that.


>
> That's cool. I wasn't talking about you. (Actually I've never read it - only second hand via RFC3339). It was just a warning for others to be a bit more precise in what they are talking about if we are proposing a syntax. I've never seen a data format or application that uses the whole of ISO8601 rather than the full thing. If we don't use a profile then we may find that Joda is the only practical implementation. I presume that the proposed java.xml.bindDatatypeConverter doesn't support ISO8601 formats outside of the XSD profile.
>
> We need to discuss whether we want to support things like ISO8601's unqualified local time - I'm not even sure how well Joda supports mixing these with absolute times. It has a high risk of being misinterpreted and causing interop issues.
>

The point of using ISO8601 is not about getting everything it does, as it clearly does too much, but rather to be compliant with a spec, rather than just making stuff up. Coming up with our own subset profile might be an acceptable middle ground, but we'll see.

Rich

Ben Smith-Mannschott

unread,
Nov 6, 2011, 6:46:07 PM11/6/11
to cloju...@googlegroups.com
On Sun, Nov 6, 2011 at 17:06, Rich Hickey <richh...@gmail.com> wrote:
>
> On Nov 6, 2011, at 8:57 AM, David Powell wrote:
>
>>
>> The Atom syndication feed format chose to use the RFC3339 profile of ISO8601.  I'd suggest reading section 3.3 of RFC4287 to understand this choice, as it explains its compatability with other profiles.  I'd also suggest reading RFC3339 which explains ISO8601 and the IETFs prefered profile.
>>
>
> I like the idea of an ISO8601-compliant, yet simplifying 'profile' like RFC3339 [1]. I am concerned about not having good parsing support like Joda's in JS.
>
> As I said, RFC3339, in making almost everything mandatory, is not a good fit for human use in several domains. But something like it, that allowed for the elision of trailing components, might be perfect, and support simple(r) parsers.
>
> Anyone want to take a crack at modifying the RFC3339 ABNF (section 5.6) to incorporate elision of trailing components and making a Clojure parser for it?

Please have a look at this:

https://github.com/bpsm/rfc3339datetime/blob/master/src/rfc3339datetime/core.clj#L119

Currently it just deposits the results of the parse into a map and is
lacking the java glue to drop it into LispReader. It does, however,
parse RFC3339-style dates, times and date-times and supports elision
of trailing components.

I'm willing to make any changes deemed necessary if you think it could
be of use in extending the reader to handle instant literals.

> No time offset means Z.

done

> T and Z must be uppercase when supplied.

done

// Ben

Rich Hickey

unread,
Nov 6, 2011, 9:59:29 PM11/6/11
to cloju...@googlegroups.com


Thanks for tackling this!

I think I was thinking of something slightly different:

date-year = 4DIGIT
date-month = 2DIGIT ; 01-12
date-mday = 2DIGIT ; 01-28, 01-29, 01-30, 01-31 based on
; month/year
time-hour = 2DIGIT ; 00-23
time-minute = 2DIGIT ; 00-59
time-second = 2DIGIT ; 00-58, 00-59, 00-60 based on leap second
; rules
time-secfrac = '.' 1*DIGIT
time-numoffset = ('+' / '-') time-hour ':' time-minute
time-offset = 'Z' / time-numoffset

time-part = time-hour [ ':' time-minute [ ':' time-second [time-secfrac] [time-offset] ] ]

timestamp = date-year [ '-' date-month [ '-' date-mday [ 'T' time-part ] ] ]

We won't independently parse time or date until we want to add time-of-day or calendar-date literals (neither of which will have offsets).

Rich

Ben Smith-Mannschott

unread,
Nov 7, 2011, 2:39:02 AM11/7/11
to cloju...@googlegroups.com

That simplifies the problem considerably. I'll make the necessary
changes this evening (+01:00).

> We won't independently parse time or date until we want to add time-of-day or calendar-date literals (neither of which will have offsets).

So, if/when time or date literals come they will be introduce by macro
characters distinguishable from #@ as otherwise there would be an
ambiguity since date would be a prefix of timestamp, correct?

// Ben

Rich Hickey

unread,
Nov 7, 2011, 7:42:45 AM11/7/11
to cloju...@googlegroups.com
>
>> We won't independently parse time or date until we want to add time-of-day or calendar-date literals (neither of which will have offsets).
>
> So, if/when time or date literals come they will be introduce by macro
> characters distinguishable from #@ as otherwise there would be an
> ambiguity since date would be a prefix of timestamp, correct?
>

Correct, if we ever do calendar date (whence come the nightmares). I was interested in possibly overloading @ for time of day, but some of the ISO8601 representations can't be distinguished. With this subset they could be, but the semantics are different. @ is just too good a fit for both :)

That said, @ may not be long for this world. If people can accept representation independence we can start talking about the tag system required to expand this facility.

Also, to answer my previous rhetorical question re: wrappers - "Would you do this for numbers too?"

The answer is no. No one wants to live in a world where they have to wonder if things have been unwrapped/parsed yet or not. And no one wants to have to post-process arbitrarily nested data structures in order to fully unwrap them, especially when more than one unwrapping is in play. I'm fine with having a protocol between any lower-level parser and the value constructors, but I think the result of read should be the thing you want to use in your program, not a package containing a recipe for a thing.

Thanks again for the help,

Rich

Sean Devlin

unread,
Nov 7, 2011, 9:17:30 AM11/7/11
to cloju...@googlegroups.com
Stu,
Would love to build a contrib.  What's the process for that currently?  Fork github & start a thread on Clojure-dev?

Sean

On Fri, Nov 4, 2011 at 5:16 PM, Stuart Halloway <stuart....@gmail.com> wrote:
> Would you like my datable protocol stuff?
>
> Sean

It or something like it, in a contrib. Will you be the Conj?

Stuart Halloway

unread,
Nov 7, 2011, 9:37:31 AM11/7/11
to cloju...@googlegroups.com
Stu,
Would love to build a contrib.  What's the process for that currently?  Fork github & start a thread on Clojure-dev?

Sean

Thread on clojure-dev that covers (or points to) the hammock-driven part of the process is the best place to start. I.e. problem or problems being addressed, tradeoffs, compare and contrast >1 possible approach, etc.

And prepare to be patient. :-/  (Although I am optimistic that the amount of face time this week might move things faster than when we don't have 300 people together in one place.)

Stu

Stuart Halloway
Clojure/core
http://clojure.com

Ben Smith-Mannschott

unread,
Nov 7, 2011, 2:07:07 PM11/7/11
to cloju...@googlegroups.com

https://github.com/bpsm/rfc3339datetime/blob/master/src/rfc3339datetime/core.clj#L96

I've updated the parser to reflect the above ABNF. It now parses only
timestamps. Internally, it produces the same kind sort of map as the
previous implementation.

I've also gone ahead and coded some conversions from this internal
representation to java.util.Date and java.util.Calendar, from the
internal representation back to an string formatted as above. I've
written some unit tests to confirm that this string representation
will round trip. In the process, I've noticed a few issues:

- I'm currently storing the fractional second as a double. This was a
punt to get around either (1) needing to store an integer and a scale
factor or (2) dictating a precision. Java doesn't appear to have a
formatting option which means "print as many digits as you need to
round-trip this number", so currently I'm just picking nanosecond
precision when converting from map to string.

Switching to option (1) by storing an integer for the fractional part
along with a scale factor makes equality (of the internal map
representation) less nice. (see BigDecimal). Switching to option (2)
is probably the most pragmatic, but what should said precision be?

- Calendar has a notion of timezone offset, so this gets preserved
when we convert from the internal map.

- Date, however, does not have a notion of timezone offset, so we get
local time when we convert Calendar to Date, which is confusing since
a timestamp entered without an explicit offset is (per definition
above) treated as Z, so what you type in will read differently than
what date prints out since we can write the timestamp to look like
local time, when in fact it isn't.

- More seriously, Date loses the timezone offset, so if the default
reader behavior is to expose timestamp literals as Date objects in
the, this will be lossy and not capable of being round-tripped with
the timezone in tact. The best we could hope for is converting the
Date objects (local time zone) back to UTC before generating their
string representation. We'd still lose the timezone offset originally
entered, but at least the point in time specified by the original
literal would not drift.

Sorry, bit rambly.

Thoughts?

// Ben

David Powell

unread,
Nov 7, 2011, 2:43:43 PM11/7/11
to cloju...@googlegroups.com
On Mon, Nov 7, 2011 at 12:42 PM, Rich Hickey <richh...@gmail.com> wrote:

> Also, to answer my previous rhetorical question re: wrappers - "Would you do this for numbers too?"
>
> The answer is no. No one wants to live in a world where they have to wonder if things have been unwrapped/parsed yet or not. And no one wants to have to post-process arbitrarily nested data structures in order to fully unwrap them, especially when more than one unwrapping is in play. I'm fine with having a protocol between any lower-level parser and the value constructors, but I think the result of read should be the thing you want to use in your program, not a package containing a recipe for a thing.

One of the great things about Lisps is that the reader is an
abstraction for syntax. Your code never needs to think about parens,
read data is a full fidelity abstraction over syntax. Data is data.
It can stand alone.

The default implementation of instants is j.u.Date - which is lossy,
given that it doesn’t preserve zone offsets or sub-milli precision.
So the fidelity of data is dependent on the context of the code that
read it. And code is dependent on some system properties or whatever
set the default implementation. And by default, you get losage. This
seems like a layering violation.


As I understand it, it is out of scope to provide a set of polymorphic
date arithmetic operations that operate on j.u.Date, Joda, Calendar,
etc.

The only operations that will be required of all implementations are
print and read. (and maybe some sort of is-instant? operation).

LexicalInstant would have a string field, but that doesn’t make it a
‘wrapper’ around string any more than Joda is a wrapper around long.
LexicalInstant would provide the required operations: print and read.
It isn’t a wrapper; it is an implementation.


I agree that you want the reader to hand you the things that you want
to use in your program. I’d want the reader to hand me Joda DateTime
objects, and I’d bind the Joda reader before I read any data. But
what about data that has been read before you had the choice to
override it? Either because my module is part of a larger system that
has no specific interest in dates, and that has read data on my behalf
and passed it to my function; or because the reader macro was embedded
in code. In the proposal I’d get given a j.u.Date; which isn’t what I
wanted. I’d prefer to receive a LexicalInstant - because at least
that hasn’t a) lost fidelity in the information; and b) expended
effort parsing text into complex object that I never wanted.


Inevitably, someone will implement a polymorphic protocol for date
operations, and there is no reason why they can’t extend
LexicalInstant to that protocol using Joda for the implementation. No
explicit unwrapping by clients would be required.

You’d only have to deal with LexicalInstants if you have no interest
in date operations, in which case they’d be a fine and performant
implementation; or in dire situations where you’ve not had control
over the reader, in which case they’d be better than being handed a
j.u.Date when you wanted a Joda object.

--
Dave

Luc Prefontaine

unread,
Nov 7, 2011, 3:26:02 PM11/7/11
to cloju...@googlegroups.com
Hi,

I would hate loosing the timezone and end up with an arbitrary object that yields
a time that suddenly becomes "local" :(.

Minimally the conversion would have to take the timezone as in input in the conversion
process but this will yield strange Date values not reflecting local time. Guaranteed fun to
debug an app with divergent Date values... sometimes UTC sometimes anchored in the local timezone
depending how they have been created.

Using a Calendar object systematically to convert to avoid this pitfall seems to me to be a significant overhead.

I agree with David Powell, why do we urgently need to conversion to an implementation dependent representation ?
Postponing the conversion as late as possible looks the best to me, especially if you are
just passing around date/time values.

Why not leave that to the implementation through some extension (lib/protocol, whatever) when it
feels appropriate to convert (user display, computations, conversions, ...) ?

I personally hate local timezones because tzdata is changing quite frequently.
That will not change soon, there are discussions in many countries in progress about removing daylight
saving changes because the expected $ return is diminishing.

Using UTC or postponing conversion until needed looks to me more promising.

Luc P.

--

Alex Miller

unread,
Nov 7, 2011, 3:40:29 PM11/7/11
to cloju...@googlegroups.com
> - I'm currently storing the fractional second as a double. This was a
> punt to get around either (1) needing to store an integer and a scale
> factor or (2) dictating a precision. Java doesn't appear to have a
> formatting option which means "print as many digits as you need to
> round-trip this number", so currently I'm just picking nanosecond
> precision when converting from map to string.
>
> Switching to option (1) by storing an integer for the fractional part
> along with a scale factor makes equality (of the internal map
> representation) less nice. (see BigDecimal). Switching to option (2)
> is probably the most pragmatic, but what should said precision be?

I think nanosecond precision is sufficient for everything I work with
regularly (databases, XML, Joda, JSR 310). There was lengthy
discussion on how to represent nanosecond instants in JSR 310 and a
lot of experimentation. In the end they settled on using two fields:
- long for seconds since the epoch
- int for nanos farther than the second with valid range [0, 999999999].

Inspired by that I'd suggest using an int with a known range plus a
printer smart enough to omit trailing 0s. I think there are many
floating point reasons to avoid using a double.

> - Calendar has a notion of timezone offset, so this gets preserved
> when we convert from the internal map.
>
> - Date, however, does not have a notion of timezone offset, so we get
> local time when we convert Calendar to Date, which is confusing since
> a timestamp entered without an explicit offset is (per definition
> above) treated as Z, so what you type in will read differently than
> what date prints out since we can write the timestamp to look like
> local time, when in fact it isn't.
>
> - More seriously, Date loses the timezone offset, so if the default
> reader behavior is to expose timestamp literals as Date objects in
> the, this will be lossy and not capable of being round-tripped with
> the timezone in tact. The best we could hope for is converting the
> Date objects (local time zone) back to UTC before generating their
> string representation. We'd still lose the timezone offset originally
> entered, but at least the point in time specified by the original
> literal would not drift.

<opinion>
I have spent painful parts of my life working with date values
returned from multiple databases and users in distributed systems
spread across timezones. All of the Java choices are broken.
java.util.Date can't do nanos or timezones. Calendar can't do nanos.
java.sql.Timestamp can't do timezones and has broken equality
semantics. They're all mutable. The Java formatter/parsers want to
interpret timezone-less dates in terms of the platform timezone.

I understand the desire to return a "host-appropriate" type, but as a
Clojure developer, the host alternatives are all insufficient. I also
don't want to include Joda in every app I ever write. There are many
cases where I need something better than Java provides (ie non-broken)
but far less than Joda provides (date arithmetic, multiple calendar
systems, etc).

I really want:
1) a Clojure data representation (be it map, record, type, etc)
2) reader and printer support to standard format (ISO8601)
3) transformations to/from host types (I'd rather stay in a non-host
representation because all host representations are broken - these are
needed for interop though)
4) optional Clojure libs for transformations to popular 3rd party libs like Joda
5) optional Clojure libs with protocols for generic accessors ala dateable
</opinion>

But I'm really curious to hear how this plays into a more generic
story re readers and tagging.

David Powell

unread,
Nov 7, 2011, 4:05:58 PM11/7/11
to cloju...@googlegroups.com
On Mon, Nov 7, 2011 at 8:26 PM, Luc Prefontaine
<lprefo...@softaddicts.ca> wrote:
> Hi,
>
> I would hate loosing the timezone and end up with an arbitrary object that yields
> a time that suddenly becomes "local" :(.

That shouldn't happen. Even j.u.Date stores times as UTC.


> I agree with David Powell, why do we urgently need to conversion to an implementation dependent representation ?
> Postponing the conversion as late as possible looks the best to me, especially if you are
> just passing around date/time values.

I was only suggesting to use a do-nothing/preserving implementation if
no better implementation had been installed.

--
Dave

Rich Hickey

unread,
Nov 7, 2011, 4:56:03 PM11/7/11
to cloju...@googlegroups.com

On Nov 7, 2011, at 2:43 PM, David Powell wrote:

> On Mon, Nov 7, 2011 at 12:42 PM, Rich Hickey <richh...@gmail.com> wrote:
>
>> Also, to answer my previous rhetorical question re: wrappers - "Would you do this for numbers too?"
>>
>> The answer is no. No one wants to live in a world where they have to wonder if things have been unwrapped/parsed yet or not. And no one wants to have to post-process arbitrarily nested data structures in order to fully unwrap them, especially when more than one unwrapping is in play. I'm fine with having a protocol between any lower-level parser and the value constructors, but I think the result of read should be the thing you want to use in your program, not a package containing a recipe for a thing.
>
> One of the great things about Lisps is that the reader is an
> abstraction for syntax. Your code never needs to think about parens,
> read data is a full fidelity abstraction over syntax. Data is data.
> It can stand alone.
>

One of the not-so-great things about Lisps is that the number of types supported by the reader is minute, non-polymorphic, and only extensible in clash-producing ways. We already cross the fidelity barrier with (), {} and #{} not returning precisely the same types by default, without catastrophe.

> The default implementation of instants is j.u.Date - which is lossy,
> given that it doesn’t preserve zone offsets or sub-milli precision.
> So the fidelity of data is dependent on the context of the code that
> read it. And code is dependent on some system properties or whatever
> set the default implementation. And by default, you get losage. This
> seems like a layering violation.
>

One of the other not-so-great things about Lisps is that in the pursuit of correctness they created islands where only they were correct, and getting to and from the mainland required conversions. These are largely uninhabited islands.

>
> As I understand it, it is out of scope to provide a set of polymorphic
> date arithmetic operations that operate on j.u.Date, Joda, Calendar,
> etc.
>
> The only operations that will be required of all implementations are
> print and read. (and maybe some sort of is-instant? operation).
>
> LexicalInstant would have a string field, but that doesn’t make it a
> ‘wrapper’ around string any more than Joda is a wrapper around long.
> LexicalInstant would provide the required operations: print and read.
> It isn’t a wrapper; it is an implementation.
>

It's an implementation of 'Wrapper', and getting from it to anything you could use in date operations would be an expensive parsing/assembly job.

>
> I agree that you want the reader to hand you the things that you want
> to use in your program. I’d want the reader to hand me Joda DateTime
> objects, and I’d bind the Joda reader before I read any data. But
> what about data that has been read before you had the choice to
> override it? Either because my module is part of a larger system that
> has no specific interest in dates, and that has read data on my behalf
> and passed it to my function; or because the reader macro was embedded
> in code.

You should presume for the moment that there will be a way to set up, prior to your program running at all, the type to be returned by all cases of reading (and embedding into code) of this literal. You certainly bear no greater risk of being passed other than what you want than you do today.

> In the proposal I’d get given a j.u.Date; which isn’t what I
> wanted. I’d prefer to receive a LexicalInstant - because at least
> that hasn’t a) lost fidelity in the information; and b) expended
> effort parsing text into complex object that I never wanted.
>

I asked you the question about numbers before and you didn't answer. But it is a vital question. No one would want numbers treated this way.

>
> Inevitably, someone will implement a polymorphic protocol for date
> operations, and there is no reason why they can’t extend
> LexicalInstant to that protocol using Joda for the implementation. No
> explicit unwrapping by clients would be required.

On a string? That would be awful performance-wise.

Rich

Phil Hagelberg

unread,
Nov 7, 2011, 5:16:03 PM11/7/11
to cloju...@googlegroups.com
On Mon, Nov 7, 2011 at 12:40 PM, Alex Miller <al...@puredanger.com> wrote:
> <opinion>
> I have spent painful parts of my life working with date values
> returned from multiple databases and users in distributed systems
> spread across timezones.  All of the Java choices are broken.
> java.util.Date can't do nanos or timezones. Calendar can't do nanos.
> java.sql.Timestamp can't do timezones and has broken equality
> semantics.  They're all mutable.  The Java formatter/parsers want to
> interpret timezone-less dates in terms of the platform timezone.
>
> I understand the desire to return a "host-appropriate" type, but as a
> Clojure developer, the host alternatives are all insufficient.  I also
> don't want to include Joda in every app I ever write.  There are many
> cases where I need something better than Java provides (ie non-broken)
> but far less than Joda provides (date arithmetic, multiple calendar
> systems, etc).

I strongly agree with Alex here. It would be wise to wait on this
until JSR 310 has landed along with its backports; rushing forward
with the deeply flawed current options would be regrettable.

-Phil

Rich Hickey

unread,
Nov 7, 2011, 6:03:40 PM11/7/11
to cloju...@googlegroups.com

So we shelve the entire thing?

Or throw when no handler is installed, until JSR 310 arrives?

I guess the latter is preferable to returning X now and Y later, for any value of X.

I am deeply opposed to wrappers or shims.

Rich

Stuart Halloway

unread,
Nov 7, 2011, 6:12:34 PM11/7/11
to cloju...@googlegroups.com
>>> <opinion>
>>> I have spent painful parts of my life working with date values
>>> returned from multiple databases and users in distributed systems
>>> spread across timezones. All of the Java choices are broken.
>>> java.util.Date can't do nanos or timezones. Calendar can't do nanos.
>>> java.sql.Timestamp can't do timezones and has broken equality
>>> semantics. They're all mutable. The Java formatter/parsers want to
>>> interpret timezone-less dates in terms of the platform timezone.
>>>
>>> I understand the desire to return a "host-appropriate" type, but as a
>>> Clojure developer, the host alternatives are all insufficient. I also
>>> don't want to include Joda in every app I ever write. There are many
>>> cases where I need something better than Java provides (ie non-broken)
>>> but far less than Joda provides (date arithmetic, multiple calendar
>>> systems, etc).
>>
>> I strongly agree with Alex here. It would be wise to wait on this
>> until JSR 310 has landed along with its backports; rushing forward
>> with the deeply flawed current options would be regrettable.
>
> So we shelve the entire thing?
>
> Or throw when no handler is installed, until JSR 310 arrives?

That works for me. Phil's intermediate option doesn't exist, and we can offer value to people willing to use Joda now (myself being one.)

Luc Prefontaine

unread,
Nov 7, 2011, 6:16:03 PM11/7/11
to cloju...@googlegroups.com

I like this idea, this leaves a number of options opened depending on the app's needs.
Would the handlers come from a predefine choice or would we be able to provide our own ?

> I guess the latter is preferable to returning X now and Y later, for
> any value of X.
>
> I am deeply opposed to wrappers or shims.
>
> Rich
>

--

Rich Hickey

unread,
Nov 7, 2011, 6:22:49 PM11/7/11
to cloju...@googlegroups.com

On Nov 7, 2011, at 3:40 PM, Alex Miller wrote:

> But I'm really curious to hear how this plays into a more generic
> story re readers and tagging.

The story is a lot like this one - bitter dispute over desirable programmatic representation types ;-), but hope of consensus over read/print representations and semantics.

The basic idea is:

#tag some-X-we-can-already-read

with agreement as to the semantics of tag and choice of the structure(s) of X.

Inside the reader (and code loader etc), the X is fed to:

pluggable-fn-of-X -> any-Y-program-wants

read then yielding:

Y-program-wants.

Non-namespace-qualified tags are reserved for the language, but namespaced ones can be used for anything, without collision.

Everything nests, as usual (one of many key drawbacks of wrappers and shims is this scenario).

There will be some config/property file thingy where handlers are registered.

So, in lieu of special support for instants, we could do:

#instant "1985-04-12T23:20:50.52Z"

and things like:

#ints [0 1 3 5 7]

#my.ns.foo {:bar 123 :baz [5 6 7]}

etc.

Rich

David Powell

unread,
Nov 7, 2011, 8:19:48 PM11/7/11
to cloju...@googlegroups.com
This sounds very interesting. I still have some issues with what I've
heard so far, but I'm going to take a step back and leave things in my
outbox until things have settled a bit. No doubt people are busy with
the Conj this week.

--
Dave

Phil Hagelberg

unread,
Nov 7, 2011, 8:26:18 PM11/7/11
to cloju...@googlegroups.com
On Mon, Nov 7, 2011 at 3:03 PM, Rich Hickey <richh...@gmail.com> wrote:
> So we shelve the entire thing?
>
> Or throw when no handler is installed, until JSR 310 arrives?

It depends on the current state of the backport, I suppose. If it's
good enough and there's high confidence it will make it into the JDK
in the future, I could see a path that avoids a lot of complexity by
simply using JSR 310 rather than supporting a myriad of types. But I
haven't looked into this in depth.

It's worth mentioning that since the JSR isn't finalized, interested
parties may find it worthwhile to join the design discussions on their
mailing list: http://sourceforge.net/mailarchive/forum.php?forum_name=threeten-develop

-Phil

Sean Corfield

unread,
Nov 8, 2011, 1:38:20 AM11/8/11
to cloju...@googlegroups.com
On Mon, Nov 7, 2011 at 3:22 PM, Rich Hickey <richh...@gmail.com> wrote:
> #instant "1985-04-12T23:20:50.52Z"
>
> and things like:
>
> #ints [0 1 3 5 7]
>
> #my.ns.foo {:bar 123 :baz [5 6 7]}

This is what I was looking for and something like this I can get
behind. I don't like the #@ syntax...
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

Meikel Brandmeyer

unread,
Nov 8, 2011, 2:36:49 AM11/8/11
to cloju...@googlegroups.com
Hi,


Am Dienstag, 8. November 2011 00:22:49 UTC+1 schrieb Rich Hickey:

#my.ns.foo {:bar 123 :baz [5 6 7]}

Will this be more like #my.ns/foo {:bar 123 :baz [5 6 7]}?

Sincerely
Meikel

 

Ben Smith-Mannschott

unread,
Nov 8, 2011, 4:04:32 AM11/8/11
to cloju...@googlegroups.com

Since I'm not going to be at the Conj, I'll lay out my understanding
of the situation and make a proposal.

Clojure should define the tag and syntax of the proposed instant literals.

If Clojure were to define a default parser for this syntax it should
return objects with the following properties:

- able to interoperate with platform APIs (no wrappers/shims)
- immutable
- sufficiently lossless (roundtripable)
(e.g. limit the precision if you must, but don't lose the time offset.)
- reasonable equality behavior

The JDK Does not currently provide a Class with the above
characteristics. Joda-time does provide a class with the above
characteristics, but we don't want to add a dependency on a
third-party library to Clojure core. There is hope that JSR 310 may
eventually provide us with a suitable solution.

Presently, no good options remain.

The default parser provided by Clojure for instant literals should
throw an exception, requiring users of instant literals to explicitly
specify a handler that makes the compromises appropriate to their
application.

A future version of Clojure may provide a default implementation that
returns a suitable JSR310 value.

Question: should the reader support for instant literal validate
against the syntax defined by Clojure before calling the parser
function? (This would keep users of the instant literal "honest" WRT
the syntax.) Or is it more important to avoid that overhead? (We're
all adults here.)

Have fun at the Conj!

// Ben

Rich Hickey

unread,
Nov 8, 2011, 8:44:10 AM11/8/11
to cloju...@googlegroups.com

Sorry, yes.

Rich

Rich Hickey

unread,
Nov 8, 2011, 9:27:29 AM11/8/11
to cloju...@googlegroups.com

On Nov 8, 2011, at 4:04 AM, Ben Smith-Mannschott wrote:

>
> Since I'm not going to be at the Conj, I'll lay out my understanding
> of the situation and make a proposal.
>
> Clojure should define the tag and syntax of the proposed instant literals.
>
> If Clojure were to define a default parser for this syntax it should
> return objects with the following properties:
>
> - able to interoperate with platform APIs (no wrappers/shims)
> - immutable
> - sufficiently lossless (roundtripable)
> (e.g. limit the precision if you must, but don't lose the time offset.)
> - reasonable equality behavior
>
> The JDK Does not currently provide a Class with the above
> characteristics. Joda-time does provide a class with the above
> characteristics, but we don't want to add a dependency on a
> third-party library to Clojure core. There is hope that JSR 310 may
> eventually provide us with a suitable solution.
>
> Presently, no good options remain.
>
> The default parser provided by Clojure for instant literals should
> throw an exception, requiring users of instant literals to explicitly
> specify a handler that makes the compromises appropriate to their
> application.
>
> A future version of Clojure may provide a default implementation that
> returns a suitable JSR310 value.
>
>

It seems people want to preserve (and round-trip) offset, but a lot of trouble is taken on in trying to do that.

An instant doesn't have an offset. It may be initially expressed with one, but offset is not part of a point in time, i.e. what was expressed was point + offset, which is not an instant, but yields one *after the arithmetic is done*. Like other arithmetic results, the operands aren't preserved. We certainly accept this elsewhere:

user=> 28/7
4

I think we should revisit that presumption of need, and separate it from desire. It certainly seems that offset-preserving times are a black art not yet mastered even after years of experience in date/time library development, as evidenced by last month's discussions on the threeten list:

http://sourceforge.net/mailarchive/forum.php?forum_name=threeten-develop&max_rows=25&style=nested&viewmonth=201110

One way of considering it is that offset preservation is a value-add, subject to numerous interpretations, and if one wants it one has to buy into a specific methodology for preserving it (especially once any operations are carried out), doing so by choosing a specific representation type.

Perhaps we ought to limit the semantics of 'instant' to, well, instants. This greatly simplifies things. Yes, instants might (or might not) have less utility than date-time-offset-zone thingies, but are more precisely understood and used.

That is to say, perhaps a proposal for instant should *explicitly not promise* offset preservation, while a future date-time-offset-thing might. It then is a representational detail of an instant type that it holds the instant in 2 parts and prints it again that way. If it matters to you, choose one that does that. This is more what the original thinking was in this proposal.

In that case, the major shortcoming of j.u.Date lies in precision being limited to msecs. That is, one could consider j.u.Date a (widely recognized) semantic container for the long msecs instant and ignore the rest of it, if undesired. java.sql.Timestamp remedies the resolution issue, but has other issues in deriving from Date but not really being interoperable with Date. (Another wonder of class design, i.e. it makes you wonder about class design :) From the javadoc:

"The inheritance relationship between Timestamp and java.util.Date really denotes implementation inheritance, and not type inheritance."

"really"? Meh.

Without diving down that hole, the point here is that we've scope-creeped instant beyond the semantics of instant, and need to recognize and reconsider that.

> Question: should the reader support for instant literal validate
> against the syntax defined by Clojure before calling the parser
> function? (This would keep users of the instant literal "honest" WRT
> the syntax.) Or is it more important to avoid that overhead? (We're
> all adults here.)


The rules might be - the parser is required to validate (and thus make the promise to help the user), freeing the the reader to (threaten to) redundantly validate at any random time (to keep the parser honest).

Rich

Ben Smith-Mannschott

unread,
Nov 8, 2011, 9:56:36 AM11/8/11
to cloju...@googlegroups.com

Ah! you're right on this, of course. Round-tripping just needs to
preserve the point in time that the instant specifies, not the
particular notation used to express it. This round-trips fine:

"2011-11-08T10:11:12+01:00"
reads as an instant representing the time
2011-11-08T09:11:12+00:00
prints as
"2011-11-08T09:11:12Z" (or +00:00)
reads as
2011-11-08T09:11:12+00:00

I retract the bit about time offset in the "sufficiently lossless" point.

> I think we should revisit that presumption of need, and separate it from desire. It certainly seems that offset-preserving times are a black art not yet mastered even after years of experience in date/time library development, as evidenced by last month's discussions on the threeten list:
>
> http://sourceforge.net/mailarchive/forum.php?forum_name=threeten-develop&max_rows=25&style=nested&viewmonth=201110
>
> One way of considering it is that offset preservation is a value-add, subject to numerous interpretations, and if one wants it one has to buy into a specific methodology for preserving it (especially once any operations are carried out), doing so by choosing a specific representation type.
>
> Perhaps we ought to limit the semantics of 'instant' to, well, instants. This greatly simplifies things. Yes, instants might (or might not) have less utility than date-time-offset-zone thingies, but are more precisely understood and used.
>
> That is to say, perhaps a proposal for instant should *explicitly not promise* offset preservation, while a future date-time-offset-thing might. It then is a representational detail of an instant type that it holds the instant in 2 parts and prints it again that way. If it matters to you, choose one that does that. This is more what the original thinking was in this proposal.
>
> In that case, the major shortcoming of j.u.Date lies in precision being limited to msecs. That is, one could consider j.u.Date a (widely recognized) semantic container for the long msecs instant and ignore the rest of it, if undesired. java.sql.Timestamp remedies the resolution issue, but has other issues in  deriving from Date but not really being interoperable with Date. (Another wonder of class design, i.e. it makes you wonder about class design :) From the javadoc:
>
> "The inheritance relationship between Timestamp and java.util.Date really denotes implementation inheritance, and not type inheritance."
>
> "really"? Meh.
>
> Without diving down that hole, the point here is that we've scope-creeped instant beyond the semantics of instant, and need to recognize and reconsider that.
>
>> Question: should the reader support for instant literal validate
>> against the syntax defined by Clojure before calling the  parser
>> function? (This would keep users of the instant literal "honest" WRT
>> the syntax.) Or is it more important to avoid that overhead? (We're
>> all adults here.)
>
>
> The rules might be - the parser is required to validate (and thus make the promise to help the user), freeing the the reader to (threaten to) redundantly validate at any random time (to keep the parser honest).
>
> Rich
>

Ben

Ben Smith-Mannschott

unread,
Nov 8, 2011, 4:34:11 PM11/8/11
to cloju...@googlegroups.com
On Tue, Nov 8, 2011 at 15:27, Rich Hickey <richh...@gmail.com> wrote:
>

I think I'm now closer to something usable:

https://github.com/bpsm/rfc3339datetime/blob/86ea5d044a37969af8849c075ad06aee19c9efbf/src/rfc3339datetime/core.clj

Parse is now pluggable, the sense that it takes a constructor function
which is called with the the result of the parse. The value returned
by the constructor is ultimately the result of the call to parse.

constructors are provided for the Java types java.util.Date,
java.util.Calendar and java.sql.Timestamp.

Functions are provided to format each of these three types as rfc3339
timestamps, with some possible minor lossage of information depending
on which type is chosen:

- Calendar preserves only milliseconds
- Date preserves only milliseconds and forgets the timezone
offset the original literal used.
- Timestamp preserves nanoseconds (yea), but forgets the
timezone.

When I say "forgets the timezone" I mean that if we format the object
in question back to an rfc3339 timestamp, we'll end up with something
like:

2000-01-02T12:00:00+00:00 and it won't be possible to tell if the
original literal ended with 13:00:00+01:00 or maybe 06:00:00-06:00.
(All three forms specify the same instant in time, so there's no real
lossage.)

The next thing I'd like to try is to use Stu's patch as guidance to
provide an implementation of his #@ literal using the code above
instead of sneakily linking in Joda-time.

// Ben

Rich Hickey

unread,
Nov 8, 2011, 10:02:31 PM11/8/11
to cloju...@googlegroups.com

Cool, thanks!

A couple of points.

The comments indicate we differ from RFC 3339 in not supporting date and time, but I don't read 3339 as supporting them stand-alone, only as subcomponents of date-time, i.e. 3339 only defines date-time. We do differ in supporting eliding trailing, and that covers offsets as well.

The pluggable ctors are function of the components, not what appeared to the reader. If you want to anticipate the model I proposed for a full tag system, they all would have to be functions of string. The string->components thing could be a public shared helper fn though.

Thanks again,

Rich


Ben Smith-Mannschott

unread,
Nov 9, 2011, 1:11:33 AM11/9/11
to cloju...@googlegroups.com

Good catch. I'll amend the comments accordingly.

> The pluggable ctors are function of the components, not what appeared to the reader. If you want to anticipate the model I proposed for a full tag system, they all would have to be functions of string. The string->components thing could be a public shared helper fn though.

Yes, the plugability I mentioned is an implementation detail. It's not
related to the plugability of custom parsers for tagged data types in
LispReader that's been discussed on this thread.

The code I posted is just intended as a building block. roughly:

(def parse-rfc3339-as-date-for-reader
(partial parse construct-date))

> Thanks again,
>
> Rich

glad to help,
Ben

Ben Smith-Mannschott

unread,
Nov 9, 2011, 3:28:20 PM11/9/11
to cloju...@googlegroups.com
On Wed, Nov 9, 2011 at 04:02, Rich Hickey <richh...@gmail.com> wrote:
>

Yes, I'd like to anticipate proposed tag system, though I haven't
thought about it deeply enough to be sure that I can pull it off. I
may fall back to a #@xxxx… as a first proof of concept.

At first blush, it looks to me like the syntax you sketched earlier in
this thread conflicts with type/record literals provided by 1.3
(CtorReader).

> So, in lieu of special support for instants, we could do:
>

> #instant "1985-04-12T23:20:50.52Z"
>
> and things like:
>
> #ints [0 1 3 5 7]
>

> #my.ns.foo {:bar 123 :baz [5 6 7]}

seems to conflict with: #myns.MyRecord{:a 1, :b 2}, #myns.MyRecord[ 1 2 ]

http://dev.clojure.org/display/design/defrecord+improvements

Or at least it's not obvious to me how I would disambiguate robustly.

Unless... (thinking here)...

#myns.MyRecord

Name has more than one segment. Last segment is separated by "."
-> myns.MyRecord names a class.

#instant "2001"

Name has exactly one segment
-> clojure.core/instant is a dynamic var bindable to a to a parse
procedure that expects a string (in this particular example).

#foo.bar/thing {:x 1}

Name has more than one segment. The final segment is separated by "/".
-> foo.bar/thing is a dynamic var bindable to a parse procedure that
expects a map (in this example).

I guess I could disambiguate it that way. Is this what you had in mind?

// Ben


> Thanks again,
>
> Rich

Rich Hickey

unread,
Nov 9, 2011, 3:57:32 PM11/9/11
to cloju...@googlegroups.com

Sorry, I didn't intend to send you off trying to go further, just sussing out how what you had already done would plug in.

Let's hold off on #tag etc. It is not yet decided that will be the syntax. But yes, were it to be, we'd have to reconcile with records.

Thanks,

Rich

Ben Smith-Mannschott

unread,
Nov 9, 2011, 4:12:57 PM11/9/11
to cloju...@googlegroups.com

Ok. I'll take what I've got and integrate it with's Stu's existing #@
syntax as a proof of concept. I generally find having something
concrete to play with helpful.

// Ben

Ben Smith-Mannschott

unread,
Nov 12, 2011, 4:13:42 PM11/12/11
to cloju...@googlegroups.com
I've attached a patch to http://dev.clojure.org/jira/browse/CLJ-871

It implements instant literals of the form
#@yyyy-mm-ddThh:mm:ss.fff+hh:mm using only classes available in the
JRE.

// Ben

Reply all
Reply to author
Forward
0 new messages