Suggestion for choro.clj
Sean Devlin
I just saw a post linking to chrono. Cool stuff. I'd like to throw an
idea out there.
It would be really cool if the date functions worked on a larger set
of inputs, like the way duck-streams works on several stream types,
and the sequence abstraction makes it easy to do all sorts of stuff.
It'd really help w/ interacting legacy code. Perhaps it could auto-
convert the following datatypes to the calendar type:
java.util.Date
java.util.GregorianCalendar
javax.sql.Date
java.lang.Long
I've created a mutlimethod that implements the following graph:
http://cloud.github.com/downloads/francoisdevlin/devlinsf-clojure-utils/date-utils_design.pdf
I could submit a patch if you're interested.
Sean Devlin
David Edgar Liebke
when you're ready.
David
Bradford Cross
Sean Devlin
and I'm tempted to start moving things around & clean stuff up. Could
someone tell me what the high level goals are for this library? That
would make it easier for me to determine what to do.
Sean
On Jan 16, 12:34 pm, Bradford Cross <bradford.n.cr...@gmail.com>
wrote:
> Really cool work Sean.
>
> On Sat, Jan 16, 2010 at 9:17 AM, David Edgar Liebke <lie...@gmail.com>wrote:
>
>
>
> > Sean, I'm absolutely interested, go ahead and submit a pull request
> > when you're ready.
>
> > David
>
> > On Sat, Jan 16, 2010 at 12:07 PM, Sean Devlin <francoisdev...@gmail.com>
> > wrote:
> > > Hey all,
> > > I just saw a post linking to chrono. Cool stuff. I'd like to throw an
> > > idea out there.
>
> > > It would be really cool if the date functions worked on a larger set
> > > of inputs, like the way duck-streams works on several stream types,
> > > and the sequence abstraction makes it easy to do all sorts of stuff.
> > > It'd really help w/ interacting legacy code. Perhaps it could auto-
> > > convert the following datatypes to the calendar type:
>
> > > java.util.Date
> > > java.util.GregorianCalendar
> > > javax.sql.Date
> > > java.lang.Long
>
> > > I've created a mutlimethod that implements the following graph:
>
> >http://cloud.github.com/downloads/francoisdevlin/devlinsf-clojure-uti...
Bradford Cross
Hey, I'm taking a look around chrono.clj. It's a little disorganized,
and I'm tempted to start moving things around & clean stuff up. Could
someone tell me what the high level goals are for this library?
Sane date/time support for clojure/jvm. This has historically been a terrible area. We want to encapsulate the badness, and provide a simple/intuitive api.
That's it.
Edmund
While this is under discussion I'd like to offer to add some basic
timezone support. For my own stuff I wrap Joda time thinly and use
UTC for all internal representation and storage which makes this
trivial, as shown below. Perhaps we could do something similar (or
entirely different) for Chrono ?
Edmund
;;
-----------------------------------------------------------------------------
;; Create
(defn str-to-time
"Return a time object based a passed in [string] with an optional
[zone]"
;; If no time zone assume UTC
([string]
(str-to-time string "UTC"))
;; If a zone is specified the use it
([string zone]
(.parseDateTime
(.withZone sql-fmt (DateTimeZone/forID zone))
string)))
(defn to-utc
"Convert a datetime to UTC in which all calculations should be
performed."
[dt]
(.withZone dt DateTimeZone/UTC))
;; Main create function
(def
#^{:doc "Main CreateTime method. See str-to-time for arguments."}
ct
(comp to-utc str-to-time))
(def
#^{:doc "Format that MySQL expects on input datetimes. Ensures UTC.
Use as (.print sql-fmt x)"}
sql-fmt
(.withZone
(org.joda.time.format.DateTimeFormat/forPatterns
"yyyy-MM-dd' 'HH:mm:ss")
DateTimeZone/UTC))
(def
#^{:doc "The datetime format that MySQL returns for datetimes"}
tst-fmt
(.withZone
(org.joda.time.format.DateTimeFormat/forPattern
"yyyy-MM-dd' 'HH:mm:ss.0")
DateTimeZone/UTC))
On Jan 19, 6:27 am, Bradford Cross <bradford.n.cr...@gmail.com> wrote:
Sean Devlin
This is a great suggestion. Part of the patch I'm working on will be to add first rate ISO8601 support & being able to create a time object from a hash map. Both of this should make it easier to implement timezone fns. I'll be sure to include a few tz fns in my patch, too.
I'll push something to my git fork tonight. I'm breaking a lot of stuff, but I hope it's for the better. I'd like to get something out there for people to review ASAP, while it's still "In Progress".
Sean
Sean Devlin
so I'd suggest reviewing it before pulling.
http://github.com/francoisdevlin/incanter
As you may have guessed, I started on this project for episode 7. You
can see the basics here:
1. to-ms
Everything is centered around a multimethod, to-ms. This multimethod
works on the following types:
Clojure Maps
java.util.Date
java.util.Calendar
java.sql.Timestamp
java.lang.Long
org.joda.time.DateTime
java.lang.String
If a map is passed in, use the keys specified in time-keys.
When a string is passed, an optional keyword can be passed to select a
parser. The available parsers can be found in the formatters hash-
map. Call display-formats to see the output for each date formatter.
If a keyword is NOT passed to the string method, the keyword bound to
default-format is used.
2. Date constructors
There are the following date constructor fns
date
sql-ts
greg-cal
joda
time-map
str-time
Each of these fns accept the same type of inputs as to-ms, because the
multimethod is called internally.
The str-time behaves slightly differently. It uses the optionally
supplied keyword to define the output, and again defaults to default-
format. i.e.
user=>(str-time a-time :a-format)
"20100119"
This fn does not handle string input well. If you need to pass it a
string, I recommend using an intermediate fn for now.
user=>(str-time (joda input-string :format-a) :format-b)
"A sucessfully converted string"
3. Time Zones
I haven't finished time zone stuff yet.
4. Predicates
This uses the following predicates:
* compare-time
* before?
* after?
* valid-range?
* is-within?
* are-overlapping?
The implementation is completely different than before. I haven't
formally tested yet.
5. Relative time fns
See the doc strings for the use of these fns
* period
* period-between
* later
* earlier
6. Interval utils
See the values in the variable time-keys to determine which fields its
possible to reset to.
* start-of
* end-of
* date-seq
Conclusion
Again, sorry about the total re-write. I understand if this is a show
stopper. Anyway, I just wanted to get it out there so everyone can
review it.
Hope you like it,
Sean
David Edgar Liebke
This is great, and I'm looking forward to merging it, once Bradford
has had a chance to make sure it works with FlightCaster's existing
code base.
Unfortunately, I'm having a bit of a problem building your branch. It
looks like 'earlier?' is missing, but it is still be referenced. Do
you intend to use 'before?' instead? Did the branch pass all the
chrono tests for you?
David
Sean Devlin
I was more interested in getting buy-in for api signatures, names, etc. Some of the details still need to be finished.
Sean
Bradford Cross
Ah, I should have been more explicit about something. This is a *preview release*. I haven't done any real testing yet.
I was more interested in getting buy-in for api signatures, names, etc. Some of the details still need to be finished.
Plumbing seems reasonable.
I wonder if we might be able to get buy with a single top level function to create a date rather than a different constructor function for each type?
I kind of feel like hiding the nasty heterogeneity of all the java date stuff is a big win.
On that note, if we have other fns with polymorphic dispatch in order to provide a consistant top level api for accessing date info -> time zone, y,m,d,h,m,s,ms etc. that might also be great.
Curious what others think (too much magic?) - but I am really into the idea of hiding the mess here and providing a single date constructor and a single top-level suite of date accessor and query fns.
For example, take a look at all the complexity that is hidden by the copy multimethod in clojure.contrib.duck-streams
Bradford Cross
Sean Devlin
Okay, just pushed another version. I was able to get it to build with
maven, but I had to temporarily comment out all of the tests in
chrono_test.clj. I also had to add a few aliases to get the other
tests to work. I'll update the tests in the next few days.
Renamed joda back to joda-date
As far as the code goes, I added a case for Integers, so the following
now works:
(date 2010 01 19)
(date 2010 01 19 0 0 10)
(date 2010 01 19 0 0 10 0)
Time zone support is coming soon.
You can use time-map to access the y, mo, d stuff, or call (bean (joda-
date)) if you need more. Currently I feel like the old accessor proxy
was overkill.
As to answer you question about the other top level constructors,
think of them as different sequence constructors. The to-ms fn is the
equivalent of seq. Things like date, joda-date, sql-ts are like vec,
hash-map, etc. Different tools for different jobs. sql-ts is damn
near useless until you have to interact w/JDBC. Then it's great.
I think the "duckiness" is baked in now. Specifically, use the
predicate, period tools, or interval tools to see it in action. Once
I add my new tests, then it should make more sense.
Oh, and I used juxt so this library now relies on Clojure 1.1. Is
this okay? If so, can I write my new tests in terms of clojure.test
instead?
That should do it for now. I"m sure I'm missing something. Have fun.
Sean
On Jan 19, 10:29 pm, Bradford Cross <bradford.n.cr...@gmail.com>
wrote:
> > > On Tue, Jan 19, 2010 at 9:03 PM, Sean Devlin <francoisdev...@gmail.com>
Bradford Cross
Edmund
David Edgar Liebke
> this okay? If so, can I write my new tests in terms of clojure.test
> instead?
>
Yep, Clojure 1.1 is already a dependency.
Edmund
As far as timezone support goes it would be pretty simple, I think:
In general its much better to specify geographical locations as simple
hour offsets are not usually what you actually want. So I'd suggest
something along the lines of
for the timzone constructor
(defn tz
"Creates a Joda Time Zone, either based on hour offset or financial
centre. Default to UTC"
([] DateTimeZone/UTC)
([z]
(cond
(integer? z) (DateTimeZone/forOffsetHours z)
(string? z) (DateTimeZone/forID z)
true (DateTimeZone/UTC))))
Then the base representation classes could take an optional third
param:
(defmethod to-ms String
([s] (to-ms s default-format))
([s f] (to-ms (.parseDateTime (formatters f) s)))
([s f z] (to-ms (.parseDateTime (.withZone (formatters f) (tz z))
s))))
Thoughts ?
Edmund
> ...
>
> read more »
Sean Devlin
Thanks for the tips on creating a time zone. I'll be adding this to
the code tonight. It'll definitely be a good addition for the integer
& maps case.
However, I'm a little concerned about adding it to the String case.
This is because time zone information is already part of the string in
many of the formats. I'm worried that overriding the time zone info
might be confusing. I'm open to a counter argument, though.
Sean
> ...
>
> read more »
Edmund Jackson
I have no deep understanding of these issues, but various burns in previous times have caused me to write code this way. Of course, that doesn't make me correct ! My reasons are:
1. When processing date from a region usually the time is not properly timezone designated in the set. To parse I'd rather pass the zone information as an argument to the parsing function rather than dabble with the string itself to insert it. So I rather make a custom parses ala (def my-parser (partial default-parser timezone)) than append "+1" to the string "2010-01-20 18:02:01".
2. Even if it is a good idea, specifying a proper timezone in the string is tricky. For instance, in ISO8601 you can specify +n, where n is an offset from UTC. While easy, this is insufficient because, as a result of DST and historical changes of timezone zoning, a particular spot on Earth will have a different offset at different times of the year or before/after a particular point. Its much more robust to use specifiers like "America/New_York" as the underlying tz database takes care of this madness for you. I do not know how to specify these properly in the current formats.
Its likely I'm just ignorant on the correct ways to solve these issues and would love to hear them :)
Edmund
Edmund
"The future is here. It's just not widely distributed yet"
-- Gibson
Sean Devlin
carlitos
On Jan 20, 3:22 pm, Sean Devlin <francoisdev...@gmail.com> wrote:
> Thanks for the tips on creating a time zone. I'll be adding this to
> the code tonight. It'll definitely be a good addition for the integer
> & maps case.
Hi Sean,
there is a bug in the following method definition (the "h" is missing
from the call to to-ms in the second case):
(defmethod to-ms Integer
([y mo d] (to-ms y mo d 0 0 0))
([y mo d h mi s] (to-ms y mo d mi s 0))
([y mo d h mi s ms & r]
(to-ms (DateTime. y mo d h mi s ms))))
Cheers,
Carlos
PS: I understand this is the latest version you've published:
http://github.com/francoisdevlin/incanter/blob/master/modules/incanter-chrono/src/main/clojure/incanter/chrono.clj
Sean Devlin
Sean Devlin
zone code that is making me reconsider the design. Relying on the
long representation is a bad choice, because it loses tz information.
Working on a fix. The final result *should* be just as clean.
Sean
> >http://github.com/francoisdevlin/incanter/blob/master/modules/incante...
Sean Devlin
I finally had some time to work on this tonight. Let's hear it for
huge snowstorms!
Anyway, you can find my latest work here:
http://github.com/francoisdevlin/incanter
Here's what I've changed:
Added a timezone multimethod that creates Joda DateTimeZone objects.
It accepts the following as input:
* DateTimeZone
* java.util.TimeZone
* DateTime
* Integer offsets
* String representations of time per the iso standard (e.g. "America/
New_York")
* Date
* Calendar objects
Add a java-tz fn that accepts the same inputs as tz, but returns a
java.util.Timezone object.
All of the fns centered around to-ms are now centered around a new
multimethod, to-joda*. This new multimethod does a much better job of
preserving tz info. Also, it is now possible to specify time zone
information for all of the types that support it. For example,
user=>(joda-time) ;creates a joda-time object now, local tz
user=>(joda-time utc) ;creates a joda-time object now, at utc
user=>(joda-time 1) ;creates a joda-time object now, at European time
user=>(joda-time "America/New_York") ;creates a joda-time object now,
at NY time
;Also works for greg-cal
user=>(greg-cal "America/New_York"); creates a greg-cal in NY
user=>(greg-cal 2008 1 1"America/New_York"); creates a greg-cal in NY
user=>(greg-cal {:year 2008 :month 1:day 1} -5); creates a greg-cal,
-5 UTC
;Chain them together
user=>(greg-cal (joda-time "America/New_York") utc) ;creates a greg
cal in NY, converts it to a joda UTC
user=>(greg-cal (joda-time "America/New_York") (joda-time 1)) ;creates
a greg cal in NY, converts it to a joda Europe
I also moved before? & after? back to earlier? & later?
Added the following convenience fns for creating periods
year
month
day
hour
minute
sec
;; Plural Support
years
months
days
hours
minutes
secs
That's it for now folks. I could use some help getting tests in
order, if there are any volunteers. I'll see what I can do tomorrow.
Sean
Edmund Jackson
Edmund
Edmund
Bradford Cross
I could use some help getting tests in order, if there are any volunteers.
Is this that you are unclear how to do good testing?
Sean Devlin
Bradford Cross
OK, looks like you have some help. I'll review once the tests are in place.
On Feb 10, 2010 8:21 AM, "Sean Devlin" <francoi...@gmail.com> wrote:
A little of that, and I find testing is something that really does benefit from many people adding cases. They "design by committee" that back a bad library also makes a great test suite.
Sean
On Feb 10, 2010, at 11:14 AM, Bradford Cross wrote:
>
>
> On Fri, Feb 5, 2010 at 9:24 PM, Sean Dev...