ANN: jedi-time 0.1.4

[dimitris]

Hi folks,

The first public release of `jedi-time` should be hitting clojars any
minute now. I am traveling next week so may be slow to reply to
feedback/bugs/PRs...

https://github.com/jimpil/jedi-time


Kind regards,

Dimitris

[Sean Corfield]

This is very cool but I would strongly recommend you try using this with REBL so you can figure out how to make the `nav` part work in a more natural way. 

nav is intended to work with a key and value (from the datafied structure), but your nav expects special values so it doesn't work with REBL. 

You can put (java.time.Instant/now) into REBL and your datafication produces a great data representation, but you can't navigate into it using the keys (and values) of the data structure itself. You can put :format into the nav-> bar and it defaults to a format you can get a string back, but none of the other nav calls will work.

You might consider combining the :format key with the actual format, e.g., :format-iso, :format-yy-MM-dd and if the key is something your don't recognize, just let it behave like regular data navigation.

I think you're trying to do too much with nav, beyond "navigation". I think you could split some of the "clever" navigation out into a transform function that takes a datafied time and produces a new datafied time, and then let nav do the "conversion" back to Java objects. You've complected the transforms and the conversion right now.

If you're on Slack, I'm happy to DM about this in more detail (when you're back from traveling).

Sean

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/8dbb9c5b-0ab0-fc76-6dc6-5e75b93d452a%40gmail.com.

--

Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles Networks, LLC. -- https://worldsinglesnetworks.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[Dimitrios Jim Piliouras]

Hi Sean,

 

Admittedly, I’ve never used REBL, and I did struggle with the shape and name of the `nav` arguments...

 

In particular I’m struggling to understand why would anyone use `nav` to navigate to a key that already exists in the map...Can’t we just use `get` or `get-in`?

You used the :format as an example, which works with nil, :iso, or a String pattern as the last arg to nav. But again, :format is NOT in the datafied representation.

 

In essence, I’ve tried to use `nav` to navigate to things that can be expensive and don’t necessarily belong in the actual datafied representation.

If the second argument to `nav`,  is expected to be a key already present in the map, then I really don’t understand what is the point of `nav`.

 

kind regards,

Dimitris

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CAD4thx-dwJkYYsGDWnD%3DAc7oaNBHqykGPzYhTHdWQRJmbk1DEw%40mail.gmail.com.

 

[Dimitrios Jim Piliouras]

I think I need to clarify  the following points in my head:

 

1. Was `nav` conceived to complement `datafy`  specifically in the context of REBL (and not really anywhere else)?
2. How useful is a datafied datetime (outside the context of REBL), if it can’t be easily formatted, queried, shifted or converted to an alternate version?
3. Why would I ever navigate the existing keys of a map via `nav` (as opposed to the standard functions)?

 

Kind regards,

Dimitris

[Sean Corfield]

Think of it as a square:

You start with an object of some sort (left side) -> datafy -> turns it into pure Clojure data (including metadata). (right side)

Given pure Clojure data, you can navigate through it with get etc and you stay in the right side (pure data).

Given that pure Clojure data, you can navigate back to the left hand wide with nav, mimicking how get etc work.

So datafy is L -> R, get is R -> R, nav is R -> L on a "diagonal" that takes you back to the object world on the left, corresponding to the place on the right that you'd get to via get etc.

See if this blog post helps https://corfield.org/blog/2018/12/03/datafy-nav/

 

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/5e3694b9.1c69fb81.8a875.d5ea%40mx.google.com.

[Dimitrios Jim Piliouras]

This is what I've done but it contradicts what we said earlier...

If I navigate to some existing key and it gives me back a Java object, then it means that the datafied representation had a key pointing to non data! 

I have read your blog post multiple times ;), but I think the situation you're describing  with the foreign keys is rather unique...

The datafied datetime cannot possibly include all its possible formats, nor all the possible alternatives - that would be extremely wasteful and meaningless the way I see it.  

Let's take an Instant as an example...it datafies to map of two keys (:epoch, :second). Does it make sense to add a :format-iso key in there pointing to a String? Is there any point navigating to that key? Is there any point navigating to :epoch or :second? The answer is no, right? Is there a point in navigating to :zoned-datetime given a zone id? I would think yes...

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CAD4thx8GRuxeGd-MrMSti%2BWRV1neRFOLWNh08xpb-Qrmya0kZw%40mail.gmail.com.

[Sean Corfield]

You're misunderstanding me. I'll try again.

I'm not saying you can't navigate to keys that don't exist in the data -- but since there would be no corresponding value, the nav call would be (nav coll k nil) essentially.

If (get coll k) produces some value v, then (nav coll k v) will take you from the right side (pure data) to the left side (objects) to the object that "corresponds" to the equivalent navigation on the right (i.e., within the data).

object -> datafy -> pure data

pure data -> get etc -> new pure data

pure data -> nav -> new object "corresponding" to new pure data

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CAE3KzwJ9QuTB7gOMdyja5LyczStJzgsEv9c%3D3_cM%2BoY2_ppPRA%40mail.gmail.com.

[dimitris]

Hi Sean,

I'm back home and trying to understand/internalize this...Unfortunately, this kind of (flat & arg-less) navigation is not going to be very useful for the majority of java.time (datafied) objects. That is for two reasons... First of all the datafied maps I'm returning are nested. This means that for example to get to the `YearMonth` object, you would need to navigate to the [:year :month] path, and in the absence of `nav-in` this is somewhat awkward. Secondly, most of the interesting/useful conversions (in the context of date-times), almost always requires some sort of argument (e.g. `Instant` to `LocalDateTime`), and so if the last arg to `nav` has to be either nil (for missing keys), or match the actual value in the map, then there is no room left for arguments.

It is true that I'm probably trying to do too much with `nav`, but now that I'm understanding its purpose better, I get the feeling that it's not going to be as useful as I originally thought (in the context of this lib). Yes, I can pull all the clever stuff into distinct functions, but ultimately for `nav` to be useful I would have to either:

1. Change the datafied representation to something flat, OR

2. accept that navigating to pure data (via `get-in`) will be done with real paths (e.g. `[:year :month]`), whereas navigating to objects (via `nav`) will be done with bogus keys (e.g. `:month-of-year`).

As things stand (with my current nested representation), only LocalDate, LocalDateTime, OffsetDateTime & ZonedDateTime can have useful navigations:

- LocalDate => :week-day , :year-month

- LocalDateTime => :local-date, :local-time

- OffsetDateTime => :local-datetime, :instant

- ZonedDateTime => :offset-datetime, :local-datetime, :instant

That is pretty much it in terms of `nav`...

Does that make (more) sense?

Many thanks in advance...

Dimitris

ps:  Sean I can be on slack but with my work email

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CAD4thx_1%2BwQi6_-BkiPLM%3DUvAm84wWF3Ob_%2B%3DZ2nZx-9uHj2fQ%40mail.gmail.com.

[Sean Corfield]

Yes, I agree with all of that I think.

For nested navigation, consider that (get-in data [:year :month) is equivalent to (get data :month (get data :year)) so you could nav one step at a time. Calling nav (& then datafy) on the intermediate steps would just bring you back to the data world at the same point as the inner (top-level) get in that case.

nav-in would be a strange operation since it would need to call datafy after each step to get the arguments needed for the next nav call. REBL provides nav-> which does this behind the scenes while it is threading data through the pipeline of nav operations (so there is a precedent).

Even with an equivalent to nav-in (or nav->) I think that using datafy/nav on Java Time objects may be an incomplete mapping -- and probably somewhat hard to work with. When you first posted, I was more focused on the confusion using non-core datafy/nav would be and interop with REBL -- I didn't look too deep into the _actual_ navigation you were proposing, sorry. 

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/470f811f-21cb-74f4-bd85-4a3fb5792fef%40gmail.com.

[dimitris]

Hi again Sean and folks,

I've had another stub at this, mostly by flattening the model but also by separating navigation from query/comparing/formatting etc. Most datafied representations are now navigable on their keys (returning either a Java object or some base value), and some on certain extra keys.

I believe this version will play much nicer with REBL, and all the wrapper style fns now live in `jedi-time.datafied` namespace and backed-by `jedi-time.protocols`. In other words, navigation is now purely for traversing the graph - everything else has its own protocol.

Unfortunately, now Instant can't really navigate to anything interesting (it falls back to data navigation via `get`).

I would be very interested in feedback on the new approach (git-sha: 8e756ecb71bbfa0b081e00d71a21c47037f1eae4). If anything, it separates navigation from the other capabilities, and makes sure that there is always a navigation path to either upgrade (by making assumptions), or downgrade (by losing information) your datafied representation .

As always, thanks in advance...

Dimitris

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CAD4thx_w554izXzEXznMmT9ac6-1iv9b9tKLdU-yJz1BO4NA%3Dg%40mail.gmail.com.

[Sean Corfield]

That is starting to work nicely with REBL for the basic datafy/nav functionality. Thank you!

Once I'd verified that, I tried this: (assoc (d/datafy (java.time.LocalDateTime/now)) :format :iso) hoping that if I nav'd to the new :format key, it would "navigate" to a formatted version but it didn't. Then I realized you hadn't added support for that in your nav implementation (you're assuming folks explicitly use your protocol-based functions on the original objects, I think?). That would be the next logical step: being able to augment a datafied date/time with additional key/value pairs that would be recognized by the implementation of nav. The datafication wouldn't need to add these keys (although, if you wanted an obvious "default" behavior, you could add some) but the navigation would need to recognize them and do the appropriate calculation/conversion. Does that make sense?

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/c9e611ee-6064-c1b2-def4-b777448a21a7%40gmail.com.

[dimitris]

Ok, I'm glad datafy/nav works as expected, but the rest of you response confuses me again :)

The reason I removed support for navigating to `:format` was because we established in previous conversations that I was doing too much with `nav`. I was using it for comparing, shifting, formatting & converting. I've put all of that behind separate protocols and extending them via metadata (similarly to `nav`). You now seem to implying going back to "doing too much" with `nav` - or have I misunderstood you (again)?

Moreover, if `:format` is recognised by `nav`, you wouldn't need to `assoc` it in order to use it - it would just work (as it used to work before my commit last night), so that bit confuses me too. In any case, I do massively appreciate the time you're putting into this...There is great confusion on-line about datafy/nav, whether they are useful on their own (vs being complementary to each other), the right arguments, where we draw the line in terms of doing too much etc etc. 

Thanks again :)

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CAD4thx8tiu__4SjEJT_mK7B1kaJvjO2fCEeM9DYBgOrmz5sJ1A%40mail.gmail.com.

[Sean Corfield]

> Ok, I'm glad datafy/nav works as expected, but the rest of you response confuses me again :)

 

Sorry ☹

 

> You now seem to implying going back to "doing too much" with `nav` - or have I misunderstood you (again)?

 

I’m suggesting that if you add certain key/value pairs to the datafied Java Time values, nav could recognize those as navigation from data to “stuff”. This gets you much closer to your original concept while staying within the datafy/nav confines.

 

Let’s take a concrete example:

 

(d/datafy (java.time.LocalDateTime/now)) -> a hash map like this:

 

{:second-milli 114, :hour 14, :second-micro 114866, :second 47,
:month {:name "FEBRUARY", :value 2, :length 29, :day 9},
:year {:value 2020, :leap? true, :length 366, :week 6, :day 40},
:weekday {:name "SUNDAY", :value 7}, :second-nano 114866000, :minute 42}

 

(get data :month) -> {:name "FEBRUARY", :value 2, :length 29, :day 9}

 

(nav data :month (get data :month)) -> a java.time.Month object for “FEBRUARY” (losing the day – which suggests :day should be part of the datafication separately BTW)

 

Now lets look for :format:

 

(get data :format) -> nil

 

(nav data :format (get data :format)) -> nil

 

As expected. But it’s data and we can augment it with other keys:

 

(let [data’ (assoc data :format :iso)] (get data’ :format)) -> :iso

 

At this point, we have a key and a value so we can call nav with those:

 

(let [data’ (assoc data :format :iso)] (nav data’ :format (get data’ :format)) -> could navigate to an ISO-formatted version of the local date time

 

> Moreover, if `:format` is recognised by `nav`, you wouldn't need to `assoc` it in order to use it

 

What I didn’t like about the original was that your nav function accepted arbitrary keys and values that weren’t related to the datait was “magic” and had extended navigation arbitrarily outside of the Clojure navigation of the data (see the nav docstring below):. I’m not suggesting that all of your original functionality maps down naturally to get/nav like this – I don’t know how I would feel about the add/subtract time periods being done this way but if you take data (the hash map produced by datafying a Java Time object) and manipulate the data in ways that preserves the nav metadata, then calling nav on that new data could do more things than calling nav on the original data, if it follows the get/nav path that is intended by the datafy/nav mappings:

 

Foo -> datafy -> data

 

(get data k) -> v

 

(nav data k v) -> v or some new Foo or…

 

But the expectation is that nav will get called as if (nav data k (get data k)) or (nav data nil v) if there’s no natural key/index associated with the value v.

 

Here’s the docstring for nav – I’ve added some emphasis:

 

“Returns (possibly transformed) v in the context of coll and k (a

  key/index or nil). Callers should attempt to provide the key/index

  context k for Indexed/Associative/ILookup colls if possible, but not

  to fabricate one e.g. for sequences (pass nil). nav returns the

  value of clojure.core.protocols/nav.”

 

Hopefully this clarifies what I was trying to express, but I’m happy to have another few goes around if we’re not both there yet 😊

 

Sean Corfield -- (904) 302-SEAN

An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/0d6c3c50-481b-cfb3-3f6b-161bd372ea6f%40gmail.com.

 

[dimitris]

On 10/02/2020 20:45, Sean Corfield wrote:
> I’m suggesting that if you add certain key/value pairs to the datafied
> Java Time values, nav could recognize those as navigation from data to
> “stuff”. This gets you much closer to your original concept while
> staying within the datafy/nav confines.

The keys recognised by `nav` have been manually hard-coded to match the
keys in the map (in my current implementation), so if I add :format it
will always be recognised regardless of whether it exists in the map.
The only way to have `nav` selectively recognise keys based on their
existence in the map, is to put its whole logic behind a `contains?`
check. However that sounds counter-intuitive/productive...

> What I didn’t like about the original was that your nav function
> accepted arbitrary keys and values that weren’t related to the datait
> was “magic” and had extended navigation arbitrarily outside of the
> Clojure navigation of the data

I'm surprised you said that right after having shown exactly how I had
navigation for :format up until yesterday. What makes :format not
arbitrary or magic? How could `nav` possibly know that a new key has
been added (or removed/updated for that matter), and dynamically add the
corresponding behaviour?

> I don’t know how I would feel about the add/subtract time periods
> being done this way

Again, I find that interesting because if there is one operation that
naturally leads you from data to another Java object, that is shifting.
For example, the :format capability we're discussing leads you to a
String, whereas `(nav datafied :+ [2 :weeks])` leads you back to a
(datafiable) java object (i.e. a nicer fit for going from data to
stuff). The `:at-zone` and `:at-offset` navigation paths were similarly
good fits for the same reason. To me, :format although convenient and
all, is totally arbitrary.

> if you take data (the hash map produced by datafying a Java Time
> object) and manipulate the data in ways that preserves the nav
> metadata, then calling nav on that new data could do more things than
> calling nav on the original data

I honestly don't see how that is possible in the open way that you seem
to imply...Someone needs to define upfront what the `nav` capabilities
will be (what keys will be recognised), and those are not tied in any
way to the keys/values in the map at any given point in time. As I said
in my first point above, one could manually force that a navigation path
only fires if the key is actually contained in the data, but that sounds
like an anti-pattern if I'm honest.

I would love to understand a bit deeper why you thought that my original
nav keys felt arbitrary and magic, whereas you clearly think that
:format isn't...To me they all are, or none are.


Thanks again...

[Sean Corfield]

> The only way to have `nav` selectively recognise keys based on their  existence in the map, is to put its whole logic behind a `contains?`  check. However that sounds counter-intuitive/productive...

 

Remember that nav is called with the key _and the value_ so it will be passed whatever value is associated with the key in the data structure. So (nav data :format nil) means either the key exists with the value nil or the key doesn’t exist, but either way nil isn’t a valid format so your nav should just return the value unchanged.

 

If (get data k) produces v, then nav will be called as (nav data k (get data k)) – if v is part of data via some other “path”, then nav will be called as (nav data nil v) so you always get v from data and you may get k from data. The key, if passed, comes from the data structure – not some fabricated key.

 

If I do this:

 

(assoc (d/datafy (java.time.LocalDateTime/now)) :month {:name "MARCH" :value 3 :length 31})

 

When I navigate through the fields of that data structure to :month, I should get a java.time.Month for March – because that’s the value passed into nav. Your current implementation still returns February. That’s why nav should pay attention to the value v that is passed in (again, per the docstring, nav “Returns (possibly transformed) v, in the context of coll and k”).

 

The nav docstring:

 

“Returns (possibly transformed) v in the context of coll and k (a

  key/index or nil). Callers should attempt to provide the key/index

  context k for Indexed/Associative/ILookup colls if possible, but not

  to fabricate one e.g. for sequences (pass nil). nav returns the

  value of clojure.core.protocols/nav.”

 

Sean Corfield -- (904) 302-SEAN

An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

 

From: dimitris
Sent: Monday, February 10, 2020 2:20 PM
To: clo...@googlegroups.com
Subject: Re: ANN: jedi-time 0.1.4

 

On 10/02/2020 20:45, Sean Corfield wrote:

--

You received this message because you are subscribed to the Google

Groups "Clojure" group.

To post to this group, send email to clo...@googlegroups.com

Note that posts from new members are moderated - please be patient with your first post.

To unsubscribe from this group, send email to

clojure+u...@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/clojure?hl=en

---

You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/4bdcef09-71db-b67d-796f-604e209f8a10%40gmail.com.