Modelling in Clojure

[Tom Oram]

Hello Clojure people,

First up, apologies because this is going to be a long message. Howver, if you do have the time to read and respond, then I would be extremely grateful!

Recently I've decided to give Clojure a proper go. Over the past year or so I've paid it a bit of attention: I've read books all about how to use it and I've spent a bit of time working through tutorials to create little web apps and so on. I understand the language (although not to any great depth yet), but I'm still unsure about the best approaches to actually working with it.

I've got many years of OOP experience, and I'm a big fan of principles like SOLID and approaches like DDD. What I want to do now is, try and learn how to build well designed models using Clojure, while using best practices! I've started having a bit of a crack at it by building a simple app, but I'm finding myself trying to transfer a lot of my existing OOP techniques into Clojure. It's working, but I'm wandering if I'm overdoing it or perhaps just not doing things "the Clojure way".

So, my very first question is are there any good books or resources on modelling, application architecture or best practices, using Clojure?

Next up, maps vs records. I've read that the typical approach is to use maps until the design starts to solidify and then you can move to records. Is the the general attitude of Clojure developers or do some like to dive straight in with records?

The next question is encapsulation: I've taken the approach that I'm kind of using namespaces like classes. If I want to create customer entity I create a namespace for it, then add a "make" function to it which returns a map of the right "shape".  I'm then shying away from accessing the map directly, and rather doing it through other methods in the namespace which take the instance as a parameter. E.g.

(ns app.domain.customer)


(defn make [name, email]
  {:name name
   :email email})


(defn get-name [customer]
  (:name customer))

Is this a reasonable approach? If not, what might be a better one?

This leads on to safety and defensive programming. Using the approach above, how much "type safety" is required? Obviously, in any API which is going to be used by other programmers, you've got to handle bad inputs well. But what is the Clojure approach in the "domain model"? Coming from the DDD mindset, where models are designed to be un-breakable, part of me wants to use records for all entities and typehint for them everywhere. However, I wander if the Clojure way is more about rigorous testing to make sure the wrong values don't get put in in the first place? Also, what about libraries like https://github.com/Prismatic/schema, this could be used in :pre conditions to be more explicit, is that a common thing?

Next up, how far do you go with creating "types" vs using primitives? Again, referring back to DDD, both the email and the name in the example above are suitable candidates for value objects. In Clojure, would you consider hiding the data behind a set of specialised functions to create, access and use it? Or would you just pass the primitive string/map/vector/whatever about and work on it directly? Example:

; Given
(defn make-customer [name email]
 {:name name, :email email})
 
; Examples
 
(def customer1 (make-customer "Tom" "em...@address.com"))
 
; vs...
 
(defn make-name [name] name)
 
(defn make-email [email] email)
 
(def customer2
 (make-customer (make-name "Tom")
 (make-email "em...@address.com")))

I think that's all I want to ask about now. I do have other questions about dependency inversions, but I'll leave that for another time. If you've read this far then thank you very very much! Also,I know that no one really wants to just sit and read though random peoples code, but, if you are interested in my (very very early stages) experimental project, then I've put in on https://github.com/tomphp/clojure-cocktails - any comments, critiques, PRs or questions would be really great!

Thanks you so much for your times and I look forward to any thoughts or suggestions!

Tom

[James Reeves]

Hi Tom,

Clojure views software architecture in a fundamentally different way to object orientated languages. In some areas, Clojure best practice is diametrically opposite to OOP best practice.

One of the key ideas of OOP is encapsulating data, but Clojure entirely rejects this. In Clojure, raw data structures are preferable over APIs. If you're approaching the idea of modelling by asking "How can I best hide this data behind an API", then you're swimming against the current.

Instead, Clojure encourages you model your data with data. In your examples, you start with a customer data structure:

    {:name "Alice", :email "al...@example.com"}

At this point, stop. You have your data model.

You may very well ask, what's to keep people adhering to this model? I could very well write:

    (assoc customer :name nil)

But that doesn't change the existing value, because values are immutable. All it does is derive a new data structure, and one that happens not to adhere to our definition of a customer. If you discard the idea of mutable data structures, you can also discard a lot of the ideas about restricting change to data.

If you want to ensure validity when dealing with data from an external source, you can use a predicate, or a library like Prismatic's Schema. Or if you want to ensure correctness within the code itself, you can use pre- and post-conditions, or use core.typed.

The advantage of not thinking about APIs is that you end up with a better API. With a raw data structure, your end users have access to all the data manipulation functions in Clojure Core, and the rest of the Clojure ecosystem.

- James

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark Engelberg]

On Thu, Oct 16, 2014 at 3:49 PM, James Reeves <ja...@booleanknot.com> wrote:

    {:name "Alice", :email "al...@example.com"}

At this point, stop. You have your data model.

I think that coming from OO, the most disconcerting piece of Clojure's philosophy is that it is relatively rare in Clojure to publish an API with data accessors like `get-name`.  The most common way to access the name would be to simply use
(:name person)

But let's say later you decide you want your data model to be {:first-name "Alice", :last-name "Beasley", :email "al...@example.com"}, and you want to change name to be a computed value that concatenates first and last names -- this is going to break all your existing code.  If name were only accessed throughout your code via the call `get-name`, then it would be trivial to make this change.

(Theoretically, it is possible to implement a revision to a person data structure by representing it as a custom map that reimplements keyword access so that (:name person) calls a function rather than does the usual keyword lookup, but I've never seen anyone do this, so I'm ignoring this possibility for the purpose of this discussion).

Many OO languages force, or encourage you stylistically, to hide everything behind setters and accessors, so you get for free the ability to change your mind about the data model later.

With Clojure, it is very common to allow your data model to be the API.  This puts some additional pressure to get it right the first time.  You have to explicitly think ahead about which fields might need to change in the future.

--Mark

P.S. To answer your specific questions about what is preferred, I'd go with your version that looks like:

(defn make-customer [name email]
 {:name name, :email email})
 


(def customer1 (make-customer "Tom" "em...@address.com"))



I wouldn't bother with records unless I needed to use protocols.  I would use a constructor like make-customer, but I would probably not use an accessor like get-name.  I think these would be the most common choices in the Clojure community (although other choices are also valid).

[Sean Corfield]

On Oct 16, 2014, at 5:08 PM, Mark Engelberg <mark.en...@gmail.com> wrote:

But let's say later you decide you want your data model to be {:first-name "Alice", :last-name "Beasley", :email "al...@example.com"}, and you want to change name to be a computed value that concatenates first and last names -- this is going to break all your existing code.  If name were only accessed throughout your code via the call `get-name`, then it would be trivial to make this change.

But presumably you'd then be writing code that does (:first-name customer) and (:last-name customer) so you're likely modifying old code anyway?

Even if you're only doing this in new code, your original make-customer presumably took "first last" "em...@addre.ss" and now will either need to change to "first" "last" "em...@addre.ss" - and you'll have to change every constructor call - or you'll modify it to split "first last" automatically to populate :first-name and :last-name - in which case you might as well still populate :name (and none of your code needs to change).

So I think that's a bit of a straw man argument.

Many OO languages force, or encourage you stylistically, to hide everything behind setters and accessors, so you get for free the ability to change your mind about the data model later.

But not the API of the class - that still forces changes to client code - and the setters and accessors are part of that API so adding getFirstName() / getLastName() changes your API anyway.

With Clojure, it is very common to allow your data model to be the API.  This puts some additional pressure to get it right the first time.  You have to explicitly think ahead about which fields might need to change in the future.

In practice, I don't think it makes it any harder than in OOP - and given the ability to add fields to maps with no client code changes required, I think it's actually _easier_ in FP, in nearly all _real world_ cases.

Sean Corfield -- (904) 302-SEAN

An Architect's View -- http://corfield.org/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[James Reeves]

On 17 October 2014 01:08, Mark Engelberg <mark.en...@gmail.com> wrote:

But let's say later you decide you want your data model to be {:first-name "Alice", :last-name "Beasley", :email "al...@example.com"}, and you want to change name to be a computed value that concatenates first and last names -- this is going to break all your existing code.  If name were only accessed throughout your code via the call `get-name`, then it would be trivial to make this change.

Sure, that's the theory behind encapsulation, but I'm not convinced there are many cases in practice where the API can remain consistent while the data changes.

Let's take names as an example. Let's assume you start with a data structure:

    {:first-name "Alice", :last-name "Beasley"}

And we construct an API around that:

    (defn get-first-name [p] (:first-name p))

    (defn get-last-name [p] (:last-name p))

What happens when we come across a name that doesn't fall this scheme, like "Isa bin Osman" or "Madurai Mani Iyer"? In those cases our getter functions become as obsolete as the fields they map to.

You might argue that a smart person would write more abstract accessors:

    (defn get-full-name [p]

      (str (:first-name p) " " (:last-name p)))

    (defn get-brief-name [p]

      (:first-name p))

But if you were smart enough to do that, then you'd clearly be smart enough to write a better data structure in the first place:

    {:full-name "Alice Beasley", :brief-name "Ms Beasley"}

Even if you accept the idea that APIs can protect against changing data structures, you have to be aware what you're giving up. Encapsulation comes at a significant cost: you need to write a custom API for every data structure in your application.

- James

[Armando Blancas]

Sure, that's the theory behind encapsulation, but I'm not convinced there are many cases in practice where the API can remain consistent while the data changes.

 

 

I'm not, either. Models that have little or no abstraction --basically aggregates of related items-- end up having APIs that just reflect their contents. For years, my tools were the Eclipse refactoring features and the code (re)generation of the Eclipse Modeling Framework, not encapsulation. I started using Clojure practices, though not totally convinced until a while ago Stuart Sierra wrote: "The data *is* the API," which helped me understand what I was really doing. Whereas I never faced the proverbial computed-field change, I've now taken good advantage of generic functions over maps and records. 

[Rui Yang]

I am new to clojure.

My question how to handle version if we expose data directly as api, any schema change will break the api which may impact third party users.

[Atamert Ölçgen]

Every programming language is a (somewhat) unique approach to programming. So they all have their ways. Java is best expressed using Java best practices. Python is best expressed using Python best practices. Of course Clojure too is best expressed using Clojure best practices. 

Having said that; I don't think object oriented paradigm makes a lot sense in a dynamic environment. Clojure is a dynamic language. Data can be the API without major issues because we can handle all kinds of data with ease. (= (seq []) nil), select-keys, fnil etc...

Encapsulation is a valuable property when we're in a statically typed environment. We want to hide more because it's a more rigid environment.

On Fri, Oct 17, 2014 at 12:23 PM, Armando Blancas <abm2...@gmail.com> wrote:

Sure, that's the theory behind encapsulation, but I'm not convinced there are many cases in practice where the API can remain consistent while the data changes.

 

 

I'm not, either. Models that have little or no abstraction --basically aggregates of related items-- end up having APIs that just reflect their contents. For years, my tools were the Eclipse refactoring features and the code (re)generation of the Eclipse Modeling Framework, not encapsulation. I started using Clojure practices, though not totally convinced until a while ago Stuart Sierra wrote: "The data *is* the API," which helped me understand what I was really doing. Whereas I never faced the proverbial computed-field change, I've now taken good advantage of generic functions over maps and records. 

--

You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Kind Regards,
Atamert Ölçgen

-+-
--+
+++

www.muhuk.com

[Mark Engelberg]

Right, my point wasn't just about "data change", it was more specifically about the addition or change of "computed fields".

In Clojure, non-computed fields are usually accessed directly by keyword, whereas computed fields require an actual API.  This difference in access style complicates things if you want to change which things are stored versus computed.

Another complicated scenario that occurs fairly commonly in my work is that there are computed fields that you want to memoize so that you only want to compute them once.  For example, imagine that the first time you look up this customer's zip code, you want to do some lookup against some time-consuming service to compute the more precise 4-digit extension to the zip code, and store it as part of the data so you never have to compute it again.  These sorts of data models are very natural in OO, but require more thought to pull off successfully in Clojure. 

Clojure's modeling capability fits my brain quite well, and I don't see these things as major problems.  But I feel obligated to point them out to people coming from an OO background as things to watch out for.

As you say, there is a motto in Clojure: "The data is the API."  What that means is that planning out the data representation becomes just as important as planning out the API is in an OO language.  In OO, the API protects you from certain kinds of data changes, whereas in Clojure the data is more visible, and once people start writing code assuming that certain fields are going to always be present and stay named the same way, you're not going to be able to change that without upsetting your users.

[Atamert Ölçgen]

On Fri, Oct 17, 2014 at 12:47 PM, Rui Yang <ryan...@gmail.com> wrote:

I am new to clojure.

My question how to handle version if we expose data directly as api, any schema change will break the api which may impact third party users.

You can't protect the consumers of your API. You publish your new API and pray.

As a consumer I do TDD. So my code is fully tested. Of course most of the 3rd party code is mocked in my tests.

When I upgrade a dependency I read the changelog and run my code to see if anything breaks. Clojure is a dynamic language, so there's actually no schema.

I suppose I can write integration tests and such... But I doubt they'll be able to catch everything.

My point is; as a producer (of libraries) there's little you can do within your code. As a consumer you can do a bit more.

 

On Friday, 17 October 2014 15:23:10 UTC+11, Armando Blancas wrote:

Sure, that's the theory behind encapsulation, but I'm not convinced there are many cases in practice where the API can remain consistent while the data changes.

 

 

I'm not, either. Models that have little or no abstraction --basically aggregates of related items-- end up having APIs that just reflect their contents. For years, my tools were the Eclipse refactoring features and the code (re)generation of the Eclipse Modeling Framework, not encapsulation. I started using Clojure practices, though not totally convinced until a while ago Stuart Sierra wrote: "The data *is* the API," which helped me understand what I was really doing. Whereas I never faced the proverbial computed-field change, I've now taken good advantage of generic functions over maps and records. 

--

You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tom Oram]

Wow! Thanks for all the replies everyone!

I had a suspicion that the general response would be that I was thinking too much in terms of OOP. I'm going to try and go more with the approach suggested in the replies (it's a mental struggle more than a technical one because I've definitely trained my brain to think in a particular way).

"The data is the API" seems to be a very strong point here. Also this comment from puzzler was very helpful:

"In Clojure, non-computed fields are usually accessed directly by keyword, whereas computed fields require an actual API."

Again, are there any suggested best practices resources available?

Thanks again everyone!

[Tom Oram]

Oh also, can anyone recommend a really well designed OS application I can take a look at and study the code?

On Thursday, 16 October 2014 22:19:32 UTC+1, Tom Oram wrote:

[Phillip Lord]

Mark Engelberg <mark.en...@gmail.com> writes:
> But let's say later you decide you want your data model to be {:first-name
> "Alice", :last-name "Beasley", :email "al...@example.com"}, and you want to
> change name to be a computed value that concatenates first and last names
> -- this is going to break all your existing code. If name were only
> accessed throughout your code via the call `get-name`, then it would be
> trivial to make this change.
>
> (Theoretically, it is possible to implement a revision to a person data
> structure by representing it as a custom map that reimplements keyword
> access so that (:name person) calls a function rather than does the usual
> keyword lookup, but I've never seen anyone do this, so I'm ignoring this
> possibility for the purpose of this discussion).

Actually, I think that this is a real problem with Clojure, and with
data access. It is very hard to change between accessing a var as a
value and through calling a value.

I can give two concrete examples of this. First, in my library I used a
var to store a factory object from a Java API. So I created one and
stored it in a var. This worked well, in general, but when I integrated
my library into an application, I realised that I need to get this
factory object from elsewhere -- in short I needed a function call. So I
had to change all of my `data-factory` calls to `(data-factory)`.

Another time the same issue hit me was with Clojure's :doc metadata.
This is stored as a string, but I wanted to be able to subvert this by
calculating the string from the object containing in the var. As far as
I can tell, this is impossible in Clojure without changing all the
client code that accesses the :doc metadata.

Java does not have this problem by common design patterns -- fields are
declared private and always accessed through a function. If I understand
things correctly, Scala avoids the problem in the same way, although it
autocodes the accessors for you, so avoids the boilerplate.

In Clojure, it seems the only way to avoid this would to wrap up values
in `constantly` and access everything as a function.

Phil

[James Reeves]

On 17 October 2014 05:47, Rui Yang <ryan...@gmail.com> wrote:

I am new to clojure.

My question how to handle version if we expose data directly as api, any schema change will break the api which may impact third party users.

Just treat breaking schema changes in the same way you'd treat breaking API changes.

If you're using semantic versioning, for instance, a breaking change to the schema would warrant an update from version 1.0 to version 2.0.

- James

[James Reeves]

On 17 October 2014 12:14, Phillip Lord <philli...@newcastle.ac.uk> wrote:

Actually, I think that this is a real problem with Clojure, and with
data access. It is very hard to change between accessing a var as a
value and through calling a value.

Curiously, this is something I don't think I've ever run into.

Perhaps I'm the outlier, but both the examples you give seem a little odd to me.

Vars are generally reserved for constants, functions, dynamics and (sometimes) application caches. If you're putting anything else in a var, it's probably wrong.

With regard to automatically :doc string, why can't you set it when the var is created?

- James

[Luc Préfontaine]

Hi Phil,

At some point in time after a prolonged exposure to Clojure, my mind shifted,
I now think about my program flow as values, not containers being passed along.

I take the problem the reverse way to find out if there's a need for a var vs a fn and state
lifecyle.
Here's a short summary of my reasonning:

a) I just find out how long a value need to 'survive' to decide how to implement it's access
and its scope.

Most of the time values are ephemeral so the answer is obvious, a local binding is enough
if there's a need to name it.

b) If a value is required during the lifespan of the process then his scope is probably global
and a var is an obvious choice.

c) If a value reflects a state (as understood by clojure) that will change in time then an
atom or ref can be used. Then its scope determines if it will end up in a var
or not.

I try to wrap a state lifecycle in a API. Most of the time leaking to the outside world involves
too much refactoring, it may look simple at the beginning but you get caught in a
forest fire at some point.

d) I try to stay away from dynamic bindings except in rare cases (syntactic sugar for DSLs, ...).

e) I do not consider Java objects as values. Their internal mutation disqualifies them to me.

They seldom make it in a var in my world except when there's no other choice
(database connection pool, ...). I try to confine them at the edge of the code and wrap
their access in a function.

If possible I try to avoid leaking them to the outside world.

I try to shorten their lifespan even if it means that they will get reallocated
more often.

The main reason being that I want to avoid having to think about things like
are they multithread safe ? To they have a finite lifecycle which prevents reuse ?

They will get buried in a value as we know it in Clojure if they have to leak out.
Makes it a bit harder to access them directly.

If a Java object lifecycle has to leak, I wrap it in a API that relates to the application state,
not to the object itself.

The net result is that my code has few vars aside from the functions themselves and less
state concerns.

I agree that it requires some brain cells rewiring. I went along that path the first year I
worked non-stop with Clojure.

Luc P.

>
> Actually, I think that this is a real problem with Clojure, and with
> data access. It is very hard to change between accessing a var as a
> value and through calling a value.
>
> I can give two concrete examples of this. First, in my library I used a
> var to store a factory object from a Java API. So I created one and
> stored it in a var. This worked well, in general, but when I integrated
> my library into an application, I realised that I need to get this
> factory object from elsewhere -- in short I needed a function call. So I
> had to change all of my `data-factory` calls to `(data-factory)`.
>
> Another time the same issue hit me was with Clojure's :doc metadata.
> This is stored as a string, but I wanted to be able to subvert this by
> calculating the string from the object containing in the var. As far as
> I can tell, this is impossible in Clojure without changing all the
> client code that accesses the :doc metadata.
>
> Java does not have this problem by common design patterns -- fields are
> declared private and always accessed through a function. If I understand
> things correctly, Scala avoids the problem in the same way, although it
> autocodes the accessors for you, so avoids the boilerplate.
>
> In Clojure, it seems the only way to avoid this would to wrap up values
> in `constantly` and access everything as a function.
>
> Phil
>

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

--
Luc Préfontaine<lprefo...@softaddicts.ca> sent by ibisMail!

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:
>>
>> Actually, I think that this is a real problem with Clojure, and with
>> data access. It is very hard to change between accessing a var as a
>> value and through calling a value.
>>
>
> Curiously, this is something I don't think I've ever run into.
>
> Perhaps I'm the outlier, but both the examples you give seem a little odd
> to me.

I don't know who is the outlier. The point is that Scala, for instance,
has explicit support to hide the distinction between accessing a value
and computing a value. The point is to support the uniform access
principle.

http://en.wikipedia.org/wiki/Uniform_access_principle

To my knowledge, Clojure cannot do this.

> Vars are generally reserved for constants, functions, dynamics and
> (sometimes) application caches. If you're putting anything else in a var,
> it's probably wrong.

When I first wrote the var, I thought it was going to be a constant.
However, during the development of my code base, I discovered in one or
two small uses cases, it could not be.

> With regard to automatically :doc string, why can't you set it when the var
> is created?

Two reasons. The first is specific to my library, which is that the var
contains a mutable Java object. So the :doc string may change over time,
independently of the value of the var. "Don't use mutable objects" would
be an obvious response, but not a useful one in this case.

The second reason is more general. I tried adding the :doc metadata to
the vars after they had all been created, but generating this string
involves pretty printing of lisp code, and this is computationally
intensive. In my hands, my code would take, say, 1 minute to load and
then I'd wait 5 minutes for the doc string metadata to update.

So, I want to do this lazily -- in practice, :doc strings are rarely
used, so why compute them all when only a few or none might ever be used.

Phil

[James Reeves]

On 17 October 2014 16:21, Phillip Lord <philli...@newcastle.ac.uk> wrote:

http://en.wikipedia.org/wiki/Uniform_access_principle

To my knowledge, Clojure cannot do this.

Yes, Clojure pretty much rejects the idea of uniform access.

 

When I first wrote the var, I thought it was going to be a constant.
However, during the development of my code base, I discovered in one or
two small uses cases, it could not be.

Sorry, I should have been clearer. By "constant" I meant an immutable data structure. I wouldn't consider a Java factory object to be a constant, even if its internal state never changes.

 

> With regard to automatically :doc string, why can't you set it when the var
> is created?

Two reasons. The first is specific to my library, which is that the var
contains a mutable Java object. So the :doc string may change over time,
independently of the value of the var. "Don't use mutable objects" would
be an obvious response, but not a useful one in this case.

If you're placing mutable objects in a top-level var, and having that var's docstring change based on their contents, that's a strong indication you're doing something wrong.

It sounds like you're trying to interoperate with a Java library that's very structured around OOP principles, and very hard to translate into idiomatic Clojure. My guess is that you wouldn't have the issues you mention in a pure Clojure solution.

- James

[Fluid Dynamics]

On Friday, October 17, 2014 11:22:22 AM UTC-4, Phillip Lord wrote:

James Reeves <ja...@booleanknot.com> writes:
>>
>> Actually, I think that this is a real problem with Clojure, and with
>> data access. It is very hard to change between accessing a var as a
>> value and through calling a value.
>>
>
> Curiously, this is something I don't think I've ever run into.
>
> Perhaps I'm the outlier, but both the examples you give seem a little odd
> to me.

I don't know who is the outlier. The point is that Scala, for instance,
has explicit support to hide the distinction between accessing a value
and computing a value. The point is to support the uniform access
principle.

http://en.wikipedia.org/wiki/Uniform_access_principle

To my knowledge, Clojure cannot do this.

It seems to me that some support could be created for this. Specifically, we'd want to make it that a) (deref foo) on a non-IDeref just evaluates to foo in a no-op, and furthermore that if foo is known at compile time not to be an IDeref the (deref foo) compiles away to just foo, so has no runtime cost; and b) (deref (delay (some-fcall))) amounts to (force (delay (some-fcall))) and where the compiler knows the thing being derefed is a delay (via type hint or whatever) is as efficient as just (some-fcall) at run-time the first time called, and as efficient as ((constantly foo)) for some value of foo thereafter.

In that case, one could hide the computed-or-not nature of some data behind expecting users to use @my-thing to access it, and if my-thing is not an IDeref it is the same as my-thing, but it can be changed later to a delay to make it a lazily-computed thing without breaking the API, and without much runtime efficiency cost, given that the new object is hinted with ^IDeref or something.

[Mars0i]

On Thursday, October 16, 2014 11:53:42 PM UTC-5, puzzler wrote:

In Clojure, non-computed fields are usually accessed directly by keyword, whereas computed fields require an actual API.  This difference in access style complicates things if you want to change which things are stored versus computed.

This also means that you have to remember which data has a keyword accessor and which uses a function.

[Mark Engelberg]

Yeah, it's hard to deny the convenience of Clojure's keyword lookups and standard assoc mechanism for getting and setting stored values, but I think Bertrand Meyer's Uniform Access Principle reflects some pretty deep thinking about the kinds of complications that arise in maintaining large programs.  Although the Clojure community mostly rejects the Uniform Access Principle right now, as people start writing larger programs in Clojure, and need to maintain them for longer periods of time, it will be interesting to see if the pendulum swings back in favor of uniform access.

It will be fun to have this conversation again in 5 years time.

The good news is that if the community does start to see more value in uniform access, achieving that is just a few macros away.

--Mark

--

[Tom Oram]

While this discussing has taken a slight tangent from my original question, it's been a very interesting read. Thanks for all your thoughts everyone. You guys rock!

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/v03o5MWys9E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[James Reeves]

On 18 October 2014 08:28, Mark Engelberg <mark.en...@gmail.com> wrote:

Yeah, it's hard to deny the convenience of Clojure's keyword lookups and standard assoc mechanism for getting and setting stored values, but I think Bertrand Meyer's Uniform Access Principle reflects some pretty deep thinking about the kinds of complications that arise in maintaining large programs.  Although the Clojure community mostly rejects the Uniform Access Principle right now, as people start writing larger programs in Clojure, and need to maintain them for longer periods of time, it will be interesting to see if the pendulum swings back in favor of uniform access.

You make it sound as if structuring an application around data, rather than APIs, is untested at scale. I'd argue the opposite: the only architecture we know works at scale is data driven.

The largest systems we've developed, including the web itself, are data driven. Above a certain size, they have to be, due to latency and consistency concerns. Structuring a large system into isolated services that communicate with data is a tried and tested architecture.

There may be a place for the Uniform Access Principle at the medium scale, where an application is large, but not so large it can't be hosted on one machine. I don't think the relative merits of data-driven vs. api-driven approaches has been proven at this scale.

That said, I think there are reasons for betting on Clojure's approach. Ultimately it comes down to whether we try to manage complexity or remove complexity. The Uniform Access Principle falls in the former camp, along with OOP and encapsulation. They're tools to manage connections between components of a codebase.

Clojure takes the more aggressive stance, and suggests that rather than managing complexity, we should be focused on getting rid of it wherever possible. For instance, where OOP languages try to manage state change though encapsulation, Clojure just removes mutable state entirely, or at least places it in confinement.

Where complexity can't be removed, then we start to get Clojure code that begins to look similar to OO designs. Stuart Sierra's components, for instance, look somewhat similar to stripped-down objects. The difference in Clojure's approach is that these constructs are a last resort, rather than the norm.

- James

[Chris Ford]

James might be too modest to mention this as an exemplar as he's the maintainer, but for me, Ring is a great example of the success of data-as-API. HTTP requests are represented as a nested map with well-known keys, and middleware works with these fields or even adds new ones.

--

[Luc Préfontaine]

+1.

Two years ago we went all data driven here. We stripped the code size and complexity by
a huge factor. All data encapsulation code was sent to the trash can.

Our processing is driven by data more than by code. We ended up with a significant
increase in generic code not linked to the business domain and the rest is made up
mostly of DSLs.

What a relief....

Luc P.

> On 18 October 2014 08:28, Mark Engelberg <mark.en...@gmail.com> wrote:
>
> > Yeah, it's hard to deny the convenience of Clojure's keyword lookups and
> > standard assoc mechanism for getting and setting stored values, but I think
> > Bertrand Meyer's Uniform Access Principle reflects some pretty deep
> > thinking about the kinds of complications that arise in maintaining large
> > programs. Although the Clojure community mostly rejects the Uniform Access
> > Principle right now, as people start writing larger programs in Clojure,
> > and need to maintain them for longer periods of time, it will be
> > interesting to see if the pendulum swings back in favor of uniform access.
> >
>
> You make it sound as if structuring an application around data, rather than
> APIs, is untested at scale. I'd argue the opposite: the only architecture
> we know works at scale is data driven.
>
> The largest systems we've developed, including the web itself, are data
> driven. Above a certain size, they have to be, due to latency and
> consistency concerns. Structuring a large system into isolated services
> that communicate with data is a tried and tested architecture.
>
> There may be a place for the Uniform Access Principle at the medium scale,
> where an application is large, but not so large it can't be hosted on one
> machine. I don't think the relative merits of data-driven vs. api-driven
> approaches has been proven at this scale.
>
> That said, I think there are reasons for betting on Clojure's approach.

> Ultimately it comes down to whether we try to *manage* complexity or
> *remove* complexity. The Uniform Access Principle falls in the former camp,

> along with OOP and encapsulation. They're tools to manage connections
> between components of a codebase.
>
> Clojure takes the more aggressive stance, and suggests that rather than
> managing complexity, we should be focused on getting rid of it wherever
> possible. For instance, where OOP languages try to manage state change
> though encapsulation, Clojure just removes mutable state entirely, or at
> least places it in confinement.
>

> Where complexity *can't* be removed, then we start to get Clojure code that

> begins to look similar to OO designs. Stuart Sierra's components, for
> instance, look somewhat similar to stripped-down objects. The difference in
> Clojure's approach is that these constructs are a last resort, rather than
> the norm.
>
> - James
>

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Mark Engelberg]

I think all of James' points about the proven value of structuring an application primarily around data rather than a complex API are right on point.  It is one of the things I love about the Clojure philosophy.

But there's nothing about the value of data-driven development that requires data lookups and data computations to be so different.  There's plenty of room for Clojure to have continued evolution in this area and still preserve the essence of its approach.

For example, consider that several years ago, Rich declared that Clojure would never have a simple mutable box.  But lo and behold, now we have volatiles.  Consider the rise of records, protocols, and components -- a lot of judiciously applied OOish concepts.

I really enjoy introducing Java programmers to the Clojure way of thinking about data.  But when I do, I like to explain the current thinking in the Clojure community, talk about some of the most triumphant success stories (e.g., Ring), acknowledge some of the pain points, talk about some of the ways that Clojure has grown to handle other data modeling pain points, and some of the ways that Clojure may continue to grow.

[Nahuel Greco]

Maybe we need some sort of lazy map where:

(def m (assoc-computed {:first-name "Robert" :last-name  "Plankton"}

                       :full-name #(str (:first-name %) " " (:last-name %))))

;; will call the function to compute the value and will memoize it:

(:full-name m)  

;; now the memoized value is returned without calling the function

(:full-name m)

;; equality / hashing will trigger computation+memoization of all m

;; computed keys if they aren't computed yet:

(= m other-map)

Computing functions must be pure, so m internally is a mutable object but you can't really distinguish it from an immutable one :) 

Saludos,
Nahuel Greco.

--

[James Reeves]

On 18 October 2014 21:02, Mark Engelberg <mark.en...@gmail.com> wrote:

I think all of James' points about the proven value of structuring an application primarily around data rather than a complex API are right on point.  It is one of the things I love about the Clojure philosophy.

But there's nothing about the value of data-driven development that requires data lookups and data computations to be so different.  There's plenty of room for Clojure to have continued evolution in this area and still preserve the essence of its approach.

It seems counter to the idea of keeping code and data separate, and also a fairly leaky abstraction. If you allow computed fields that are indistinguishable from value fields, then you remove many of the guarantees that you have with a pure data structure.

For example, how would you serialise a computed field? Would you just ignore it? Does that mean that changing the computed fields around would result in different serialisation? Is there any way of connecting a serialised data structure with computed fields to the right code?

So the approach isn't without tradeoffs and increased complexity. I'd also need convincing this is even a problem, as I don't recall a time when this would have been useful in my own work.

- James

[Brandon Bloom]

I don't know who is the outlier. The point is that Scala, for instance,
has explicit support to hide the distinction between accessing a value
and computing a value. The point is to support the uniform access
principle.

http://en.wikipedia.org/wiki/Uniform_access_principle

In one sense, accessing data and calling functions uniform: keyword lookups are just function calls themselves. In another sense, lookup by keyword requires implementing ILookup etc rather than changing a simple function.

If you take the stance that the data is your API (and you should), then it makes sense that switching from a keyword lookup to a function call is a breaking change! It shouldn't be easy to rip the data-rug out from under your callers. At least it *is* possible, depending on how far you're willing to go down the Clojure interfaces rabbit hole. Of course, this equation changes even more in favor of Clojure's keywords approach if your client lives across the network.

That said, if you're not sure if you want to make the data your API, you can reserve the right to change your mind later quite easily. Let's say you have (ns user) and want to create a username function:

(def username :username) ; tada!

Yes, this requires some foresight. Back in my C# days, the guidance was "always make everything a property". That was especially important if you cared about binary compatibility, since changing from a readonly field to a getter was a breaking ABI change. But you know what, over quite a few major and minor releases of our widely used APIs, I don't recall ever once changing from a trivial `return foo` getter to more complex getter logic. The expectation that a getter is a simple data read was always a core part of the public interface.

[Alex Baranosky]

I've maintained 5+ year-old Clojure applications and the Uniform Access Principle was not a concern for me.

Bigger concerns for me were the Single-Responsibility Principle, and conversely, the Big Ball of Mud Anti-pattern. But I think these are both concerns on any large, old program in any language.

--

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:

> On 17 October 2014 16:21, Phillip Lord <philli...@newcastle.ac.uk> wrote:
>> http://en.wikipedia.org/wiki/Uniform_access_principle
>>
>> To my knowledge, Clojure cannot do this.
>>
>
> Yes, Clojure pretty much rejects the idea of uniform access.

I don't think it does. I think it just does not support it which is a
somewhat different thing.

>> > With regard to automatically :doc string, why can't you set it when the
>> var
>> > is created?
>>
>> Two reasons. The first is specific to my library, which is that the var
>> contains a mutable Java object. So the :doc string may change over time,
>> independently of the value of the var. "Don't use mutable objects" would
>> be an obvious response, but not a useful one in this case.
>>
>
> If you're placing mutable objects in a top-level var, and having that var's
> docstring change based on their contents, that's a strong indication you're
> doing something wrong.

Yes, this is certainly a possibility.

> It sounds like you're trying to interoperate with a Java library that's
> very structured around OOP principles, and very hard to translate into
> idiomatic Clojure. My guess is that you wouldn't have the issues you
> mention in a pure Clojure solution.

Which is the actuality. While using mutable objects and global state may
be an indication that I am doing something wrong, my belief is that
starting a software project by ignoring existing java libraries and
rewriting everything in Clojure is a much bigger error. I got to a
usable piece of software in two months; this would not have happened
from scratch.

Still, although my example might appear irrelevant to Clojure in
general, I don't think it is. Clojure's current docstring support is not
very good (and there was a thread here about it not long ago). It would
be nice to have something richer. For really rich documentation,
maintaining the docstrings is a different file might make sense. Clojure
can support this at the moment, but only in the way that I have -- the
var metadata could be updated at load time. What it cannot do (easily)
is support lazy loading of the file documentation at the point of first
use, because the decision was made originally that :doc metadata is a
value and is NOT computed.

Phil

[Phillip Lord]

Fluid Dynamics <a209...@trbvm.com> writes:
>> I don't know who is the outlier. The point is that Scala, for instance,
>> has explicit support to hide the distinction between accessing a value
>> and computing a value. The point is to support the uniform access
>> principle.
>>
>> http://en.wikipedia.org/wiki/Uniform_access_principle
>>
>> To my knowledge, Clojure cannot do this.
>>
>
> It seems to me that some support could be created for this. Specifically,
> we'd want to make it that a) (deref foo) on a non-IDeref just evaluates to
> foo in a no-op, and furthermore that if foo is known at compile time not to
> be an IDeref the (deref foo) compiles away to just foo, so has no runtime
> cost; and b) (deref (delay (some-fcall))) amounts to (force (delay
> (some-fcall))) and where the compiler knows the thing being derefed is a
> delay (via type hint or whatever) is as efficient as just (some-fcall) at
> run-time the first time called, and as efficient as ((constantly foo)) for
> some value of foo thereafter.
>
> In that case, one could hide the computed-or-not nature of some data behind
> expecting users to use @my-thing to access it, and if my-thing is not an
> IDeref it is the same as my-thing, but it can be changed later to a delay
> to make it a lazily-computed thing without breaking the API, and without
> much runtime efficiency cost, given that the new object is hinted with
> ^IDeref or something.

It's definately supportable. Symbol macros could also achieve the same
thing; Clojure has these, but not globally.

Phil

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:

> On 18 October 2014 08:28, Mark Engelberg <mark.en...@gmail.com> wrote:
>
>> Yeah, it's hard to deny the convenience of Clojure's keyword lookups and
>> standard assoc mechanism for getting and setting stored values, but I think
>> Bertrand Meyer's Uniform Access Principle reflects some pretty deep
>> thinking about the kinds of complications that arise in maintaining large
>> programs. Although the Clojure community mostly rejects the Uniform Access
>> Principle right now, as people start writing larger programs in Clojure,
>> and need to maintain them for longer periods of time, it will be
>> interesting to see if the pendulum swings back in favor of uniform access.
>>
>
> You make it sound as if structuring an application around data, rather than
> APIs, is untested at scale. I'd argue the opposite: the only architecture
> we know works at scale is data driven.
>
> The largest systems we've developed, including the web itself, are data
> driven.

Interesting. So, if you resolve http://www.clojure.org, is this data or
is it computed?

I don't think you can tell. The web supports the Uniform Access
Principle.

Phil

[Gary Verhaegen]

On Monday, 20 October 2014, Phillip Lord <philli...@newcastle.ac.uk> wrote:

Interesting. So, if you resolve http://www.clojure.org, is this data or
is it computed?

You're dereferencing a ref (url) to get an immutable value (string).

Maybe it would be worth exploring ways to implement IDeref with custom data types?

[James Reeves]

On 20 October 2014 12:23, Phillip Lord <philli...@newcastle.ac.uk> wrote:

James Reeves <ja...@booleanknot.com> writes:

> Yes, Clojure pretty much rejects the idea of uniform access. 

I don't think it does. I think it just does not support it which is a
somewhat different thing.

I thought it was pretty clear that Clojure prefers data over APIs. The uniform access principle is about preferring APIs over data, which seems counter to Clojure's ideology.

Still, although my example might appear irrelevant to Clojure in
general, I don't think it is. Clojure's current docstring support is not
very good (and there was a thread here about it not long ago). It would
be nice to have something richer. For really rich documentation,
maintaining the docstrings is a different file might make sense. Clojure
can support this at the moment, but only in the way that I have -- the
var metadata could be updated at load time. What it cannot do (easily)
is support lazy loading of the file documentation at the point of first
use, because the decision was made originally that :doc metadata is a
value and is NOT computed.

There's no reason why you have to use the :doc metadata for that. Just write another function that takes a var and spits out some documentation.

On 20 October 2014 12:26, Phillip Lord <philli...@newcastle.ac.uk> wrote:

> The largest systems we've developed, including the web itself, are data
> driven.

Interesting. So, if you resolve http://www.clojure.org, is this data or
is it computed?

I don't think you can tell. The web supports the Uniform Access
Principle.

The response may be computed, but once it's sent to the client it's immutable data. The response returned has no inherent API associated with it.

To put it another way, consider the function:

    (defn build-user [first-name last-name]

      {:first-name first-name

       :last-name last-name

       :full-name (str first-name " " last-name)})

Like a website, a function may perform computations, but its return value is immutable data. Would you say the above conforms the the Uniform Access Principle?

- James

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:
>> > Yes, Clojure pretty much rejects the idea of uniform access.
>
>
>> I don't think it does. I think it just does not support it which is a
>> somewhat different thing.
>>
>
> I thought it was pretty clear that Clojure prefers data over APIs. The
> uniform access principle is about preferring APIs over data, which seems
> counter to Clojure's ideology.

The uniform access principle is about having uniform access to data and
APIs. It's not about prefering one or the other.

> Still, although my example might appear irrelevant to Clojure in
>> general, I don't think it is. Clojure's current docstring support is not
>> very good (and there was a thread here about it not long ago). It would
>> be nice to have something richer. For really rich documentation,
>> maintaining the docstrings is a different file might make sense. Clojure
>> can support this at the moment, but only in the way that I have -- the
>> var metadata could be updated at load time. What it cannot do (easily)
>> is support lazy loading of the file documentation at the point of first
>> use, because the decision was made originally that :doc metadata is a
>> value and is NOT computed.
>>
>
> There's no reason why you have to use the :doc metadata for that. Just
> write another function that takes a var and spits out some documentation.

Yes, which is what I have done, of course. Now it won't work in any IDE
which looks for the docstring as :doc metadata. It is totally
unextensible. I do not think that this is good.

>> > The largest systems we've developed, including the web itself, are data
>> > driven.
>>
>> Interesting. So, if you resolve http://www.clojure.org, is this data or
>> is it computed?
>>
>> I don't think you can tell. The web supports the Uniform Access
>> Principle.
>
>
> The response may be computed, but once it's sent to the client it's
> immutable data.

Well, that's the point, you cannot tell. So, for example, I can provide
a website implemented over a relational database. Or I can serialize it
out as static files. Or I can add a caching layer which serializes
lazily, and redoes every hour.

> The response returned has no inherent API associated with it.

That sort of depends on the response.

> To put it another way, consider the function:
>
> (defn build-user [first-name last-name]
> {:first-name first-name
> :last-name last-name
> :full-name (str first-name " " last-name)})
>
> Like a website, a function may perform computations, but its return
> value is immutable data. Would you say the above conforms the the
> Uniform Access Principle?

As I said at the beginning, it is possible to achieve UAP by making
*everything* a function. So, we can automatically achieve UAP by ONLY
using functions and never values.

So consider for example, this case.

(:doc (meta #'concat))

The return value of this expression can never be computed (short of
reimplementing ILookup or IMeta which, of course, we could do).

On the other hand

((:doc (meta #'concat)))

could be computed or not, since the function returned could just be
returning a value. So, the latter supports the UAP, the former doesn't.

In fact, this is how I squared the circle in my case. I changed

data-factory

to

(data-factory)

In most cases the data-factory function just returns a constant value,
but it can use computation when I choose.

Phil

[James Reeves]

On 20 October 2014 14:02, Phillip Lord <philli...@newcastle.ac.uk> wrote:

The uniform access principle is about having uniform access to data and
APIs. It's not about prefering one or the other.

Right, but Clojure does heavily prefer data over APIs, and therein lies the conflict.

 

Yes, which is what I have done, of course. Now it won't work in any IDE
which looks for the docstring as :doc metadata. It is totally
unextensible. I do not think that this is good.

Clojure prefers "simple" solutions over "easy" solutions.

Unrestricted polymorphism and universal access would make solving this problem easier, but they also create more complexity.

Or to put it another way, rather than trying to shoehorn a solution into a system that wasn't built for it, we should design a new system around the desired solution. In the case of documentation, the most obvious approach is to decide on a new protocol all IDEs can implement.

> The response may be computed, but once it's sent to the client it's
> immutable data.

Well, that's the point, you cannot tell. So, for example, I can provide
a website implemented over a relational database. Or I can serialize it
out as static files. Or I can add a caching layer which serializes
lazily, and redoes every hour.

Yes, and so can a function. Clojure doesn't say, "you shouldn't compute", it says, "you should separate computations and data".

I suppose it's possible to build a service where every value lookup requires hitting an external server, but no-one sane builds a web service like that. Latency concerns alone mean that data and computation need to be separated in distributed systems.

 

> The response returned has no inherent API associated with it.

That sort of depends on the response.

I suppose one could design architecture around passing around executable code that's executed in a sandbox by the client, but see my previous point about sane design.

 

> To put it another way, consider the function:
>
>     (defn build-user [first-name last-name]
>       {:first-name first-name
>        :last-name last-name
>        :full-name (str first-name " " last-name)})
>
> Like a website, a function may perform computations, but its return
> value is immutable data. Would you say the above conforms the the
> Uniform Access Principle?

As I said at the beginning, it is possible to achieve UAP by making
*everything* a function. So, we can automatically achieve UAP by ONLY
using functions and never values.

The point I was trying to make was that computed values (such as those produced by a web server) do not necessarily imply UAP.

I don't disagree that UAP has some benefits, but it comes with a huge number of tradeoffs. Given that the problem it tries to solve is, in my experience at least, exceeding rare, it really doesn't seem worth the additional complexity.

- James

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:
>> Yes, which is what I have done, of course. Now it won't work in any IDE
>> which looks for the docstring as :doc metadata. It is totally
>> unextensible. I do not think that this is good.
>>
>
> Clojure prefers "simple" solutions over "easy" solutions.

A nice aphorism sometimes, but content free in this case, I think.

> Unrestricted polymorphism and universal access would make solving this
> problem easier, but they also create more complexity.
>
> Or to put it another way, rather than trying to shoehorn a solution into a
> system that wasn't built for it, we should design a new system around the
> desired solution.

So, there is this tourist, and he asks a local, "how do I get into
town". "Hmmm," says the local, "if I want to get into town, I wouldn't
start from here".

Software engineering is a compromise, and designing a new system is a
pain, especially if multiple people have to update to it. I think
Clojure's doc string support is weak. Part of the reason for this, is
because it is not extensible. The main reason that it is not extensible
is that, in Clojure, once you have decided that something is a value,
you are stuck.

Don't care about my use-case? That's fine. Have a look at Andy
Fingerhuts thalia which helps to fix Clojure's poor documentation.

https://github.com/jafingerhut/thalia

Same problem -- all has to be done up front, which is ugly and nasty. As
a result, I don't use thalia, which is a shame.

>> The response may be computed, but once it's sent to the client it's
>> > immutable data.
>>
>> Well, that's the point, you cannot tell. So, for example, I can provide
>> a website implemented over a relational database. Or I can serialize it
>> out as static files. Or I can add a caching layer which serializes
>> lazily, and redoes every hour.
>>
>
> Yes, and so can a function. Clojure doesn't say, "you shouldn't compute",
> it says, "you should separate computations and data".

Which is nice and simple, yes. And has negative consequences in terms of
extensibility. I understand if you are happy with this compromise. But
it is a compromise.

> The point I was trying to make was that computed values (such as those
> produced by a web server) do not necessarily imply UAP.
>
> I don't disagree that UAP has some benefits, but it comes with a huge
> number of tradeoffs. Given that the problem it tries to solve is, in my
> experience at least, exceeding rare, it really doesn't seem worth the
> additional complexity.

Well, the question is, where does this additional complexity come from.
In Java, it results in enormous quantities of boilerplate get/set
methods. In Scala, these are autocoded away.

In Clojure, I don't see an easy solution. Not going to stop me using the
language, but it's a weak spot.

Phil

[Brandon Bloom]

Well, the question is, where does this additional complexity come from.
In Java, it results in enormous quantities of boilerplate get/set
methods. In Scala, these are autocoded away.

Boilerplate isn't complexity: It's inefficiency.

I'll grant that it creates complexity-potential-energy via many increased space for complexity to hide. However, it's far more complex to eliminate boilerplate via dynamic mechanisms, such as interface dispatch or, heaven forbid, Ruby-style abuses such as method_missing. Simpler is to eliminate boilerplate by developing terser encodings in terms of values, and bringing computation to bear to interpret (or translate) those values.

But I'll argue that avoiding the UAP isn't about complexity. It's about an intentional modeling of non-uniform access. Clojure data is built, constructively, out of lists, maps, sets, vectors, symbols, keywords, numbers, strings, etc. Any inductive data types, by the very nature of computation, are implemented in terms of co-inductive operations on codata types. You can't observe something in your computer without executing code to interpret some representation which models some abstraction. Any representation is built concretely out of the abstractions below it. By committing to :keyword style access, you're making a proclamation that you're operating on a "concrete" representation. If that concrete representation happens to be implemented abstractly (as it must be), you can override it (again with ILookup, etc). However, if you're going to override it, you damn well better provide value-like semantics.

UAP grants you flexibility in changing an abstraction. However, it grants *too much* flexibility. It lets you change from a convincing emulation of a true mathematical value to a full-blown computation object. Making operations on data source-incompatible with operations on codata is a feature, not a bug.

[James Reeves]

On 20 October 2014 17:08, Phillip Lord <philli...@newcastle.ac.uk> wrote:

James Reeves <ja...@booleanknot.com> writes:

> Clojure prefers "simple" solutions over "easy" solutions.

A nice aphorism sometimes, but content free in this case, I think.

Well, no... The whole point is that "simple" and "easy" in this context have objective definitions.

If you go by Rich's definition of "simple", then a value is objectively simpler than a function.

 

So, there is this tourist, and he asks a local, "how do I get into
town". "Hmmm," says the local, "if I want to get into town, I wouldn't
start from here".

That makes perfect sense, if the tourist is making the trip only once.

But that hardly applies in this case, because a documentation system will be used many times.

So there's this business owner, and he asks a local, "What's the quickest route for my vehicles to get to town?" The local considers, and replies, "You'd be able to get into town much faster if your vehicles were stationed on the other side of town."

A small context shift, and suddenly the local's advice is extremely useful.

 

Which is nice and simple, yes. And has negative consequences in terms of
extensibility. I understand if you are happy with this compromise. But
it is a compromise.

I don't disagree, but I do consider the compromise to be a minor one.

UAP has huge disadvantages in terms of complexity, repetition, scalability, reliability, isolation, and a whole bunch of other things that Clojure is explicitly trying to avoid.

On the other hand, I can't personally recall a situation where it would have actually been useful to me.

Well, the question is, where does this additional complexity come from.
In Java, it results in enormous quantities of boilerplate get/set
methods. In Scala, these are autocoded away.

By "complexity" I'm again referring to Rich's more objective definition of the term.

Complexity in this sense is a measurement of how many things can affect part of your program. An immutable value is the simple, because nothing can affect it. A pure function is more complex, because it's affected by its arguments, and a side-effectful function has even an greater complexity, because it can be affected by pretty much anything.

The simple vs. easy idea is a short way of saying we should prioritise reducing coupling between components (simplicity), even at the cost of short term gains (easiness).

- James

[Daniel]

To their a wrench in the works, "keywords are functions that look themselves up in a map". Therefore, a traditional map is just as much api as data, and map values can be functions too, blurring the lines between static and computed values. So, you know, have fun with that.

[Jirka Daněk]

I asked the question Lightweight data modeling vs traditional classes on Programmers.StackExchange three days ago. I am happy I found this discussion, because the evil moderators at SE hate my question, and want to close it. As it was originally, the question was much longer than it is now, and I feel bad about throwing all what I wrote away, so I want to send it here.

I want to ask a follow-up question to [Q: Functional Programming vs. OOP](https://programmers.stackexchange.com/questions/9730/functional-programming-vs-oop) regarding the data representation.

To explain what I mean by "lightweight data representation", I'll quote from [a Mark Engelberg's blog](http://programming-puzzler.blogspot.cz/2013/12/clojure-vs-scala.html)

> Just as one example, consider modeling a deck of cards. In Clojure,

> you'd be more likely to come up with a simple representation for a

> card, perhaps: [10 :spades]. Depending on the card game, you might

> choose to represent a face card as [:king :clubs] or [13 :clubs]. A

> deck would likely be modeled as just a sequence of cards, and all the

> built-in sequence functions would apply, for example, shuffle, take,

> drop, etc. Serializing the data (for example, if you want to keep a

> database tracking all the shuffled decks you've ever used in a given

> game) comes for free.

> 

> On the other hand, in Scala, you'd be more likely to create a card

> Class with a rank and suit field. The Suit class would be comprised of

> four case classes, because the philosophy is to enumerate all the

> possible suits as separate entities -- there's nothing in Scala like

> Clojure's convenient keywords. For the rank, you'd be steered towards

> representing all the ranks as integers. The possibility of

> representing face cards with a name would likely never occur to you,

> because it would be too complicated to go through the effort of

> defining the type of a rank to be a "integer or a class comprised of

> four case classes -- jack,queen,king,ace". For modeling the deck, you

> probably wouldn't say a Deck is-a sequence, because composition is

> favored over inheritance. So you'd probably have a Deck class which

> would contain a sequence of cards. This means that you'd have to

> reimplement methods like shuffle, take, and drop on your Deck class to

> turn around and dispatch those methods to the underlying sequence of

> cards. If you're not careful, years of object-oriented training might

> kick in and before you know it, you're representing the deck as a

> class where methods like shuffle, take, and drop destructively update

> the underlying sequence -- it feels so natural to do that once you've

> encapsulated the underlying sequence of cards in a class. If you want

> to serialize a deck, that's more code to write (although general

> "pickling" of a Scala object is an active area of research).

> 

> This example pretty much sums up what I prefer about Clojure. I like

> to tell people that a big part of what makes Clojure special is its

> **philosophy of lightweight data modeling**. It leads to delightfully

> simple systems. Scala remains deeply rooted in the OO philosophy,

> which all too often leads to an over-engineered muddle.

Lightweight data modeling is of course not limited to FP languages, although the ease of creating immutable values in many FP languages makes it very attractive there. [Peter Norvig's Python programming class](https://www.udacity.com/course/cs212) on Udacity started by discussing card games and the representation he suggested for a card was a tuple. That is my second example to illustrate that lightweight data modeling is popular nowadays.

The example from an opposing camp which I've read somewhere illustrates the advantage of classes as a mechanism to deal with changing requirements in large systems. Imagine you've decided to represent prices in your system as some sort of arbitrary precision integer and only later realize that the system would be dealing with EUR as well as USD. Then you need to go back, create a class with an amount and currency field and change all the code that works with prices.

Then, there is this interesting debate in the Python world, of course not all is relevant, because Python does not embrace immutability

Stop writing classes, PyCon US 2012

Start writing more classes, some blog

Stop writing stupid classes

Some more from the Clojure camp, this time by the creator himself in [the talk Simple Made Easy](http://www.infoq.com/presentations/Simple-Made-Easy).

@56:30 into the talk:

> Finally in this area. Information. It is simple. Right? The only thing

> you can possibly do with information is ruin it. Right? Don't do it.

> Right? Don't do this stuff. I mean. We got objects. Objects are made

> to like encapsulate I/O devices. So there is a screen, but I can't

> like touch the screen, so I have an object. There's the mouse. I can't

> touch the mouse so there's an object. Right? That's all they're good

> for. They were never supposed to be applied to information, we apply

> them to information. That's just wrong. It's wrong. I can now say it's

> wrong for a reason, right? It's wrong because it's complex. In

> particular, it ruins your ability to build generic data manipulation

> things. If you leave data alone, right, you can build things once that

> manipulate data. And you can reuse them all over the place. And you

> know they are right once and you're done. The other thing about it,

> which also applies to ORM is that it will tie your logic to

> representation things. […] So represent data as data, please. Start

> using maps and sets directly. Don't feel like I have to write a class

> now, 'cause I have a new piece of information. That's just silly.

Are there any other relevant arguments that have been made in this discussion besides what I've just tried to summarize?

Having in mind the "expression problem" mentioned in one of the answers to the original question, it seems to me that lightweight data representation is only suitable for small systems or isolated parts of large systems where the problem domain is well understood. If that is not the case, then we need to use algebraic data types or classes, each is extendable, but each in a different direction.

Finally I want to make the following comments about the previous discussion, mostly to prove I am not just posting heaps of text without reading what people said before me:

I don't know who is the outlier. The point is that Scala, for instance, has explicit support to hide the distinction between accessing a value and computing a value. The point is to support the uniform access principle. http://en.wikipedia.org/wiki/Uniform_access_principle To my knowledge, Clojure cannot do this.

Dart does that too. Every property access, even to globals, is a method call. It is pretty much the only new feature the language has (not counting .. method cascades, lambdas and not requiring everything to be in a class, which is all hardly new for somebody outside the Java world) and all talks that give Dart a pitch praise UAP, maybe because there is not much else to praise.

But there's nothing about the value of data-driven development that requires data lookups and data computations to be so different.

Is calling a pure function on an immutable value a data access? I think it is. In my mind, 7+5 is a thing, 13, not a computation. A sufficiently smart compiler/interpreter may evaluate the access function at object creation and store all as a value {:fst 7 :snd 5 :sum 13} .

The good news is that if the community does start to see more value in uniform access, achieving that is just a few macros away.

Implementing Uniform access principle as a macro is fine, the good thing is that it won't create the problem we know from Javascript OO libraries. "My UAP macro works differently than your UAP macro so instead of UA we are having NA (no access)".

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:
>> Which is nice and simple, yes. And has negative consequences in terms of
>> extensibility. I understand if you are happy with this compromise. But
>> it is a compromise.
>>
>
> I don't disagree, but I do consider the compromise to be a minor one.
>
> UAP has huge disadvantages in terms of complexity, repetition, scalability,
> reliability, isolation, and a whole bunch of other things that Clojure is
> explicitly trying to avoid.
>
> On the other hand, I can't personally recall a situation where it would
> have actually been useful to me.

Okay. I can give you a very concrete example, and one where I think that
it probably has been actually useful to you.


Imagine you write the following piece of code:

(first l)

This returns a value. If we obeyed the universal access principle,
however, we would not know whether this resulted from computational or
otherwise. And, in fact, Clojure does exactly this. It's why you can
write:

(take 10 (range))

The argument that Clojure prefers data over APIs is, as far as I can
see, just wrong. If that were the case, why not drop ISeq (which is an
API) and use a cons cell (which is data)? Answer, because hiding the
distinction between accessing data and performing computation allows you
to do some very useful things. Including infinitely long sequences. In
fact, that Clojure is built over abstractions (i.e an API) is a major
selling point.

The distinction of Clojure is not that it prefers data over APIs, but
that it prefers relatively few APIs, reused where possible.

Phil

[Tom Oram]

I think I've not got a much better idea of "data as the API" concept. What I'm still wandering is where do you do the validation of the data. For example, say a customer has a name, email and favourite colour. This could be represented as a map like so:

 {:name "Tom", :email "t...@whatever.com", :fav-colour :blue}

So I'm guessing that an add customer usecase would be implemented through a function like so:

(defn add-customer [customer]
  ; ...
  )

The system has rules which state that a customer must have an email address and a name, fav colour is option though. My mind then says that, because add-customer is an API call (as in someone implementing a gui will want to make call to this function), then this

should perform some error checking that name and email exist in the map. Reasonable?

Let's also say that maybe there's a send-message function, which takes a customer and a record, and sends the customer a message. This function is assumed to never be needed to be called externally from other functions in the system, so at this point do we skip the customer map validation, and assume that the API functions will protect against invalid customer maps.

Example:

(defn send-message [customer message]
  (send-email (:email customer) message)) ; no error checking because called from internal functions only


(defn add-customer [customer]
  (when (customer-is-valid customer) ; error checking because likely to be called externally
    (add-to-database customer)
    (send-message customer "Welcome")))

Does that seem reasonable? (I hope I'm making sense!?)

Also, what's the attitude towards adding entries into the data which are no expected, for example if I define my customer map with an :age. Is it the general attitude that adding an unknown field should cause an error, or should it just be accepted? (I'm guessing this may be a more application specific answer).

@Jirka: I'm afraid I don't have any answers to your questions (hopefully others will) but I very much  enjoyed reading the material you referenced. Thanks for the post!

On Thursday, 16 October 2014 22:19:32 UTC+1, Tom Oram wrote:

Hello Clojure people,

First up, apologies because this is going to be a long message. Howver, if you do have the time to read and respond, then I would be extremely grateful!

Recently I've decided to give Clojure a proper go. Over the past year or so I've paid it a bit of attention: I've read books all about how to use it and I've spent a bit of time working through tutorials to create little web apps and so on. I understand the language (although not to any great depth yet), but I'm still unsure about the best approaches to actually working with it.

I've got many years of OOP experience, and I'm a big fan of principles like SOLID and approaches like DDD. What I want to do now is, try and learn how to build well designed models using Clojure, while using best practices! I've started having a bit of a crack at it by building a simple app, but I'm finding myself trying to transfer a lot of my existing OOP techniques into Clojure. It's working, but I'm wandering if I'm overdoing it or perhaps just not doing things "the Clojure way".

So, my very first question is are there any good books or resources on modelling, application architecture or best practices, using Clojure?

Next up, maps vs records. I've read that the typical approach is to use maps until the design starts to solidify and then you can move to records. Is the the general attitude of Clojure developers or do some like to dive straight in with records?

The next question is encapsulation: I've taken the approach that I'm kind of using namespaces like classes. If I want to create customer entity I create a namespace for it, then add a "make" function to it which returns a map of the right "shape".  I'm then shying away from accessing the map directly, and rather doing it through other methods in the namespace which take the instance as a parameter. E.g.

(ns app.domain.customer)


(defn make [name, email]
  {:name name
   :email email})


(defn get-name [customer]
  (:name customer))

Is this a reasonable approach? If not, what might be a better one?

This leads on to safety and defensive programming. Using the approach above, how much "type safety" is required? Obviously, in any API which is going to be used by other programmers, you've got to handle bad inputs well. But what is the Clojure approach in the "domain model"? Coming from the DDD mindset, where models are designed to be un-breakable, part of me wants to use records for all entities and typehint for them everywhere. However, I wander if the Clojure way is more about rigorous testing to make sure the wrong values don't get put in in the first place? Also, what about libraries like https://github.com/Prismatic/schema, this could be used in :pre conditions to be more explicit, is that a common thing?

Next up, how far do you go with creating "types" vs using primitives? Again, referring back to DDD, both the email and the name in the example above are suitable candidates for value objects. In Clojure, would you consider hiding the data behind a set of specialised functions to create, access and use it? Or would you just pass the primitive string/map/vector/whatever about and work on it directly? Example:

; Given
(defn make-customer [name email]
 {:name name, :email email})
 
; Examples
 
(def customer1 (make-customer "Tom" "em...@address.com"))
 
; vs...
 
(defn make-name [name] name)
 
(defn make-email [email] email)
 
(def customer2
 (make-customer (make-name "Tom")
 (make-email "em...@address.com")))

I think that's all I want to ask about now. I do have other questions about dependency inversions, but I'll leave that for another time. If you've read this far then thank you very very much! Also,I know that no one really wants to just sit and read though random peoples code, but, if you are interested in my (very very early stages) experimental project, then I've put in on https://github.com/tomphp/clojure-cocktails - any comments, critiques, PRs or questions would be really great!

Thanks you so much for your times and I look forward to any thoughts or suggestions!

Tom

[James Reeves]

On 21 October 2014 12:52, Phillip Lord <philli...@newcastle.ac.uk> wrote:

Okay. I can give you a very concrete example, and one where I think that
it probably has been actually useful to you.

Imagine you write the following piece of code:

(first l)

This returns a value. If we obeyed the universal access principle,
however, we would not know whether this resulted from computational or
otherwise. And, in fact, Clojure does exactly this.

So you're saying laziness and UAP are the same thing in your view?

I'm not sure I'd necessarily consider deferring computation to be the same as uniform access. Deferring computation can occur once, so you couldn't, for instance, use it to solve the :doc problem you mentioned earlier, where the :doc string of a var would depend on the contents of the var. 

That said, I think you could reasonably argue that that "immutable UAP = laziness". 

However, even with this restriction, lazy data structures are still more complex than strict structures. They may pull in side effects and are not confined in size. But they are, as you point out, extremely useful, which is why I like Clojure's compromise of having lazy seqs and strict everything else.

To be clear, I don't think Clojure is necessarily against complexity, just against unnecessary complexity. "The simplest thing possible, but no simpler."

The distinction of Clojure is not that it prefers data over APIs, but
that it prefers relatively few APIs, reused where possible.

I don't think those are mutually exclusive ideas; I think they both stem from the premise of reducing coupling, or in Clojure terminology, "complexity".

Clojure prefers both idea because data is simpler than APIs, and fewer APIs are simpler than more APIs.

- James

[James Reeves]

I think there's should be a strong distinction between data that comes from an external source, and data that comes from an internal source.

External data should be verified as soon as it comes in. There should be a thorough check at the top level that ensures everything is correct before continuing. There's a reason why security checkpoints are carried out at the borders of countries.

Verifying internal data is a matter of program correctness. In an ideal world, you'd make no mistakes and there would be no need to ever check. In practice, it's sometimes helpful to add constraints that limit what can go wrong.

For example, let's take your send-message function:

(defn send-message [customer message]

  (send-email (:email customer) message))

We could add preconditions that raise an error if things go wrong:

(defn send-message [customer message]

  {:pre [(email? (:email customer)) (string? message)]}

  (send-email (:email customer) message))

But in this case I'd be inclined to put the checks on the send-mail function instead:

(defn send-email [email message]

  {:pre [(email? email) (string? message)]}

  ...)

If these preconditions are triggered, then something has gone wrong in your code, but at least they limit the damage that can be done. Preconditions like this might stop your app from flooding your SMTP server with thousands of incorrectly formatted emails.

With regards to whether you should test for additional fields in maps, I'd say generally "no", because functions should only care about the part of the map they're interested in. However, if you're iterating over the map, or checking external data, then you probably do want to check.

- James

--

[Tom Oram]

Thanks for such a great reply James, it was exactly the answer I was hoping for. Also, the point about functions only checking the part of the map that are interested in is something which makes perfect sense but I'd not even considered. Having come from the idea that the data should be perfectly encapsulated and not contain anything unknown, I naturally wanted to ensure this couldn't happen. But now you've started me off on a whole new thought process of possibilities. Very interesting and thank you very much for your thoughts! 

[Mike Haney]

I can't remember if someone posted this already, and the thread is too long and I am too lazy to go back and check, so I apologize if it's already been mentioned.

Anyway, I found this talk very helpful when I started learning Clojure, getting used to data-oriented thinking:

http://www.infoq.com/presentations/Thinking-in-Data

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:

> On 21 October 2014 12:52, Phillip Lord <philli...@newcastle.ac.uk> wrote:
>
>>
>> Okay. I can give you a very concrete example, and one where I think that
>> it probably has been actually useful to you.
>>
>> Imagine you write the following piece of code:
>>
>> (first l)
>>
>> This returns a value. If we obeyed the universal access principle,
>> however, we would not know whether this resulted from computational or
>> otherwise. And, in fact, Clojure does exactly this.
>
>
> So you're saying laziness and UAP are the same thing in your view?

I am saying that UAP enables you to implement laziness freely.

> I'm not sure I'd necessarily consider deferring computation to be the same
> as uniform access. Deferring computation can occur once, so you couldn't,
> for instance, use it to solve the :doc problem you mentioned earlier, where
> the :doc string of a var would depend on the contents of the var.

No, for that you computational in general. Deferred computation would be
enough to support, for example, thalia's extended documentation,
although you'd have to decide up front.

Regardless, we have a nice example in Clojure, where we not
distinguishing between data and computation allows us to do something
nice.

Another example is memoize, which does the opposite -- it makes data
appear to be computation, and means that you can, for example, use a
intuitive recursive definition of fibonacci, without exploding the
compute time.

> To be clear, I don't think Clojure is necessarily against complexity, just
> against unnecessary complexity. "The simplest thing possible, but no
> simpler."

Unfortunately, this distinction is one made at an application level, I
think, and not at a code level.

All of this discussion has made me think I should revist the issue. I
could put my own Map implementation on my vars as metadata, and have
this map work lazily, so that the calculation of the :doc metadata
happens through computation.

That would be evil.

Phil

[James Reeves]

On 21 October 2014 17:15, Phillip Lord <philli...@newcastle.ac.uk> wrote:

James Reeves <ja...@booleanknot.com> writes:

> So you're saying laziness and UAP are the same thing in your view?

I am saying that UAP enables you to implement laziness freely.

Sure, laziness feels like a subset of UAP.

 

Regardless, we have a nice example in Clojure, where we not
distinguishing between data and computation allows us to do something
nice.

Yes... I agree it allows us to do something, but let's agree to disagree on whether that something is "nice" :)

All of this discussion has made me think I should revist the issue. I
could put my own Map implementation on my vars as metadata, and have
this map work lazily, so that the calculation of the :doc metadata
happens through computation.

That would be evil.

It would... but if you're set on this path of evilness, you might want to look at def-map-type in https://github.com/ztellman/potemkin

- James

[Tom Oram]

@Mike Hanley - Really great video, thanks for the link!

[Jason Wolfe]

Hi Tom, 

Maybe this post from would be of use?  

https://github.com/Prismatic/eng-practices/blob/master/clojure/20130926-data-representation.md

It's my best attempt (albeit a year or so old) to answer many of these questions.  Happy to answer questions if you've got them.

Cheers,

Jason

[Linus Ericsson]

Jason,

the summary is good, but I'm missing the more efficient data structure array-map that probably wastes less space than the hash-map for the same size of object. [1]

Also Zach Tellman has made some effort with clj-tuple which however use indexes, not keys. [2]

[1] http://clojuredocs.org/clojure.core/array-map
[2] https://github.com/ztellman/clj-tuple

/Linus

--

[Phillip Lord]

James Reeves <ja...@booleanknot.com> writes:
>
> Sure, laziness feels like a subset of UAP.
>
>
> Regardless, we have a nice example in Clojure, where we not
>> distinguishing between data and computation allows us to do something
>> nice.
>>
>
> Yes... I agree it allows us to do something, but let's agree to disagree on
> whether that something is "nice" :)

Laziness? In my experience it is not nearly as useful as it is suggested
to be, but it does seem to be pushed as a feature of clojure.

> All of this discussion has made me think I should revist the issue. I
>> could put my own Map implementation on my vars as metadata, and have
>> this map work lazily, so that the calculation of the :doc metadata
>> happens through computation.
>>
>> That would be evil.
>>
>
> It would... but if you're set on this path of evilness, you might want to
> look at def-map-type in https://github.com/ztellman/potemkin

That's useful indeed. One of the things putting me off my evil plan was
the prospect of implementing a clojure data structure in clojure; I know
what a pain that is. This might help to circumvent all of that.

Phil

[James Reeves]

On 22 October 2014 08:24, Linus Ericsson <oscarlinu...@gmail.com> wrote:

2014-10-22 6:25 GMT+02:00 Jason Wolfe <ja...@w01fe.com>:

Maybe this post from would be of use?  

https://github.com/Prismatic/eng-practices/blob/master/clojure/20130926-data-representation.md

the summary is good, but I'm missing the more efficient data structure array-map that probably wastes less space than the hash-map for the same size of object.

Yes, it's a little misleading to compare a hash-map to a record, because records rarely have more than 32 defined keys. It should really be a comparison between records and array-maps, which are much closer in terms of space usage and performance.

- James

[James Reeves]

On 22 October 2014 10:01, Phillip Lord <philli...@newcastle.ac.uk> wrote:

James Reeves <ja...@booleanknot.com> writes:
>
>> Regardless, we have a nice example in Clojure, where we not
>> distinguishing between data and computation allows us to do something
>> nice.
>
> Yes... I agree it allows us to do something, but let's agree to disagree on
> whether that something is "nice" :)

Laziness? In my experience it is not nearly as useful as it is suggested
to be, but it does seem to be pushed as a feature of clojure.

No, I mean unrestricted uniform access.

Clojure's laziness is restricted to seqs and is guaranteed to always produce the same value for the same field. That limits its complexity.

That said, seqs still have their issues, even with their inherent restrictions. For instance, it's easy to accidentally hold onto the head of a seq, particularly if it's passed in as an argument to a function. For example:

    (defn print-seq [xs]

      (doseq [x xs]
        (println x)))

In this case the behaviour of the seq complects with the outer lexical scope, causing an unintended side effect.

Lazy seqs are certainly useful, but I also wonder if we're going to see them being used less in future, in favour of solutions built around CollReduce. Is it possible to do without them entirely? I'm not sure, but it would be interesting to find out.

- James

[Fluid Dynamics]

On Wednesday, October 22, 2014 6:09:04 AM UTC-4, James Reeves wrote:

On 22 October 2014 10:01, Phillip Lord <philli...@newcastle.ac.uk> wrote:

James Reeves <ja...@booleanknot.com> writes:
>
>> Regardless, we have a nice example in Clojure, where we not
>> distinguishing between data and computation allows us to do something
>> nice.
>
> Yes... I agree it allows us to do something, but let's agree to disagree on
> whether that something is "nice" :)

Laziness? In my experience it is not nearly as useful as it is suggested
to be, but it does seem to be pushed as a feature of clojure.

No, I mean unrestricted uniform access.

Clojure's laziness is restricted to seqs

Not quite; there's also delay and force, and it's possible to use these to construct other lazy data-structures.

and is guaranteed to always produce the same value for the same field.

Nope:

=> (def foo (int-array [1 2 2 5 9 3]))
#'user/foo
=> (def bar (seq foo))
#'user/bar
=> bar
(1 2 2 5 9 3)
=> (aset foo 3 3)
3
=> bar
(1 2 2 3 9 3)

That said, seqs still have their issues, even with their inherent restrictions. For instance, it's easy to accidentally hold onto the head of a seq, particularly if it's passed in as an argument to a function. For example:

    (defn print-seq [xs]

      (doseq [x xs]
        (println x)))

Locals clearing should prevent head-holding in this case, since xs isn't referenced after the doseq, shouldn't it?

[James Reeves]

On 22 October 2014 14:32, Fluid Dynamics <a209...@trbvm.com> wrote:

On Wednesday, October 22, 2014 6:09:04 AM UTC-4, James Reeves wrote:

No, I mean unrestricted uniform access.

Clojure's laziness is restricted to seqs

Not quite; there's also delay and force, and it's possible to use these to construct other lazy data-structures.

I'm not sure I'd consider a delay to be a lazy structure, at least not in the same way as a seq, because dereferencing is a very explicit operation, even with force.

 

and is guaranteed to always produce the same value for the same field.

Nope:

=> (def foo (int-array [1 2 2 5 9 3]))
#'user/foo
=> (def bar (seq foo))
#'user/bar
=> bar
(1 2 2 5 9 3)
=> (aset foo 3 3)
3
=> bar
(1 2 2 3 9 3)

Okay, I'll grant that Clojure has very few concrete guarantees, especially when interoperating with mutable Java structures. Often guarantees of correctness are traded for performance.

However, this is clearly a side effect of performance concerns rather than something put in to give seqs mutability.

That said, seqs still have their issues, even with their inherent restrictions. For instance, it's easy to accidentally hold onto the head of a seq, particularly if it's passed in as an argument to a function. For example:

    (defn print-seq [xs]

      (doseq [x xs]
        (println x)))

Locals clearing should prevent head-holding in this case, since xs isn't referenced after the doseq, shouldn't it?

I think you're right in this case. I ran into this issue a few months ago, and there was some combination of effects that caused the head to be retained. I remember it was counterintuitive. It might have involved destructuring.

- James

[Stephen Gilardi]

Clojure's laziness is restricted to seqs

and is guaranteed to always produce the same value for the same field.

Nope:

=> (def foo (int-array [1 2 2 5 9 3]))
#'user/foo
=> (def bar (seq foo))
#'user/bar
=> bar
(1 2 2 5 9 3)
=> (aset foo 3 3)
3
=> bar
(1 2 2 3 9 3)

There’s no laziness in that example. With laziness introduced by map, the caching done by lazy seq shows:

user> (def foo (int-array [1 2 2 5 9 3]))

#'user/foo

user> (def baz (map identity foo))

#'user/baz

user> baz

(1 2 2 5 9 3)

user> (aset foo 3 3)

3

user> baz

(1 2 2 5 9 3)

—Steve

[Ambrose Bonnaire-Sergeant]

On Wed, Oct 22, 2014 at 9:32 AM, Fluid Dynamics <a209...@trbvm.com> wrote:

and is guaranteed to always produce the same value for the same field.

Nope:

=> (def foo (int-array [1 2 2 5 9 3]))
#'user/foo
=> (def bar (seq foo))
#'user/bar
=> bar
(1 2 2 5 9 3)
=> (aset foo 3 3)
3
=> bar
(1 2 2 3 9 3)

What is happening here? (seq foo) certainly does not look like something that should extend ISeq from its behaviour.

Thanks,

Ambrose

[Fluid Dynamics]

=> (instance? clojure.lang.ISeq bar)
true

[Jozef Wagner]

Not every ISeq is lazy. In this case the seq is not lazy but is backed by a mutable array, thus the mentioned behavior.

Jozef

[Ambrose Bonnaire-Sergeant]

I missed this in the sequence spec: When seq is used on native Java arrays, changes to the underlying array will be reflected in the seq - you must copy the source array to get full immutability.

I understand the rationale, still quite shocking.

Ambrose

On Wed, Oct 22, 2014 at 3:26 PM, Jozef Wagner <jozef....@gmail.com> wrote:

Yes, seqs are presented as persistent collections, so this situation
is quite unfortunate. But in the case of seq over mutable array, I
think practical/performance reasons win over the correctness, which
would require caching of realized items.

Jozef

On Wed, Oct 22, 2014 at 7:51 PM, Ambrose Bonnaire-Sergeant
<abonnair...@gmail.com> wrote:
> Sorry, to be specific I'm disturbed by the apparent mutation of an immutable
> data structure after it has been observed!
>
> Thanks,
> Ambrose

[Fluid Dynamics]

On Wednesday, October 22, 2014 4:51:23 PM UTC-4, Ambrose Bonnaire-Sergeant wrote:

I missed this in the sequence spec: When seq is used on native Java arrays, changes to the underlying array will be reflected in the seq - you must copy the source array to get full immutability.

I understand the rationale, still quite shocking.

Ambrose

On Wed, Oct 22, 2014 at 3:26 PM, Jozef Wagner <jozef....@gmail.com> wrote:

Yes, seqs are presented as persistent collections, so this situation
is quite unfortunate. But in the case of seq over mutable array, I
think practical/performance reasons win over the correctness, which
would require caching of realized items.

As another poster noted, you can get caching of realized items if you really really want it, by using (map identity the-array). And if you want a snapshot at a particular time you had the array, regardless of how much later a particular sequence item will be first consumed, you can copy it upfront (with (into [] the-array), or just another array and System/arraycopy and don't let a reference to the new array escape). Although, mutation *while* you're taking the snapshot is harder to deal with. You and the potential mutator would have to agree on an object (say, the array itself) to lock. Once you have an immutable object in Clojureland though you can dispense with explicit locking and use the STM, atoms, and suchlike.

[Ambrose Bonnaire-Sergeant]

There is no point considering workarounds (but thanks either way). Every line of Clojure code

near a call to `seq` has local assumptions about an immutable data structure. The implications

run deep.

If this issue is going to be tackled, ArraySeq needs to be made immutable.

I'll continue this discussion elsewhere, sorry for derailing the OP.

Ambrose

--

[James Reeves]

On 21 October 2014 01:23, Daniel <double...@gmail.com> wrote:

To their a wrench in the works, "keywords are functions that look themselves up in a map".  Therefore, a traditional map is just as much api as data

I'm not sure why you think one follows from the other.

- James

[Jirka Daněk]

On Thursday, October 16, 2014 11:19:32 PM UTC+2, Tom Oram wrote:

In Clojure, would you consider hiding the data behind a set of specialised functions to create, access and use it? Or would you just pass the primitive string/map/vector/whatever about and work on it directly?

Stuart Sierra recommends keeping ALL your state in a map and structure your application as a chain of functions that take the map and return a new map. It is a very good functional style because it is similar to the state monad in Haskell.

;; Bad
(defn complex-process []
  (let [a (get-component @global-state)
        b (subprocess-one a) 
        c (subprocess-two a b)
        d (subprocess-three a b c)]
    (reset! global-state d)))

;; Good
(defn complex-process [state]
  (-> state
    subprocess-one
    subprocess-two
    subprocess-three))

So your customer would be a map in an array of customers in the state map.

Clojure has the functions update-in and similar to simplify this kind of data manipulations. I saw it mentioned at https://programmers.stackexchange.com/questions/208154/everything-is-a-map-am-i-doing-this-right