Code arrangement for understandability

[ngocdaothanh]

Hi,

I want to ask about code arrangement and understandability
(readability).

Consider this "normal" code:
x = 1
...
f(x)
...
y = x + 2
...
g(y)

x, f, y, g have the same "abstractness" level, so they are indented to
the same level.

My Clojure code:
(let [x 1]
...
(f x)
...
(let [y (+ x 2)]
...
(g y)))

It is very difficult to capture the "algorithm" behind the Clojure
code because things of the same "abstractness" level do not have the
same indent level.

How to rearrange the Clojure code for understandability?

I have been studying Clojure for several months now but I could never
"get" it. But I couldn't explain why I couldn't get it, I just didn't
know why. But today I think I have found the reason: I think the
problem of Clojure (and Lisp) is not parenthesis, its problem is
indent.

I think if I can "let over" this problem, I will be enlightened. Would
you please help me?

Thank you,
Ngoc

[Graham Fawcett]

On Thu, Dec 10, 2009 at 9:15 AM, ngocdaothanh <ngocda...@gmail.com> wrote:
> Hi,
>
> I want to ask about code arrangement and understandability
> (readability).
>
> Consider this "normal" code:
> x = 1
> ...
> f(x)
> ...
> y = x + 2
> ...
> g(y)
>
> x, f, y, g have the same "abstractness" level, so they are indented to
> the same level.

In what sense do they have the 'same abstractness level'? What do you
mean by abstractness? g(y) depends upon y; y depends upon x; nothing
depends upon f(x), so presumably it is being called for side effects.
I don't see anything similar about them.

You could also write

(let [x 1
_ (f x)
y (+ x 2)
_ (g y)]
...)

which puts them all at the same nesting level.

Best,
Graham

>
> My Clojure code:
> (let [x 1]
>  ...
>  (f x)
>  ...
>  (let [y (+ x 2)]
>    ...
>    (g y)))
>
> It is very difficult to capture the "algorithm" behind the Clojure
> code because things of the same "abstractness" level do not have the
> same indent level.
>
> How to rearrange the Clojure code for understandability?
>
> I have been studying Clojure for several months now but I could never
> "get" it. But I couldn't explain why I couldn't get it, I just didn't
> know why. But today I think I have found the reason: I think the
> problem of Clojure (and Lisp) is not parenthesis, its problem is
> indent.
>
> I think if I can "let over" this problem, I will be enlightened. Would
> you please help me?
>
> Thank you,
> Ngoc
>

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

[Laurent PETIT]

I'm not sure either what you mean by "abstraction level", but I would say that I tend more to think about abstraction level to be at function definition level.

2009/12/10 ngocdaothanh <ngocda...@gmail.com>

--

[Chouser]

On Thu, Dec 10, 2009 at 9:15 AM, ngocdaothanh <ngocda...@gmail.com> wrote:
>

> My Clojure code:
> (let [x 1]
>  ...
>  (f x)
>  ...
>  (let [y (+ x 2)]
>    ...
>    (g y)))
>
> It is very difficult to capture the "algorithm" behind the Clojure
> code because things of the same "abstractness" level do not have the
> same indent level.

Note, however, that in this example only the value of (g y) will
be returned. Granted, if the ... includes conditionals (with
more levels of indenting that aren't shown) there may be other
possible results. But with the structure given, (f x) for
example must be producing side-effects or it wouldn't be there at
all. That pushes this code over into the "less than perfectly
idiomatic" category and gets us a little off the topic of why
idiomatic Clojure is written in one particular way or another.

So let's imagine a more complex example, with multiple 'if's and
therefore multiple tail positions, each computing the return
value a different way:

(let [x 1]
(if ...
(let [y (+ x 1)]
(if ...
y
(f x)))

(let [y (+ x 2)]

(if ...
(if ...
x
y)
(g y)))))

Let's set aside for the moment questions about weather all this
really belongs in a single function or if y should be defined in
multiple places. If we're trying to understand this code we're
going to be repeatedly confronted with the question "where is the
value of foo computed" where foo is x or y.

In almost all languages our eyes would wander up the screen
looking for hints. In imperative code, we would have to be alert
for any variable assignments like "x = y + 1", regardless of
where they show up -- being in some unrelated "then" clause is no
guarantee you can ignore it if there's any chance it might have
executed before reaching the code in question. We'd also need to
be alert to any procedure calls that might name x as a parameter
if there's any chance the procedure might modify x.

In Clojure we still need to look farther up the screen, but only
to find out-dented 'let's. When x is a local holding an
immutable value like an Integer, there is simply no way any code
can muck with it except a 'let' that is directly in the parentage
of the code in question. This does mean you need to look as
different levels of indentation to find the definition, but it is
a good deal less code and fewer kinds of code than in an
imperative language.

--Chouser

[Sean Devlin]

A few things might help:

* Judging by you examples, I looks like you're still getting used to
the lisp style of coding. Everything is a chained function call.
Write your code to support with that in mind.
* Practice using comp & partial. Write a few small apps where you
NEVER use a let form.
* Practice using ->. Try to write a few other other apps only using
this.
* Master the editor you're using. Emacs has great tab support, and
will help you arrange your code accordingly. I'm sure Vim does
something similar. Enclojure has a netbeansy "Format my code" option
from the popup menu.

Sean

PS - Do you come from a Python background?

[David Brown]

On Thu, Dec 10, 2009 at 09:26:07AM -0500, Graham Fawcett wrote:

>(let [x 1
> _ (f x)
> y (+ x 2)
> _ (g y)]
> ...)

What do people in general think of this style? I remember using this
trick a lot with O'Caml, and I've certainly used it a few times in
Clojure, but something feels icky about it.

Where it's most useful, though is with stuff like this:

(let [x ...
y ...
_ (prn "y is" y)
...]
...)

I have found I sometimes find something like:

(let [x ...
x (... x ...)
x (... x ...)
x (... x ...)]
x)

easier to write, even if it is just how I write it the first time, and
then later change it to something looking more like function
application. Sometimes, I've found the let-chain is easier to modify
in the future.

I guess, realizing it's still not imperative (necessarily), it
shouldn't bother me as much.

David

[samppi]

If it's for side-effects, then using _ is fine, in my opinion—they're
nicely identifiable.

They're especially useful for inserting println calls for seeing the
value of something when I'm debugging; in fact, this is the primary
way I do debugging. I would say, though, that usually if you're going
to truly do side-effects, it's better to isolate them as much as
possible, rather than interspersing them within essential logic.

I've seen the

(let [x ...
x (... x ...)
x (... x ...)
x (... x ...)]
x)

notation before, and it is fine. For my tastes, however, I think that
it repeats the symbol (in this case, 'x) too much. Sometimes it may be
the best way, but usually I would instead use ->, ->>, and/or letfn.

[Richard Newman]

> How to rearrange the Clojure code for understandability?

One approach I've used in Common Lisp to avoid multiple lets is to do

(let* ((x (let ((v 1))
(f v) ; for side-effects
v))
(y (+ x 2)))
(g y))

This calls to mind Clojure's doto, which instantiates a Java object,
does some stuff to it, then returns the object. It would be easy to
define a macro to describe this.

Of course, if `f` *returns* the value you want to bind to `x`, you
don't have this problem (you can just put (f x) directly in the let
binding form).

You also don't have this problem if you don't need to call `f` before
binding `y`. (+ x 2) is side-effect free, and (f x) cannot mutate `x`,
so your Clojure code can easily be written

(let [x 1
y (+ x 2)]
(f x)
(g y))

[Richard Newman]

> They're especially useful for inserting println calls for seeing the
> value of something when I'm debugging; in fact, this is the primary
> way I do debugging.

(defn tee (x)
(println "Debug: " (prn-str x))
x)

(let [x (tee (foo 1))]
...)

[ataggart]

The spy macro from c.c.logging does that. ;)

[Richard Newman]

> The spy macro from c.c.logging does that. ;)

I knew it was there somewhere; I just couldn't remember where, or what
it was called! :)

[David Brown]

On Thu, Dec 10, 2009 at 09:13:33AM -0800, samppi wrote:

>notation before, and it is fine. For my tastes, however, I think that
>it repeats the symbol (in this case, 'x) too much. Sometimes it may be
>the best way, but usually I would instead use ->, ->>, and/or letfn.

The problem I have using -> and ->> is that I often find inconsistency
as to which argument the value should be placed at.

David

[ngocdaothanh]

> Do you come from a Python background?

For the sake of this discussion, I would say I come from Erlang.

> Judging by you examples, I looks like you're still getting used to the lisp style of coding. Everything is a chained function call.

You're correct. Comming from Erlang:
* I tend to see things as if they are in a chain, concatted by commas.
* I feel that I have to write lots of nested "let"s for temporary
immutables. I think to avoid adding "let"s, Clojurians would just
don't use temporary immutables. This makes Clojure code hard to
understand, because temporary immutables with good names help explain
the code. The tricks to avoid adding "let"s in previous posts are very
ugly in my opinion. Is this style of "let" common and a good practice
to follow? (I just want to know, sorry if my expression is offensive)

> What do you mean by abstractness?

By "abstractness level" in the previous post, I mean level of code
block. For example I would say B1 and B4 are of the same level.

B1
| B2
| B3
B4

Because of indents, my previous Clojure code lied to my eyes that x,
y, f, g are not at the same block level. This is my difficulty with
Clojure. In short, I can't see a rough algorithm from any Clojure code
any more just by seeing the shape (levels) of blocks. To understand a
Clojure code, I have to look at every bit of code, look closer.

> In Clojure we still need to look farther up the screen...

Things in Erlang are immutable, so I think Clojure has no advantage
over Erlang here.

> Practice using comp & partial...

What do you mean by "comp & partial"? Could you please explain?

> (let [x 1
> y (+ x 2)]
> (f x)
> (g y))

Well, I know my example code can be rewritten like that, but I just
could not give a better one.

Rearranging like this reminds me of C, in which every variables must
be listed beforehand. Since all Clojurians say "lazy" is good, I would
say I prefer C++ because I can be lazy in it, I only have to declare a
variable when I actually use it.

I'm lost in Clojure, please light me a way.

Lots of thanks.

[Jarkko Oranen]

On Dec 11, 11:14 am, ngocdaothanh <ngocdaoth...@gmail.com> wrote:

> Because of indents, my previous Clojure code lied to my eyes that x,
> y, f, g are not at the same block level. This is my difficulty with
> Clojure. In short, I can't see a rough algorithm from any Clojure code
> any more just by seeing the shape (levels) of blocks. To understand a
> Clojure code, I have to look at every bit of code, look closer.
>

You should come up with a real code example instead of a bunch of (f
x) (y z) expressions... The example you show is not very idiomatic
clojure simply because it has side-effects. (Of course you need side-
effects, but they should be isolated).

If the code structure gets too complicated, simplify it by writing
functions or macros to better express your intent.

> > In Clojure we still need to look farther up the screen...
>
> Things in Erlang are immutable, so I think Clojure has no advantage
> over Erlang here.
>
> > Practice using comp & partial...
>
> What do you mean by "comp & partial"? Could you please explain?

comp is function composition, and partial is partial application. eg:

((comp negate inc) 6) -> -7

((partial + 1) 5) -> 6

using these to create new functions is called point-free style
I'm not so much a fan, and I don't think point-free style is
particularly idiomatic in Clojure (unlike in Haskell for example), but
it's sometimes less noisy than #() or fn expressions, so it's good to
know.

They're usually best used with filter and map.

>
> > (let [x 1
> >       y (+ x 2)]
> >   (f x)
> >   (g y))
>
> Well, I know my example code can be rewritten like that, but I just
> could not give a better one.

I don't think there's any use for comp or partial in this case. it's
too simple.

>
> Rearranging like this reminds me of C, in which every variables must
> be listed beforehand. Since all Clojurians say "lazy" is good, I would
> say I prefer C++ because I can be lazy in it, I only have to declare a
> variable when I actually use it.

Your code examples are a bit pathological. Usually (hopefully), you
would have Clojure code that looks like this:

(if-let [source (get-stuff-from-database)]
(reduce +
(map somefn
(filter somepred (extract-data source))))
0) ; no data

Which is more functional in style. One thing you can do is to learn to
read these from right to left, which takes a bit of practice, but pays
off in the end, in my opinion. The ->> and -> macros are sometimes
useful for transforming the right-to-left -readable forms into forms
that look more like pipelines:

(->> (extract-data source)
(filter somepred)
(map somefn)
(reduce +))

There's no need for temporary names here. If a step needs explanation,
just add comments.

Hope this helps.

--
Jarkko

[Jarkko Oranen]

> ((comp negate inc) 6) -> -7

Hm, I was sure negate existed... But seems like it doesn't.

Oh well. (comp - inc) works. :)

[Richard Newman]

> This makes Clojure code hard to understand

I'd phrase this "This makes it hard for me to understand Clojure
code". Lots of us do just fine most of the time.

Try spending some time reading point-free Haskell: there are *no*
local names. You can do that in Clojure, too.

> Is this style of "let" common and a good practice
> to follow? (I just want to know, sorry if my expression is offensive)

Generally, if a local is used only once, I don't bother naming it.
That means I don't have too many lets, and usually not nested ones.

>> (let [x 1
>> y (+ x 2)]
>> (f x)
>> (g y))
>
> Well, I know my example code can be rewritten like that, but I just
> could not give a better one.

The point was that when you're programming without side-effects, *all*
your code can be written in this way, because there is no ordering
dependency between `f` and `g`.

You know your code is spreading its use of side-effects too thinly
when your code starts looking procedural. Ordering dependencies are a
code smell.

> Rearranging like this reminds me of C, in which every variables must
> be listed beforehand. Since all Clojurians say "lazy" is good, I would
> say I prefer C++ because I can be lazy in it, I only have to declare a
> variable when I actually use it.

Even lazier is not using a variable at all.

(Note, though, that "laziness" in Clojure means delaying evaluation,
not being lazy as a programmer.)

[ataggart]

It would be much easier if you pasted some real code. So far it just
looks like complaining that you find it awkward to write clojure in an
imperative style.

[ngocdaothanh]

Below is a snippet from Rich's presentation at QCon 2009 (http://
qconlondon.com/london-2009/file?path=/qcon-london-2009/slides/
RichHickey_Clojure.pdf):

(reduce (fn [model f] (assoc model f (inc (get model f 1))))
{} features))

Do Clojurians usually arrange like that? Can it be rearrange for more
understandability?

Thanks

[Richard Newman]

> (reduce (fn [model f] (assoc model f (inc (get model f 1))))
> {} features))
>
> Do Clojurians usually arrange like that? Can it be rearrange for more
> understandability?

I can't speak for everyone, but I don't think it's common to jam
*everything* on one line. I imagine Rich was laying out for space on
slides.

For maximum readability I would arrange that thusly:

(reduce
(fn [model f]
(assoc model
f (inc (get model f 1))))
{}
features))

or, less verbosely

[ataggart]

Or one could be ridiculously verbose:

(reduce #(let [[model feature] %&
val (get model feature 1)
val (inc val)]
(assoc model feature val))
{}
features)

Its worth noting that the only reason for these extra steps is that,
unlike get, update-in doesn't take a not-found arg (much to my
chagrin). If it had a signature as has recently been discussed
(http://groups.google.com/group/clojure/browse_thread/thread/
99cc4a6bfe665a6e), then the code could simplified down to:

(reduce #(update-in %1 [%2] 2 inc) {} features)

[Adrian Cuthbertson]

> (reduce (fn [model f] (assoc model f (inc (get model f 1))))
> {} features))

> Do Clojurians usually arrange like that? Can it be rearrange for more
> understandability?

I would write it exactly like that. What happens as you become
familiar with Clojure is that the patterns of the api become almost
stamped on your brain, something like a set of nested boxes. When you
see a piece of code that starts with "(reduce" your mind immediately
forms a kind of template like;
... reduce [
a_function_with_2_args,
the_initial_thing_to_be_reduced_into,
the_coll_to_be_reduced
]
and you just know that the 2 args of the function are;
... [the_thing_being_reduced_into_at_each_step,
the_element_of_the_coll_at_each_step
]
and that it should return;
... _the_coll_being_reduced_at_each_step
which will be the final result of the "(reduce"

Similarly your mind then "chunks" the function's contents and applies
a similar pattern matching process to that.

This ability just comes with practice and continual use of the
language and the best (perhaps only) way of acquiring it is by
experimenting with these main core api calls (reduce, map, assoc,
update-in, loop, etc) at the repl.

It's like driving a car, once you can drive you just use the api
(clutch, accelerator, gears, etc) without having to think where the
bits are and that you have to press the clutch before changing the
gears. You would simply ignore, or be irritated with labels that said
"Clutch Here", "Brake Here" and so on.

This is not meant to be patronising, but I think it does speak to the
problem of "disjoint" between experienced Clojure/Lisp'ers and noobs
learning the language. They tend to code as above, but when trying to
help people who are learning, they try to bridge the (imperative) gap
by breaking down the code, adding let placeholders, etc. But they
don't code like that.

Not sure what the best way of bridging the gap is and it is
patronising to just say "experiment at the repl", but for me
personally, that was how the patterns started "clicking" in.

Perhaps our learning resources should include examples like the above
to help learners at least see what the patterns are?

-Rgds, Adrian.

[Richard Newman]

> This is not meant to be patronising, but I think it does speak to the
> problem of "disjoint" between experienced Clojure/Lisp'ers and noobs
> learning the language. They tend to code as above, but when trying to
> help people who are learning, they try to bridge the (imperative) gap
> by breaking down the code, adding let placeholders, etc. But they
> don't code like that.

This discussion reminds me of

http://www.willamette.edu/~fruehr/haskell/evolution.html

[Sean Devlin]

Adrian's comments are spot on, and they can be a bit intimidating to a
newcomer. Let me offer a piece of encouragement.

There's an interesting result from the way Clojurians code as
described above. Every once in a while, someone will do something so
radically brilliant with the built in constructs of the language that
it catches veterans off guard. If you go through the archives of this
list, you'll see several threads where someone does something strange,
and fifty other people respond "WTF???... oh, I get it... DAMN THAT'S
AWESOME". In that order.

It's possible to learn the language quickly, and become proficient in
a year or two. True expertise takes time, and I'd argue that even
Rich hasn't *mastered* Clojure yet. This is not a reflection on
Rich's aptitude with his own language, rather it has to do with how
daunting a task mastering a LISP truly is.

So keep your chin up. Write code, constantly practice. Ask
questions. We're all learning things all of the time here. Who
knows, given enough time you might just write one of those radically
different forms yourself.

Happy Hacking,
Sean

On Dec 12, 11:44 pm, Adrian Cuthbertson <adrian.cuthbert...@gmail.com>
wrote: