tools for minimizing forward declaration

[Phillip Lord]

One of the things that I find unusual with clojure is the requirement
for forward declaration. While I can see the advantages, managing it by
hand can be a pain.

So I was wondering, are there any tools for adding declare statements
when necessary. And better for working out how to reorder function
declarations to minimize the requirement for forward declarations.

Phil

[Stuart Sierra]

Forward declarations are rarely necessary in my experience: you just get used to defining your namespaces with low-level primitive functions at the top and higher-level functions at the bottom. You only need forward declarations (`declare`) in cases of mutual recursion, i.e. two functions that call each other.

-S

[Timothy Baldridge]

And also note that protocol functions and multimethods also don't need forward declarations as they are polymorphic and the interface is defined ahead of time. So mutually recursive multimethods don't need to use declare at all. 

Timothy Baldridge

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Phillip Lord]

That would be true, if I knew what my code was going to do when I
started. But most of my code is used to investigate things that I don't
understand; so it evolves slowly over time. I don't know when I start
what "low-level" is going to be. So, I'm left with the task of removing
forward declarations at the end.

I've also been using clojure for representing things other than
functions -- logical statements in this case -- and mutational
referencing is very common there.

If there aren't any tools, then I'm left with doing it by hand.

> --

--
Phillip Lord, Phone: +44 (0) 191 222 7827
Lecturer in Bioinformatics, Email: philli...@newcastle.ac.uk
School of Computing Science, http://homepages.cs.ncl.ac.uk/phillip.lord
Room 914 Claremont Tower, skype: russet_apples
Newcastle University, twitter: phillord
NE1 7RU

[John D. Hume]

On Aug 19, 2013 5:53 AM, "Phillip Lord" <philli...@newcastle.ac.uk> wrote:
>
> That would be true, if I knew what my code was going to do when I
> started. But most of my code is used to investigate things that I don't
> understand; so it evolves slowly over time. I don't know when I start
> what "low-level" is going to be. So, I'm left with the task of removing
> forward declarations at the end.

I don't think Stuart was suggesting that you start by writing low-level fns at the top but that, as you extract or introduce low-level things to support the idea you're working on, you put them above where you're working (instead of just putting the forward declarations there).

[Tim Visher]

The most annoying thing to me about forward declaration is that it
prevents what Uncle Bob calls 'Newspaper Style Code' where I can
structure my code in such a way that the high-level functions are
right at the top and the primitives that they might need are found
below so that I or someone else who needs to read me code can stop
reading at any point and only miss some details about how things work.

I'm a big fan of this and it's one of the things I miss most about
other languages when I'm in Clojure. Keepin a bunch of primitives in
my head without seeing the big picture so that when I see the big
picture it makes more sense is difficult and for whatever reason
scrolling from the bottom of the file feels unnatrural enough that I
still haven't gotten used to it.

One advantage though is that the requiring forward declaration does
gaurentee that if it compiles the code will not have a mixture of the
two styles.

[Phillip Lord]

Tim Visher <tim.v...@gmail.com> writes:
> The most annoying thing to me about forward declaration is that it
> prevents what Uncle Bob calls 'Newspaper Style Code' where I can
> structure my code in such a way that the high-level functions are
> right at the top and the primitives that they might need are found
> below so that I or someone else who needs to read me code can stop
> reading at any point and only miss some details about how things work.

I tend to agree with this. I like to write in a relatively literate
style when I can, and group functions according to, er, their function.
Without forward declarations, as Stuart suggests, the primary axis of
classification becomes "primitiveness", which can probably be reduced to
"how many other functions does this one refer to".

> I'm a big fan of this and it's one of the things I miss most about
> other languages when I'm in Clojure. Keepin a bunch of primitives in
> my head without seeing the big picture so that when I see the big
> picture it makes more sense is difficult and for whatever reason
> scrolling from the bottom of the file feels unnatrural enough that I
> still haven't gotten used to it.
>
> One advantage though is that the requiring forward declaration does
> gaurentee that if it compiles the code will not have a mixture of the
> two styles.

True. And it also stops the spell mistake errors that I am used to in
Elisp where you call a function by the wrong name. Although, then
clojure brings the problems of accidentally capturing a clojure core
name:

(defn thing [class]
(do-other-thing name))

which compiles just fine but is going to do the wrong thing.

Compromises, compromises.

Phil

[Michael Gardner]

On Aug 19, 2013, at 06:38 , Tim Visher <tim.v...@gmail.com> wrote:

> The most annoying thing to me about forward declaration is that it
> prevents what Uncle Bob calls 'Newspaper Style Code' where I can
> structure my code in such a way that the high-level functions are
> right at the top and the primitives that they might need are found
> below so that I or someone else who needs to read me code can stop
> reading at any point and only miss some details about how things work.

On the other hand, it doesn't take much to get used to reading source files from the bottom up. And I'd consider it a (minor) benefit that Clojure encourages this organization, as opposed to languages that don't care where in the file you put your functions.

Not that I'd be against doing things another way; I just haven't been bothered much by Clojure's "backwards" orientation and the occasional forward-declaration.

[Mikera]

I've found the requirement to define things in order to be a major pain in the following reasonably common situation:

A) public API in one namespace

B) various functions used to implement public API in another namespace

A clearly depends on B. But B often needs to depend on A also: you typically want to use calls to the public API as part of your implementation (e.g. recursively processing subcomponents in a tree-like data structure)

I haven't found a good solution to this yet in Clojure. Typical workarounds involve duplication / facades, such as having "api-function" in A call "api-function-impl" in B. 

My personal opinion (before anyone starts flaming - please regard as constructive criticism) is that Clojure is weak in this regard. Even if it is possible to make this sort of code work, expecting developers to jump through hoops, restructure their code and/or spend a lot of time refactoring because the language doesn't support proper forward declarations is not great..... Clojure would be a better language IMHO if the environment allowed arbitrary declaration order and I think this could feasibly be achieved (via some judicious compiler / namespace handling enhancements).

[Mark Engelberg]

I agree this is a huge pain, although I don't know if I'd call it a "forward declaration" issue as much as it is an issue with Clojure not allowing circular dependencies among modules.

Potemkin seems to be the best way to deal with this particular scenario, but I personally think that this is an important enough problem that a standard mechanism for referring-and-reexporting a var should be built into Clojure's core.

[u1204]

Or you could use (*cough*) a literate (*cough*) programming (*cough*)
style :-)

[Tim Visher]

I'll point out as well that though I thought Yegge's criticisms of
Clojure were a bit polemical (I guess that's his style), the single
pass compiler issue was one of his biggest gripes, and I do think it
still rings true. I feel like I have to babysit clojure in this
regard, when I usually feel like clojure is babysitting me! :)

[Armando Blancas]

I'll point out as well that though I thought Yegge's criticisms of
Clojure were a bit polemical (I guess that's his style), the single
pass compiler issue was one of his biggest gripes, and I do think it
still rings true. I feel like I have to babysit clojure in this
regard, when I usually feel like clojure is babysitting me! :)

Have you seen this? https://news.ycombinator.com/item?id=2467359

[Tim Visher]

That's what I was referring to. Was there something specific about it
that you wanted to call out? :)

--

In Christ,

Timmy V.

http://blog.twonegatives.com/
http://five.sentenc.es/ -- Spend less time on mail

[Phillip Lord]

Tim Visher <tim.v...@gmail.com> writes:

> On Mon, Aug 19, 2013 at 11:09 PM, Armando Blancas <abm2...@gmail.com> wrote:
>>> I'll point out as well that though I thought Yegge's criticisms of
>>> Clojure were a bit polemical (I guess that's his style), the single
>>> pass compiler issue was one of his biggest gripes, and I do think it
>>> still rings true. I feel like I have to babysit clojure in this
>>> regard, when I usually feel like clojure is babysitting me! :)
>>
>> Have you seen this? https://news.ycombinator.com/item?id=2467359
>
> That's what I was referring to. Was there something specific about it
> that you wanted to call out? :)

Actually, that's quite interesting, and I hadn't seen it before.

I can understand the reasons why forward declaration makes the
implementation of clojure easy, but it does seem to be pushing the
burden onto the programmer, rather than the language designer. I'm
particularly not convinced by the argument that "Lisps were designed to
receive a set of interactions/forms via a REPL"; forward declaration
introduces another circumstance where the state of your REPL can get in
a twist: your source might look like this...

(def y)
(defn x[] y)

then get changed to

(defn x[] y)
(def y)

and working quite happily *until* you restart the REPL. Re-starting just
in case is a problem with most REPLs, and this doesn't help. On the
whole, I think I prefer compile warnings, rather than everything
breaking.

I seem to have stirred up a bit of a nest.

Phil

[Softaddicts]

REPL versus compilation in Lisp have for many decades been two different
things.

In the REPL, you were allowed to refer to symbols that were undefined in the current
lexical scope, hoping that at runtime they would eventually exists some where
on the stack.

To compile such code you needed to add declarations for this
stuff not in the lexical scope of the compiler to tell it that it would be defined later
else where. Either later in the same file or in a different one, this did not matter.
Otherwise the compiler would simply fail.

Compilation was an odd thing in Lisp initially and it was sufficiently different from
the REPL to require twists in your code when you wanted to compile it.
It was seen as a second step, after dev had been done in the REPL.

The REPL is there to swallow forms which are unrelated.
The kind of compilation you would like to see requires something more
(the "file approach ?").

If we fall too much on the "compilation/optimization"
side, then this relative independence of forms will eventually break and
create it's own set of problems at the REPL.

Yes Lisps were designed mainly for REPL interaction, compiling came afterward.
I was a witness of this era mainly because I am a dinosaur :)

Clojure's approach is somewhere closer to the compilation side in view of the
requirements of it's supporting container but I do not think
that it can get much closer without breaking some of that dynamic REPL stuff that
speeds up development and that we like.

Luc P.

> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

--
Softaddicts<lprefo...@softaddicts.ca> sent by ibisMail from my ipad!

[Armando Blancas]

That's what I was referring to. Was there something specific about it
that you wanted to call out? :)

Nope; just wanted to bring it up.

[Mikera]

There's a way to reconcile all of this: if you treat the "environment as a value" then I believe you can achieve both:

- Incremental REPL-style commands that work immediately in the current environment

- Arbitrary declaration order in files

From a technical perspective, the environment would contain deferred compilation units that get compiled only when the symbols they require are all defined. This could be managed fairly efficiently using a dependency graph maintained by the compiler. Automatic recompilation of dependency graphs would also be possible, I believe - this would potentially allow some fun/scary stuff if you redefine things at runtime but I think it would work.

I wrote about this idea a while back, there wan't very much interest at the time but I still think it's a powerful approach that could be applied to Clojure:
https://groups.google.com/d/topic/clojure-dev/S8BawG7nzJA/discussion