Namespace circular dependencies

[Colin Fleming]

Hi all,

I'm a little confused about circular namespace dependencies and what is allowed and not allowed. My understanding was that Clojure did not permit cycles in the namespace dependency graph. I dimly recall seeing an error about this at some point and I had seen code in e.g. tools.namespace which throws an error on dependency cycles, so I added some validation in Cursive to produce an error and to prevent namespaces being loaded when there were cycles.

The initial implementation would throw an error if there was a cycle anywhere in the transitive closure of all dependencies from a namespace. This was clearly a little too enthusiastic and has now been fixed, but it did reveal several libraries which have obvious circular dependencies with no apparent issues. One example of this is Zach Tellman's Manifold library, which contains a central manifold.stream namespace which is required by several manifold.stream.<whatever> namespaces. However manifold.stream requires them back with bare require's halfway through its source, and Cursive would complain about this.

So, I'm confused. I'm not sure what is allowed and under what circumstances, and I'm not sure what I should be validating here. Zach very reasonably argued that Clojure clearly permits this since Manifold works, but I'm not sure if this is just a happy accident. Looking at the clojure.core code, it seems that there is only circular dependency checking when using load, not when using require. tools.namespace does throw an error on circular dependencies but wouldn't catch this because it only looks at the ns form, not at any later requires.

Any clarity on what is actually allowed, and any suggestions about the right thing to be validating here are much appreciated.

Cheers,
Colin

[Stephen Gilardi]

> So, I'm confused. I'm not sure what is allowed and under what circumstances, and I'm not sure what I should be validating here. Zach very reasonably argued that Clojure clearly permits this since Manifold works, but I'm not sure if this is just a happy accident. Looking at the clojure.core code, it seems that there is only circular dependency checking when using load, not when using require.

I took a look at what’s going on in the manifold case. I’m not sure this explanation is enough to yield a solution to the problem you’re trying to solve, but it should be a good start.

The cyclic load dependency check fails if `load` is asked to load a file while that same file has started loading but not yet finished loading. It’s a runtime check.

`require` calls `load` conditionally to implement its basic function: “load a lib if it isn’t already loaded”. `require`’s processing of a given libspec may or may not result in a call to `load`: If the lib is already loaded, it isn’t loaded again.

As of commit f11e70726553685bda222ccd37c28addcfc519c7 (29 November 2012, first in clojure-1.5.0-beta2), a lib is considered loaded as soon as an ns form for the corresponding namespace has been successfully evaluated.

In the case of manifold.stream, the explicit calls to `require` occur after (ns manifold.stream …). Since manifold.stream is already considered loaded at that point, the indirect calls to (require ‘manifold.stream) that they generate do not result in calls to `load` and therefore do not trigger the cyclic load dependency check.

—Steve

[Colin Fleming]

I see - thanks very much Stephen, that's very helpful. So it seems like this check should only catch circular dependencies when they appear in ns forms, since those will produce a cycle error if none of the other namespaces are currently loaded. I'll investigate doing this. I can't think of a case when you wouldn't want this check, since when a cycle exists in ns forms there should be no way to load any of the related namespaces without receiving an error.

Cheers,

Colin

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.