[ANN] clindex 0.2.3 - a Clojure[Script] source file indexer

Juan Monetta

unread,

Nov 4, 2019, 7:37:49 AM11/4/19

to Clojure

Clindex is a general and extensible Clojure[Script] source file indexer.

It scans a Clojure[Script] project together with all its dependencies and generates a datascript database with facts about them.

It is intended to be used as a platform for building dev tools so they don't have to deal with the complexities of understanding Clojure code by reading the filesystem.

Instead as an api for talking about your code it gives you a datascript db full of facts you can use together with d/q, d/pull, d/entity, etc.

--

Juan Monetta

Juraj Martinka

unread,

Nov 5, 2019, 12:43:29 AM11/5/19

to Clojure

This looks really cool!

I'm trying this on a leiningen project (https://github.com/jumarko/clojure-experiments/) but I'm getting a NPE:

Reflector.java: 426 clojure.lang.Reflector/invokeNoArgInstanceMember

utils.clj: 11 clindex.utils/normalize-path

utils.clj: 10 clindex.utils/normalize-path

...

REPL: 280 clindex.scanner/eval22432/scan-namespace-decl

scanner.clj: 354 clindex.scanner/scan-namespaces/fn

core.clj: 2755 clojure.core/map/fn

LazySeq.java: 42 clojure.lang.LazySeq/sval

LazySeq.java: 51 clojure.lang.LazySeq/seq

RT.java: 535 clojure.lang.RT/seq

core.clj: 137 clojure.core/seq

protocols.clj: 24 clojure.core.protocols/seq-reduce

protocols.clj: 75 clojure.core.protocols/fn

protocols.clj: 13 clojure.core.protocols/fn/G

core.clj: 6828 clojure.core/reduce

core.clj: 6810 clojure.core/reduce

scanner.clj: 313 clindex.scanner/merge-namespaces

scanner.clj: 309 clindex.scanner/merge-namespaces

scanner.clj: 358 clindex.scanner/scan-namespaces

scanner.clj: 346 clindex.scanner/scan-namespaces

api.clj: 142 clindex.api/index-project!

api.clj: 128 clindex.api/index-project!

REPL: 11 clojure-experiments.parsers.clindex/eval22438

It looks like the `meta` on `ns-decl` returns nil here: https://github.com/jpmonettas/clindex/blob/master/src/clindex/scanner.clj#L278

Is that something that's a result of my setup (using leiningen)?

Juan Monetta

unread,

Nov 5, 2019, 6:28:08 AM11/5/19

to Clojure

Thanks Juraj!

If I have to guess it is probably including a different version of tools.namespace.

I forked tools.namespace to return some extra metadata when parsing namespace declarations.

You can try to run lein deps :tree and check the only tools.namespace is `jpmonettas/tools.namespace 0.3.2`, if you see some `org.clojure/tools.namespace` you can add it to that project exclusions.

I hope that helps!

Juan

Juraj Martinka

unread,

Nov 5, 2019, 8:47:32 AM11/5/19

to Clojure

Thanks, adding exclusions helped: https://github.com/jumarko/clojure-experiments/commit/ad069fa1cc465874dce70d36a9d4d4b9244bb47e.

However, I'm now facing another issue and that is this [I added some debug output]:

full-path: /Users/jumar/workspace/clojure/clojure-experiments/src/clojure_experiments/books/sicp/ch2_abstractions_data/s1_intro.clj

form: 1

Execution error (IllegalArgumentException) at clindex.scanner/read-namespace-forms$fn (form-init934221917976668977.clj:181).

Don't know how to create ISeq from: java.lang.Long

It seems that it has trouble parsing standalone numbers in the middle of a clojure file like this: https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/books/sicp/ch1_abstractions_procedures/exercise.clj#L191

Is that a known limitation? I can see that's unusual in a production codebase but I guess this is a completely valid content.

Juraj Martinka

unread,

Nov 5, 2019, 9:01:59 AM11/5/19

to Clojure

Btw. I've also tried it on one of our closed source projects but there it's failing in clojure.tools.deps.alpha.util.maven/make-session

Execution error (NullPointerException) at clojure.tools.deps.alpha.util.maven/make-session (maven.clj:179).

maven.clj: 179 clojure.tools.deps.alpha.util.maven/make-session

maven.clj: 176 clojure.tools.deps.alpha.util.maven/make-session

maven.clj: 94 clojure.tools.deps.alpha.extensions.maven/eval19538/fn/fn

session.clj: 23 clojure.tools.deps.alpha.util.session/retrieve

session.clj: 14 clojure.tools.deps.alpha.util.session/retrieve

maven.clj: 94 clojure.tools.deps.alpha.extensions.maven/eval19538/fn

MultiFn.java: 244 clojure.lang.MultiFn/invoke

alpha.clj: 185 clojure.tools.deps.alpha/expand-deps/fn

alpha.clj: 184 clojure.tools.deps.alpha/expand-deps

alpha.clj: 165 clojure.tools.deps.alpha/expand-deps

alpha.clj: 232 clojure.tools.deps.alpha/resolve-deps

alpha.clj: 214 clojure.tools.deps.alpha/resolve-deps

scanner.clj: 108 clindex.scanner/scan-all-projects

This ls likely because the (maven/make-system) call returns nil for some reason but, unfortunately, I ran out of time before I could investigate this further.

Juan Monetta

unread,

Nov 5, 2019, 10:38:31 AM11/5/19

to Clojure

Thanks for reporting that!, it should be fixed in latest master commit. I tried your clojure-experiments repo and it is indexing fine now.

Juan Monetta

unread,

Nov 5, 2019, 10:41:27 AM11/5/19

to Clojure

No idea what can be causing this. Current master has a fix for honoring :mvn/repos in deps.edn, maybe that fixes it?

Juraj Martinka

unread,

Nov 6, 2019, 4:24:45 AM11/6/19

to Clojure

Wow, that was quick - thanks!

I can confirm it now works on the clojure-experiments project.

For the closed source one I'm still getting NPE though. Will have a second look in the future if I find enough time.

Reply all

Reply to author

Forward