[ANN] tools.deps.alpha

[Alex Miller]

tools.deps.alpha is a new contrib library API for transitive dependency graph expansion and the creation of classpaths.

For more information, I would invite you to check out:

* the README from tools.deps.alpha

* the slides from my EuroClojure talk (video coming in the next week or two)

There are some additional pieces still to come that will provide installers for various systems as well. This is still a work in progress but is targeted for completion along with Clojure 1.9.

[Sean Corfield]

Looks very interesting. The manual install works – looking forward to platform installers!

 

There are some formatting glitches in the README with nested lists.

 

Also, this line:

 

                Invoke: clj -Porg.clojure=/Users/me/code/clojure/target/classes

 

Looks like it should be:

 

                Invoke: clj -Porg.clojure/clojure=/Users/me/code/clojure/target/classes

 

Based on the previous example’s :dev classpath-overrides?

 

Question: is the thinking that tools like Boot and Leiningen could (should?) switch over to using this, instead of their own, home-grown Aether-based stack?

 

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

[Alex Miller]

On Tue, Jul 25, 2017 at 3:47 PM, Sean Corfield <se...@corfield.org> wrote:

Looks very interesting. The manual install works – looking forward to platform installers!

 

There are some formatting glitches in the README with nested lists.

Don't see it, let me know and I'll fix.

 

 

Also, this line:

 

                Invoke: clj -Porg.clojure=/Users/me/code/clojure/target/classes

 

Looks like it should be:

 

                Invoke: clj -Porg.clojure/clojure=/Users/me/code/clojure/target/classes

 

Based on the previous example’s :dev classpath-overrides?

Yep, fixed.

 

 

Question: is the thinking that tools like Boot and Leiningen could (should?) switch over to using this, instead of their own, home-grown Aether-based stack?

Not necessarily. tools.deps.alpha and the clj script are intended to traverse dependencies and build classpaths. The clj script does not compile code, build artifacts, deploy artifacts, and do many of the other things that build tools like Leiningen or Boot do and clj is not intended to be a replacement for any of those things.

The primary use case for clj initially will be new users or existing users that want to build a repl-focused project without any real intent to deploy artifacts (for example, someone using a repl to munge data or do other exploratory work, rather than build libs or applications). Over time we have ideas about other things we can build out from this core - I hinted at some in my EuroClojure talk. 

It will likely also be useful to build bridges for boot or lein to use a deps.edn (possibly in either direction), but I am not confident enough yet in how this will evolve to predict exactly how that will look. I suspect it's a small job to make boot leverage a deps.edn file now as a source of dependencies and probably a bit larger for Leiningen to do so if desired. But I haven't looked at either yet. Would be happy to discuss more.

Note that tools.deps.alpha is still built on (mostly) the same stack as Pomegranate/lein/boot with the difference that Aether is no longer an Eclipse project and has been rebranded into the Apache Maven project as Maven Resolver. tools.deps.alpha uses the latest Maven Resolver api directly. Pomegranate is using an older (but quite similar) version of Aether. I did look at using Pomegranate, but Pom used extra deps for the dynamic classpath and was doing quite a bit more than I needed (and using older APIs), so in the end I decided to just reimplement exactly what I needed.

[Alex Miller]

On Tue, Jul 25, 2017 at 4:28 PM, Alex Miller <al...@puredanger.com> wrote:

On Tue, Jul 25, 2017 at 3:47 PM, Sean Corfield <se...@corfield.org> wrote:

Looks very interesting. The manual install works – looking forward to platform installers!

 

There are some formatting glitches in the README with nested lists.

Don't see it, let me know and I'll fix.

I see it now and have fixed it!

[Mark Derricutt]

On 26 Jul 2017, at 2:19, Alex Miller wrote:

There are some additional pieces still to come that will provide installers for various systems as well. This is still a work in progress but is targeted for completion along with Clojure 1.9.

Looks good - however with the release of Java 9 soon, has there been any consideration to supporting the module path at all? To me, regardless of peoples thoughts on jigsaw, releasing a new JVM dependency management tool now, and not supporting it seems wrong.

Mark

---

"The ease with which a change can be implemented has no relevance at all to whether it is the right change for the (Java) Platform for all time." — Mark Reinhold.

Mark Derricutt
http://www.theoryinpractice.net
http://www.chaliceofblood.net
http://plus.google.com/+MarkDerricutt
http://twitter.com/talios
http://facebook.com/mderricutt

[Gregg Reynolds]

Dumb question of the day: given that Clojurescript evidently (as I understand) will soon work seamlessly with npm, can (chunks of) this be extended to cljs?  E.g. (s/def ::providers (s/keys :opt-un [::npm])) or some such.  Note that I ask without understanding. ;)

 

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[Sean Corfield]

Looks much better, thank you!

 

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

 

--

[Sean Corfield]

The primary use case for clj initially will be new users or existing users that want to build a repl-focused project without any real intent to deploy artifacts

 

Fair enough. Not convinced clj is very beginner-friendly (compared to Boot – it’s certainly more friendly than Leiningen) but I suspect when there are installers available – and documentation out there is updated to reflect that – it’ll seem easier.

 

FWIW, Boot allows you to play with arbitrary libraries in the REPL like this:

 

                boot -d group/artifact repl

 

No project needed, no build.boot etc, just a REPL with the specified libraries available, wherever you are.

 

Over time we have ideas about other things we can build out from this core - I hinted at some in my EuroClojure talk. 

 

Looking forward to this appearing on ClojureTV! Heard interesting things about it from attendees!

 

I suspect it's a small job to make boot leverage a deps.edn file now as a source of dependencies

 

Yeah, that’s how we handle all our dependencies at World Singles – and it sure is easy with Boot: (merge-env! :dependencies (edn/read-string (slurp “deps.edn”))) – although, of course, the deps.edn file we use is just a vector of vectors, like the regular :dependencies key in Boot/Leiningen.

 

tools.deps.alpha uses the latest Maven Resolver api directly.

 

Good to know, thanks.

 

Sean

 

 

[Alex Miller]

On Tue, Jul 25, 2017 at 5:07 PM, Mark Derricutt <ma...@talios.com> wrote:

Looks good - however with the release of Java 9 soon, has there been any consideration to supporting the module path at all? To me, regardless of peoples thoughts on jigsaw, releasing a new JVM dependency management tool now, and not supporting it seems wrong.

No, there has been no consideration thus far about modulepath. Our goal here is to create classpaths for Clojure apps. I think we have a long while if ever before using modulepaths with Clojure is common. Modules seem fairly useless to me from a Clojure perspective (perhaps even actively harmful) - everything about them is about making things less dynamic and more closed.

[Alex Miller]

Dumb question of the day: given that Clojurescript evidently (as I understand) will soon work seamlessly with npm, can (chunks of) this be extended to cljs?  E.g. (s/def ::providers (s/keys :opt-un [::npm])) or some such.  Note that I ask without understanding. ;)

This effort is really about working on the problem of creating JVM classpaths and was not really intended to extend to cljs. Perhaps it could be (but I'm not sure where or why that would be useful).

[Alex Miller]

On Tue, Jul 25, 2017 at 6:57 PM, Sean Corfield <se...@corfield.org> wrote:

The primary use case for clj initially will be new users or existing users that want to build a repl-focused project without any real intent to deploy artifacts

 

Fair enough. Not convinced clj is very beginner-friendly (compared to Boot – it’s certainly more friendly than Leiningen) but I suspect when there are installers available – and documentation out there is updated to reflect that – it’ll seem easier.

Yeah, with the installers, the manual steps will just be replaced with "brew install clojure". I actually have that working, but I have to finish some of the build scripting stuff before I can release it.

 

FWIW, Boot allows you to play with arbitrary libraries in the REPL like this:

 

                boot -d group/artifact repl

 

No project needed, no build.boot etc, just a REPL with the specified libraries available, wherever you are.

Yeah, boot really starts from dynamic and that drives some of the choices. We decided to start with static and build toward dynamic classpaths. Honestly, I rarely need or want a dynamic classpath and it requires a lot of extra machinery. Typically I think most people add deps to a project comparatively rarely and the cost of a restart is not a big deal.

 

Over time we have ideas about other things we can build out from this core - I hinted at some in my EuroClojure talk. 

 

Looking forward to this appearing on ClojureTV! Heard interesting things about it from attendees!

 

I suspect it's a small job to make boot leverage a deps.edn file now as a source of dependencies

 

Yeah, that’s how we handle all our dependencies at World Singles – and it sure is easy with Boot: (merge-env! :dependencies (edn/read-string (slurp “deps.edn”))) – although, of course, the deps.edn file we use is just a vector of vectors, like the regular :dependencies key in Boot/Leiningen.

Yeah, a lot of people do stuff like this. Good opportunity for some glue code.

[Mark Derricutt]

On 26 Jul 2017, at 13:53, Alex Miller wrote:

This effort is really about working on the problem of creating JVM classpaths and was not really intended to extend to cljs. Perhaps it could be (but I'm not sure where or why that would be useful).

If node modules were declared with a type of :npm that could lead to a nightmare of having mixed dependency type which wouldn't really make sense.

If it was an archive/dependency of purely *.cljc files that could work as their platform independent, but arbitrary node packages would seem to the path of trouble.

[Gregg Reynolds]

On Tue, Jul 25, 2017 at 8:53 PM, Alex Miller <al...@puredanger.com> wrote:

Dumb question of the day: given that Clojurescript evidently (as I understand) will soon work seamlessly with npm, can (chunks of) this be extended to cljs?  E.g. (s/def ::providers (s/keys :opt-un [::npm])) or some such.  Note that I ask without understanding. ;)

This effort is really about working on the problem of creating JVM classpaths and was not really intended to extend to cljs. Perhaps it could be (but I'm not sure where or why that would be useful).

Just wondering (in near-total ignorance) about clj/cljs parallels, e.g. maybe resolve-deps for npm would be useful in cljs.

BTW, minor doc error I think:

resolve-deps

Usage: (resolve-args deps-map resolve-args)

Should that be (resolve-deps ...)? 

thanks,

gregg

[Alex Miller]

On Tuesday, July 25, 2017 at 9:46:55 PM UTC-5, Gregg Reynolds wrote:

BTW, minor doc error I think:

resolve-deps

Usage: (resolve-args deps-map resolve-args)

Should that be (resolve-deps ...)? 

Yes, fixed. Thanks!

[Rick Moynihan]

Just checking this out... Looks really interesting.

Might be worth mentioning the rlwrap dependency in the installation instructions:

/Users/rick/repos/3rdparty/tools.deps.alpha/src/main/resources/clj: line 106: rlwrap: command not found

resolved on a mac with:

brew install rlwrap

R.

[Steve Miner]

I’ve been using my own launch script that does something similar to the new clj script. (I’m caching a Leiningen generated classpath.)  I had some bugs when my project directory was copied to another user or machine because the cached classpath was not validate in the new environment, particularly with a different Maven repository.  The same kind of problem can happen with a remotely mounted project directory.  File time checks did not necessarily protect against these relocations.

I fixed my problem with two changes.  First, I put the cache in the PROJECT/target subdirectory so that a “lein clean” would allow the user to recover.  Second, I named my cache file with the inode of the project.clj file.  If the project is copied somewhere else, it is highly unlikely that the inode numbers would match.  The inode of the Maven repository or project deps.edn should work with the new clj script.

I use rlwrap, except within an Emacs shell.  I suggest you test for the presence of rlwrap as you do for java.  $(type -p rlwrap)  and ignore it if it’s not found.  I think Emacs is popular enough that it’s worth testing for within your clj script ($EMACS will be defined).  It would be good enough if you provided an option to turn off rlwrap so I could wrap your script with my own.

I can file bugs and provide patches if you want.

Steve Miner

steve...@gmail.com

[Alex Miller]

On Wed, Jul 26, 2017 at 11:48 AM, Steve Miner <steve...@gmail.com> wrote:

I’ve been using my own launch script that does something similar to the new clj script. (I’m caching a Leiningen generated classpath.)  I had some bugs when my project directory was copied to another user or machine because the cached classpath was not validate in the new environment, particularly with a different Maven repository.  The same kind of problem can happen with a remotely mounted project directory.  File time checks did not necessarily protect against these relocations.

I fixed my problem with two changes.  First, I put the cache in the PROJECT/target subdirectory so that a “lein clean” would allow the user to recover.  

This is a non-starter - we need the cache to survive a rebuild by separate build tool (also don't want to create target dir if not needed).

 

Second, I named my cache file with the inode of the project.clj file.  If the project is copied somewhere else, it is highly unlikely that the inode numbers would match.  The inode of the Maven repository or project deps.edn should work with the new clj script.

Interesting, will think about that (portability issues though).

 

I use rlwrap, except within an Emacs shell.  I suggest you test for the presence of rlwrap as you do for java.  $(type -p rlwrap)  and ignore it if it’s not found.  I think Emacs is popular enough that it’s worth testing for within your clj script ($EMACS will be defined).  It would be good enough if you provided an option to turn off rlwrap so I could wrap your script with my own.

I've had feedback from others on rlwrap too (since it's currently missing from the installation instructions, which is just my oversight). The idea was that the installer would ensure it's installed so it can be assumed it exists. But there are surely cases where you might not want it so a jira for having a way to add more control around it would be fine. I don't want to check for $EMACS but wouldn't mind having a new env var that you could set to skip it or something like that.

[Rick Moynihan]

Regarding the cross platform nature of rlwrap, at the cost of an extra JVM dependency might it not be better to use jline?

http://jline.sourceforge.net/
https://github.com/jline/jline3

I used to use it with clojure many many years ago and it seemed to work quite well...

R.

[Alex Miller]

I considered jline as well, but using it would require mixing it into the classpath, which I did not wish to do. Many have died on the bikeshed of rlwrap vs jline and I've made the decision for now. Changes were made to the clj script yesterday to make it tolerant of rlwrap being unavailable and there is a ticket working toward some additional changes to give you more control. 

If you wish to use jline it is of course, a Java library, so you can add an alias with :extra-deps dependency on jline, then include it on the command line with "clj -Rjline". With the changes mentioned above you would also be able to disable rlwrap.

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/FpMqtNu0TCE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+unsubscribe@googlegroups.com.