nREPL feedback

[Chas Emerick]

Whatever new REPL capabilities may or may not land in Clojure proper in
the future, I wanted to make something explicit:

If you're using nREPL, and you have a problem, I want to hear
from you.

This is particularly true if you are working on some kind of
Clojure[Script] tooling that uses nREPL: you're in a unique position to
know when some particular REPL behaviour is user error, a bug in the
tools you're developing, or actually a bug in nREPL itself. Percolate
those user reports up when appropriate!

Of course, any issue reports that contain actionable information (esp. a
way to reproduce an observed problem) go straight to the top of the heap.

Beyond bugs and such, I'm also interested to hear about (and see!) ways
nREPL could be made better. Things like nrepl-discover[1] and other ways
to make nREPL an easy-to-use and reliable substrate are of particular
interest. There's also the wide-open space of supporting things like
"rich content", one of the vectors that got me interested in building
nREPL in the first place (that the default wire format can carry binary
data offers a wide-open field in this area). And, chances are many of
the biggest improvements can be made without touching nREPL itself at
all; I'm happy to look at and advise anyone working on related stuff,
whether it's destined for inclusion in nREPL or not.

Anyway, that's it. I just wanted to make sure it's clear that "my door
is open".

Happy travels,

- Chas

[1] https://github.com/technomancy/nrepl-discover

[Colin Fleming]

Hi Chas,

In general, nREPL has been very pleasant to work with. I've had occasional hiccups but very few. The one real problem I've had has been around CLJS REPLs, and the pain around maintaining source/line info:

http://dev.clojure.org/jira/browse/NREPL-59
https://github.com/clojure-emacs/cider/issues/830
https://github.com/cemerick/piggieback/issues/31

That said, I'm starting to think that trying to maintain line number info for anything other than load-file is pretty much a lost cause anyway, since unless the form you're loading has exactly the same number of lines as its previous version, the existing line numbers are going to be wrong for all forms following it in the file anyway. But eval should support this, even so.

I also remember several cases when I found the operation doc missing vital info, but looking at it now I can't remember what those cases were. I definitely had to read the code a lot to build a client.

My main philosophical problem with nREPL is with middleware and using it to add new verbs. I think there are few things done this way that couldn't be done more simply and clearly using straight eval, but I understand that's a controversial position and one that's unlikely to be adopted any time soon. Personally, I think the main verbs that come out of the box (eval, load-file and interrupt, session management) are pretty much all most people need. I know there are cases that do actually require middleware, but I think they're such a vanishingly small minority that I'm not sure the complexity is justified.

--
You received this message because you are subscribed to the Google Groups "clojure-tools" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-tools+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chas Emerick]

Thanks for the feedback, Colin, it's appreciated.

Yeah, source form tracking is a bit of a sticky issue. I was hoping to push the top-level form use case onto clients (especially insofar as certain clients have very nontraditional ways of organizing code), but the resulting complexity was just too much. I think we have a reasonable plan to add optional line/column/sourcefile information to eval operations, maybe for 0.3.0.

I think perhaps you started before I fully fleshed out the operation docs, etc? They're far more complete now than they were soon after 0.2.0 landed. I would like to have (contributed, perhaps?) more comprehensive documentation, but the audience is so small (just those building nREPL clients, not big-U Users) that it's been hard to justify expanding the expositional documentation.

I sympathize with your point re: middlewares as the only (obvious) way to implement operations. But there's two things here: the mechanics of using middleware to define operations, and then the whole "why not just use eval for ~everything?" question. I completely agree on the former: defining and using middleware is very verbose, but very flexible; something like nrepl-discover would make defining new operations _way_ simpler for the majority of use cases. I've long planned on adding something like it to nREPL, but it's never dinged the top of the TODO list.

The answer to "why not just use eval for ~everything?" comes in a few different flavours:

First, I wanted to minimize the pain associated with building Clojure tooling in environments that aren't themselves built using Clojure, or some other kind of lisp. Within this context, the data model offered by bencode (a few atomic types, plus nestable lists and maps) plus a "dictionary" of operations with named parameters (this constitutes an nREPL request) yields something that nearly any programming language can target, even relatively baroque things like VimScript. Saying "just use eval" means that any tooling outside of a lisp will be string-templating expressions to be evaluated.

Second (and more substantively), operations provide a more constrained, abstract interface than `eval`, allowing clients to "call" operations without knowing at all how they're implemented. This allows e.g. tools that were originally designed to work strictly with Clojure code to evaluate and `load-file` ClojureScript code with zero changes via piggieback (for example).  The alternative would be that every tool be aware of what type of REPL its working with, and change the fns being `eval`'d accordingly. Likewise, operations can provide data *about* the evaluation context without running within that context; cider (and maybe others?) uses this to be able to issue all sorts of operations (implementing code completion, macroexpansion, apropos, etc) against REPLs, whether they're Clojure or ClojureScript.

(I don't remember where I saw it, but someone once hacked up a set of nREPL middleware similar to piggieback, but which turned a user's session into a JRuby REPL, with hooks for basic code completion, etc. I took this as a good signal. :-)

Of course, in the end, if you want to use `eval` for tooling stuffs, it's there for you.

I hope the above helps in explaining some of the background on the topic.

Thanks,

- Chas

To unsubscribe from this group and stop receiving emails from it, send an email to clojure-tool...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "clojure-tools" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure-tool...@googlegroups.com.

[Robert Krahn]

Hi, Chas --

Thanks to you and everyone involved in nREPL! It's a reliable and flexible

toolset to use.

I recently hacked together some cljx support, somewhat similar to [1] but not

per connection and more configurable. I was wondering about what's planned for

nREPL cljx support in general in regards to the upcoming support for feature

expressions?

Also generally I'm mostly using plain eval to build tools and would be in favor of

keeping nREPL lean and simple in the future.

[1] https://github.com/lynaghk/cljx/blob/master/src/cljx/repl_middleware.clj

Thanks,

Robert

[Chas Emerick]

I'd love to see your alternative take on nREPL cljx support. Perhaps put it in its own project so people can experiment with it? If it's an improvement over what's offered "officially" now, I'd be happy to look at a PR.

cljx will continue to live on independent of whatever lands in Clojure[Script] itself to support similar use cases; tons of projects use cljx already (many quite popular), so supporting maintenance of them is necessary at a minimum. Also, whatever the strengths of the official reader-level approach, it may yet be the case that some people continue to prefer cljx's preprocessor approach to the problem for whatever reason, so the full lifetime of cljx isn't obvious.

In any case, ping clojure-tools, or open an issue/PR/whatever, but we should probably keep cljx topics off of clojure-dev. :-)

Cheers,

- Chas

[Laurent PETIT]

Hello,

Le lundi 30 mars 2015, Justin Smith <noise...@gmail.com> a écrit :

The two things on my "perfect nrepl" wish list (definitely not urgent, but the two improvements I can think of) are some method of making it more secure (that is, some access control beyond "local connections only") and startup time.

Addressing these would eliminate the two reasons I ever don't run nrepl when I run Clojure.

Are you talking specifically of nREPL here, or using a clojure REPL in general?

Talking about the startup time, can you elaborate a bit (through which command line / tool do you start a repl server/client, how many times a day -since in my experience new REPLs aren't started a lot per day...). 

Knowing this would help bridge the implicit assumptions it may have about your use cases, and thus better understand the problem you encounter with (n)REPLs. 

 

You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.

For more options, visit https://groups.google.com/d/optout.

--

Laurent Petit

[Colin Fleming]

I just added a clojure.main REPL option to Cursive, and subjectively the startup time is something that several people have commented on - I haven't done any measurements or tried to identify why the difference exists, but it's quite noticeable. This just came out two days ago, so I think it's fair to say that every user who has commented on the new feature after using it has commented on the difference in startup time.

Of course, whether shaving two seconds off really makes a big difference in the day of an average developer is doubtful, but first impressions count for a lot. It doesn't help that the new REPL also feels snappier in normal use - this is almost certainly because it does less (no completion over the connection, for example, which adds a lot of back and forth traffic).

On 31 March 2015 at 16:07, Justin Smith <noise...@gmail.com> wrote:

that comparison wasn't totally fair - the nREPL one was loading a namespace and making it current, and the clojure.main one was not

$ time echo '(System/exit 0)' | lein run -m clojure.main -e "(do (require 'lazybot.run) (in-ns 'lazybot.run))"

#<Namespace lazybot.run>

real 0m6.303s

user 0m8.620s

sys 0m0.255s

On Monday, March 30, 2015 at 8:01:47 PM UTC-7, Justin Smith wrote:

Hi Laurent,

This is a comparison between the vanilla clojure.main repl as opposed to nREPL.

The startup time difference between `lein run -m clojure.main` and `lein repl` is significant, so I find myself doing the former to get a faster loading repl.

~/clojure/lazybot$ time echo '(System/exit 0)' | lein run -m clojure.main
Clojure 1.7.0-alpha1
user=>
real 0m3.269s
user 0m4.435s
sys 0m0.332s
~/clojure/lazybot$ time echo '(System/exit 0)' | lein repl
nREPL server started on port 55211 on host 127.0.0.1 - nrepl://127.0.0.1:55211
REPL-y 0.3.5, nREPL 0.2.6
Clojure 1.7.0-alpha1
Java HotSpot(TM) 64-Bit Server VM 1.8.0_05-b13
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

            lazybot.run=> (System/exit 0)
            SocketException The transport's socket appears to have lost its connection to the nREPL server
            clojure.tools.nrepl.transport/bencode/fn--7491/fn--7492 (transport.clj:95)
            clojure.tools.nrepl.transport/bencode/fn--7491 (transport.clj:95)
            clojure.tools.nrepl.transport/fn-transport/fn--7463 (transport.clj:42)
            clojure.core/binding-conveyor-fn/fn--4145 (core.clj:1910)
            java.util.concurrent.FutureTask.run (FutureTask.java:266)
            java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1142)
            java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:617)
            java.lang.Thread.run (Thread.java:745)
            Bye for now!


           real 0m9.817s
           user 0m17.238s
           sys 0m0.536s

The clojure.main repl does not listen on an unsecured socket, so it doesn't provide full access to my account (or the account running the repl at least, usually mine) for any process local to my machine (acknowledged that local holes are numerous for every OS, but there are Clojure users who cannot use nREPL because of this concern).

To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev+unsubscribe@googlegroups.com.

To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

--

Laurent Petit

[Chas Emerick]

(Culling the copy to clojure-dev.)

There's definitely some overhead involved in `lein repl` vs. `lein run -m clojure.main`, but very little of that is fundamentally nREPL-related: Leiningen has to load its own nREPL support, which includes reply, a bunch of code-completion bits, and other supporting utilities.

On the other hand, if you are only using `lein repl` to start a Clojure process with a useful project classpath and stand up an nREPL server to communicate with through Cursive/CIDER/CCW/etc, this should be entirely sufficient:

lein run -m clojure.main -e "(require '[clojure.tools.nrepl.server :as nrepl])(nrepl/start-server)"

Of course, mixing in tooling support (code completion etc) will narrow the gap back towards `lein repl`.

- Chas

To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

--

Laurent Petit

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

[Bozhidar Batsov]

Guess we should suggest the addition of a bare-repl lein task. :-)

As I don't really care about reply or clojure-complete when using CIDER I'd appreciate such a task.

[Laurent PETIT]

"The Repl has no clothes!" ;)

Laurent Petit

[Chas Emerick]

I don't know if a bare-repl task would make a lot of sense. Either it's exactly the snippet I included in my previous mail (in which case, it's trivial/reasonable to expect tools to just invoke clojure.main in the same way), or it's something that would need to be customized to suit each development environment, in which case it's reasonable for every env to include+use their own little "bare++" nREPL script.

- Chas