REPL and ^C

[martin odersky]

I noted that in the 2.10 nightlies the REPL no longer catches ^C

/Users/odersky/gits/courses/progfun-slides> scala

Welcome to Scala version 2.10.0-20120828-145227-3313b3049c (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_31).

Type in expressions to have them evaluated.

Type :help for more information.

scala> while (true) {}   

/Users/odersky/gits/courses/progfun-slides> 

When I type ^C I get back the OS prompt. Is that intentional? I kind of miss the "have to take her out ..." spiel.

Cheers

 - Martin

[Paul Phillips]

On Wed, Aug 29, 2012 at 5:03 AM, martin odersky <martin....@epfl.ch> wrote:
> When I type ^C I get back the OS prompt. Is that intentional? I kind of miss
> the "have to take her out ..." spiel.

At some point it was suggested that the compiler code footprint had
become too large. I removed a number of files where I was the sole
author; among these was the signal handling code. That wasn't my only
motivation; the possibility of interrupting long computations required
running every repl line in its own thread, which brought about some
surprising/undesirable behaviors, e.g. with anything using
threadlocals. I figured I'd bring it back someday in a separate repl.

[Daniel Spiewak]

Why would this require running each line in its own thread?  Why not simply have a handler for InterruptedException around the main dispatch site for a single line?  Then, SIGINT can (once again) raise the InterruptedException, aborting the current execution without killing the REPL.  The only problem I can see here is if some cleanup is required that previously would have been handled by just destroying the thread.

Daniel

[Paul Phillips]

How do you intend to interrupt it? There is no way to interrupt an
arbitrary computation on the jvm short of calling Thread.stop, which
is what it used to do, and which is also why it has to be in its own
thread. If you think otherwise, I encourage you to send a pull
request.

[Daniel Spiewak]

The entire REPL would need to be in a separate thread, but it could be the same thread for the entire REPL, rather than per computation.  You still need to use Thread.stop, but the thread would not in fact, er, stop.  Instead, it would catch the InterruptedException and move on to the next REPL prompt.  Basically, this proposal lifts the Thread wrapping from the level of a single line up to the entire dispatch loop.

I'll look at sending a pull request.  It doesn't sound too tricky.

Daniel

[Paul Phillips]

On Wed, Aug 29, 2012 at 11:13 AM, Daniel Spiewak <djsp...@gmail.com> wrote:
> You still need to use Thread.stop, but the thread would not in fact, er,
> stop. Instead, it would catch the InterruptedException and move on to the
> next REPL prompt. Basically, this proposal lifts the Thread wrapping from
> the level of a single line up to the entire dispatch loop.

After calling Thread.stop, it is not InterruptedException you would be
catching, but ThreadDeath. Having read "Pet Sematary", I find it
improbable that everything would end well.

[Roland Kuhn]

Hi Daniel,

29 aug 2012 kl. 20:13 skrev Daniel Spiewak:

The entire REPL would need to be in a separate thread, but it could be the same thread for the entire REPL, rather than per computation.  You still need to use Thread.stop, but the thread would not in fact, er, stop.  Instead, it would catch the InterruptedException and move on to the next REPL prompt.  Basically, this proposal lifts the Thread wrapping from the level of a single line up to the entire dispatch loop.

I'll look at sending a pull request.  It doesn't sound too tricky.

Thread.stop() can easily mess up arbitrary data structures which happen to be referenced from the call stack at the point it is “interrupted”: this is a very real problem, with locks which are never released or wake-ups which never happen, or …

In short: Thread.stop() itself and catching ThreadDeath is not tricky, that’s right, but most libraries (including the JRE) are not prepared to survive it. The only safe thing to do is confine a set of Threads to an isolated ClassLoader and kill them as a whole, throwing away all classes in there.

Regards,

Roland

Daniel

On Wed, Aug 29, 2012 at 12:03 PM, Paul Phillips <pa...@improving.org> wrote:

How do you intend to interrupt it? There is no way to interrupt an
arbitrary computation on the jvm short of calling Thread.stop, which
is what it used to do, and which is also why it has to be in its own
thread.  If you think otherwise, I encourage you to send a pull
request.

On Wed, Aug 29, 2012 at 10:53 AM, Daniel Spiewak <djsp...@gmail.com> wrote:
> Why would this require running each line in its own thread?  Why not simply
> have a handler for InterruptedException around the main dispatch site for a
> single line?  Then, SIGINT can (once again) raise the InterruptedException,
> aborting the current execution without killing the REPL.  The only problem I
> can see here is if some cleanup is required that previously would have been
> handled by just destroying the thread.

Roland Kuhn
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn

[Daniel Spiewak]

I was thinking of Thread.interrupt, not Thread.stop, but unfortunately Thread.interrupt doesn't have the semantics necessary to implement this (it can't raise an InterruptedException in an arbitrary instruction interleaving, only at actual thread blocking points).  So as you say, there's no way to do it the way I had been thinking of.

Daniel

[nafg]

Couldn't you wrap the entire REPL in a thread, call Thread.stop, and discard that thread, and then transparently start a new thread for subsequent commands?

[Daniel Spiewak]

The only advantage to that over the previous strategy is the weirdness with ThreadLocal(s) would be confined to just when you hit Ctrl-C, rather than with every command.  Not sure that's an improvement.

Daniel

[Naftoli Gugenheim]

How often do you expect to have to hit Ctrl-C? I think it's reasonable for Ctrl-C to start a new thread, when the only alternative is to abort the whole process (and having to warm up the JVM again).

[Ivan Todoroski]

If I understand correctly, pressing Ctrl-C now would exit the REPL
completely and cause you to lose all the state you have accumulated in
the REPL?

So, I guess the question is which is more detrimental to general
developer productivity:

A) losing accumulated state of a potentially long REPL session just
because you wanted to interrupt an infinite loop (or a long computation)
you created by mistake , or

B) not being able to use ThreadLocals across REPL commands

This is not a loaded question btw, I am not trying to be
confrontational. I honestly don't know the answer to this question. I am
leaning towards A, but then again I don't know how many libraries are
using ThreadLocals internally that one would like to use in the REPL.

On the other hand, there might be different ways to solve the problem.

For example, you could keep creating new threads on every REPL command,
but before every new REPL command you just copy the entries in the
Thread.threadLocals map from the old thread to the new thread (you would
need to also copy Thread.inheritableThreadLocals for completeness).

If you don't detest too much the use reflection to poke around private
JDK fields, this could work nicely.

Another solution could be to do what you do now (only one thread for the
whole REPL that dies on Ctrl-C), but then enhance the :replay command to
be able to replay all commands from the previous REPL session. That way
you can at least regain the REPL state you had before you hit that
unfortunate infinite loop.


Of course, when I say "you" above it is meant rhetorically, I am not
suggesting that you (Paul) should personally implement any of this. I'm
just throwing out a couple of ideas to discuss their feasibility.

[Alex Cruise]

My apologies in advance if any of this is obvious, naive and/or already discussed and discarded.

Ideally, the function of Ctrl-C should depend on whether the REPL is waiting for user input, or running something in the foreground.  

Waiting at a prompt, my preference would be:

- If there's no text entered at the prompt, print a message like "(interrupt) type :quit <Enter> or Ctrl-D to exit"

  - it's handy to trap Ctrl-C here, because people often press it repeatedly without intending to completely exit their current activity--mysql client, I'm looking at you

- If there is text entered, discard it and bring up a new, empty prompt--this is what bash does.

With a foreground task running, ideally Ctrl-C should kill the task and return control to the REPL.  I guess this implies running each command in a separate thread from the REPL itself, so it can be forcibly stopped--but of course, this doesn't necessarily imply starting a new thread for each command--we could just keep one of them around to run user code in--until it dies, only then we start a new one.

As for the mechanics of stopping the task, so long as we're baking pie in the sky...

while (!thread.isDead) {

  thread.youShouldExitNow() // presumably this is a custom thread class

  thread.interrupt()

  Thread.sleep(n) // say 2-5 seconds

  if (!thread.isDead) {

    val yn = prompt "The thread hasn't exited yet. Should we try to forcibly stop it?"

    if (yn == "y") {

      thread.stop()

    } 

  }

}

If anything is running in the background, may FSM have mercy on its soul. Out of scope. :)

If we felt like being clever, we could rewrite any line that has a looping construct at the root of its tree to be breakable.  If it's a method call that loops, well, too bad.

-0xe1a

[Paul Phillips]

On Thu, Aug 30, 2012 at 12:01 PM, Alex Cruise <al...@cluonflux.com> wrote:

My apologies in advance if any of this is obvious, naive and/or already discussed and discarded.

It is all good ideas.  Do me a favor and open a ticket with that text verbatim...

[Ivan Todoroski]

On 30.08.2012 21:01, Alex Cruise wrote:
> *With a foreground task running*, ideally Ctrl-C should kill the task

> and return control to the REPL. I guess this implies running each
> command in a separate thread from the REPL itself, so it can be forcibly

> stopped--but of course, this doesn't necessarily imply /starting a *new*
> thread/ for each command--we could just keep one of them around to run

> user code in--until it dies, only then we start a new one.

Wouldn't this still leave the problem with ThreadLocals though? Any
previously defined ThreadLocals will suddenly lose all their values
after you press Ctrl-C.

I don't know how big of a problem this is in practice, but it was
explicitly mentioned by Paul as one of the reasons for not using
different threads for different REPL commands.

So you would still end up having to do the nasty reflection hack of
copying ThreadLocal values into the new thread...

[Alex Cruise]

On Fri, Aug 31, 2012 at 3:33 AM, Ivan Todoroski <grnch...@gmx.net> wrote:

On 30.08.2012 21:01, Alex Cruise wrote:

*With a foreground task running*, ideally Ctrl-C should kill the task and return control to the REPL.  I guess this implies running each command in a separate thread from the REPL itself, so it can be forcibly stopped--but of course, this doesn't necessarily imply /starting a *new* thread/ for each command--we could just keep one of them around to run user code in--until it dies, only then we start a new one.

Wouldn't this still leave the problem with ThreadLocals though? Any previously defined ThreadLocals will suddenly lose all their values after you press Ctrl-C.

Yes, I would consider this a minor drawback of the suggestion.

 

I don't know how big of a problem this is in practice, but it was explicitly mentioned by Paul as one of the reasons for not using different threads for different REPL commands.

If you ask us to kill the thread, we kill the thread.  If this breaks your program, you get to keep both pieces. :) Running *each line in its own thread* would be guaranteed to cause problems like this, but we shouldn't do that.

 

So you would still end up having to do the nasty reflection hack of copying ThreadLocal values into the new thread...

*shrug* if it turns out to be a problem in practice, we can revisit.

-0xe1a

[Alex Cruise]

Thanks! https://issues.scala-lang.org/browse/SI-6302

-0xe1a