Julia multiline input on dumb terminals?

[Vitalie Spinu]

Hi,

I am trying to integrate julia with ESS (http://ess.r-project.org/), as
it offers sub-process interaction (such as sending functions, lines,
regions directly form the editing buffer in emacs). But I am stuck with
making multiline input to work.

First of all "julia" exec is not working at all. One line input works
fine with julia-release-basic.

Multiline problem is as follows:

Start emacs shell (M-x shell), start julia-release-basic, and run:

function f(x,y)
x + y
end

You will get:

julia> function f(x,y)
x + y
end
not supported

julia> x not defined

julia> syntax error: unexpected end

Which means that julia sees each line as a complete input.

I've tried sending input to subprocess both with \n and \r, same
result.

Any ideas?

Thanks,
Vitalie.

[Mark Henderson]

Just a guess, do you see the same result with julia and/or julia-release-readline?

On Sunday, April 1, 2012 12:32:00 PM UTC-5, Vitalie Spinu wrote:

Hi,

I am trying to integrate julia with ESS (http://ess.r-project.org/), as
it offers sub-process interaction (such as sending functions, lines,
regions directly form the editing buffer in emacs). But I am stuck with
making multiline input to work.

First of all "julia" exec is not working at all. One line input works
fine with julia-release-basic.

Multiline problem is as followStart emacs shell (M-x shell), start julia-release-basic, and run:

   
    function f(x,y)
      x + y
    end
   

You will get:

                                   
    julia> function f(x,y)            
      x + y                            
    end                                
    not supported                      
                                       
    julia> x not defined              
                                       
    julia> syntax error: unexpected end
   

Which means that julia sees each line as a complete input.

I've tried sending input to subprocess both with \n and \r,  same
result.

Any ideas?

Thanks,
Vitalie.

[Douglas Bates]

On Sunday, April 1, 2012 12:32:00 PM UTC-5, Vitalie Spinu wrote:

Hi,

I am trying to integrate julia with ESS (http://ess.r-project.org/), as
it offers sub-process interaction (such as sending functions, lines,
regions directly form the editing buffer in emacs). But I am stuck with
making multiline input to work.

First of all "julia" exec is not working at all. One line input works
fine with julia-release-basic.

I'm glad to hear that you are working on ESS and Julia, Vitale.  I got about as far as you did, using julia-release-basic, and encountered much the same problems.

I will be delighted when you learn how to solve these problems.

It may be a roundabout approach but you could write the multiline input to a temporary file and use the Julia function load() to load that file.  This doesn't seem like a good long-term solution but it may help just to get things going.

I assume you are aware of the file contrib/julia-mode.el in the Julia source tree as an editing mode.  Linking the ESS-style running an inferior process with the editing mode would be wonderful.

  

[Vitalie Spinu]

>>>> Mark Henderson <mark...@gmail.com>

>>>> on Sun, 1 Apr 2012 11:04:14 -0700 (PDT) wrote:

> Just a guess, do you see the same result with julia and/or
> julia-release-readline?

Just julia executable is not working at all. Once started, it gets stuck.

[Vitalie Spinu]

>>>> Douglas Bates <dmb...@gmail.com>

>>>> on Sun, 1 Apr 2012 11:39:25 -0700 (PDT) wrote:

> I assume you are aware of the file contrib/julia-mode.el in the Julia
> source tree as an editing mode.

Yes, I am basing on that one.

> Linking the ESS-style running an inferior process with the editing
> mode would be wonderful.

Plenty of functionality is already in place in ESS. Things like
completions, eldoc and interactive help could be done in a day or
so. Just to figure out what's the problem with "\n".

It's some sort of IO glitch in Julia. Simple proc spawning like this:

(setq proc (start-process "julia" "julia"
"~/VC/julia/julia-release-basic"))
(process-send-string proc "function next(s::IntSet, i)\n")

gives "julia> not supported".

By the way, it would be nice to have a secondary prompt for incomplete
input, something like "+ " at bol. It's easier to track input/output
regions this way.

Vitalie.

[Viral Shah]

This is indeed great. Although I do not use ESS, I have friends that swear by it. @StefanKarpinski, since you implemented the multi-line stuff, can you tell what might be going on?

-viral

[Stefan Karpinski]

Sorry for the delayed reply. I'm super behind on email due to the Lang.NEXT trip. I implemented the multiline handling in our readline repl ui, but I'm not sure what the basic repl is doing. Possibly just reading line line at a time and evaling each one, which is obviously not going to work for multiline expressions. The thing to do there would be to emulate the readline repls ability to read a line of input and use the parser to decide whether it's syntactically invalide (raise an immediate error), incomplete (wait for more lines), or a complete and syntactically valid expression (eval it and show the result). From subsequent emails it seems like you may have figured this out — is that the case?

[Vitalie Spinu]

Hi Stefan,

No I didn't figured it out, just implemented an workaround in ESS --
save to a temp file and load. It works fine for sending chunks of code
to julia sub-process. It doesn't work for line by line accumulation of
an input, both from script and process buffers.

As far as I understand the wait-till-valid-input mechanism is
implemented. Would it be possible to add it to julia process running
with no readline facility?

Thanks.

>>>> Stefan Karpinski <ste...@karpinski.org>

[Stefan Karpinski]

I'm not sure how ESS works, but might it not make more sense for emacs code to be able to link against libjulia directly and be a completely separate UI instead of trying to wrap the dumb terminal with all the complications entailed by that?

[Vitalie Spinu]

Sorry Stefan, I somehow missed your message, and saw it only now after
emacs shell problem popped again on the list.

Linking is not an option for two reasons. It's not possible to call C
code without recompiling Emacs. And you don't want to be tied to only
one version of Julia, R, stata etc. There could be dozens of different
versions installed and ESS should work with all of them.

The only option is to communicate with Julia is with a spawned
sub-process. It's very similar (if not identical) to how named pipes
work:

mkfifo ju
(tail -f ju) | ./julia-release-basic

Now from other terminal

echo 5 + 6 > ju

You should see 11 on the first console.

Currently julia executable is tangled with readline library. Beyond the
obvious disadvantage of not being able to run julia on terminals not
supporting readline, it's also messes up custom readline
configuration. I have plenty of shortcuts and other readline config in
my .inputrc. When Julia starts, none of them works anymore. Note that
this also is a problem of "julia-release-basic" and my readline
shortcuts are also useless there. This is not the case of other
sub-processes which I've tried - R, python, bash, stata.

For now, it would be really great if the original issue could be
solved. I.e. make julia-release-basic to wait for the complete input.

Would be also nice to have a different secondary prompt when Julia is
expecting input, such as "+ ". It's nicer for the user and easier to
track from external IDEs.

Thanks a lot for looking into this.

>>>> Stefan Karpinski <ste...@karpinski.org>

[Jeff Bezanson]

Ok, we should make julia-release-basic wait for complete input.

What do you mean by "tangled" with readline? Linked to? :) What would
we be doing with readline that messes with your configuration?
julia-release-basic does not use readline at all so of course readline
configuration won't work.

[Jameson Nash]

I've setup the libuv fork to handle multiline input in basic mode (because of the mention in this thread, in fact). However, the plan ahead for it may depend upon the merge plan and how long people are willing to wait for this.
-Jameson

[Keno Fischer]

I can port that over. The repl changes to basic are minimal.

[Vitalie Spinu]

>>>> Keno Fischer <ke...@stanford.edu>

>>>> on Sun, 6 May 2012 23:59:44 -0400 wrote:

>> > What do you mean by "tangled" with readline? Linked to? :) What would
>> > we be doing with readline that messes with your configuration?
>> > julia-release-basic does not use readline at all so of course readline
>> > configuration won't work.

>> >> Currently julia executable is tangled with readline library. Beyond the
>> >> obvious disadvantage of not being able to run julia on terminals not
>> >> supporting readline, it's also messes up custom readline
>> >> configuration. I have plenty of shortcuts and other readline config in
>> >> my .inputrc. When Julia starts, none of them works anymore. Note that
>> >> this also is a problem of "julia-release-basic" and my readline
>> >> shortcuts are also useless there. This is not the case of other
>> >> sub-processes which I've tried - R, python, bash, stata.

I've written this without trying more recent version of
julia-release-readline. All my readline shortcuts are working now, so
it's solved. BTW, why not shorter names julia-basic, julia-readline?

Somehow python and R manage to handle dumb terminals from the same
executable. I.e. they work with readline in xterm based consoles, and
they also work in emacs shell. I wonder if same thing would be possible
for Julia. I guess this comes to checking $TERM environment variable,
which is "dumb" in emacs shell.

Vitalie.

[Jeff Bezanson]

Glad at least one problem is solved.

I suppose we could load our command line code dynamically. Might be a
nice simplification.