new REPL

[Stefan Karpinski]

Good news, everyone! The pure-Julia read eval print loop (REPL) that Keno Fischer developed and Mike Nolta integrated into base Julia has just been merged. There are a number of nice things about changing from the old REPL to this new one, in no particular order:

• The old REPL used the GNU readline library, which we had hacked far beyond what it was ever meant to do. This made modifying it a bit terrifying and thus issues with it tended to get ignored or shelved as "we'll be able to do that in the new REPL".
• The new REPL, is pretty clean, simple Julia code. Seriously – terminal support, line editing, and the REPL itself are less than 2000 lines of code – total. This works out to a net code reduction of 33233 lines of code (GNU readline is 34640 lines of C), while gaining functionality. That has to be a project record.
• The new code is infinitely easier to modify, fix and improve, so REPL-replated bugs will probably get fixed lickety split going forward.
• The old GNU readline REPL was one of our GPL library dependencies that make the total Julia "product" GPL. We'd like to shed these or make them optional to allow for a non-GPL, MIT-licensed Julia distribution and this is a major step toward that goal.
• The new REPL code already has fancy features that you wouldn't even think about doing with readline. Try typing "?" or ";" at the prompt and see the REPL mode change form "julia>" to "help>" or "shell>". Cool, huh?
• The new REPL is noticeably snappier than the old one. Combined with the static compilation of julia introduced in 0.3, going from zero to REPL is pretty quick these days.
• Since full-fledged line editing functionality is now built into Base Julia, we can use it everywhere without worrying what libraries people have installed. Once we settle on a good API, you can expect that user code that needs to prompt for input will be just as slick as the REPL itself.

There will, of course, be some hitches and road bumps, but now that this is merged and everyone using Julia master will be testing it, they should get sorted out in short order. Much applause for Keno and Mike for this excellent work.

Stefan

[John Myles White]

This is really great news.

I’m finding that the new REPL has trouble on OS X if you give it input that starts with a tab. For example, the input "\tP = 1” seems to execute an empty expression rather than assign 1 to P.

 — John

<good news everyone.jpg>

[Stefan Karpinski]

As usual, when you discover a problem file an issue.

[Jake Bolewski]

This is really great work Keno and Mike.

I think a great improvemnt would be to make completions were a bit more modular.  That way custom completion callbacks could be added in at runtime in your .juliarc file.  I'm trying to get zsh to do the shell completions but that would only interest people who use zsh :-)

Do we have an ETA on when 0.3 will be released?  This seemed like one of the bigger blockers.

Best,
Jake

[J Luis]

Where does it keeps the commands history? The old  .julia_history is not used (and lost) anymore but the commands "memory" is preserved, though reset to blank.

[Keno Fischer]

~/.julia_history2 unless Mike changed it. The format is different.

[Shaun Walbridge]

Wonderful work Keno and Mike! This is a great addition.

Would it be possible to retain the standard Readline history file format of separating elements by newlines? It looks like history elements in my new .julia_history2 file are NULL terminated, which makes the file harder to use with line-oriented shell tools like ag/ack/grep.

[Keno Fischer]

The problem is the ambiguity between whether you typed the newline or whether it's a record separator. There is hacks to make it work, but I would prefer to have a non-text character to keep it simple. If you don't care when grepping, couldn't you just do 

~/.julia_history2 > tr '\0' '\n' | grep ...

?

[J Luis]

Domingo, 30 de Março de 2014 2:09:03 UTC+1, Keno Fischer escreveu:

~/.julia_history2 unless Mike changed it. The format is different.

Hmm, strange I don't have it (Windows) but I do have a ~/.julia_history
 

[Laszlo Hars]

Cool!

Would it be difficult to hack it, such that if some text is already at the prompt, the UP and DOWN arrows would only recall entries from the history, which start with that text? It really speeds up things.

Another wish would be a way to disable sound (or select a very very short tone). My laptop beeps so loudly when I reach the end of history, that wakes up my family. If I reduce the master sound volume (Win7), I would not hear email notifications. An editable configuration file would be great.

[Stefan Karpinski]

I don't get any sound at all. Sound should probably be off by default. I think that the Matlab style history search would be quite welcomed if you wanted to try to hack it in. Should be fairly doable.

[Keno Fischer]

The sounds is your computer's notification beep, which unfortunately I don't have control over (in volume). An option to turn it on/off would be easy to do though. 

[Guillermo Garza]

Would it be difficult to hack it, such that if some text is already at the prompt, the UP and DOWN arrows would only recall entries from the history, which start with that text? It really speeds up things.

Looking at the logic here might be a good start: https://github.com/zsh-users/zsh-history-substring-search/blob/master/zsh-history-substring-search.zsh 

[Shaun Walbridge]

Yeah, fair enough -- that'd work fine for my needs.

[Amit Murthy]

If we are on the latest master, I guess we can safely `rm -Rf deps/readline-6.2` ?

[Job van der Zwan]

On Saturday, 29 March 2014 20:59:19 UTC+1, Stefan Karpinski wrote:

• The new REPL, is pretty clean, simple Julia code. Seriously – terminal support, line editing, and the REPL itself are less than 2000 lines of code – total. This works out to a net code reduction of 33233 lines of code (GNU readline is 34640 lines of C), while gaining functionality. That has to be a project record.
• The new code is infinitely easier to modify, fix and improve, so REPL-replated bugs will probably get fixed lickety split going forward.

 TIL infinity is approximately 33/2. Seriously though, that's amazing. Well done guys!

[Stefan Karpinski]

Hey, in networking protocols, infinity is often 15.

[Tomas Lycken]

After updating to latest master and recompiling julia, firing up a terminal and doing "julia" gives a user experience that is comparable to "python" in speed (i.e. both are ready to roll in less than a second), and the Julia REPL has a bunch of nice colors. I call that a win =)

This is great work indeed!

// Tomas

[Stefan Karpinski]

Starting up Python is still 10x faster, but we're definitely getting there. Once we get down to 20 ms, I'll be really happy with startup time. Then it will be completely reasonable to write short-lived command-line tools in Julia. Of course, that might happen sooner than we can make the REPL startup that fast since it may be possible to just compile the tools to executables that don't require most of Julia.

[J Luis]

One thing I'm seeing is that my local build starts way faster (less than a sec) than the nightly (~3 sec)
 (Windows here)

[Ismael VC]

It would be nice if one could switch to the "shell" prompt but leaving it there after the command ends instead of returning to the "julia" prompt, in order to type an arbitrary number of short commands, for example using doulbe ";;" or whatever:

I noticed that while in the "shell" prompt my zsh aliases still work!

Is the idea of the new REPL to be like the ipython REPL?

ismaelvc@toybox ~ % ipython

Python 3.4.0 (default, Mar 17 2014, 22:57:51) 

Type "copyright", "credits" or "license" for more information.

IPython 1.2.1 -- An enhanced Interactive Python.

?         -> Introduction and overview of IPython's features.

%quickref -> Quick reference.

help      -> Python's own help system.

object?   -> Details about 'object', use 'object??' for extra details.

In [1]: test = 'hello'

In [2]: !echo $test

hello

In [3]: files = !ls

In [4]: files

Out[4]: 

['003_overview.pdf',

 'Algebra_Lineal.ipynb',

 'Apuntes_TESCI',

 'ArduinoSketches',

 'bash_script.sh',

 'Books',

 'circular_tree.png',

 'Contabilidad_140314.ipynb',

 'Devel',

 'Downloads',

 'Dropbox',

 'ejemplos.py',

 'example1_graph.png',

 'example2_graph.png',

 'fizzbuzz.jl',

 'hola.py',

 'HORARIO_2.png',

 'imageDownloader',

 'julia',

 'julia_hola.jl',

 'Junk',

 'learn_julia.ipynb',

 'mail_dual',

 'Music',

 'networkx_example.py',

 'nothing',

 'Pictures',

 'Probabilidad_Estadistica.ipynb',

 'problem_1.py',

 'Proyecto_Euler',

 '__pycache__',

 'pydot_example_directed.py',

 'pydot_example.py',

 'pyramid.py',

 'python2_test.py',

 'python_hola.py',

 'python_segfault',

 'qt-recordMyDesktop-crash.log',

 'Roms',

 'sketchbook',

 'suma_modulo.py',

 'TESCI',

 'testing_julia.ipynb',

 'test.py',

 'Untitled0.ipynb',

 'VirtualBox VMs']

In [5]:

Do you really want to exit ([y]/n)?

ismaelvc@toybox ~ % julia

               _

   _       _ _(_)_     |  A fresh approach to technical computing

  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org

   _ _   _| |_  __ _   |  Type "help()" to list help topics

  | | | | | | |/ _` |  |

  | | |_| | | | (_| |  |  Version 0.3.0-prerelease+2282 (2014-03-31 04:34 UTC)

 _/ |\__'_|_|_|\__'_|  |  Commit 1651a97 (1 day old master)

|__/                   |  i686-pc-linux-gnu

julia> test = "helo"

"helo"

shell> echo $test

helo

julia> files = ;ls

ERROR: syntax: unexpected ;

Ctrl+Left/Right to move the cursor one word at a time, doesn't work, I suppose, that functionality comes from readline.

It would be nice to have julia as my default "shell" with custom profiles and the avility to mix shell commands with the language commands, I have used ipython as a shell with some succes thanks to goodies like in the paste above.

Startup is also noticeably faster, keep up the good work! :-)

[Patrick O'Leary]

On Tuesday, April 1, 2014 12:56:09 AM UTC-5, Ismael VC wrote:

It would be nice if one could switch to the "shell" prompt but leaving it there after the command ends instead of returning to the "julia" prompt, in order to type an arbitrary number of short commands, for example using doulbe ";;" or whatever:

https://github.com/JuliaLang/julia/issues/6320

[Ismael VC]

Also having colors by default is nice, but I'm wondering if those can also e aplied to the backtraces, like in ipython:

# PYTHON (no colors but "descriptive")

>>> "test" / 0

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

TypeError: unsupported operand type(s) for /: 'str' and 'int'

# IPYTHON (uses colors and is descriptive)

In [1]: "test" / 0

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-1-24bdacdad2fc> in <module>()

----> 1 "test" / 0

TypeError: unsupported operand type(s) for /: 'str' and 'int'

# JULIA (uses only red, not descriptive)

julia> "test" / 0

ERROR: no method /(ASCIIString, Int32)

 

[Job van der Zwan]

Still more accurate than most approximations used in astronomy ;)

[Kevin Lin]

The new repl's great.  One very minor question: occasionally I find it convenient to be able to run julia inside emacs, and the new repl doesn't interact well with either ess (I have v13.09) or just a plain shell.  Is there an easy way to e.g. tell julia that it's running in a dumb terminal?

Thanks,

Kevin

[cdm]

i suspect that there is a way to do this ...

suppose that one needed to change the

Julia prompt from

julia>

to say, something like

j~

is this defined in Base, or somewhere else?

many thanks,

cdm

[Steve Kelly]

The prompt is defined here: https://github.com/JuliaLang/julia/blob/2eee58701052caf6b6927604732e197b1054ac7b/base/REPL.jl#L191

I was looking into it so I could number lines for my SaveREPL.jl package, but it looked like a lot of refactoring.

[Keno Fischer]

No, that's not the right REPL. The one in the terminal is the
LineEditREPL. We need better repl configuration, but you can change
the prompt like this:

Base.active_repl.interface.modes[1].prompt="abc> "

[cdm]

perfect ...

this is expected to aid the implementation

of Julia magics from Sage Notebooks over

on SMC {cloud.sagemath.com}.

many thanks !

cdm

[cdm]

loosely related request ...

could someone point me to where

customizations of the REPL are

described?

for example, suppose the user was

interested in changing the color

schemes in the REPL so that

the "julia>" prompt was say

yellow instead of the green from

the three dots logo, and user

entry was in the same purple

from the three dots logo, etc.

also is there a command to

generate the Julia logo header

that is served at the start of

every REPL launch.

many thanks,

cdm

[T3X]

On Thursday, 30 October 2014 16:46:55 UTC, cdm wrote:

also is there a command to

generate the Julia logo header

that is served at the start of

every REPL launch.

Base.banner()

[Steven G. Johnson]

On Thursday, October 30, 2014 12:46:55 PM UTC-4, cdm wrote:

for example, suppose the user was

interested in changing the color

schemes in the REPL so that

the "julia>" prompt was say

yellow instead of the green from

the three dots logo, and user

entry was in the same purple

from the three dots logo, etc.

I don't think this is easily customizable right now; it doesn't seem to be documented at all.

The prompt color is set by the default LineEditREPL constructor (in REPL.jl).  The result color can be changed via the environment variable JULIA_ANSWER_COLOR (black/red/green/yellow/blue/magenta/cyan/white/normal/bold) and the input color can be changed via the environment variable JULIA_INPUT_COLOR.  The shell and help colors are also set in the LineEditREPL constructor.

In principle, I think it would be possible to replace the REPL with a different REPL (e.g. LineEditREPL called with different colors passed to its constructor) by changing Base.active_repl in your .juliarc file (since this is loaded before the REPL is started); you'll also need to call popdisplay() to remove the original REPL from the display stack and push the new REPL display via pushdisplay(Base.REPL.REPLDisplay(active_repl)).  However, this seems a bit painful...you'll have to replicate a bunch of the logic in the _start() function from base/client.jl.

[Steven G. Johnson]

On Thursday, October 30, 2014 1:46:11 PM UTC-4, Steven G. Johnson wrote:

On Thursday, October 30, 2014 12:46:55 PM UTC-4, cdm wrote:

for example, suppose the user was

interested in changing the color

schemes in the REPL so that

the "julia>" prompt was say

yellow instead of the green from

the three dots logo, and user

entry was in the same purple

from the three dots logo, etc.

I don't think this is easily customizable right now; it doesn't seem to be documented at all.

Actually, there is a much easier way to customize it.  In your juliarc file (~/.juliarc.jl), you can do e.g.:

Base.active_repl.prompt_color = Base.text_colors[:cyan]

to change the prompt color.  Similarly for the other fields of active_repl (e.g. input_color, answer_color, shell_color, and help_color).   You also need to set Base.active_repl.envcolors = false in order to override the input_color and the answer_color.

[cdm]

figures, it would be easy.

why is everything so easy

in Julia ... ?

cdm

[Jiahao Chen]

On Thu, Oct 30, 2014 at 3:02 PM, cdm <cdmcle...@gmail.com> wrote:

why is everything so easy

in Julia ... ?

Why is everything so hard in not-Julia?

[Stefan Karpinski]

LoL. That's flattering. There are certainly other languages that like to make things easy, we just follow in that tradition as much as possible.

[Rob J. Goedman]

Wow, works! No more compiling of 0.3.2 for me!!!

Thank you!

Rob J. Goedman

goe...@icloud.com

On Aug 21, 2014, at 11:57 AM, Rob J. Goedman <goe...@icloud.com> wrote:

Just wondering, I used to change the color of the help prompt by editing REPL.jl and re-making julia.

With the release of 0.3 I'm trying to stick to the official version.

Is there a way to change these colors (:yellow for help conflicts with my background color), maybe when starting Julia REPL?

Thanks,

Rob J. Goedman

goe...@icloud.com

[cdm]

On Thursday, October 30, 2014 1:04:52 PM UTC-7, Stefan Karpinski wrote:

LoL. That's flattering. There are certainly other languages that like to make things easy, we just follow in that tradition as much as possible.

flattering and COMPLEMENTary !