Gorilla REPL in Light Table

[Jony Hudson]

Hi Light Tablers,

 I'm the author of a browser-based rich Clojure nREPL client called Gorilla REPL. It's just like a normal REPL, but has flexible rendering of output values - so you can plot things like graphs, tables etc - and it lets you save REPL sessions as "worksheets" which combine code, markdown notes, and rendered output. People often compare it to iPython notebook. Probably the most common feature request I get is integration with Light Table, so I thought I'd post up here to canvas opinion, and get some technical pointers :-)

Personally, I think it would be great to integrate with Light Table, as while the core of Gorilla works nicely, it's rough around the edges - in areas like file handling, usability (keymaps aren't configurable, commands aren't so easy to discover ...), managing nREPL connections. These are all things that Light Table has got sorted out really nicely. The question is how best to integrate it.

Perhaps the most direct option would be to try and drop the Gorilla "editor" directly into Light Table. Currently, Gorilla's editor is written in JS, using knockout.js to connect its data to the UI, and uses multiple CodeMirror instances per worksheet to handle text editing. I could imagine packaging the core into a JS library that could have API to hook in to a Light Table plugin. The big questions here would be: can knockout.js work inside Light Table? would Gorilla's approach of expecting to handle key/mouse events directly be compatible? how does Light Table handle document data - could the Gorilla editor just have API to get a plain-text serialised model on save, or is it more complicated than that?

If dropping the full Gorilla app in as an editor isn't possible, then there are a number of possibilities that I can see, but perhaps it's best to stay focussed on that idea first.

I'd love to hear thoughts on this, and any suggestions on how to implement it would be very welcome. Also, any pointers to understanding the structure of LT's code would be great - I think I understand the basics of BOT, but I'm finding it difficult to put together all of the pieces to see how the whole works. Is there any documentation on the overall structure, more specific than the generalities of BOT?

Thanks,

Jony

[Rainer Schuster]

Basically LightTable and Gorilla REPL are doing the same thing. There is a nREPL Backend/Middleware LT sends json serialised messages to and receives from. When you first try to eval, LT is going to establish a connection/session. A good walkthrough is the 5 post series about the Groovy Plugin. This is really a good series:

http://codewader.blogspot.de/2014/02/a-groovy-light-table-client-step-1.html

http://codewader.blogspot.de/2014/02/a-groovy-light-table-client-step-2.html

http://codewader.blogspot.de/2014/05/a-groovy-light-table-client-step-3.html

http://codewader.blogspot.de/2014/05/a-groovy-light-table-client-step-4.html

http://codewader.blogspot.de/2014/06/a-groovy-light-table-client-step-5.html

You can use whatever Javascript you like to use. There is also a Tutorial about using JS but can't find it ... either search the Forum here or wait for someone else to answer. 

The only thing you probably have to do is hooking in a behavior that reacts on the result. Probalby it already works just returning the html. And all you have to provide is your renderer as dependency.

You don't need to have several Codemirror editors, as the one you are using is sufficient. there is a ::eval behaviour which evals all of the editors content and eval.one, which in the case of Clojre evals the outermost form under your cursor or the previous if pointed at the end.

the :result behaviour then needs to do something with the result. It contains the client id, which was sending the request, the editor positions and the result (IIRC the structure is described in the Groovy Blogpost Series). 

As far is I understand it (from my limited knowledge) all you have to do is write a behaviour for the :result and display the html content. Because the rendering is part of the REPL session and only an added dependency. Therefore you only have to handle displaying the html content from your renderer properly

Also have a look at the Clojure-Plugin which you are going to address later: https://github.com/LightTable/Clojure/blob/master/src/lt/plugins/clojure.cljs 

If you really like to have your interface as a first step you can simply create a browser tab and launch the interface, without further LT integration. and create 2 commands, one for launching the server and one for shutting the server down. If you wanna go this route http://app.kodowa.com/playground/lein-light is probably helpful

Here is a plugin to create a simple project skeleton

https://github.com/mdhaney/lt-plugin-template 

[Jony Hudson]

Hi Rainer,

 thanks, this is all really helpful :-) I'll take a close look at those posts.

I've been playing a little, and have got knockout controlling the contents of a tab, so am hopeful that the plan of dropping the Gorilla editor in "as is" might work out. If this works, it would be a good first step I think.

It would be interesting to bring the option of using Gorilla's renderer to format the inline results, as you describe, and this is definitely something I'll look at.

There's also a bigger question of re-implementing the editor portion of Gorilla using the editing facilities that LT provides. This would certainly have some advantages, and would provide a uniform editing experience for the user. I'll have to look into the details of CodeMirror to see whether it seems practical to provide the sort of rich-document thing I'm after within it.

Thanks again for your help,

Jony

[Matt Mower]

Jony I've no idea if I can help... I'm not so well versed in the LT internals although I did mess with CodeMirror to write a simple plugin so if you have an issue tracker for this project I'll keep an eye on it.

But in any case I loved what you've done with Gorilla REPL and I'd love to see it integrated into LT.

Best,

Matt

[cldwalker]

On Wednesday, July 9, 2014 7:59:39 AM UTC-4, Jony Hudson wrote:

Hi Light Tablers,

 I'm the author of a browser-based rich Clojure nREPL client called Gorilla REPL. It's just like a normal REPL, but has flexible rendering of output values - so you can plot things like graphs, tables etc - and it lets you save REPL sessions as "worksheets" which combine code, markdown notes, and rendered output. People often compare it to iPython notebook. Probably the most common feature request I get is integration with Light Table, so I thought I'd post up here to canvas opinion, and get some technical pointers :-)

Welcome. Would be great to get Gorilla working within LT :) 

Personally, I think it would be great to integrate with Light Table, as while the core of Gorilla works nicely, it's rough around the edges - in areas like file handling, usability (keymaps aren't configurable, commands aren't so easy to discover ...), managing nREPL connections. These are all things that Light Table has got sorted out really nicely. The question is how best to integrate it.

Perhaps the most direct option would be to try and drop the Gorilla "editor" directly into Light Table. Currently, Gorilla's editor is written in JS, using knockout.js to connect its data to the UI, and uses multiple CodeMirror instances per worksheet to handle text editing. I could imagine packaging the core into a JS library that could have API to hook in to a Light Table plugin. The big questions here would be: can knockout.js work inside Light Table? would Gorilla's approach of expecting to handle key/mouse events directly be compatible? how does Light Table handle document data - could the Gorilla editor just have API to get a plain-text serialised model on save, or is it more complicated than that?

If you still want to trigger a behavior on save, you'll need to create a custom behavior that triggers on an editor object's :save like ::file-save [1]. To create an editor object, see how pool.cljs does it [2]. Once you have an editor object, you can access the underlying CodeMirror editor using editor/->cm-ed [3]. With direct access to the editor object, you can convert your codemirrorVM.js to optionally take it as an arg. However, I'm not sure if LT's editor object makes assumptions about rendering that will conflict with your current rendering.

[1] https://github.com/LightTable/LightTable/blob/master/src/lt/objs/editor/file.cljs#L11-L23

[2] https://github.com/LightTable/LightTable/blob/master/src/lt/objs/editor/pool.cljs#L196

[3] https://github.com/LightTable/LightTable/blob/master/src/lt/objs/editor.cljs#L58-L61

Cheers,

Gabriel

[Jony Hudson]

Hi Gabriel,

 thanks for the pointers, very useful.

I've got a toy plugin together, but I'm having a bit of trouble with implementing "Save". I _think_ it raises a wider question ... If I understand correctly, when the user hits Cmd+s, or File->Save then, assuming something with tag :editor is active, this triggers the :save command in opener.cljs [1] (i.e. not any save behaviour directly). The :exec key for this command then gets the last active editor from the pool, using a function call (not via raise), and raises a :save trigger on it. And the pool is closely coupled with the default CodeMirror editor implementation. I guess what I'm saying is, it looks like the editor implementation is "hard-coded" in, and it's not possible to plug in a new editor component using BOT. Does that sound like I've understood the situation correctly?

I guess, if that is the case, then a further question would be whether there's any desire to make editor components pluggable, or whether that's just too much customisation?

Jony

[1] https://github.com/LightTable/LightTable/blob/master/src/lt/objs/opener.cljs#L220-L224

[cldwalker]

On Saturday, July 12, 2014 7:07:13 AM UTC-4, Jony Hudson wrote:

Hi Gabriel,

 thanks for the pointers, very useful.

I've got a toy plugin together, but I'm having a bit of trouble with implementing "Save". I _think_ it raises a wider question ... If I understand correctly, when the user hits Cmd+s, or File->Save then, assuming something with tag :editor is active, this triggers the :save command in opener.cljs [1] (i.e. not any save behaviour directly). The :exec key for this command then gets the last active editor from the pool, using a function call (not via raise), and raises a :save trigger on it. And the pool is closely coupled with the default CodeMirror editor implementation. I guess what I'm saying is, it looks like the editor implementation is "hard-coded" in, and it's not possible to plug in a new editor component using BOT. Does that sound like I've understood the situation correctly?

You've read correctly that pool/last-active just pulls the current editor object. However, the current CM editor object can be customized at will. As a proof of concept, here's a command that changes the undoDepth option:

(cmd/command {:command :customize-editor

              :desc "TODO"

              :exec (fn []

                      (let [ed (pool/last-active)]

                        (prn (editor/option ed :undoDepth))

                        (editor/set-options ed {:undoDepth 3000})

                        (prn (editor/option ed :undoDepth))))})

See the console log to verify the change takes effect.

Whether you use editor/set-options or call something directly on the CodeMirror object e.g. (.someFn (editor/->cm-ed ed) ...) , you should be able to customize the current editor.

Alternatively, if it's not possible to update the current CM editor object to be gorillable, you could try updating the :ed key of the current LT editor object with a new CodeMirror object:

(let [ed (pool/last-active)]

  (lt.object/update! ed [:ed] (js/CodeMirror. ...)))

However, I haven't tried that and don't know if that would have any unexpected consequences.

Hope this helps,

Gabriel

[Michael O'Keefe]

Just happened upon this post.

Light Table has the "browser tab" feature. One thing I tried to do with the Gorilla REPL in the past and again right now is just to get Gorilla running inside the "browser tab" and use it from there. One thing that prevents me from going forward with that workflow is that the Gorilla REPL keybindings don't seem to reach the Gorilla REPL instance: Shift+Enter, Ctrl+g Ctrl+x, and friends don't do anything.

As a fan of both Light Table and Gorilla REPL, I would find it valuable just to get Gorilla's key bindings to be recognized in the "browser tab" in Light Table -- that would be an 80% solution for me. Is that something that can be addressed via a Light Table behavior configuration?

Cheers,

Michael