Question about running / driving leo 'headless' to integrate it in my own GUI

[Robert Cholette]

First I'd like to apologize for just asking this on github, I didnt realise here was perhaps a better place to ask this kind of thing... 

Hi! Long time leo user here. I use it mainly for the 'core' features: outline organisation of my code : @clean nodes, structure with @others. And I alt-tab leo alongside with another editor/IDE for running/debugging/compiling/linting/beautifying/etc...

I dont care so much about vim/emacs integration nor dont even understand what those buffers/minibuffers are and all around feel like that qt-gui framework didnt age very well. (no offence meant here, as I would be devastated to think I offended edream, he's like my programming 'idol')

I also do not use @buttons and internal scripting in leo altough i can see its use for some people. 

I mainly use Leo for its 'file-generation'/'file-reading' (mainly @clean) feature via the outline structure that it provides. Organising a program with an outline, clones and @others is the best! Which also, if I may say so, is Leo's 'killer feature'.

So i'd like to try and integrate, or 'roll my own' leo in my other favorite editor so that I have the subset of leo's features that I just defined as its 'killer features' available without having Leo 'opened'. 

Is leoBridge the way to go? ...or is there a way to start leo with no GUI and have it listen for commands on a specific port for input/output of commands and answers? 

Many thanks in advance!!  

[vitalije]

I am interested what editor you intend to use? Maybe I could help.

Vitalije

[Matt Wilkie]

First I'd like to apologize for just asking this on github, I didnt realise here was perhaps a better place to ask this kind of thing...

I've just cross linked to this thread from there (https://github.com/leo-editor/leo-editor/issues/1281).  Perhaps after the discussion has yielded answers the recipe or outline of what to do can be added to the issue. Following the general principle of exploration and following avenues in the mailing list, and fixes and solutions in the issue tracker.

-matt

[Robert Cholette]

I am interested what editor you intend to use? Maybe I could help.

 

I started thinking about this a couple months ago when I mainly used sublime (scriptable in python), but now since I 'switched' to using vsCode & found out it has a flexible outline control available for expansions/plugins : I figured it would be easier & better for me.

So far I've made some tests with leobridge & gonna fiddle a bit more in the next few days. I'm eager to try things out.  I'll report findings on this thread if anything comes out of this!

I've just cross linked to this thread from there

Thank you both Vitalije & Matt !

[vitalije]

I 'switched' to using vsCode & found out it has a flexible outline control available for expansions/plugins : I figured it would be easier & better for me.

I have never had any interest in vsCode before, but after just brief look at the vsCode's "writing extension guides", I do believe it is possible to integrate Leo outline into vsCode quite easily. Of course this is just my first impression. 

Recently I wrote in rust server to keep track of the history of any Leo outline. It communicates through http. vsCode has an extension API for registering virtual document providers. That seems to be a perfect spot to start.

If you are interested we can join the efforts to make it work. You may find useful to look at the rust crates I have recently published:

1. fossil-delta
2. leo-ver-serv  which uses fossil-delta
3. mini_leo (which recently got rust implementation of parsing Leo external files i.e. at-file files and loading .leo files)

I just need to copy relevant code from mini_leo/rust for opening .leo files and reading at-file files into leo-ver-serv and it could be a server which serves both:

1. any requested node
2. outline structure

To use it from vsCode, one have to register a virtualDocumentProvider for some specific URI schema. For example  leo://(path to leo file)/#(some gnx) gives body of the node with given gnx in the given leo file.

The server part I could build in just a few days. It is just a single executable file, that one can put anywhere on the system path.

In leo-ver-serv there is already an implementation of drawing outline. It is written in CoffeeScript. That code can be used for displaying outline in the vsCode. If necessary it can be translated in TypeScript.

Vitalije

[Edward K. Ream]

On Monday, August 12, 2019 at 2:37:26 AM UTC-5, vitalije wrote:

>> I switched to using vsCode & found out it has a flexible outline control available for expansions/plugins : I figured it would be easier & better for me.

> I have never had any interest in vsCode before, but after just brief look at the vsCode's "writing extension guides", I do believe it is possible to integrate Leo outline into vsCode quite easily. Of course this is just my first impression.

> Recently I wrote in rust server to keep track of the history of any Leo outline. It communicates through http. vsCode has an extension API for registering virtual document providers. That seems to be a perfect spot to start.

> If you are interested we can join the efforts to make it work.

Oh joy.  Thanks for this.

Edward

[vitalije]

Oh joy.  Thanks for this.

Edward

I am glad you like the idea. This morning I have installed vsCode and a few of its extension modules. Editing rust code gives code completion, displays info on types, fields and methods on mouse hover, warns about code mistakes,... All this functionality comes from rls - rust language services a server written in rust which analyzes code, builds it, runs it,...

It also analyzes git and displays blame information for current line  (who and when the line was last edited) if the file is under version control.

Perhaps Leo should use rls at least when editing rust. There might be similar language services for other languages too. I vaguely remember hearing about some standard for implementing such services.

And definitely Leo could use leo-service for heavy lifting. At least two areas could be improved with the code I have already written:

1. parsing external files
2. calculating visible tree items for fast drawing only visible nodes 

Vitalije

[Edward K. Ream]

On Mon, Aug 12, 2019 at 6:06 AM vitalije <vita...@gmail.com> wrote:

All this functionality comes from rls - rust language services a server written in rust which analyzes code, builds it, runs it,...

Cool.

Perhaps Leo should use rls at least when editing rust.

I'll leave this question for you.  At present all my brain cells are engaged with pyzo ;-)

Edward

[Robert Cholette]

I started reading on vscode's extensions capabilities, and so I started fiddling a bit and made a skeleton expansion for now, and I have started a project at https://github.com/boltex/leointeg

clone, run npm install, F5 to run in vscode. You should see the icon for the 'leo mode' on the left activity bar, an empty outline pane, and no body pane for now...

Just an empty framework, started from the default expansion examples from microsoft. If you have python support enabled in vscode, the python file in the src folder should run without errors if you right-click in the editor window & run in terminal... it's intended to be executed by the expansion to deal with leo via leoBridge.

i've outlined a bit of concepts/ideas in the readme.

I guess there's more done on the icon than anything else for now, I'll have time to work on it this week end :) 

cheers!

[Robert Cholette]

ok, First working version which permits outline browsing for now. 

This is promising! 

Next step will be showing proper node icons -  I have made icons for nodes states. Then I will flesh out more in days to come and add editing functionality. 

Thanks for your support and insights, driving via leoBridge is easy and simple, thanks to Edward's examples of p_to_ap and ap_to_p. 

Vitalije, thanks also for your input. although good ideas and suggestions, I want something in the short term so i guess i wont be redoing leo in typescript ;)

edit: typos and again, the link for this project is : https://github.com/boltex/leointeg

[Robert Cholette]

Got so far as to browse with appropritate node icons. 

a part from editing as in leo, long term goals are to Go-to appropriate line in generated files when debugging/breakpoints etc.. (Reproducing part of xcc-nodes behaviour, see http://xccnode.sourceforge.net/) 

and have file-generating 'at' nodes show their derived line number instead of the body-pane's line number (Also reproducing xcc-nodes)

Added screenshot to readme: 

( yeah i've got to de-saturate the colors in the icons! )

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[john lunzer]

Impressive progress. This is a pretty exciting development. While many "viewers" have been demonstrated this is one of the first "Leo in an editor" type demonstrations I've seen. Nice work. I look forward to seeing where this goes.

[Robert Cholette]

Thanks! 

Now that I've reached this milestone I'm going to cleanup/refactor the code : The js (typescript) part is not too bad, but the python side has no classes (just functions and globals) so this needs a little organisation!

Then next (maybe this weekend) I'll make the body pane show actual body of selected node and permit tree and body edition and saving.

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[Edward K. Ream]

On Thu, Aug 29, 2019 at 6:11 AM john lunzer <lun...@gmail.com> wrote:

Impressive progress. This is a pretty exciting development. While many "viewers" have been demonstrated this is one of the first "Leo in an editor" type demonstrations I've seen. Nice work. I look forward to seeing where this goes.

Me too.  And discussing this here is just fine ;-)

Edward

[Edward K. Ream]

On Thu, Aug 29, 2019 at 9:10 AM Robert Cholette <felix...@gmail.com> wrote:

Now that I've reached this milestone I'm going to cleanup/refactor the code : The js (typescript) part is not too bad, but the python side has no classes (just functions and globals) so this needs a little organisation!

Then next (maybe this weekend) I'll make the body pane show actual body of selected node and permit tree and body edition and saving.

Thanks for the update.  I've bookmarked your repo and look forward to studying it in detail.  This will be useful as potential bridge to Joe Orr's leovue project.

Edward

[Robert Cholette]

First Prototype Goal Achieved! 

Browsing Now shows body text too, just like browsing in Leo, casually without editing body nor headlines. (read only for now)

Next steps I'll use the 'FileSystemProvider' API in vscode to 'fake' the body text so that it's editable. 

Other than that for now editing the headline would be nice. I'll get to that shortly. :)

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[Edward K. Ream]

On Friday, August 30, 2019 at 10:57:23 PM UTC-5, Robert Cholette wrote:

First Prototype Goal Achieved! 

Well done.  Very exciting.

Edward

[Robert Cholette]

(in case someone's trying some kind of vscode integration)

Just a note about 'force open' a particular file when writing a vscode extension: I just spent time reading about vsCode's definition and behaviour of what it considers a 'workspace', and also, a project's root folder.

Turns out that by default, they are the same, the root folder you've opened, lets say a project's folder, is what it considers its workspace and that there can only be one workspace at a time in an opened instance of vscode. 

But most importantly, when changed it *restarts* all extensions! oops! Goes to show i'm pretty new to vscode! 

That's what kept me from being able to 'force open' properly a view of my body text from a virtual filesystem. (i used a workspace-changing function to open it instead of a simpler 'open file' thingy function...)

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[Robert Cholette]

leointeg now updated with 'real' editable body pane made out of a fake document in a custom filesystem. does not update leo side yet.

 
On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[Edward K. Ream]

On Tue, Sep 3, 2019 at 7:33 PM Robert Cholette <felix...@gmail.com> wrote:

(in case someone's trying some kind of vscode integration)

Just a note about 'force open' a particular file when writing a vscode extension: I just spent time reading about vsCode's definition and behaviour of what it considers a 'workspace', and also, a project's root folder.

Turns out that by default, they are the same, the root folder you've opened, lets say a project's folder, is what it considers its workspace and that there can only be one workspace at a time in an opened instance of vscode. 

But most importantly, when [the workspace is] changed it *restarts* all extensions! oops! Goes to show i'm pretty new to vscode! 

Thanks for this.  There is no substitute for experience.

Edward

[Edward K. Ream]

On Tue, Sep 3, 2019 at 11:06 PM Robert Cholette <felix...@gmail.com> wrote:

leointeg now updated with 'real' editable body pane made out of a fake document in a custom filesystem. does not update leo side yet.

Glad to hear that work continues!

Edward

[Robert Cholette]

Just a note about placement of the outline pane in vscode as an extension: a tree view widget can also easily placed on the same panel as the main file-explorer, like this 'nodedependencies' below : 

So that's something I might go with, or offer as an option.

If anyone has sugestions for integration into vscode that would not be obvious, like keyboard shortcus to be mapped over when having a leo element selected, or anything that might be going through your head as you ponder about having leo seamlesly integrated in a modern IDE, please go ahead and mention your ideas, suggestions or requests. Thanks!

--

Félix

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[Robert Cholette]

Added right click menu and hover actions on all items in outline pane.

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[Robert Cholette]

Just a copy / paste in case someone joins this thread:

About a month ago I did a bunch of tests with vscode's extension capabilities along with tests with leoBridge's control of Leo. Mostly throwing spaghetti at the wall and seeing what sticks. I got excited, you guys kind of liked it I guess, but it all added up to a whole bunch of spaghetti code mess as you may have noticed if you browsed my sources. Mostly so in the 'communication' between vscode and the python script (Leo) part.

I sincerely apologize for this and present you now instead with a cleaned up version, where code-duplication was eliminated and all processes are now streamlined and simplified!

I think this is a great moment for testing that "basic browsing" works fine before adding outline editing, and map mostly all or the rest of Leo's functionality to be used seamlessly from vscode...

https://github.com/boltex/leointeg

Please try it out (browsing along with headline and body editing for now) and ask about anything I may have overlooked in this short invitation to beta-test my stuff for free ;) 

here is the last changelog entry:

- Major 'Browsing' update before adding outline editing and file saving & derivation

- Refactored and simplified communication between vscode and leoBridge

- Stabilized browsing with multiple simultaneous (and different) body panes

- Added Leo Outline into explorer view (in addition to the dedicated Leo sidebar view)

- Supports new command to open a node on the side from the context menu

- Multiple configuration options: Open Settings with CTRL+',' type 'leo', or look for 'leo integration' in extensions

Many thanks!!

--

Félix

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote:

[Edward K. Ream]

On Sat, Oct 5, 2019 at 1:49 AM Robert Cholette <felix...@gmail.com> wrote:

Just a copy / paste in case someone joins this thread:

I've given #1281 the "Enhancement" and "Cool", and renamed it to:

"Integrate Leo with Visual Studio Code".

Thanks for this work. It will be listed as one of the major features of Leo 6.1.

Edward

[Robert Cholette]

Many Thanks! 'Real' 

Editing functionality is coming soon as this first phase of 'leoBridge communication & browsing stability' is almost done.

--

Félix

On Saturday, August 10, 2019 at 3:49:34 PM UTC-4, Robert Cholette wrote: