ENB: Live coding in Leo?

[Edward K. Ream]

In this Engineering Notebook post I'll explore changing Leo while Leo is running.

Imo, changing Leo "on the fly" is impossible, but thinking about what would be necessary might suggest potential improvements to Leo

Background

Live coding is supposedly a killer feature of smalltalk. I have my doubts, discussed below.

Live coding has another meaning in the arts.  See this wikipedia page and this page.

pylivecoding is a python module that purports to support live coding in python.  From its readme file:

QQQ

pylivecoding is a live coding environment implementation inspired by Smalltalk.

Essentially this library reloads modules and updates all live instances of classes defined in those modules to the latest code of the class definition without losing any of the data/state. This way you can change code in your favorite code editor and IDE and immediately see the results without any delays after you save the module.

QQQ

All "live" classes must be subclasses of livecoding.LiveEnviroment(), which is just a wrapper for python's reload statement. Imo, the pylivecoding "architecture" could work only in small apps.

Barriers to changing running apps

All apps, regardless of language, are inherently inflexible:

1. The architecture can't change.  In particular, APIs are contracts that can't easily be broken.  Refactoring implies restarting the app.

2. Objects (instances of classes) contain data that must be retained. In general, changing any module might change the data that the module computes.

Live coding apps (in the arts) give the illusion of flexibility:

1. The app's architecture and primitives remain fixed (and inaccessible to the artist).

2. The artist's code is rerun completely whenever the code changes.

Summary

Live coding in the arts gives the illusion of dynamism.

It's not possible to change the architecture of an app on the fly, regardless of language.

Reducing the static data in some of Leo's classes might simplify Leo. Or not. I'll discuss this topic in another post.

Edward

[tbp1...@gmail.com]

On Thursday, March 3, 2022 at 8:52:57 AM UTC-5 Edward K. Ream wrote:

In this Engineering Notebook post I'll explore changing Leo while Leo is running.

Imo, changing Leo "on the fly" is impossible, but thinking about what would be necessary might suggest potential improvements to Leo

[snip]

All apps, regardless of language, are inherently inflexible:

1. The architecture can't change.  In particular, APIs are contracts that can't easily be broken.  Refactoring implies restarting the app.

2. Objects (instances of classes) contain data that must be retained. In general, changing any module might change the data that the module computes.

I generally agree with these statements, but they aren't always as rigid as is stated here.  I'm going mostly by my ancient experience with Forth, which I've mentioned in another thread.  Back in those days, there was basically no fancy memory management, and DOS or even CP/M would be the operating system.  Forth did not use OS calls, and as it came for DOS, read and wrote raw disk sectors, not OS files.  So you could run Forth or DOS, but not both at the same time from the same drive (a floppy disk drive).

Forth commands were, except for a few primitives, just lists of other Forth commands.  They were invoked by name, and the commands were executed by looking up the address of the next command, and executing its code list.  All data was kept on a stack, and commands had no arguments.  They were expected to push and pop the stack to get or save data.  The Forth primitives were coded in assembler, and all other Forth commands - called "words" - were built up ("defined") from those, even flow control commands like IF.  You didn't save programs, you saved that Forth image which included all the commands defined to date.  In this respect it was rather like Smalltalk.

Here's what I've been leading up to.  I was able to convert my Forth image to use standard DOS files instead of raw disk sectors.  I did it on the fly.  First I defined a command to perform interrupts (there was a way to have your command list be a series of ASM instructions instead of being Forth words).  The BIOS used Int21, and DOS used Int27, IIRC.  Once I had this working, I could define new words that called DOS and BIOS services.

From there, I defined new Forth words to read and write files using DOS files.  Then I could save even the Forth image to a DOS file.  After that, Forth and DOS could co-exist on the same disk.  Life became much easier.  I was also able to change the original line-by-line editor to a full-screen editor bit by bit in a similar way.  That was interesting because you needed the editor to write your new words, so it always had to work even as you changed it..

I would call that "changing Forth on the fly".  I think that one key to the ability to make such changes is that there was no global data in Forth.  Each word pushed data onto the stack and popped data from it.  So words had to cooperate to maintain the contents of the stack correctly - that was the equivalent of an API contract.  But the data was produced and consumed locally in an ephemeral way.

Of course, this has almost nothing to do with Leo and "live programming".  But it's an interesting perspective on the question.

[Offray Vladimir Luna Cárdenas]

Hi,

On 3/03/22 8:52, Edward K. Ream wrote:

In this Engineering Notebook post I'll explore changing Leo while Leo is running.

Imo, changing Leo "on the fly" is impossible, but thinking about what would be necessary might suggest potential improvements to Leo

Background

Live coding is supposedly a killer feature of smalltalk. I have my doubts, discussed below.

It is a killer feature. Most of the time when I see doubts about a not well known/understood technology or alternative, it is mostly formulated from the direct experience with the technology against what we're are comparing it, in contrast to what we have heard about the alternative. I not saying that this is your case, but it has happen to me, particularly in programmers circles, when they are comparing lets say Pharo against more popular computer languages/environments or Fossil against Git. Another point is that the alternative is not similar enough to the

Barriers to changing running apps

All apps, regardless of language, are inherently inflexible:

By some fundamental comparison, all apps can share some inherently feature, being all Turing machines or equivalent in (virtual) machine code. The issue for me, is the metaphors and experiences they provide when apps are presented to us. That's why direct experience with alternatives, particularly when they different to what we're used to is important.

1. The architecture can't change.  In particular, APIs are contracts that can't easily be broken.  Refactoring implies restarting the app.

2. Objects (instances of classes) contain data that must be retained. In general, changing any module might change the data that the module computes.

Live coding apps (in the arts) give the illusion of flexibility:

1. The app's architecture and primitives remain fixed (and inaccessible to the artist).

2. The artist's code is rerun completely whenever the code changes.

I like the definition of architecture as the stuff that is difficult to change [1][2] and in that sense the important thing would be how an app bring us affordances to traverse/understand architecture (and maybe to change it). Also language primitives are not fixed in Smalltalk based environments. If you want, you could change the way booleans work on the fly (and assume the consequences).

[1] "Good Enogh" Architecture: https://youtu.be/PzEox3szeRc
[2] Architecture: The Stuff That's Hard to Change https://youtu.be/3LtQWxhqjqI

Coming back to different metaphors and experiences, instead of some fundamental metric of equivalence (like rerunning on changes), the distinctive factor would be the kind of understanding and conversations that a live coding environment is enabling between the user/dev and the artifact itself and how this have consequences in the creations/tasks at hand. To have a glimpse of it, there is the, now classic, video "Inventing on principle"[3] and for a more developer/coding oriented one, there is "Moldable development"[4]. In both cases, because programming is not mainly dealing with text and blindly manipulating symbols, and instead being engage in a "artifact conversation" where different representations are provided to easy such conversations and we can see the consequences of our code in a more direct way over the artifacts.

[3] https://vimeo.com/38272912
[4] https://www.youtube.com/watch?v=Pot9GnHFOVU

In my case, the conversation I had with textual artifacts, was easy by the Leo affordances, when I was able to split and recombine the text at hand (prose or code), according to my understanding of it. Pharo, Lepiter and other live coding environments provide me with the possibility to run small snippets of code and extend the representation of the system I'm dealing with on the fly to increase my understading. I want. Grafoscopio is my attempt to combine both experiences: live coding and outlining in a single tool and to see the consequences of such tools in agency of grassroots communities, starting at our local hackerspace (HackBo).

Summary

Live coding in the arts gives the illusion of dynamism.

It's not possible to change the architecture of an app on the fly, regardless of language.

Reducing the static data in some of Leo's classes might simplify Leo. Or not. I'll discuss this topic in another post.

My summary would be in the lines of the importance of which innevitable illusions/metaphors we bring while building digital tools (as we're are not dealing with changes in current in microprocessors) and the affordances/capabilities they bring to our comunities (for example of users).

Cheers,

Offray

[tbp1...@gmail.com]

There is one area in Leo that is akin to live programming.  That is running a Leo script that is in a subtree but is not a file.  You can make a change and hit CNTL-b to rerun it.  Behind the scenes the script gets turned into a file and the file gets run, but the user doesn't need to know that.  I've got an app I'm working on right now that runs in a tab in the Log frame.  I was able to devise a script the re-creates that tab and loads the app into it.  It was a bit tricky to make work because it has to reload my app (to import it again with its changes).  I made a button for that script, so I can make a code change, hit the button, and I'm ready to go in the tab with the changed code.

This is essentially live programming - as far as my personal perceptions are concerned - but it's a very simple, nearly self-contained thing.  It doesn't change any of Leo's data structures, although it does get data from the outline's tree (well, it can change the current c.p position).  I don't know how one could treat other parts of Leo this way, but I'm enjoying this bit of live programming.

[Edward K. Ream]

On Thu, Mar 3, 2022 at 11:42 AM Offray Vladimir Luna Cárdenas <off...@riseup.net> wrote:

> [Live coding] is a killer feature.

Thanks for these links.  I've bookmarked them and will study them with great interest.

A comment about recent ENB's:

pylivecoding may be useful during development. The dev would add the necessary scaffolding to do proper reloads of code under development.

But a framework for reloading isn't as big a deal as one might think, because Leo's new (traditional) unit tests do a complete restart of Leo for every single each test. This means that unit tests are, in fact, live coding.

Edward

[tbp1...@gmail.com]

I haven't been able to find it again.  But I seem to remember from some time ago reading a bit on reloading Leo plugins without stopping/restarting Leo.  Anyone have a pointer to it?

[Edward K. Ream]

On Thursday, March 3, 2022 at 11:42:10 AM UTC-6 [Offray] wrote:

I like the definition of architecture as the stuff that is difficult to change

...

[1] "Good Enough" Architecture: https://youtu.be/PzEox3szeRc

[2] Architecture: The Stuff That's Hard to Change https://youtu.be/3LtQWxhqjqI

These videos discuss organizational-level issues. I have no experience at this level, and you shouldn't take anything I say to apply to this level. When I talk about architecture I am speaking of Leo's architecture at the module and class level.

 

the distinctive factor would be the kind of understanding and conversations that a live coding environment is enabling between the user/dev and the artifact itself and how this have consequences in the creations/tasks at hand

...

[3] https://vimeo.com/38272912

This video, Inventing on principle, is indeed intriguing. The underlying language isn't smalltalk, so the same ideas could, I suppose, be used in python. Afaik, the source code isn't available. I assume, as a kind of reverse engineering, that Victor has enhanced the drawing and "physics" portions of the game app. It's a cool demo.

[4] https://www.youtube.com/watch?v=Pot9GnHFOVU

The moldable development environment shows what is possible when fundamental objects can be augmented. It's an interesting idea.

Summary

Thanks for these links. The first two deal with system design issues that do not relate directly to programming languages. The second two links relate to issues closer to my world.

Edward

[tbp1...@gmail.com]

Here's an interesting and maybe relevant project - Ficl.  It's old and probably defunct, but still seems interesting in this context.  From the summary:

" Ficl is a complete programming language interpreter designed to be embedded into other systems (including firmware based ones) as a command, macro, and development prototype language. Unlike other scripting interpreters, Ficl:

....

Where most Forths view themselves as the center of the system and expect the rest of the system to be coded in Forth, Ficl acts as a component of the system. It is easy to export code written in C or ASM to Ficl in the style of TCL, or to invoke Ficl code from a compiled module. This allows you to do incremental development in a way that combines the best features of threaded languages (rapid development, quick code/test/debug cycle, reasonably fast) with the best features of C (everyone knows it, easier to support large blocks of code, efficient, type checking). In addition, Ficl provides a simple and powerful object model that can act as an object oriented adapter for code written in C (or asm, Forth, C++...)."

Although it's apparently abandoned, SourceForge stil; seems to have the code.

Instead of Forth/C, I  think of Python/Rust.

[Offray Vladimir Luna Cárdenas]

Hi,

On 5/03/22 7:09, Edward K. Ream wrote:

Summary

Thanks for these links. The first two deal with system design issues that do not relate directly to programming languages. The second two links relate to issues closer to my world.

Edward

Thanks for taking a detailed look of the videos. Knowing which definition of architecture are we using is helpful to clarify what we mean by the intention to change it. I think there is some correlation between concrete architectures in specific programs, and more general questions. I'm now developing some software combining Nim, Pharo, TiddlyWiki, Mardeep and Fossil and I think about decisions in that stack that allow us to be pretty fluent and which ones are difficult to change, even in its concrete incarnations.

I agree that the concepts and experiences that Victor and Griba show in their particular videos are not tied with a particular technology and is more about the kind of experience we would like to have with some environments. They could be found in other places. The place I have found it in a more pervasive and practical way is in Smalltalk environments, and was one of my switching points, coming from Python, Scheme and other places in my previous explorations.

For me, traversing this issues, from infrastructure to architecture, to "humanware/wetware", is kind of my place, as I care more about that bridges instead of being an specialist in the connected spots. In some way that allow me to participate of conversation about the upcoming Leo futures/features regarding live coding and is a joy to participate more actively of this community.

Cheers,

Offray