Please tell how it was that you got the Leo Aha

[Edward K. Ream]

Inspired by Terry's recent post, I'm wondering how many of you can
tell the story of what you were doing and thinking when you first
"got" what Leo was all about. Such stories might help newbies, and
they will help me understand the pathways people take to understanding
Leo.

These stories don't have to be long and detailed, but long and
detailed is good too.

In other words, the next big thing in Leo will likely be writing, but
it may not all be my writing :-)

Edward

[Edward K. Ream]

On Sep 3, 12:40 am, "Edward K. Ream" <edream...@gmail.com> wrote:
> Inspired by Terry's recent post, I'm wondering how many of you can
> tell the story of what you were doing and thinking when you first
> "got" what Leo was all about.

My own story is different, since it started before Leo ever
existed :-) The first part of the following is adapted from the start
of Leo's History chapter,
http://webpages.charter.net/edreamleo/history.html The second part is
new.

Beginnings

Leo grew out of my efforts to use Donald Knuth’s “CWEB system of
Structured documentation.” I had known of literate programming since
the mid 1980’s, but I never understood how to make it work for me.
Knuth's huge TeX system is presented in as a literate program in the
TeXBook:
http://www.amazon.com/TeXbook-Donald-E-Knuth/dp/0201134489
This book contains thousands of sections, arranged mostly in "stream-
of-consciousness" order.

In hindsight, I see now that the major problem was that there was no
way for me to keep track of all these "sections". Knuth used various
devices to try to make sense of this, including cross references,
indices and literary-style introductions, summaries and transitions
between sections. But I knew that I was not really understanding the
program.

In November 1995 I started thinking about literate programming in
earnest. Over the holidays I mused about making literate programs more
understandable. In January 1996 the fog of confusion suddenly cleared.
I summarized my thinking with the phrase, web are outlines in
disguise. I suspect I was thinking something like this: most
sections in Knuth's program are called from only one other section.
If the called section is represented as a child of the calling
section, much of the essential structure of the program will be made
visible.

Naturally, the same remarks apply to functions, methods and even
subclasses. I use the term "helper" to mean a function that is called
from a small number of calling functions. Most helpers, in fact, help
only one caller. So the natural place to place a helper in an outline
is as the child of the function it helps.

Again, this is perfectly clear in hindsight, but at the time many
details remained obscure.

Breakthrough

March 5, 1996, is the most important date in Leo’s history. While
returning from a day of skiing, I discussed my thoughts with Rebecca.
During that conversation I realized that I could use the MORE outliner
as a prototype for a “literate outliner.”

I immediately started work on my first literate outline. It quickly
became apparent that outlines work. I could immediately see that
placing "helper" sections as child nodes of the section they helped
removed all the confusion about the overall structure of literate
program.

The first "outline-oriented" program I wrote was a bootstrap: I
hacked a translator called M2C so I could use MORE to write real code.
I would write code in MORE, copy the text to the clipboard in MORE
format, then run M2C, which would tangle the outline into C code. This
process was useful, if clumsy. I called the language used in the
outline SWEB, for simplified CWEB. Much later Leo started supporting
the noweb language.

The @others directive dates from the very first hour of using MORE as
a prototype for a literate outliner. I realized that there was no
particular need to give names to every chunk of code: the @others
directive allowed me to access nodes without caring what their names
were.

Two later Ahas

Back in 1996 I was still using C (not C++) almost exclusively. It
seems like the dark ages. Python existed, but I didn't know about
it. The internet existed, but I didn't have access to it. So I was
cut off from the world of development in a way that is hard to imagine
today.

There were at least two more Aha's that shaped my view of Leo, but I
don't have clear recollections about how they came to be. These
realization likely happened more slowly, not in one Aha moment.

The first Aha is that browsers do not, and **can not**, know how to
organize my programs. The reason is clear: I don't know myself! That
is, the proper organization of my code depends on what I want to do
with it at the moment. For example, when I'm fixing a but, the proper
organization is one that ignores all nodes except those relating to
the bug. So there is no single one "right" organization of programs
(or any other complex data).

I had always been vaguely dissatisfied with class browsers. Even the
SmallTalk browser, which in some ways is pretty cool, did not help me
**in any way** to understand any organization except the "official"
organization. Browsers did not then, nor do they have today, any
memory of the nodes (or other organizations) that I like to create in
Leo. Far from remembering my task-oriented organizations, most
browsers do not even remember the part of the class hierarchy that I
last visited! Class browsers suffer from Alzheimer's disease :-)

It was an incredible piece of luck that I started with MORE as the
prototype of Leo. MORE *does* have a memory, because MORE has
clones. Leo's clones work *exactly* like MORE clones. So the Aha is
this: clones create permanent views of data that can be remembered.

The second aha took even longer to develop. Once Leo became an
independent program, it was possible to write scripts that would
access the nodes **in the outline containing the script**. The first
versions of Leo were written in C, and then Borland C++, and I don't
remember what kind of scripting features Leo had then. But in late
2001 Leo was born again as a Python program. It was natural to define
the current node of an outline, and to provide getters and setters for
that node.

It took *years* of using the Python version of Leo before it became
clear that one of Leo's biggest strengths is this: Leo has a rich DOM
(Document Object Model). Other editors don't remember anything
because **there is nothing except files to remember**. At best,
editors can remember bookmarks, but these are merely locations in
otherwise **completely uniform** text. In contrast, Leo scripts have
instant access to full outline organization. But that means that my
scripts have instant access to **my** organizations!!!

In particular, another extremely fortunate side effect of using MORE
as Leo's prototype was the notion that nodes have a headline as well
as body text. This has allowed headlines to be descriptions of the
data the node contains. The fancy term for this is that headlines
contain meta-data; body text contains data. By convention, both Leo's
internal scripts and user scripts use @x in the headline as a user-
extensible type system. This turns out to be profoundly important.

As I write this, I am struck by how fortunate I have been in
developing Leo. First, that I was motivated by my inability to
understand Knuth's TeX program. Second, that MORE was the prototype
for Leo. Third, that I stumbled upon Python. And most importantly,
that so many of Leo's users have made absolutely brilliant
suggestions. In particular, Marc-Antoine Parent suggested XML for
Leo's file format, Kent Tenney saw how to share subtrees, the great
Bernhard Mulder convinced me to use positions (iterators), 'e'
suggested what became @button nodes, and Ville created the bridge
between IPython and Leo.

It's been quite a ride. Thanks to you all.

Edward

[Edward K. Ream]

On Sep 3, 8:15 am, "Edward K. Ream" <edream...@gmail.com> wrote:

> And most importantly, that so many of Leo's users have made absolutely brilliant suggestions.

I risk offending those not mentioned. I've certainly got to thank
Paul Paterson for creating Leo's plugin architecture, and LeoUser for
creating the prototype for what turned into Leo's minibuffer. And on
and on and on...

Edward

[Matt Wilkie]

Thanks for the story Edward. It helps :)

> During that conversation I realized that I could use
> the MORE outliner as a prototype for a “literate outliner.”

And all this time I thought the 'more' the leo docs referred to was
the "less is more" type (http://www.gnu.org/software/less/), and
pondered why you thought it was possibly the most elegant program ever
written!

http://www.outliners.com/more31 - Is this the More, which perhaps
should be spelt Muse here, of your inspiration?

--
-matt

[rhh]

I found Python about 10 years ago. I use Python programming to
support my engineering research and design. Since I can remember, I
have wanted a three pane type calculation interface with an outliner
for all the reasons Edward mentions. Programs like Treepad gave me a
sense of how well the paradigm worked - but I still needed a
programming API. Since structural engineering calculation documents
are a form of reusable code (bridge and building design algorithms
don't change that much) it was natural to also begin to think about an
integrated calculation document environment in addition to a coding
environment. Discovering Leo (when Leo was at v4.5) was like walking
into a candy store. My aha moment was the feeling of relief that
comes when a great programmer and team of programmers creates
something that met my needs better than I could myself.

[Edward K. Ream]

Yes.  More was developed by Dave Weimer, sold to Symantec for millions of dollars, and then disappeared.

Edward

[Edward K. Ream]

On Thu, Sep 3, 2009 at 8:15 AM, Edward K. Ream <edre...@gmail.com> wrote:

Two later Ahas

I realize that I described the Ahas, but not how they came about.
 

The first Aha is that browsers do not, and **can not**, know how to
organize my programs.

 

It was an incredible piece of luck that I started with MORE as the

prototype of Leo.  MORE *does* have a memory, because MORE has
clones.  Leo's clones work *exactly* like MORE clones.  So the Aha is
this: clones create permanent views of data that can be remembered.

The Aha arose naturally as the result of using clones.  After cloning a node, it is natural to put it somewhere else, and that "somewhere else" usually contains other clones.   So you could say that any (usually regular) node containing clones becomes a summary (view) of those clones.  Alternatively, the reason I made clones was that I had something to do, and I wanted to focus my attention on that something (project).  I create a (regular) node to represent the view, and then drag clones to that node.  Either way, the result is the same.

Once one starts using clones, it seems almost inevitable that they become part of a project-oriented organization of the data in the outline.  The cool thing is that a single outline can contain arbitrarily many views of itself.  This came directly from MORE.  Leo simply borrowed it.

It took *years* of using the Python version of Leo before it became
clear that one of Leo's biggest strengths is this:  Leo has a rich DOM (Document Object Model).

The realization that Leo's DOM is what fundamentally distinguishes Leo from all other editors, came within the last year.  I knew that there were many things that you can do in Leo that you can't even *think of* in other editors.  But the Aha that Leo's DOM is what makes Leo unique happened while I was writing documentation.

Edward

P.S.  The evolution of Leo's DOM itself (as distinct from the Aha that Leo has a DOM) happened something like this:

Ever since Leo became a stand-alone program (that is, after Leo no longer depended on MORE), Leo used @root and @file nodes to denote parts of an outline that would generate external files.  Later, @thin, @auto, @shadow and @edit nodes appeared.

The C versions of Leo did support a DOM, but nothing much happened because there was no easy way to run C scripts from Leo.  This all changed radically in late 2001 when the first Python version of Leo appeared.  Leo's Python code created an implicit DOM of the outline.

Leo's scripting abilities developed slowly. I don't recall when a usable execute-script command first saw the light of day, but I know for sure it was quite some time before e's brilliant suggestion to "bring scripts to data" resulted in @button nodes.  These have made a huge difference in Leo's scripting.  For example, before  @button, Leo had a very clumsy "script find" mechanism, which was made completely obsolete by @button.

@button begat @test, and @test nodes typically using headlines to denote data.  The result was a notion that @x nodes denote user-defined types.  Leo has a *user-extensible* DOM.

Bernhard Mulder's suggestion to use positions (iterators) to traverse the tree was another key ingredient in the evolution of Leo's DOM.  This allowed Leo's data structure to develop in new directions, culminating in the one-node world.

EKR

[Ville M. Vainio]

On Thu, Sep 3, 2009 at 8:40 AM, Edward K. Ream <edre...@gmail.com> wrote:

> Inspired by Terry's recent post, I'm wondering how many of you can
> tell the story of what you were doing and thinking when you first
> "got" what Leo was all about.  Such stories might help newbies, and
> they will help me understand the pathways people take to understanding
> Leo.

My "story":

- I bumped into leo 5 years back. This tutorial sparked my fascination:

http://www.3dtree.com/ev/e/sbooks/leo/sbframetoc_ie.htm

(yeah, they are not really up-to-date anymore. The best tutorial is
probably the "Outlining" one, with Friends, Enemies and People that
owe me money)

I played around with it a bit, but found myself unable to use it for
programming (the @root thing is not all that practical). Also, back
then Leo was too slow for large projects (we've come a long way
since). Still, a weird "charm" about the whole thing stuck to the back
of my head.

Here are some of my c.l.p postings from 2004:

http://mail.python.org/pipermail/python-list/2004-July/270849.html
http://mail.python.org/pipermail/python-list/2004-July/270910.html

I guess I found myself not needing Leo that much back then - my job
was boring at the time, and I didn't have all that much to keep track
off.

Later, a colleague introduced me to Freemind (and my job had got
interesting enough to actually keep track of what i'm doing). I used
that for a while to keep notes about my defect fixing work (with chat
logs, tracebacks, interesting functions etc. associated with each
defect).

However, I soon hit the glass ceiling with Freemind - it just didn't
scale for lots of text and complex graphs. It was more of an
attractive toy, really. I then remembered Leo, and switched to using
that. Leo was a much better fit for my needs.

This was coupled with Edwards post to ipython mailing list:

http://osdir.com/ml/python.ipython.devel/2005-07/msg00015.html

This resulted in ILeo, which solved some of the woes of using %edit in
IPython, and revealed a whole new avenue of thinking about interactive
programming.

I got seriously sucked into development of Leo itself when the Qt ui
started happening. I had a very selfish motive for promoting the Qt ui
stuff - I had started working on Linux at the time (on the maemo/Nokia
N900 stuff I posted the link about), and the Tk ui was bad enough on
Linux that I either had to reimplement a Leo-like program myself
(possibly based on qleolite, branch still on LP), or wait for other
GUI plugin to emerge. But here we are.

--
Ville M. Vainio
http://tinyurl.com/vainio

[Kent Tenney]

My excitement about Leo has only increased since this post:
http://longrun.zwiki.org/LeoLog