Thoughts about navigating the leo outline programmatically

12 views
Skip to first unread message

zpcspm

unread,
Jan 1, 2009, 2:12:48 PM1/1/09
to leo-editor
Preface

Some time ago I was fiddling with a Greasemonkey script that wasn't
written by me. My goal was to provide some refactoring hints to the
authors. Obviously, my first task was to understand what it does and
how. I can read JavaScript code because of some C/Java background, but
not more than that. So my quest included reading the wikipedia
articles (and a little other info) about JQuery and XPath, because I
was totally clueless about them. I won't claim that I could write
something useful now, but now I know at least what are these. Both
JQuery and XPath seem to provide functional query languages.

I was thinking about these from leo's perspective. In leo, we have
something like this:

for p in c.allNodes_iter():
<do something>

Basically, this is "manual" processing.

I wonder if leo could provide such a query language in the future,
that would be much more flexible than UNL and would spare the
programmer from manually scanning the outline for nodes that match
certain criteria (perhaps regexp for the node headline, regexp for the
node body) by providing a way to write [nested] queries.

I recall some ideas about storing leo nodes in a database popping up
earlier in this group. If this ever happens, it could provide an
alternative way to perform search queries (assuming that the given
database backend will understand regexp in queries).

This is not a feature request, it's rather a call for brainstorming.

Ville M. Vainio

unread,
Jan 1, 2009, 2:45:36 PM1/1/09
to leo-e...@googlegroups.com
On Thu, Jan 1, 2009 at 9:12 PM, zpcspm <zpc...@gmail.com> wrote:

> I wonder if leo could provide such a query language in the future,
> that would be much more flexible than UNL and would spare the
> programmer from manually scanning the outline for nodes that match
> certain criteria (perhaps regexp for the node headline, regexp for the
> node body) by providing a way to write [nested] queries.

In ILeo, there is a trivial match_h:

ileo[C:p/leo]|5> list(wb.match_h('Non-'))
<5> [<LeoNode <pos 67824464 childIndex: 4 lvl: 2 [2]
Non-essential>>]

This does a regexp match. Yeah, it's just an allnodes_iter in drag.

Generalized hierarchical node selection is easy to do as well:

node = wb.select('Gui wrap.*').select(".*Qt").select('Font')

The magic here would be a new "LeoNodeList" class that

- inherits from python 'list' type
- provides 'select' method that returns all nodes under any node in
the LeoNodeList (self) that match the pattern as a new LeoNodeList. It
would only traverse one tree level (only work with direct children),
to avoid circularities & other weirdness.

The special "first" select (here wb.select, but it could be c.select()
as well ) would scan the whole tree. Throw in select_b('some body
text') to search nodes by body contents, and we have a winner.

Hey, it could work.

import IPython
IPython.genutils.SList??

Will show you an example implementation of a custom list type that
provides some additional methods (this time provided for string lists,
e.g. system command output).

--
Ville M. Vainio
http://tinyurl.com/vainio

zpcspm

unread,
Jan 1, 2009, 3:26:06 PM1/1/09
to leo-editor
I think it's time to add a "take a closer look at IPython" item to my
TODO list :)
My first post was a perspective of somebody who uses leo, but doesn't
use IPython at all: nor IPython itself, neither a combo of leo and
IPython. It happened because leo doesn't enforce using IPython.

Ville M. Vainio

unread,
Jan 1, 2009, 3:59:44 PM1/1/09
to leo-e...@googlegroups.com
On Thu, Jan 1, 2009 at 10:26 PM, zpcspm <zpc...@gmail.com> wrote:

> My first post was a perspective of somebody who uses leo, but doesn't
> use IPython at all: nor IPython itself, neither a combo of leo and
> IPython. It happened because leo doesn't enforce using IPython.

This is why I have been advocating integrating the LeoWorkbook from
ILeo code to Leo itself (or at least the concept!). There is nothing
very IPython-specific about the code, it's just something I thought
would be a more "convenient" API for using data in leo. There is some
help in ILeo to provide code completion for workbook, but that is
outside the class itself (i.e. it doesn't matter whether the class is
in plain-old-leo, or IPython integration layer). Likewise, LeoWorkbook
returns LeoNode objects, which are just thin wrappers around position,
that provide e.g. node.h and node.b (properties to access headString
and bodyString(). Those properties should be in leo's position code in
the first place, to make scripts shorter and easier to read/write.

Kent Tenney

unread,
Jan 2, 2009, 7:52:55 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 1:45 PM, Ville M. Vainio <viva...@gmail.com> wrote:
>
> On Thu, Jan 1, 2009 at 9:12 PM, zpcspm <zpc...@gmail.com> wrote:
>
>> I wonder if leo could provide such a query language in the future,
>> that would be much more flexible than UNL and would spare the
>> programmer from manually scanning the outline for nodes that match
>> certain criteria (perhaps regexp for the node headline, regexp for the
>> node body) by providing a way to write [nested] queries.
>
> In ILeo, there is a trivial match_h:
>
> ileo[C:p/leo]|5> list(wb.match_h('Non-'))
> <5> [<LeoNode <pos 67824464 childIndex: 4 lvl: 2 [2]
> Non-essential>>]
>
> This does a regexp match. Yeah, it's just an allnodes_iter in drag.

{-:

Such a nice image. A wrapper is a pretty boring garment, whereas
being in drag usually implies an interesting story.

Edward K. Ream

unread,
Jan 2, 2009, 7:54:23 AM1/2/09
to leo-e...@googlegroups.com
On Thu, Jan 1, 2009 at 2:59 PM, Ville M. Vainio <viva...@gmail.com> wrote:

On Thu, Jan 1, 2009 at 10:26 PM, zpcspm <zpc...@gmail.com> wrote:

> My first post was a perspective of somebody who uses leo, but doesn't
> use IPython at all: nor IPython itself, neither a combo of leo and
> IPython. It happened because leo doesn't enforce using IPython.

This is why I have been advocating integrating the LeoWorkbook from
ILeo code to Leo itself (or at least the concept!).

This is going to happen.  Probably this week.  Maybe today.

Edward

Kent Tenney

unread,
Jan 2, 2009, 8:05:45 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 2:26 PM, zpcspm <zpc...@gmail.com> wrote:
>
> I think it's time to add a "take a closer look at IPython" item to my
> TODO list :)

It's fast becoming indispensable for me,
The easiest idiom for me to adapt has been
tab completion, ? and ??

These 3 tools have reduced the time I spend working with APIs
I'm unfamiliar with.

The ? feature makes me wish that Leo had more
thorough docstring coverage.

Edward K. Ream

unread,
Jan 2, 2009, 8:09:38 AM1/2/09
to leo-e...@googlegroups.com
On Thu, Jan 1, 2009 at 1:12 PM, zpcspm <zpc...@gmail.com> wrote:

Preface

Some time ago I was fiddling with a Greasemonkey script that wasn't
written by me. My goal was to provide some refactoring hints to the
authors. Obviously, my first task was to understand what it does and
how. I can read JavaScript code because of some C/Java background, but
not more than that. So my quest included reading the wikipedia
articles (and a little other info) about JQuery and XPath, because I
was totally clueless about them. I won't claim that I could write
something useful now, but now I know at least what are these. Both
JQuery and XPath seem to provide functional query languages.

Thanks for the clear intro.  It makes an enjoyable, easy-to-understand story :-)

 
I was thinking about these from leo's perspective. In leo, we have
something like this:

for p in c.allNodes_iter():
   <do something>

Basically, this is "manual" processing.

I wonder if leo could provide such a query language in the future,
that would be much more flexible than UNL and would spare the
programmer from manually scanning the outline for nodes that match
certain criteria (perhaps regexp for the node headline, regexp for the
node body) by providing a way to write [nested] queries.

We all agree that the iLeo workbook is a good thing to do. However, the payoff of the workbook is for users using IPython with Leo; the payoff for script writers is small.

Indeed, xml needs languages like XSLT and JQuery and XPath because otherwise the "user" would have to parse xml.  These languages are in effect wrappers that expose the underlying structure (DOM, Document Object Model) of the xml document.

In contrast, when Leo reads files (.leo files or external files), Leo does a full parse of those files and converts them to trees of first-class objects (vnodes, and the helper tnodes).  So Leo has already done the heavy lifting for script writers: scripts just need to traverse trees.  Leo's iterators make this kind of task trivial.

Furthermore, p.bodyString(), p.headString() and other getters and setters provide trivial access to all parts of Leo's DOM.

In short, little or nothing more is needed.

I'm glad you raised this question, because it shows that even experienced users sometimes don't fully grasp the power of Leo's DOM.  You could say that Leo's DOM is so easy to use that doesn't *look* like a DOM, but in fact it is.  Furthermore, Leo's DOM is much easier to use that the xml DOM, and is much richer that the trivial DOM's provided by Emacs and Vim.  This is a story that absolutely must be told in Leo's introduction.

Edward

Edward K. Ream

unread,
Jan 2, 2009, 8:12:48 AM1/2/09
to leo-e...@googlegroups.com
On Thu, Jan 1, 2009 at 2:59 PM, Ville M. Vainio <viva...@gmail.com> wrote:

Likewise, LeoWorkbook
returns LeoNode objects, which are just thin wrappers around position,
that provide e.g. node.h and node.b (properties to access headString
and bodyString(). Those properties should be in leo's position code in
the first place, to make scripts shorter and easier to read/write.

I agree.

Edward

Edward K. Ream

unread,
Jan 2, 2009, 8:18:15 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 7:05 AM, Kent Tenney <kte...@gmail.com> wrote:

The ? feature makes me wish that Leo had more
thorough docstring coverage.

Yes, that would be nice.  It's a major, thankless, task that may be a long time coming.

In the meantime, Leo's auto-completer (Alt-1, Alt-2) is a good alternative.  A method's name usually tells me what it does, but I need the autocompleter to remind me what arguments can be used.  I assume IPython can remind you what the arguments of a function and method are...

Edward

Edward K. Ream

unread,
Jan 2, 2009, 8:34:10 AM1/2/09
to leo-editor


On Jan 2, 6:54 am, "Edward K. Ream" <edream...@gmail.com> wrote:

> > This is why I have been advocating integrating the LeoWorkbook from
> > ILeo code to Leo itself (or at least the concept!).
>
> This is going to happen.  Probably this week.  Maybe today.

There is a potential performance pitfall in the present
implementation.
wb[node-name] is a wrapper for c.allNodes_iter. This is benign at the
command-line, but could create nasty surprises for scripts.

Any thoughts about what to do? Documentation is my last choice.

Edward

Edward K. Ream

unread,
Jan 2, 2009, 8:39:49 AM1/2/09
to leo-editor


On Jan 2, 7:34 am, "Edward K. Ream" <edream...@gmail.com> wrote:

> wb[node-name] is a wrapper for c.allNodes_iter.  This is benign at the
> command-line, but could create nasty surprises for scripts.
>
> Any thoughts about what to do?  Documentation is my last choice.

Hmm. Maybe scripts should have to do::

wb = c.noteBook()

This would populate the underlying dictionary once, so wb[node-name]
is a simple dictionary lookup. The down side is that wb won't be up-
to-date if the outline changes.

Hmm again. As I write this, I think the commander should always
provide an up-to-date c.wb object. This implies that all changes to
outline be reflected immediately in c.wb. It takes more work
initially, but anything else is going to result in endless variations
on one or more FAQ's.

Any comments or suggestions?

Edward

Ville M. Vainio

unread,
Jan 2, 2009, 8:56:44 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 3:09 PM, Edward K. Ream <edre...@gmail.com> wrote:

> We all agree that the iLeo workbook is a good thing to do. However, the
> payoff of the workbook is for users using IPython with Leo; the payoff for
> script writers is small.

Yoe

>
> Indeed, xml needs languages like XSLT and JQuery and XPath because otherwise
> the "user" would have to parse xml. These languages are in effect wrappers
> that expose the underlying structure (DOM, Document Object Model) of the xml
> document.
>
> In contrast, when Leo reads files (.leo files or external files), Leo does a
> full parse of those files and converts them to trees of first-class objects
> (vnodes, and the helper tnodes). So Leo has already done the heavy lifting
> for script writers: scripts just need to traverse trees. Leo's iterators
> make this kind of task trivial.
>
> Furthermore, p.bodyString(), p.headString() and other getters and setters
> provide trivial access to all parts of Leo's DOM.
>
> In short, little or nothing more is needed.
>
> I'm glad you raised this question, because it shows that even experienced
> users sometimes don't fully grasp the power of Leo's DOM. You could say
> that Leo's DOM is so easy to use that doesn't *look* like a DOM, but in fact
> it is. Furthermore, Leo's DOM is much easier to use that the xml DOM, and
> is much richer that the trivial DOM's provided by Emacs and Vim. This is a
> story that absolutely must be told in Leo's introduction.
>
> Edward
>
>
> >
>



Ville M. Vainio

unread,
Jan 2, 2009, 9:06:32 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 3:09 PM, Edward K. Ream <edre...@gmail.com> wrote:

> We all agree that the iLeo workbook is a good thing to do. However, the
> payoff of the workbook is for users using IPython with Leo; the payoff for
> script writers is small.

Not if you want to write the script "quickly". It gets you around
low-level way of iterating through nodes (effectively reducing code).

I think my "select" method proposition could be a more "rigid" way of
approaching the problem, and would not rely on magic getattr stuff
(which helps with ipython, because it can provide tab completion).
Still, provided that we add tab completion (that fetches from IPython
- i.e. the WorkBook completion works ok) to leo text editor, it'll be
much easier to get to the data of a node.

> Furthermore, p.bodyString(), p.headString() and other getters and setters
> provide trivial access to all parts of Leo's DOM.
>
> In short, little or nothing more is needed.

Providing a convenience wrapper to find nodes may be "little", but it
will certainly make the scripts easier to read and write.

Using the 'select' proposition before, you have to write quite a bit
of code to cover what an oneliner with select() does - and it can
actually encourage people to write more code that uses "live" data in
the leo documents.

Ville M. Vainio

unread,
Jan 2, 2009, 9:14:31 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 3:34 PM, Edward K. Ream <edre...@gmail.com> wrote:

> wb[node-name] is a wrapper for c.allNodes_iter. This is benign at the

It's actually wb.NodeName. But the truth is that it's not fast. This
could be made much, much faster by updating the node dict whenever
changes happen.

> Any thoughts about what to do? Documentation is my last choice.

I think you should look at the possibility of implementing the
"select" method first. It's more general purpose and less "magical" -
notably, it works for all kinds of headlines, as opposed to ones that
look like a variable name.

I can provide a demo implementation if my previous explanation was not
clear enough.

Edward K. Ream

unread,
Jan 2, 2009, 9:52:18 AM1/2/09
to leo-editor


On Jan 2, 8:14 am, "Ville M. Vainio" <vivai...@gmail.com> wrote:

> > wb[node-name] is a wrapper for c.allNodes_iter.  This is benign at the
>
> It's actually wb.NodeName. But the truth is that it's not fast. This
> could be made much, much faster by updating the node dict whenever
> changes happen.

The more I think about this, the more unhappy I am. Here are my
present thoughts:

1. Clearly, Leo should support the following getters and setters,
where p is a position and s is a string::

s = p.b
s = p.h
p.b = s
p.h = s

This is convenient, clear, safe, and requires no large-scale changes
to Leo's core.

2. There appear to be *severe* problems with c.wb.nodeName outside the
context of the command-line:

First, c.wb.nodeName returns the first node of the tree whose name is
nodeName. This is well defined, but it is also badly defined :-)
There may be many nodes in a tree with the same name. In general,
there is no reason to prefer the first node found, so it seems to me
we are building in confusion.

Second: it does not appear easy to quickly update c.wb when inserting,
deleting or changing (the headline of) nodes, at least with the
present ("bad") definition. It might be easier to do if c.wb.nodeName
returned a list of positions p such that p.h == nodeName, but
naturally that makes using c.wb.nodeName clumsy.

Not good. We can not allow fundamental confusion about what
wb.nodeName means, and we can not compromise Leo's speed. More
invention is needed.

The first invention that comes to mind is @wb trees. The idea is that
c.wb would restrict its operation to descendants of @wb nodes.
However, this doesn't solve either problem very well. True, direct
operations on c.wb might enforce the requirement that c.wb.nodeName
yield a unique node, but there are many other ways of violating this
requirement.

In short, c.wb has serious flaws in a scripting context. We must
either fix these flaws or do without c.wb.

The inventing lamp is lit.

Edward

Ville M. Vainio

unread,
Jan 2, 2009, 10:32:13 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 4:52 PM, Edward K. Ream <edre...@gmail.com> wrote:

> 2. There appear to be *severe* problems with c.wb.nodeName outside the
> context of the command-line:
>
> First, c.wb.nodeName returns the first node of the tree whose name is
> nodeName. This is well defined, but it is also badly defined :-)
> There may be many nodes in a tree with the same name. In general,
> there is no reason to prefer the first node found, so it seems to me
> we are building in confusion.

Yes - wb is for "quick-and-dirty" node discovery, where this tradeoff
has been made purposefully. The idea is that the leo document will
have headlines that are named like variables (i.e. hopefully in unique
fashion, with variable-like names). This does *not* work for discovery
of nodes in a naively created leo outline.

However, here is my implementation of "select()" functionality that
seems to work (and IS appropriate for scripting usage):

class PosList(list):
def select(self, pat):
res = PosList()
for p in self:
#print "po",p
for chi_p in p.children_iter():
#print "chi",chi_p
if re.match(pat, chi_p.headString()):
res.append(chi_p.copy())
return res

Here is example usage:

ileo[~]|194> pl = PosList([c.currentPosition()])

ileo[~]|206> pl
<206> [<pos 145246028 childIndex: 2 lvl: 0 [0] qt gui stuff>]
ileo[~]|207> pl.select('.*')
<207>
[<pos 161789100 childIndex: 0 lvl: 1 [1] To do>,
<pos 161791660 childIndex: 1 lvl: 1 [1] Dumps>,
<pos 161788780 childIndex: 2 lvl: 1 [1] Unit tests>,
<pos 161788396 childIndex: 3 lvl: 1 [1] Unused>,
<pos 161788268 childIndex: 4 lvl: 1 [1] @auto ../test/qtui_generate.p
y>,
<pos 161788204 childIndex: 5 lvl: 1 [1] @auto qt_main.py>,
<pos 161791628 childIndex: 6 lvl: 1 [1] @auto qt_main.ui>,
<pos 161790636 childIndex: 7 lvl: 1 [1] @auto qt_quicksearch.py>,
<pos 161790060 childIndex: 8 lvl: 1 [1] @auto qt_quicksearch.ui>,
<pos 161790796 childIndex: 9 lvl: 1 [1] @thin qtGui.py>,
<pos 161791276 childIndex: 10 lvl: 1 [1] Key problems>]
ileo[~]|208> pl.select(' Unit.*')
<208> [<pos 161788748 childIndex: 2 lvl: 1 [1] Unit tests>]
ileo[~]|209> pl.select(' Unit.*').select('.*')
<209>
[<pos 161788620 childIndex: 0 lvl: 2 [2] Pass>,
<pos 161789228 childIndex: 1 lvl: 2 [2] Fail>]
ileo[~]|210> pl.select(' Unit.*').select('Pass')
<210> [<pos 161790508 childIndex: 0 lvl: 2 [2] Pass>]
ileo[~]|211> pl.select(' Unit.*').select('Pass').select('leo.*')
<211>
[<pos 161789036 childIndex: 4 lvl: 3 [3] leoBridge>,
<pos 161790956 childIndex: 5 lvl: 3 [3] leoConfig>,
<pos 161791532 childIndex: 6 lvl: 3 [3] leoDialog>,
<pos 161791980 childIndex: 7 lvl: 3 [3] leoGlobals (no tnodeList)>,
<pos 161791084 childIndex: 8 lvl: 3 [3] leoGui>,
<pos 161789068 childIndex: 9 lvl: 3 [3] leoKeys>,
<pos 161791500 childIndex: 10 lvl: 3 [3] leoPlugins>,
<pos 161788364 childIndex: 11 lvl: 3 [3] leoTangle>,
<pos 161791948 childIndex: 12 lvl: 3 [3] leoTest>]

zpcspm

unread,
Jan 2, 2009, 10:46:52 AM1/2/09
to leo-editor
On Jan 2, 3:09 pm, "Edward K. Ream" <edream...@gmail.com> wrote:
> We all agree that the iLeo workbook is a good thing to do. However, the
> payoff of the workbook is for users using IPython with Leo; the payoff for
> script writers is small.

I've just installed IPython. The installation was painless (even if I
had to build a package for my Linux distribution, but this is
irrelevant), mainly because IPython doesn't have other dependencies
that Python itself.
So I've installed IPython, I've managed to get the IPython console
from leo, I saw how outline nodes can be accessed from IPython by
repeating Ville's examples. All that I need now is just practice to
get used to it.
I wrote earlier that leo doesn't enforce the use of IPython. This fact
has pros and cons. On a second thought, leo doesn't encourage too much
the use of IPython as well. Okay, the manual has a chapter related to
IPython, it has examples. But perhaps it lacks a story about a very
generic use case that would clearly show the advantages of using leo
+IPython over using only leo, thus motivating a leo newbie to pay
attention to the IPython plugin from the start.

Edward K. Ream

unread,
Jan 2, 2009, 10:51:44 AM1/2/09
to leo-editor
On Jan 2, 8:52 am, "Edward K. Ream" <edream...@gmail.com> wrote:

> The inventing lamp is lit.

My intuition isn't clear yet. Here are my present, somewhat
contradictory, thoughts:

1. Surely, we must support wb on the IPython command line. There, at
least, it's an excellent idea.

2. There is potential confusion because several nodes (vnodes) may
have the same headline. This suggests that vnodes or gnx's are the
proper keys to use behind-the-scenes.

3. We can sidestep the issue of updating the workbook by having any
Leo operation that changes a node set a c.workBookDirty flag. If this
flag is clear, the workbook can use cached values; otherwise the
workbook can recompute the cached values using something like
c.allNodes_iter. Script writers must be aware that changing the tree
while using c.wb can be expensive.

4. An @wb tree is similar to @chapter wb. We may want to consider a
workbook as a special kind of chapter. This has appeal: it would be
natural to have c.wb.nodeName refer to the first node that matches in
@chapter wb.

OTOH, requiring workbook nodes to be descendants of an @chapter wb
node lessens the usefulness for workbooks for general scripts. Likely
this can't be helped. If workbook nodes can appear anywhere there
seems to be no good way to disambiguate which node is meant when
several nodes have the same headline. *Maybe* using vnodes as keys is
a way forward, but I have my doubts.

In short, Leo can, and probably should, define a c.wb class, but this
class will exist mainly to support access from the IPython command
line. General Leo scripts could also use this class, but they must be
aware that all nodes will be members of the wb chapter. I still see
no way to make c.wb be a surrogate for c.allNodes_iter in a way that
is clear, useful, non-surprising and efficient.

Edward

Ville M. Vainio

unread,
Jan 2, 2009, 11:08:23 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 5:32 PM, Ville M. Vainio <viva...@gmail.com> wrote:

> However, here is my implementation of "select()" functionality that
> seems to work (and IS appropriate for scripting usage):

I added PosList to ileo - it may turn redundant if it's added to core
(along with .h, .b properties).

Here's an example oneliner that prints the list of @shadow files of my
Enemy Territory config directory ;-) :

ileo[~/ipython-extensions]|21> print " ".join( n.h.lower().split()[1]
for n in wb.match_h('Enemy
territory').select('@path.*').select('@shadow.*') )

soldier1.cfg field.cfg covert2.cfg superselector.cfg tweaks.cfg
engineer.cfg covert.cfg medic.cfg soldier2.cfg soldier3.cfg
soldier4.cfg toggle.cfg talkbinds.cfg close.cfg long.cfg tuc-main.cfg
controls.cfg rec-stop-demo.cfg etconfig.cfg

Note that this makes it easy to write leo document traversal without
really knowing about different iterators at all.

Edward K. Ream

unread,
Jan 2, 2009, 11:11:13 AM1/2/09
to leo-editor
On Jan 2, 9:32 am, "Ville M. Vainio" <vivai...@gmail.com> wrote:

> However, here is my implementation of "select()" functionality that
> seems to work (and IS appropriate for scripting usage):
>
> class PosList(list):
>     def select(self, pat):
>         res = PosList()
>         for p in self:
>             #print "po",p
>             for chi_p in p.children_iter():
>                 #print "chi",chi_p
>                 if re.match(pat, chi_p.headString()):
>                     res.append(chi_p.copy())
>         return res

Very clever. PosList is a subclass of the list class, so no ctor is
required. The statement::

pList = PosList([c.currentPosition()])

is equivalent to::

pList = [c.currentPosition]

except that a select method defined for the list pList.

The second clever thing about this is that the select method doesn't
cache anything, so there is no need to update the cache when the
outline changes. Naturally, pList will become invalid if any node in
pList gets deleted. This could be fixed by adding a guard that calls
the (newly fixed) c.positionExists.

I'm not sure how this relates to the general problems with c.wb, but
it's good to have in any case.

Edward

Ville M. Vainio

unread,
Jan 2, 2009, 11:20:18 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 6:11 PM, Edward K. Ream <edre...@gmail.com> wrote:

> The second clever thing about this is that the select method doesn't
> cache anything, so there is no need to update the cache when the
> outline changes. Naturally, pList will become invalid if any node in
> pList gets deleted. This could be fixed by adding a guard that calls
> the (newly fixed) c.positionExists.
>
> I'm not sure how this relates to the general problems with c.wb, but
> it's good to have in any case.

It basically means that wb can be relegated to it's role in IPython
(i.e. interactive use). Of course anyone that enables ipython plugin
can use it, but select() is better for general-purpose scripting usage
(and should be aesthetically pleasing in its simplicity).

BTW, subclassing list type is very cool - SList is one of my favourite
features in ipython, enabling stuff like:

ileo[~/leo-editor]|27> leofiles = !ls -l
ileo[~/leo-editor]|28> leofiles
<28> SList (.p, .n, .l, .s, .grep(), .fields(),
sort() available):
0: total 116
1: -rw------- 1 ville ville 196 2008-12-21 15:52 bzr_log.aSxdUM~
2: -rwxr-xr-x 1 ville ville 1415 2008-11-29 14:54 install
3: -rw-r--r-- 1 ville ville 129 2008-09-18 21:05 launchLeo.py
4: drwxr-xr-x 13 ville ville 384 2008-09-20 14:04 leo
5: -rw-r--r-- 1 ville ville 68184 2008-12-21 20:46 leo.e4p
6: -rw-r--r-- 1 ville ville 250 2008-12-21 20:47 leo.e4q
7: -rw-r--r-- 1 ville ville 188 2008-12-21 20:47 leo.e4t
8: drwxr-xr-x 2 root root 208 2008-09-18 21:07 leo_editor.egg-info
9: -rw-r--r-- 1 ville ville 1460 2008-09-18 21:05 leo_to_html.xsl
10: -rw-r--r-- 1 ville ville 19267 2008-11-29 14:54 pylint-leo.bat
11: -rw-r--r-- 1 ville ville 1399 2008-09-18 21:05 setup.py
ileo[~/leo-editor]|29> leofiles.grep('leo',prune=1)
<29> SList (.p, .n, .l, .s, .grep(), .fields(),
sort() available):
0: total 116
1: -rw------- 1 ville ville 196 2008-12-21 15:52 bzr_log.aSxdUM~
2: -rwxr-xr-x 1 ville ville 1415 2008-11-29 14:54 install
3: -rw-r--r-- 1 ville ville 1399 2008-09-18 21:05 setup.py

Similar convenience methods can be added to PosList at a later date,
as need arises.

Edward K. Ream

unread,
Jan 2, 2009, 11:33:52 AM1/2/09
to leo-editor
On Jan 2, 10:08 am, "Ville M. Vainio" <vivai...@gmail.com> wrote:

> Here's an example one-liner that prints the list of @shadow files of my
> Enemy Territory config directory ;-) :
>
> ileo[~/ipython-extensions]|21> print " ".join( n.h.lower().split()[1]
> for n in wb.match_h('Enemy
> territory').select('@path.*').select('@shadow.*') )

Excellent example! "Enemy territory" is, in effect, a namespace from
which nodes can selected.

match_h is defined in ipython-extensions:ipy_leo.py. It returns a
list of all nodes in the entire outline whose headline matches the
regular expression.

> Note that this makes it easy to write leo document traversal without
> really knowing about different iterators at all.

I'm convinced. This is a much better solution than I was
contemplating. These methods are clear, unambiguous and useful, and
they require no change to Leo's core, other than supporting the
workbook class, that is. Scripts are unlikely to suffer serious
performance problems when using these methods.

Edward

Ville M. Vainio

unread,
Jan 2, 2009, 11:37:06 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 6:33 PM, Edward K. Ream <edre...@gmail.com> wrote:

>> Note that this makes it easy to write leo document traversal without
>> really knowing about different iterators at all.
>
> I'm convinced. This is a much better solution than I was
> contemplating. These methods are clear, unambiguous and useful, and
> they require no change to Leo's core, other than supporting the
> workbook class, that is. Scripts are unlikely to suffer serious
> performance problems when using these methods.

You don't have to support workbook class, either. Just add match_h to
controller. This is unrelated to chief raison d'etre of WorkBook, that
of providing easy access to leo positions through getattr magic.

As it stands, we can leave WorkBook as ILeo feature.

Ville M. Vainio

unread,
Jan 2, 2009, 11:43:20 AM1/2/09
to leo-e...@googlegroups.com
On Fri, Jan 2, 2009 at 5:51 PM, Edward K. Ream <edre...@gmail.com> wrote:

> 3. We can sidestep the issue of updating the workbook by having any
> Leo operation that changes a node set a c.workBookDirty flag. If this
> flag is clear, the workbook can use cached values; otherwise the
> workbook can recompute the cached values using something like
> c.allNodes_iter. Script writers must be aware that changing the tree
> while using c.wb can be expensive.

Here's an idea:

- Add a set dirtySet to controller.
- Every time the tree becomes dirty, leo clears that set.
- Plugins can add own items to that set, and if that item does not
exist, the plugin will know that the outline is dirty since the last
time it checked.

Example::

cached_data = {}

def get_cache():
if 'myplugin' in c.dirtySet:
# tree is still not dirty
return cached_data
generate_cache(cached_data)
c.dirtySet.add('myplugin')
return cached_data

> 4. An @wb tree is similar to @chapter wb. We may want to consider a
> workbook as a special kind of chapter. This has appeal: it would be
> natural to have c.wb.nodeName refer to the first node that matches in
> @chapter wb.

That was the original implementation in ILeo (@ipy-root node), but it
didn't turn out to be that handy. Typically, you want the special
nodes just next to the scripts you are working with, for easy access.
Having to clone is a drag then.

Lorenz Köhl

unread,
Jan 2, 2009, 1:41:48 PM1/2/09
to leo-e...@googlegroups.com
zpcspm wrote the following:
> [...] Okay, the manual has a chapter related to IPython,

> it has examples. But perhaps it lacks a story about a
> very generic use case that would clearly show the
> advantages of using leo +IPython over using only leo,
> thus motivating a leo newbie to pay attention to the
> IPython plugin from the start.

I'm not very experienced with using LEO as a Python IDE, but
for me it makes all the difference. It eliminates the edit,
save, run cycle to test my code. How do people do that
without a push-to-interpreter function (provided by ILeo)?

The helpful image for me is that I don't edit code in an
editor buffer/file -- I change/add code to a running
interactive environment. LISP was the first programming
language to support this style of writing programs as
growing organisms.

I'm currently reading through some of Dijkstra's essays and
he strongly opposes the use of allegory to think about
computer programming. His structured programming approach
tries to organize code so one can prove it's correctness.
Knuth's Literate Programming can be seen as an enhancement
on structured programs sequencing/ordering so I think LEO
(and ILeo) can provide the best of two approaches to program
development -- interactive and well structured.

Edward K. Ream

unread,
Jan 10, 2009, 7:38:44 AM1/10/09
to leo-e...@googlegroups.com


On Fri, Jan 2, 2009 at 12:41 PM, Lorenz Köhl <lor...@quub.de> wrote:

I'm not very experienced with using LEO as a Python IDE, but
for me it makes all the difference. It eliminates the edit,
save, run cycle to test my code. How do people do that
without a push-to-interpreter function (provided by ILeo)?

See this FAQ entry:

http://webpages.charter.net/edreamleo/FAQ.html#how-can-i-use-leo-to-develop-leo-itself

This is the workflow I use myself, and it is extremely efficient.

Edward
Reply all
Reply to author
Forward
0 new messages