> I wonder if leo could provide such a query language in the future,
> that would be much more flexible than UNL and would spare the
> programmer from manually scanning the outline for nodes that match
> certain criteria (perhaps regexp for the node headline, regexp for the
> node body) by providing a way to write [nested] queries.
In ILeo, there is a trivial match_h:
ileo[C:p/leo]|5> list(wb.match_h('Non-'))
<5> [<LeoNode <pos 67824464 childIndex: 4 lvl: 2 [2]
Non-essential>>]
This does a regexp match. Yeah, it's just an allnodes_iter in drag.
Generalized hierarchical node selection is easy to do as well:
node = wb.select('Gui wrap.*').select(".*Qt").select('Font')
The magic here would be a new "LeoNodeList" class that
- inherits from python 'list' type
- provides 'select' method that returns all nodes under any node in
the LeoNodeList (self) that match the pattern as a new LeoNodeList. It
would only traverse one tree level (only work with direct children),
to avoid circularities & other weirdness.
The special "first" select (here wb.select, but it could be c.select()
as well ) would scan the whole tree. Throw in select_b('some body
text') to search nodes by body contents, and we have a winner.
Hey, it could work.
import IPython
IPython.genutils.SList??
Will show you an example implementation of a custom list type that
provides some additional methods (this time provided for string lists,
e.g. system command output).
--
Ville M. Vainio
http://tinyurl.com/vainio
> My first post was a perspective of somebody who uses leo, but doesn't
> use IPython at all: nor IPython itself, neither a combo of leo and
> IPython. It happened because leo doesn't enforce using IPython.
This is why I have been advocating integrating the LeoWorkbook from
ILeo code to Leo itself (or at least the concept!). There is nothing
very IPython-specific about the code, it's just something I thought
would be a more "convenient" API for using data in leo. There is some
help in ILeo to provide code completion for workbook, but that is
outside the class itself (i.e. it doesn't matter whether the class is
in plain-old-leo, or IPython integration layer). Likewise, LeoWorkbook
returns LeoNode objects, which are just thin wrappers around position,
that provide e.g. node.h and node.b (properties to access headString
and bodyString(). Those properties should be in leo's position code in
the first place, to make scripts shorter and easier to read/write.
This is why I have been advocating integrating the LeoWorkbook from
On Thu, Jan 1, 2009 at 10:26 PM, zpcspm <zpc...@gmail.com> wrote:
> My first post was a perspective of somebody who uses leo, but doesn't
> use IPython at all: nor IPython itself, neither a combo of leo and
> IPython. It happened because leo doesn't enforce using IPython.
ILeo code to Leo itself (or at least the concept!).
Preface
Some time ago I was fiddling with a Greasemonkey script that wasn't
written by me. My goal was to provide some refactoring hints to the
authors. Obviously, my first task was to understand what it does and
how. I can read JavaScript code because of some C/Java background, but
not more than that. So my quest included reading the wikipedia
articles (and a little other info) about JQuery and XPath, because I
was totally clueless about them. I won't claim that I could write
something useful now, but now I know at least what are these. Both
JQuery and XPath seem to provide functional query languages.
I was thinking about these from leo's perspective. In leo, we have
something like this:
for p in c.allNodes_iter():
<do something>
Basically, this is "manual" processing.
I wonder if leo could provide such a query language in the future,
that would be much more flexible than UNL and would spare the
programmer from manually scanning the outline for nodes that match
certain criteria (perhaps regexp for the node headline, regexp for the
node body) by providing a way to write [nested] queries.
Likewise, LeoWorkbook
returns LeoNode objects, which are just thin wrappers around position,
that provide e.g. node.h and node.b (properties to access headString
and bodyString(). Those properties should be in leo's position code in
the first place, to make scripts shorter and easier to read/write.
The ? feature makes me wish that Leo had more
thorough docstring coverage.
> We all agree that the iLeo workbook is a good thing to do. However, the
> payoff of the workbook is for users using IPython with Leo; the payoff for
> script writers is small.
Not if you want to write the script "quickly". It gets you around
low-level way of iterating through nodes (effectively reducing code).
I think my "select" method proposition could be a more "rigid" way of
approaching the problem, and would not rely on magic getattr stuff
(which helps with ipython, because it can provide tab completion).
Still, provided that we add tab completion (that fetches from IPython
- i.e. the WorkBook completion works ok) to leo text editor, it'll be
much easier to get to the data of a node.
> Furthermore, p.bodyString(), p.headString() and other getters and setters
> provide trivial access to all parts of Leo's DOM.
>
> In short, little or nothing more is needed.
Providing a convenience wrapper to find nodes may be "little", but it
will certainly make the scripts easier to read and write.
Using the 'select' proposition before, you have to write quite a bit
of code to cover what an oneliner with select() does - and it can
actually encourage people to write more code that uses "live" data in
the leo documents.
> wb[node-name] is a wrapper for c.allNodes_iter. This is benign at the
It's actually wb.NodeName. But the truth is that it's not fast. This
could be made much, much faster by updating the node dict whenever
changes happen.
> Any thoughts about what to do? Documentation is my last choice.
I think you should look at the possibility of implementing the
"select" method first. It's more general purpose and less "magical" -
notably, it works for all kinds of headlines, as opposed to ones that
look like a variable name.
I can provide a demo implementation if my previous explanation was not
clear enough.
> 2. There appear to be *severe* problems with c.wb.nodeName outside the
> context of the command-line:
>
> First, c.wb.nodeName returns the first node of the tree whose name is
> nodeName. This is well defined, but it is also badly defined :-)
> There may be many nodes in a tree with the same name. In general,
> there is no reason to prefer the first node found, so it seems to me
> we are building in confusion.
Yes - wb is for "quick-and-dirty" node discovery, where this tradeoff
has been made purposefully. The idea is that the leo document will
have headlines that are named like variables (i.e. hopefully in unique
fashion, with variable-like names). This does *not* work for discovery
of nodes in a naively created leo outline.
However, here is my implementation of "select()" functionality that
seems to work (and IS appropriate for scripting usage):
class PosList(list):
def select(self, pat):
res = PosList()
for p in self:
#print "po",p
for chi_p in p.children_iter():
#print "chi",chi_p
if re.match(pat, chi_p.headString()):
res.append(chi_p.copy())
return res
Here is example usage:
ileo[~]|194> pl = PosList([c.currentPosition()])
ileo[~]|206> pl
<206> [<pos 145246028 childIndex: 2 lvl: 0 [0] qt gui stuff>]
ileo[~]|207> pl.select('.*')
<207>
[<pos 161789100 childIndex: 0 lvl: 1 [1] To do>,
<pos 161791660 childIndex: 1 lvl: 1 [1] Dumps>,
<pos 161788780 childIndex: 2 lvl: 1 [1] Unit tests>,
<pos 161788396 childIndex: 3 lvl: 1 [1] Unused>,
<pos 161788268 childIndex: 4 lvl: 1 [1] @auto ../test/qtui_generate.p
y>,
<pos 161788204 childIndex: 5 lvl: 1 [1] @auto qt_main.py>,
<pos 161791628 childIndex: 6 lvl: 1 [1] @auto qt_main.ui>,
<pos 161790636 childIndex: 7 lvl: 1 [1] @auto qt_quicksearch.py>,
<pos 161790060 childIndex: 8 lvl: 1 [1] @auto qt_quicksearch.ui>,
<pos 161790796 childIndex: 9 lvl: 1 [1] @thin qtGui.py>,
<pos 161791276 childIndex: 10 lvl: 1 [1] Key problems>]
ileo[~]|208> pl.select(' Unit.*')
<208> [<pos 161788748 childIndex: 2 lvl: 1 [1] Unit tests>]
ileo[~]|209> pl.select(' Unit.*').select('.*')
<209>
[<pos 161788620 childIndex: 0 lvl: 2 [2] Pass>,
<pos 161789228 childIndex: 1 lvl: 2 [2] Fail>]
ileo[~]|210> pl.select(' Unit.*').select('Pass')
<210> [<pos 161790508 childIndex: 0 lvl: 2 [2] Pass>]
ileo[~]|211> pl.select(' Unit.*').select('Pass').select('leo.*')
<211>
[<pos 161789036 childIndex: 4 lvl: 3 [3] leoBridge>,
<pos 161790956 childIndex: 5 lvl: 3 [3] leoConfig>,
<pos 161791532 childIndex: 6 lvl: 3 [3] leoDialog>,
<pos 161791980 childIndex: 7 lvl: 3 [3] leoGlobals (no tnodeList)>,
<pos 161791084 childIndex: 8 lvl: 3 [3] leoGui>,
<pos 161789068 childIndex: 9 lvl: 3 [3] leoKeys>,
<pos 161791500 childIndex: 10 lvl: 3 [3] leoPlugins>,
<pos 161788364 childIndex: 11 lvl: 3 [3] leoTangle>,
<pos 161791948 childIndex: 12 lvl: 3 [3] leoTest>]
> However, here is my implementation of "select()" functionality that
> seems to work (and IS appropriate for scripting usage):
I added PosList to ileo - it may turn redundant if it's added to core
(along with .h, .b properties).
Here's an example oneliner that prints the list of @shadow files of my
Enemy Territory config directory ;-) :
ileo[~/ipython-extensions]|21> print " ".join( n.h.lower().split()[1]
for n in wb.match_h('Enemy
territory').select('@path.*').select('@shadow.*') )
soldier1.cfg field.cfg covert2.cfg superselector.cfg tweaks.cfg
engineer.cfg covert.cfg medic.cfg soldier2.cfg soldier3.cfg
soldier4.cfg toggle.cfg talkbinds.cfg close.cfg long.cfg tuc-main.cfg
controls.cfg rec-stop-demo.cfg etconfig.cfg
Note that this makes it easy to write leo document traversal without
really knowing about different iterators at all.
> The second clever thing about this is that the select method doesn't
> cache anything, so there is no need to update the cache when the
> outline changes. Naturally, pList will become invalid if any node in
> pList gets deleted. This could be fixed by adding a guard that calls
> the (newly fixed) c.positionExists.
>
> I'm not sure how this relates to the general problems with c.wb, but
> it's good to have in any case.
It basically means that wb can be relegated to it's role in IPython
(i.e. interactive use). Of course anyone that enables ipython plugin
can use it, but select() is better for general-purpose scripting usage
(and should be aesthetically pleasing in its simplicity).
BTW, subclassing list type is very cool - SList is one of my favourite
features in ipython, enabling stuff like:
ileo[~/leo-editor]|27> leofiles = !ls -l
ileo[~/leo-editor]|28> leofiles
<28> SList (.p, .n, .l, .s, .grep(), .fields(),
sort() available):
0: total 116
1: -rw------- 1 ville ville 196 2008-12-21 15:52 bzr_log.aSxdUM~
2: -rwxr-xr-x 1 ville ville 1415 2008-11-29 14:54 install
3: -rw-r--r-- 1 ville ville 129 2008-09-18 21:05 launchLeo.py
4: drwxr-xr-x 13 ville ville 384 2008-09-20 14:04 leo
5: -rw-r--r-- 1 ville ville 68184 2008-12-21 20:46 leo.e4p
6: -rw-r--r-- 1 ville ville 250 2008-12-21 20:47 leo.e4q
7: -rw-r--r-- 1 ville ville 188 2008-12-21 20:47 leo.e4t
8: drwxr-xr-x 2 root root 208 2008-09-18 21:07 leo_editor.egg-info
9: -rw-r--r-- 1 ville ville 1460 2008-09-18 21:05 leo_to_html.xsl
10: -rw-r--r-- 1 ville ville 19267 2008-11-29 14:54 pylint-leo.bat
11: -rw-r--r-- 1 ville ville 1399 2008-09-18 21:05 setup.py
ileo[~/leo-editor]|29> leofiles.grep('leo',prune=1)
<29> SList (.p, .n, .l, .s, .grep(), .fields(),
sort() available):
0: total 116
1: -rw------- 1 ville ville 196 2008-12-21 15:52 bzr_log.aSxdUM~
2: -rwxr-xr-x 1 ville ville 1415 2008-11-29 14:54 install
3: -rw-r--r-- 1 ville ville 1399 2008-09-18 21:05 setup.py
Similar convenience methods can be added to PosList at a later date,
as need arises.
>> Note that this makes it easy to write leo document traversal without
>> really knowing about different iterators at all.
>
> I'm convinced. This is a much better solution than I was
> contemplating. These methods are clear, unambiguous and useful, and
> they require no change to Leo's core, other than supporting the
> workbook class, that is. Scripts are unlikely to suffer serious
> performance problems when using these methods.
You don't have to support workbook class, either. Just add match_h to
controller. This is unrelated to chief raison d'etre of WorkBook, that
of providing easy access to leo positions through getattr magic.
As it stands, we can leave WorkBook as ILeo feature.
> 3. We can sidestep the issue of updating the workbook by having any
> Leo operation that changes a node set a c.workBookDirty flag. If this
> flag is clear, the workbook can use cached values; otherwise the
> workbook can recompute the cached values using something like
> c.allNodes_iter. Script writers must be aware that changing the tree
> while using c.wb can be expensive.
Here's an idea:
- Add a set dirtySet to controller.
- Every time the tree becomes dirty, leo clears that set.
- Plugins can add own items to that set, and if that item does not
exist, the plugin will know that the outline is dirty since the last
time it checked.
Example::
cached_data = {}
def get_cache():
if 'myplugin' in c.dirtySet:
# tree is still not dirty
return cached_data
generate_cache(cached_data)
c.dirtySet.add('myplugin')
return cached_data
> 4. An @wb tree is similar to @chapter wb. We may want to consider a
> workbook as a special kind of chapter. This has appeal: it would be
> natural to have c.wb.nodeName refer to the first node that matches in
> @chapter wb.
That was the original implementation in ILeo (@ipy-root node), but it
didn't turn out to be that handy. Typically, you want the special
nodes just next to the scripts you are working with, for easy access.
Having to clone is a drag then.
I'm not very experienced with using LEO as a Python IDE, but
for me it makes all the difference. It eliminates the edit,
save, run cycle to test my code. How do people do that
without a push-to-interpreter function (provided by ILeo)?
The helpful image for me is that I don't edit code in an
editor buffer/file -- I change/add code to a running
interactive environment. LISP was the first programming
language to support this style of writing programs as
growing organisms.
I'm currently reading through some of Dijkstra's essays and
he strongly opposes the use of allegory to think about
computer programming. His structured programming approach
tries to organize code so one can prove it's correctness.
Knuth's Literate Programming can be seen as an enhancement
on structured programs sequencing/ordering so I think LEO
(and ILeo) can provide the best of two approaches to program
development -- interactive and well structured.
I'm not very experienced with using LEO as a Python IDE, but
for me it makes all the difference. It eliminates the edit,
save, run cycle to test my code. How do people do that
without a push-to-interpreter function (provided by ILeo)?