ILeo (IPython-Leo bridge) introduction
Ville M. Vainio
doc/examples/leo_bridge_demo.leo. But here it is, to whet the appetite
for IPython 0.8.3 (for those who want to wait for releases for some
reason) ;-)
I'll also post this non-final version to c.l.py, but here it goes:
Introduction
============
The purpose of ILeo, or leo-ipython bridge, is being a two-way communication
channel between Leo and IPython. The level of integration is much deeper than
conventional integration in IDEs; most notably, you are able to store *data* in
Leo nodes, in addition to mere program code. The possibilities of this are
endless, and this degree of integration has not been seen previously
in the python
world.
IPython users are accustomed to using things like %edit to produce non-trivial
functions/classes (i.e. something that they don't want to enter directly on the
interactive prompt, but creating a proper script/module involves too much
overhead). In ILeo, this task consists just going to the Leo window,
creating a node
and writing the code there, and pressing alt+I (push-to-ipython).
Obviously, you can save the Leo document as usual - this is a great advantage
of ILeo over using %edit, you can save your experimental scripts all at one
time, without having to organize them into script/module files (before you
really want to, of course!)
Installation
============
You need at least Leo 4.4.7, and the development version of IPython (ILeo
will be incorporated to IPython 0.8.3).
You can get IPython from Launchpad by installing bzr and doing
bzr branch lp:ipython
and running "setup.py install".
You need to enable the 'ipython.py' plugin in Leo:
- Help -> Open LeoSettings.leo
- Edit @settings-->Plugins-->@enabled-plugins, add/uncomment 'ipython.py'
- Restart Leo. Be sure that you have the console window open (start
leo.py from console, or double-click leo.py on windows)
- Press alt+5 OR alt-x start-ipython to launch IPython in the console that
started leo. You can start entering IPython commands normally, and Leo will keep
running at the same time.
Accessing IPython from Leo
==========================
IPython code
------------
Just enter IPython commands on a Leo node and press alt-I to execute
push-to-ipython to execute the script in IPython. 'commands' is interpreted
loosely here - you can enter function and class definitions, in addition to the
things you would usually enter at IPython prompt - calculations,
system commands etc.
Everything that would be legal to enter on IPython prompt is legal to execute
from ILeo.
Results will be shows in Leo log window for convenience, in addition
to the console.
Suppose that a node had the following contents:
{{{
1+2
print "hello"
3+4
def f(x):
return x.upper()
f('hello world')
}}}
If you press alt+I on that done, you will see the following in Leo log
window (IPython tab):
{{{
In: 1+2
<2> 3
In: 3+4
<4> 7
In: f('hello world')
<6> 'HELLO WORLD'
}}}
(numbers like <6> mean IPython output history indices).
Plain Python code
-----------------
If the headline of the node ends with capital P, alt-I will not run the code
through IPython translation mechanism but use the direct python 'exec' statement
(in IPython user namespace) to execute the code. It wont be shown in IPython
history, and sometimes it is safer (and more efficient) to execute things as
plain Python statements. Large class definitions are good candidates for P
nodes.
Accessing Leo nodes from IPython
================================
The real fun starts when you start entering text to leo nodes, and are using
that as data (input/output) for your IPython work.
Accessing Leo nodes happens through the variable 'wb' (short for "WorkBook")
that exist in the IPython user namespace. Nodes that are directly accessible are
the ones that have simple names which could also be Python variable names;
'foo_1' will be accessible directly from IPython, whereas 'my scripts' will not.
If you want to access a node with arbitrary headline, add a child node '@a foo'
(@a stands for 'anchor'). Then, the parent of '@a foo' is accessible through
'wb.foo'.
You can see what nodes are accessible be entering (in IPython)
wb.<TAB>. Example:
[C:leo/src]|12> wb.
wb.b wb.tempfile wb.rfile wb.NewHeadline
wb.bar wb.Docs wb.strlist wb.csvr
Suppose that we had a node with headline 'spam' and body:
['12',2222+32]
we can access it from IPython (or from scripts entered into other Leo
nodes!) by doing:
C:leo/src]|19> wb.spam.v
<19> ['12', 2254]
'v' attribute stands for 'value', which means the node contents will be run
through 'eval' and everything you would be able to enter into IPython prompt
will be converted to objects. This mechanism can be extended far beyond direct
evaluation (see '@cl definitions').
'v' attribute also has a setter, i.e. you can do:
wb.spam.v = "mystring"
Which will result in the node 'spam' having the following text:
'mystring'
What assignment to 'v' does can be configured through generic functions
(simplegeneric module, will be explained later).
Besides v, you can set the body text directly through wb.spam.b =
"some\nstring", headline by wb.spam.h = 'new_headline' (obviously you must
access the node through wb.new_headline from that point onwards), and access the
contents as string list (IPython SList) through 'wb.spam.l'.
If you do 'wb.foo.v = 12' when node named 'foo' does not exist, the node titled
'foo' will be automatically created and assigned body 12.
@cl definitions
===============
If the first line in the body text is of the form '@cl sometext', IPython will
will evaluate 'sometext' and call the result with the rest of the body when you
do 'wb.foo.v'. An example is in place here. Suppose that we have defined a class
(I use the term class in a non-python sense here)
{{{
def rfile(body,n):
""" @cl rfile
produces a StringIO (file like obj) of the rest of the body """
import StringIO
return StringIO.StringIO(body)
}}}
Now, let's say you node 'spam' with text
{{{
@cl rfile
hello
world
and whatever
}}}
Now, on IPython, we can do this:
{{{
[C:leo/src]|22> f = wb.spam.v
[C:leo/src]|23> f
<23> <StringIO.StringIO instance at 0x04E7E490>
[C:leo/src]|24> f.readline()
<24> u'hello\n'
[C:leo/src]|25> f.readline()
<25> u'world\n'
[C:leo/src]|26> f.readline()
<26> u'and whatever'
[C:leo/src]|27> f.readline()
<27> u''
}}}
You should declare new @cl types to make ILeo as convenient your
problem domain as possible. For example, a "@cl etree" could return
the elementtree object for xml content, or
Special node types
==================
@ipy-startup
------------
If this node exist, the *direct children* of this will be pushed to IPython when
ILeo is started (you press alt+5). Use it to push your own @cl definitions etc.
The contents of of the node itself will be ignored.
@ipy-results
------------
When you create a new node (wb.foo.v = 'stuff'), the node foo will be created as
a child of this node. If @ipy-results does not exist, the new node
will be created after the currently selected node.
@a nodes
--------
You can attach these as children of existing nodes to provide a way to access
nodes with arbitrary headlines, or to provide aliases to other nodes. If
multiple @a nodes are attached as children of a node, all the names can be used
to access the same object.
Acknowledgements & History
==========================
This idea got started when I (Ville) saw this post by Edward Ream (the author of
Leo) on IPython developer mailing list:
http://lists.ipython.scipy.org/pipermail/ipython-dev/2008-January/003551.html
I was using FreeMind as mind mapping software, and so I had an immediate use
case for Leo (which, incidentally, is superior to FreeMind as mind mapper). The
wheels started rolling, I got obsessed with the power of this concept
(everything clicked together), and Edwards excitement paralleled mine.
Everything was mind-bogglingly easy/trivial, something that is typical of all
revolutionary technologies (think Python here).
The discussion that "built" ILeo is here:
http://sourceforge.net/forum/forum.php?thread_id=1911662&forum_id=10226
--
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
Edward K. Ream
> I wrote a quick intro to ILeo (IPython + Leo bridge), available in
Ville, I never dreamed that I would have as good a day as yesterday,
as I happily read and re-read this post. One reason for my joy was
the knowledge that Leo is destined to outlive me--the IPython
community is going to take this work of yours and make it their own.
Furthermore, you are going to make me famous: you write so much better
about Leo than I do :-)
ILeo follows in the footsteps of e's great scripting plugin. It opens
new directions for scripting; it creates new relationships between
programs, scripts, data, metadata and outline structure.
I wonder how many of Leo's users understand the true meaning of the
first words on Leo's home page:
"Leo shows user-created relationships among any kind of data."
Clearly, you understand this sentence: ILeo is one specific example of
its power.
I had the following Aha while thinking about this post: there is an
intimate interplay between specifics and generalities. Each supports
the other. Neither works as well without the other. Generalities like
the quote above are almost impossible to understand without specific
examples. OTOH, the quote above captures the essence of @whatever
nodes, including @ipy-whatever nodes. So generalities call forth more
specific examples.
I'll say more later, but for now I'll just say, great work Ville!
Edward
Ville M. Vainio
> as I happily read and re-read this post. One reason for my joy was
> the knowledge that Leo is destined to outlive me--the IPython
> community is going to take this work of yours and make it their own.
The thing about Leo is that it's a *concept*, not an editor. I now
think of it as an editable "window" to Python object space. Such a
concept deserves to outlive all of us :-).
What I think ILeo does well here is that it makes the (illusory)
disconnect between leo objects and python objects disappear - nobody
should be intimidated by this snippet that sets all headlines in the
workbook to 'hello':
for node in wb:
node.h = 'hello'
And when they see that both IPython and Leo keep running at the same
time, I hope they'll have the same mindblowing AHA that I had last
month :-)
(and yeah, I've been having those mind blowing AHA's all the time
since I started - this is a very inspiring environment to play in).
BTW, if you still wonder why Leo keeps running when IPython mainloop
has started - there is some special sauce in Python that makes Tkinter
event loop run even when python code is running. Or at least that is
what Fernando seems to be saying here:
http://mail.python.org/pipermail/python-dev/2005-November/058057.html
:
"Tk worked out of the box (because of the Tkinter event loop
integration in Python)"
> examples. OTOH, the quote above captures the essence of @whatever
> nodes, including @ipy-whatever nodes. So generalities call forth more
> specific examples.
Yeah, it is worth stressing that headlines and bodystrings, even with
those @ characters, are just strings and Leo is not going to explode
when you enter @whatever there. @whatever means @whatever you and your
problem domain want it to mean :-)
Brian Theado
> I wrote a quick intro to ILeo (IPython + Leo bridge), available in
Nice introduction. I was able to follow your instructions and get iLeo running.
[...]
> 'v' attribute also has a setter, i.e. you can do:
>
> wb.spam.v = "mystring"
>
> Which will result in the node 'spam' having the following text:
>
> 'mystring'
When I played with this, my first thought was that since I was
modifying a leo outline from iPython, that I would want to save it
from there as well. That led me to thinking it would be useful to be
able to run leo minibuffer commands from iPython. This can be done
with:
leox.c.executeMinibufferCommand "save-file"
and it is convenient to make an alias in order to shorten the command:
mb = leox.c.executeMinibufferCommand
so now this works:
mb "save-file"
iPython lacks command-completion for these commands, so it isn't as
convenient as using the minibuffer. That led me to the thought that
iPython has such nice tab completion, is there a way to hook
completion into the minibuffer commands. My only idea so far is to
dynamically create a python object containing one method for each
minibuffer command (I guess dashes would need to be mapped to
underscores or something to make them valid python identifiers). Then
since it is a python object, iPython will automatically support tab
completion. Any other ideas?
One thing I notice with both my 'mb' object above and the iLeo 'wb'
object is that once iPython is launched with Alt-5, these objects are
bound to the original leo document. If I open another leo document,
the 'wb' object still operates on the original leo document. Maybe
this behavior makes the most sense, but it seems like there should be
any easy way to switch context of the wb object. I've seen there is a
leox.g.app.windowList, but I don't know what to do with it to get a
different commander hooked into iPython.
Brian
Ville M. Vainio
> iPython lacks command-completion for these commands, so it isn't as
> convenient as using the minibuffer. That led me to the thought that
> iPython has such nice tab completion, is there a way to hook
> completion into the minibuffer commands. My only idea so far is to
Of course there is, and it's easy as well :-)
See how ipy_leo does the completer for workbook:
@IPython.generics.complete_object.when_type(LeoWorkbook)
def workbook_complete(obj, prev):
return all_cells().keys() + [s for s in prev if not s.startswith('_')]
We just introduce LeoMinibuffer class, implement getattr for that
(that does the dash-underscore mapping), and implement a completer for
that. Should be easy. And great idea btw.
Can the minibuffer commands take arguments? If that is the case, we
should rather make it a magic function and create a custom completer
for that.
> One thing I notice with both my 'mb' object above and the iLeo 'wb'
> object is that once iPython is launched with Alt-5, these objects are
> bound to the original leo document. If I open another leo document,
> the 'wb' object still operates on the original leo document. Maybe
> this behavior makes the most sense, but it seems like there should be
> any easy way to switch context of the wb object. I've seen there is a
> leox.g.app.windowList, but I don't know what to do with it to get a
> different commander hooked into iPython.
Yeah, we need to figure out a way to dig up a new commander from Leo
(using something in g). Any pointers?
Ville M. Vainio
> Can the minibuffer commands take arguments? If that is the case, we
> should rather make it a magic function and create a custom completer
> for that.
Now that I think of it more, this minibuffer thing probably *is* more
elegant with magic functions. We can keep using dashes instead of
translating to underscores.
Ville M. Vainio
> On Sun, Feb 24, 2008 at 11:35 AM, Ville M. Vainio <viva...@gmail.com> wrote:
>
> > Can the minibuffer commands take arguments? If that is the case, we
> > should rather make it a magic function and create a custom completer
> > for that.
>
> Now that I think of it more, this minibuffer thing probably *is* more
> elegant with magic functions. We can keep using dashes instead of
> translating to underscores.
Easy as pie, done in launchpad:
def mb_f(self, arg):
""" Execute leo minibuffer commands """
c.executeMinibufferCommand(arg)
def mb_completer(self,event):
""" Custom completer for minibuffer """
cmds = c.commandsDict.keys()
cmds.sort()
return cmds
pass
ip.set_hook('complete_command', mb_completer, str_key = 'mb')
ip.expose_magic('mb',mb_f)
Ville M. Vainio
> One thing I notice with both my 'mb' object above and the iLeo 'wb'
> object is that once iPython is launched with Alt-5, these objects are
> bound to the original leo document. If I open another leo document,
> the 'wb' object still operates on the original leo document. Maybe
> this behavior makes the most sense, but it seems like there should be
> any easy way to switch context of the wb object. I've seen there is a
> leox.g.app.windowList, but I don't know what to do with it to get a
> different commander hooked into iPython.
You would need to update leox, c and g global variables in ipy_leo
module. That is easy.
I'll fix this in ipython.py plugin (i.e. Leo side) so that alt+5
always updates the IPython session with current commander. I think
that's pretty intuitive and handy.
Brian Theado
> > elegant with magic functions. We can keep using dashes instead of
> > translating to underscores.
>
> Easy as pie, done in launchpad:
Thanks for this. It worked for me after I figured out to use 'mb' and
not '%mb'. When I type '%mb<space><tab>', then I'm presented with a
list of what looks like toplevel python methods. When I type
'mb<space><tab>', then I'm presented with the list of minibuffer
commands. Just inexperience with iPython (and python), I guess. Now
that I know, it works fine.
With this, iPython is almost viable as a minibuffer replacement.
iPython has the advantage of being able to recall previous commands
with the up arrow. It has the disadvantage of being in a separate
console window instead of attached to the Leo GUI. It also has the
disadvantage that you mention in another thread about not being able
to input arguments. When you call a minibuffer command that presents
an interactive prompt (i.e. clone-find-all), then you have to switch
to the leo gui in order to switch to enter the string to find.
Brian
Ville M. Vainio
> Thanks for this. It worked for me after I figured out to use 'mb' and
> not '%mb'. When I type '%mb<space><tab>', then I'm presented with a
> list of what looks like toplevel python methods. When I type
> 'mb<space><tab>', then I'm presented with the list of minibuffer
> commands. Just inexperience with iPython (and python), I guess. Now
> that I know, it works fine.
No, it was a bug, which is now fixed (in ileo-exp branch).
Edward K. Ream
> doc/examples/leo_bridge_demo.leo. But here it is, to whet the appetite
> for IPython 0.8.3 (for those who want to wait for releases for some
> reason) ;-)
post. The stuff has really hit the fan recently: bounced tax checks,
multiple furnace and chimney problems, an ice storm. Just a few
questions to start...
1. How much will the user be expected to know about ipy_leo.py? If the
answer is "nothing", then how much needs to be documented?
2. Is the documentation supposed to replace Leo's ipython bridge
documentation, or is it supposed to be in addition to Leo's
documentation? In particular, it seems like you either need to
reference the Leo docs for the bridge, or duplicate some of the
salient features. Imo, it's especially important to emphasize that c
& g (or perhaps leox.c and leox.g, see below) give access to
everything needed: everything else is just convenience.
3. Are any parts of the present Leo docs at
http://webpages.charter.net/edreamleo/IPythonBridge.html obsolete?
Some comments:
1. It seems dubious to put c and g into the IPython space. Both have
well-known meanings in physics. Why not just put leox into the
space?
2. I've changed my mind about moving some of the ideas from ipy_leo.py
to Leo's core. (from another thread) Your code was more elegant than
your initial proposal :-) I was thrown off by having shortcuts to
eliminate c.begin/endUpdate. In fact, properties are a great idea.
p.h = x (and the rest) is a great simplification. I should have
thought of it before.
All these questions and comments are pretty much just nits, but we are
getting to that point, it seems to me.
Edward
Ville M. Vainio
> 1. How much will the user be expected to know about ipy_leo.py? If the
> answer is "nothing", then how much needs to be documented?
Pretty much nothing, unless the user wants to extend the behaviour
(i.e. add own 'push' handlers).
The documentation needs are pretty much covered by what I have written
already. I can move that document to LeoDocs, if that makes sense.
> 2. Is the documentation supposed to replace Leo's ipython bridge
> documentation, or is it supposed to be in addition to Leo's
> documentation? In particular, it seems like you either need to
I think the ipython bridge documentation is way too detailed /
advanced for basic use. The chapter 'Scripting IPython from Leo" is
probably out of the scope that what most people will "get" - and most
scripting will happen to other direction. What people need to know
from leo side is that alt+I works, and that's it.
> reference the Leo docs for the bridge, or duplicate some of the
> salient features. Imo, it's especially important to emphasize that c
> & g (or perhaps leox.c and leox.g, see below) give access to
> everything needed: everything else is just convenience.
Yeah, that needs to be mentioned. My intro was just a quick rundown, I
will add more detail in the following days, including the fact that
most stuff is just convenience wrapping for already existing leo
functionality. But IPython is all about convenience :-)
> 3. Are any parts of the present Leo docs at
> http://webpages.charter.net/edreamleo/IPythonBridge.html obsolete?
Yes, there is no leox anymore, it's called _leo now. Likewise, this is
not IPShellEmbed, just say "manages a singleton IPython shell object".
> 1. It seems dubious to put c and g into the IPython space. Both have
> well-known meanings in physics. Why not just put leox into the
> space?
I liked to play with those variables directly while learning to play
with leo and I'm lazy typist (hey, that's why I became such a loyal
IPython user and later on the maintainer in the first place) ;-)
Of course I can remove them now; it's easy enough to add c,g =
_leo.c,_leo.g to my @startup nodes. And most stuff can be done now w/o
c and g, through ipy_leo.LeoNode and LeoWorkbook.
All normal ipython scripts also push _p (a LeoNode instance) for
current node to ipython namespace.
> 2. I've changed my mind about moving some of the ideas from ipy_leo.py
> to Leo's core. (from another thread) Your code was more elegant than
> your initial proposal :-) I was thrown off by having shortcuts to
> eliminate c.begin/endUpdate. In fact, properties are a great idea.
> p.h = x (and the rest) is a great simplification. I should have
> thought of it before.
Yeah, the beginUpdate was not the point there :-). Consider adding
__iter__ too, though a meaning of that is pretty ambiguous... direct
children/all children/whatever.