Plotting (once again)
Robert Schroll
The discussion about basemap have gotten me thinking about other ways to
handle plotting in Reinteract. Both replot and refigure work on the
model of recording plotting commands as they are entered, but saving
them for execution until the plot is actually displayed. This works
well as long as the plot commands are known ahead of time. But new
plotting commands cannot be introduced without some additional work.
An alternative, which I've started exploring
(http://jfi.uchicago.edu/~rschroll/refigure/refigure2.py) is to put all
the plotting commands in a block, so that they are all executed at once,
setting up the figure at the beginning of the block, and displaying at
the end. Because of the __enter__ and __exit__ methods, I thought that
a with block might work. The goal is to have a syntax like
>>> with figure() as f:
... <plot command>
... <plot command>
... <plot command>
with setup handled in f.__enter__() and display and tear down in
f.__exit__(). This does eliminate the single line plot() command (as it
must live within the with block), but allows you to use any unaltered
plotting command within the block. I think this is a fair trade-off.
Unfortunately, the current version of refigure2 can't actually display
the figure from within the f.__exit__() method. (Instead, you need to
make the last line of the with block be 'f', which is the CustomResult
object.) I think this is fundamental - I don't see anyway to return a
CustomResult to Reinteract from the with statement itself.
Since there have been other discussions about how to create code blocks
within Reinteract, I thought there might be a legitimate case for
extending Reinteract's handling of with blocks. Here's a few brainstorms:
1) Reinteract could check the return value of the __exit__ method of a
with block. If it is a CustomResult, this value is embedded in the
notebook.
2) Reinteract could check for a special __reinteract_exit__ method, call
it if exists, and embed the return value.
3) Reinteract could check if the variable created by a with statement is
a CustomResult, and embed it in the notebook after the block is executed.
These three all require Reinteract to have some knowledge about what's
going on within a block of code. If I understand correctly, that's not
how Reinteract currently works.
4) Reinteract could trap 'CustomResultException's that indicate a
CustomResult is to be embedded in the notebook. I think that this would
require fewer changes to Reinteract and give us more use cases, at the
cost of interrupting execution to display the CustomResult. (You'd
better throw that execption only at the very end!)
5) Reinteract could implement it's own block type that would behave much
like with, but would return the value of the __exit__ method to the
notebook. The avoids changing the meaning of existing syntax at the
cost of introducing new syntax.
Are any of these ideas worthwhile? Or have I missed a basic problem
with this scheme?
Let me know,
Robert
Robert Schroll
I've been playing around with my refigure2 module, and I think I like it
better than its predecessor. The thing I really like about it is I can
use all my old plotting routines, originally written for the
interpreter, from within Reinteract. No more must I split my time
between Reinteract and ipython.
I've tidied up refigure2 a bit and added a few features. If you're
interested, you can get the new version and a brief tutorial from my
website (http://jfi.uchicago.edu/~rschroll/refigure/). The new features
are:
1) Single plotting commands can be used on their own, instead of being
wrapped in a with block. This means you can do `plot(x,y)` instead of
`with figure() as f: plot(x,y); f`.
2) The f returned by `with figure() as f` now inherits from matplotlib's
Figure(), so you can do matplotlib object-oriented stuff to it.
3) Locking is implemented, so if two worksheets are executing at the
same time, they won't draw onto each other's figures. (Note: this makes
execution hang with nested `with figure()` blocks. So don't do that.)
4) refigure2 is less picky about backends. If matplotlib's
rcParams['backend_fallback'] is True (which it is by default), refigure2
will choose one of the GTK backends for you.
I'd appreciate any comments on refigure2, especially from those of you
who have also spent time thinking about plotting in Reinteract.
I've also been thinking about these ideas:
> 1) Reinteract could check the return value of the __exit__ method of a
> with block. If it is a CustomResult, this value is embedded in the
> notebook.
Based on my current understanding, there's no easy (or even moderately
difficult) way to do this. Moreover, this could interfere with the
stated purpose of the __exit__ return value, dealing with exceptions.
>
> 2) Reinteract could check for a special __reinteract_exit__ method, call
> it if exists, and embed the return value.
>
> 3) Reinteract could check if the variable created by a with statement is
> a CustomResult, and embed it in the notebook after the block is executed.
It seems to me that these could be implemented relatively easily. In
rewrite.py, Reinteract is already messing with the AST of the code
chunks before executing them. I think Reinteract could rewrite with
statements like so:
with <thing> as <var>:
<commands>
to
with <thing> as <var>:
<commands>
try:
<var>.__reinteract_exit__()
except AttributeError:
pass
This would run into trouble for with statements without the as clause.
But they could be dealt with by first rewriting
with <thing>:
to
with <thing> as _internal_var:
Which would work until we run into nested with blocks. We could replace
_internal_var with a stack, but it seems to me to be easier to instead
put the try block first within the with block:
with <thing> as <var>:
<commands>
to
with <thing> as <var>:
try:
<var>.__reinteract_display__()
except Attribute Error:
pass
<commands>
As long as __reinteract_display__() returns a CustomResult, it's
create_widget() method won't be called until after the entire chunk
containing the with block has been executed, so the displayed widget can
reflect the processing that happens in <commands>.
> 4) Reinteract could trap 'CustomResultException's that indicate a
> CustomResult is to be embedded in the notebook. I think that this would
> require fewer changes to Reinteract and give us more use cases, at the
> cost of interrupting execution to display the CustomResult. (You'd
> better throw that execption only at the very end!)
I think this would be easy, but probably not a good idea. You couldn't
put one of these inside a for loop, for example.
>
> 5) Reinteract could implement it's own block type that would behave much
> like with, but would return the value of the __exit__ method to the
> notebook. The avoids changing the meaning of existing syntax at the
> cost of introducing new syntax.
I think this would require all the work from a (2)/(3) type scheme, plus
lots more (adding syntax highlighting, modifying the parser).
What do you think? I'm willing to start working on the (2)/(3) scheme,
if it might be incorporated in Reinteract. But if these sorts of
changes to the language are inappropriate, I won't waste my time.
Thanks,
Robert
Robert Schroll
> 3) Locking is implemented, so if two worksheets are executing at the
> same time, they won't draw onto each other's figures. (Note: this makes
> execution hang with nested `with figure()` blocks. So don't do that.)
FYI, locking was not implemented correctly in this version. I've redone
that bit and uploaded the new version to the same place
(http://jfi.uchicago.edu/~rschroll/refigure/). While the new version is
doing better, it likely still has flaws. Let me know when you find them.
On the plus side, refigure2 no longer hangs when `with figure()` blocks
are nested. It may not deal with them correctly, though. On the
downside, you must now use `f = figure(locking=False)` to make a useable
figure outside of a with block. (As before, this syntax has the
potential to cause all sorts of problems, so only use it while putting
figures together. Don't keep it permanently.)
Thanks,
Robert