instructions how to host sage notebook

[Ondrej Certik]

Hi,

is there some place with instructions how to host the sage notebook on
my own server, including security etc.? I found this so far:

http://wiki.sagemath.org/DanDrake/JustEnoughSageServer

I want to host a C++ finite element library that we are developing[1]
on the web, using Python (resp. Cython) for people to play with, so
that they don't have to download and fiddle with installation and
dependencies and setup. So I need a Python shell and images, 2D are
ok, but maybe I can even find ways to hook it to the 3D plotting.

I hate preparsing, e.g. 1 returning Integer(1), so I was living with a
feeling that sage notebook is too much hooked into sage, but I just
realized that if I turn the Python mode in, I can use it as a Python
shell, with tab-completion and everything. So that's really cool. And
I can host sympy or anything else I want in there the same way.

Another option is to use knoboo, but I think that currently sage
notebook is more usable[2].

Yet another option is to use ipython web frontend, but I am not sure
about it's current status, I'll ask on the ipython list.

Ondrej

[1] http://hpfem.math.unr.edu/hermes/
[2] http://groups.google.com/group/knoboo-devel/browse_thread/thread/7e034c11b6b46338

[Ondrej Certik]

> Yet another option is to use ipython web frontend, but I am not sure
> about it's current status, I'll ask on the ipython list.

http://lists.ipython.scipy.org/pipermail/ipython-dev/2009-February/004904.html

O.

[William Stein]

On Tue, Feb 10, 2009 at 11:46 AM, Ondrej Certik <ond...@certik.cz> wrote:
>
> Hi,
>
> is there some place with instructions how to host the sage notebook on
> my own server, including security etc.? I found this so far:
>
> http://wiki.sagemath.org/DanDrake/JustEnoughSageServer

Basically:

(1) Setup a virtual machine with your favorite linux distro. Use vmware
or VirtualBox or whatever.

(2) Install Sage into it.

(3) Type notebook? and read it.

There is also a nice document that Luiz Felipe Martins wrote and sent
me offlist.
I just forwarded it to you.

> I want to host a C++ finite element library that we are developing[1]
> on the web, using Python (resp. Cython) for people to play with, so
> that they don't have to download and fiddle with installation and
> dependencies and setup. So I need a Python shell and images, 2D are
> ok, but maybe I can even find ways to hook it to the 3D plotting.

>
> I hate preparsing, e.g. 1 returning Integer(1), so I was living with a
> feeling that sage notebook is too much hooked into sage, but I just
> realized that if I turn the Python mode in, I can use it as a Python
> shell, with tab-completion and everything. So that's really cool. And
> I can host sympy or anything else I want in there the same way.

It would also be trivial to slightly modify the sage notebook code so that
the *default* system at the top selected is Python instead of Sage.
You could also setup things so whatever libraries are relevant to finite
element modeling are preloaded.

William

[Ondrej Certik]

On Tue, Feb 10, 2009 at 12:57 PM, William Stein <wst...@gmail.com> wrote:
>
> On Tue, Feb 10, 2009 at 11:46 AM, Ondrej Certik <ond...@certik.cz> wrote:
>>
>> Hi,
>>
>> is there some place with instructions how to host the sage notebook on
>> my own server, including security etc.? I found this so far:
>>
>> http://wiki.sagemath.org/DanDrake/JustEnoughSageServer
>
> Basically:
>
> (1) Setup a virtual machine with your favorite linux distro. Use vmware
> or VirtualBox or whatever.
>
> (2) Install Sage into it.
>
> (3) Type notebook? and read it.
>
> There is also a nice document that Luiz Felipe Martins wrote and sent
> me offlist.
> I just forwarded it to you.

Thanks a low William. That document is exaclty what I was looking for.

>
>> I want to host a C++ finite element library that we are developing[1]
>> on the web, using Python (resp. Cython) for people to play with, so
>> that they don't have to download and fiddle with installation and
>> dependencies and setup. So I need a Python shell and images, 2D are
>> ok, but maybe I can even find ways to hook it to the 3D plotting.
>
>>
>> I hate preparsing, e.g. 1 returning Integer(1), so I was living with a
>> feeling that sage notebook is too much hooked into sage, but I just
>> realized that if I turn the Python mode in, I can use it as a Python
>> shell, with tab-completion and everything. So that's really cool. And
>> I can host sympy or anything else I want in there the same way.
>
> It would also be trivial to slightly modify the sage notebook code so that
> the *default* system at the top selected is Python instead of Sage.
> You could also setup things so whatever libraries are relevant to finite
> element modeling are preloaded.

Yes, I didn't realise it is that trivial. Thanks for the help, I'll
report back when we get this working.

Ondrej

[William Stein]

On Tue, Feb 10, 2009 at 1:02 PM, Ondrej Certik <ond...@certik.cz> wrote:
>
> On Tue, Feb 10, 2009 at 12:57 PM, William Stein <wst...@gmail.com> wrote:
>>
>> On Tue, Feb 10, 2009 at 11:46 AM, Ondrej Certik <ond...@certik.cz> wrote:
>>>
>>> Hi,
>>>
>>> is there some place with instructions how to host the sage notebook on
>>> my own server, including security etc.? I found this so far:
>>>
>>> http://wiki.sagemath.org/DanDrake/JustEnoughSageServer
>>
>> Basically:
>>
>> (1) Setup a virtual machine with your favorite linux distro. Use vmware
>> or VirtualBox or whatever.
>>
>> (2) Install Sage into it.
>>
>> (3) Type notebook? and read it.
>>
>> There is also a nice document that Luiz Felipe Martins wrote and sent
>> me offlist.
>> I just forwarded it to you.
>
> Thanks a low William. That document is exaclty what I was looking for.

You're welcome.

>>
>>> I want to host a C++ finite element library that we are developing[1]
>>> on the web, using Python (resp. Cython) for people to play with, so
>>> that they don't have to download and fiddle with installation and
>>> dependencies and setup. So I need a Python shell and images, 2D are
>>> ok, but maybe I can even find ways to hook it to the 3D plotting.
>>
>>>
>>> I hate preparsing, e.g. 1 returning Integer(1), so I was living with a
>>> feeling that sage notebook is too much hooked into sage, but I just
>>> realized that if I turn the Python mode in, I can use it as a Python
>>> shell, with tab-completion and everything. So that's really cool. And
>>> I can host sympy or anything else I want in there the same way.
>>
>> It would also be trivial to slightly modify the sage notebook code so that
>> the *default* system at the top selected is Python instead of Sage.
>> You could also setup things so whatever libraries are relevant to finite
>> element modeling are preloaded.
>
> Yes, I didn't realise it is that trivial. Thanks for the help, I'll
> report back when we get this working.

If you have trouble trying to make the above changes -- which is
entirely possible -- please complain.

William

[Ondrej Certik]

>> Yes, I didn't realise it is that trivial. Thanks for the help, I'll
>> report back when we get this working.
>
> If you have trouble trying to make the above changes -- which is
> entirely possible -- please complain.

I am not there yet, but I created this howto:

http://wiki.sagemath.org/SageVirtualBox

I didn't like that I had to install vmware as root and it was touching
my kernel, so I just installed virtualbox from ubuntu repositories and
that seems to work nice. It uses nat, but it forwards the 22 port, so
I can ssh into the image from the host, so that's all I need.

However, after I tried to also forward the port 8000 (e.g. last 3
commands from the howto a, so that I can access the notebook outside
the image, and I started to get:

sage: notebook(port=8000)
The notebook files are stored in: /home/ondrej/.sage//sage_notebook
**************************************************
* *
* Open your web browser to http://localhost:8000 *
* *
**************************************************
2009-02-10 16:41:12-0800 [-] Log opened.
2009-02-10 16:41:12-0800 [-] twistd 8.1.0
(/home/ondrej/ext/sage-3.2.3-Debian4.0-32bit-Intel-i686-Linux/local/bin/python
2.5.2) starting up
2009-02-10 16:41:12-0800 [-] reactor class: <class
'twisted.internet.selectreactor.SelectReactor'>
2009-02-10 16:41:12-0800 [-] twisted.web2.channel.http.HTTPFactory
starting on 8000
2009-02-10 16:41:12-0800 [-] Starting factory
<twisted.web2.channel.http.HTTPFactory instance at 0xab0236c>
http://localhost:8000/?startup_token=73e83ab5dad44c5f96b5fcd635b1933d:
No such file or directory

Notice the last line. Any ideas what is happening here? I googled a
bit and I think you suggested in some other email, that it is the
vmware problem (in my case virtualbox)?

Ondrej

[William Stein]

On Wed, Feb 11, 2009 at 7:54 AM, Ondrej Certik <ond...@certik.cz> wrote:
>
>>> Yes, I didn't realise it is that trivial. Thanks for the help, I'll
>>> report back when we get this working.
>>
>> If you have trouble trying to make the above changes -- which is
>> entirely possible -- please complain.
>
> I am not there yet, but I created this howto:
>
> http://wiki.sagemath.org/SageVirtualBox
>
> I didn't like that I had to install vmware as root and it was touching
> my kernel, so I just installed virtualbox from ubuntu repositories and
> that seems to work nice. It uses nat, but it forwards the 22 port, so
> I can ssh into the image from the host, so that's all I need.
>
> However, after I tried to also forward the port 8000 (e.g. last 3
> commands from the howto a, so that I can access the notebook outside
> the image, and I started to get:

That is not how the notebook works. By default the notebook
listens *only* on localhost, which means nobody else. This is a very
important security feature. If you want the notebook to listen to the
outside world, just do

sage: notebook(address="")

and the notebook will listen for connections on all interfaces.

> sage: notebook(port=8000)
> The notebook files are stored in: /home/ondrej/.sage//sage_notebook
> **************************************************
> * *
> * Open your web browser to http://localhost:8000 *
> * *
> **************************************************
> 2009-02-10 16:41:12-0800 [-] Log opened.
> 2009-02-10 16:41:12-0800 [-] twistd 8.1.0
> (/home/ondrej/ext/sage-3.2.3-Debian4.0-32bit-Intel-i686-Linux/local/bin/python
> 2.5.2) starting up
> 2009-02-10 16:41:12-0800 [-] reactor class: <class
> 'twisted.internet.selectreactor.SelectReactor'>
> 2009-02-10 16:41:12-0800 [-] twisted.web2.channel.http.HTTPFactory
> starting on 8000
> 2009-02-10 16:41:12-0800 [-] Starting factory
> <twisted.web2.channel.http.HTTPFactory instance at 0xab0236c>
> http://localhost:8000/?startup_token=73e83ab5dad44c5f96b5fcd635b1933d:
> No such file or directory
>
>
> Notice the last line. Any ideas what is happening here? I googled a
> bit and I think you suggested in some other email, that it is the
> vmware problem (in my case virtualbox)?
>
> Ondrej
>
> >
>

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

[Ondrej Certik]

On Wed, Feb 11, 2009 at 9:15 AM, William Stein <wst...@gmail.com> wrote:
>
> On Wed, Feb 11, 2009 at 7:54 AM, Ondrej Certik <ond...@certik.cz> wrote:
>>
>>>> Yes, I didn't realise it is that trivial. Thanks for the help, I'll
>>>> report back when we get this working.
>>>
>>> If you have trouble trying to make the above changes -- which is
>>> entirely possible -- please complain.
>>
>> I am not there yet, but I created this howto:
>>
>> http://wiki.sagemath.org/SageVirtualBox
>>
>> I didn't like that I had to install vmware as root and it was touching
>> my kernel, so I just installed virtualbox from ubuntu repositories and
>> that seems to work nice. It uses nat, but it forwards the 22 port, so
>> I can ssh into the image from the host, so that's all I need.
>>
>> However, after I tried to also forward the port 8000 (e.g. last 3
>> commands from the howto a, so that I can access the notebook outside
>> the image, and I started to get:
>
> That is not how the notebook works. By default the notebook
> listens *only* on localhost, which means nobody else. This is a very
> important security feature. If you want the notebook to listen to the
> outside world, just do
>
> sage: notebook(address="")
>
> and the notebook will listen for connections on all interfaces.

Oops, you are right, I forgot about this. Now everything works, I
updated the howto:

http://wiki.sagemath.org/SageVirtualBox

Ondrej

[mabshoff]

On Feb 11, 9:22 am, Ondrej Certik <ond...@certik.cz> wrote:
> On Wed, Feb 11, 2009 at 9:15 AM, William Stein <wst...@gmail.com> wrote:

<SNIP>

> > That is not how the notebook works.    By default the notebook
> > listens *only* on localhost, which means nobody else.  This is a very
> > important security feature.  If you want the notebook to listen to the
> > outside world, just do
>
> >  sage: notebook(address="")
>
> > and the notebook will listen for connections on all interfaces.
>
> Oops, you are right, I forgot about this. Now everything works, I
> updated the howto:
>
> http://wiki.sagemath.org/SageVirtualBox

But do you get that oddity with the token? Because that if you can
reproducibly hit that problem you are the first one who has a chance
to debug this as someone we know with a technical understanding of
Python.

> Ondrej

Cheers,

Michael

[Ondrej Certik]

I do. Can you tell me more information about it? I am now setting up
the users and permissions and updating the how to on the wiki. Then I
can look at it.

Btw, I am behind a university firewall that doesn't allow me to use
IRC. Does anyone remember the iptables or route rules to forward the
IRC connection through my server? I know I can setup ssh port
forwarding, but then I would have to log to localhost and fiddle with
the IRC client, so I'd prefer to forward all outgoing connections to
the IRC port via my own external server. Don't loose time with this, I
can figure it out after I study it, but maybe you know it immediately.

Ondrej

[William Stein]

I just ssh to sage.math and type irssi.

William

[Ondrej Certik]

>> Btw, I am behind a university firewall that doesn't allow me to use
>> IRC. Does anyone remember the iptables or route rules to forward the
>> IRC connection through my server? I know I can setup ssh port
>> forwarding, but then I would have to log to localhost and fiddle with
>> the IRC client, so I'd prefer to forward all outgoing connections to
>> the IRC port via my own external server. Don't loose time with this, I
>> can figure it out after I study it, but maybe you know it immediately.
>
> I just ssh to sage.math and type irssi.

Nice, that works, thanks.

>>> It would also be trivial to slightly modify the sage notebook code so that
>>> the *default* system at the top selected is Python instead of Sage.
>>> You could also setup things so whatever libraries are relevant to finite
>>> element modeling are preloaded.
>>

>> Yes, I didn't realise it is that trivial. Thanks for the help, I'll
>> report back when we get this working.
>
> If you have trouble trying to make the above changes -- which is
> entirely possible -- please complain.

Indeed, it was a oneliner:

diff -r b0aa7ef45b3c sage/server/notebook/user_conf.py
--- a/sage/server/notebook/user_conf.py Mon Jan 05 23:03:45 2009 -0800
+++ b/sage/server/notebook/user_conf.py Wed Feb 11 08:58:59 2009 -0800
@@ -4,7 +4,7 @@
import conf

defaults = {'max_history_length':500,
- 'default_system':'sage',
+ 'default_system':'python',
'autosave_interval':3*60, # (in seconds)
'default_pretty_print': False
}

Is there some better way, so that I don't have to modify Sage sources?
It looks like some user configuration, but I haven't yet figured out
how to use it the way it is intended.

Also I want to control what is imported and what is not (in the empty
worksheet), looking into it now.

Ondrej

[Nick Alexander]

> Indeed, it was a oneliner:
>
> diff -r b0aa7ef45b3c sage/server/notebook/user_conf.py
> --- a/sage/server/notebook/user_conf.py Mon Jan 05 23:03:45 2009 -0800
> +++ b/sage/server/notebook/user_conf.py Wed Feb 11 08:58:59 2009 -0800
> @@ -4,7 +4,7 @@
> import conf
>
> defaults = {'max_history_length':500,
> - 'default_system':'sage',
> + 'default_system':'python',
> 'autosave_interval':3*60, # (in seconds)
> 'default_pretty_print': False
> }
>
>
> Is there some better way, so that I don't have to modify Sage sources?

You could monkey patch it from init.sage (you don't want to see my
init.sage):

import sage.server.notebook.user_conf
sage.server.notebook.user_conf.defaults['default_system'] = 'python'

Untested!
Nick

[Dan Drake]

On Wed, 11 Feb 2009 at 07:54AM -0800, Ondrej Certik wrote:
> I am not there yet, but I created this howto:
>
> http://wiki.sagemath.org/SageVirtualBox

I mentioned this on IRC, but so that anyone who is looking through list
archives knows, I wrote a similar guide:

http://wiki.sagemath.org/DanDrake/JustEnoughSageServer

Dan

--
--- Dan Drake <dr...@kaist.edu>
----- KAIST Department of Mathematical Sciences
------- http://mathsci.kaist.ac.kr/~drake

[Ondrej Certik]

Thanks for the tip, I'll try it.

So I got our software running in Sage over the web! Here is a very
simple example:

http://nb.hpfem.org:8000/home/pub/5/

It uses a mesh, that I uploaded to the worksheet, and then it loads it
into our C++ code. That's it. I had to disable more interesting stuff,
because it was depending on umfpack, but I had some problems
installing it in Sage, and it is not needed anyway, as I want to use
scipy.sparse for solving stuff, or pysparse, or pets4py or slepc4py,
but I want to call it from Python, not from C++.
So I need to work on this more to get everything polished, but now I
can see that it will work.

Great job with the notebook, it's exactly what I need!

I have couple questions:

1) how can I change what gets imported by default in an empty
worksheet? I use Python mode, but sage still gets imported and it
polutes the namespace. :)

2) How can I get better tracebacks when something goes wrong? Example:

import hermes2d
m = hermes2d.Mesh()
m.load()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ondrej/nbfiles/worksheets/ondrej/0/code/7.py", line 6, in <module>
print _support_.syseval(python, ur'''m.load()''',
'/home/ondrej/nbfiles/worksheets/ondrej/0/cells/12')
File "/home/ondrej/ext/sage-3.2.3-Debian4.0-32bit-Intel-i686-Linux/local/lib/python2.5/site-packages/sage/server/support.py",
line 323, in syseval
return system.eval(cmd)
TypeError: eval() takes at least 3 arguments (2 given)

The m.load() expects one parameter, I didn't give it any, but it is
not apparent from the tracebakc what went wrong.

3) I want to use matplotlib for plotting, or maybe some other library,
in any case the result will be a png image. Here are couple examples
how it can be used in the notebook:

http://nb.hpfem.org:8000/home/pub/2/
http://nb.hpfem.org:8000/home/pub/3/
http://nb.hpfem.org:8000/home/pub/4/

Slide to the bottom of the page to see the image. However, I noticed
with this approach, that if I update the plotting commands and
regenerate the .png image, the result gets messed up sometimes and I
had to stop the worksheet, restart it and reevaluate all cells again,
then it works. Is this a known problem? What is the workaround?

I am impressed how usable the notebook is.

Ondrej

[William Stein]

On Sun, Feb 15, 2009 at 3:33 PM, Ondrej Certik <ond...@certik.cz> wrote:
>
> On Wed, Feb 11, 2009 at 7:13 PM, Nick Alexander <ncale...@gmail.com> wrote:
>>
>>> Indeed, it was a oneliner:
>>>
>>> diff -r b0aa7ef45b3c sage/server/notebook/user_conf.py
>>> --- a/sage/server/notebook/user_conf.py Mon Jan 05 23:03:45 2009 -0800
>>> +++ b/sage/server/notebook/user_conf.py Wed Feb 11 08:58:59 2009 -0800
>>> @@ -4,7 +4,7 @@
>>> import conf
>>>
>>> defaults = {'max_history_length':500,
>>> - 'default_system':'sage',
>>> + 'default_system':'python',
>>> 'autosave_interval':3*60, # (in seconds)
>>> 'default_pretty_print': False
>>> }
>>>
>>>
>>> Is there some better way, so that I don't have to modify Sage sources?
>>
>> You could monkey patch it from init.sage (you don't want to see my
>> init.sage):
>>
>> import sage.server.notebook.user_conf
>> sage.server.notebook.user_conf.defaults['default_system'] = 'python'
>
> Thanks for the tip, I'll try it.

Doing that isn't evil either. Well, at least I can tell you that it matches
with the author's (me) original intentions for how

sage.server.notebook.user_conf.defaults

might be used.

>
> So I got our software running in Sage over the web! Here is a very
> simple example:
>
> http://nb.hpfem.org:8000/home/pub/5/
>
> It uses a mesh, that I uploaded to the worksheet, and then it loads it
> into our C++ code. That's it. I had to disable more interesting stuff,
> because it was depending on umfpack, but I had some problems
> installing it in Sage, and it is not needed anyway, as I want to use
> scipy.sparse for solving stuff, or pysparse, or pets4py or slepc4py,
> but I want to call it from Python, not from C++.
> So I need to work on this more to get everything polished, but now I
> can see that it will work.
>
> Great job with the notebook, it's exactly what I need!

Cool. By the way, I made an account on your server above and tried
the above example, and all went well.

>
> I have couple questions:
>
> 1) how can I change what gets imported by default in an empty
> worksheet? I use Python mode, but sage still gets imported and it
> polutes the namespace. :)

Hmmm... First, this can definitely be fixed.
[takes 10 minutes]
OK, I wrote a small patch that modifies the sage notebook so that it
starts up all worksheet processes in Pure Python mode without
importing the sage library. It's a 2-line modification to
notebook/worksheet.py. See attached. After applying this the
individual worksheets will actually startup a lot faster and be more
lightweight, but of course no specific sage functionality is available
unless one does "import sage.all". Tab completion and introspection
still works fine though without importing sage.all.
gh

> 2) How can I get better tracebacks when something goes wrong? Example:
>
> import hermes2d
> m = hermes2d.Mesh()
> m.load()
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/home/ondrej/nbfiles/worksheets/ondrej/0/code/7.py", line 6, in <module>
> print _support_.syseval(python, ur'''m.load()''',
> '/home/ondrej/nbfiles/worksheets/ondrej/0/cells/12')
> File "/home/ondrej/ext/sage-3.2.3-Debian4.0-32bit-Intel-i686-Linux/local/lib/python2.5/site-packages/sage/server/support.py",
> line 323, in syseval
> return system.eval(cmd)
> TypeError: eval() takes at least 3 arguments (2 given)
>
>
> The m.load() expects one parameter, I didn't give it any, but it is
> not apparent from the tracebakc what went wrong.

That's a bug in the sage notebook when using another mode (e.g.,
Python) and TypeError is raised running some code. It's interesting
that nobody seems to have reported this before. I've reported this as
trac #5282:

http://trac.sagemath.org/sage_trac/ticket/5282

I have posted a patch there that fixes this bug. You should review it
so that it gets into sage asap.

> 3) I want to use matplotlib for plotting, or maybe some other library,
> in any case the result will be a png image. Here are couple examples
> how it can be used in the notebook:
>
> http://nb.hpfem.org:8000/home/pub/2/
> http://nb.hpfem.org:8000/home/pub/3/
> http://nb.hpfem.org:8000/home/pub/4/
>
> Slide to the bottom of the page to see the image. However, I noticed
> with this approach, that if I update the plotting commands and
> regenerate the .png image, the result gets messed up sometimes and I
> had to stop the worksheet, restart it and reevaluate all cells again,
> then it works. Is this a known problem? What is the workaround?

In sage mode, I've never seen problems like this. Weird. It should
just work. Maybe this is a problem with using python mode, and the
directory
not being properly changed or something. What happens if you make
sure all the images have *different* filenames (say something random)?

> I am impressed how usable the notebook is.

Thanks.

-- William

[Jason Grout]

William Stein wrote:
>
>> 3) I want to use matplotlib for plotting, or maybe some other library,
>> in any case the result will be a png image. Here are couple examples
>> how it can be used in the notebook:
>>
>> http://nb.hpfem.org:8000/home/pub/2/
>> http://nb.hpfem.org:8000/home/pub/3/
>> http://nb.hpfem.org:8000/home/pub/4/
>>
>> Slide to the bottom of the page to see the image. However, I noticed
>> with this approach, that if I update the plotting commands and
>> regenerate the .png image, the result gets messed up sometimes and I
>> had to stop the worksheet, restart it and reevaluate all cells again,
>> then it works. Is this a known problem? What is the workaround?
>
> In sage mode, I've never seen problems like this. Weird. It should
> just work. Maybe this is a problem with using python mode, and the
> directory
> not being properly changed or something. What happens if you make
> sure all the images have *different* filenames (say something random)?

Is this problem characterized by:

1. Create a plot in a cell

2. change the plotting code, evaluate the cell. Then maybe delete the
cell. Save the worksheet and come back to it later.

3. Run a different plotting comand, or update a cell. When you evaluate
the first time, the graphics are old graphics from other cells.
Evaluate All refreshes the graphics, though.

or something like the above? It seems like the root problem is
something like #5177, where the old image stays around for longer than
it should.

Jason

[Ondrej Certik]

Hi,

so I got back to it again, thanks for all the support I got here and
thanks William for the patches. Here are my replies:

On Wed, Feb 11, 2009 at 10:22 PM, Dan Drake <dr...@kaist.edu> wrote:
> On Wed, 11 Feb 2009 at 07:54AM -0800, Ondrej Certik wrote:
>> I am not there yet, but I created this howto:
>>
>> http://wiki.sagemath.org/SageVirtualBox
>
> I mentioned this on IRC, but so that anyone who is looking through list
> archives knows, I wrote a similar guide:
>
> http://wiki.sagemath.org/DanDrake/JustEnoughSageServer

Thanks, I added a link to your wiki to:

http://wiki.sagemath.org/SageVirtualBox

On Wed, Feb 11, 2009 at 10:41 AM, William Stein <wst...@gmail.com> wrote:

>
> On Wed, Feb 11, 2009 at 10:18 AM, Ondrej Certik <ond...@certik.cz> wrote:
>> Btw, I am behind a university firewall that doesn't allow me to use
>> IRC. Does anyone remember the iptables or route rules to forward the
>> IRC connection through my server? I know I can setup ssh port
>> forwarding, but then I would have to log to localhost and fiddle with
>> the IRC client, so I'd prefer to forward all outgoing connections to
>> the IRC port via my own external server. Don't loose time with this, I
>> can figure it out after I study it, but maybe you know it immediately.
>
> I just ssh to sage.math and type irssi.

That works like a charm, I am now using it too. Thanks!

On Wed, Feb 11, 2009 at 9:36 AM, mabshoff <mabs...@googlemail.com> wrote:
>
>
>
> On Feb 11, 9:22 am, Ondrej Certik <ond...@certik.cz> wrote:
>> On Wed, Feb 11, 2009 at 9:15 AM, William Stein <wst...@gmail.com> wrote:
>
> <SNIP>
>
>> > That is not how the notebook works. By default the notebook
>> > listens *only* on localhost, which means nobody else. This is a very
>> > important security feature. If you want the notebook to listen to the
>> > outside world, just do
>>
>> > sage: notebook(address="")
>>
>> > and the notebook will listen for connections on all interfaces.
>>
>> Oops, you are right, I forgot about this. Now everything works, I
>> updated the howto:
>>
>> http://wiki.sagemath.org/SageVirtualBox
>
> But do you get that oddity with the token? Because that if you can
> reproducibly hit that problem you are the first one who has a chance
> to debug this as someone we know with a technical understanding of
> Python.

Unfortunately I stopped getting it. Well, it's fortunate for me that
it works, unfortunate that I can't debug it easily.

>>> Slide to the bottom of the page to see the image. However, I noticed
>>> with this approach, that if I update the plotting commands and
>>> regenerate the .png image, the result gets messed up sometimes and I
>>> had to stop the worksheet, restart it and reevaluate all cells again,
>>> then it works. Is this a known problem? What is the workaround?
>>
>> In sage mode, I've never seen problems like this. Weird. It should
>> just work. Maybe this is a problem with using python mode, and the
>> directory
>> not being properly changed or something. What happens if you make
>> sure all the images have *different* filenames (say something random)?
>
>
> Is this problem characterized by:
>
> 1. Create a plot in a cell
>
> 2. change the plotting code, evaluate the cell. Then maybe delete the
> cell. Save the worksheet and come back to it later.
>
> 3. Run a different plotting comand, or update a cell. When you evaluate
> the first time, the graphics are old graphics from other cells.
> Evaluate All refreshes the graphics, though.
>
> or something like the above? It seems like the root problem is
> something like #5177, where the old image stays around for longer than
> it should.

Thanks for all this help. I think it was my fault. Here is how to reproduce it:

1) create a new worksheet (no matter which mode), for example at sagenb.org

2)

import pylab
pylab.plot([1, 2, 3], [1, 2, 1])
pylab.savefig("a.png")

# it plots a plot, all a-ok

3) go back and change [1, 2, 1] to [1, 2, 3], reevaluate, it will
produce this messed up plot:

http://www.sagenb.org/home/pub/281/

And here is how to fix it:

1) add pylab.cla() right after "import pylab" and rerun it.

So it's a matplotlib feature. Sorry about the confusion.

>> 1) how can I change what gets imported by default in an empty
>> worksheet? I use Python mode, but sage still gets imported and it
>> polutes the namespace. :)
>
> Hmmm... First, this can definitely be fixed.
> [takes 10 minutes]
> OK, I wrote a small patch that modifies the sage notebook so that it
> starts up all worksheet processes in Pure Python mode without
> importing the sage library. It's a 2-line modification to
> notebook/worksheet.py. See attached. After applying this the
> individual worksheets will actually startup a lot faster and be more
> lightweight, but of course no specific sage functionality is available
> unless one does "import sage.all". Tab completion and introspection
> still works fine though without importing sage.all.
> gh

Thanks William for the patch, it does exactly what I wanted. I updated
my server, now I have a better address:

http://nb.hpfem.org/

If yo go there, you can see that I customized it a bit --- when I find
more time, I'll probably change colors or something to make it more
different, so that people don't confuse my local modifications with
the default sage notebook.

Is it ok with you if I rename it from Sage notebook to Hermes notebook
and only leave a link to sagemath.org at the front page and in the
help? Well, I know it's opensource, so I could do that, but I meant if
you have some long term strategy how you would like people to use &
install the modified sage notebooks.

Some other things I have been doing with the notebook:

Since my research topis is electronic structure calculations, I took
the most famous electronic structure program abinit.org, which is in
fortran, I replaced their build system with cmake, installed it into
Sage, wrote Python wrappers that produce input files and read output
files from it, installed it to Sage and here is how it looks like:

http://nb.hpfem.org/home/pub/7/

Now I need to play with graphics, e.g. create plots from all the
programs that I am interested in and then it will start to rock. Then
I'll try to interface all the other programs too.

I am surprised noone did that before (I mean except Sage for some math
programs), but it's so obvious thing to do. I want all my programs and
everything I use to be easily available in the notebook, so that I can
play with it whenever I am and show it live during
teaching/presentations. And also invite other people to play with it
too, if it's just a few mouse clicks. Well, I just wasn't thinking
this way before either, but now it's clear to me that this is the way
to go.

Ondrej

[William Stein]

> 1) create a new worksheet (no matter which mode), for example at sagenb.org
>
> 2)
>
> import pylab
> pylab.plot([1, 2, 3], [1, 2, 1])
> pylab.savefig("a.png")
>
> # it plots a plot, all a-ok
>
> 3) go back and change [1, 2, 1] to [1, 2, 3], reevaluate, it will
> produce this messed up plot:
>
> http://www.sagenb.org/home/pub/281/
>
>
> And here is how to fix it:
>
> 1) add pylab.cla() right after "import pylab" and rerun it.
>
> So it's a matplotlib feature. Sorry about the confusion.

I've been frequently very confused by this feature of
matplotlib/MATLAB. It's less confusing when there is an interactive
window that pops up -- when rendering to an off-screen buffer it is
often easy to forget to clear things.

>
>
>>> 1) how can I change what gets imported by default in an empty
>>> worksheet? I use Python mode, but sage still gets imported and it
>>> polutes the namespace. :)
>>
>> Hmmm... First, this can definitely be fixed.
>> [takes 10 minutes]
>> OK, I wrote a small patch that modifies the sage notebook so that it
>> starts up all worksheet processes in Pure Python mode without
>> importing the sage library.  It's a 2-line modification to
>> notebook/worksheet.py.  See attached.  After applying this the
>> individual worksheets will actually startup a lot faster and be more
>> lightweight, but of course no specific sage functionality is available
>> unless one does  "import sage.all".  Tab completion and introspection
>> still works fine though without importing sage.all.
>> gh
>
> Thanks William for the patch, it does exactly what I wanted. I updated
> my server, now I have a better address:
>
> http://nb.hpfem.org/
>
> If yo go there, you can see that I customized it a bit --- when I find
> more time, I'll probably change colors or something to make it more
> different, so that people don't confuse my local modifications with
> the default sage notebook.

Very nice!

> Is it ok with you if I rename it from Sage notebook to Hermes notebook
> and only leave a link to sagemath.org at the front page and in the
> help? Well, I know it's opensource, so I could do that, but I meant if
> you have some long term strategy how you would like people to use &
> install the modified sage notebooks.

It is *definitely* OK. You have my official written permission to
rebrand your server "Hermes notebook".

> Some other things I have been doing with the notebook:
>
> Since my research topis is electronic structure calculations, I took
> the most famous electronic structure program abinit.org, which is in
> fortran, I replaced their build system with cmake, installed it into
> Sage, wrote Python wrappers that produce input files and read output
> files from it, installed it to Sage and here is how it looks like:
>
> http://nb.hpfem.org/home/pub/7/
>
> Now I need to play with graphics, e.g. create plots from all the
> programs that I am interested in and then it will start to rock. Then
> I'll try to interface all the other programs too.
>
> I am surprised noone did that before (I mean except Sage for some math
> programs), but it's so obvious thing to do. I want all my programs and
> everything I use to be easily available in the notebook, so that I can
> play with it whenever I am and show it live during
> teaching/presentations. And also invite other people to play with it
> too, if it's just a few mouse clicks. Well, I just wasn't thinking
> this way before either, but now it's clear to me that this is the way
> to go.
>
> Ondrej

+1

It's very nice to have everything under one roof.

-- William

[Ondrej Certik]

>> Is it ok with you if I rename it from Sage notebook to Hermes notebook
>> and only leave a link to sagemath.org at the front page and in the
>> help? Well, I know it's opensource, so I could do that, but I meant if
>> you have some long term strategy how you would like people to use &
>> install the modified sage notebooks.
>
> It is *definitely* OK.   You have my official written permission to
> rebrand your server "Hermes notebook".

Thanks! :)

Ondrej

[Jason Grout]

Ondrej Certik wrote:
>
> Since my research topis is electronic structure calculations, I took
> the most famous electronic structure program abinit.org, which is in
> fortran, I replaced their build system with cmake, installed it into
> Sage, wrote Python wrappers that produce input files and read output
> files from it, installed it to Sage and here is how it looks like:
>
> http://nb.hpfem.org/home/pub/7/
>
> Now I need to play with graphics, e.g. create plots from all the
> programs that I am interested in and then it will start to rock. Then
> I'll try to interface all the other programs too.

I'm curious: is there any chance that you'll use JMOL as a 3d molecular
viewer? I've been curious about any uses in Sage of JMOL for its
original intended purpose.

Please note that the default menu for JMOL changed in 3.3, but that it's
very easy to get the old menu back. The old menu showed lots of stuff
that is very nice to have if you are actually looking at atoms and
molecules.

Thanks,

Jason

[Brian Granger]

> I'm curious: is there any chance that you'll use JMOL as a 3d molecular
> viewer?  I've been curious about any uses in Sage of JMOL for its
> original intended purpose.

This is a great idea. I am working with Ondrej on this project and
there is not reason we can't do this. Currently, we don't support any
of the file formats that JMOL can read, but that will hopefully change
soon.

Brian