notebook() hangs in SXCE and OpenSolaris

[Brandon...@gmail.com]

Hello,

I'm very happy about the improved Solaris support. Until recently I
was running sage in a linux zone on my workstation, but unfortunately
modern linux version support for branded zones is lagging. I confess
I've also been using sagenb.org a bit (which will be running on a
T2000 soon?) I've tried to run the notebook in 3 different
environments (all by simply calling notebook() from the sage shell).

1: Updated OpenSolaris 2008.11 (snv_111a)

2: SXCE global zone (snv_109)

3: SXCE sparse root zone (same machine as (2))

On all of these, the startup of the notebook seems to hang for several
hours at this point:
"""
Retype new password:
Please login to the notebook with the username 'admin' and the above
password.
Password changed for user 'admin'.
**************************************************
* *
* Open your web browser to http://localhost:8000 *
* *
**************************************************
"""

After going to sleep and leaving the process running in the sparse
root zone, the next morning I found that the notebook server had
finally completed initialization and was able to view the notebook
from the sparse root zone (but not from anywhere else, despite some
routing efforts - probably my own fault).

Currently I'm trying this in opensolaris, and here is some output that
may be useful (from ps -a):

732 pts/5 S 0:00 bash ./sage
739 pts/5 S 0:00 bash /export/home/brandon/sage-3.4.1-beta-i86pc
+toolchain-SunOS/local/bin/sage-sage
1103 pts/5 S 0:00 python /export/home/brandon/sage-3.4.1-beta-
i86pc+toolchain-SunOS/local/bin/sage-cleaner
1104 pts/5 S 0:07 sage.bin /export/home/brandon/sage-3.4.1-beta-
i86pc+toolchain-SunOS/local/bin/sage-ipython -i
1138 pts/5 S 0:00 bash /export/home/brandon/sage-3.4.1-beta-i86pc
+toolchain-SunOS/sage -twistd --pidfile=sage_notebook/twistd.pid -ny
sage_notebook/twistedconf.tac
1145 pts/5 S 0:00 bash /export/home/brandon/sage-3.4.1-beta-i86pc
+toolchain-SunOS/local/bin/sage-sage -twistd --pidfile=sage_notebook/
twistd.pid -ny sage_notebook/twistedconf.tac
1157 pts/5 S 0:07 python /export/home/brandon/sage-3.4.1-beta-
i86pc+toolchain-SunOS/local/bin/twistd --pidfile=sage_notebook/
twistd.pid -ny sage_notebook/twistedconf.tac
11862 pts/5 S 0:00 bash
1720 pts/6 S 0:00 bash
1189 pts/7 S 0:00 bash /export/home/brandon/sage-3.4.1-beta-i86pc
+toolchain-SunOS/sage -python -u
1198 pts/7 S 0:00 bash /export/home/brandon/sage-3.4.1-beta-i86pc
+toolchain-SunOS/local/bin/sage-sage -python -u
1210 pts/7 S 0:02 python -u

[mabshoff]

On May 3, 6:48 pm, "Brandon.Bar...@gmail.com"
<Brandon.Bar...@gmail.com> wrote:
> Hello,

Hi,

> I'm very happy about the improved Solaris support.  Until recently I
> was running sage in a linux zone on my workstation, but unfortunately
> modern linux version support for branded zones is lagging.  I confess
> I've also been using sagenb.org a bit (which will be running on a
> T2000 soon?)

Probably, but the GMP/MPIR performance on the T2000 is pathetic per
"thread" and in effect only 8 or 16 out of 64 threads can use the
integer or FP units at a time, so scaling out kind of sucks, too.

>  I've tried to run the notebook in 3 different
> environments (all by simply calling notebook() from the sage shell).
>
> 1: Updated OpenSolaris 2008.11 (snv_111a)
>
> 2: SXCE global zone (snv_109)
>
> 3: SXCE sparse root zone (same machine as (2))
>
> On all of these, the startup of the notebook seems to hang for several
> hours at this point:

If the notebook takes forever to initialize this indicates a problem
with the source of randomness. I know too little about your setup to
determine if it is the system not providing enough entropy to the zone
or what else could be interfering here. libgcrypt (which is used to
create the the keys) used to have a bug that caused it to use way more
entropy then required and this caused a very long hang on Solaris 10.
We have since updated and at least on plain vanilla Solaris 10 (i.e.
no zone) the notebook start is instant. Any chance this gives you a
hint what is wrong? Does the problem also happen outside the zone?

Cheers,

Michael

[bbarker]

Hi Michael,

1 and 2 are both outside of zones (they are in the "global zone"), and
they also hang seemingly indefinitely (I haven't kept them running
long enough to prove me wrong yet, other than the one running in the
zone).
Since it is happening on fairly different systems - SXCE (workstation/
server) vs OpenSolaris 2008.11 (laptop), I'm inclined to think it
isn't anything I've done with configuration ... but its hard to say
for sure.
Later I could try running it in a fresh install of some sort, in
virtual box, if you have a specific version/configuration you'd like
me to test with.

I haven't tried this on an official Solaris 10 release yet, but it
seems strange that the bug would crop back up in newer releases of the
OS.

[mabshoff]

On May 3, 7:14 pm, bbarker <Brandon.Bar...@gmail.com> wrote:
> Hi Michael,

Hi Brandon,

> 1 and 2 are both outside of zones (they are in the "global zone"), and
> they also hang seemingly indefinitely (I haven't kept them running
> long enough to prove me wrong yet, other than the one running in the
> zone).

Well, it should be instant. Why don't you start it under gdb, CTRL-C
into it and get a backtrace. If it is trying to aquire entropy it
should be clear that this is the issue.

> Since it is happening on fairly different systems - SXCE (workstation/
> server) vs OpenSolaris 2008.11 (laptop), I'm inclined to think it
> isn't anything I've done with configuration ... but its hard to say
> for sure.

Ok.

> Later I could try running it in a fresh install of some sort, in
> virtual box, if you have a specific version/configuration you'd like
> me to test with.
>
> I haven't tried this on an official Solaris 10 release yet, but it
> seems strange that the bug would crop back up in newer releases of the
> OS.

Sure, I would be very surprised if this is the issue here since
Solaris is supposed to be compatible :)

Cheers,

Michael

[bbarker]

Some additional information:

Trying to exit the server during hang by pressing Ctrl-C repeatedly
will only yield the following (repeatedly):
^Cselect() error: Interrupted system call
^Cselect() error: Interrupted system call
^Cselect() error: Interrupted system call
^Cselect() error: Interrupted system call

Doing the same after running sage -dbg, notebook() ... will eventually
crash and fall back to the sage shell:

~/sage-3.4.1-beta-i86pc+toolchain-SunOS >./sage -gdb
----------------------------------------------------------------------
| Sage Version 3.4.1, Release Date: 2009-04-21 |
| Type notebook() for the GUI, and license() for information. |
| WARNING: This is a beta release that does not pass all tests. |
| For more info see http://wiki.sagemath.org/solaris |
----------------------------------------------------------------------
/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/local/bin/
sage-ipython
GNU gdb 6.3.50_2004-11-23-cvs
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and
you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "i386-pc-solaris2.11"...
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
warning: Lowest section in /lib/librt.so.1 is .dynamic at 00000074
Python 2.5.2 (r252:60911, Apr 20 2009, 12:01:15)
[GCC 4.3.2] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
sage: notebook()
The notebook files are stored in: /export/home/brandon/.sage//
sage_notebook

**************************************************
* *
* Open your web browser to http://localhost:8000 *
* *
**************************************************

Traceback (most recent call last):
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/twisted/application/app.py", line
614, in run
runApp(config)
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/twisted/scripts/twistd.py", line 23,
in runApp
_SomeApplicationRunner(config).run()
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/twisted/application/app.py", line
330, in run
self.application = self.createOrGetApplication()
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/twisted/application/app.py", line
416, in createOrGetApplication
application = getApplication(self.config, passphrase)
--- <exception caught here> ---
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/twisted/application/app.py", line
427, in getApplication
application = service.loadApplication(filename, style, passphrase)
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/twisted/application/service.py",
line 368, in loadApplication
application = sob.loadValueFromFile(filename, 'application',
passphrase)
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/twisted/persisted/sob.py", line 214,
in loadValueFromFile
exec fileObj in d, d
File "sage_notebook/twistedconf.tac", line 19, in <module>
worksheet.init_sage_prestart(twist.notebook.get_server(),
twist.notebook.get_ulimit())
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/sage/server/notebook/worksheet.py",
line 156, in init_sage_prestart
_a_sage = initialized_sage(server, ulimit)
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/sage/server/notebook/worksheet.py",
line 118, in initialized_sage
S._start(block_during_init=False)
File "/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/
local/lib/python2.5/site-packages/sage/interfaces/expect.py", line
471, in _start
raise RuntimeError, "Unable to start %s"%self.__name
exceptions.RuntimeError: Unable to start sage

Failed to load application: Unable to start sage

---------------------------------------------------------------------------
error Traceback (most recent call
last)

/export/home/brandon/.sage/<ipython console> in <module>()

/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/local/lib/
python2.5/site-packages/sage/server/notebook/notebook_object.pyc in
__call__(self, *args, **kwds)
163 """
164 def __call__(self, *args, **kwds):
--> 165 return self.notebook(*args, **kwds)
166
167 notebook = run_notebook.notebook_twisted

/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/local/lib/
python2.5/site-packages/sage/server/notebook/run_notebook.pyc in
notebook_twisted(self, directory, port, address, port_tries, secure,
reset, accounts, require_login, server_pool, ulimit, timeout,
open_viewer, sagetex_path, start_path, fork, quiet)
273 if open_viewer:
274 "Open viewer automatically isn't fully implemented.
You have to manually open your web browser to the above URL."
--> 275 return run(port)
276
277

/export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/local/lib/
python2.5/site-packages/sage/server/notebook/run_notebook.pyc in run
(port)
257 e = os.system(cmd)
258 if e == 256:
--> 259 raise socket.error
260 return True
261 # end of inner function run

error:

[bbarker]

I posted the above before reading your reply about running gdb ... so
just to be clear, notebook() crashes before Ctrl-C is pressed (but
only when running in sage -gdb).

On May 3, 10:36 pm, bbarker <Brandon.Bar...@gmail.com> wrote:
> Some additional information:
>
> Trying to exit the server during hang by pressing Ctrl-C repeatedly
> will only yield the following (repeatedly):
> ^Cselect() error: Interrupted system call
> ^Cselect() error: Interrupted system call
> ^Cselect() error: Interrupted system call
> ^Cselect() error: Interrupted system call
>
> Doing the same after running sage -dbg, notebook() ... will eventually
> crash and fall back to the sage shell:
>
> ~/sage-3.4.1-beta-i86pc+toolchain-SunOS >./sage -gdb
> ----------------------------------------------------------------------
> | Sage Version 3.4.1, Release Date: 2009-04-21                       |
> | Type notebook() for the GUI, and license() for information.        |
> | WARNING: This is a beta release that does not pass all tests.      |

> | For more info seehttp://wiki.sagemath.org/solaris                |

[mabshoff]

On May 3, 7:36 pm, bbarker <Brandon.Bar...@gmail.com> wrote:
> Some additional information:
>
> Trying to exit the server during hang by pressing Ctrl-C repeatedly
> will only yield the following (repeatedly):
> ^Cselect() error: Interrupted system call
> ^Cselect() error: Interrupted system call
> ^Cselect() error: Interrupted system call
> ^Cselect() error: Interrupted system call

Hmmm, select() - not sure what is going on here. Can you use 'truss'
and send some relevant excerpt from the logs?

> Doing the same after running sage -dbg, notebook() ... will eventually
> crash and fall back to the sage shell:
>
> ~/sage-3.4.1-beta-i86pc+toolchain-SunOS >./sage -gdb
> ----------------------------------------------------------------------
> | Sage Version 3.4.1, Release Date: 2009-04-21                       |
> | Type notebook() for the GUI, and license() for information.        |
> | WARNING: This is a beta release that does not pass all tests.      |

> | For more info seehttp://wiki.sagemath.org/solaris                |

> ----------------------------------------------------------------------
> /export/home/brandon/sage-3.4.1-beta-i86pc+toolchain-SunOS/local/bin/
> sage-ipython
> GNU gdb 6.3.50_2004-11-23-cvs
> Copyright 2004 Free Software Foundation, Inc.

<SNIP>

This is a little long in the tooth. I compile gdb 6.8 a couple days
ago and I was quite happy with it. It needed one trivial source code
fix to get it to compile on Solaris 10, but other than that it worked.

Cheers,

Michael