rebasing: I got it to work -- sage-4.4.3 under Cygwin

362 views
Skip to first unread message

William Stein

unread,
Jun 6, 2010, 5:37:14 AM6/6/10
to Mike Hansen, sage-windows
Hi Mike (cc: Sage Windows)

Here's the absurd way I got my sage-4.4.3 install to fork.
Anyways, I did the following:

(0) Read about rebasing: /usr/share/doc/Cygwin/rebase-3.0.1.README

(1) Shutdown all cygwin processes using the task manager

(2) Using command.exe run c:\cygwin\bin\ash then

/bin/rebaseall -v

which didn't help at all since it only rebase /usr/.

(3) Tried Gary Zablackis's 2005 script, but by hand:

cd /home/wstein/sage-4.4.3/
/bin/find -name *.dll > Sage-dlls.lst
/bin/rebaseall -v -T Sage-dlls.lst

(4) The above failed with an error rebasing libpython.dll. So I
deleted the libpython.dll line from Sage-dlls.lst, then
again did

/bin/rebaseall -v -T Sage-dlls.lst

(5) This appeared to work... but Sage still did *not* fork. The
following popped up when
trying to fork in Sage: local/lib/libcurvesntl.dll

So, I edited Sage-dlls.lst, and deleted everything *before* the
libcurvesntl.dll line. Then I did

/bin/rebaseall -v -T Sage-dlls.lst

and amazingly after this forking with Sage works, as illustrated in
the screenshots.

---

Here's my build notes to go from stock sage-4.4.3 to something that I
built under Cygwin and can fully use:

1. Put the following four spkg's in place of the ones that are in
spkg/standard:

boehm_gc-7.1.p4.spkg ecl-10.4.1.spkg libpng-1.2.35.p2.spkg maxima-5.21.0.spkg

I put these at http://sage.math.washington.edu/home/wstein/patches/sage-cygwin-4.4.3.tar
temporarily, for convenience.

2. don't worry about cvxopt, which doesn't build, and is the last
thing. The build fails, but we just ignore it.

3. Sage doesn't start. Using "cygcheck" on problemative dll, we find
that a fix is to copy

libntl.dll.a to libntl.dll

in SAGE_ROOT/local/lib/

4. Then sage starts up.

* But it can't fork. We have to rebase. See above.

---

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

Screen shot 2010-06-06 at 2.22.29 AM.png

William Stein

unread,
Jun 6, 2010, 11:51:44 PM6/6/10
to Mike Hansen, sage-windows
Hi,

I put a tarball of sage-4.4.3 built as below for cygwin, and the complete output
of "sage -t" on it at

http://sage.math.washington.edu/home/wstein/binaries/cygwin/

Thus if you want to try running sage-4.4.3 on Windows, this is an easy
way to try.

William

William Stein

unread,
Jun 7, 2010, 1:48:25 AM6/7/10
to sage-windows
Hi,

I've updated

http://trac.sagemath.org/sage_trac/wiki/CygwinPort

so it lists most of the problems with *using* (not building) Sage on
Cygwin that were uncovered by doctesting, along with individual
tickets for each problem.

-- William

William Stein

unread,
Jun 7, 2010, 2:01:06 AM6/7/10
to sage-windows
Hi,

One other remark: the notebook *does* just work with this build of
Sage-4.4.3 for Cygwin :-)

William

Screen shot 2010-06-06 at 11.00.40 PM.png

Andrew Trapani

unread,
Jun 11, 2010, 5:12:13 PM6/11/10
to sage-windows
Hi,
I downloaded this and followed the rebasing instructions.
It works.

I ran all of the quick little tests I wrote for sage-4.3.4 binary.
Everything I try works.

I will leave it on for a while. In sage-4.3.4, eventually I would get
forking problems and I wouldn't be able to get into any of my
worksheets, although it worked for the first little while.

Also when I exit and I try to exit the cygwin bash shell, it asks to
'terminate batch process', but this python process won't quit until I
kill it manually. This is the python process that starts when I run
sage.

Andrew
> > I put these athttp://sage.math.washington.edu/home/wstein/patches/sage-cygwin-4.4.3...

Andrew Trapani

unread,
Jun 14, 2010, 2:42:16 PM6/14/10
to sage-windows
Hi again,
I managed to reproduce the 'forking' problem.

In the notebook,
1. I loaded up four test worksheets, noted their individual PIDs, ran
all of the commands, and kept them in running state by hitting the
home button.
2. I came back about 1 and a half hour later and tried one at a time
to save and quit. When I tried to close the last one, the Cython demo,
the python process didn't close the any python processes. Turns out
time has nothing to do with it.
3. The notebook had gone back to the home page, but when I try to
load any worksheet it gives an error. On the command line is copy/
pasted below.
4. I manually close the PID associated with the Worksheet that failed
to quit. Of note, the python process was only using 740 KB when I
force quit it, while it was using over 100 MBs while running. So it
almost got there...
5. I can load worksheets again and continue working.

To reproduce simply: If I figure out how, I will attach the Cython
worksheet, if requested. Not sure if it has to do with Cython
specifically, but it seems to reproduce the bug every time using this
process:
1. Start the Cython worksheet and note PID.
2. Shift+Enter the Cython worksheet and let it finish processing
3. Hit HOME.
4. Go back into the Cython worksheet and Save and Quit.
5. Check to see if the PID has closed. Never does on my system.
6. Repeat 1-5 until the worksheet no longer loads.
7. Force quit all unclosed PIDs associated with the Cython worksheet.
8. Should be able to open up the worksheet and run it, as long as all
of the PIDs are closed.


Here is an attempt to copy/paste the worksheet:
sage: def slow_logistic(n,a,lamb = 3.82):
... """
... A slow logistic map.
... """
... q = a
... for i in range(n):
... q = lamb*q*(1-q)
... return q
sage: time a = slow_logistic(10^6,.5)
Time: CPU 3.72 s, Wall: 3.72 s
sage: %cython
sage: def cython_logistic(int n, float a, float lamb = 3.82):
... cdef float q
... q = a
... for i in range(n):
... q = lamb*q*(1-q)
... return q
sage: time a = cython_logistic(10^6,.5)
Time: CPU 0.03 s, Wall: 0.03 s
sage: 3.83 / 0.3
12.7666666666667
sage: def f_slow(x):
... return sin(x**2)
...
sage: def integrate_f_slow(a, b, N):
... s = 0
... dx = (b-a)/N
... for i in range(N):
... s += f_slow(a+i*dx)
... return s * dx
sage: %cython
sage: cdef extern from "math.h":
... double sin(double)
...
sage: cdef double f(double x):
... return sin(x*x)
...
sage: def integrate_f(double a, double b, int N):
... cdef int i
... cdef double s, dx
... s = 0
... dx = (b-a)/N
... for i in range(N):
... s += f(a+i*dx)
... return s * dx
sage: time integrate_f(0.0,1000.0,100000)
4.3718220505138667
Time: CPU 0.00 s, Wall: 0.00 s
sage: time integrate_f_slow(0.0, 1000.0, 100000)
4.37182205051387
Time: CPU 3.91 s, Wall: 3.94 s
sage: time integrate_f(0.0,1000.0,10000000)
0.62620775465178535
Time: CPU 0.89 s, Wall: 0.89 s
sage: time integrate_f(0.0,1000.0,100000000)
0.6261904582599962
Time: CPU 7.84 s, Wall: 7.86 s


Here is the full command line:

sage: notebook()
The notebook files are stored in: sage_notebook.sagenb
Removing stale pidfile sage_notebook.sagenb/twistd.pid
**************************************************
* *
* Open your web browser to http://localhost:8000 *
* *
**************************************************
Note: GNUTLS not available.
2010-06-14 09:28:10-0700 [-] Log opened.
2010-06-14 09:28:10-0700 [-] twistd 9.0.0 (/cygdrive/c/sage-4.4.3/
local/bin/python 2.6.4) starting up.
2010-06-14 09:28:10-0700 [-] reactor class:
twisted.internet.selectreactor.SelectReactor.
2010-06-14 09:28:10-0700 [-] twisted.web2.channel.http.HTTPFactory
starting on 8000
2010-06-14 09:28:10-0700 [-] Starting factory
<twisted.web2.channel.http.HTTPFactory instance at 0x7be48f4c>
2010-06-14 09:28:31-0700 [HTTPChannel,1,127.0.0.1] User 'atrapani'
logged in.
2010-06-14 11:10:54-0700 [HTTPChannel,40,127.0.0.1] ERROR initializing
compute process:
2010-06-14 11:10:54-0700 [HTTPChannel,40,127.0.0.1]
2010-06-14 11:10:54-0700 [HTTPChannel,40,127.0.0.1] Pexpect:
pty.fork() failed: out of pty devices
2010-06-14 11:10:54-0700 [HTTPChannel,40,127.0.0.1] Exception
rendering:
2010-06-14 11:10:54-0700 [HTTPChannel,40,127.0.0.1] Unhandled Error
Traceback (most recent call last):
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/internet/defer.py", line 181, in addCallbacks
self._runCallbacks()
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/internet/defer.py", line 323, in _runCallbacks
self.result = callback(self.result, *args, **kw)
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/internet/defer.py", line 284, in _continue
self.unpause()
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/internet/defer.py", line 280, in unpause
self._runCallbacks()
--- <exception caught here> ---
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/internet/defer.py", line 323, in _runCallbacks
self.result = callback(self.result, *args, **kw)
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/web2/server.py", line 296, in <lambda>
d.addCallback(lambda res, req: res.renderHTTP(req), self)
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/web2/resource.py", line 85, in renderHTTP
return method(request)
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/web2/resource.py", line 202, in http_GET
return super(Resource, self).http_GET(request)
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/twisted/web2/resource.py", line 128, in http_GET
return self.render(request)
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/sagenb-0.8-py2.6.egg/sagenb/notebook/twist.py", line 1534, in
render
self.worksheet.sage()
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/sagenb-0.8-py2.6.egg/sagenb/notebook/worksheet.py", line
2852, in sage
self.initialize_sage()
File "/cygdrive/c/sage-4.4.3/local/lib/python2.6/site-
packages/sagenb-0.8-py2.6.egg/sagenb/notebook/worksheet.py", line
2825, in initialize_sage
raise RuntimeError, msg
exceptions.RuntimeError: Pexpect: pty.fork() failed: out of
pty devices

Andrew Trapani

unread,
Jun 14, 2010, 2:52:12 PM6/14/10
to sage-windows
I quit and rebooted the notebook() command and it no longer has
problems killing this Worksheet.

Time may be an important factor after all.
> ...
>
> read more »

Stephen Loo

unread,
Jul 25, 2010, 11:26:53 PM7/25/10
to sage-windows
Hi,

I installed cywin as follow,

sage subshell$ uname -a
CYGWIN_NT-5.1 PC4482 1.7.5(0.225/5/3) 2010-04-12 19:07 i686 Cygwin

After untar sage-4.4.3 in my home directory, I start sage and get
following error

$ ./sage
----------------------------------------------------------------------
| Sage Version 4.4.3, Release Date: 2010-06-04 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/stephenloo/sage-4.4.3/local/bin/sage-ipython", line 18,
in <module
>
import IPython
File "/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/
IPython/__
init__.py", line 58, in <module>
__import__(name,glob,loc,[])
File "/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/
IPython/ip
struct.py", line 22, in <module>
from IPython.genutils import list2dict2
File "/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/
IPython/ge
nutils.py", line 59, in <module>
from IPython.external.path import path
File "/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/
IPython/ex
ternal/path.py", line 35, in <module>
import md5
File "/home/stephenloo/sage-4.4.3/local/lib/python/md5.py", line 10,
in <modul
e>
from hashlib import md5
File "/home/stephenloo/sage-4.4.3/local/lib/python/hashlib.py", line
136, in <
module>
md5 = __get_builtin_constructor('md5')
File "/home/stephenloo/sage-4.4.3/local/lib/python/hashlib.py", line
63, in __
get_builtin_constructor
import _md5
ImportError: No module named _md5

I have followed above guide to rebaseall. But the error still occurs.

Please help, thanks.

Stephen
> ...
>
> 閱讀更多 »

William Stein

unread,
Jul 25, 2010, 11:37:45 PM7/25/10
to sage-w...@googlegroups.com

I think this has nothing to do with rebasing. I think you need to
install some
Cygwin library packages, maybe openssl...?

William

> --
> You received this message because you are subscribed to the Google Groups "sage-windows" group.
> To post to this group, send email to sage-w...@googlegroups.com.
> To unsubscribe from this group, send email to sage-windows...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sage-windows?hl=en.

Stephen Loo

unread,
Jul 26, 2010, 12:19:22 AM7/26/10
to sage-windows
Thanks, I install openssl library and fix this problem, but another
problem appear as below:

---------------------------------------------------------------------------
ImportError Traceback (most recent call
last)

/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/IPython/
ipmaker.py
c in force_import(modname)
64 reload(sys.modules[modname])
65 else:
---> 66 __import__(modname)
67
68

/home/stephenloo/sage-4.4.3/local/bin/ipy_profile_sage.py in
<module>()
5 preparser(True)
6
----> 7 import sage.all_cmdline
8 sage.all_cmdline._init_cmdline(globals())
9

/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/sage/
all_cmdline.p
y in <module>()
12 try:
13
---> 14 from sage.all import *
15 from sage.calculus.predefined import x
16 preparser(on=True)

/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/sage/
all.py in <mo
dule>()
71
72 from sage.rings.all import *
---> 73 from sage.matrix.all import *
74
75 # This must come before Calculus -- it initializes the Pynac
library.


/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/sage/
matrix/all.py
in <module>()
----> 1
2
3 from matrix_space import MatrixSpace, is_MatrixSpace
4 from constructor import matrix, Matrix, random_matrix,
diagonal_matrix,
identity_matrix, block_matrix, block_diagonal_matrix, jordan_block,
zero_matrix
5 from matrix import is_Matrix
6 from berlekamp_massey import berlekamp_massey
7

/home/stephenloo/sage-4.4.3/local/lib/python2.6/site-packages/sage/
matrix/matrix
_space.pyc in <module>()
38 import matrix_modn_sparse
39
---> 40 import matrix_mod2_dense
41 #import matrix_mod2_sparse

42

ImportError: No such file or directory
Error importing ipy_profile_sage - perhaps you should run %upgrade?
WARNING: Loading of ipy_profile_sage failed.

sage:


Please help again.

Thanks,
Stephen


On 7月26日, 上午11時37分, William Stein <wst...@gmail.com> wrote:
> ...
>
> 閱讀更多 »

William Stein

unread,
Jan 19, 2011, 4:33:44 PM1/19/11
to sage-windows, andrew...@gmail.com
I'm just forwarding this to Sage-windows...

---------- Forwarded message ----------
From: Andrew Trapani <andrew...@gmail.com>
Date: Wed, Jan 19, 2011 at 1:29 PM
Subject: Re: rebasing: I got it to work -- sage-4.4.3 under Cygwin
To: William Stein <wst...@gmail.com>


I found a very easy to reproduce bug using the maxima function solve:
http://www.sagemath.org/doc/tutorial/tour_algebra.html

x = var('x')
solve(x^2 + 3*x + 2, x)

It starts up maxima, solves it, but then when I save and quit, the
maxima process is still running. A process leak.

Every time I reload this worksheet and re-run the solve function, it
starts a new maxima process which stays connected until I save/discard
and quit the worksheet.

I updated cygwin today, 1.7.1-1, and updated all the libraries,
restarted.

Another bug, potentially solved,
Before updating I tried it and I was getting the associated python
process with a worksheet wouldn't necessarily get killed when I save/
discard and quit from this worksheet. This would eventually cause me
to be unable to load any worksheets until I manually kill these python
processes.

After updating, I haven't been able to reproduce this worksheet
process leak. It used to be very easy to reproduce, by just opening
and closing worksheets randomly, eventually (a minute or two) it would
have a process leak. It may be the update to cygwin has solved, or
partially solved, the worksheet process leak.

Still looking forward to future work on windows Sage.

Andrew

> > I put these athttp://sage.math.washington.edu/home/wstein/patches/sage-cygwin-4.4.3...


> > temporarily, for convenience.
>
> > 2. don't worry about cvxopt, which doesn't build, and is the last
> > thing.  The build fails, but we just ignore it.
>
> > 3. Sage doesn't start.  Using "cygcheck" on problemative dll, we find
> > that a fix is to copy
>
> >   libntl.dll.a to libntl.dll
>
> > in SAGE_ROOT/local/lib/
>
> > 4. Then sage starts up.
>
> > * But it can't fork.  We have to rebase.  See above.
>
> > ---
>
> > --
> > William Stein
> > Professor of Mathematics
> > University of Washington
> >http://wstein.org
>
> --
> William Stein
> Professor of Mathematics

> University of Washingtonhttp://wstein.org

Dima Pasechnik

unread,
Dec 3, 2011, 12:53:42 AM12/3/11
to sage-w...@googlegroups.com, Mike Hansen


On Monday, 7 June 2010 11:51:44 UTC+8, William stein wrote:
Hi,

I put a tarball of sage-4.4.3 built as below for cygwin, and the complete output
of "sage -t" on it at

   http://sage.math.washington.edu/home/wstein/binaries/cygwin/

Thus if you want to try running sage-4.4.3 on Windows, this is an easy
way to try.

William

On Sun, Jun 6, 2010 at 2:37 AM, William Stein <wst...@gmail.com> wrote:
> Hi Mike (cc: Sage Windows)
>
> Here's the absurd way I got my sage-4.4.3 install to fork.
> Anyways, I did the following:
>
>  (0) Read about rebasing:   /usr/share/doc/Cygwin/rebase-3.0.1.README
>
>  (1) Shutdown all cygwin processes using the task manager
>
>  (2) Using command.exe run c:\cygwin\bin\ash then
>
> /bin/rebaseall -v
>
> which didn't help at all since it only rebase /usr/.
>
> (3) Tried Gary Zablackis's 2005 script, but by hand:
>
> cd /home/wstein/sage-4.4.3/
> /bin/find -name *.dll > Sage-dlls.lst
> /bin/rebaseall -v -T Sage-dlls.lst
>
> (4) The above failed with an error rebasing libpython.dll.  So I

this is apparently a file permission problem, taken care of in #12096
Reply all
Reply to author
Forward
0 new messages