Re: Sage 4.6 'make' failing due to missing Python Module- Very Frustrating!

[Volker Braun]

Can you try to compile a non-ancient version of Sage? I don't remember when we updated from python-2.6.4.p9 but it was a while ago.



On Monday, May 20, 2013 9:54:41 PM UTC+1, ghm...@mst.edu wrote:

math module OK
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/george/APCG/apcg/mcp/sage/local/lib/python/hashlib.py", line 138, in <module>
    sha224 = __get_builtin_constructor('sha224')
  File "/home/george/APCG/apcg/mcp/sage/local/lib/python/hashlib.py", line 66, in __get_builtin_constructor
    import _sha256
ImportError: No module named _sha256
hashlib module failed to import

real        2m6.118s
user        1m20.713s
sys        0m7.724s
sage: An error occurred while installing python-2.6.4.p9
Please email sage-devel http://groups.google.com/group/sage-devel
explaining the problem and send the relevant part of
of /home/george/APCG/apcg/mcp/sage/install.log.  Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9 and type 'make check' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
(cd '/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9' && '/home/george/APCG/apcg/mcp/sage/sage' -sh)
When you are done debugging, you can type "exit" to leave the
subshell.
make[1]: *** [installed/python-2.6.4.p9] Error 1
make[1]: Leaving directory `/home/george/APCG/apcg/mcp/sage/spkg'

real        2m14.313s
user        1m23.997s
sys        0m8.637s
Error building Sage.
./sage -docbuild all html  2>&1 | tee -a dochtml.log

I have installed multiple packages, updated Python, and any number of other things.  Unfortunately, I must use this version of Sage and would like some help with this.  Please explain why this is so difficult and why the error messages are astoundingly unhelpful.  Thank you, any help is appreciated!

[François Bissey]

On Mon, 20 May 2013 14:25:36 Volker Braun wrote:
> Can you try to compile a non-ancient version of Sage? I don't remember when
> we updated from python-2.6.4.p9 but it was a while ago.
>

We moved away from it at sage 5.0 - I should know. Early 2012, late 2011?

[ghm...@mst.edu]

I'm going to attempt to use the newest version of Sage, but say, just in case, that I will be forced to use 4.6. Is there a way to debug this message or get some help on the issue?

[François Bissey]

I'd say you don't have libcrypto (part of openssl) installed on your system
as an educated guess.

Francois

[leif]

ghm...@mst.edu wrote:
> Hey Sage Devs!
>
> I have encountered a problem during my attempt to build Sage for the first time. As the program requests, I attempt to 'make' Sage and most of it seems to work fine until it attempts to test Python. At this point, I encounter the error below:

>
> math module OK
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> File "/home/george/APCG/apcg/mcp/sage/local/lib/python/hashlib.py", line 138, in <module>
> sha224 = __get_builtin_constructor('sha224')
> File "/home/george/APCG/apcg/mcp/sage/local/lib/python/hashlib.py", line 66, in __get_builtin_constructor
> import _sha256
> ImportError: No module named _sha256
> hashlib module failed to import

Install OpenSSL? (There's even a Sage package for that.)

And/or if you're on Debian, Ubuntu, or one of their derivatives, try the
Python 2.6.4.p11 spkg, which is still available e.g. here:

https://spkg-upload.googlecode.com/files/python-2.6.4.p11.spkg

> I have installed multiple packages, updated Python, and any number of other things. Unfortunately, I must use this version of Sage and would like some help with this. Please explain why this is so difficult and why the error messages are astoundingly unhelpful. Thank you, any help is appreciated!

Next time, how about googling the error message?

And

> Describe your computer, operating system, etc.

-leif

--
() The ASCII Ribbon Campaign
/\ Help Cure HTML E-Mail

[leif]

ghm...@mst.edu wrote:
> I'm going to attempt to use the newest version of Sage, but say, just in case, that I will be forced to use 4.6. Is there a way to debug this message or get some help on the issue?

Inspect the spkg's build log: spkg/logs/python-*.log

(There you'll presumably also find "Failed to find the necessary bits to
build these modules: ...".)

[ghm...@mst.edu]

I spent hours Googling the error message before posting here. And tried anything even remotely related to my issue, to no avail.

[leif]

ghm...@mst.edu wrote:
> On Monday, May 20, 2013 5:53:16 PM UTC-5, leif wrote:
>> ghm...@mst.edu wrote:

>> <SNIP>

>>> ImportError: No module named _sha256
>>
>>> hashlib module failed to import
>>
>>
>> Install OpenSSL? (There's even a Sage package for that.)
>>
>>
>> And/or if you're on Debian, Ubuntu, or one of their derivatives, try the
>>
>> Python 2.6.4.p11 spkg, which is still available e.g. here:
>>
>> https://spkg-upload.googlecode.com/files/python-2.6.4.p11.spkg
>>
>>
>>> I have installed multiple packages, updated Python, and any number of other things. Unfortunately, I must use this version of Sage and would like some help with this. Please explain why this is so difficult and why the error messages are astoundingly unhelpful. Thank you, any help is appreciated!
>>
>>

>> And
>>
>> > Describe your computer, operating system, etc.
>>

> I spent hours Googling the error message before posting here. And tried anything even remotely related to my issue, to no avail.

Well, if you provide us with some information on your system (see
above), and upload your Python spkg build log somewhere, we can probably
help you debug the issue.

It wouldn't hurt to tell us why you have to (or want to) stick with Sage
4.6, released years ago, either... ;-)

[ghm...@mst.edu]

Alright, here's all of the information that seems relevant:

I'm running Ubuntu 12.04 as a Virtual Machine (More than enough memory - up to 100 GB if needed for everything!) hosted on a Windows 8 laptop.  I inherited a massive code base reliant on Sage to do the background mathematics and many of the function calls and background work makes use of Sage, in addition to the fact that I am not the only programmer and have very little to do with Sage, needing it only to work correctly so that I may work on the GUI side of things.  The code base uses Sage 4.6 along with some overwritten sage functions to accomplish what we need to do...only available in 4.6.

My python spkg build log is attached (or at least I think it is.  I'm more familiar with pypy and have little experience installing stuff like this from scratch.  If this isn't it, please direct me to the correct file.  Thanks!)

Anything else that would be useful? 

On Monday, May 20, 2013 3:54:41 PM UTC-5, ghm...@mst.edu wrote:

Hey Sage Devs!

I have encountered a problem during my attempt to build Sage for the first time.  As the program requests, I attempt to 'make' Sage and most of it seems to work fine until it attempts to test Python. At this point, I encounter the error below:

math module OK
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/george/APCG/apcg/mcp/sage/local/lib/python/hashlib.py", line 138, in <module>
    sha224 = __get_builtin_constructor('sha224')
  File "/home/george/APCG/apcg/mcp/sage/local/lib/python/hashlib.py", line 66, in __get_builtin_constructor
    import _sha256

ImportError: No module named _sha256
hashlib module failed to import

real        2m6.118s

user        1m20.713s
sys        0m7.724s
sage: An error occurred while installing python-2.6.4.p9
Please email sage-devel http://groups.google.com/group/sage-devel
explaining the problem and send the relevant part of
of /home/george/APCG/apcg/mcp/sage/install.log.  Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9 and type 'make check' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
(cd '/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9' && '/home/george/APCG/apcg/mcp/sage/sage' -sh)
When you are done debugging, you can type "exit" to leave the
subshell.
make[1]: *** [installed/python-2.6.4.p9] Error 1
make[1]: Leaving directory `/home/george/APCG/apcg/mcp/sage/spkg'

real        2m14.313s
user        1m23.997s
sys        0m8.637s
Error building Sage.
./sage -docbuild all html  2>&1 | tee -a dochtml.log

I have installed multiple packages, updated Python, and any number of other things.  Unfortunately, I must use this version of Sage and would like some help with this.  Please explain why this is so difficult and why the error messages are astoundingly unhelpful.  Thank you, any help is appreciated!

[kcrisman]

On Tuesday, May 21, 2013 1:24:08 AM UTC-4, ghm...@mst.edu wrote:

Alright, here's all of the information that seems relevant:

I'm running Ubuntu 12.04 as a Virtual Machine (More than enough memory - up to 100 GB if needed for everything!) hosted on a Windows 8 laptop.  I inherited a massive code base reliant on Sage to do the background mathematics and many of the function calls and background work makes use of Sage, in addition to the fact that I am not the only programmer and have very little to do with Sage, needing it only to work correctly so that I may work on the GUI side of things.  The code base uses Sage 4.6 along with some overwritten sage functions to accomplish what we need to do...only available in 4.6.

Presumably "last available in".  Is this perchance due to 

http://trac.sagemath.org/sage_trac/ticket/10043

http://trac.sagemath.org/sage_trac/ticket/10151

in which case it's possible that we here (or on sage-support, or wherever) could get your group up to speed with the new interface.  Nathann, do you know how hard it would be to update legacy code?  Maybe more than just solver=solver added everywhere... 

Naturally, this all assumes the OP's code is in fact GPL-compatible ;-)  We would be very interested in helping advertise projects which use Sage, so if you let us know that would be wonderful!

 

My python spkg build log is attached (or at least I think it is.  I'm more familiar with pypy and have little experience installing stuff like this from scratch.  If this isn't it, please direct me to the correct file.  Thanks!)

Yes, here we aren't installing a Python package, but rather building Python itself from scratch.

The following seems relevant.

building 'crypt' extension

gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9/src/./Include -I. -IInclude -I./Include -I/home/george/APCG/apcg/mcp/sage/local/include -I/usr/local/include -I/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9/src/Include -I/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9/src -c /home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9/src/Modules/cryptmodule.c -o build/temp.linux-i686-2.6/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9/src/Modules/cryptmodule.o

gcc -pthread -shared -L/home/george/APCG/apcg/mcp/sage/local/lib -I. -IInclude -I./Include -I/home/george/APCG/apcg/mcp/sage/local/include build/temp.linux-i686-2.6/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9/src/Modules/cryptmodule.o -L/home/george/APCG/apcg/mcp/sage/local/lib -L/usr/local/lib -L. -lpython2.6 -o build/lib.linux-i686-2.6/crypt.so

*** WARNING: renaming "crypt" since importing it failed: build/lib.linux-i686-2.6/crypt.so: undefined symbol: crypt

Failed to find the necessary bits to build these modules:

_bsddb             _curses            _curses_panel   

_hashlib           _ssl               bsddb185        

dbm                gdbm               linuxaudiodev   

ossaudiodev        sunaudiodev                        

To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Failed to build these modules:

crypt              nis                                

 

The first hit on a Google search for the crypt error message is

http://groups.google.com/group/sage-devel/browse_thread/thread/593b9a4124f5075d

which leads to 

http://trac.sagemath.org/sage_trac/ticket/11243

and it looks like 

http://boxen.math.washington.edu/home/kirkby/patches/python-2.6.4.p10.spkg

might be what you want.  This issue went away in Python 2.7 so it is no longer an issue.

[kcrisman]

> http://trac.sagemath.org/sage_trac/ticket/11243
>
> and it looks like
>

> http://boxen.math.washington.edu/home/kirkby/patches/python-2.6.4.p10...

>
> might be what you want.  This issue went away in Python 2.7 so it is no
> longer an issue.

And it's possible that this won't work with Ubuntu 12.04 since Sage
4.7 was released, with this ticket in it, in May 2011... nearly a year
before the Ubuntu release you are using. But the fix to the spkg
should be relatively trivial.

[ghm...@mst.edu]

You say "relatively trivial" and I feel increasingly unintelligent.  I downloaded the python-2.6.4.p10, extracted it, and copied it in place of the p9 that was in the sage spkg directory, with no change in results.

Is this the proper procedure or does something else needed to be modified?  Again, this is the first time I've done anything like this and it's a bit over my head.

The ticket you linked me to was helpful, and I'm currently running through the suggested changes in the patch.  I'll post another update after those are made and another 'make' is attempted.

Thank you!

[kcrisman]

On May 21, 12:45 pm, ghm...@mst.edu wrote:
> You say "relatively trivial" and I feel increasingly unintelligent.  I

Sorry, I meant relatively trivial for someone who knows how spkgs work
- I was thinking someone might cook one up.

Try replacing your spkg with http://boxen.math.washington.edu/home/kcrisman/python-2.6.4.p10.spkg
just to see if the most trivial possible change works.

> downloaded the python-2.6.4.p10, extracted it, and copied it in place of
> the p9 that was in the sage spkg directory, with no change in results.

No, by design it shouldn't change results. It only is testing for
Ubuntu 11.

But it doesn't matter. Do a normal (i.e. failing at Python) Sage
build. When it gives you the message you saw

Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
(cd '/home/george/APCG/apcg/mcp/sage/spkg/build/python-2.6.4.p9' && '/
home/george/APCG/apcg/mcp/sage/sage' -sh)

and then it's possible to make the changes with all environment
variables appropriately set. Though I don't think it would be as easy
as typing "make" there, since the Python spkg has a little more to it
than most.

[ghm...@mst.edu]

Just checking in, I replaced the python install retrieved by Sage with the downloaded Python spkg, with no change in error message.  Any other ideas?

Thank you!

[kcrisman]

On May 21, 1:57 pm, ghm...@mst.edu wrote:
> Just checking in, I replaced the python install retrieved by Sage with the
> downloaded Python spkg, with no change in error message.  Any other ideas?
>

I'm not sure what you mean by this. What you'd have to do is remove
the python-x.y.z.spkg in spkg/standard, replace it with the one I
uploaded, and then try make. At any rate, the underlying cause here
is almost certainly the same as at the ticket in question.

If for some reason that doesn't work, then someone more expert than me
needs to help. And it would be useful to have a link to the
downstream project, as we said :) since perhaps you just need a minor
tweak in the interface to make it work with a more current Sage.

[leif]

kcrisman wrote:
> On May 21, 1:57 pm, ghm...@mst.edu wrote:
>> Just checking in, I replaced the python install retrieved by Sage with the
>> downloaded Python spkg, with no change in error message. Any other ideas?
>>
>
> I'm not sure what you mean by this. What you'd have to do is remove
> the python-x.y.z.spkg in spkg/standard, replace it with the one I
> uploaded, and then try make. At any rate, the underlying cause here
> is almost certainly the same as at the ticket in question.

As already suggested, I'd try the .p11 spkg. (You shouldn't have to
delete the .p9 or .p10, but in any case you have to copy the new /
alternate Python spkg into $SAGE_ROOT/spkg/standard/. Afterwards simply
type 'make' again.)

If it's a (short-term) option for you, use Lucid (10.04.4 LTS) in the
VM, which doesn't have those problems.

Otherwise also make sure you have 'dpkg-dev' installed ("sudo apt-get
install dpkg-dev").

[ghm...@mst.edu]

p11 and p10 both fail to resolve the problem.  dpkg-dev installed and fully updated.

Thank you for the suggestion though!

[François Bissey]

Do you have libcrypto installed on the system at all. I think it would
trigger this particular module build to fail.

It is a bit of a shot in the dark I'll admit.

Francois

[leif]

ghm...@mst.edu wrote:
> p11 and p10 both fail to resolve the problem. dpkg-dev installed and
> fully updated.

Use the .p11, which works for me (Ubuntu 12.04.2 x86_64, Ubuntu GCC
4.6.3), and was supposed to fix exactly that issue.

Prior to that, install OpenSSL: sudo apt-get install libssl-dev

(That should provide or pull in libcrypto as well.)


Anyway, even if you finally succeed installing the Python spkg, you'll
get into further trouble.

You'll have to replace further spkgs by ones from later releases (such
as symmetrica-2.0.p7, lcalc-20100428-1.23.p6, singular-3-1-1-4.p12), and
install some "manually" be specifying specific flags.

[Probably more to come; I'm still building...]


As mentioned, you'd have less trouble with Lucid (Kernel 2.x, GCC 4.4.3,
"traditional" file system layout w.r.t. headers and libraries).


-leif

> As already suggested, I'd try the .p11 spkg. (You shouldn't have to
> delete the .p9 or .p10, but in any case you have to copy the new /
> alternate Python spkg into $SAGE_ROOT/spkg/standard/. Afterwards
> simply
> type 'make' again.)
>
> If it's a (short-term) option for you, use Lucid (10.04.4 LTS) in the
> VM, which doesn't have those problems.
>
> Otherwise also make sure you have 'dpkg-dev' installed ("sudo apt-get
> install dpkg-dev").

[leif]

leif wrote:
> ghm...@mst.edu wrote:
>> p11 and p10 both fail to resolve the problem. dpkg-dev installed and
>> fully updated.
>
> Use the .p11, which works for me (Ubuntu 12.04.2 x86_64, Ubuntu GCC
> 4.6.3), and was supposed to fix exactly that issue.
>
> Prior to that, install OpenSSL: sudo apt-get install libssl-dev
>
> (That should provide or pull in libcrypto as well.)
>
>
> Anyway, even if you finally succeed installing the Python spkg, you'll
> get into further trouble.
>
> You'll have to replace further spkgs by ones from later releases (such
> as symmetrica-2.0.p7, lcalc-20100428-1.23.p6, singular-3-1-1-4.p12), and
> install some "manually" be specifying specific flags.
>
> [Probably more to come; I'm still building...]

Next stop: r-2.10.1.p4 (doesn't properly detect readline)

=> use the readline-6.2.p3 spkg


-leif

[leif]

leif wrote:
> leif wrote:
>> ghm...@mst.edu wrote:
>>> p11 and p10 both fail to resolve the problem. dpkg-dev installed and
>>> fully updated.
>>
>> Use the .p11, which works for me (Ubuntu 12.04.2 x86_64, Ubuntu GCC
>> 4.6.3), and was supposed to fix exactly that issue.
>>
>> Prior to that, install OpenSSL: sudo apt-get install libssl-dev
>>
>> (That should provide or pull in libcrypto as well.)

It does.

>> Anyway, even if you finally succeed installing the Python spkg, you'll
>> get into further trouble.
>>
>> You'll have to replace further spkgs by ones from later releases (such
>> as symmetrica-2.0.p7, lcalc-20100428-1.23.p6, singular-3-1-1-4.p12), and
>> install some "manually" be specifying specific flags.
>>
>> [Probably more to come; I'm still building...]
>
> Next stop: r-2.10.1.p4 (doesn't properly detect readline)
>
> => use the readline-6.2.p3 spkg

----------------------------------------------------------------------
| Sage Version 4.6, Release Date: 2010-10-30 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
sage: !cat /etc/issue; uname -rsm; gcc --version
Ubuntu 12.04.2 LTS \n \l

Linux 3.2.0-41-generic x86_64
gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

sage: !ls spkg/installed/
atlas-3.8.3.p16 gd-2.0.35.p5 mpfi-1.3.4-cvs20071125.p8
rubiks-20070912.p12
blas-20070724 gdmodule-0.56.p7 mpfr-2.4.2 sage-4.6
boehm_gc-7.1.p6 genus2reduction-0.3.p8 mpir-1.2.2.p1 sagenb-0.8.7
boost-cropped-1.34.1 gfan-0.4plus.p1 mpmath-0.15 sage_scripts-4.6
bzip2-1.0.5 givaro-3.2.13rc2.p2 networkx-1.2.p1 sagetex-2.2.5
cddlib-094f.p8 glpk-4.44 ntl-5.4.2.p12 scipy-0.7.p5
cephes-2.8 gnutls-2.2.1.p5 numpy-1.3.0.p4 scipy_sandbox-20071020.p7
cliquer-1.2.p7 graphs-20070722.p1 opencdk-0.6.6.p5 scons-1.2.0
conway_polynomials-0.2 gsl-1.14 palp-1.1.p3 setuptools-0.6c9.p0
cvxopt-0.9.p9 iconv-1.13.1.p3 pari-2.4.3.svn-12577.p9
singular-3-1-1-4.p12
cython-0.13.p1 iml-1.0.1.p13 pexpect-2.0.p4 sphinx-0.6.3.p4
dir-0.1 ipython-0.9.1.p0 pil-1.1.6.p2 sqlalchemy-0.5.8
docutils-0.5.p0 jinja2-2.1.1.p0 polybori-0.6.4.p6 sqlite-3.6.22
ecl-10.2.1.p3 lapack-20071123.p1 polytopes_db-20100210 symmetrica-2.0.p7
eclib-20100711 lcalc-20100428-1.23.p6 prereq-0.7 sympow-1.018.1.p8
ecm-6.2.1.p2 libfplll-3.0.12.p1 pycrypto-2.1.0 sympy-0.6.4.p0
elliptic_curves-0.1 libgcrypt-1.4.4.p3 pygments-0.11.1.p0
tachyon-0.98beta.p11
examples-4.6 libgpg_error-1.6.p3 pynac-0.2.1 termcap-1.3.1.p1
extcode-4.6 libm4ri-20100701.p1 python-2.6.4.p11 twisted-9.0.p2
f2c-20070816.p2 libpng-1.2.35.p2 python_gnutls-1.1.4.p7 weave-0.4.9.p0
flint-1.5.0.p5 linbox-1.1.6.p3 r-2.10.1.p4 zlib-1.2.5
flintqs-20070817.p5 matplotlib-1.0.0 ratpoints-2.1.3.p1 zn_poly-0.9.p5
fortran-20100629 maxima-5.20.1.p0 readline-6.0.p4 zodb3-3.7.0.p4
freetype-2.3.5.p3 mercurial-1.3.1.p2 readline-6.2.p3
gap-4.4.12.p4 moin-1.9.1.p1 rpy2-2.0.8
sage:


q.e.d. ;-)


I could compile a source tarball with these spkgs if you want, although
you'd still have to build the Singular spkg with specific flags (at
least LDFLAGS="-ldl", probably CXXFLAGS="-fno-tree-vectorize"; if you
set CFLAGS etc. when building the Sage library, you also have to add
"-fno-strict-aliasing"), but I could also provide a shell script to do
so (building that distribution from scratch in "one step").

That's probably better than a binary distribution, and building a 32-bit
version (you may want) for Precise would take even more time.


-leif