trouble building pycrypto-2.6.1 (for sage 6.1)

[Niles Johnson]

I had no trouble building sage (using make) for 6.1.beta1 or 6.1.beta2, but for 6.1.beta3 there is a failure building pycrypto on Red Hat.  Apparently a new version was merged here

http://git.sagemath.org/sage.git/log/?h=fe9ba66e75a8817910fe74eb6c196225cf0cbc2f

Here are some excerpts from the log file, with the full thing attached.  Ideas on what I can do?

Thanks,

Niles

Found local metadata for pycrypto-2.6.1.p0

Found local sources at /home/johnson.5320/sage/upstream/pycrypto-2.6.1.tar.gz

Checksum: aeda3ed41caf1766409d4efc689b9ca30ad6aeb2 vs aeda3ed41caf1766409d4efc689b9ca30ad6aeb2

pycrypto-2.6.1.p0

====================================================

Setting up build directory for pycrypto-2.6.1.p0

Finished set up

****************************************************

Host system:

Linux kepler.math.osu.edu 2.6.32-431.1.2.0.1.el6.x86_64 #1 SMP Fri Dec 13 13:06:13 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

****************************************************

C compiler: gcc

C compiler version:

Using built-in specs.

Target: x86_64-redhat-linux

Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux

Thread model: posix

gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) 

****************************************************

patching file src/libtom/tomcrypt_pk.h

running install

running build

...

running build_ext

running build_configure

configure: error: cannot find sources (src/pycrypto_compat.h) in /home/johnson.5320/sage or ..

Traceback (most recent call last):

  File "setup.py", line 456, in <module>

    core.setup(**kw)

...

  File "setup.py", line 278, in run

    raise RuntimeError("autoconf error")

RuntimeError: autoconf error

Error: Failed to install PyCrypto.

real 0m0.423s

user 0m0.172s

sys 0m0.223s

************************************************************************

Error installing package pycrypto-2.6.1.p0

[Niles Johnson]

Oh, my build on Mac OS 10.7.5 just finished and I have the same problem there :(  

Here are the equivalent excerpts:

Found local metadata for pycrypto-2.6.1.p0

Attempting to download package pycrypto-2.6.1.p0

>>> Trying to download http://www.sagemath.org/packages/upstream/pycrypto/pycrypto-2.6.1.tar.gz

[.......................................................]

Checksum: aeda3ed41caf1766409d4efc689b9ca30ad6aeb2 vs aeda3ed41caf1766409d4efc689b9ca30ad6aeb2

pycrypto-2.6.1.p0

====================================================

Setting up build directory for pycrypto-2.6.1.p0

Finished set up

****************************************************

Host system:

Darwin galileo.newark-campus.com 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64

****************************************************

C compiler: gcc

C compiler version:

Using built-in specs.

COLLECT_GCC=gcc

COLLECT_LTO_WRAPPER=/Applications/sage/local/libexec/gcc/x86_64-apple-darwin11.4.2/4.7.3/lto-wrapper

Target: x86_64-apple-darwin11.4.2

Configured with: ../src/configure --prefix=/Applications/sage/local --with-local-prefix=/Applications/sage/local --with-gmp=/Applications/sage/local --with-mpfr=/Applications/sage/local --with-mpc=/Applications/sage/local --with-system-zlib --disable-multilib --disable-nls  

Thread model: posix

gcc version 4.7.3 (GCC) 

****************************************************

patching file src/libtom/tomcrypt_pk.h

running install

running build

...

running build_ext

running build_configure

configure: error: cannot find sources (src/pycrypto_compat.h) in /Applications/sage or ..

Traceback (most recent call last):

  File "setup.py", line 456, in <module>

...

  File "setup.py", line 278, in run

    raise RuntimeError("autoconf error")

RuntimeError: autoconf error

Error: Failed to install PyCrypto.

real 0m0.399s

user 0m0.073s

sys 0m0.092s

[Niles Johnson]

On Monday, February 3, 2014 3:09:57 PM UTC-5, Niles Johnson wrote:

 

configure: error: cannot find sources (src/pycrypto_compat.h) in /Applications/sage or ..

After some trial and error, I found the file pycrypto_compat.h in what looks like a weird place:

<SAGE_ROOT>/local/var/tmp/sage/build/pycrypto-2.6.1.p0/src/src/pycrypto_compat.h

I copied this to <SAGE_ROOT>/src, as per the error message from the log file, and that seems to have done the trick.  Building 6.1 on linux finished successfully; I'm still waiting for my Mac OS build to finish, but it's already past the point of building pycrypto-2.6.1.p0.

Even though this worked, I don't understand why it was necessary.  There must be some other problem with my sage install (on both machines) . . . during my trial/error phase, I made fresh installs with

git clone git://github.com/sagemath/sage.git

but I had the same build problems.  Confusing.

-Niles

 

[Volker Braun]

I don't know why its not working for you, but what is failing is the following autoconf check:

AC_CONFIG_SRCDIR([src/pycrypto_compat.h])

My guess would be that you have some wonky environment variables pre-defined in your bashrc (e.g.). $srcdir, perhaps. You using the same bashrc (on passing through environment, did you ssh log into one of the systems?) on both systems seems to be the only explanation possible.

[Niles Johnson]

On Tuesday, February 4, 2014 11:34:26 AM UTC-5, Volker Braun wrote:

I don't know why its not working for you, but what is failing is the following autoconf check:

AC_CONFIG_SRCDIR([src/pycrypto_compat.h])

 

My guess would be that you have some wonky environment variables pre-defined in your bashrc (e.g.). $srcdir, perhaps. You using the same bashrc (on passing through environment, did you ssh log into one of the systems?) on both systems seems to be the only explanation possible.

Thanks for this explanation/guess.  Indeed, I did ssh to the linux system from the Mac.  I tried ssh from a different machine to the linux one, and then had no problems at all (make was successful for the 6.1 commit and the latest 6.1.1, and I did remove <SAGE_ROOT>/src/pycrypto_compat.h before testing).  I'll have to check environment variables on the Mac when I'm back in the office tomorrow . . . I didn't realize I was passing them through the ssh connection, but I agree that would explain the strange behavior.

-Niles

[Niles Johnson]

For the record, I would like to confess that I've now figured the problem out, and it was indeed a wonky environment variable: $PATH!  Years ago when I started using sage, I just added the sage root directory to my path, and I've been doing it ever since.  I now realize that is a bad idea.

The configure script for pycrypto was getting confused and calling the configure script in my sage root directory because it was findable from the $PATH.  This caused all manner of misconfiguration.  I'm a little shocked that somehow I've avoided a problem like this up until now.  Anyway, I've learned a lesson and I'm posting this in case anyone else makes the same mistake.

thanks,

Niles

p.s.  If you'd like to know, future Googler, here are the details of what I learned, and what I think happened:

0. I tried downloading and building pycrypto separately, independently of sage, and had a similar problem during the configuration stage.  So I dove into configure.


1. The configure script seems to execute itself several times, and so needs to know what command was used to run it.  It does this early on, and stores this in the variable $as_myself.


2. If I'm understanding correctly, when configure is executed with a directory separator in the command name (as in ./configure), it uses that command ($0), but if not it tries to figure out what command to run by searching the $PATH:

https://github.com/dlitz/pycrypto/blob/master/configure#L5470

So if there is some other configure script in $PATH, then it will find and execute that script!


3. pycrypto is configured and built from a python script, setup.py.  Line 307 there sets the configure command as "sh configure":

https://github.com/dlitz/pycrypto/blob/master/setup.py#L307

which causes $PATH to be searched.  Only if no configure is found in the $PATH does configure give up and default back to $0 (and therefore finish configuring correctly).


4.  Googling indicates that (1) and (2) are entirely standard parts of many many configure scripts (maybe all of them).  On the other hand, line 307 of setup.py seems special to pycrypto.  Maybe line 307 is part of the "Modifications" that they mention at the beginning of that file.  Google didn't find other files containing the comment about mingw32 that appears on that line.

[Volker Braun]

Ah, that explains it. We just recently added a "configure" script, which is why you didn't trip over that before.