Building packages with Python 2.7

[Anthony Scopatz]

Hello Again, 

Does anyone know how to specify which version of Python that you want to build your package with?  Python 2.7 is clearly available at run time, but when I try to build a Cython extension it uses the old Python 2.6 version....

Thanks for your help in advance.

Be Well

Anthony

[Aaron Staley]

Currently the build system only works with python 2.6.  We're working on adding support for python2.7, but that may be a few weeks away.  Can you use python2.6 for now or is 2.7 build support necessary?

Thanks,

Aaron Staley

PiCloud, Inc.

[Anthony Scopatz]

Hi Aaron, 

Thanks for the reply.  It is not the python version so much as the Cython version that matters.  Version 0.12 and below didn't have C++ support and thus don't have the 'libc' module that you can cimport.  

EPD went from Cython 0.12 -> 0.13 at the 6.2 / 6.3 split.  So if you are at EPD at or below 6.2, maybe an update is all that is needed....

EPD version 7 uses Cython 0.14.1, which it seems is already on PiCloud.  Which is why I asked my original question ;)  

Thanks again!

Be Well

Anthony

[Aaron Staley]

Python 2.6 has access to EPD 6.3.  Python 2.7 accesses EPD 7.

Thanks,

Aaron

[Anthony Scopatz]

Ahh I think I see what is going on.  My build path must automatically include Cython's Includes directory (on my system '/usr/lib64/python2.7/site-packages/Cython/Includes/').  However, this is not automatically on the path in PiCloud.  

Rather than me spend a ton of time looking for it, do you know off hand Aaron what the path to the PiCloud site-packages is that holds Cython?  '/usr/lib/python2.6/site-packages/' doesn't seem to exist and '/usr/lib/python2.6/dist-packages/' doesn't have Cython in it....

Thanks again!

Be Well

Anthony

[Aaron Staley]

Hi Anthony,

Cython is stored on our shared file system.

To access it, use the site packages specified by:

/afs/picloud.com/toolkits/epd-6.2-2-rh5-x86_64/lib/python2.6/site-packages/

cython.py is there, as is the Cython directory which contains the necessary Includes directory.

Please let me know how everything goes.  

Regards,

Aaron Staley

[Anthony Scopatz]

Hi Aaron, 

Thanks for pointing me in the right direction.  I added a few lines to my setup.py which attempts to discover the 'Cython/Includes/' dir and it confirms the directory you mentions.  

However, my build still fails because as you can see in the dir's path you are still on EPD 6.2 (epd-6.2-2-rh5-x86_64), not EPD 6.3 as you mentioned before (at least for building packages).  There are some pretty serious differences between Cython 0.12 (EPD 6.2) and Cython 0.13 (EPD 6.3).  The major issue here is that the structure of the 'Cython/Includes/' dir is completely different.

I'd rather not have to support an old (and inferior) version of Cython in my code.  Is there any way for you to update to EPD 6.3?  I'd imagine it is easier than EPD 7.  Or at least use enpkg to update Cython on your 6.2 install...

(Really I think that the long term solution here is allowing users to select what version(s) of Python/EPD they want to build with.  I understand that might take a while to implement ;).

Thanks again!

Anthony

[Ken Elkabany]

Hi Anthony,

Unfortunately, we'll have to get back to you on when your issue will be resolved. We're currently in the process of revamping the entire package management system. Our new system will allow you to take control of the compilation process on our servers, including choosing the libraries you wish to compile against. We'll be sure keep you posted.

Thanks,

Ken

[Anthony Scopatz]

On Wed, Apr 13, 2011 at 10:47 PM, Ken Elkabany <k...@picloud.com> wrote:

Hi Anthony,

Unfortunately, we'll have to get back to you on when your issue will be resolved. We're currently in the process of revamping the entire package management system. Our new system will allow you to take control of the compilation process on our servers, including choosing the libraries you wish to compile against.

Fantastic!  I saw that you are going down for maintenance on Twitter and figured this might be related.  If it isn't, thanks for acknowledging this issue. 

 

We'll be sure keep you posted.

Thanks again.

Be Well

Anthony

[Ken Elkabany]

The new package management system won't be available this Friday. Don't despair! We're making significant backend changes this Friday that will be the precursor to many new features, including package management.