best practices for deploying a cross-platform Cython project - or examples?

[Tom Swirly]

Hello, Cythoners.  

I'm getting towards the end of my second Cython project, and it's gone extremely smoothly.  (You can see it here: https://github.com/rec/tdsp - just what you want, if what you want is fast and powerful color math...)

Now I want to start packaging a release, which means making a bunch of builds for platforms like Windows and OS/X where the users might not even have a compiler, as well as making convenience builds for other platforms like Ubuntu.

Other people must have done this before, and also learned by feedback from their users.  

Are there any good examples of this, or best practice documents?   I found some examples on this list but they were dealing with "multiple extensions" whereas we have one extension library, and multiple release targets.

Thanks in advance!

--

     /t

http://radio.swirly.com - art music radio 24/7 366/1000

[Denis Akhiyarov]

conda-forge seems to be nice way to take advantage of various CI like Appveyor and Travis and developers are converging around this build tool.

and here is example for you with cython project:

https://github.com/conda-forge/cython-feedstock

[Tom Swirly]

Thanks for the reply!

But I'm not seeing how this works.  Where are all the .so files in this project?  (I believe that Windows uses something else, but I'm going to call them .so for now...)

The big issue is keeping track of these three or four pre-built .sos and making sure that the correct one gets installed into the correct place.

Heck, there doesn't even seem to be a setup.py - so how does a user actually install this example?

[Denis Akhiyarov]

I recommend that you continue the discussion in conda-forge issue tracker and just cross-link it here.

The starting point is to look at YAML files for each CI, it is using conda build under the hood:

https://github.com/conda-forge/cython-feedstock/blob/master/appveyor.yml

[Tom Swirly]

I think you're misunderstanding what it is we need - I think you're being confused by the word "packaging".  :-)

We need to be able to create an installer for non-technical people who won't have a compiler.  This requires pre-built binaries that we check in - there's no way around it - and it requires a setup.py for us to be able to distribute through PyPi.

The example you showed me cannot be what we need.  There is no setup.py;  there are no binaries.  A naive user would not be able to install that package.

(I know Appveyor pretty well, but we have no great need for CI yet, or I'd have set it up already... so far I'm the only programmer who's contributed to this, and while I have several very interested customers for this who will probably contribute somewhat, my theory is that it might be months before even a second developer ever contributes...)

[Chris Barker - NOAA Federal]

On May 19, 2016, at 8:24 AM, Tom Swirly <t...@swirly.com> wrote:

I think you're misunderstanding what it is we need - I think you're being confused by the word "packaging".  :-)

No, you are confused by what you were pointed to -- it is a recipe for creating s a package -/ not the package itself.

What is being recommended is that you use the conda packaging system, rather than PyPi +pip.

The packages themselves go up on anaconda.org for distribution.

https://anaconda.org

https://anaconda.org/conda-forge/packages

Your uses would install Anaconda or miniconda, then install your packages.

Conda-forge provides tooling to make distributing multi-platform packages pretty automatic.

Alternatively, you can build binary wheels and put them up on PyPi. But I'm not aware of any tooling to make that easy to do for multiple platforms with CI systems.

We need to be able to create an installer for non-technical people who won't have a compiler.  This requires pre-built binaries that we check in - there's no way around it - and it requires a setup.py for us to be able to distribute through PyPi.

If you want binaries on PyPi you need wheels.

CHB

https://groups.google.com/d/optout.