Support PEP 427 wheel distrubtions on PyPi

[Tom Forbes]

Hello,

Are there any current plans for distributing bob in a PEP 427 wheel format on PyPi? Currently source distributions are uploaded but it would be good to have at least a MacOS wheel available.

Based on some cursory reading of the Gitlab CI config that the various bob sub-packages use it should be possible to do this, unless I am missing something? If there are no current plans and others agree this is worthwhile I would love to offer my help in getting this going.

Tom

[Amir Mohammadi]

Hi Tom, 

Bob has a lot of C/C++ dependencies.

It's not a pure Python software that you can easily distribute with wheels. We use conda to distribute our software. Have you considered using that? Conda is a binary package manager that can package libraries in any language. With conda we can package libraries in C like ffmpeg and bob which is a mix of C++ and Python.

Thanks,

Amir

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tom Forbes]

Thanks for your reply Amir,

Python Wheels are useful for exactly this use case, and if you have a pip based workflow then integrating Conda into it can be annoying and add friction. I really believe it is quite simple to add support for distributing wheels on PyPi, as a test I cloned the bob.ap repo here (https://gitlab.idiap.ch/bob/bob.ap/tree/master) and ran `python setup.py bdist_wheel`. This created a .whl file in the dist/ directory which contains all the compiled extensions and can be installed without a C compiler on your system. The test scripts included all appear to pass. I've attached this wheel file (built on MacOS 10.13 64 bit) and you should be able to install this on a MacOS python 3.6 environment by doing `pip install [wheel path]`.

Wheels do not help in the case where you have runtime system libraries (like an ffmpeg executable, or a library that's dynamically linked) that need to be available but this can be up to the user to handle. I would just like to be able to install bob modules via pip without needing the entire build chain and all the associated libraries and compilers, and PEP 427 wheels really help here.

If you're already building the libraries on MacOS/Linux in the CI it may be as simple as just doing 'python setup.py bdist_wheel' on those platforms and then uploading the resulting wheel files to PyPi alongside the source, which is handled by twine or your current release script. 

[Amir Mohammadi]

Hi Tom,

We used to work with wheels in our CI before and we recently replaced them with

conda packages because they are superior to wheels. I invite you to read:

https://stackoverflow.com/q/20994716

or https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/

It took us two years to completely switch to Conda. We are not going back ;)

It is not just as simple as creating wheels in our CI.

We have to spend a large portion of our time implementing this

and maintaining this which is just a waste of our time even if you

help us do it.

If you don't want to use Conda, I recommend that you read the docs of our

developer modules which let's you compile Bob from source:

https://www.idiap.ch/software/bob/docs/bob/docs/master/index.html#modules-for-developers

and if you think the docs can be improved, send us a patch.

At Idiap, we use both conda and buildout to develop Bob.

If you cannot install Bob from source using pip or buildout,

please let us know and we will help you. If you use pip,

you have to pip install bob packages one by one in order of dependencies.

Something like:

pip install bob.extension

pip install bob.blitz

pip install bob.core

pip install bob.io.base

....

Installing all of them at once will not work because of limitations of pip.

Thanks,

Amir

[Amir Mohammadi]

Hi Tom,

I created and answered a stackoverflow question that will help you

install Bob without conda:

https://stackoverflow.com/questions/50803148/how-to-install-bob-from-source-without-conda/50803149#50803149

I hope that helps.

This was recycled and updated from our old documentation:

https://www.idiap.ch/software/bob/docs/bob/bob/v3.0.0/source.html

Best,

Amir