Conda packages for Ubuntu 16.04 / GCC version 5.4 and greater

[Manuel Günther]

Dear Idiapers,

yesterday I tried to install Bob using conda on Ubuntu 16.04 LTS. As I am modifying some packages, I tried to build the GitLab package ``bob.io.image`` against the conda install. After building ``bob.io.image``, I tried to import it. Unfortunately, this does not work:

$ ./bin/python
>>> import bob.io.image

OSError: [..]/bob/io/image/libbob_io_image.so: undefined symbol: _ZNK3bob2io4base5array8typeinfo3strB5cxx11Ev

$ c++filt _ZNK3bob2io4base5array8typeinfo3strB5cxx11Ev

bob::io::base::array::typeinfo::str[abi:cxx11]() const

After some time of googling, I found out (https://github.com/klee/klee/issues/336) that there is a problem with the GCC version. Apparently, GCC 4 and GCC 5 are binary incompatible as they have reimplemented the STL in GCC 5.

I have tried the solution provided in the above mentioned thread, i.e., setting 

$ export CXXFLAGS="-D_GLIBCXX_USE_CXX11_ABI=0"
$ ./bin/buildout

However, when I tried to link other libraries (like google protobuf, required by Caffe), I found that I would need to recompile these libraries with the same flag (or with an older GCC version).

Of course, I would like to avoid that. So, one solution is to build all Bob packages from scratch. This works as expected. 

A nicer solution would be to have a conda install for Bob, which relies on GCC 5.4 or greater. How difficult would this be to generate?

Cheers

Manuel

[Manuel Günther]

Update: I said that compiling from scratch worked. Unfortunately, this is not true. Running the first three command lines from https://gitlab.idiap.ch/bob/bob/wikis/Dependencies#ubuntu, I get the following error:

Package libtiff4-dev is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  libtiff5-dev:i386 libtiff5-dev

Package python-support is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'python-support' has no installation candidate
E: Unable to locate package libblitz1-dev
E: Package 'libtiff4-dev' has no installation candidate
E: Unable to locate package libvl-dev

So, either the build instructions simply don't work for Ubuntu 16.04 LTS, or they are outdated. When looking into the conda installation, I find that it installed ``libblitz`` in version ``0.10``, and not in version 1 (the one that is provided with the previous Bob binaries, see above).

I thought that there was a good reason for having our own blitz version compiled: the original version does not support Arrays larger than 2GB. I think, this limitation still applies. Can someone please tell me, if it now got unsafe to create larger Arrays?

Cheers

Manuel

[Amir Mohammadi]

Dear Manuel,

You need to compile every c++ package with the new ABI not just bob packages, so you need to compile blitz too for example.

The best way for you is to compile Google protobuf and cafe with the old ABI and against your conda installation.

Best,

Amir

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Amir Mohammadi]

That dependency list is definitely outdated. 

--

[Amir Mohammadi]

The latest version of blitz is 0.10 which is what we ship in conda. You are confusing version of blitz with its soname version probably.

On Tue, Oct 18, 2016, 8:21 PM 'Manuel Günther' via bob-devel <bob-...@googlegroups.com> wrote:

--

[Manuel Günther]

I see, what you are saying, and I don't want to do that. Google protobuf is just one of the many libraries that Caffe requires.

Note that you only need to recompile C++ libraries that export functions using STL (e.g., std::string). As blitz does not export such functions, I can link against blitz that was compiled with GCC 4.

My current work-around looks like follows:

1. Install a conda environment without bob, but with bob's dependencies: 

$ conda create -n base python=2.7 libblitz
$ conda install -n base cython pip giflib libtiff

2. Compile all Bob packages from source, using the conda ``base`` environment

3. Compile OpenCV (required by Caffe) and install it manually into the conda ``base`` environment

    I had to disable TIFF as this does not seem to work with Caffe (next step)

4. Compile Caffe using Bob (from source) and OpenCV (from the base conda environment)

This seems to have worked fine.

So, my question still is: is there a way to provide Bob packages compiled with GCC 5 in conda -- without overwriting the ones compiled with GCC 4? If yes, how difficult, you think, would it be to do that?

Thanks

Manuel

[André Anjos]

On Wed, Oct 19, 2016 at 2:12 AM, 'Manuel Günther' via bob-devel <bob-...@googlegroups.com> wrote:

So, my question still is: is there a way to provide Bob packages compiled with GCC 5 in conda -- without overwriting the ones compiled with GCC 4? If yes, how difficult, you think, would it be to do that?

I don't think this is a feasible alternative as we have to follow conda-forge if we'd like to keep maximum compatibility with other deployment environments.

@Amir: what do you think about this?

Best, Andre

--

Dr. André Anjos
Idiap Research Institute
Centre du Parc - rue Marconi 19
CH-1920 Martigny, Suisse
Phone: +41 27 721 7763
Fax: +41 27 721 7712
http://andreanjos.org

[Amir Mohammadi]

Hi Manuel,

You can compile bob packages with GCC 5 and the new C++ ABI and create conda packages of them but we cannot do that in conda-forge. You need to take the feedstocks (e.g. https://github.com/conda-forge/bob.io.image-feedstock) , modify them, and generate your packages locally and use those instead.

What you did is also fine too but you should be very careful to use only C and not C++ from conda-forge.

Best,

Amir

--