Compiling for CentOS 6.4

[Sachin Shrestha]

Hey John/Daniel,

The latest builds work well on CentOS 7.x and I'm able to build from source without any issues. However, it looks like they have been compiled for glib 2.14 or higher from version 0.47.x onwards so the new builds don't work on our CentOS 6.4 (2.6.32-358.11.1.el6.x86_64) environment at work. Hence I wanted to build at our end. Are there any caveats or gotchas that I need to watch out for when building for the older OS version? Do I need to build all the dependencies as well?

Thanks,

Sachin

[Daniel Dresser]

This old thread got a bit confusing, since it was orignally about a completely different topic:

https://groups.google.com/forum/#!msg/gaffer-dev/uxJlfyU0NVU/T6juEjb-AgAJ

Reading that, it sounds like John was able to do Centos 6 builds up until fairly recently ( and stopped just because it was an extra burden on him for the public releases, and VFXPlatform has moved on ).  So that sounds somewhat hopeful that things should work - maybe John will chime in with any gotchas.

[John Haddon]

I'm afraid I don't recall the specifics of the last problem I had with CentOS 6. I suspect it might have been something relatively trivial like a Sphinx compatibility issue when building the docs. I don't think there's any fundamental issue with CentOS 6, but like Daniel said, it became too much of a burden to maintain, especially given that VFXPlatform has moved on.

You will need to build all the dependencies too, which you can find here :

https://github.com/gafferhq/dependencies

And you might find it useful to refer to the last CentOS 6 Docker setup we used :

https://github.com/GafferHQ/build/blob/4e6313971dacdf02203a2d2d4f5f256e3a1be4f4/Dockerfile

Cheers...

John

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sachin Shrestha]

Thanks John. I'm building the dependencies right now and it seems to be going well so far. Will post here if I have any trouble 9and I'm also referring to the exhaustive thread that Daniel linked).

Cheers,
Sachin

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.

[Sachin Shrestha]

Hi John,

We installed all the dependencies listed in your docker file with the only difference being the gcc tools are not installed in the system default path. Instead, I installed it in a separate location. We are setting this environment for the build:

export BUILD_DIR=/repo/gaffer/0.48.0.0/build/compiled
export ARNOLD_ROOT=/repo/SolidAngle/sdk/Arnold-5.1.1.0-linux
export RMAN_ROOT=/repo/3delight/3delight-12.5.9/Linux-x86_64
export CC=/repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/bin/gcc
export CXX=/repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/bin/c++
export LD_LIBRARY_PATH=/repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1:/repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/lib64
export PATH=/repo/Python/linux/python2.7_UCS4/bin:/repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/bin:/repo/Python/linux/python2.7_UCS4/bin:/usr/site/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/lsd/bin:

The build runs fine until it hits OpenEXR where it completes all the ilmbase builds but fails during pyilmbase build with these errors:

checking for C++ compiler default output file name...
configure: error: in `/repo/gaffer/0.48.0.0/build/dependencies/pyilmbase-2.2.0':
configure: error: C++ compiler cannot create executables
See `config.log' for more details.
mv: cannot stat `/repo/gaffer/0.48.0.0/build/compiled/lib/python*/site-packages/iexmodule.so': No such file or directory
mv: cannot stat `/repo/gaffer/0.48.0.0/build/compiled/lib/python*/site-packages/imathmodule.so': No such file or directory

We tried debugging this and it seems to fail at this point in the buildEXR.sh script:

./configure --prefix=$BUILD_DIR --with-boost-include-dir=$BUILD_DIR/include --without-numpy

Further debugging the configure script for pyilmbase, we noticed that the below file was either missing or  configure is not able to find it due to which it may not be able to run some tests:

cat >conftest.$ac_ext <<_ACEOF
/* confdefs.h.  */
_ACEOF
cat confdefs.h >>conftest.$ac_ext
cat >>conftest.$ac_ext <<_ACEOF
/* end confdefs.h.  */

It looks like it didn't find the confdefs.h file to cat into the conftest.cpp file. Do we need to install any other packages besides the ones mentioned in your docker file? Also, as you may have noticed above, we are overriding the default system python (which is 2.6) with 2.7 via the PATH and LD_LIBRARY_PATH env vars.

Would appreciate any pointers or hints on this.

Cheers,
Sachin

[John Haddon]

On 8 August 2018 at 16:01, Sachin Shrestha <noizf...@gmail.com> wrote:

The build runs fine until it hits OpenEXR where it completes all the ilmbase builds but fails during pyilmbase build with these errors:

See `config.log' for more details.

Have you looked in config.log to see what it says? You should be able to find it at pyilmbase-2.2.0/config.log.

Do we need to install any other packages besides the ones mentioned in your docker file?

I imagine that might depend on whether or not your base image has the same packages as the base image in the Docker file. But my guess would be that you should have everything you need already. What I can say definitively is that the Docker setup you see in GafferHQ/build is exactly what we use to build Gaffer and all of its dependencies.

Also, as you may have noticed above, we are overriding the default system python (which is 2.6) with 2.7 via the PATH and LD_LIBRARY_PATH env vars.

We did the same when we were building on CentOS 6, so that sounds reasonable...

 

[Sachin Shrestha]

Hi John,

Sorry, I forgot to post our findings from that log file. Yes, we did look at it and while I don't remember the specific details but off-hand I think I noticed that the python version variable was being passed an empty string instead of 2.7, the ilmbase include path was pointing to my local /usr/include/OpenEXR instead of the ones that got built into the dependencies and the ilmbase libs were also pointing to my local install paths. I could fix the last one by removing my system default pkgconfig file for ilmbase. But I couldn't override the openexr include paths. I even tried moving the directory but the build files still referred to this path.

Sorry for the hazy details but I can check tomorrow once I'm at work and provide more specifics.

Thanks again,

Sachin

[Sachin Shrestha]

Hi John,

I have attached the config.log file.  The python version is set as 2.7 but there are some other python related variables that are empty so not sure if that's a problem. There are these errors in the log:

Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-6/root/usr --mandir=/opt/rh/devtoolset-6/root/usr/share/man --infodir=/opt/rh/devtoolset-6/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --enable-plugin --with-linker-hash-style=gnu --enable-initfini-array --disable-libgcj --with-default-libstdcxx-abi=gcc4-compatible --with-isl=/builddir/build/BUILD/gcc-6.3.1-20170216/obj-x86_64-redhat-linux/isl-install --enable-libmpx --with-mpc=/builddir/build/BUILD/gcc-6.3.1-20170216/obj-x86_64-redhat-linux/mpc-install --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 6.3.1 20170216 (Red Hat 6.3.1-3) (GCC)
configure:2884: $? = 0
configure:2891: /repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/bin/g++ -V >&5
g++: error: unrecognized command line option '-V'
g++: fatal error: no input files
compilation terminated.

There is no -V flag for g++. Instead there is a -v or --verbose flag. Further down there are these errors:

configure:2921: checking for C++ compiler default output file name
configure:2943: /repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/bin/g++    conftest.cpp  >&5
/repo/gaffer/0.48.0.0/tools/opt/rh/devtoolset-6/root/usr/bin/../libexec/gcc/x86_64-redhat-linux/6.3.1/as: error while loading shared libraries: libopcodes-2.27-10.el6.1.so: cannot open shared object file: No such file or directory
configure:2947: $? = 1
configure:3018: checking whether the C++ compiler works
configure:3028: ./
./configure: line 3030: ./: is a directory
configure:3032: $? = 126
configure:3039: error: in `/repo/gaffer/0.48.0.0/build/dependencies/pyilmbase-2.2.0':
configure:3043: error: cannot run C++ compiled programs.

libopcodes-2.27-10.el6.1.so: cannot open shared object file: No such file or directory

Looks like this lib comes from another path in the devtoolset-6 repo which was not added in my env so I'm trying that now. Will let you know how it goes.

Cheers,
Sachin

[John Haddon]

Hi Sachin,

As you say, it looks like the problems you are having are related to differences between your build machine and the one I was using - the different compiler location and the existence of a system install of OpenEXR. Have you considered using the Docker setup directly to make your builds?

Cheers...

John

--

You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

John Haddon - R&D Programmer

Image Engine

studio: +1 (604) 874-5634 | jo...@image-engine.com | www.image-engine.com



15 West 5th Avenue, Vancouver, BC, V5Y 1H4, Canada

If you are not the intended recipient, disclosure, copying, distribution and use of this email is prohibited. Please notify us immediately and delete this email from your systems. You may contact us at in...@image-engine.com if you do not wish to receive further commercial electronic messages. We may still send you messages for which we do not require consent.

[Sachin Shrestha]

Hi John,

That is a possibility so I can try out a docker setup too. For now I’m building a separate build host for this, failing which I’ll most likely move to docker.

Cheers,
Sachin

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

John Haddon - R&D Programmer

Image Engine

studio: +1 (604) 874-5634 | jo...@image-engine.com | www.image-engine.com



15 West 5th Avenue, Vancouver, BC, V5Y 1H4, Canada

If you are not the intended recipient, disclosure, copying, distribution and use of this email is prohibited. Please notify us immediately and delete this email from your systems. You may contact us at in...@image-engine.com if you do not wish to receive further commercial electronic messages. We may still send you messages for which we do not require consent.

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.

[Sachin Shrestha]

Hi John,

I setup a docker container with your example dockerfile and ran the build for el6. I ran it interactively since there were some issues during the build which turned out to be the gxx info not being passed to the build script even though the devtoolset-6 paths were in the PATH env (and I was running the scl enable commands too). I had to manually set the CXX environment variable. Apart from this and a gcc error about strict-overflow (which also needed to be set explicitly instead of the build python script setting that, the build ran fine and i was able to finally compile for el6. However, when I try to run the compiled gaffer on CentOS 6.4, it throws this error (works fine on CentOS 7 though):

Traceback (most recent call last):
  File "/repo/RnD/software/gaffer/gaffer-0.48.0.0-linux-el6/bin/gaffer.py", line 52, in <module>
    from Gaffer._Gaffer import _nameProcess
  File "/repo/RnD/software/gaffer/gaffer-0.48.0.0-linux-el6/python/Gaffer/__init__.py", line 40, in <module>
    from _Gaffer import *
ImportError: /repo/RnD/software/gaffer/gaffer-0.48.0.0-linux-el6/lib/libGaffer.so: undefined symbol: _ZNSt6thread20hardware_concurrencyEv

I compared the libGaffer.so from 0.46.0.0 and this build and this is the output:

[sshrestha@pww0828 buildhost_vfx2018_el6]$ nm /repo/RnD/software/gaffer/gaffer-0.48.0.0-linux-el6/lib/libGaffer.so | grep _ZNSt6thread20hardware_concurrencyEv
                 U _ZNSt6thread20hardware_concurrencyEv
[sshrestha@pww0828 buildhost_vfx2018_el6]$ nm /repo/RnD/software/gaffer/gaffer-0.46.0.0-linux/lib/libGaffer.so | grep _ZNSt6thread20hardware_concurrencyEv
0000000000141870 T _ZNSt6thread20hardware_concurrencyEv

I'm not sure which library it is looking up that symbol in so am I missing any additional steps besides following the dockerfile example? My host machine was a CentOS7.5 machine and the docker container was built from the official centos:6 docker repo with additional packages from your dockerfile.

Cheers,
Sachin

[Daniel Dresser]

Hey - you may not get particularly prompt responses from most of the Gaffer team at the moment - I just got home from a Siggraph party.

But anyway, std::thread::hardware_concurrency comes from libstdc++

When I run:

ldd install/gaffer-0.47.0.0-linux/lib/libGaffer.so 

One of the lines says:

libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fcbfb081000)

Are you seeing libstdc++ not being found?

[Sachin Shrestha]

Hi Daniel,

Yes, I figured that the responses may be delayed this week due to Siggraph so no worries.

The libstdc++ is being reported properly in the ldd output:

libstdc++.so.6 => /opt/gcc-4.6.1/lib64/libstdc++.so.6 (0x00002b7c018d4000)

This is our default version in the CentOS6.4 image that we use in production. Both the official 0.46.0.0 build and my 0.48.0.0.el6 build report the same output, the former works while the latter doesn't.

Cheers,
Sachin

[Daniel Dresser]

The difference between your nm output for 0.46 and 0.48 is that the 0.46 build is defining this symbol for some reason - it's like it was statically linked to libstdc++ ( but also dynamically linked?  Weird.  I'm not really an expert in this stuff. )

What I don't get is why it wouldn't pick up this symbol properly at runtime anyway, since it is dynamically linking to libstdc++.

Just to confirm, when I run:

nm -D /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep hardware_concurrency

I get:

00000000000b8bf0 T _ZNSt6thread20hardware_concurrencyEv

What does:

nm -D /opt/gcc-4.6.1/lib64/libstdc++.so.6 | grep hardware_concurrency

return for you?

[Sachin Shrestha]

Hi Daniel,

I'm not at work right now so don't have access to my workstation but I have a stock CentOS6.4 setup at home for testing this build and it doesn't work there either (it reports the same problem). Here is the output from that:

uname -r

2.6.32-754.2.1.el6.x86_64

gcc -v
gcc version 4.4.7 20120313 (Red Hat 4.4.7-23) (GCC)

nm /usr/lib64/libstdc++.so.6 | grep hardware_concurrency
nm: /usr/lib64/libstdc++.so.6: no symbols

And strangely enough, if I run the same on my stock CentOS7.x machine where the compiled gaffer version works, it outputs the same:

uname -r

3.10.0-862.3.2.el7.x86_64

gcc -v
gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)

nm /usr/lib64/libstdc++.so.6 | grep hardware_concurrency
nm: /usr/lib64/libstdc++.so.6: no symbols

Further, checking this in the docker container environment that was used for the build, it reports even stranger results:

nm /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.so | grep hardware_concurrency
nm: /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.so: File format not recognized

The devtoolset-6 gets installed to those paths and  I don't see a libstdc++.so.6 library in there. The libstdc++.so seems to be either corrupt or not an actual .so library though. Running a strace on gaffer in both el6 and el7 seem to give some clues. There is a whole bunch of missing libs that show up in the el6 log and the paths that are being looked up don't seem to exist in the devtoolset-6 repo. I'll continue to investigate this.

Cheers,

Sachin

[Daniel Dresser]

OK, sounds like your strace might be the best clue so far.

If you want to use "nm" on those libs, you need to pass the "-D" flag, though.

[Sachin Shrestha]

Hi Daniel,

Sorry, I missed using the -D flag to nm. That certainly doesn't return anything on el6.

So, I rebuilt my CentOS 6.4 buildhost but this time I did not use docker. I only followed the docker instructions to install the required build dependencies. Post that both the dependencies build and the gaffer build went fine and now I have gaffer 0.48.1.0 build working on el6 without any issues. This el6 buildhost does not return anything for hardware_concurrency either when I run the nm command but despite that the compiled gaffer version  works fine this time. I'm not sure what was wrong with the previous el6 docker container but maybe some components were missing form the stock el6 container image. Will give the docker workflow another try at some point but at least I have the latest build working on el6 for now.

I still had to explicitly pass the devtoolset-6 gcc version to CXX in the scons file as it wasn't pickling it up from the PATH environment that was already set and I also had to skip the qtVersion check by explicitly setting it.

Cheers,

Sachin

[Sachin Shrestha]

Leaving the link to the build here if anyone else needs it for el6:

https://github.com/noizfactory/gaffer/releases/tag/0.48.1.0

Cheers,

Sachin