Failure to compile on FreeBSD 10.1

[J Mathew]

I'm using FreeBSD 10.1-RELEASE and I'm trying to compile arango db. From various old threads I've heard this is possible. 

I started with the guide here: 

https://docs.arangodb.com/Installing/Compiling.html

And downloaded a source tarball from here:

https://www.arangodb.com/download/

I believe I've installed all the dependencies, however make keeps failing with the following output. So my question is, does anyone know what is causing this failure?

make: "/usr/src/ArangoDB-2.7.0/Makefile" line 10489: warning: duplicate script for target "js/apps/system/_admin/aardvark/APP/frontend/js/modules/%.js" ignored

make: "/usr/src/ArangoDB-2.7.0/Makefile" line 10486: warning: using previous script for "js/apps/system/_admin/aardvark/APP/frontend/js/modules/%.js" defined here

make: don't know how to make etc/arangodb/arango-dfdb.conf. Stop

make: stopped in /usr/src/ArangoDB-2.7.0

The following is the output from configure:

configure: configure started in '/usr/src/ArangoDB-2.7.0

configure: with CC=''

configure: with CXX=''

configure: with CPPFLAGS=''

configure: with CFLAGS=''

configure: with CXXFLAGS=''

configure: with LDFLAGS=''

checking for go... go

configure: ................................................................................

configure: CHECKING BUILD SYSTEM

configure: ................................................................................

checking build system type... x86_64-unknown-freebsd10.1

checking host system type... x86_64-unknown-freebsd10.1

checking target system type... x86_64-unknown-freebsd10.1

checking for a BSD-compatible install... /usr/bin/install -c

checking whether build environment is sane... yes

checking for a thread-safe mkdir -p... config/install-sh -c -d

checking for gawk... no

checking for mawk... no

checking for nawk... nawk

checking whether make sets $(MAKE)... yes

checking whether make supports nested variables... yes

checking whether make supports nested variables... (cached) yes

configure: ................................................................................

configure: CHECKING C/C++ COMPILER AND LINKER

configure: ................................................................................

checking for style of include used by make... GNU

checking for g++... no

checking for c++... c++

checking whether the C++ compiler works... yes

checking for C++ compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C++ compiler... yes

checking whether c++ accepts -g... yes

checking dependency style of c++... gcc3

checking how to run the C++ preprocessor... c++ -E

checking for gcc... no

checking for clang... clang

checking whether we are using the GNU C compiler... yes

checking whether clang accepts -g... yes

checking for clang option to accept ISO C89... none needed

checking whether clang understands -c and -o together... yes

checking dependency style of clang... gcc3

checking whether we are using the GNU C++ compiler... (cached) yes

checking whether c++ accepts -g... (cached) yes

checking dependency style of c++... (cached) gcc3

checking how to run the C preprocessor... clang -E

checking whether ln -s works... yes

checking whether make sets $(MAKE)... (cached) yes

checking whether c++ supports C++11 features by default... no

checking whether c++ supports C++11 features with -std=c++11... yes

checking -Wstrict-null-sentinel for c++... no

checking -Qunused-arguments for c++... yes

checking -std=gnu90 for clang... yes

checking -std=c++11 for c++... yes

checking whether C++ has support for std::unordered_map::emplace()... yes

checking for ranlib... ranlib

checking for special C compiler options needed for large files... no

checking for _FILE_OFFSET_BITS value needed for large files... no

configure: ................................................................................

configure: CHECKING FOR PTHREADS

configure: ................................................................................

checking for the pthreads library -lpthreads... no

checking whether pthreads work without any flags... no

checking whether pthreads work with -Kthread... no

checking whether pthreads work with -kthread... no

checking for the pthreads library -llthread... no

checking whether pthreads work with -pthread... yes

checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE

checking if more special flags are required for pthreads... -D_THREAD_SAFE

checking broken __cxa_guard... no

configure: ................................................................................

configure: CHECKING FOR DOCUMENTATION UTILS

configure: ................................................................................

checking for markdown... no

checking for html2text... no

checking whether to enable maintainer-specific portions of Makefiles... no

checking for futimes... yes

checking for clock_gettime... yes

configure: --------------------------------------------------------------------------------

configure: CHECKING FOR MATH

configure: --------------------------------------------------------------------------------

checking for sincos in -lm... no

configure: --------------------------------------------------------------------------------

configure: CHECKING FOR READLINE

configure: --------------------------------------------------------------------------------

checking for grep that handles long lines and -e... /usr/bin/grep

checking for egrep... /usr/bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking readline/readline.h usability... yes

checking readline/readline.h presence... yes

checking for readline/readline.h... yes

checking for readline in -lreadline... yes

checking READLINE support... readline

checking READLINE version... 5.2

configure: --------------------------------------------------------------------------------

configure: CHECKING FOR OPENSSL

configure: --------------------------------------------------------------------------------

checking openssl/ssl.h usability... yes

checking openssl/ssl.h presence... yes

checking for openssl/ssl.h... yes

checking for CRYPTO_num_locks in -lcrypto... yes

checking for SSL_get_error in -lssl... yes

checking return type of SSLv23_method... SSL_METHOD*

checking OPENSSL version... "OpenSSL 1.0.1j-freebsd 15 Oct 2014"

checking whether your system knows about splice()... no

checking that generated files are newer than configure... done

configure: creating ./config.status

config.status: creating Makefile

config.status: creating Installation/MacOSX/Bundle/Info.plist

config.status: creating config/config.h

config.status: creating lib/Basics/local-configuration.h

config.status: executing depfiles commands

configure: --------------------------------------------------------------------------------

configure: BISON: disabled

configure: FIGURES: disabled

configure: FLEX: disabled

configure: GO BINARY: go

configure: HTML2TEXT support: disabled

configure: LOGGER: enabled

configure: MARKDOWN support: disabled

configure: RELATIVE PATHS: disabled

configure: TCMALLOC support: disabled

configure: TIMING: disabled

configure: --------------------------------------------------------------------------------

configure: READLINE VERSION: 5.2

configure: READLINE_CPPFLAGS: -DTRI_READLINE_VERSION='"5.2"'

configure: READLINE_LDFLAGS:

configure: READLINE_LIBS: -lreadline

configure: .

configure: OPENSSL VERSION: "OpenSSL 1.0.1j-freebsd 15 Oct 2014"

configure: OPENSSL_CPPFLAGS:

configure: OPENSSL_LDFLAGS:

configure: OPENSSL_LIBS: -lssl -lcrypto

configure: .

configure: BOOST: 3rdParty version

configure: BOOST VERSION: 1.58.0

configure: BOOST_CPPFLAGS: -I./3rdParty/boost/1.58.0

configure: .

configure: V8: 3rdParty version

configure: V8 VERSION: 4.3.61

configure: V8_TARGET: x64.release

configure: V8_CPPFLAGS: -I./3rdParty/V8-4.3.61/include -DTRI_V8_VERSION='"4.3.61"'

configure: V8_LDFLAGS:

configure: V8_LIBS: ./3rdParty/V8-4.3.61/out/x64.release/obj.target/tools/gyp/libv8_base.a ./3rdParty/V8-4.3.61/out/x64.release/obj.target/tools/gyp/libv8_libbase.a ./3rdParty/V8-4.3.61/out/x64.release/obj.target/tools/gyp/libv8_libplatform.a ./3rdParty/V8-4.3.61/out/x64.release/obj.target/tools/gyp/libv8_nosnapshot.a -lexecinfo

configure: .

configure: ICU: 3rdParty version

configure: ICU VERSION: 52.2

configure: ICU_CPPFLAGS: -D_REENTRANT -I./3rdParty/V8-4.3.61/third_party/icu/source/common -I./3rdParty/V8-4.3.61/third_party/icu/source/i18n -I./3rdParty/V8-4.3.61/third_party/icu/source/io

configure: ICU_LDFLAGS:

configure: ICU_LIBS: -ldl -lm ./3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicui18n.a ./3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicuuc.a ./3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicudata.a -ldl -lm

configure: .

configure: LIBEV: 3rdParty version

configure: LIBEV VERSION: 4.11

configure: LIBEV_CPPFLAGS: -I./3rdParty/libev -DTRI_LIBEV_VERSION='"4.11"'

configure: LIBEV_LDFLAGS:

configure: LIBEV_LIBS: ./3rdParty/libev/BUILD/.libs/libev.a

configure: .

configure: ZLIB: 3rdParty version

configure: ZLIB VERSION: 1.2.7

configure: ZLIB_CPPFLAGS:

configure: ZLIB_LDFLAGS:

configure: ZLIB_LIBS: ./3rdParty/zlib-1.2.7/libz.a

configure: --------------------------------------------------------------------------------

configure: REPOSITORY VERSION:

configure: BITS: 64

configure: CPPFLAGS:  -DTRI_BITS=64 -DHAVE_READLINE=1

configure: CFLAGS: -std=gnu90 -g -O2  -Wall -Winit-self -Wno-long-long -Wno-variadic-macros -Wshadow -Wstrict-prototypes -Wdeclaration-after-statement -D_THREAD_SAFE -pthread

configure: CXXFLAGS: -std=c++11 -g -O2 -std=c++11  -Wall -Winit-self -Wno-long-long -Wno-variadic-macros -Woverloaded-virtual -Qunused-arguments -D_THREAD_SAFE -pthread

configure: LDFLAGS:  -D_THREAD_SAFE -pthread

configure: LIBS:

configure: CC: clang

configure: CC VERSION: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512

Target: x86_64-unknown-freebsd10.1

Thread model: posix

configure: CXX: c++

configure: CXX VERSION: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512

Target: x86_64-unknown-freebsd10.1

Thread model: posix

configure: --------------------------------------------------------------------------------

[Wilfried Gösgens]

arango-dfdb.conf is just a configuration; so probably the make install fails for you?
Can you retry with V=1
so we get a little more output about whats it doing there?



[J Mathew]

Here you go:

prompt# make V=1

make: "/usr/src/ArangoDB-2.7.0/Makefile" line 10489: warning: duplicate script for target "js/apps/system/_admin/aardvark/APP/frontend/js/modules/%.js" ignored

make: "/usr/src/ArangoDB-2.7.0/Makefile" line 10486: warning: using previous script for "js/apps/system/_admin/aardvark/APP/frontend/js/modules/%.js" defined here

make: don't know how to make etc/arangodb/arango-dfdb.conf. Stop

Seems it doesnt produce more output.

[Jan]

I am not sure if the 3rd line in the make output is related to the first two, but the problem with the first two have been fixed in this commit:

    https://github.com/arangodb/arangodb/commit/38a6afc1e6f9c77fe2ba8d0a48b6d9e677390996

If you want to check it, you can pull this commit from the 2.7 branch if you like and apply it locally.
As it changes the Makefile, you may need to reconfigure the local installation like this:

rm -f .file-list-js .setup-js-directories  # clear some cache files
make setup
./configure --your-configure-options-go-here
make

I am not sure if that fixes anything, as make does not report a failure (not even the first two warnings) here.
Apart from that, we're not compiling and testing and supporting FreeBSD, so there may be additional issues when compiling ArangoDB.
Errors and warnings from compiling on FreeBSD are welcome though, and there's a chance that we can fix them even without having a FreeBSD instance around.

Best regards
Jan

[J Mathew]

Thanks Jan. That info helped me realize a few things.

So part of the problem is that I've been using 'make' which on freebsd is bsd make. 'make setup' fails because bsd make can't find the setup target. 

Switching to gmake gets me a bit further down the building road.

Then I realized I'm running on csh and switched to bash, just in case.

After all that I've got this new error:

--------------------------------------------------------------------------------

BUILDING V8

--------------------------------------------------------------------------------

cd ./3rdParty/V8-4.3.61 \

  export GYPFLAGS="-Dstandalone_static_library=1 -Dlinux_use_bundled_gold=0 -Dlinux_use_gold_flags=0" \

  && gmake \

        CC="clang" \

        CC.host="clang" \

        CXX="c++" \

        CXX.host="c++" \

        LINK="c++" \

        CFLAGS="-O3 -fomit-frame-pointer -g " \

        CXXFLAGS="-O3 -fomit-frame-pointer -g " \

        library=static strictaliasing=off snapshot=off werror=no x64.release

cd: too many arguments

Makefile:10555: recipe for target '.v8-build-64' failed

gmake: *** [.v8-build-64] Error 2

Changing export to '&& export' got me just a bit further. To yield my latest error:

--------------------------------------------------------------------------------

BUILDING V8

--------------------------------------------------------------------------------

cd ./3rdParty/V8-4.3.61 \

  && export GYPFLAGS="-Dstandalone_static_library=1 -Dlinux_use_bundled_gold=0 -Dlinux_use_gold_flags=0" \

  && gmake \

        CC="clang" \

        CC.host="clang" \

        CXX="c++" \

        CXX.host="c++" \

        LINK="c++" \

        CFLAGS="-O3 -fomit-frame-pointer -g " \

        CXXFLAGS="-O3 -fomit-frame-pointer -g " \

        library=static strictaliasing=off snapshot=off werror=no x64.release

gmake[1]: Entering directory '/usr/src/arangodb/3rdParty/V8-4.3.61'

PYTHONPATH="/usr/src/arangodb/3rdParty/V8-4.3.61/tools/generate_shim_headers:/usr/src/arangodb/3rdParty/V8-4.3.61/build::/usr/src/arangodb/3rdParty/V8-4.3.61/build/gyp/pylib:" \

GYP_GENERATORS=make \

build/gyp/gyp --generator-output="out" build/all.gyp \

              -Ibuild/standalone.gypi --depth=. \

              -Dv8_target_arch=x64 \

               -Dtarget_arch=x64 \

               \

              -S.x64.release -Dstandalone_static_library=1 -Dlinux_use_bundled_gold=0 -Dlinux_use_gold_flags=0 -Dv8_enable_backtrace=1 -Dv8_use_snapshot='false' -Dwerror='' -Dv8_no_strict_aliasing=1 -Darm_fpu=default -Darm_float_abi=default

build/gyp/gyp: not found

Makefile:425: recipe for target 'out/Makefile.x64.release' failed

gmake[1]: *** [out/Makefile.x64.release] Error 127

gmake[1]: Leaving directory '/usr/src/arangodb/3rdParty/V8-4.3.61'

Makefile:10555: recipe for target '.v8-build-64' failed

gmake: *** [.v8-build-64] Error 2

I'm not really sure what 'gyp' is and why it's not been built so I stopped there. Any ideas?

[J Mathew]

Since my problem is building the V8 library and bsd ports already has a V8 build. Can I just point to an existing V8 library? How do I do that? I checked the configure script for an option, but my untrained eyes don't see it. 

[Jan Steemann]

gyp is a Python-based "make"-like utility that's used by V8 (and others).

When ArangoDB builds V8, the V8 makefile will invoke the gyp binary.

I am not sure how to install this on FreeBSD, but on Ubuntu it's as simple as `sudo apt-get install gyp`.

Best regards

Jan

--
You received this message because you are subscribed to the Google Groups "ArangoDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arangodb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jan Steemann]

I don't think there is currently an option to link ArangoDB against an existing V8 library on the system.
We had such option a while ago but removed it, because the V8 library versions shipped with most systems are hopelessly outdated, and because we require a specific version of V8 to build ArangoDB. We're using several of its newer features that simply are not available in many of the packaged versions.

Best regards

Jan

[J Mathew]

I havent tested it but I finally got it to build. Thanks for all the help!

All of the problems I had were relatively simple, but in case anyone else is curious I've provided a summary below.

Summary of changes:

• Install additional ports:
• clang37 (Might not be required)
• gyp
• Remove test targets from V8  build/all.gyp as per:
  
• https://github.com/arangodb/arangodb/issues/1272
• Change bash path from /bin/bash to /usr/local/bin/bash in various files. I ended up just making link.
• grep -rl '#\!\/bin\/bash' . # Finds problematic scripts
• ln -s /usr/local/bin/bash /bin/bash
• Modify Makefile (after configure) to do' cd && export' instead of 'cd export' when building V8 as per my first post.
• Replace -ld flags from Makefile with -lc. 
• Don't have the error output but it looks like: /usr/bin/ld cannot find -ldl
• According to the dlopen docs its in libc
• Set environment vars to use Clang37
• V8 recommends at least clang 35 
• 'Manually' build V8 library by calling 'gmake x64', prior to main build. This doesn't seem to get built automatically for some reason.

Basic Build Method:

export CC=/usr/local/bin/clang37
export CXX=/usr/local/bin/clang++37
export GYP_DEFINES=clang=1
cd  3rdParty/V8-4.3.61/
gmake clean
cd ../..
gmake clean
gmake setup
./configure --enable-relative
...Modify make file...
cd  3rdParty/V8-4.3.61/
gmake x64
cd ../..
gmake

The following is a list of build errors I ran into for the benefit of any Googlers.

This is the build error for a missing x64.release folder

In file included from arangod/Wal/RecoverState.h:34:

In file included from ./arangod/Utils/transactions.h:34:

./arangod/Utils/CollectionNameResolver.h:242:28: warning: moving a temporary object prevents copy elision

      [-Wpessimizing-move]

          std::string name(std::move(getCollectionName(cid)));

                           ^

./arangod/Utils/CollectionNameResolver.h:242:28: note: remove std::move call here

          std::string name(std::move(getCollectionName(cid)));

                           ^~~~~~~~~~                      ~

1 warning generated.

  CXX      arangod/Wal/arangod_libarangod_a-RemoverThread.o

  CXX      arangod/Wal/arangod_libarangod_a-Slot.o

  CXX      arangod/Wal/arangod_libarangod_a-Slots.o

  CXX      arangod/Wal/arangod_libarangod_a-SynchronizerThread.o

  AR       arangod/libarangod.a

  CXX      arangosh/ArangoShell/bin_arangob-ArangoClient.o

  CXX      arangosh/Benchmark/bin_arangob-arangob.o

  CXXLD    bin/arangob

clang-3.7: error: no such file or directory: './3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicuii8n.a'

clang-3.7: error: no such file or directory: './3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicuuu.a'

clang-3.7: error: no such file or directory: './3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicuddta.a'

clang-3.7: error: no such file or directory: './3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicuii8n.a'

clang-3.7: error: no such file or directory: './3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicuuu.a'

clang-3.7: error: no such file or directory: './3rdParty/V8-4.3.61/out/x64.release/obj.target/third_party/icu/libicuddta.a'

Makefile:3422: recipe for target 'bin/arangob' failed

gmake[1]: *** [bin/arangob] Error 1

gmake[1]: Leaving directory '/usr/src/arangodb'

Makefile:2062: recipe for target 'all' failed

gmake: *** [all] Error 2

This is the build error for the wrong bash path while building ETCD

  CXXLD    bin/arangosh

  CXX      arangosh/ArangoShell/bin_arangoimp-ArangoClient.o

  CXX      arangosh/V8Client/bin_arangoimp-ImportHelper.o

  CXX      arangosh/V8Client/bin_arangoimp-arangoimp.o

  CXXLD    bin/arangoimp

  CXX      arangod/RestServer/bin_arangod-arangod.o

In file included from arangod/RestServer/arangod.cpp:37:

In file included from ./arangod/RestServer/ArangoServer.h:39:

In file included from ./arangod/Aql/QueryRegistry.h:33:

In file included from ./arangod/Aql/Query.h:40:

In file included from ./arangod/Utils/AqlTransaction.h:37:

./arangod/Utils/CollectionNameResolver.h:242:28: warning: moving a temporary object prevents copy elision

      [-Wpessimizing-move]

          std::string name(std::move(getCollectionName(cid)));

                           ^

./arangod/Utils/CollectionNameResolver.h:242:28: note: remove std::move call here

          std::string name(std::move(getCollectionName(cid)));

                           ^~~~~~~~~~                      ~

1 warning generated.

  CXXLD    bin/arangod

--------------------------------------------------------------------------------

BUILDING etcd

--------------------------------------------------------------------------------

cd ./3rdParty/etcd && ./build

./build: not found

Makefile:10726: recipe for target 'bin/etcd-arango' failed

gmake[1]: *** [bin/etcd-arango] Error 127

gmake[1]: Leaving directory '/usr/src/arangodb'

Makefile:2062: recipe for target 'all' failed

gmake: *** [all] Error 2

When building V8 there's a minor error. Changing 'reinterpret_cast' to 'static_cast' seems to fix it.

../src/runtime/runtime-i18n.cc:630:37: error: reinterpret_cast from 'nullptr_t' to 'v8::internal::Smi *' is not

      allowed

  local_object->SetInternalField(1, reinterpret_cast<Smi*>(NULL));

                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

Other things I noticed:

• I've been doing all of this on the 2.7 branch. But I briefly jumped to the devel branch to try and build the current. However, it looks like the BSD #defines are not there like they are in the 2.7 branch. Any reason that's the case?
  
• After switching to the 2.7 branch and running 'gmake clean'. It ran configure. Afterwards it ran only the rm commands. Not important. Just unexpected.

[Frank Celler]

Hi J Mathew,

thanks a lot for your instructions on how to compile on FreeBSD. I'll try to setup a FreeBSD myself and integrate it into the source code into the devel branch. I let you know, if I succeed.

best

  Frank

[Szczepan Bednarz]

Hi J Mathew,

Thanks for hints how to compile Arangodb on FreeBSD. I have used them as a good starting point.

My setup is FreeBSD 10.2-RELEASE-p2 without root privileges, and Arangodb 2.7.3.

Here is my (successful) method. I may miss something, but more or less is what I did:

 

1. Compilation & local instalation of gcc 5.3 system

2. Force to use the gcc toolchain, set CXX and CC to gcc 5.3 compilers paths

3. Add to CFLAGS and CXXFLAGS: -D_GLIBCXX_USE_C99

4. Add to LDFLAGS: -fno-use-linker-plugin

5. ./configure

6. Modification of Makefile: cd ./3rdParty/$(V8DIR) &&\ ('&&' characters, line ~10735)

7. Remove ‘-ldl’ from the Makefile

8. Modification of 3rdParty/V8-4.3.61/build/all.gyp: remove tests targets

9. Add a line ‘#include <stdio.h>’ to '3rdParty/V8-4.3.61/src/base/platform/platform-freebsd.cc'

10. gmake && gmake install 

 

Now I’m a happy Arangodb user on FreeBSD machine:

INFO ArangoDB 2.7.3 [freebsd] 64bit, using ICU 54.1, V8 4.3.61, OpenSSL 1.0.2d 9 Jul 2015

 

Regards,

Szczepan