Static Linking or use Folly without install to the system

[jeffp]

Hi,

We are doing a POC with https://github.com/facebook/UdpPinger on Ubuntu 1404.  which depends on folly.  

We want to deploy the UdpPinger binary with pre built package, which either statically link folly library.  or can just copy the folly pre built one and then point the folly library from a local folder without installation. 

We want to install the UdpPinger without installing folly to the local system. Is there a way to achieve this? Can we copy a pre built Folly binary and let the UdpPinger to point to that?

Thanks,

Jeff

[Milosz Tanski]

Jeff,

The folly build system (automake/autoconf) does build static libaries and install them in $PREFIX during make install. You would need to locally change the UdpPinger's Makefile to link against the static libraries.

If you want to link against static folly, but dynamic other libraries you would need to change the UdpPinger Make file to look something like this:

- LDLIBS=-lfolly -lgflags -lpthread -lthrift -ldouble-conversion -lglog

+ LDLIBS=$PREFIX/libfolly.a -lgflags -lpthread $PREFIX/libthrift.a -ldouble-conversion -lglog

Keep in mind I didn't test this, and if folly or any other static library has additional dependencies you will have to track them down and include them as well.

Here's some more information on how to deal with this:

http://stackoverflow.com/questions/4156055/gcc-static-linking-only-some-libraries

Best,

- Milosz

--

---
You received this message because you are subscribed to the Google Groups "Folly: the Facebook Open-source LibrarY" group.
To unsubscribe from this group and stop receiving emails from it, send an email to facebook-foll...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: mil...@adfin.com

[jeffp]

Thanks so much Milosz for the timely reply. We will try it out and let you know!

Thanks,

Jeff

[jeffp]

Do you know a quickly to figure out which lib the libfolly.a also depends on?  ....  I tried to add all of those Demangle.o to Makefile but seems not working. 

Alternatively, can we try to use LD_LIBRARY_PATH and load the dependency without static linking?

~/netmon/UdpPinger/upong$ make

g++ -g -Igen-cpp -std=c++11 NoradTargetThread.o Main.cpp -o upong libfolly.a -lgflags -lpthread -lglog

libfolly.a(Demangle.o): In function `folly::demangle(char const*, char*, unsigned long)':

/home/ypei/folly/folly/Demangle.cpp:105: undefined reference to `cplus_demangle_v3_callback'

/home/ypei/folly/folly/Demangle.cpp:105: undefined reference to `cplus_demangle_v3_callback'

libfolly.a(Format.o): In function `folly::FormatValue<double, void>::formatHelper(folly::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, folly::fbstring_core<char> >&, int&, folly::FormatArg&) const':

/home/ypei/folly/folly/Format.cpp:99: undefined reference to `double_conversion::DoubleToStringConverter::ToFixed(double, int, double_conversion::StringBuilder*) const'

libfolly.a(Format.o): In function `double_conversion::DoubleToStringConverter::ToShortest(double, double_conversion::StringBuilder*) const':

/usr/include/double-conversion/double-conversion.h:158: undefined reference to `double_conversion::DoubleToStringConverter::ToShortestIeeeNumber(double, double_conversion::StringBuilder*, double_conversion::DoubleToStringConverter::DtoaMode) const'

libfolly.a(Format.o): In function `folly::FormatValue<double, void>::formatHelper(folly::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, folly::fbstring_core<char> >&, int&, folly::FormatArg&) const':

/home/ypei/folly/folly/Format.cpp:118: undefined reference to `double_conversion::DoubleToStringConverter::ToExponential(double, int, double_conversion::StringBuilder*) const'

libfolly.a(String.o): In function `std::enable_if<std::is_floating_point<double>::value, double>::type folly::to<double>(folly::Range<char const*>*)':

/home/ypei/folly/folly/./../folly/Conv.h:1142: undefined reference to `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int, int*) const'

collect2: error: ld returned 1 exit status

make: *** [upong] Error 1

//here we found the .a file

~/netmon/folly/folly/.libs$ ls

Benchmark.o          json.o                        MacAddress.o

Bits.o               libfolly.a                    MemoryMapping.o

Checksum.o           libfollybase.a                Random.o

Conv.o               libfollybase.la               SafeAssert.o

Demangle.o           libfollybenchmark.a           SharedMutex.o

dynamic.o            libfollybenchmark.la          Singleton.o

EscapeTables.o       libfollybenchmark.lai         SocketAddress.o

File.o               libfollybenchmark.so          SpookyHashV1.o

FileUtil.o           libfollybenchmark.so.57       SpookyHashV2.o

FingerprintTables.o  libfollybenchmark.so.57.0.0   StringBase.o

Format.o             libfollybenchmark.so.57.0.0T  String.o

FormatTables.o       libfolly.la                   Subprocess.o

GroupVarint.o        libfolly.lai                  ThreadCachedArena.o

GroupVarintTables.o  libfolly.so                   TimeoutQueue.o

IPAddress.o          libfolly.so.57                Unicode.o

IPAddressV4.o        libfolly.so.57.0.0            Uri.o

IPAddressV6.o        LifoSem.o                     Version.o

Thanks,

[Milosz Tanski]

On Tue, Mar 8, 2016 at 4:26 PM, jeffp <jeff...@gmail.com> wrote:

Do you know a quickly to figure out which lib the libfolly.a also depends on?  ....  I tried to add all of those Demangle.o to Makefile but seems not working. 

Alternatively, can we try to use LD_LIBRARY_PATH and load the dependency without static linking?

~/netmon/UdpPinger/upong$ make

g++ -g -Igen-cpp -std=c++11 NoradTargetThread.o Main.cpp -o upong libfolly.a -lgflags -lpthread -lglog

libfolly.a(Demangle.o): In function `folly::demangle(char const*, char*, unsigned long)':

/home/ypei/folly/folly/Demangle.cpp:105: undefined reference to `cplus_demangle_v3_callback'

/home/ypei/folly/folly/Demangle.cpp:105: undefined reference to `cplus_demangle_v3_callback'

libfolly.a(Format.o): In function `folly::FormatValue<double, void>::formatHelper(folly::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, folly::fbstring_core<char> >&, int&, folly::FormatArg&) const':

/home/ypei/folly/folly/Format.cpp:99: undefined reference to `double_conversion::DoubleToStringConverter::ToFixed(double, int, double_conversion::StringBuilder*) const'

libfolly.a(Format.o): In function `double_conversion::DoubleToStringConverter::ToShortest(double, double_conversion::StringBuilder*) const':

/usr/include/double-conversion/double-conversion.h:158: undefined reference to `double_conversion::DoubleToStringConverter::ToShortestIeeeNumber(double, double_conversion::StringBuilder*, double_conversion::DoubleToStringConverter::DtoaMode) const'

libfolly.a(Format.o): In function `folly::FormatValue<double, void>::formatHelper(folly::basic_fbstring<char, std::char_traits<char>, std::allocator<char>, folly::fbstring_core<char> >&, int&, folly::FormatArg&) const':

/home/ypei/folly/folly/Format.cpp:118: undefined reference to `double_conversion::DoubleToStringConverter::ToExponential(double, int, double_conversion::StringBuilder*) const'

libfolly.a(String.o): In function `std::enable_if<std::is_floating_point<double>::value, double>::type folly::to<double>(folly::Range<char const*>*)':

/home/ypei/folly/folly/./../folly/Conv.h:1142: undefined reference to `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int, int*) const'

collect2: error: ld returned 1 exit status

make: *** [upong] Error 1

Static libraries don't encode their dependencies. There's some solutions to this like pkgconfig (where it can save it's static dependencies). 

What I usually do is use ldd on the dynamic version to see what it depends on and include those as well. It's not a 100% 1:1 between the static and dynamic one but close enough.

mtanski@buildbox:~$ ldd /usr/local/lib/libfolly.so

linux-vdso.so.1 =>  (0x00007ffec0be2000)

libboost_context.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_context.so.1.58.0 (0x00007f0dd2581000)

libboost_program_options.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_program_options.so.1.58.0 (0x00007f0dd2303000)

libboost_thread.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0 (0x00007f0dd20dc000)

libboost_filesystem.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.58.0 (0x00007f0dd1ec4000)

...

With this information in hand you can line up the symbol errors with their libraries.

[jeffp]

Thanks so much Milosz,

This is a 20+item long list and I feel it depends on many kernel  share libraries. Also they are .so share libraries rather than the .a static ones.  Can we still add those .so into the make file?  These so are with the kernel I think and are so deeply integrated with the system. 

Was reading : http://www.network-theory.co.uk/docs/gccintro/gccintro_25.html

Do you if know we can build a static folly with all its dependencies;  

or build sth like a JVM or python virtual environment that is independent from the system's glibc library? 

Thanks,

Jeff

[jeffp]

OK....

figured it out: this is working for upong: also the order matters

LDLIBS=libfolly.a /usr/lib/x86_64-linux-gnu/libiberty.a /usr/lib/x86_64-linux-gnu/libdouble-conversion.a -lgflags -lpthread -lglog