Statically linked envoy binary?

[verve.ne...@gmail.com]

Hello,

We are currently evaluating envoy for use within our infrastructure. 

Envoy's build dependencies are quite contemporary and so far docker has been necessary to get it to run since it required glibc 2.18 (CentOS 7 has glibc 2.17 and CentOS 6.8 has glibc 2.12).

One other possibility that came to mind was making a statically linked binary, but while investigating this it was found that envoy is linked to "libdl" -- and I'm wondering if that would preclude static linkage? No?

Thanks

[Matt Klein]

We are discussing this now in Gitter. We can circle back here once there is some resolution.

--
You received this message because you are subscribed to the Google Groups "Envoy Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/4542052f-3cac-40fd-bd6b-f77f4e7d467c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Klein

Software Engineer

mkl...@lyft.com / 206.327.4515

[bill.h...@gmail.com]

On Monday, October 31, 2016 at 3:31:17 PM UTC-7, Matt Klein wrote:

We are discussing this now in Gitter. We can circle back here once there is some resolution.

Has there been any update? I just built using:

• bazel.release.server_only — build Envoy static binary under -c opt with gcc.

and see

ldd /tmp/envoy-docker-build/envoy/source/exe/envoy

/tmp/envoy-docker-build/envoy/source/exe/envoy: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /tmp/envoy-docker-build/envoy/source/exe/envoy)

linux-vdso.so.1 =>  (0x00007ffdfcbfe000)

librt.so.1 => /lib64/librt.so.1 (0x00007fb71a9bb000)

libm.so.6 => /lib64/libm.so.6 (0x00007fb71a6b9000)

libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb71a49c000)

libc.so.6 => /lib64/libc.so.6 (0x00007fb71a0da000)

/lib64/ld-linux-x86-64.so.2 (0x0000558888a06000)

 

On Mon, Oct 31, 2016 at 3:04 PM, <verve.ne...@gmail.com> wrote:

Hello,

We are currently evaluating envoy for use within our infrastructure. 

Envoy's build dependencies are quite contemporary and so far docker has been necessary to get it to run since it required glibc 2.18 (CentOS 7 has glibc 2.17 and CentOS 6.8 has glibc 2.12).

One other possibility that came to mind was making a statically linked binary, but while investigating this it was found that envoy is linked to "libdl" -- and I'm wondering if that would preclude static linkage? No?

Thanks

--
You received this message because you are subscribed to the Google Groups "Envoy Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/4542052f-3cac-40fd-bd6b-f77f4e7d467c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Klein]

No update. I strongly recommend not fully statically linking. I've run into problems in the past trying to do this. Basically, you really need to know what you are doing to have this work and not run into any production issues. 

The only type of static linking we officially support is what is done in our official CI builds. If you want to go beyond this you are on your own.

Thanks,

Matt

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/624d06a9-171c-4eec-bbfc-4479a6d44cd8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[steve...@gmail.com]

Matt,

Is it possible for Envoy to use glibc 2.17?  Currently in Istio we are seeing people wanting to custom build glibc2.18 in the implementation.  I am concerned this will leak over into a scenario where both glibc 2.17 and glibc2.18 are required on the system to support CentOS 7 environments.

Reference:

https://github.com/istio/istio/pull/9391#issuecomment-431248604

[Matt Klein]

Sorry no idea. I would give it a shot and see what breaks.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/b0474df5-a13d-498f-8026-e588d97d4e4d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Matt Klein

Software Engineer

mkl...@lyft.com

https://calendly.com/mattklein123

[Lizan Zhou]

I commented in the Istio issue, from objdump looks like the only symbol envoy needed in glibc 2.18 is

objdump -T bazel-bin/source/exe/envoy-static | grep GLIBC_2.18

0000000000000000      DF *UND*  0000000000000000  GLIBC_2.18  __cxa_thread_atexit_impl

There are similar discussion in Chromium:

https://bugs.chromium.org/p/chromium/issues/detail?id=749077

tl;dr: This is because of libstdc++ linked against newer version of glibc, will try to utilize __cxa_thread_atexit_impl to when it is available (to support thread_local C++11 scope qualifier)

So the solution could be similar to what Chromium ended up (which is pretty ugly): replacing the symbol in libc.so.6 so libstdc++ won't utilize it from glibc.

Or just build with older version of glibc.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CA%2BOFKguv-%2Bx85chrM6xWUBY-bNau7qMLKe2ebAfNiaqW1Pma1w%40mail.gmail.com.

[gn...@twitter.com]

If i'm reading this correctly, it seems like as long as we're building envoy from source on CentOS 7, glibc >= 2.18 isn't really a hard requirement and we should be ok with 2.17? I suppose if the build succeeds, that's as good a confirmation as any that as far as a successful build is concerned, we're ok. Want to check if there are any obvious runtime implications of this though. From Lizan's response, it seems like there's just the one symbol that's needed from 2.18. Infact even on a binary built against 2.17, the following symbol seems like the only one used:

0000000000000000      DF *UND* 0000000000000000  GLIBC_2.17  secure_getenv

Thanks!

On Friday, October 19, 2018 at 8:02:39 PM UTC-7, Lizan Zhou wrote:

I commented in the Istio issue, from objdump looks like the only symbol envoy needed in glibc 2.18 is

objdump -T bazel-bin/source/exe/envoy-static | grep GLIBC_2.18

0000000000000000      DF *UND*  0000000000000000  GLIBC_2.18  __cxa_thread_atexit_impl

There are similar discussion in Chromium:

https://bugs.chromium.org/p/chromium/issues/detail?id=749077

tl;dr: This is because of libstdc++ linked against newer version of glibc, will try to utilize __cxa_thread_atexit_impl to when it is available (to support thread_local C++11 scope qualifier)

So the solution could be similar to what Chromium ended up (which is pretty ugly): replacing the symbol in libc.so.6 so libstdc++ won't utilize it from glibc.

Or just build with older version of glibc.

On Fri, Oct 19, 2018 at 5:46 PM 'Matt Klein' via envoy-users <envoy...@googlegroups.com> wrote:

Sorry no idea. I would give it a shot and see what breaks.

On Fri, Oct 19, 2018 at 6:31 AM <steve...@gmail.com> wrote:

Matt,

Is it possible for Envoy to use glibc 2.17?  Currently in Istio we are seeing people wanting to custom build glibc2.18 in the implementation.  I am concerned this will leak over into a scenario where both glibc 2.17 and glibc2.18 are required on the system to support CentOS 7 environments.

Reference:

https://github.com/istio/istio/pull/9391#issuecomment-431248604

On Monday, October 31, 2016 at 3:04:48 PM UTC-7, verve.ne...@gmail.com wrote:

Hello,

We are currently evaluating envoy for use within our infrastructure. 

Envoy's build dependencies are quite contemporary and so far docker has been necessary to get it to run since it required glibc 2.18 (CentOS 7 has glibc 2.17 and CentOS 6.8 has glibc 2.12).

One other possibility that came to mind was making a statically linked binary, but while investigating this it was found that envoy is linked to "libdl" -- and I'm wondering if that would preclude static linkage? No?

Thanks

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/b0474df5-a13d-498f-8026-e588d97d4e4d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Klein

Software Engineer

mkl...@lyft.com

https://calendly.com/mattklein123

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy...@googlegroups.com.

[Lizan Zhou]

Yes that's right. But the problem of CentOS 7 is its libstdc++ doesn't have C++11 ABI so the std::string isn't thread-safe (See https://github.com/envoyproxy/envoy/blob/master/source/common/common/compiler_requirements.h#L12). So ideally building with libc++ (You have to build your own LLVM toolchain for this though) on CentOS 7 is better, that's how we build packages for GetEnvoy.io.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/06cebbdd-06cc-4a64-bbf5-2661ded9aedc%40googlegroups.com.