fleuntd kubernetes_metadata plugin

[Anil]

Hello,

I am trying to install plugin fluent-plugin-kubernetes_metadata_filter  (2.6.0) in fluentd docker image v1.12.0-debian-1.0 and it failed with following error log -

Building native extensions. This could take a while...

ERROR:  Error installing fluent-plugin-kubernetes_metadata_filter:

ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/unf_ext-0.0.7.7/ext/unf_ext

/usr/local/bin/ruby -I /usr/local/lib/ruby/2.6.0 -r ./siteconf20210303-21-1sa0zta.rb extconf.rb

checking for -lstdc++... *** extconf.rb failed ***

Could not create Makefile due to some reason, probably lack of necessary

libraries and/or headers.  Check the mkmf.log file for more details.  You may

need configuration options.

Provided configuration options:

--with-opt-dir

--without-opt-dir

--with-opt-include

--without-opt-include=${opt-dir}/include

--with-opt-lib

--without-opt-lib=${opt-dir}/lib

--with-make-prog

--without-make-prog

--srcdir=.

--curdir

--ruby=/usr/local/bin/$(RUBY_BASE_NAME)

--with-static-libstdc++

--without-static-libstdc++

--with-stdc++lib

--without-stdc++lib

/usr/local/lib/ruby/2.6.0/mkmf.rb:467:in `try_do': The compiler failed to generate an executable file. (RuntimeError)

You have to install development tools first.

from /usr/local/lib/ruby/2.6.0/mkmf.rb:552:in `try_link0'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:570:in `try_link'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:789:in `try_func'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:1016:in `block in have_library'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:959:in `block in checking_for'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:361:in `block (2 levels) in postpone'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:331:in `open'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:361:in `block in postpone'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:331:in `open'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:357:in `postpone'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:958:in `checking_for'

from /usr/local/lib/ruby/2.6.0/mkmf.rb:1011:in `have_library'

from extconf.rb:6:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /usr/local/bundle/extensions/x86_64-linux/2.6.0/unf_ext-0.0.7.7/mkmf.log

extconf failed, exit code 1

Could you help me identify and fix the issue ? i tried to install ruby-dev and no luck. the one possibility is to create new docker image with required extensions mentioned in the following luck.

https://docs.fluentd.org/deployment/plugin-management#gem-and-native-extension

Regards,

Anil

[Thomas Müller]

Anil schrieb am Mittwoch, 3. März 2021 um 23:42:53 UTC+1:

Hello,

I am trying to install plugin fluent-plugin-kubernetes_metadata_filter  (2.6.0) in fluentd docker image v1.12.0-debian-1.0 and it failed with following error log -

Building native extensions. This could take a while...

ERROR:  Error installing fluent-plugin-kubernetes_metadata_filter:

ERROR: Failed to build gem native extension.

    current directory: /usr/local/bundle/gems/unf_ext-0.0.7.7/ext/unf_ext

/usr/local/bin/ruby -I /usr/local/lib/ruby/2.6.0 -r ./siteconf20210303-21-1sa0zta.rb extconf.rb

checking for -lstdc++... *** extconf.rb failed ***

...

extconf failed, exit code 1

Could you help me identify and fix the issue ? i tried to install ruby-dev and no luck. the one possibility is to create new docker image with required extensions mentioned in the following luck.

it looks like it fails on "stdc++"  check. possibly install some dev package of libstdc++

- Thomas

[Anil]

HI Thomas,

Thanks for the response.

i tried install std++ and libstd++ and no luck.

Adding details of gem_make.out -

"gcc -o conftest -I/usr/local/include/ruby-2.6.0/x86_64-linux -I/usr/local/include/ruby-2.6.0/ruby/backward -I/usr/local/include/ruby-2.6.0 -I.    -O3 -ggdb3 -Wall -Wextra -Wdeclaration-after-statement -Wdeprecated-declarations -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wrestrict -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable  -fPIC conftest.c  -L. -L/usr/local/lib -Wl,-rpath,/usr/local/lib -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic     -Wl,-rpath,/usr/local/lib -L/usr/local/lib -lruby  -lm   -lc"

checked program was:

/* begin */

1: #include "ruby.h"

2:

3: int main(int argc, char **argv)

4: {

5:   return 0;

6: }

/* end */ 

Regards,

Anil

[Kentaro Hayashi]

Hi, maybe the compiler and related toolchain is not installed.

sudo apt install -y libstdc++-8-dev gcc make g++

Regards,

2021年3月5日金曜日 1:59:41 UTC+9 Anil:

[Anil]

HI Ken,

Thanks for the response. 

sudo apt install is not working with fluentd container because it is setting the user as fluent.

The only option i see is to create a new docker image from base docker and install required plugins with root user and it is working.

Please let me know if you have different opinion. 

Regards

[Kentaro Hayashi]

Hi,

The following example command may work as you expected when running container.

docker exec -u root (YOUR_CONTAINER_ID) apt install -y libstdc++-8-dev gcc make g++

docker exec -u root (YOUR_CONTAINER_ID) gem install fluent-plugin-kubernetes_metadata_filter

Regards,

2021年3月19日金曜日 0:17:59 UTC+9 Anil:

[Anil]

HI Ken,

i can't use explicit docker exec command as i am using kubernetes and it requires installation of plugins before pipeline start processing the logs. 

As i said before, sudo command is also not working in default fluentd docker image. 

Regards

[cosmo09...@gmail.com]

Hi,

To use within k8s, building custom image with your defined Dockerfile is better instead of installing Fluentd plugin on initContainers.

Please refer to the document: https://github.com/fluent/fluentd-docker-image#how-to-build-your-own-image

Cheers,

Hiroshi

2021年3月23日火曜日 2:44:48 UTC+9 Anil: