td-agent2 rpm on CentOS installs td-agent version 0.14.11

[Edward Adasiak]

Just installed td-agent in a CentOS 7 based docker container yesterday (1/25/2017) using the script:

curl -L https://toolbelt.treasuredata.com/sh/install-redhat-td-agent2.sh | sh 

It appears that the td-agent binary is 0.14.11 instead of the 0.12.31 version I was expecting:

[root@69fa4df8a363 /]# rpm -qa | grep td-agent

td-agent-2.3.4-0.el7.x86_64

[root@69fa4df8a363 /]# /usr/sbin/td-agent --version

td-agent 0.14.11

Anyone else notice this?

[Mr. Fiber]

I'm not sure what happen on your environment

because td-agent 2.3.4 on CentOS 7 includes fluentd v0.12.31.

You can check it by downloading rpm directly and use localinstall.

$ curl -O http://packages.treasuredata.com/2/redhat/7/x86_64/td-agent-2.3.4-0.el7.x86_64.rpm

$ yum localinstall td-agent-2.3.4-0.el7.x86_64.rpm

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bobby M.]

I wanted to extend this.  Currently the Amazon Linux 2016.09 releases for 2.3.3 and 2.3.4 are both Fluentd 14, even though the changelog specifies Fluentd 12.

I confirmed this now by downloading from the following links:

http://packages.treasuredata.com.s3.amazonaws.com/2/redhat/2016.09/x86_64/td-agent-2.3.4-1.el2016_09.x86_64.rpm

http://packages.treasuredata.com.s3.amazonaws.com/2/redhat/2016.09/x86_64/td-agent-2.3.3-0.el2016.x86_64.rpm

The redhat 7 packages appear to install the correct version of Fluentd 12x

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.

[Mr. Fiber]

I didn't reproduce it.

$ curl -O http://packages.treasuredata.com.s3.amazonaws.com/2/redhat/2016.09/x86_64/td-agent-2.3.4-1.el2016_09.x86_64.rpm
$ sudo yum localinstall td-agent-2.3.4-1.el2016_09.x86_64.rpm
$ sudo /usr/sbin/td-agent-gem list | grep fluentd
fluentd (0.12.31)
fluentd-ui (0.4.3)

This is my test result. I confirmed one amazon linux user about this problem

and he said there is no problem for now.

Do you have a cache mechanizm for rpm or downloading file?

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.

[Bobby M.]

It's possible it was resolved.  This is from a buffer from earlier today when I noticed it.  I pinned the version of `fluent-plugin-grok-parser` to 1.0.0 since it works/requires fluentd 12

/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:2112:in `raise_if_conflicts': Unable to activate fluent-plugin-grok-parser-1.0.0, because fluentd-0.14.11 conflicts with fluentd (>= 0.10.58, ~> 0.12.0) (Gem::ConflictError)
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:1280:in `activate'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems.rb:198:in `rescue in try_activate'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems.rb:195:in `try_activate'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:126:in `rescue in require'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:39:in `require'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-grok-parser-1.0.0/lib/fluent/plugin/parser_grok.rb:1:in `<top (required)>'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/registry.rb:94:in `block in search'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/registry.rb:92:in `each'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/registry.rb:92:in `search'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/registry.rb:43:in `lookup'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/plugin.rb:146:in `new_impl'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/plugin.rb:123:in `new_parser'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/plugin_helper/parser.rb:89:in `block in configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/plugin_helper/parser.rb:85:in `each'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/plugin_helper/parser.rb:85:in `configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/plugin/filter_parser.rb:41:in `configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/agent.rb:148:in `add_filter'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/agent.rb:68:in `block in configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/agent.rb:63:in `each'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/agent.rb:63:in `configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/root_agent.rb:86:in `configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/engine.rb:113:in `configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/engine.rb:87:in `run_configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/supervisor.rb:721:in `run_configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/supervisor.rb:495:in `dry_run'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/supervisor.rb:442:in `run_supervisor'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/lib/fluent/command/fluentd.rb:286:in `<top (required)>'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.14.11/bin/fluentd:5:in `<top (required)>'
        from /opt/td-agent/embedded/bin/fluentd:23:in `load'
        from /opt/td-agent/embedded/bin/fluentd:23:in `<top (required)>'
        from /usr/sbin/td-agent:7:in `load'
        from /usr/sbin/td-agent:7:in `<main>'

[Edward Adasiak]

Thanks for checking on this so quickly!  Looks like this was an issue with my environment after all.  I neglected to pin a gem to a particular version, and it appears that the latest version branch of it has a dependency on 0.14.x, while 0.12.x compatibility is handled via another (lesser) version branch.

Pinning the gem to the proper version resulted in the td-agent/fluentd version binary I was expecting.  Thanks again, and sorry for the false alarm!

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.

[Bobby M.]

I found the same issue.  Thanks for the follow up Edward. 

[James Pettit]

Any chance you guys could share the gem that had the 0.14 dependency? I'm using omnibus to create a build for my environment, and it keeps coming up as 0.14, rather than 0.12. 

[Bobby M.]

Apologies for just seeing this.  The gem was the webfs plugin.

[TamirR]

Is there a new update? I;m using the latest Amazon AMI and while trying this now, I get two versions.

Why two versions? and how can force to use only the 0.12.31 as in the past as this cause errors in the log:

$ /usr/sbin/td-agent-gem list | grep fluentd

fluentd (1.1.0, 0.12.31)

fluentd-ui (0.4.3)

$ td-agent --version

td-agent 1.1.0

From the log:

2018-01-24 08:53:19 +0000 [warn]: #0 emit transaction failed: error_class=ArgumentError error="time must be a Fluent::EventTime (or Integer): Float" location="/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.0/lib/fluent/plugin/output.rb:788:in `metadata'" tag="resource-staging-ecs-XXXXXXXXXXX"

2018-01-24 08:53:19 +0000 [warn]: #0 suppressed same stacktrace

These errors cause the td-agent not releasing memory and the agent get to more than ~50GB, that needed to killed manually.

Please your help.

Thanks