Webhdfs with Kerberos

16 views
Skip to first unread message

Mujeeb Baig

unread,
Jun 5, 2018, 9:02:04 AM6/5/18
to Fluentd Google Group
Hi,

I am trying to store logs on HDFS with Kerberos enabled, but getting hit by gssapi file loading error (see attached file).
I am forwarding docker logs to HDFS

This is my td-config section for webhdfs

 
# Output
<match docker.*>
  type copy
  <store>
    @type file
    path /test/
  </store>
  <store>
    @type webhdfs
    host  192.168.12.30
    port  50070
    path  /logs/es_cluster/access.log.%Y%m%d_%H.#{Socket.gethostname}.log
    kerberos true
    kerberos_keytab /etc/security/fluentd-user.headless.keytab
  </store>
</match>

I did tried with
gem install gssapi
on log aggregator vm but no success.

Am I missing something?
Thanks in Advance
Mujeeb
fluentd-log-aggregator.txt

Mr. Fiber

unread,
Jun 6, 2018, 6:26:56 AM6/6/18
to Fluentd Google Group
You need to check you install gssapi is installed in td-agent's ruby, not system ruby.



Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mujeeb Baig

unread,
Jun 6, 2018, 7:14:17 AM6/6/18
to Fluentd Google Group
Thanks for reply Masahiro,

I tried to install as follows
sudo /usr/sbin/td-agent-gem install gssapi
but got error (see attached file)

I am new to Ruby, haven't used it before!!! Looks like I am missing something in installation

Thanks


On Wednesday, June 6, 2018 at 11:26:56 AM UTC+1, repeatedly wrote:
You need to check you install gssapi is installed in td-agent's ruby, not system ruby.



MasahiroI

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
fluentd-gem-install-gssapi-error.txt

Mujeeb Baig

unread,
Jun 6, 2018, 7:26:34 AM6/6/18
to Fluentd Google Group
mkmf.log attached, complaining about libffi is not found
fluentd-gem-install-gssapi-error-mkmf.txt

Mujeeb Baig

unread,
Jun 6, 2018, 7:54:45 AM6/6/18
to Fluentd Google Group
yum install gcc
resolved the gssapi issue.



On Wednesday, June 6, 2018 at 12:14:17 PM UTC+1, Mujeeb Baig wrote:
Reply all
Reply to author
Forward
0 new messages