Fluentd to read the syslog

[kumar]

Hi Guys,

I'm running the fluentd docker to read the syslog from the same host. Here my fluent.conf file,

<source>
  type syslog
  port 5140
  tag system
</source>


and my I have set the syslog redirect to 5140 port and restarted the rsyslog service. But the container is not reading the syslog. Need help guys.
 

[kumar]

Or guys help me with the step to configure through fluentd docker container to read the syslog and write to specify location? 

[Satoshi Tagomori]

Hi kumar,

Do you have <match> sections for output plugins?

And/or are you specifying -p5140:5140 option for "docker run" command?

tagomoris.

2015年5月21日木曜日 19時18分07秒 UTC+9 kumar:

[kumar]

Yes,

<source>
  type syslog
  port 5140
  tag system
</source>

<match system.**>
  type stdout
</match>

Also I'm running the docker as ,

docker run -td  -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/containers:/var/lib/docker/containers  --name collector1 docker-sys:latest

I have tested with -p option,

docker run -td -p 5140:5140  -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/containers:/var/lib/docker/containers  --name collector1 docker-sys:latest

[kumar]

please find the same issue on stack overflow,

http://stackoverflow.com/questions/30373063/in-syslog-pulgin-influentd-docker-container/30374648#30374648

[Satoshi Tagomori]

for syslog UDP packets, i found an article that "/udp" suffix is needed:

http://stackoverflow.com/questions/27223786/docker-udp-port-support

can you try that?

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

田籠 聡 (TAGOMORI Satoshi)
Mail: tago...@gmail.com

[kumar]

Thanks. But it looks like they are trying to access the UDP port from the container. I'm trying to run the app from the container that will contact the UDP port from the host.

[Mr. Fiber]

I'm not familiar with Docker but could you show me the fluentd's log?

Masahiro

[kumar]

I have raised this issue over stack overflow.  Please take look at this,

http://stackoverflow.com/questions/30373063/in-syslog-pulgin-influentd-docker-container/30374648#30374648

[Mr. Fiber]

Sorry I don't want to use stackoverflow.

I posted an answer to my library question before,

but stackoverflow deleted my answer.

So I reply to you here.

> Errno::EADDRNOTAVAIL: Cannot assign requested address - bind(2) for "192.168.1.37" port 5140

From your logs, fluentd can't bind 192.168.1.37 and port 5140.

I don't know the Docker but Does Docker have some limitations for binding address?

Or how about removing bind parameter?

[Lance N.]

I'm going to guess that 192.168.1.37 is the IP of the Docker host? Your Docker container for td-agent will have an IP that starts with 172. Docker will (probably) only let you bind to that address, localhost, or 0.0.0.0. Also, you will have to start the Docker container with the option that exposes that port number, which and Docker will then open 192.168.1.37:5140 and forward it to the container's 172.* IP. 

Here's how to test this:

Run 'docker ps' and find out the hex code for your running container (for example deadbeef). Run:

docker exec -it deadbeef ifconfig

and it will print the various network interfaces visible to your td-agent program.

Docker is cool but it is not simple :)

Lance

[kumar]

I tried to forward to the container, I find the error as

2015-05-25 16:55:08 +0000 [info]: adding source type="syslog"
2015-05-25 16:55:08 +0000 [error]: unexpected error error_class=Errno::EACCES error=#<Errno::EACCES: Permission denied - bind(2) for "0.0.0.0" port 514>
  2015-05-25 16:55:08 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_syslog.rb:173:in `bind'
  2015-05-25 16:55:08 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_syslog.rb:173:in `listen'
  2015-05-25 16:55:08 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_syslog.rb:106:in `start'
  2015-05-25 16:55:08 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/root_agent.rb:111:in `block in start'
  2015-05-25 16:55:08 +0000 [error]: /op

Also I tried to get the data from 514 using nc from the container ,

nc -l 514

the result data as,

30>May 25 22:19:30 5gws056 docker/34cead996122[1865]: Hello Mon May 25 16:49:30 UTC 2015
<30>May 25 22:19:31 5gws056 docker/34cead996122[1865]: Hello Mon May 25 16:49:31 UTC 2015
<30>May 25 22:19:32 5gws056 docker/34cead996122[1865]: Hello Mon May 25 16:49:32 UTC 2015
<30>May 25 22:19:33 5gws056 docker/34cead996122[1865]: Hello Mon May 25 16:49:33 UTC 2015
<30>May 25 22:19:34 5gws056 docker/34cead996122[1865]: Hello Mon May 25 16:49:34 UTC 2015
<30>May 25 22:19:35 5gws056 docker/34cead996122[1865]: Hello Mon May 25 16:49:35 UTC 2015
<30>May 25 22:19:36 5gws056 docker/34cead996122[1865]: Hello Mon May 25 16:49:36 UTC 2015

[Mr. Fiber]

514 is the well-know ports so normal user, td-agent, can't use 514.