Add custom logs to fluent-bit collection

[Gil Cohen]

Hi,

I am using the stable/fluent-bit helm chart (chart) to collect the pod logs and forward them to a central fluentd pod.

This works great, but I would also like fluent-bit to forward custom logs created by the pods in the same way. (the container running in the pod creates some debug logs I want to keep separately).

Is there a way to do this?

Thanks,

Gil

[Sid Stuart]

I can't say I'm expert, but I'm picking up syslog as well as the kubernetes logs with a configuration that looks like this,

input-kubernetes.conf: |

    [INPUT]

        Name              tail

        Tag               kube.*

        Path              /var/log/containers/*.log

        Path_Key          filename

        Parser            docker

        DB                /var/log/flb_kube.db

        Mem_Buf_Limit     5MB

        Skip_Long_Lines   On

        Refresh_Interval  10

    [INPUT]

        Name              tail

        Tag               syslog

        Path              /var/log/syslog

        Path_Key          filename

        Parser            syslog

        DB                /var/log/flb_kube.db

        Mem_Buf_Limit     5MB

        Skip_Long_Lines   On

        Refresh_Interval  10

The tough part would be to find or define a parser that works for you.

Sid

--
You received this message because you are subscribed to the Google Groups "Fluent-Bit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluent-bit+unsubscribe@googlegroups.com.
To post to this group, send email to fluen...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fluent-bit/69d45890-bf9d-4670-8ae8-794dbc7233d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eduardo Silva]

Hi Gil, 

if your custom application running in a Pod generate it own log files in a mounted volume, the best way is to run Fluent Bit in the same Pod as a side-car container, pretty much as you would do in a non kubernetes environment, 

best

--

You received this message because you are subscribed to the Google Groups "Fluent-Bit" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fluent-bit+...@googlegroups.com.

To post to this group, send email to fluen...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fluent-bit/69d45890-bf9d-4670-8ae8-794dbc7233d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Eduardo Silva

Open Source, Treasure Data

http://www.treasuredata.com/opensource

 

[Gil Cohen]

Thank you, I'll try that