Fluentbit stops processing (tail input / graylog output)

1,419 views
Skip to first unread message

Jimmy

unread,
Nov 26, 2021, 8:01:22 PM11/26/21
to Fluent Bit
I have a simple setup with tail input file sending to graylog (gelf) output.  It works great until some time (can be hours or days) it quits sending.  There is nothing in the fluentbit log to tell me what is wrong.  Also, the monitoring api responds fine so our monitoring sytem cant even detect when it's stopped working when it hits the fluentbit monitoring endpoint.   If I try to stop the service, it hangs too so I suspect it is caught in some weird waiting state.  I can kill the process and restart it fine.  But then of course after several hours/days it will get into this weird state.  I tried all conceivable options but I can't get around this issue.  Any ideas? I took a procdump (windows) and every time it seems to be in the middle of doing a tls disconnect.  There's plenty of memory and disk.  Any help would be greatly appreciated.

[INPUT]
    Name tail
    parser gelf
    Path D:\Logs\log-*.txt
    DB D:\Logs\logdb.db
    DB.locking true
    DB.sync full
    Refresh_Interval 10
    Read_from_Head true
    Skip_Long_Lines On
    Buffer_Max_Size 1MB
    Mem_Buf_Limit 100MB
    Buffer_Chunk_Size 1MB
    storage.type filesystem

[OUTPUT]
    name  gelf
    match *
    host  somehost
    port 12201
    mode tls
    Gelf_Short_Message_Key short_message
    Gelf_Full_Message_Key @m
    Gelf_Timestamp_Key @t
    Retry_Limit false
    tls on
    tls.verify off

Eduardo Silva

unread,
Nov 26, 2021, 8:06:59 PM11/26/21
to Jimmy, Fluent Bit
Let's move this ticket to Github for assistance, also on that ticket provide

- Full OS name and version
- Fluent Bit version
- full Fluent Bit configuration 
- log file

--
You received this message because you are subscribed to the Google Groups "Fluent Bit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluent-bit+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fluent-bit/eb17bf1e-32d4-4963-9d14-51eea8d6518cn%40googlegroups.com.


--

Jimmy

unread,
Nov 27, 2021, 8:49:53 PM11/27/21
to Fluent Bit
Thanks for responding.  I added it here.. https://github.com/fluent/fluent-bit/issues/4376
Reply all
Reply to author
Forward
0 new messages