I have only files with "buffer." prefix with @type file, how can I final log file without this prefix?

24 views
Skip to first unread message

Stéphane Klein

unread,
Jul 18, 2017, 4:43:54 AM7/18/17
to Fluentd Google Group
Hi,

I have this directive in fluentd config file:

<match **>
    @type file
    path /fluentd/log/output
</match>

I have this files in /fluentd/log/output

# tree fluentd-log/output
fluentd-log/output
├── buffer.b5548a854de17d7e88ea319ea591d00fb.log
└── buffer.b5548a854de17d7e88ea319ea591d00fb.log.meta

I see "buffer." prefix. I understand it is the file buffer but how can I get the final log file, not the buffer?

Best regards,
Stéphane

Stéphane Klein

unread,
Jul 18, 2017, 5:11:39 AM7/18/17
to Fluentd Google Group

Le mardi 18 juillet 2017 10:43:54 UTC+2, Stéphane Klein a écrit :

<match **>
    @type file
    path /fluentd/log/output
</match>


If I use this configuration:

  <store>
    @type file
    path /fluentd/log/output
    buffer_type memory
    append false
  </store>

I have:

# tree fluentd-log/
fluentd-log/
├── fluentd.log
└── output.20170717_0.log

All is good. But if I don't use memory buffer_type, how can I get this output log file?

Best regards,
Stéphane

Mr. Fiber

unread,
Jul 18, 2017, 5:16:03 AM7/18/17
to Fluentd Google Group
I understand it is the file buffer but how can I get the final log file, not the buffer?



Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Stéphane Klein

unread,
Jul 18, 2017, 7:07:26 AM7/18/17
to Fluentd Google Group


Le mardi 18 juillet 2017 11:16:03 UTC+2, repeatedly a écrit :
I understand it is the file buffer but how can I get the final log file, not the buffer?



I see but when I set buffer_path and path, I see buffer files in buffer_path but I have nothing in path :(

Mr. Fiber

unread,
Jul 18, 2017, 7:13:08 AM7/18/17
to Fluentd Google Group
It seems not flushed yet.

--

Stéphane Klein

unread,
Jul 18, 2017, 7:18:44 AM7/18/17
to Fluentd Google Group


Le mardi 18 juillet 2017 13:13:08 UTC+2, repeatedly a écrit :
It seems not flushed yet.


Thanks.

Ok with:

    flush_interval 5s


# tree fluentd-log/
fluentd-log/
├── fluentd.log
├── output.20170718_0.log
└── output.buffer
    ├── buffer.b55495a7a48148d1faf838fe4963217a5.log
    └── buffer.b55495a7a48148d1faf838fe4963217a5.log.meta

complete config file:

<system>
  log_level debug
</system>

<source>
  @type syslog
  port 5140
  bind 0.0.0.0
  tag journal
</source>

<match **>
  @type copy


  <store>
    @type file
    path /fluentd/log/output
    buffer_type file
    buffer_path /fluentd/log/output.buffer
    append false
    flush_interval 5s
  </store>

  <store>
    @type elasticsearch
    host elasticsearch
    buffer_type memory
    flush_interval 1s
    logstash_dateformat %Y%m%d
    port 9200
    logstash_format true
    type_name fluentd
    index_name logstash
    include_tag_key true
    tag_key _key
    buffer_chunk_limit 512k
    reload_connections false
    reconnect_on_error true
    max_retry_wait 60
    disable_retry_limit
  </store>
</match>
Reply all
Reply to author
Forward
0 new messages