flush_interval does not seem to work

[Fere]

Hi,

I am trying to use flush_interval to make sure the buffered data to be sent out every few seconds rather than every minute.  But it doesn't work.  The buffered data is flushed and sent out every minute regardless of speifying flush_interval or its value.

Here is the match part:

<match ind9a.mtn>

  @type file

  @id nonkafka.children.ind9a.mtn

  output_tag false  # Optional, defaults to true. Output the tag field if true.

  output_time false  # Optional, defaults to true. Output the time field if true.

  path /var/log/td-agent/buffer/ind9a.mtn/ind9a.mtn

  format csv

  fields transaction_id,src_id,amount,time

  force_quotes false

  time_slice_format %Y%m%d%H%M

  time_slice_wait 0s

  time_format %Y%m%dT%H%M%S%z

  flush_interval  5s

</match>

What am I missing?

Appreciate your help

[Mr. Fiber]

I tested your configuration and it worked as expected.

- fluent.conf

```
<source>
  @type forward
</source>

<match ind9a.mtn>
  @type file
  @id nonkafka.children.ind9a.mtn

  path log/ind9a.mtn
  format json

  time_slice_format %Y%m%d%H%M
  time_slice_wait 0s
  time_format %Y%m%dT%H%M%S%z
  flush_interval 5s
</match>

```

- test client

```
require 'fluent-logger'
require 'json'

log = Fluent::Logger::FluentLogger.new(nil, :host=>'localhost', :port=>24224)
hash = {'hello' => 'world'}

loop {
  log.post("ind9a.mtn", hash)
  sleep 1
}

```

- output result

You can see files are created by 5 seconds interval

```
% ls -T -l log
total 56
-rw-r--r--  1 repeatedly  staff    36B 11 30 13:03:02 2017 ind9a.mtn.201711301303_0.log
-rw-r--r--  1 repeatedly  staff    90B 11 30 13:03:07 2017 ind9a.mtn.201711301303_1.log
-rw-r--r--  1 repeatedly  staff    90B 11 30 13:03:12 2017 ind9a.mtn.201711301303_2.log
-rw-r--r--  1 repeatedly  staff    90B 11 30 13:03:17 2017 ind9a.mtn.201711301303_3.log
-rw-r--r--  1 repeatedly  staff    90B 11 30 13:03:22 2017 ind9a.mtn.201711301303_4.log
-rw-r--r--  1 repeatedly  staff    90B 11 30 13:03:27 2017 ind9a.mtn.201711301303_5.log
-rw-r--r--  1 repeatedly  staff    72B 11 30 13:03:32 2017 ind9a.mtn.201711301303_6.log

```

> What am I missing?

I'm not sure... one possible case using old configuration on your test server.

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Fere]

Thank you for your reposnse.

I do see the buffered files too, however they are sitting there and are only sent to the destination every minute.  Is there a way to make the actual sending of the output to the destination every few seconds (to be more like realtime)?

Thanks,

Fereshteh

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.

[Mr. Fiber]

> however they are sitting there and are only sent to the destination every minute.

What does this sentence mean? You don't use out_file in the actual situation?

To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.