Buffer file keeps growing without flushing data to destination

[Gayatri S]

Hi, We are facing issues with logs not being pushed to ES. Below is our Buffer config

 <store>

        @type elasticsearch

        host "#{ENV['ES_HOSTNAME']}"

        port 9243

        user "#{ENV['ES_USERNAME']}"

        password "#{ENV['ES_PASSWORD']}"

        scheme https

        with_transporter_log true

        @log_level debug

        ssl_verify false

        ssl_version TLSv1_2

        index_name ${indexprefix}

        reconnect_on_error true

        reload_connections false

        reload_on_failure true

        suppress_type_name true

        request_timeout 30s

        prefer_oj_serializer true

        type_name _doc

        <buffer indexprefix>

          @type "file"

          path "#{ENV['BufferPath']}"

          flush_thread_count 10

          flush_mode interval

          flush_interval 30s

          flush_at_shutdown true

          overflow_action throw_exception

          compress gzip

          retry_forever true

          retry_type periodic

          retry_wait 30s

          chunk_limit_size 64MB

          total_limit_size 64GB

        </buffer>

        </store>

when tried to look at the buffer file, the file kept growing in size shown below. 

What is going wrong here. ES is reachable and other applications are still able to push logs.

Fluentd version - 1.11.4

There are no error logs sometimes see the below warning 

buffer flush took longer time than slow_flush_log_threshold: elapsed_time=20.126440424006432 slow_flush_log_threshold=20.0 plugin_id="object:b18"

[Vazel]

have you resolved?

[José Lecaros]

Hello there,

I think there are more active users in the fluentd slack channel slack.Fluentd.org

If you haven't solved you're very welcome to make your questions there.