Fluentd Logging Driver multiple destinations ... and Docker remote logging fragility

[Traiano Welcome]

Hi

I'd like to be able to specify multiple destinations as "fluentd-server" to be used in a failover scenario if the fluentd server is no longer available (dns failure, network node failure, server failure).

Currently it appears that only one "fluentd-server" destination is accepted as a docker daemon or run option. This means no failover possible if the logging destination is unreachable.

Unfortunately, docker daemon itself panics and shuts down if it cannot reach the fluentd logging server, as described here:

https://github.com/moby/moby/issues/32567

If the fluentd logging driver supported multiple fluent logging destinations, docker could failover to the next server without crashing.

Does fluentd driver have such a feature currently, or is one planned? If not, is there a recommended setup for this requirement?

Many thanks in advance,

Traiano

[Mr. Fiber]

> Does fluentd driver have such a feature currently, or is one planned? If not, is there a recommended setup for this requirement?

Hmm... there are two approaches:

- https://github.com/fluent/fluent-logger-golang supports multiple destinations

- fluentd logging driver re-creates fluent-logger-golang with other servers

The problem is fluent-logger-golang needs active maintainer.

fluent-logger-golang is first logger in golang but other fluent loggers has better APIs.

So we need to update fluent-logger-golang but no resource in fluentd committers for now...

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Traiano Welcome]

On Fri, Jun 8, 2018 at 11:17 PM, Mr. Fiber <repea...@gmail.com> wrote:

> Does fluentd driver have such a feature currently, or is one planned? If not, is there a recommended setup for this requirement?

Hmm... there are two approaches:

- https://github.com/fluent/fluent-logger-golang supports multiple destinations

- fluentd logging driver re-creates fluent-logger-golang with other servers

The problem is fluent-logger-golang needs active maintainer.

fluent-logger-golang is first logger in golang but other fluent loggers has better APIs.

So we need to update fluent-logger-golang but no resource in fluentd committers for now...

Ok, so would it be fair to say fluentd-logger-golang is not suitable for production environments currently?

Currently, I'm using the following docker configuration for both the docker daemon and containers, to provide some resilience to an outage of the fluentd aggregator of a few hours:

    "log-opts": {
        "fluentd-address": "127.0.0.1:24226",
        "fluentd-buffer-limit": "512m",
        "mode": "non-blocking",
        "tag": "docker.{{.ID}}",
        "fluentd-max-retries": "144",
        "fluentd-retry-wait": "600s",
        "fluentd-async-connect": "true"
    }

I suppose this is the best possible solution until fluentd-driver has a production ready multi-destination alternative :-(