logging - how to integrate with graylog2 (logback-gelf)

[Lior Harel]

We're trying to send all services' logs to centralised logging system (graylog2) and having issues with integrating the dropwizard services.

I've seen few threads about custom log appender configuration, and the common response is it's not supported using the yaml configuration.

do you have any suggestion how this can be achieved?

do you consider a more pluggable architecture for the logging aspect? 

[Coda Hale]

You'll have to write it yourself.

---
Coda Hale
http://codahale.com

[Kurt Harriger]

I actually had the exact same question this morning.  I too want to send all logs to graylog, currently all of our other projects use log4j.  Is it possible to use log4j with dropwizard instead of logback, this seems easiest to me as I can leverage existing configuration files? 

[Coda Hale]

Nope. I'm not interested in making Dropwizard a one-size-fits-all solution.

If you want the logging to do something different, programmatically re-configure it to do so.

---
Coda Hale
http://codahale.com

[Kurt Harriger]

The enterprise ready solution that requires programmers to change logging configuration?... Last time I worked for an enterprise ops and development were entirely separate teams! (partly why I no longer work for an enterprise, but just sayin doesn't quite jive with the enterprise ready messaging) 

[Tatu Saloranta]

If you want to use log4j, you can do that. But there is little point
in trying to make DropWizard pluggable with more than one logging
framework. The whole point is to reduce complexity, not increase it.

But isn't there slf4j component that allows one to pipe either backend
to log4j, or log4j calls to go to slf4j backend? Not everything has to
be provided by platform itself.

-+ Tatu +-

[Coda Hale]

There's literally no reference to "enterprise" in the Dropwizard repo, either in the code or the documentation. It's not a term I use, nor do I care whether people who self-identify as such like or dislike Dropwizard.

High-performance? Yes.

Ops-friendly? Yes.

Enterprise? Don't care.

Dropwizard's primary purpose is to enable feature teams at Yammer to ship high-quality, ops-friendly services as quickly as possible. The fact that this goal dovetails with other people's goals is a happy coincidence.

---
Coda Hale
http://codahale.com

[Kurt Harriger]

For some reason I thought the website said "enterprise-ready", but it says "production-ready" my bad.  Production-ready in my world means logs are sent to graylog, but I'm just trolling now so you can ignore me.  

[Coda Hale]

And if you want it to do that, write that code.

If you want that code to be configurable, write that code.

---
Coda Hale
http://codahale.com

[Nick Telford]

It's pretty simple to write a bundle that does this. I was feeling generous this morning so I hacked together a gist using https://github.com/Moocar/logback-gelf: https://gist.github.com/dd5e000c3327484540a8

I haven't compiled/tested this, so there's bound to be a few typos etc. but you should get the general idea. You can break the Bundle out in to a separate module that all your projects depend on, then you only have to deal with the Service boilerplate that adds the Bundle and hooks up your Service's Configuration to that of the Bundle.

I mostly did this to demonstrate that Dropwizard, for all its opinions, is still very flexible.

Regards,

Nick Telford

[Kurt Harriger]

Thanks Nick, slightly better than my 10 minute hack... however, I disagree that this demonstrates its flexibility, rather I think it highlights the problem.  We also add couple extended attributes about the deployment environment, so although cleaner than my code a bit it still requires modification to add features that are ALREADY configurable with documentation with examples is in logback-gelf project using the standard logback.xml or groovy configuration files.  

My initial assumption was that dropwizard simply provided a yaml rather than xml or groovy syntax to the underlying logback configuration, however the documentation didn't seem to match up and left me bit confused until I found this post basically stating hey we don't expose the underlying configuration write your own wrapper.  

I don't expect nor want dropwizard to provide native support for every logging appender logback provides. This creates a huge burden of documenting and supporting all these logging wrappers and as a "library" they SHOULD NOT NEED TO!  Yet dropwizard disables native configuration and replaces their own configuration DSL.   However, Logback already provides a well documented and extensible configuration file that supports custom extenders, so my argument is WE SHOULDN'T NEED TO EITHER!  This is where I feel dropwizard crosses the line from library to framework.  Why should the dropwizard community need to write and maintain their own logging configuration wrappers when Logback already solves this problem? 

Logging frameworks can get complex, and I think it is nice that dropwizard tries to provide to minimize the complexity.  However, this should adhere to the library principle and not the framework principle... use it if you want if you don't well then don't! 

Perhaps the most ideal solution would be simply this:

logging:

  externalConfiguration = true

If externalConfiguration = true then dropwizard should do nothing! use it if you want... if you don't then don't!

[Nick Telford]

I hadn't intended to spark a debate, merely demonstrate that it can be done without more than 15 mins or so of work. If you use Graylog across all your systems, then you can spend 15 minutes adding support for it to Dropwizard and be done with it.

If you want to make Dropwizard configure pluggable Logback Appenders natively, feel free to fork and contribute. Coda has repeatedly stated that he'll accept patches that improve modularity/pluggability provided it doesn't make things more painful for Yammer's engineers.

Regards,

Nick Telford

[Kurt Harriger]

Fair enough.  It appears that doing nothing is a bit difficult since the configuration is reset in a static method on service and later reconfigured from yaml after the configuration is parsed.  I guess this is partly to ensure that errors prior to loading configuration are at least sent to the console if no logback configuration file exists.  I'm not very familiar with logback yet, but it looks like the configuration file can be loaded pragmatically with the JordanConfigurator.  The simplest hack then would probably be to just put this call in my service initialize method, but I'd be willing to add a externalConfiguration = filename to LoggingFactory and submit a patch to make this core functionality if there is interest.  

[Coda Hale]

As I've said before, the only reason Dropwizard exists at all is to provide opinions on what a service should be.

It uses fat JARs because I think they work better.

It embeds Jetty because I think that works better.

It uses Jackson because I think that works better.

It uses Jersey because I think that works better.

It has a single YAML configuration file because I think that works better.

It wraps Logback because I think that works better.

Dropwizard makes choices about the vast majority of things that a RESTful service might do. It does so because it allows Yammer's feature teams to get working on the things that actually deliver business value (i.e., the feature) more quickly, and because it allows our infrastructure teams to make a maximal number of assumptions about the operational profile of a service in production.

I'm willing — eager, even — to hear feature requests which improve Dropwizard for everyone. But I'm fundamentally unwilling to compromise the benefits that Dropwizard provides my co-workers in order to please arbitrary people on the internet. My employer — the organization which signs my paychecks — will always come first. Such is the nature of paychecks.

So: yes, there are existing configuration file formats for Logback, for servlet configuration, for Jetty configuration, for Jersey configuration, for all of the shit that Dropwizard depends on. If you want to go down that route, you just can deploy your WAR files to a Jetty app server. I'm OK with that.

---
Coda Hale
http://codahale.com

[Tatu Saloranta]

Just to make sure you are aware of this: usually you extend
Configuration object, and in your sub-class you can add whatever new
fields you want. You can also sub-class any of existing configuration
Object (like HttpConfiguration etc).

What this means is that it is easy to add more configuration options,
including serializations of objects from other libs, frameworks, as
long as they can be data-bound easily. And if embedding does not work,
you can add URLs and reference from there, adding explicit handling.

This means that you can aggregate all kinds of configuration without
straying too far from the single root configuration file (and matching
config POJO) idea.

One final tip: anything that takes Properties for configuration is
trivially easy to map from YAML/JSON: basic object structure is
automatically bound to Properties field in the configuration object.

-+ Tatu +-