Flogger: A Fluent Logging API for Gerrit?

464 views
Skip to first unread message

Edwin Kempin

unread,
Apr 17, 2018, 10:56:50 AM4/17/18
to gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion

Hi,


today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

https://docs.google.com/presentation/d/1mE6Gc3Tb0cqYRjFk1FRg-5QsgceEHlDlZKewNBRydjY/edit?usp=sharing 


Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

https://github.com/google/flogger

https://google.github.io/flogger/


Benefits:

- Better Readability

- Better Performance

- Extensible API

- Will allow us to implement request tracing


Flogger API examples:

logger.atSevere() .withCause(err) .log("Error in %s %s", req.getMethod(), uri);
logger.atInfo().atMostEvery(1, TimeUnit.HOURS).log(...)
logger.atInfo().withStackTrace(StackSize.FULL).log(...)

The idea is to use Flogger with a (yet to be written) log4j backend so that existing log4 configuration files continue to work.

Within Google Flogger is long-established and used by many projects.

Please let me know if you have any concerns about this.

I'm also looking for volunteers to help with the migration effort. So if this sounds interesting to you and you have time to help with this please let me know. One of the first things to do would be to write a log4j backend for Flogger (estimated effort ~2weeks).

Edwin

thomasmu...@yahoo.com

unread,
Apr 17, 2018, 12:09:19 PM4/17/18
to Repo and Gerrit Discussion

thomasmu...@yahoo.com

unread,
Apr 17, 2018, 12:14:14 PM4/17/18
to Repo and Gerrit Discussion
Will this support sending logs to logstash?

And also because of the migration to another logger and the fact that log4j on logstash is being removed, will this support gelf?

Or will we develop a logstash plugin for flogger?


On Tuesday, April 17, 2018 at 3:56:50 PM UTC+1, Edwin Kempin wrote:

Edwin Kempin

unread,
Apr 17, 2018, 4:34:35 PM4/17/18
to Paladox none, Repo and Gerrit Discussion
On Tue, Apr 17, 2018 at 6:09 PM thomasmulhall410 via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Edwin Kempin

unread,
Apr 17, 2018, 4:44:17 PM4/17/18
to Paladox none, Repo and Gerrit Discussion
On Tue, Apr 17, 2018 at 6:14 PM thomasmulhall410 via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
Will this support sending logs to logstash?

And also because of the migration to another logger and the fact that log4j on logstash is being removed, will this support gelf?
I think this shouldn't effect any existing integration with logstash since Flogger is an API to write logs, but the resulting log files will be the same as before and we want to continue using log4j as backend. 
How is the logstash integration currently working? Is it using an log4j appender as data source, or is it reading directly from the log files?
I don't know gelf. How is it working and how is it used to get Gerrit logs into logstash?
 

Or will we develop a logstash plugin for flogger?

On Tuesday, April 17, 2018 at 3:56:50 PM UTC+1, Edwin Kempin wrote:

Hi,


today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

https://docs.google.com/presentation/d/1mE6Gc3Tb0cqYRjFk1FRg-5QsgceEHlDlZKewNBRydjY/edit?usp=sharing 


Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

https://github.com/google/flogger

https://google.github.io/flogger/


Benefits:

- Better Readability

- Better Performance

- Extensible API

- Will allow us to implement request tracing


Flogger API examples:

logger.atSevere() .withCause(err) .log("Error in %s %s", req.getMethod(), uri);
logger.atInfo().atMostEvery(1, TimeUnit.HOURS).log(...)
logger.atInfo().withStackTrace(StackSize.FULL).log(...)

The idea is to use Flogger with a (yet to be written) log4j backend so that existing log4 configuration files continue to work.

Within Google Flogger is long-established and used by many projects.

Please let me know if you have any concerns about this.

I'm also looking for volunteers to help with the migration effort. So if this sounds interesting to you and you have time to help with this please let me know. One of the first things to do would be to write a log4j backend for Flogger (estimated effort ~2weeks).

Edwin

thomasmu...@yahoo.com

unread,
Apr 17, 2018, 5:17:06 PM4/17/18
to Repo and Gerrit Discussion
Currently log4j works with logstash by using SocketAppender but logstash is removing that plugin for security reasons. And for integrating gelf you include https://gerrit-review.googlesource.com/c/gerrit/+/146214


Log4j logstash is done like:


But it's very limited in what that will do, where as gelf seems to include better stack traces from the logs.

Matthias Sohn

unread,
Apr 17, 2018, 5:33:17 PM4/17/18
to Edwin Kempin, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
On Tue, Apr 17, 2018 at 4:56 PM, 'Edwin Kempin' via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:

Hi,


today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

https://docs.google.com/presentation/d/1mE6Gc3Tb0cqYRjFk1FRg-5QsgceEHlDlZKewNBRydjY/edit?usp=sharing 


Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

https://github.com/google/flogger

https://google.github.io/flogger/


Benefits:

- Better Readability

- Better Performance

- Extensible API

- Will allow us to implement request tracing


Flogger API examples:

logger.atSevere() .withCause(err) .log("Error in %s %s", req.getMethod(), uri);
logger.atInfo().atMostEvery(1, TimeUnit.HOURS).log(...)
logger.atInfo().withStackTrace(StackSize.FULL).log(...)


looks like a good improvement
 
The idea is to use Flogger with a (yet to be written) log4j backend so that existing log4 configuration files continue to work.

Within Google Flogger is long-established and used by many projects.

Please let me know if you have any concerns about this.

I'm also looking for volunteers to help with the migration effort. So if this sounds interesting to you and you have time to help with this please let me know. One of the first things to do would be to write a log4j backend for Flogger (estimated effort ~2weeks).

Edwin

would it make sense to also adopt flogger in jgit ?

-Matthias 

Luca Milanesio

unread,
Apr 17, 2018, 5:36:49 PM4/17/18
to Matthias Sohn, Luca Milanesio, Edwin Kempin, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
+1 IMHO. It is crucial to have a consistent way of logging throughout the Gerrit stack, including JGit :-)

Welcome Flogger !

Luca.


-Matthias 

Matthias Sohn

unread,
Apr 17, 2018, 6:20:11 PM4/17/18
to Luca Milanesio, Edwin Kempin, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
On Tue, Apr 17, 2018 at 11:36 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:
On 17 Apr 2018, at 22:33, Matthias Sohn <matthi...@gmail.com> wrote:

On Tue, Apr 17, 2018 at 4:56 PM, 'Edwin Kempin' via Repo and Gerrit Discussion <repo-discuss@googlegroups.com> wrote:
Hi,

today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

is the binary available on some central artefact repository like Maven central ?
This one has a strange groupId which I wouldn't expect from a google open source project
 
Benefits:
- Better Readability
- Better Performance
- Extensible API
- Will allow us to implement request tracing

Flogger API examples:
logger.atSevere() .withCause(err) .log("Error in %s %s", req.getMethod(), uri);
logger.atInfo().atMostEvery(1, TimeUnit.HOURS).log(...)
logger.atInfo().withStackTrace(StackSize.FULL).log(...)


looks like a good improvement
 
The idea is to use Flogger with a (yet to be written) log4j backend so that existing log4 configuration files continue to work.

Within Google Flogger is long-established and used by many projects.

Please let me know if you have any concerns about this.

I'm also looking for volunteers to help with the migration effort. So if this sounds interesting to you and you have time to help with this please let me know. One of the first things to do would be to write a log4j backend for Flogger (estimated effort ~2weeks).

Edwin

would it make sense to also adopt flogger in jgit ?

+1 IMHO. It is crucial to have a consistent way of logging throughout the Gerrit stack, including JGit :-)

ok, I can look into this part
 
Welcome Flogger !

Luca.


-Matthias 

-- 
-- 
To unsubscribe, email repo-discuss+unsub...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

Matthias Sohn

unread,
Apr 17, 2018, 6:25:45 PM4/17/18
to Luca Milanesio, Edwin Kempin, Repo and Gerrit Discussion
can we remove the Gerrit maintainers list from this conversation ?
I keep getting error bounces and the mail server complains I am not allowed to post on the maintainer list

-Matthias

 
Welcome Flogger !

Luca.


-Matthias 

-- 
-- 
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com

thomasmu...@yahoo.com

unread,
Apr 17, 2018, 6:31:51 PM4/17/18
to Repo and Gerrit Discussion
Also  log4j.xml must also be supported too alongside log4j.properties.


On Tuesday, April 17, 2018 at 3:56:50 PM UTC+1, Edwin Kempin wrote:

Edwin Kempin

unread,
Apr 18, 2018, 4:16:05 AM4/18/18
to Paladox none, Repo and Gerrit Discussion
On Tue, Apr 17, 2018 at 11:17 PM thomasmulhall410 via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
Currently log4j works with logstash by using SocketAppender
OK, I expect that this continues to work since we want to use Flogger with a log4j backend.
 
but logstash is removing that plugin for security reasons. And for integrating gelf you include https://gerrit-review.googlesource.com/c/gerrit/+/146214
Thanks. I see, this is an orthogonal issue. Using Flogger doesn't solve this for you, but also shouldn't make it any more difficult to address.
Writing a logstash integration specifically for Flogger might be a good idea, but it's nothing that I want to investigate right now.

Edwin Kempin

unread,
Apr 18, 2018, 4:25:21 AM4/18/18
to Matthias Sohn, Luca Milanesio, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
On Wed, Apr 18, 2018 at 12:20 AM Matthias Sohn <matthi...@gmail.com> wrote:
On Tue, Apr 17, 2018 at 11:36 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:
On 17 Apr 2018, at 22:33, Matthias Sohn <matthi...@gmail.com> wrote:

On Tue, Apr 17, 2018 at 4:56 PM, 'Edwin Kempin' via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
Hi,

today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

is the binary available on some central artefact repository like Maven central ?
This one has a strange groupId which I wouldn't expect from a google open source project
These are indeed not the correct artifacts.
I think the Flogger artifacts are not yet published to Maven Central, but this should happen soon.
Meanwhile [1] describes how to build the jars from source.


 

 
Benefits:
- Better Readability
- Better Performance
- Extensible API
- Will allow us to implement request tracing

Flogger API examples:
logger.atSevere() .withCause(err) .log("Error in %s %s", req.getMethod(), uri);
logger.atInfo().atMostEvery(1, TimeUnit.HOURS).log(...)
logger.atInfo().withStackTrace(StackSize.FULL).log(...)


looks like a good improvement
 
The idea is to use Flogger with a (yet to be written) log4j backend so that existing log4 configuration files continue to work.

Within Google Flogger is long-established and used by many projects.

Please let me know if you have any concerns about this.

I'm also looking for volunteers to help with the migration effort. So if this sounds interesting to you and you have time to help with this please let me know. One of the first things to do would be to write a log4j backend for Flogger (estimated effort ~2weeks).

Edwin

would it make sense to also adopt flogger in jgit ?

+1 IMHO. It is crucial to have a consistent way of logging throughout the Gerrit stack, including JGit :-)

ok, I can look into this part

Nice that you like the idea :)
Having jgit migrating to Flogger too would be ideal indeed.
Thanks Matthias for offering to look into this.
 
 
Welcome Flogger !

Luca.


-Matthias 

-- 
-- 
To unsubscribe, email repo-discuss...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

Edwin Kempin

unread,
Apr 18, 2018, 4:25:38 AM4/18/18
to Paladox none, Repo and Gerrit Discussion
On Wed, Apr 18, 2018 at 12:31 AM thomasmulhall410 via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
Also  log4j.xml must also be supported too alongside log4j.properties.
Yes, thanks for reminding about this.
 

Matthias Sohn

unread,
Apr 18, 2018, 11:33:57 AM4/18/18
to Edwin Kempin, Luca Milanesio, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
On Wed, Apr 18, 2018 at 10:24 AM, Edwin Kempin <eke...@google.com> wrote:


On Wed, Apr 18, 2018 at 12:20 AM Matthias Sohn <matthi...@gmail.com> wrote:
On Tue, Apr 17, 2018 at 11:36 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:


On 17 Apr 2018, at 22:33, Matthias Sohn <matthi...@gmail.com> wrote:

On Tue, Apr 17, 2018 at 4:56 PM, 'Edwin Kempin' via Repo and Gerrit Discussion <repo-discuss@googlegroups.com> wrote:
Hi,

today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

is the binary available on some central artefact repository like Maven central ?
This one has a strange groupId which I wouldn't expect from a google open source project
These are indeed not the correct artifacts.
I think the Flogger artifacts are not yet published to Maven Central, but this should happen soon.

For JGit we need to add the library to the Orbit project for signing it with the Eclipse certificate,
for that we'd ideally download the binary artefact from Maven central.

 
 
 
Welcome Flogger !

Luca.


-Matthias 

-- 
-- 
To unsubscribe, email repo-discuss+unsub...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

Edwin Kempin

unread,
Apr 23, 2018, 2:43:29 AM4/23/18
to Matthias Sohn, Luca Milanesio, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
On Wed, Apr 18, 2018 at 5:33 PM Matthias Sohn <matthi...@gmail.com> wrote:
On Wed, Apr 18, 2018 at 10:24 AM, Edwin Kempin <eke...@google.com> wrote:


On Wed, Apr 18, 2018 at 12:20 AM Matthias Sohn <matthi...@gmail.com> wrote:
On Tue, Apr 17, 2018 at 11:36 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:


On 17 Apr 2018, at 22:33, Matthias Sohn <matthi...@gmail.com> wrote:

On Tue, Apr 17, 2018 at 4:56 PM, 'Edwin Kempin' via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
Hi,

today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

is the binary available on some central artefact repository like Maven central ?
This one has a strange groupId which I wouldn't expect from a google open source project
These are indeed not the correct artifacts.
I think the Flogger artifacts are not yet published to Maven Central, but this should happen soon.

For JGit we need to add the library to the Orbit project for signing it with the Eclipse certificate,
for that we'd ideally download the binary artefact from Maven central.

The flogger artifacts are now available on Maven Central. The group ID is com.google.flogger:
 

 
 
 
Welcome Flogger !

Luca.


-Matthias 

-- 
-- 
To unsubscribe, email repo-discuss...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

Matthias Sohn

unread,
Apr 23, 2018, 4:35:03 PM4/23/18
to Edwin Kempin, Luca Milanesio, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
On Mon, Apr 23, 2018 at 8:42 AM, Edwin Kempin <eke...@google.com> wrote:


On Wed, Apr 18, 2018 at 5:33 PM Matthias Sohn <matthi...@gmail.com> wrote:
On Wed, Apr 18, 2018 at 10:24 AM, Edwin Kempin <eke...@google.com> wrote:


On Wed, Apr 18, 2018 at 12:20 AM Matthias Sohn <matthi...@gmail.com> wrote:
On Tue, Apr 17, 2018 at 11:36 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:


On 17 Apr 2018, at 22:33, Matthias Sohn <matthi...@gmail.com> wrote:

On Tue, Apr 17, 2018 at 4:56 PM, 'Edwin Kempin' via Repo and Gerrit Discussion <repo-discuss@googlegroups.com> wrote:
Hi,

today at the hackathon I presented the idea to use a new logging framework, Flogger, for Gerrit:

Flogger is an open source project (Apache 2 License) from Google that provides a fluent JAVA API for logging:

is the binary available on some central artefact repository like Maven central ?
This one has a strange groupId which I wouldn't expect from a google open source project
These are indeed not the correct artifacts.
I think the Flogger artifacts are not yet published to Maven Central, but this should happen soon.

For JGit we need to add the library to the Orbit project for signing it with the Eclipse certificate,
for that we'd ideally download the binary artefact from Maven central.

The flogger artifacts are now available on Maven Central. The group ID is com.google.flogger:

thanks. 

Are you aware of any documentation explaining when we need which of those 4 jars ?
The javadoc for flogger packages isn't really helpful.
 

 
 
 
Welcome Flogger !

Luca.


-Matthias 

-- 
-- 
To unsubscribe, email repo-discuss+unsub...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

Edwin Kempin

unread,
Apr 24, 2018, 2:57:46 AM4/24/18
to Matthias Sohn, Luca Milanesio, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion
Could you please post this question on the flogger mailing list?

 
 

 
 
 
Welcome Flogger !

Luca.


-Matthias 

-- 
-- 
To unsubscribe, email repo-discuss...@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

Matthias Sohn

unread,
Apr 24, 2018, 6:45:45 AM4/24/18
to Edwin Kempin, Luca Milanesio, gerritcoderevi...@googlegroups.com, Repo and Gerrit Discussion

thomasmu...@yahoo.com

unread,
May 22, 2018, 9:26:26 AM5/22/18
to Repo and Gerrit Discussion
I wonder if this is a perfect time to update to log4j2 and make a log4j2 backend for flogger?


On Tuesday, April 17, 2018 at 3:56:50 PM UTC+1, Edwin Kempin wrote:

Edwin Kempin

unread,
May 22, 2018, 9:35:43 AM5/22/18
to Paladox none, Repo and Gerrit Discussion
On Tue, May 22, 2018 at 3:26 PM thomasmulhall410 via Repo and Gerrit Discussion <repo-d...@googlegroups.com> wrote:
I wonder if this is a perfect time to update to log4j2 and make a log4j2 backend for flogger?
I would prefer to first finish the migration to Flogger with the log4j backend that I wrote for this.
If you want log4j2 you can start implementing the log4j2 backend for Flogger. Once this is there we can do the switch to log4j2.
 

Edwin Kempin

unread,
May 22, 2018, 9:39:53 AM5/22/18
to Paladox none, Repo and Gerrit Discussion
The changes that migrate Gerrit to Flogger are now up for review:

Any testing that you can do is much appreciated.

We still must wait until new Flogger jars have been published to Maven Central before we can submit this series.

Paladox none

unread,
May 10, 2019, 7:24:56 PM5/10/19
to Repo and Gerrit Discussion
Im going to give a shot at supporting log4j2 in flogger (after the Wikimedia foundation requested it :)).

Paladox none

unread,
May 11, 2019, 8:22:41 PM5/11/19
to Repo and Gerrit Discussion
I've uploaded this WIP https://github.com/google/flogger/pull/82 (im stuck on replacing callAppenders).

log4j2/src/main/java/com/google/common/flogger/backend/log4j2/Log4j2LoggerBackend.java:76: error: cannot find symbol

    logger.callAppenders(logEntry.asLoggingEvent());

          ^

  symbol:   method callAppenders(LogEvent)

Reply all
Reply to author
Forward
0 new messages