Logging libraries and standard library compatibility
301 views
Skip to first unread message
Amit Saha
Aug 28, 2021, 7:38:50 PM8/28/21
to golang-nuts
Hi all, a huge disclaimer - I am not familiar with the design
decisions and internal details of anything that follows. They are
coming from the point of view of a consumer of the language and the
ecosystem and also someone trying to write about these things for an
audience.
Say I have an application - a HTTP server, where I initialize a
log.Logger as implemented by the standard library and then I inject
that into the handler functions where they call the Printf() or the
other methods to log stuff. Say I have also configured a middleware to
log data for every request, once again using the Printf() and family
functions.
Now, I want to implement some levelled and/or structured logging. So,
here's my expectation - I should just be able to update the initial
code and everything else should not be changed. Of course, in due
course of time, I will update those to take advantage of the actual
functionality offered by those libraries - levelled logging,
structured logging. But, initially, my code should just build and
work. Many years back, that's how I really found
https://github.com/sirupsen/logrus useful and exciting. However, some
more performant and recent packages don't seem to have that
compatibility as a goal. For example, https://github.com/uber-go/zap
doesn't AFAICT implement the Printf() or other functions. However,
https://github.com/rs/zerolog does, which is great.
I imagine this possibly is due to the standard library not exporting
publicly available types for - logging handlers, formatters (encoders)
and such that third party packages have difficulty implementing these
in standard library compatible way?
To summarize, I would have really wanted a way to avoid my code
becoming tightly coupled with the logging library I am using but that
seems to be hard at this stage.
What does the community think?
decisions and internal details of anything that follows. They are
coming from the point of view of a consumer of the language and the
ecosystem and also someone trying to write about these things for an
audience.
Say I have an application - a HTTP server, where I initialize a
log.Logger as implemented by the standard library and then I inject
that into the handler functions where they call the Printf() or the
other methods to log stuff. Say I have also configured a middleware to
log data for every request, once again using the Printf() and family
functions.
Now, I want to implement some levelled and/or structured logging. So,
here's my expectation - I should just be able to update the initial
code and everything else should not be changed. Of course, in due
course of time, I will update those to take advantage of the actual
functionality offered by those libraries - levelled logging,
structured logging. But, initially, my code should just build and
work. Many years back, that's how I really found
https://github.com/sirupsen/logrus useful and exciting. However, some
more performant and recent packages don't seem to have that
compatibility as a goal. For example, https://github.com/uber-go/zap
doesn't AFAICT implement the Printf() or other functions. However,
https://github.com/rs/zerolog does, which is great.
I imagine this possibly is due to the standard library not exporting
publicly available types for - logging handlers, formatters (encoders)
and such that third party packages have difficulty implementing these
in standard library compatible way?
To summarize, I would have really wanted a way to avoid my code
becoming tightly coupled with the logging library I am using but that
seems to be hard at this stage.
What does the community think?
Amnon
Aug 29, 2021, 2:10:54 AM8/29/21
to golang-nuts
Yes, this is a massive pain.
The standard library's log.Logger is not an interface, despite ending in er.
If it was an interface, it would be easy for third party packages to implement it, and then it would
be a lowest common denominator for any package where doing some logging could be useful.
Unfortunately it is too late to change this.
The standard library's log.Logger is not an interface, despite ending in er.
If it was an interface, it would be easy for third party packages to implement it, and then it would
be a lowest common denominator for any package where doing some logging could be useful.
Unfortunately it is too late to change this.
Sean Liao
Aug 29, 2021, 3:18:16 AM8/29/21
to golang-nuts
zap for example provides both:
RedirectStdLog to capture all output and
NewStdLog to produce a preconfigured std log.Logger
Leo Baltus
Aug 30, 2021, 6:43:32 PM8/30/21
to golang-nuts
Maybe it is worth checking out:
On Sunday, August 29, 2021 at 8:10:54 AM UTC+2 Amnon wrote:
Kevin Chowski
Aug 30, 2021, 6:58:23 PM8/30/21
to golang-nuts
Can you clarify why having log.Logger be an interface would be helpful?
Any library can define their own Logger interface with the same methods as the log.Logger type (e.g. Printf, Fatalf), and that would allow that library to take a reference to some generic "logging implementation" without being tied to the standard library. One could argue that there should be a centrally-defined interface which all possible loggers should "definitely" implement, but given that one could define their own interface with a subset of necessary functions it does not seem like a big deal.
Any library can define their own Logger interface with the same methods as the log.Logger type (e.g. Printf, Fatalf), and that would allow that library to take a reference to some generic "logging implementation" without being tied to the standard library. One could argue that there should be a centrally-defined interface which all possible loggers should "definitely" implement, but given that one could define their own interface with a subset of necessary functions it does not seem like a big deal.
To the original poster: can you just declare your own Logger-like interface and allow users of your library to pass whatever logging implementation they have? In what ways are you thinking that you're forced to be coupled with the standard library's implementation?
Amit Saha
Aug 30, 2021, 10:19:29 PM8/30/21
to Kevin Chowski, golang-nuts
On Tue, Aug 31, 2021 at 8:58 AM Kevin Chowski <Ke...@chowski.com> wrote:
<snip>
> To the original poster: can you just declare your own Logger-like interface and allow users of your library to pass whatever logging implementation they have? In what ways are you thinking that you're forced to be coupled with the standard library's implementation?
Yes, I can do that - that's a good idea actually - although more
complicated than one I would have liked for my use-case (more on this
in my original post and below), but certainly a feasible one with long
term benefits.
> In what ways are you thinking that you're forced to be coupled with the standard library's implementation?
When I first add some logging to my application, I am doing it using
the standard library's log.Printf(), log.Println(), log.Fatal() and
friends. Now, I reach a point of time where I want to start structured
logging but i want to do it incrementally. My existing Printf(),
Println() calls should continue to build. The only change I want to do
initially is simply change how I am constructing the log.Logger
object.
<snip>
> To the original poster: can you just declare your own Logger-like interface and allow users of your library to pass whatever logging implementation they have? In what ways are you thinking that you're forced to be coupled with the standard library's implementation?
complicated than one I would have liked for my use-case (more on this
in my original post and below), but certainly a feasible one with long
term benefits.
> In what ways are you thinking that you're forced to be coupled with the standard library's implementation?
the standard library's log.Printf(), log.Println(), log.Fatal() and
friends. Now, I reach a point of time where I want to start structured
logging but i want to do it incrementally. My existing Printf(),
Println() calls should continue to build. The only change I want to do
initially is simply change how I am constructing the log.Logger
object.
Rupert Paddington
Aug 31, 2021, 4:47:47 AM8/31/21
to golang-nuts
If Logger was an interface it would be pretty large and to some extent application-specific, which is undesirable.
Instead inject the sink (io.Writer).
This is preferable in every way:
* the interface is small and universal
* it eliminates the coupling with any specific library present and future
* it requires to test the behaviour instead of the collaboration
These are the same principles why for example the http.Handler interface takes an http.ResponseWriter, and then the outcome of the response can be tested by reading from the writer that is passed at testing time.
Robert Engels
Aug 31, 2021, 7:47:02 AM8/31/21
to Rupert Paddington, golang-nuts
The io.Writer is too low level to be useful. Imagine a logger that wants to send an sms on severe errors only.
The best solution is to declare your own logger interface - but similar to the collections discussion it would better if the stdlib declared an ILogger interface (or similar)
On Aug 31, 2021, at 3:48 AM, Rupert Paddington <blm...@gmail.com> wrote:
If Logger was an interface it would be pretty large and to some extent application-specific, which is undesirable.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/c76931fb-4d7e-40f2-84d3-8923dd280748n%40googlegroups.com.
Amit Saha
Aug 31, 2021, 5:02:32 PM8/31/21
to Robert Engels, Rupert Paddington, golang-nuts
On Tue, Aug 31, 2021 at 9:46 PM Robert Engels <ren...@ix.netcom.com> wrote:
>
> The io.Writer is too low level to be useful. Imagine a logger that wants to send an sms on severe errors only.
>
> The best solution is to declare your own logger interface - but similar to the collections discussion it would better if the stdlib declared an ILogger interface (or similar)
I found this FAQ entry on zap's project:
>
> The io.Writer is too low level to be useful. Imagine a logger that wants to send an sms on severe errors only.
>
> The best solution is to declare your own logger interface - but similar to the collections discussion it would better if the stdlib declared an ILogger interface (or similar)
"""
Why aren't Logger and SugaredLogger interfaces?
Unlike the familiar io.Writer and http.Handler, Logger and
SugaredLogger interfaces would include many methods. As Rob Pike
points out, "The bigger the interface, the weaker the abstraction."
Interfaces are also rigid — any change requires releasing a new major
version, since it breaks all third-party implementations.
Making the Logger and SugaredLogger concrete types doesn't sacrifice
much abstraction, and it lets us add methods without introducing
breaking changes.
Your applications should define and depend upon an interface that
includes just the methods you use.
"""
The last sentence is interesting here which is pointing application
authors to, I think define their own interface for logging which can
then be implemented by a concrete type such as standard library's
logger or zap's or zerolog's.
Robert Engels
Aug 31, 2021, 5:54:18 PM8/31/21
to Amit Saha, Rupert Paddington, golang-nuts
I don’t agree with Rob’s assessment. Standardized interfaces is what has allowed the Java 3P library ecosystem to be so robust.
It takes more design work upfront and you can get things wrong - but it’s a price and risk worth taking in my opinion.
> On Aug 31, 2021, at 4:02 PM, Amit Saha <amits...@gmail.com> wrote:
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANODV3kQ0uy%3Doi2FvJXtvSeY-k9zVU0AX5BB6HpJJCV8D3D5qQ%40mail.gmail.com.
It takes more design work upfront and you can get things wrong - but it’s a price and risk worth taking in my opinion.
> On Aug 31, 2021, at 4:02 PM, Amit Saha <amits...@gmail.com> wrote:
Reply all
Reply to author
Forward
0 new messages