Akka and MDC

[Michael Slinn]

I was reading the Logback chapter on MDC when I realized that this feature probably is a very bad idea to use with Akka. From the docs:

The MDC manages contextual information on a per thread basis. A child thread automatically inherits a copy of the mapped diagnostic context of its parent. Typically, while starting to service a new client request, the developer will insert pertinent contextual information, such as the client id, client's IP address, request parameters etc. into the MDC. Logback components, if appropriately configured, will automatically include this information in each log entry.

Comments?

Mike

[Patrik Nordwall]

In case you didn't notice we answered a question about MDC a few days ago. Yes, it's not possible to use MDC with akka's async logging.

/Patrik

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/_mOOuQu2SXcJ.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[Brice Figureau]

On Thu, 2012-07-26 at 08:26 +0200, Patrik Nordwall wrote:
> In case you didn't notice we answered a question about MDC a few days
> ago. Yes, it's not possible to use MDC with akka's async logging.

But it would be awesome if the akka async logger would have an API to
emulate the MDC.

I partially solved the issue (I was trying to assign somewhat unique ids
to messages sent and was planning to use the MDC to decorate my actor
receive log entries with this id) by (ab)using a custom LogSource that
decorates the ActorRef/Actor LogSource string result with this message
id. I feel dirty :)

--
Brice Figureau
My Blog: http://www.masterzen.fr/

[√iktor Ҡlang]

On Thu, Jul 26, 2012 at 10:40 AM, Brice Figureau <bric...@daysofwonder.com> wrote:

On Thu, 2012-07-26 at 08:26 +0200, Patrik Nordwall wrote:
> In case you didn't notice we answered a question about MDC a few days
> ago. Yes, it's not possible to use MDC with akka's async logging.

But it would be awesome if the akka async logger would have an API to
emulate the MDC.

Sounds cool, care to do it?

 

I partially solved the issue (I was trying to assign somewhat unique ids
to messages sent and was planning to use the MDC to decorate my actor
receive log entries with this id) by (ab)using a custom LogSource that
decorates the ActorRef/Actor LogSource string result with this message
id. I feel dirty :)

Sounds like a good reason to get your hands dirty ;-)

Cheers,

√

 

--
Brice Figureau
My Blog: http://www.masterzen.fr/

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[Brice Figureau]

On Thu, 2012-07-26 at 10:42 +0200, √iktor Ҡlang wrote:
>
>
> On Thu, Jul 26, 2012 at 10:40 AM, Brice Figureau
> <bric...@daysofwonder.com> wrote:
> On Thu, 2012-07-26 at 08:26 +0200, Patrik Nordwall wrote:
> > In case you didn't notice we answered a question about MDC a
> few days
> > ago. Yes, it's not possible to use MDC with akka's async
> logging.
>
>
> But it would be awesome if the akka async logger would have an
> API to
> emulate the MDC.
>
>
> Sounds cool, care to do it?

Well, why not?
Let's see if I can find some time in the next couple of weeks to produce
a pull request.

[Michael Slinn]

I thought ThreadLocal storage should not be used with Akka. Seems likely that MDC uses ThreadLocal storage. Am I wrong, or are there caveats regarding when ThreadLocal is a good/bad idea with Akka?

I missed the mentions of MDC in the previous threads, so I took a look. I did not see any cautions mentioned about this in those threads.

Mike

[√iktor Ҡlang]

On Thu, Jul 26, 2012 at 11:35 PM, Michael Slinn <msl...@gmail.com> wrote:

I thought ThreadLocal storage should not be used with Akka. Seems likely that MDC uses ThreadLocal storage. Am I wrong, or are there caveats regarding when ThreadLocal is a good/bad idea with Akka?

ThreadLocalRandom

 

I missed the mentions of MDC in the previous threads, so I took a look. I did not see any cautions mentioned about this in those threads.

Mike

--

You received this message because you are subscribed to the Google Groups "Akka User List" group.

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/4BflVjfHtrAJ.

To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[Mike Slinn]

Say what?

[Patrik Nordwall]

26 jul 2012 kl. 23:35 skrev Michael Slinn <msl...@gmail.com>:

I thought ThreadLocal storage should not be used with Akka. Seems likely that MDC uses ThreadLocal storage.

Right

Am I wrong, or are there caveats regarding when ThreadLocal is a good/bad idea with Akka?

Bad

What is unclear?

I missed the mentions of MDC in the previous threads, so I took a look. I did not see any cautions mentioned about this in those threads.

Mike

--

You received this message because you are subscribed to the Google Groups "Akka User List" group.

To view this discussion on the web visit https://groups.google.com/d/msg/akka-user/-/4BflVjfHtrAJ.

[Brice Figureau]

Well, I just realized something:
* MDC is a logback only feature
* akka-slf4j has no hard dependencies on logback

If I want to implement an MDC like feature that ends up filling up the
MDC in akka-slf4j, at one point there will be a dependency to logback...

What would be the optimal way to manage this issue?

[√iktor Ҡlang]

http://akka.io/community/

 

--
Brice Figureau
My Blog: http://www.masterzen.fr/

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[Brice Figureau]

I was thinking about a patch against akka, since there must be
modifications of the log API.

[√iktor Ҡlang]

Doesn't that depend on the solution?

 

--
Brice Figureau
My Blog: http://www.masterzen.fr/

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

[Patrik Nordwall]

27 jul 2012 kl. 13:40 skrev Brice Figureau <bric...@daysofwonder.com>:

> On Thu, 2012-07-26 at 10:53 +0200, Brice Figureau wrote:
>> On Thu, 2012-07-26 at 10:42 +0200, √iktor Ҡlang wrote:
>>>
>>>
>>> On Thu, Jul 26, 2012 at 10:40 AM, Brice Figureau
>>> <bric...@daysofwonder.com> wrote:
>>> On Thu, 2012-07-26 at 08:26 +0200, Patrik Nordwall wrote:
>>>> In case you didn't notice we answered a question about MDC a
>>> few days
>>>> ago. Yes, it's not possible to use MDC with akka's async
>>> logging.
>>>
>>>
>>> But it would be awesome if the akka async logger would have an
>>> API to
>>> emulate the MDC.
>>>
>>>
>>> Sounds cool, care to do it?
>>
>> Well, why not?
>> Let's see if I can find some time in the next couple of weeks to produce
>> a pull request.
>
> Well, I just realized something:
> * MDC is a logback only feature

No, the API for MDC is in slf4j.

> * akka-slf4j has no hard dependencies on logback
>
> If I want to implement an MDC like feature that ends up filling up the
> MDC in akka-slf4j, at one point there will be a dependency to logback...
>
> What would be the optimal way to manage this issue?
> --
> Brice Figureau
> My Blog: http://www.masterzen.fr/
>

[Brice Figureau]

I finally had time to start working on this, but I couldn't find a
satisfactory API.

I'd like to have insights from akka dev, as to what would be preferred.

The main "concept", is that the mdc data is sent as a Map[String,String]
as part of the various Debug, Info... messages that are sent on the bus.
Then on the akka-slf4j side, everything is feed into the slf4j MDC
before the logging happens.

That's the easy part.

Now, back to akka.event.LoggingAdapter, my initial implementation would
have been used like this:

val log = Logging(...)
implicit val mdc = LoggingContext
...
mdc.put("akka", "is great!")
log.info("hello world!")
mdc.remove("akka")
...

LogginContext is just a small wrapper around a Map[String,String] that
more or less has the MDC api.

Where LoggingAdapter.info would be defined with;

trait LoggingAdapter {
...
def info(message: String)(implicit mdc: LoggingContext) { if (isInfoEnabled) notifyInfo(message, mdc) }
...
}

I thought this was correct and nice.

Unfortunately this breaks the Java logging API in a bad way.

So back to the drawing board, what about:

val mdc = LoggingContext
val log = Logging(context.system, this, mdc)
...
mdc.put("akka", "is great!")
log.info("hello world!")
mdc.remove("akka")
...

With info implemented as:

def info(message: String) { if (isInfoEnabled) notifyInfo(message, mdc) }
(where mdc would be a field in the trait)

Would that work?

Anyway, I'm open to any suggestions at that point :)

[Patrik Nordwall]

yes, that's nice for scala

 

Unfortunately this breaks the Java logging API in a bad way.

but not for java :-)

 

So back to the drawing board, what about:

val mdc = LoggingContext
val log = Logging(context.system, this, mdc)
...
mdc.put("akka", "is great!")
log.info("hello world!")
mdc.remove("akka")

LoggingContext could even be constructed inside LoggingAdapter, instead of being passed in, or is there any advantage of passing it in?

val log = Logging(context.system, this)

...

log.mdc.put("akka", "is great!")

log.info("hello world!")

log.mdc.remove("akka")

The thing I'm most concerned about here is how to avoid problems for multi-threaded use of a LoggingAdapter instance. In actors that is not an issue, but LoggingAdapter is not limited to that.

To avoid multi-threaded surprises I see only two alternatives:

* passing it in as a parameter to info (and all other) methods

  - drawback is the explosion of method parameter combinations and scala/java differences for implicit param as already mentioned

* passing it in as a thread local

  - drawback is leak and cleanup issues with thread local

I don't see a perfect solution for this.

Another thing that will be asked for pretty soon is to automatically pass the mdc across actors, futures (and remote nodes). Typical example, setting user id when web request enters the system. People are used to set stuff in the mdc and then it's propagated everywhere across method calls, and will expect that across actors as well.

We should probably not support that, and then we should ask ourselves how valuable mdc is in an asynchronous world without that feature.

 

...

With info implemented as:

def info(message: String) { if (isInfoEnabled) notifyInfo(message, mdc) }
(where mdc would be a field in the trait)

Would that work?

Anyway, I'm open to any suggestions at that point :)

--
Brice Figureau
My Blog: http://www.masterzen.fr/

--

>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: http://akka.io/faq/
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
---

You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.

Visit this group at http://groups.google.com/group/akka-user?hl=en.

--

Patrik Nordwall

Typesafe -  The software stack for applications that scale

Twitter: @patriknw

[Gastón Tonietti]

Hi,

There was some activity on this subject recently. I've written something about it.

http://ktonga.tumblr.com/post/67780441208/akka-meets-mdc

Hope it helps.

Gaston.