Smart logger?

[Сергей Жуматий]

Hello!

In my program I use several dynamically created goroutines and I want to write logs with goroutine name prefix. Name is set inside goroutine. I cannot just pass this name to logger function, because I use logging in many functions and passing name to EVERY function is crazy.

Is there way to make something like lambda or something else to implement "local" version of logger function, which is different for each goroutine context? I don't want to replace log package, just to implemet function like my_log(format, ...), which prints prefix and then formatted string.

Any ideas? Is is even possible in go?

[Konstantin Khomoutov]

Use a dead-simple type wrapping the standard logger; create and use an
instance of it in each goroutine: [1].

[Konstantin Khomoutov]

On Wed, 15 Apr 2015 15:25:08 +0300
Konstantin Khomoutov <flat...@users.sourceforge.net> wrote:

[...]

> > Is there way to make something like lambda or something else to
> > implement "local" version of logger function, which is different for
> > each goroutine context? I don't want to replace log package, just to
> > implemet function like my_log(format, ...), which prints prefix and
> > then formatted string.

[...]

> Use a dead-simple type wrapping the standard logger; create and use an
> instance of it in each goroutine: [1].

Sorry, forgot to include the playground link:
http://play.golang.org/p/DWhYqdG8LH

[C Banning]

Or a utility function: https://play.golang.org/p/l17VczVNDP

[gi...@iki.fi]

The language developers are not keen on allowing programmers to know which goroutine their code is running in for “it’s abusive power” of being able to use it to implement goroutine-local-storage.

• https://groups.google.com/d/topic/golang-nuts/lGtALXDJQIE/discussion
• https://groups.google.com/d/topic/golang-nuts/0HGyCOrhuuI/discussion
• https://www.google.fi/#q=golang+cpuid — though this one is thread-id, not goroutine id.
• https://github.com/jtolds/gls — for “acquiring the goroutine id” with a closure… no, don’t use it ;)

It appears that the only solution logging-wise is to pass common data along the function calls. People usually accomplish this by having some kind central structure that’s given to passed onto every function. One such is: go.net/context.

Hopefully my comment wasn’t completely off-topic.

​

[Сергей Жуматий]

> Use a dead-simple type wrapping the standard logger; create and use an
> instance of it in each goroutine: [1].

Sorry, forgot to include the playground link:
http://play.golang.org/p/DWhYqdG8LH

  Actually I need something for case like this: http://play.golang.org/p/51ka0IyFXP
  Here worker function can be called from any goroutine, but it must be able to log with prefix defined in caller and shouldnt' receive this prefix via arguments (I have tonns of such functions).
 

[Сергей Жуматий]

среда, 15 апреля 2015 г., 15:57:34 UTC+3 пользователь gi...@iki.fi написал:

The language developers are not keen on allowing programmers to know which goroutine their code is running in for “it’s abusive power” of being able to use it to implement goroutine-local-storage.

  Ough... That's bad fo me :)

It appears that the only solution logging-wise is to pass common data along the function calls. People usually accomplish this by having some kind central structure that’s given to passed onto every function. One such is: go.net/context.

  Yes, it is a working solution, but I need to add new argument to ALL my functions. It's hard and looks ugly , so I try to find more elegant solution.

 

[Konstantin Khomoutov]

As gima at iki.fi explained, you can't do that: goroutines do now have
identity, and so there's no way to query it.

In any case, what's the reason to do what you want? If you experience
a crash you want to debug, start with race detector. Or/and study the
call stack after the crash. Note that adding a logger into the mix
*will* affect the way your goroutines are scheduled, because executing
logging code has side effects. Moreover, IIUC, the standard logger
serializes its output, so your goroutines will unexpectedly contend on
the logger.

[C Banning]

http://play.golang.org/p/TBf8wN4keF

[Konstantin Khomoutov]

On Wed, 15 Apr 2015 09:47:54 -0700 (PDT)
C Banning <clba...@gmail.com> wrote:

> http://play.golang.org/p/TBf8wN4keF

It's not really gonna work as is: http://play.golang.org/p/DD2DLdl3lg

The problem is that the logging function might be called from an
unknown level on the call stack.

Supposedly this might be somehow fixed by crawling up the call stack
until "level 0" is reached, and using the name of the function at that
level.

[Konstantin Khomoutov]

On Wed, 15 Apr 2015 20:23:58 +0300
Konstantin Khomoutov <flat...@users.sourceforge.net> wrote:

> > http://play.golang.org/p/TBf8wN4keF
>
> It's not really gonna work as is: http://play.golang.org/p/DD2DLdl3lg
>
> The problem is that the logging function might be called from an
> unknown level on the call stack.
>
> Supposedly this might be somehow fixed by crawling up the call stack
> until "level 0" is reached, and using the name of the function at that
> level.

Well, this http://play.golang.org/p/-ypqnGA-Bt would work as a poor
man's solution to the stated problem.

Of course, this is not really a solution but this is at least something.

Still, I'm inclined to think the OP is going against the grain and
would be better off solving real problem rather than trying to mess
with logging.

[Сергей Жуматий]

Well, this http://play.golang.org/p/-ypqnGA-Bt would work as a poor
man's solution to the stated problem.

  Thank you! It's really working solution. Not so elegant, but the best possible, I think. If goroutines cannot be indentified it is the only way...
 

Of course, this is not really a solution but this is at least something.

  It is usefull for my goals :)
 

Still, I'm inclined to think the OP is going against the grain and
would be better off solving real problem rather than trying to mess
with logging.

  Of course. But my problem is significally dynamic and I can't use debugger to solve it. Let me get some explanations.
  My goroutines execute special data processors and pass data each other. Sometimes (it depends on input data) I caught internal logical error (like divide by zero or format mismatch) and then I must figure out which processor got it. Because cheks are inside basic classes used by all processors I need the subject :)

  Thank you all for answers! Go is really good language but I miss many things from ruby actually :) I understand - it is a cost of speed...

[Benjamin Measures]

> Sometimes (it depends on input data) I caught internal logical error (like divide by zero or format mismatch) and then I must figure out which processor got it.

Yeah, things are really hard to debug when one of your processors are faulty.

[Konstantin Khomoutov]

On Wed, 15 Apr 2015 23:05:30 -0700 (PDT)
Сергей Жуматий <serg...@gmail.com> wrote:

[...]

> > Still, I'm inclined to think the OP is going against the grain and
> > would be better off solving real problem rather than trying to mess
> > with logging.
> >
> Of course. But my problem is significally dynamic and I can't use
> debugger to solve it. Let me get some explanations.
> My goroutines execute special data processors and pass data each
> other. Sometimes (it depends on input data) I caught internal logical
> error (like divide by zero or format mismatch) and then I must figure
> out which processor got it. Because cheks are inside basic classes
> used by all processors I need the subject :)

If you have "processors", why not give them some identity so that your
logging subsystem knows about it? I mean, looks like you don't
actually want to answer the question "which goroutine?" but rather
"which processor?".

[Сергей Жуматий]

 You're right. But. Every processor calls many service functions and class methods. They also can call other functions, etc. I didn't want to pass special context (e.g. processor name) to EVERY function in my code, it's too ugly and non-portable. But, it seems, that I have no chioce :)

[Henrik Johansson]

Doesn't glog print line numbers? It is usually sufficient for me.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Janne Snabb]

On 2015-04-15 14:47, Сергей Жуматий wrote:
> Is there way to make something like lambda or something else to
> implement "local" version of logger function, which is different for
> each goroutine context?

This might help: https://github.com/inconshreveable/log15

It makes it easy to create new loggers which carry around some context.
You can also stack them on top of each other. You just need to pass
around the logger instance to your functions. For example:

sandLogger := logger.New("proc", "sand")
sandLogger.Info("start")
sandInputLogger := sandLogger.New("task", "input")
sandLogger.Info("waiting")

The above will produce something like the following:

t=2015-04-20T17:21:27-0400 lvl=info msg=start proc=sand
t=2015-04-20T17:21:27-0400 lvl=info msg=waiting proc=sand task=input

--
Janne Snabb
sn...@epipe.com