Re: Introducting Mule - A worker process pool for CPU intensive tasks

[kuhnza]

One thing I am keen to know right off the bat is what's the standard practice for logging within node libraries? Right now mule simply uses console.log but I don't think this is an ideal solution.

What are others doing here?

On Monday, August 6, 2012 10:32:52 AM UTC-7, kuhnza wrote:

Hey guys & girls,

First time open sourcing something for the node community. Hope some find it useful. You can get it here:

https://github.com/Hubify/node-mule, or here;

npm install mule

We created it for use at Hubify after trying some of the other options out there such as Q-Oper8 and found they weren't particularly up to date or suited to our problem.

We'd love to hear your suggestions on how it could be made it better.

Cheers,
Dave

[Tim Caswell]

I use console.log. I override the function when I want to redirect the output.

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

[kuhnza]

Interesting, thanks for that...what do you think about using something like winston (which we currently use for all our custom stuff)? Do the same principle still apply, or is it bad form to use third-party logging in libs?

Dave

> nodejs+unsubscribe@googlegroups.com

[Tim Caswell]

It's only "bad form" if it doesn't fit your use case. Every situation
is different. That's the nature of software. I tend to write mostly
small libraries and micro-sites. Winston is overkill for me. I don't
know what's best for other people because I'm not in their shoes.

>> > nodejs+un...@googlegroups.com

>> > For more options, visit this group at
>> > http://groups.google.com/group/nodejs?hl=en?hl=en
>

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to

> nodejs+un...@googlegroups.com

[Matt]

I prefer if you at least have a level of indirection away from console.log, so that I can override it (or pass in a "log" function to a constructor of some sort) without having to stomp on console.log.

--

[Arnout Kazemier]

Just create an EventEmitter instance and emit your log events to there. People who then want to have logging enabled can hook up their own logging library. Or just listen to the emitted log messages using console.log

--

[Tim Caswell]

On Tue, Aug 7, 2012 at 9:54 AM, Matt <hel...@gmail.com> wrote:
> I prefer if you at least have a level of indirection away from console.log,
> so that I can override it (or pass in a "log" function to a constructor of
> some sort) without having to stomp on console.log.

The thing is, what is the purpose of console.log? It is a "log"
function after all. If I want to write data to stdout, I use
process.stdout.write(). If I want to log something to the console, I
use console.log. In vfs-child where I use stdout as a data channel, I
redirect console.log to stderr and all code continues working as
expected.

[Thomas Blobaum]

I use console.log with rconsole which adds some extra functionality

https://github.com/tblobaum/rconsole


Thomas Blobaum

[kuhnza]

Thanks Tim. Based on what you're saying I'm leaning to just leaving the console.log statements in there. Though, I'm a little unclear on how you'd  actually go about overriding console.(log|warn|error) etc. in client code. Are you able to provide an example of how you'd do it?

Something like this in my client code doesn't work because my winston config has the console logger transport setup which consequently blows the call stack:

console.log = function(msg) {

  logger.info(msg);

};

Dave

>> > nodejs+unsubscribe@googlegroups.com

>> > For more options, visit this group at
>> > http://groups.google.com/group/nodejs?hl=en?hl=en
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to

> nodejs+unsubscribe@googlegroups.com

[kuhnza]

Not a bad idea. Are you aware of any other libraries that do this? Trying to gauge how common this approach is.

nodejs+unsubscribe@googlegroups.com

[Matt]

On Tue, Aug 7, 2012 at 11:28 AM, Tim Caswell <t...@creationix.com> wrote:

On Tue, Aug 7, 2012 at 9:54 AM, Matt <hel...@gmail.com> wrote:
> I prefer if you at least have a level of indirection away from console.log,
> so that I can override it (or pass in a "log" function to a constructor of
> some sort) without having to stomp on console.log.

The thing is, what is the purpose of console.log?  It is a "log"
function after all.  If I want to write data to stdout, I use
process.stdout.write().  If I want to log something to the console, I
use console.log.  In vfs-child where I use stdout as a data channel, I
redirect console.log to stderr and all code continues working as
expected.

OP asked what was preferred and I gave my opinion. Not everyone is going to use console.log in exactly the desired way. I'm just stating what I'd prefer to see.

Plus this way if I don't want to see messages from library X but I do from library Y I can do so without having to carefully grep.

Matt.

[kuhnza]

Yeah that's exactly what I'm trying to avoid too Matt. I hate it when libs indiscriminately fill up my logs with no easy way to control the verbosity/formatting of their messages.

Is there anything like SLF4J for node? That way you could simply set the logging implementation at the module level and be done with it.

[Arnout Kazemier]

Nope most libs create their own logging libs or make it really hard to silence the logs. Console log statements are a pita because as a developer you really dont want to override build in functionality because some module is using that as a "logger" 

nodejs+un...@googlegroups.com

[Tim Caswell]

I should mention that I only leave console.log statements in
production code for rare cases (like noting an http server was created
and is listening). I try to never do it in libraries I publish
because my users might not care about that information.

I like the idea of https://github.com/visionmedia/debug for
user-configurable (via environment variables) debug logs. Node core
has something like this built-in as well.

[kuhnza]

That's my feeling also. Just did a quick survey of my node_modules folder and found that most libs in there simply don't perform any logging at all (or even if they once did it's been stripped out). It's almost as though folks have thrown it in the too hard basket and moved on. Doesn't feel right.

[Matt]

I agree - it's not right. It's very annoying to have libraries do no logging at all. It's fine when they work... but when they go wrong I want logs!

nodejs+un...@googlegroups.com

[kuhnza]

Found this discussion on the lists from last year: https://groups.google.com/forum/?fromgroups#!topic/nodejs/YoHblrE8JJM but it appears as though the discussion stalled. 

One of the comments says it's pretty easy to re-route console output which is true, but what do you do in the instance that your logging framework of choice is also logging to console? Surely you shouldn't have to sift through looking for what to keep and what to throw away/reformat...or am I looking at this the wrong way?

[Tim Caswell]

Matt, David,

Ignore what I said about how I use console.log, but what do you think
about TJ's debug library? To me it seems to solve your problems in a
really elegant way.

>>>>> nodejs+un...@googlegroups.com

>>>>> For more options, visit this group at
>>>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>>>>
>>>>>
>>>> --
>>>> Job Board: http://jobs.nodejs.org/
>>>> Posting guidelines:
>>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>>> You received this message because you are subscribed to the Google
>>>> Groups "nodejs" group.
>>>> To post to this group, send email to nod...@googlegroups.com
>>>> To unsubscribe from this group, send email to

>>>> nodejs+un...@googlegroups.com

>>>> For more options, visit this group at
>>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>>
>>> --
>>> Job Board: http://jobs.nodejs.org/
>>> Posting guidelines:
>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To post to this group, send email to nod...@googlegroups.com
>>> To unsubscribe from this group, send email to

>>> nodejs+un...@googlegroups.com

>>> For more options, visit this group at
>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to

> nodejs+un...@googlegroups.com

[Matt]

On Tue, Aug 7, 2012 at 1:58 PM, Tim Caswell <t...@creationix.com> wrote:

Matt, David,

Ignore what I said about how I use console.log, but what do you think
about TJ's debug library?  To me it seems to solve your problems in a
really elegant way.

Yup absolutely. And I much prefer being able to control these things with env vars.

[Arnout Kazemier]

debug is quite okay, but again, the problem with it is that you cannot supply it your own logging

instance, it writes it to console.log and console.error.

[kuhnza]

TJ's lib looks alright, having the option to control via environment variables is probably an acceptable solution.

This discussion has led me to delve a little deeper into the winston source code and I think there's a problem there for which I've raised an issue/pull request on GitHub (https://github.com/flatiron/winston/issues/162). To your original point Tim I think I should be able to override console.log without nasty side effects and redirect those messages to my logging implementation of choice. The current implementation of winston makes that impossible.

Maybe that combined with some environment vars to suppress logging is the way to go?

nodejs+unsubscribe@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/nodejs?hl=en?hl=en

--

Job Board: http://jobs.nodejs.org/

Posting guidelines:

https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines

You received this message because you are subscribed to the Google

Groups "nodejs" group.

To post to this group, send email to nod...@googlegroups.com

To unsubscribe from this group, send email to

nodejs+unsubscribe@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/nodejs?hl=en?hl=en

--

Job Board: http://jobs.nodejs.org/

Posting guidelines:

https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines

You received this message because you are subscribed to the Google

Groups "nodejs" group.

To post to this group, send email to nod...@googlegroups.com

To unsubscribe from this group, send email to

nodejs+unsubscribe@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/nodejs?hl=en?hl=en

--

Job Board: http://jobs.nodejs.org/

Posting guidelines:

https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines

You received this message because you are subscribed to the Google

Groups "nodejs" group.

To post to this group, send email to nod...@googlegroups.com

To unsubscribe from this group, send email to

nodejs+unsubscribe@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/nodejs?hl=en?hl=en

--

Job Board: http://jobs.nodejs.org/

Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines

You received this message because you are subscribed to the Google

Groups "nodejs" group.

To post to this group, send email to nod...@googlegroups.com

To unsubscribe from this group, send email to

nodejs+unsubscribe@googlegroups.com