Cut-off log level?

[David Dossot]

Hi loggers,

First: thanks a lot for this project!

I have noticed that each logger has the responsibility to drop a log
message if it is below its level, which is fine.

The only problem with this approach is when you emit a lot of messages
that have no chance of being logged by any logger (say at debug level
with a production system where all loggers are at info level). The
gen_server's inbox gets cluttered with messages that will be ignored
which slows down the logging of valid messages, as they wait for their
turn to be processed.

Could it be possible to have the client optimized so it pre-emptively
drops log messages that have no hope of being handled by any logger?
This new behaviour could be off by default and activated with a
boolean to maintain backwards compatibility.

Cheers,
D.

[Ahmed Ali]

Hi David,

This is one the optomizations that I've been thinking of for Log4erl.
Until now, I don't have a clear idea of how to do this in a clean way.
The simplest, and probably, the best solution, is to have a MACRO
which calls log4erl if a predefined log level is active. Something
like the marco below. I could have this come with Log4erl but clients
needs to include needed hrl file. (p.s. the code below is not tested!.
It's more like psodu-code).

If you have a better idea of how to do this, please share it with the
rest of us.

% 1=always, 2=debug, 3=info, 4=warn, 5=fatal
-define(LEVEL, 3).

-define(WARN(X), case ?LEVEL < 3 of true -> log4erl:warn(X); _ -> ok).
-define(DEBUG(X), case ?LEVEL < 2 of true -> log4erl:debug(X); _ -> ok).
-define(INFO(X), case ?LEVEL < 4 of true -> log4erl:info(X); _ -> ok).
-define(FATAL(X), case ?LEVEL < 5 of true -> log4erl:fatal(X); _ -> ok).

-define(WARN2(X,D), case ?LEVEL < 3 of true -> log4erl:warn(X,D); _ -> ok).
%% ...etc

[David Dossot]

Hi Ahmed,

I already had to add a macro for turning log4erl:debug when DEBUG!
=true but I really don't like it.

I think when you use a logging framework, you don't really want to
recompile to change its logging behavior.

Since this is merely a matter of optimization, what about my proposal
for an extra global optional configuration parameter that would be the
minimal log level above which log4erl client would send log messages
to the gen_server?

Thanks for your attention ;-)
David

On Oct 19, 10:01 am, Ahmed Ali <ahmed.naw...@gmail.com> wrote:
> Hi David,
>
> This is one the optomizations that I've been thinking of for Log4erl.
> Until now, I don't have a clear idea of how to do this in a clean way.
> The simplest, and probably, the best solution, is to have a MACRO
> which calls log4erl if a predefined log level is active. Something
> like the marco below. I could have this come with Log4erl but clients
> needs to include needed hrl file. (p.s. the code below is not tested!.
> It's more like psodu-code).
>
> If you have a better idea of how to do this, please share it with the
> rest of us.
>
> % 1=always, 2=debug, 3=info, 4=warn, 5=fatal
> -define(LEVEL, 3).
>
> -define(WARN(X), case ?LEVEL < 3 of true -> log4erl:warn(X); _ -> ok).
> -define(DEBUG(X), case ?LEVEL < 2 of true -> log4erl:debug(X); _ -> ok).
> -define(INFO(X), case ?LEVEL < 4 of true -> log4erl:info(X); _ -> ok).
> -define(FATAL(X), case ?LEVEL < 5 of true -> log4erl:fatal(X); _ -> ok).
>
> -define(WARN2(X,D), case ?LEVEL < 3 of true -> log4erl:warn(X,D); _ -> ok).
> %% ...etc
>

[Ahmed Ali]

Hi David,

First of all, I apologize in advanced for the long email. However, I
would be happy to receive your input for the possible solutions I'm
listing below.

I understand your proposal. What I'm thinking of right now is the
correct way to add this into Log4erl. The issue is that I don't have
access to client's data structure and I cannot do that without adding
some restrictions on how code that's using log4erl.

For example, the way log4erl is implemented currently is by having 1
process for each logger. So, if I wanted to add a check for log level
before sending log message, then I'll have to call the process for
querying log level, check it's OK to pass log message, which I would
expect to be as fast, if not slower, than just sending the log message
anyways.

Another solution is to add an extra parameter to Log4erl API to always
send log level used by client. This is not a particularly good
solution as well.

A third solution is to have 1 process between the actual logger
process and the client, which adds to overhead in terms of message
passing and delay before processing log messages. I'm not sure how
this solution fares compares to not having this complexity in Log4erl
code base.

A forth solution is to have Log4erl automatically generate logger code
using something like smerl (from erlyweb) and it has check for
different levels in code that is run by client. This of course will
require an extra step when creating loggers and will mean a small
change in Log4erl API (e.g. instead of log4erl:warn/1 ->
Logger:warn/1).

From all of the solutions mentioned above, I think the last option is
the best. If you have a better implementation idea, please let me
know.

Best regards,

Ahmed

[David Dossot]

I agree that from the 4 solutions you suggest, the last one is the
most attractive.

Cheers,
David

On Oct 23, 1:37 pm, Ahmed Ali <ahmed.naw...@gmail.com> wrote:
> Hi David,
>

[Ahmed Ali]

Hi David,

It'll take more time though! I don't think it'll be available for the
next release.

Anyways, thanks for the discussion.

Best regards,

Ahmed

[David Dossot]

Hey Amhed,

FWW+FYI I have implemented a very basic client side cut off using code
generation. You can take a look at it on GitHub: http://github.com/ddossot/log4erl

Cheers,
David

On Oct 24, 4:24 pm, Ahmed Ali <ahmed.naw...@gmail.com> wrote:
> Hi David,
>

> It'll take more time though! I don't think it'll be available for the
> next release.
>
> Anyways, thanks for the discussion.
>
> Best regards,
>
> Ahmed
>