What is missing in Logger?

[José Valim]

Folks,

When we started the Logger library, we have brought in many features but some of them were postponed to after 1.0. You can see the full list here:

https://github.com/josevalim/logger

The remaining features we have pending today:

1. Tracing support: it mostly includes support to perform certain actions based on metadata

2. File logging support and log rotation: the ability to log to files and rotate logs

I believe tracing support would be useful and feedback on what you would like to see when it comes to tracing is very welcome. However, I am not convinced on the usefulness of file logging support nor log rotations as tools like logrotate do an excellent job and it is easy to pipe the existing log to a file. Thoughts?

Also, is there any other feature you would like to see in Logger?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Alexei Sholik]

I remember there was something in OTP for managing log files and their rotation. Is that tightly couple with error_logger so that we cannot use it with our Logger?

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Lzgv8F3q9VU4Q%3DzBVW6wqwFwF4yGnU%2BntkS1CgVJmQTQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Booker Bense]

Maybe I'm hopelessly old school, but I'd like logs to go to syslog. 

I would like to see at least a couple choices of backend in the standard

library. File, syslog, stdin to an external process. Getting logging data 

into your monitoring system is a key feature for modern applications. 

While it seems straightforward enough to "just roll your own" for this,

it's something I would expect of any standard logging API. 

Interacting effectively with logrotate also requires a bit of twiddling 

to reset the log based on an external command. It would be nice to

standardize that. As someone that most runs other people's code for

a living I hate it when they create non-standard ways to do logging. 

- Booker C. Bense 

[José Valim]

Agreed syslog is useful. There is a backend end for it too:

https://github.com/smpallen99/syslog

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/55782096-fc7e-44b1-b413-c53d092aa5df%40googlegroups.com.

[Peter Hamilton]

+1 for using logrotate rather than have the application handle it.

On Sun, May 10, 2015 at 4:43 AM Robert Stockdale <robert.s...@gmail.com> wrote:

Custom log levels per module/logger would be a nice enhancement.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4Lzgv8F3q9VU4Q%3DzBVW6wqwFwF4yGnU%2BntkS1CgVJmQTQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAB0F1_xi1f38YaDhBcKTZ6PSqC70zsiWPeYpO5tFnOs2%3DRdXNw%40mail.gmail.com.

[miwee]

I run Elixir on my windows machine for development. logrotate, syslog and friends are not available. However, I don't intend to run production code on windows so that is not the issue. All I want is file based logging, so that I can debug some long running applications. There is a backend available, which meets my need fine.
https://github.com/onkel-dirtus/logger_file_backend

I'm mentioning this, just in case if it's considered useful enough to make it part of the default logger library.

thanks
miwee

[Josh Adams]

Custom log levels per application are all I feel is missing

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Lzgv8F3q9VU4Q%3DzBVW6wqwFwF4yGnU%2BntkS1CgVJmQTQ%40mail.gmail.com.

[José Valim]

Just to be clear. What does everyone mean by custom log levels per module/logger? Would you like to say: print all log messages that is happening in this module/application as debug?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

On Mon, May 11, 2015 at 10:05 AM, David Rouchy <dro...@gmail.com> wrote:

+1 for Custom log levels per module/logger

that'd be very handy when I need to debug only one part of the system without being flooded by the debug statements

David

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/93b0f89e-7f32-470e-88ce-5e7380abb576%40googlegroups.com.

[Josh Adams]

I want to be able to configure how logger behaves depending on the application. Others suggested module, which sounds potentially better. Imagine doing this in config.exs of the hosting application

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4LU%3D4tKXuvmsGu3TnvtGHMXQEtss7ZpqBn_Pm-Bdw%3DOag%40mail.gmail.com.

[Drew Olson]

Big +1 to custom configuration based on module / application / library / whatever. I want to be able to log warnings for my app but tell ecto to log basically nothing.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAA1-O0zZqFQq%3D%2B7E0cO1-TJ8MgROJdGyceTe6Ur5AP0_va7vJw%40mail.gmail.com.

[Dmitry Aleksandrov]

Would be nice to have application somewhere as __CALLER__.application to be accessible, in the same way, as __MODULE__, that is really, what is lacking in erlang. 

On compiling of a module, it is possible to detect, which module it is, but it is not possible automatic detect application, which will be a very nice feature (for tracing on application/module).

Now, Logger doing: [module: module, function: form_fa(fun), line: line], if we can add :application to it, than it will be possible to have per module/application tracing facilities. And it is really, what all want, I think. With it it will be possible to do Logger.trace(application: my_app, :debug).

I'll personally think, that logger should support backends: for a best case lager-compatible backends(? - do not exactly, if it makes sense).

And, it should be performant and stable at least as lager is become over the time: ( I think, it is known presentation: https://www.youtube.com/watch?t=2614&v=8BNpOHFvg_Q ).

[Dmitry Aleksandrov]

I was needed to find, in what lager_parsetransform generates. And, lager have application attribute as metadata.

info() ->

   case {whereis(lager_event),

         lager_config:get(loglevel, {0, []})}

       of

     {undefined, _} ->

         fun () -> {error, lager_not_running} end();

     {Pidtest10, {Leveltest10, __Tracestest10}}

         when __Leveltest10 band 64 /= 0 orelse

                __Tracestest10 /= [] ->

         lager:do_log(info,

                      [{application, test}, {module, test},

                       {function, info}, {line, 10},

                       {pid, pid_to_list(self())}, {node, node()}

                       | lager:md()],

                      "info:~s", ["test"], 4096, 64, __Leveltest10,

                      Tracestest10, Pidtest10);

     _ -> ok

   end.

Logger doesn't it at the moment.

[José Valim]

Do you have any idea how they find out which application is it? Do they hook into rebar? I would love to support this feature but I don't know how to do it without some sort of Mix + Logger integration (which may be fine).

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/48d21676-5bc3-4216-abb9-964e03546c68%40googlegroups.com.

[Dmitry Aleksandrov]

They do it directly in lager:

https://github.com/basho/lager/blob/master/src/lager_transform.erl#L265

I think, that __CALLER__ meta data information should have something like application field, which can be modified by mix for such use cases. It can be practical in other situation, not only in Logger.

https://github.com/elixir-lang/elixir/blob/master/lib/mix/lib/mix/compilers/elixir.ex#L112-L115

Like adding `application:`, which adds this information to enviroments, like module, function? Do you see problems in this approach?

[José Valim]

The way they do it won't work with Mix/Logger because they are assuming there is an .app or .app.src in the output directory at compilation time which is not true for Mix projects.

We would need some sort of configuration, Application.get_env(:logger, :current_application), and Mix could set it before compiling stuff.

Please open up an issue about this particular bit.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Pedro Medeiros]

It's a good idea. Really liked it. maybe we can do it with macros and instead of calling Logger.log(:my_level "Mu Log message") we can just call Logger.my_level "log message"

On Thu, Jul 2, 2015 at 3:39 PM, Roman Chvanikoff <chvan...@gmail.com> wrote:

Could be nice to have ability to use custom, dynamic log-levels. Currently we only have "error", "warn", "info" and "debug" levels and what I want is ability to do something like Logger.log(:my_level, "My log message") to get "21:34:31.786 [my_level]  My log message". All custom "log-levels" might be treated as "debug" level.

воскресенье, 10 мая 2015 г., 11:59:14 UTC+3 пользователь José Valim написал:

--

You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/2817a16a-ecf5-4bcb-878f-2076daf1bd93%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Pedro Henrique de Souza Medeiros
----------------------------------
Cel: +55 (61) 9197-0993
Email: pedr...@gmail.com

Beautiful is better than ugly,
Explicit is better than implicit,
Simple is better than complex,
Complex is better than complicated.

The Zen of Python, by Tim Peters

[José Valim]

We can't dynamically define functions on the Logger module. I am still reluctant to add new levels, wouldn't metadata help solve this?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[José Valim]

Once we support tracing, this should be achievable with metadata too. :)



--

[Devin Torres]

The feature I have always wanted but am too stupid to implement is the ability to adjust log level in config.exs *per application*. As it is today, I cannot enable debug level without all the applications my library uses spamming me.

On Fri, Jul 3, 2015 at 12:47 AM, Roman Chvanikoff <chvan...@gmail.com> wrote:

Ok, I still think syntax format of Logger.log(:mylevel, msg) is better comparing to Logger.debug(msg, level: :mylevel) but in case it's not necessary to achieve routing to different files, I'm ok with it.

пятница, 3 июля 2015 г., 8:19:59 UTC+3 пользователь José Valim написал:

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/27501c0d-4654-4e96-a1f0-0e50a1514907%40googlegroups.com.

[Bruno Matos Tavares]

Hi,

When the process state or last message is "big" the stacktrace might not be visible because is truncated. Is it possible to have more granularity on what we truncate? 

Bruno.

[José Valim]

We have an open issue to move the stacktrace information before the state due to such issues. There is also a truncate configuration which you can change.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Bruno Matos Tavares]

The truncate configuration is for the all message. 

Having a bit more granularity would be great as we can express:

config :logger, :console,

truncate: 8192,

truncate_last_message: 1000, # defaults to `truncate`

truncate_state: 2000 # defaults to `truncate`

Then stackstrace would be whatever space remains to the truncate. 

I can’t find the open issue that you refer to. 

Bruno.

[James Fish]

The order has been changed in master already: https://github.com/elixir-lang/elixir/commit/6cd1478a2ec1a3dad7f02fb5b9291abe3a3a427f.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/BAFFDDFF-6333-4225-B9F8-8BB8381B0C10%40gmail.com.

[Alex Shneyderman]

However, I am not convinced on the usefulness of file logging support nor log rotations as tools like logrotate do an excellent job and it is easy to pipe the existing log to a file.

This train of thought clashes with releases. VM is started with -daemon flag which starts things with no console at all. 

I guess in deciding of what backend should be included in Logger there is a need for a consistent criteria. To me console and file backends seem to satisfy criteria of being built with the tools that are already shipping with elixir. They do not require any thrid-parties. For example, same can not be said of syslog or loggly or other stuff that requires additional services. 

I believe file backend is a must. ATM we are handicapped in a particular deployment scenario (releases as I mentioned earlier). So it seems like file and console  backends are  complementary in that sense.

Sure, we can develop our own backend but in that case what's special about console? I see no reason for it being part of the core distribution ATM.

I think making file backend as simple as  possible is also acceptable as tools like logrotate can do their magic to help with the rest of the needs.

Cheers,

Alex.

[Bruno Matos Tavares]

Alex you have a point there, however, how likely one will use a simple file backend in production?

Just by speaking on having a file backend we had/have a discussion if the backend should rotate the logs. IMO, this kind of indicates that having a file backend will open a pandora box. We could discuss if the file backend or even Logger should allow to swap the Logger.Formatter for a custom formatter because one would like JSON logs entries or log to different files based on the log level.

Speaking by my own experience (I've develop ExSyslog https://hex.pm/packages/exsyslog) I had to allow the features above. So, I'd always have to develop or use a custom backend.

Cheers,

Bruno.