lnav picks generic_log over custom

142 views
Skip to first unread message

Sam Bagatin

unread,
May 21, 2020, 11:08:06 AM5/21/20
to lnav
I made a custom log format that I want to use for all our internal logs. Can you help me understand what's wrong?

I have lnav 0.8.5

Thanks!

Logs have this kind of format
[2020-05-18 13:36:23,988] - [main]- DEBUG Configuration:1421 - hibernate.cache.use_second_level_cache=true
[2020-05-18 13:36:23,988] - [main]- DEBUG Configuration:1421 - hibernate.use_sql_comments=true
[2020-05-18 13:36:23,989] - [main]- DEBUG Configuration:1421 - hibernate.generate_statistics=true



{
        "custom_log" : {
                "title"                 : "Java log format",
                "description"           : "Log format used by log4j and output by most java programs",
                "regex" : {

                        "custom1" : { "pattern": "^(?<timestamp>\\[\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2},\\d{3}\\]) - \\[\\w+\\]- (?<level>\\w+) (?<class>\\w+):\\d{3} - (?<body>.*)$"
                        }
                },
                "level-field" : "level",
                "level" : {
                        "error" : "ERROR",
                        "warning" : "WARN",
                        "debug" : "DEBUG",
                        "info" : "INFO"
                },
                "timestamp-format":[
                        "[%Y-%m-%d %H:%M:%S,%L]"
                ],
                "value" : {
                        "class"         : { "kind" : "string"},
                        "body"          : { "kind" : "string"}
                },
                "highlights" : {
                        "debug" : {
                                "pattern" : "IMPORTANT",
                                "color" : "Red"
                        }
                },
                "sample" : [
                        {

                                "line"  : "[2020-05-18 15:28:58,333] - [main]- DEBUG FileExtractor:410 - about to open the file"
                        }
                ]
        }
}                                                                                                                                            

Timothy Stack

unread,
May 21, 2020, 1:57:29 PM5/21/20
to Sam Bagatin, lnav
On Thu, May 21, 2020 at 8:08 AM Sam Bagatin <samuele...@gmail.com> wrote:
I made a custom log format that I want to use for all our internal logs. Can you help me understand what's wrong?

In the pattern, you have -- (?<class>\\w+):\\d{3} -- which will only match line numbers
with three digits.  But, the example log has four digits "Configuration:1421".

tim

 
--
You received this message because you are subscribed to the Google Groups "lnav" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lnav+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lnav/b93e9388-d19a-4ddd-8722-5c9ceeb642bf%40googlegroups.com.
Message has been deleted

Sam Bagatin

unread,
May 21, 2020, 2:32:22 PM5/21/20
to lnav
Thanks Timothy. I changed the regex to

"custom1" : { "pattern": "^(?<timestamp>\\[\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2},\\d{3}\\]) - \\[\\w+\\]- (?<level>\\w+) (?<class>\\w+):\\d+ - (?<body>.*)$"


But the problem persists.
To unsubscribe from this group and stop receiving emails from it, send an email to ln...@googlegroups.com.

Timothy Stack

unread,
May 21, 2020, 2:53:41 PM5/21/20
to Sam Bagatin, lnav

Add some of the log messages from the example you gave into the sample
lines in the log format.  lnav will validate the sample lines against the format
and should print out some helpful errors about which part of the message
did and did not match the pattern.

tim

To unsubscribe from this group and stop receiving emails from it, send an email to lnav+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lnav/0d18445c-8f68-45d4-8a39-1358f394dbbe%40googlegroups.com.

Sam Bagatin

unread,
May 21, 2020, 5:24:43 PM5/21/20
to lnav
Hi Timothy,
s
Thank you so much! There were quite a few lines which have a nasty extra space and that was causing the issue.
The next issue I have relates to the highlights, which doesn't work

        highlights" : {
            "main" : {
                "pattern" : "main",
                "color" : "#ff0000"
            }
        },
 I am expecting this line to have main red colored
[2020-05-18 15:28:58,333] - [main]- DEBUG AbstractBatcher:4100 - about to open PreparedStatement


Timothy Stack

unread,
May 21, 2020, 11:59:13 PM5/21/20
to Sam Bagatin, lnav

Hmm, yeah, the highlight seems to be broken on v0.8.5.  It seems
to work on the top-of-tree.  Would you be able to give that a try?

tim
To unsubscribe from this group and stop receiving emails from it, send an email to lnav+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lnav/65a51854-dc6d-46b4-a9b5-88add7c40bc9%40googlegroups.com.

Sam Bagatin

unread,
May 22, 2020, 4:44:45 AM5/22/20
to lnav
Umh, doesn't seem to change much.

It works, but only in the <body section>. Everything after "Successfully"

[18:10:25,751] DEBUG DispatcherServlet:591 - Successfully completed request


Reply all
Reply to author
Forward
0 new messages