Multi-line regular expressions do not work in search and in filters (GH issue 788)

[Piotr Dobrogost]

Hi,

I raised "Multi-line regular expressions do not work in search and in filters" issue (https://github.com/tstack/lnav/issues/788) but as I got no response there I thought I would ask here. Could we please have this working? It is the single biggest missing feature when working with log files with multi-line entries.

Regards,

Piotr Dobrogost

[Timothy Stack]

On Sun, Nov 22, 2020 at 4:40 PM Piotr Dobrogost <p.dob...@gmail.com> wrote:

Hi,

I raised "Multi-line regular expressions do not work in search and in filters" issue (https://github.com/tstack/lnav/issues/788) but as I got no response there I thought I would ask here. Could we please have this working? It is the single biggest missing feature when working with log files with multi-line entries.

I saw the issue, but I'm not sure it's something I can fix right away.  The lnav internals

work on a line-oriented basis and not a message oriented-basis.  I was starting to make

changes to make things message-oriented, but haven't had time to complete it.  It might

be possible to do it in the current code-base, though.  I'll try to find some time to

investigate.

Regards,

Piotr Dobrogost

thanks,

tim

--
You received this message because you are subscribed to the Google Groups "lnav" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lnav+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lnav/ae5d0fa3-ed79-4435-881b-8185753a28bbn%40googlegroups.com.

[Timothy Stack]

Hi Piotr,

I've pushed changes for using a SQL expression to do filtering (as

mentioned in https://github.com/tstack/lnav/issues/568).  Since the

SQL integration gives you access to the full log message you

should be able to use a command like the following to accomplish

what you want:

  :filter-expr :log_text LIKE '%<some-tag>%<some-subsequent-tag>%'

I think this should satisfy your other request as well since you can

write arbitrarily complex SQL expressions to do filtering.

thanks,

tim

On Sun, Nov 22, 2020 at 4:40 PM Piotr Dobrogost <p.dob...@gmail.com> wrote:

Hi,

I raised "Multi-line regular expressions do not work in search and in filters" issue (https://d/tstack/lnav/issues/788) but as I got no response there I thought I would ask here. Could we please have this working? It is the single biggest missing feature when working with log files with multi-line entries.

Regards,

Piotr Dobrogost

[Piotr Dobrogost]

On Monday, November 30, 2020 at 6:19:35 AM UTC+1 Tim wrote:

Hi Piotr,

I've pushed changes for using a SQL expression to do filtering (as

mentioned in https://github.com/tstack/lnav/issues/568).  Since the

SQL integration gives you access to the full log message you

should be able to use a command like the following to accomplish

what you want:

  :filter-expr :log_text LIKE '%<some-tag>%<some-subsequent-tag>%'

I think this should satisfy your other request as well since you can

write arbitrarily complex SQL expressions to do filtering.

Tim,

I would love to try it out but I'm not sure I will find time to build it myself.

I thought using SQL would be the only way to use this (as shown above

with the "LIKE" operator) but I see there is a way to use regexp on SQL fields like

shown at https://github.com/tstack/lnav/issues/689#issuecomment-740253147 , yes?

If so that would be really awesome!

Regards,

Piotr

 

[Timothy Stack]

On Tue, Dec 8, 2020 at 5:26 AM Piotr Dobrogost <p.dob...@gmail.com> wrote:

Tim,

I would love to try it out but I'm not sure I will find time to build it myself.

If you're using a 64-bit linux, you could try to use one of the CI artifacts, like

here:

  https://github.com/tstack/lnav/actions/runs/406368388

Download the "Linux 64-bit" zip file, unzip it and "chmod +x" the lnav file inside.

 

I thought using SQL would be the only way to use this (as shown above

with the "LIKE" operator) but I see there is a way to use regexp on SQL fields like

shown at https://github.com/tstack/lnav/issues/689#issuecomment-740253147 , yes?

If so that would be really awesome!

You can use any functionality in SQLite and the extensions that are included

in lnav, see:

  https://lnav.readthedocs.io/en/latest/sqlext.html#reference

So, yes, you can use regexp functions to match a string.  If there is a JSON field,

you should be able to use the JSON functions to examine it for a particular value.

There's no real limit other than the performance cost of a complicated

expression.

Regards,

Piotr

tim