Does lnav sort multiple files by timestamp on open? (looks like no)

[hagf...@gmail.com]

Hey everyone,

I just loaded two different log files into lnav hoping it would sort by timestamp so I could see events in parallel between the two, but I discovered two separate instances of a given timestamp, about 1000 lines apart, with lots of other time in between.

It's worth noting that one file records timestamps as Apr 3 01:25:48 whereas the other is Apr 03 01:25:48. While lnav can handle the timestamp formats on each of these files individually, they don't sort together when opened with lnav.

I tried catting both files this way:

cat /var/log/messages /var/log/cluster/corosync.log | sort -k1 > combined.log

and then opened combined.log... after ~ 3 minutes of watching it load, I gave up at 17%. The machine this is running on Centos 7.0.1406, 8GB RAM, 8-2Ghz cores on an 8Gb/s FC VMware RDM backed by a Compellent SAN. The hardware isn't likely a limitation in this case; the combined line count of combined.log was 211k lines... so that just may be a lot of stuff to deal with.

Is there a way to make this sort of thing work that's more elegant than catting the files together? Correlating events across multiple logs is a marvelous thing.

Thanks!

[Timothy Stack]

On Fri, Apr 3, 2015 at 8:19 AM, <hagf...@gmail.com> wrote:

Hey everyone,

I just loaded two different log files into lnav hoping it would sort by timestamp so I could see events in parallel between the two, but I discovered two separate instances of a given timestamp, about 1000 lines apart, with lots of other time in between.

Yes, ​lnav does ​sort by timestamp as it loads logs.  However, lnav expects the log messages in

the original files to be in time-sorted order.  If a message in a log was not in order, lnav will

ignore the timestamp in the message and use the timestamp from the previous message.

So, it might be the case that the original log files you're examining have out-of-order messages.

​You can check this by moving to the line in question and pressing 'p' to open the parsing view,

which will show the UTC times that lnav is using for that log message.

Example: http://static1.squarespace.com/static/51bd4e13e4b0052d7873ad34/t/51fe68cae4b0c31f219051af/1375627468272/lnav-parser.png?format=1000w

This is a bit of a no-win situation.  Earlier versions of lnav would do sorting and not maintain

the original ordering in the file, but that was confusing in some situations.  So, I ended up changing

to this behavior.  I suppose I could highlight the line if timestamp in the message was different

than the timestamp used for sorting...

It's worth noting that one file records timestamps as Apr 3 01:25:48 whereas the other is Apr 03 01:25:48.  While lnav can handle the timestamp formats on each of these files individually, they don't sort together when opened with lnav.

​Can you send some sections of the log so I can try it out and see what the problem might be?​

thanks,

tim

I tried catting both files this way:

cat /var/log/messages /var/log/cluster/corosync.log | sort -k1 > combined.log

and then opened combined.log... after ~ 3 minutes of watching it load, I gave up at 17%.  The machine this is running on Centos 7.0.1406, 8GB RAM, 8-2Ghz cores on an 8Gb/s FC VMware RDM backed by a Compellent SAN.  The hardware isn't likely a limitation in this case; the combined line count of combined.log was 211k lines... so that just may be a lot of stuff to deal with.

Is there a way to make this sort of thing work that's more elegant than catting the files together?  Correlating events across multiple logs is a marvelous thing.

Thanks!

--
You received this message because you are subscribed to the Google Groups "lnav" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lnav+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.