Help on multi-line tail issue

[Simon Geard]

Hi all...

I'm currently trying to wrangle fluent-bit into parsing the log format produced by a framework we're using (4Js Genero, if anyone is familiar with it). The format looks something like this:

Date: 02/09/2021    Time: 17:09:57
Hello world!
Date: 02/09/2021    Time: 17:09:58
Program error at 'main.4gl', line number 8.
SQL statement error number -1803 (-1).
Connection does not exist.

i.e. a standard line for the date/time, followed by one or more lines of freeform text.

I'm using the following definition for Parser_FirstLine:

[PARSER]
    Name        log-first-line
    Format      regex
    Regex       ^(?<datetime>Date: \d{2}\/\d{2}\/\d{4}\s+Time: \d{2}\:\d{2}:\d{2})(?<content>.*)
    Time_Key    datetime
    Time_Format Date: %d/%m/%Y    Time: %H:%M:%S
    Time_Keep   on

...and this is producing the following:

{"date":1630602597.0,"path":"test.err","datetime":"Date: 02/09/2021    Time: 17:09:57\nHello world!"}
{"date":1630602598.0,"path":"test.err","datetime":"Date: 02/09/2021    Time: 17:09:58\nProgram error at 'main.4gl', line number 8.\nSQL statement error number -1803 (-1).\nConnection does not exist."}

[Simon Geard]

Apologies, I posted that too soon, not used to Google Groups.  But the problem is that it's not splitting the "datetime" and "content" correctly, and I'm struggling to see what I'm missing.

What I did notice is that unlike most of the configuration examples (which are typically Java logs), there's no message after the time key - just the end of the line. And with some experimenting, I found that if I add some trailing whitespace to the end of the date lines, it correctly splits the rows:

{"date":1630602597.0,"path":"test.err","datetime":"Date: 02/09/2021    Time: 17:09:57","content":" \nHello world!"}
{"date":1630602598.0,"path":"test.err","datetime":"Date: 02/09/2021    Time: 17:09:58","content":" \nProgram error at 'main.4gl', line number 8.\nSQL statement error number -1803 (-1).\nConnection does not exist."}

So it feels like I'm very close, but I've been unable to figure out what I'm missing. Can anyone help on this?

Simon Geard.