Hi,
I wanted to create a dashboard to visualize postfix logs.
Currently, I am decoding them and storing them as alerts using the following:
Sample logs: (Postfix generates 5 non-sequential logs for each event. They share the same ID.)
Aug 23 07:23:02 mail postfix/pickup[30829]: D002A464A90: uid=1000 from=<ad...@example.com>
Aug 23 07:23:02 mail postfix/cleanup[31758]: D002A464A90: message-id=<20240823072302.2jDN_%ad...@example.com>
Aug 23 07:23:02 mail postfix/qmgr[20645]: D002A464A90: from=<ad...@example.com>, size=375, nrcpt=1 (queue active)
Aug 23 07:23:02 mail postfix/local[31766]: D002A464A90: to=<us...@example.com>, orig_to=<con...@example.com>, relay=local, delay=0.01, delays=0/0/0/0, dsn=2.0.0, status=sent (delivered to maildir)
Aug 23 07:23:02 mail postfix/qmgr[20645]: D002A464A90: removed
Decoder:
<decoder name="postfix">
<program_name>^postfix</program_name>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>(\w+):</regex>
<order>id</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>uid=(\d+)</regex>
<order>uid</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>from=\<(\.+)></regex>
<order>from</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>message-id=\<(\.+)></regex>
<order>msgid</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>size=(\d+)</regex>
<order>size</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>nrcpt=(\d+)</regex>
<order>nrcpt</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>to=\<(\.+)></regex>
<order>to</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>orig_to=\<(\.+)></regex>
<order>orig_to</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>relay=(\w+),</regex>
<order>relay</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>delay=(\.+),</regex>
<order>delay</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>delays=(\.+),</regex>
<order>delays</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>dsn=(\.+),</regex>
<order>dsn</order>
</decoder>
<decoder name="postfix">
<parent>postfix</parent>
<regex>status=(\w+)</regex>
<order>status</order>
</decoder>
Rules:
<group name="postfix,">
<!-- Rule to capture the first log entry with a specific queue ID -->
<rule id="100001" level="12">
<decoded_as>postfix</decoded_as>
<description>Initial Postfix log entry with q ID</description>
<group>postfix,</group>
</rule>
<!-- Rule to capture subsequent log entries with the same queue ID -->
<rule id="100002" level="12">
<decoded_as>postfix</decoded_as>
<if_matched_sid>100001</if_matched_sid>
<description>Subsequent Postfix log entries for the same queue ID</description>
<group>postfix,</group>
<same_id/>
</rule>
</group>
I want all the fields separately extracted from these logs to be combined under the index, based on the common ID.
My question is: How should I feed these decoded postfix logs to the index and if there are any guides on creating this index?
Thank you!