event_id in json

[st...@sroskam.nl]

I just tried the latest version on github to create rules purely on event_id's from json, but I think I've spotted an issue to trigger an alert.

- In line 679 - 692 in engine.c the code is looking if the logmessage contains the event_id, but it isn't found, therefore it isn't reaching line 779 to look at the json event_id key mapping.

The command I'm testing with:

echo "192.0.2.1|local0|info|info|sshd|2001-01-01|00:00:00|sshd| {\"EventID\":1234}" | sudo tee /var/sagan/fifo/sagan.fifo

Maybe I'm using this branch a bit to early, but I thought it was better to mention it already. Also I haven't tested the complete flow yet, so maybe I will report more about this later today.

Best regards,

Stef

[Da Beave]

I'll take a look at this ASAP.

--
You received this message because you are subscribed to the Google Groups "sagan-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sagan-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sagan-users/f351a5bc-d8b0-41bd-8722-877fb3fedc1dn%40googlegroups.com.

[st...@sroskam.nl]

I just took some time to analyze it a bit more and it also depends on the type of input I think:

With Pipe Input: JSON_LOCAL->json_count=0 is present when calling the engine, no JSON parsing is performed before the engine

With JSON Input: JSON_LOCAL-> Optional event_id and other JSON fields are present, because JSON is parsed, so in that case it depends if you map the message.EventID to event_id in the json-input.map or that you only map the complete message to the message field and use the rule mapping keyword to parse EventID to event_id later in the engine.

In the engine the order is:

1. pre-match checks

 - syslog_program

 - syslog_facility

 - syslog_level

 - syslog_tag

 - syslog_priority

2. matching checks

- content

- pcre

- meta_content

- json_pcre

- json_content

- json_meta_content

- event_id

3. field manipulations: 

a. json_map ->

- syslog_message -> I think this should be above "2. matching checks"

- syslog_program -> I think this should be above "1. pre-match checks"

- event_id -> I think this should be above "2. matching checks - event_id"

b. normalize

- event_id -> I think this should be above "2. matching checks - event_id"

I also found a minor issue when using json in the program field, I will create a pull request for that one. I can also create a pull request with a changed order.

Best regards,

Stef

Op donderdag 1 april 2021 om 22:35:27 UTC+2 schreef Da Beave:

[Da Beave]

We found a big.iasue with event_id parsing.last night.  I think we fixed the issue but if you could verify that would be great.

--

[Stef Roskam]

What was the issue you encountered? And was it related to my previous order change? 

I will try to reproduce it first on my development branch and after that I will retest it on the main sagan branch.

Best regards,

Stef

To view this discussion on the web visit https://groups.google.com/d/msgid/sagan-users/CAEAd5jkGPPPSAoh9B_XPaA-VERiewUeascCvCN%3DpNp4jyOo5xA%40mail.gmail.com.

[Da Beave]

Hello Stef, 

It does not appear like it was related to any of your changes. 

To view this discussion on the web visit https://groups.google.com/d/msgid/sagan-users/CABimJt3d%2BvKUx4eCUuWACCm_vAHamgpP0iR%2BYacTcpae9eRW%2BA%40mail.gmail.com.

[Stef Roskam]

I've tested both my development branch and the changed code.

Both are working with my testrule with the event_id in it, so I think I don't hit the bug with my code path.

Best regards,

Stef

To view this discussion on the web visit https://groups.google.com/d/msgid/sagan-users/CAEAd5jnTzyzVEJED6W%2Bpwdxns2wzattmFNMB4RaB1%2BUzh26cWw%40mail.gmail.com.