Unix Epoch Time Stamp

[Utkarsh Bhargava]

Hi Community,

Can Wazuh read and decode Unix Epoch Timestamp.

If yes how to do that ?

I am trying to ingest some logs which are having epoch timestamp and due to that I am facing some unknown error.

here's the sample log :

1651494502.077973527 HQ2_BLR events content_filtering_block url='https://play.google.com/...' category0='Shopping' server='172.217.160.238:443' client_mac='CC:15:31:CD:EC:6D'

Bold and italic numbers are epoch timestamp.

thanks and regards

[Mariano Koremblum]

Hi!

Unfortunately, the Wazuh pre-decoder does not yet support Unix Epoch tImestamps.

Have you created your own rules and decoders to process this log? In the case you did, please, can you share it with us? So we can help you decode the timestamp.

I will be waiting for your reply,

Mariano Koremblum

[Utkarsh Bhargava]

Hi Mariano,

Thank you for your quick response.

Here's the decoder that I wrote to decode that log.

<decoder name="cisco_meraki">

    <prematch>\d \d+.\d+ \S+</prematch>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>src=(\d+.\d+.\d+.\d+):(\d+)</regex>

    <order>srcip,srcport</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>mac=(\S+)</regex>

    <order>mac</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>dst=(\S+)</regex>

    <order>dstip</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>request: (\S+ \S+)</regex>

    <order>request</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>client_mac=(\S+)</regex>

    <order>client_mac</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>server=(\S+)</regex>

    <order>server</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>server=(\S+)</regex>

    <order>server</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>content_filtering_block url(\S+)</regex>

    <order>block_url</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>type(\S+)</regex>

    <order>events_type</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>vpn_type(\S+)</regex>

    <order>vpn_type</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>peer_contact(\S+)</regex>

    <order>peer_contact</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>peer_ident(\S+)</regex>

    <order>peer_ident</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>connectivity(\S+)</regex>

    <order>connectivity</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>category(\S+)</regex>

    <order>category</order>

</decoder>

<decoder name="cisco_meraki_child">

    <parent>cisco_meraki</parent>

    <regex>(\d+.\d+)\s(\w+)</regex>

    <order>time-epoch, location</order>

</decoder>

thank you.

On May 2 2022, at 7:09 pm, Mariano Koremblum <mariano....@wazuh.com> wrote:

Hi!

Unfortunately, the Wazuh pre-decoder does not yet support Unix Epoch tImestamps.

Have you created your own rules and decoders to process this log? In the case you did, please, can you share it with us? So we can help you decode the timestamp.

I will be waiting for your reply,

Mariano Koremblum

On Monday, May 2, 2022 at 9:49:26 AM UTC-3 utk...@null.co.in wrote:

Hi Community,

Can Wazuh read and decode Unix Epoch Timestamp.

If yes how to do that ?

I am trying to ingest some logs which are having epoch timestamp and due to that I am facing some unknown error.

here's the sample log :

1651494502.077973527 HQ2_BLR events content_filtering_block url='https://play.google.com/...' category0='Shopping' server='172.217.160.238:443' client_mac='CC:15:31:CD:EC:6D'

Bold and italic numbers are epoch timestamp.

thanks and regards

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/7ac63aec-9913-4dbc-ad36-09a89400360dn%40googlegroups.com.

[Mariano Koremblum]

I think the problem is located in the first decoder.

You have this:

<decoder name="cisco_meraki">
    <prematch>\d \d+.\d+ \S+</prematch>
</decoder>

But your log does not start with a single integer followed by a space. So the correct form would be:

<decoder name="cisco_meraki">
    <prematch>^\d+.\d+ \S+</prematch>
</decoder>

I hope my answer helps you!

Best regards,

Mariano Koremblum

​

[Mariano Koremblum]

Always remember that you can test your custom rules and decoders by using the wazuh-logtest tool. Here there are some links of interest related to it:

• https://documentation.wazuh.com/current/user-manual/capabilities/wazuh-logtest/
• https://documentation.wazuh.com/current/development/wazuh-logtest.html

Best Regards!

​

[Utkarsh Bhargava]

I changed the decoder but still facing the error.

here's the error log :

May 3 06:03:14 ip-172-31-30-174 filebeat[268468]: 2022-05-03T06:03:14.393Z#011INFO#011[monitoring]#011log/log.go:145#011Non-zero metrics in the last 30s#011{"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":30856818}},"memory":{"mem":{"usage":{"bytes":253952}}}},"cpu":{"system":{"ticks":20,"time":{"ms":3}},"total":{"ticks":160,"time":{"ms":31},"value":160},"user":{"ticks":140,"time":{"ms":28}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":11},"info":{"ephemeral_id":"b280940d-f694-4654-ba18-e757af220bee","uptime":{"ms":90025}},"memstats":{"gc_next":8084992,"memory_alloc":4191352,"memory_total":24239288,"rss":389120},"runtime":{"goroutines":25}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"open_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":1,"batches":1,"total":1},"read":{"bytes":575,"errors":1},"write":{"bytes":1898}},"pipeline":{"clients":1,"events":{"active":0,"published":1,"total":1},"queue":{"acked":1}}},"registrar":{"states":{"current":1,"update":1},"writes":{"success":1,"total":1}},"system":{"load":{"1":0.04,"15":0.03,"5":0.08,"norm":{"1":0.01,"15":0.0075,"5":0.02}}}}}}

regards

On May 2 2022, at 8:59 pm, Mariano Koremblum <mariano....@wazuh.com> wrote:

I think the problem is located in the first decoder.

You have this:

<decoder name="cisco_meraki">
    <prematch>\d \d+.\d+ \S+</prematch></decoder>

But your log does not start with a single integer followed by a space. So the correct form would be:

<decoder name="cisco_meraki">
    <prematch>^\d+.\d+ \S+</prematch></decoder>

I hope my answer helps you!

Best regards,

Mariano Koremblum

​

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/bdd38896-0c89-4c2f-ad8a-3c224836b683n%40googlegroups.com.

[Utkarsh Bhargava]

Here's another error log :

May 3 06:05:00 ip-172-31-30-174 kibana[254672]: {"type":"log","@timestamp":"2022-05-03T06:05:00Z","tags":["error","elasticsearch","data"],"pid":254672,"message":"[ResponseError]: Response Error"}

Please help me how I can fix it.

On May 2 2022, at 9:04 pm, Mariano Koremblum <mariano....@wazuh.com> wrote:

Always remember that you can test your custom rules and decoders by using the wazuh-logtest tool. Here there are some links of interest related to it:

• https://documentation.wazuh.com/current/user-manual/capabilities/wazuh-logtest/
• https://documentation.wazuh.com/current/development/wazuh-logtest.html

Best Regards!

​

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/a8423913-9222-4c8d-b932-5d57ad07f1fan%40googlegroups.com.

[Mariano Koremblum]

Hi again!

Where are you receiving such error messages?

[Utkarsh Bhargava]

I am seeing all these error logs in Kibana.

On May 3 2022, at 8:45 pm, Mariano Koremblum <mariano....@wazuh.com> wrote:

Hi again!

Where are you receiving such error messages?

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/1239b30b-7c7a-40e7-8bef-573b143bdf91n%40googlegroups.com.

[Mariano Koremblum]

Hi Utkarsh,

I don't think these errors are related to the decoders at all.

When you say you are seeing these error logs in Kibana, do you mean that such errors are displayed as any other log on the website or the service does not start and throws such messages? If you can explain better the problem and give us more information we will be able to help you faster and better. If you can also provide screenshots, it might help to understand better what it is going on.

I will be waiting for your reply.

Mariano

[Utkarsh Bhargava]

I have enabled log all json option in wazuh and these error logs are these in archives.json file.

If I disable the log all json logs, I won't see this error.

On May 4 2022, at 6:27 pm, Mariano Koremblum <mariano....@wazuh.com> wrote:

Hi Utkarsh,

I don't think these errors are related to the decoders at all.

When you say you are seeing these error logs in Kibana, do you mean that such errors are displayed as any other log on the website or the service does not start and throws such messages? If you can explain better the problem and give us more information we will be able to help you faster and better. If you can also provide screenshots, it might help to understand better what it is going on.

I will be waiting for your reply.

Mariano

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/bb394123-9c83-428e-ab1c-236e62a01a36n%40googlegroups.com.

[Mariano Koremblum]

Dear Utkarsh,

The logall option logs every single log that passes through the manager on the archives files, not only the alerting-level ones (logs of relevance) which are logged to the alerts files, as it is explained here: https://documentation.wazuh.com/current/user-manual/capabilities/log-data-collection/how-it-works.html#alert.

You usually should not worry about this kind of log if the rest of the programs (Wazuh, Kibana and such) are functioning well, as they might just be some expected runtime events. You do need to take care of them if you realize that something is not working as expected.

Please, let us know if you still need help with this.

Mariano

​

[Utkarsh Bhargava]

I don't have alert rules for Cisco Meraki Firewall. I am collecting all the logs in archives and reading that file using wazuh.

Is there any way I can see Cisco Meraki Firewall logs without reading the archive or without having alert rules.

regards

On May 4 2022, at 7:26 pm, Mariano Koremblum <mariano....@wazuh.com> wrote:

Dear Utkarsh,

The logall option logs every single log that passes through the manager on the archives files, not only the alerting-level ones (logs of relevance) which are logged to the alerts files, as it is explained here: https://documentation.wazuh.com/current/user-manual/capabilities/log-data-collection/how-it-works.html#alert.

You usually should not worry about this kind of log if the rest of the programs (Wazuh, Kibana and such) are functioning well, as they might just be some expected runtime events. You do need to take care of them if you realize that something is not working as expected.

Please, let us know if you still need help with this.

Mariano

​

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/449e7e70-979a-49d3-ac4a-c95669194a82n%40googlegroups.com.

[Mariano Koremblum]

Well no, that is exactly the two possible ways of collecting logs. I would recommend you create your own rules to trigger alerts related to Cisco Meraki, as using the logall option may result in a lot of disk usage. To do so, please take a look at the following links:

• https://documentation.wazuh.com/current/user-manual/ruleset/custom.html
• https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/

If you need help building your custom rules, you can always create a new thread to receive our free support.

I hope my answer helps you! Best regards,

Mariano Koremblum

​