Grafana Logs - Local File

[OrenYa]

Hi Guys,

I've added monitoring to get all Grafana logs in Kibana using the ossec agent conf file with the following code:

  <localfile>

    <location>C:\\Program Files\\grafana-5.4.0\\data\\log\\grafana.log</location>

    <log_format>syslog</log_format>

    <frequency>30</frequency>

  </localfile>

Now, I can see new lines added to the file using the archive.log but I can't see them on Kibana.

I don't want to create specific rules to catch certain type of events but to see all file logs on Kibana.

Can you update how this can be achieved?

Log example [JSON]:

{"logger":"alerting.scheduler","lvl":"dbug","msg":"Oren update","ruleCount":0,"t":"2018-12-16T11:43:27.2108526Z"}

{"lvl":"dbug","msg":"Plugins: Adding proxy route /api/plugin-proxy/raintank-worldping-app/api/*","t":"2018-12-16T11:43:26.2754617Z"}

{"id":"add unique index dashboard_acl_dashboard_id_team_id","logger":"migrator","lvl":"dbug","msg":"Skipping migration: Already executed","t":"2018-12-16T11:43:26.2089268Z"}

Thanks,
Oren 

[Victor Fernandez]

Hi Oren,

We don't have to make specific rules for each event types, but we need at less one rule that matches all Grafana logs.

Since you have configured Grafana to print logs in JSON format, we don't need to define custom decoders —JSON objects are decoded out-of-the-box—.

Your localfile configuration works, however, let me suggest you this little change:

<localfile>
  <log_format>json</log_format>
  <location>C:\grafana-5.4.2\data\log\grafana.log</location>
</localfile>

The option log_format=json will ensure that all input events are in JSON format. On the other hand, we don't need frequency=30 because that setting applies to command only.

Now, let's add our rule to the ruleset. I recommend you add this content to /var/ossec/etc/decoders/local_decoders.xml:

<group name="local,">

  <!-- JSON logs from grafana.log -->
  <rule id="100001" level="5">
    <decoded_as>json</decoded_as>
    <location>grafana.log$</location>
    <description>Log from Grafana.</description>
    <options>no_full_log</options>
    <group>grafana,</group>
  </rule>

</group>

Let me suggest you add this complementary rule, to build the rule description message from the msg string of the Grafana log:

  <!-- JSON logs from grafana.log containing message -->
  <rule id="100002" level="5">
    <if_sid>100001</if_sid>
    <field name="msg">\.+</field>
    <description>Grafana: $(msg)</description>
    <options>no_full_log</options>
    <group>grafana,</group>
  </rule>

Feel free to change the rule ID and the level according to your needs.

Now let's see the result. For this log from Grafana:

{"address":"0.0.0.0:3000","logger":"http.server","lvl":"info","msg":"HTTP Server Listen","protocol":"http","socket":"","subUrl":"","t":"2018-12-16T06:01:15.8358958-08:00"}

We will have this plain-text alert (file alerts.log):

** Alert 1544967922.270870: - local,grafana,
2018 Dec 16 14:45:22 (WIN-DC13ROCR412) 192.168.33.1->\Program Files\grafana-5.4.2\data\log\grafana.log
Rule: 100002 (level 5) -> 'Grafana: HTTP Server Listen'
{"address":"0.0.0.0:3000","logger":"http.server","lvl":"info","msg":"HTTP Server Listen","protocol":"http","socket":"","subUrl":"","t":"2018-12-16T06:01:15.8358958-08:00"}
address: 0.0.0.0:3000
logger: http.server
lvl: info
msg: HTTP Server Listen
t: 2018-12-16T06:01:15.8358958-08:00

This is the same alert in JSON format (file alerts.json) as we would see in Kibana:

{
  "timestamp": "2018-12-16T14:45:22.525+0100",
  "rule": {
    "level": 5,
    "description": "Grafana: HTTP Server Listen",
    "id": "100002",
    "firedtimes": 26,
    "mail": false,
    "groups": [
      "local",
      "grafana"
    ]
  },
  "agent": {
    "id": "004",
    "name": "WIN-DC13ROCR412",
    "ip": "192.168.33.1"
  },
  "manager": {
    "name": "stretch64"
  },
  "id": "1544967922.270870",
  "decoder": {
    "name": "json"
  },
  "data": {
    "protocol": "http",
    "address": "0.0.0.0:3000",
    "logger": "http.server",
    "lvl": "info",
    "msg": "HTTP Server Listen",
    "t": "2018-12-16T06:01:15.8358958-08:00"
  },
  "location": "\\Program Files\\grafana-5.4.2\\data\\log\\grafana.log"
}

Hope it helps. Let me know if this works for you.

Best regards,

Victor Manuel Fernandez-Castro 
Core Engineering | vic...@wazuh.com

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/18e08ce5-2808-416c-a4c6-e73f0efd8b7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[OrenYa]

Thanks Victor, I've added the rules based on your input and now I receive notifications to the email address, however I still don't see the details on Kibana which is pretty strange.

Do you have a suggestion how to check this issue? 

Additionally, I think there is a mistake in the input you provided where indicated that /var/ossec/etc/decoders/local_decoders.xml should be instead local_rules.xml. Is it correct? 

Now, let's add our rule to the ruleset. I recommend you add this content to /var/ossec/etc/decoders/local_decoders.xml:

<group name="local,">

  <!-- JSON logs from grafana.log -->

  <rule id="100001" level="5">

    <decoded_as>json</decoded_as>

    <location>grafana.log$</location>

    <description>Log from Grafana.</description>

    <options>no_full_log</options>

    <group>grafana,</group>

  </rule>

</group>

Hi Guys,

I've added monitoring to get all Grafana logs in Kibana using the ossec agent conf file with the following code:

  <localfile>

    <location>C:\\Program Files\\grafana-5.4.0\\data\\log\\grafana.log</location>

    <log_format>syslog</log_format>

    <frequency>30</frequency>

  </localfile>

Now, I can see new lines added to the file using the archive.log but I can't see them on Kibana.

I don't want to create specific rules to catch certain type of events but to see all file logs on Kibana.

Can you update how this can be achieved?

Log example [JSON]:

{"logger":"alerting.scheduler","lvl":"dbug","msg":"Oren update","ruleCount":0,"t":"2018-12-16T11:43:27.2108526Z"}

{"lvl":"dbug","msg":"Plugins: Adding proxy route /api/plugin-proxy/raintank-worldping-app/api/*","t":"2018-12-16T11:43:26.2754617Z"}

{"id":"add unique index dashboard_acl_dashboard_id_team_id","logger":"migrator","lvl":"dbug","msg":"Skipping migration: Already executed","t":"2018-12-16T11:43:26.2089268Z"}

Thanks,
Oren 

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

[OrenYa]

Hi Victor,

Just wanted to update that now I can see all local file logs in Kibana. I guess it took some time to populate the logs.

Thanks for your help!

Oren 

[Victor Fernandez]

Hi Oren,

Sorry for the late reply. I'm glad you got to see the alerts in Kibana!

Regarding the actual name of the custom rule definition(local_rules.xml), my fault, I told you the wrong name file.

Best regards,

Victor Manuel Fernandez-Castro 
Core Engineering | vic...@wazuh.com

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/18e08ce5-2808-416c-a4c6-e73f0efd8b7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e0828b61-dbb5-4fb3-9933-12c4eb6df94c%40googlegroups.com.