Kibana showing curly bracket

[rwagner_inf]

Has that ever happened to anyone?

Kibana displaying the character { in the logs view.

The logs are in json format, I am using filebeat to send the logs to the logstash.

Anyone knows how to fix this?

[Polkan Garcia]

Hi,

The {} in json format denotes objects or membership inside objects.

So, in the raw json file, using brackets is a normal view. [1]

The fields inside json are showing in the table view that you selected.

[1] http://www.json.org/

—PG

IT Security Engineer

Wazuh Inc.

As a computer, I find your faith in technology amusing.

[Santiago Bassett]

It looks like you are indexing archive events instead of only the alerts. What is the name of the index? 

Here is a brief tutorial on how to integrate archives.json (thank you Pedro, who authored it):

1.- Filebeat

On the manager, configure /etc/filebeat/filebeat.yml to read alerts.json and archives.json data. Adding as well a new field "event_type" for "event" or "alert" value:

filebeat:
  prospectors:
   - input_type: log
     paths:
      - "/var/ossec/logs/alerts/alerts.json"
     fields:
      event_type: alerts
     fields_under_root: true
     document_type: json
     json.message_key: log
     json.keys_under_root: true
     json.overwrite_keys: true

   - input_type: log
     paths:
      - "/var/ossec/logs/archives/archives.json"
     fields:
      event_type: archives
     fields_under_root: true
     document_type: json
     json.message_key: log
     json.keys_under_root: true
     json.overwrite_keys: true

2.- Logstash

Set Logstash output configuration to use "event_type" value:

 output {
     elasticsearch {
         hosts => ["localhost:9200"]
         index => "wazuh-%{event_type}-%{+YYYY.MM.dd}"
         document_type => "wazuh"
         template => "/etc/logstash/wazuh-elastic5-template.json"
         template_name => "wazuh"
         template_overwrite => true
     }
 }

3.- Elasticsearch

Check indices status to be sure there are already inserted data.

curl -XGET localhost:9200/_cat/indices/wazuh-archives*

green open wazuh-archives-2017.05.31 ZD3z28srRSara_FISH-sQg 5 0 9 0 34kb 34kb

4.- Kibana

Add a new index pattern Open Kibana WUI, go to "Management" (Gear icon, left bar).

"Index patterns", create a new one with pattern wazuh-archives-*.

Now you have three index patterns, you could pick one:

wazuh-alerts-*
wazuh-archives-*
wazuh-monitoring-*

If for some reason, you want to display all the data on same visualizations or discover tab, you could create a new index pattern with pattern "wazuh-*" which will group everything.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/4C8E2291-6C04-4357-880B-55FF3068BDCE%40wazuh.com.

For more options, visit https://groups.google.com/d/optout.

[Pedro Sanchez]

Hi,

I can understand what are your concerns with the "brackets", Kibana by default is not selecting any field as a "column" to display data, that is why "_source" field is being displayed there.

Try to use left bar to add some columns and get a nicer view.

Example:

​

Best regards,

Pedro.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CAEb-Ba873BUNTb6RUepM%2BXqoZM2%2BfDF4pqdCmG4Ko39%3D_vLeCA%40mail.gmail.com.

[rwagner_inf]

Hi Santiago! Thank's for the reply!

I did it but it did not work.

The wazuh template file is in the correct folder (/etc/logstash/wazuh-elastic5-template.json) and with correct permissions (the owner is the logstash user).

I also did this configuration that you described, but I had the same result. And before I reconfigured, I erased all the indexes that existed.

But I liked the solution to send the two files through the filebeat creating a type. Before I was created a tag and chacando if the tag existed with the logstash.

wazuh-alerts indexes:

wazuh-archives indexes:

It may be something related to the "_source" field. But I do not know what it is.

[rwagner_inf]

Hi Pedro, thank's for the reply!

Please, read the answer for Santiago.

[Pedro Sanchez]

Hi Rwagner,

Your configuration looks right, still we can notice the difference between how Kibana shows _source fields for archives* and alerts*.

I can see how "_source" field is present in "Selected fields" left column for wazuh-alerts-* pattern, but it is not for wazuh-archives-*.

Did you try to refresh your index pattern ? for loading new fields ? Management -> Index patterns -> Choose wazuh-archives-* and click on "Refresh" icon.

Best,

Pedro.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/2b24575a-e62c-4086-b2e5-b1de68e453f5%40googlegroups.com.

[rwagner_inf]

Pedro, I tried but did not show up. :(

Em quinta-feira, 1 de junho de 2017 14:56:33 UTC-3, Pedro Sanchez escreveu:

Hi Rwagner,

Your configuration looks right, still we can notice the difference between how Kibana shows _source fields for archives* and alerts*.

I can see how "_source" field is present in "Selected fields" left column for wazuh-alerts-* pattern, but it is not for wazuh-archives-*.

Did you try to refresh your index pattern ? for loading new fields ? Management -> Index patterns -> Choose wazuh-archives-* and click on "Refresh" icon.

Best,

Pedro.

[rwagner_inf]

I tried updating the wazuh-alerts there he got the same problem. The _source field is gone. Lol

Is it not because of the template file "wazuh-elastic5-template.json"?

I'm using the wazuhapp plugin, it creates the wazuh-alerts index. It may be that when I refreshed, the indices have updated to the default of the file "wazuh-elastic5-template.json".

[Pedro Sanchez]

Hi Rwagner, 

Did you get it done? I can't think about this as an "issue", I believe this is standard Kibana behaviour and it is not preventing anything to work right.

Still it looks ugly and I prefer the see _source field with some "labels" for fields, but I am not quite sure what is causing the "switch" from cool to ugly.

Could be related to template, we could try to set up the environment without template (temporally) and pushing the events manually (using cURL to Elastic), create index-pattern, not using Wazuh App, and detect if we are causing the issue or it is just Kibana tricking us.

Let us know if you figure out what is Kibana doing with the default _source field visualization on Discover.

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/9b3c7db9-c697-49c4-a2ae-46f571708c1b%40googlegroups.com.

[Leandro Maciel]

Hello,

Anyone has a solution for this? I'm having the same problem, double checked the configuration, but Kibana shows the raw json in the _source field, and I'm only indexing the alerts and monitoring, on the alert index I have the raw json in the field _source, but in the monitoring index I have the _source field with labels, both index uses the same mapping.

On Tuesday, June 6, 2017 at 8:38:46 AM UTC-3, Pedro Sanchez wrote:

Hi Rwagner, 

Did you get it done? I can't think about this as an "issue", I believe this is standard Kibana behaviour and it is not preventing anything to work right.

Still it looks ugly and I prefer the see _source field with some "labels" for fields, but I am not quite sure what is causing the "switch" from cool to ugly.

Could be related to template, we could try to set up the environment without template (temporally) and pushing the events manually (using cURL to Elastic), create index-pattern, not using Wazuh App, and detect if we are causing the issue or it is just Kibana tricking us.

Let us know if you figure out what is Kibana doing with the default _source field visualization on Discover.

Thanks!

[Pedro Sanchez]

Hi Leandro, rwagner_inf, 

We have found what was the issue with this "curly brackets" in Kibana, not showing properly the fields list without any specific field selected.

We will solve it in future versions, in case you want to make it fix before new releases, and view the _source field formatted, follow these steps:

• Go to Management > Index Patterns > Advanced Settings: metaFields
• Click on "Clear"
• Go to Management > Index Patterns
• Click on "Refresh field list"

Now, in the Discover tab you should see something like:

Best regards,

Pedro.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/bc0aea34-6dd5-4725-b702-fb6009f80b1f%40googlegroups.com.

[Leandro Maciel]

Hello,

the metaFields variable on Advanced Settings was empty, but I followed your tip and when I put _source as the default value and refreshed the field list, it worked for the wazuh index and for other indices that had the same problem.

It seems that for some reason the default values for the metaFields where not working, but if you force them, it solves the problem.

I'm using the following values in metaFields (all of them are default values, but right now they only work if I explicit declare them on the advanced settings)

I don't know if this is the normal behavior, if I missed something in the documentation or if this is a bug, but at least it is working.

Thanks!

[Pedro Sanchez]

Hi Leandro,

I would say it is on us, internally on Wazuh App we are forcing the meta fields in order to have a clean/readable document with unnecessary fields, removing all the default fields.

In our effort of making everything clear and easy, we detected that Kibana does not like not having any metaFields set.

We would take another approach and solve this for further versions.

Thanks again for the feedback, don't doubt to come at us if you detect something is again off.

Best,

Pedro.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/27aeb75a-20f8-4641-9075-32ec2818d275%40googlegroups.com.