Elastic Stack 5

[Kevin Branch]

With the current stable version of all Elastic products now hitting version 5, I wanted to ask if there are any known issues related to integrating the current Wazuh HIDS server with this this newer stack.  Has anyone else tried it out yet?  For example, is the the OSSEC alerts template and the set of Wazuh Kibana dashboards compatible with version 5?

Thanks,

Kevin

[Santiago Bassett]

Hi Kevin,

we are putting together a new guide for integration with Elastic 5. The very good news is that we are about to publish a new release that will also incorporate a Kiabana app to monitor configuration and agents status. 

New version will have full integration with OpenSCAP and enhancements for log analysis and file integrity monitoring capabilities (among other things).

I would advice to wait for the new documentation (coming out in a few weeks). In the meanwhile, we do have a beta version available. Please email me if you want to test it and I'll share with you the installation instructions.

Best regards,

Santiago.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CA%2BdGL9Ft9jgCcohxy-HnS468K-5T20VDArAhArxAqmx6jAN6cQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew So]

Hi Santiago,

I would also like to start using Elastic 5 with Wazuh, Would I be able to get a copy of the install instructions

Thanks
Andrew

[Santiago Bassett]

Hi Andrew,

I'll prepare an email with the instructions and publish it in this mailing list tomorrow the latest.

Thanks so much for the interest.

Santiago.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/ee267788-3bcd-4ea4-a8d6-f7ca3233af30%40googlegroups.com.

[Josh]

Hi Santiago,

Any idea when the new documentation will be released for Elastic 5 and the new Kibana app? I am very eager to update my installation and try out the new features. Thanks!

[Marcio Costa]

Hello guys.

fyi: I do a quick test, only switching the kibana version from 4.5 to 5, unpacking the tar.gz and replacing the folder, while we are waiting for the new steps for elk-5; sure.., it not works and I get this messages:

ui settings	Elasticsearch plugin is red
plugin:kib...@5.2.0	Ready
plugin:elasti...@5.2.0	This version of Kibana requires Elasticsearch v5.2.0 on all nodes. I found the following incompatible nodes in your cluster: v2.4.4 @ 127.0.0.1:9200 (127.0.0.1)
plugin:con...@5.2.0	Ready
plugin:time...@5.2.0	Ready

[Pedro Sanchez]

Hi Marcio,

Thanks for the feedback. You are right, in order to run Wazuh App you will need to install/upgrade all the Elastic Stack components to latest 5.x version (5.2.0 currently), otherwise Kibana won't start.

Elasticsearch itself is not compatible between 2.x and 5.x versions, please upgrade all the components and you will be able to install Wazuh app.

We are still preparing the new documentation, you can take a look into "new_template" branch on our wazuh-documentation repository: https://github.com/wazuh/wazuh-documentation/blob/new_template/source/installation_guide/installing-manager/packages-installation/elastic_server_deb.rst

Thanks again for the feedback, we keep working hard to have everything prepared as soon as posible.

Best regards,

Pedro Sanchez.

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/6a6218e5-584c-4f0e-b679-d11cc674d196%40googlegroups.com.

[Fabio Sbano]

Eu instalei o wazuh com elastic stack 5.0 :-)

[Ricardo Galossi]

Hi guys,

I'm doing some tests with Wazuh + ELK5 and I have some problems when I try to import dashboards template (wazuh-kibana5-dashboards.json) in kibana, the errors prints are attachment.

I'm using logstash, elasticsearch and kibana 5.2.0. My logstash conf file /etc/logstash/config.d/01-ossec-singlehost.conf is:

input {

        file {

                type => "ossec-alerts"

                path => "/var/ossec/logs/alerts/alerts.json"

                codec => "json"

        }

}

filter {

        geoip {

                source => "srcip"

                target => "geoip"

                database => "/etc/logstash/GeoLite2-City.mmdb"

                add_field => [ "[geoip][location]", "%{[geoip][longitude]}" ]

                add_field => [ "[geoip][location]", "%{[geoip][latitude]}"  ]

        }

        mutate {

                convert => [ "[geoip][location]", "float"]

                rename => [ "geoip", "GeoLocation" ]

                remove_field => [ "timestamp" ]

        }

}

output {

        elasticsearch {

                hosts => ["localhost:9200"]

                index => "wazuh-alerts-%{+YYYY.MM.dd}"

                document_type => "wazuh"

                template => "/etc/logstash/wazuh-elastic5-template.json"

                template_name => "wazuh"

                template_overwrite => true

        }

}

Could someone help me?

[Pedro Sanchez]

Hi everyone,

Please let me remind you we are still in a beta phase, it's posible you will find some errors, said so, we REALLY appreciate your feedback, I love the community being supportive and contributing to our project, we study and analyze in detail each emai you sent, it is really important for us to know your impressions.

Fabio, I am happy you have Wazuh+Elastic+App installed successfully, I really love to see the App working! Send more screenshots! haha.

Ricardo, I would recommend you to use new Logstash configuration file, it's similar to the oldest one but including GeoIP database is not longer needed, it is simpler and effective, take a look at (https://github.com/wazuh/wazuh/blob/master/extensions/logstash/01-wazuh.conf)

Anyway, your error is not related to Logstash configuration, it is related to internal Kibana index patterns, last Kibana version was released 6 days ago and we are still in process of fixing latest issues that new version brought up.

Being more specific, your error prompt when Kibana detects you are adding new Dashboards/Visualizations which contains fields you don't have indexed. For example, you be maybe adding a Visualization for "OpenSCAP" but you don't have alerts (events) indexed for OpenSCAP.

This problematic have being a pain in the *** for the last weeks (Since Kibana 5.0.2) and we, and Kibana community are struggling with it.

We have a workaround to this current issue (Fabio upside got everything working), Ricardo try to install Wazuh Kibana App and restart Kibana server, that will automatically: Install index patterns you need, install Elastic templates/mappings, install dashboards and visualizations.

We will keep working hard! Give us a few days to get Wazuh App fully working, at the end the installation process must be smooooth and clean :D

Best regards,

Pedro Sanchez.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/5f0010a0-c465-4b37-b32d-ea704b6a5935%40googlegroups.com.

[esqu...@gmail.com]

Hi Santiago,

I can test the elastic 5 option? 

thanks

[Fabio Sbano]

How to clean install with RHEL/CentOS 7 and Wash-2.0-ELK-5.x

step by step

Install jdk-8u121-linux-x64.rpm

cat  > /etc/yum.repos.d/epel.repo  <<EOF

[epel]

name=Extra Packages for Enterprise Linux 7 - $basearch

baseurl=https://dl.fedoraproject.org/pub/epel/7/x86_64/

enabled=1

gpgcheck=0

EOF

cat > /etc/yum.repos.d/elastic.repo << EOF

[elastic-5.x]

name=Elastic repository for 5.x packages

baseurl=https://artifacts.elastic.co/packages/5.x/yum

gpgcheck=0

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

EOF

cat > /etc/yum.repos.d/nodesource-el.repo  <<EOF

[nodesource]

name=Node.js Packages for Enterprise Linux 7 - $basearch

baseurl=https://rpm.nodesource.com/pub_6.x/el/7/x86_64/

failovermethod=priority

enabled=1

gpgcheck=0

gpgkey=file:///etc/pki/rpm-gpg/NODESOURCE-GPG-SIGNING-KEY-EL

EOF

yum install -y make gcc gcc-c++ git python-pip logstash elasticsearch kibana filebeat zip nodejs

mkdir -p /root/ossec-wazuh && cd /root/ossec-wazuh

git clone https://github.com/wazuh/wazuh-api

git clone https://github.com/wazuh/wazuh

git clone https://github.com/wazuh/wazuh-kibana-app kibana/wazuh

zip -r kibana.zip kibana

cp -av /root/ossec-wazuh/wazuh/extensions/logstash/01-wazuh.conf /etc/logstash/conf.d/

cp -av /root/ossec-wazuh/wazuh/extensions/elasticsearch/wazuh-elastic5-template.json /etc/logstash/

cat /root/ossec-wazuh/wazuh/extensions/filebeat/filebeat.yml > /etc/filebeat/filebeat.yml 

sed -ie 's/YOUR_ELASTIC_SERVER_IP/127.0.0.1/g' /etc/filebeat/filebeat.yml

/usr/share/kibana/bin/kibana-plugin install file:///root/ossec-wazuh/kibana.zip

sed -ie 's/#server.host: "localhost"/server.host: "0.0.0.0"/g' /etc/kibana/kibana.yml

systemctl enable elasticsearch filebeat logstash kibana

systemctl start elasticsearch filebeat logstash kibana

cd /root/ossec-wazuh/wazuh && ./install.sh

>> select server installation

/var/ossec/bin/ossec-control start

cd /root/ossec-wazuh/wazuh-api/ && ./install_api.sh

[esqu...@gmail.com]

Works!

Thanks a lot

[esqu...@gmail.com]

Hi,

Everything works great but i got and error in some pages like "scap" tab

• Saved "field" parameter is now invalid. Please select a new field.

  OK

  289s
• 5

  Visualize: "field" is a required parameter

The rest work fine.

Any solution?

thanks,

Iván

[Fabio Sbano]

I will go see tonight 

You received this message because you are subscribed to a topic in the Google Groups "Wazuh mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/BC2VzbUc6MI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+unsubscribe@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/43b18eec-6d4e-44c1-bd46-d5e5ad46eef6%40googlegroups.com.

[Pedro Sanchez]

Hi everyone!

We are still working on that issue, it is a well-known error, give us a couple of days and we will solve it.

FYI, in the past, Kibana allowed us to create Visualizations chosing any field we like, no matter if the field was present on some alerts (documents) or it wasn't.

Since version 5.0.2, Kibana has new types for each field on the index pattern, "searchable" and "aggregatable". Visualizations need "agregatable" fields in order to render, a field become "agregatable" when there are some alerts containing it and we refresh manually the index pattern on Kibana "Management" tab.

At Wazuh App, we already prepared and created some visualizations, but some of them won't render cause you don't have those fields, the error on your screenshots will prompt.

We are working on fixing the issue, the error should not appear and a "No results" message will appear instead (like it was on earlier Kibana versions). We have several solutions and we are studying how to apply them.

In the meantime, if you want to fix the error, wait until some OpenSCAP alerts be generated (you can look for them on Discover tab), once you have alerts, refresh the index pattern and that will solve the error. 

Best regards,

Pedro Sanchez.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CANfOmJFAn7o7UPcRVjHeqLekZ5AyFh3zsJZ6DVCz%3D47A9DuVXQ%40mail.gmail.com.

[0x2a]

Hi,

I just encountered the same issue with the "OSSEC Alerts" dashboard (installed by the wazuh app), it might be this bug: https://github.com/elastic/kibana/issues/9571 https://github.com/elastic/elasticsearch/issues/22438

It seems that if there is data for a search missing, all the other visualizations will also not load.


This seems rather important, as it breaks the dashboard if one of the searches/visualizations returns no data.

[Pedro Sanchez]

Hi 0x2a,

Thanks for your feedback, like you said, it is kind of a bug from Elasticsearch guys, I explained it on my previous post.

They are working on something called "FieldCapabilities API" which will allow us, and them, to figure out what a visualization could be rendered.

We created a workaround on Wazuh app panels, the workaround will force Wazuh index-pattern fields to be aggregatable and searchable at any cost, that way we prevent the panels failure, no matter if there is data or is not.

Could you tell us if you got that error on standard Kibana dashboards? I mean, if you go out Wazuh App and open OSSEC Alerts dashboards on Kibana>Dashboards, is the error still there? or the error appears on Wazuh app panels General/Agents tabs?

We will keep waiting until Kibana guys give us a solution, until that point, we can not prevent the error since it is on Kibana internal code.

Thanks again for your feedback, it is so much appreciate.

Best regards,

Pedro Sanchez.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e6cc0b92-4890-4847-9462-5b20c9b385fc%40googlegroups.com.

[Luke Salsich]

Is this still an issuewith the recent ES and Kibana updates? I'm still getting the ""field" is a required parameter" on Kibana 5.3.1 :(

[Pedro Sanchez]

Hi Luke,

Yes, it is still an issue on Kibana, this is not on us but still we are working hard to find a definitive solution, in the meantime, please take a look into my comment in this issue:

https://github.com/wazuh/wazuh/issues/111#issuecomment-297758448

The error appears but is something that we can fix, there are a lot of current installation with Wazuh App which don't have the error.

I will update you with any progress in a easy solution for this issue.

Best,

Pedro.

Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/fa4fc6d7-fc1d-414c-83d6-62273c577035%40googlegroups.com.

[Leandro Maciel]

Hello,

Is there any update on this issue?

[Leandro Maciel]

Hello again,

 

I've found another workaround. 

Since the problem is that Kibana visualizations won't work if a field in the visualization has no data and the dashboards won't load if some of the visualizations have errors, I manually created a dummy document with the fields that had no data and now I can see the dashboard populated.

I will now try to generate a dummy document with all the mapping fields so I can see the other dashboards that are showing errors, It's not the best solution, but it solved the problem until the fields got some real data. 

[Pedro Sanchez]

Hi Leandro,

It makes total sense in fact, we did the same thing some weeks ago, sorry to not update this email thread with the solution we took.

Please take a look into our documentation, we have a "sample document" covering all the fields that dashboards are going to use later.

• https://documentation.wazuh.com/current/installation-guide/installing-elastic-stack/elastic_server_rpm.html#elasticsearch
  

Command to insert the sample alert:

curl https://raw.githubusercontent.com/wazuh/wazuh-kibana-app/master/server/startup/integration_files/alert_sample.json | curl -XPUT "http://localhost:9200/wazuh-alerts-"`date +%Y.%m.%d`"/wazuh/sample" -H 'Content-Type: application/json' -d @- 

URL to sample alert:

https://github.com/wazuh/wazuh-kibana-app/blob/master/server/startup/integration_files/alert_sample.json

Please feel free to review it and compare it with yours, we keep having some issues in certain installations I am sure you can help us improving it.

Thanks for the feedback, interest and contributions, they are too much appreciated.

Best,

Pedro. 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/34f0add1-61de-4c99-ab4d-3be59b0b1197%40googlegroups.com.