wazuhapp-3.10.1_7.3.1.zip + Elasticsearch - Kibana

[Rick Gutierrez]

Hi list , I hope to be in the right direction when writing on this
list, I am trying to put my own SIEM with elasticsearch + kibana +
wazuh app, I managed to install searchguard to protect the kibana
access, so far so good, after that I started installing wazuhapp,
install the plugin, after finishing the installation I try to access
kibana and it tells me: "Kibana server is not ready yet"

versions i'm using
=====================================================
CentOS 7 x64
kibana-7.3.1-1.x86_64
elasticsearch-7.3.1-1.x86_64
logstash-7.3.1-1.noarch
wazuh-manager-3.10.1-1.x86_64
wazuh-api-3.10.1-1.x86_64
Python 2.7.5


wazuhapp installation process
=====================================================

I don't know if I'm right here , but this was the procedure I did, and
the strange thing is that when installing the plugin I noticed that it
was fast and I don't know if it was done correctly.

curl https://raw.githubusercontent.com/wazuh/wazuh/18af17e5afca7c15e23dc554f4cd75f1ab37bdc1/extensions/elasticsearch/wazuh-elastic7-template-alerts.json
| curl -XPUT 'http://172.16.249.184:9200/_template/wazuh' -H
'Content-Type: application/json' -d @-

/usr/share/kibana/bin/kibana-plugin install
https://packages.wazuh.com/wazuhapp/wazuhapp-3.10.1_7.3.1.zip
--allow-root
Found previous install attempt. Deleting...
Attempting to transfer from
https://packages.wazuh.com/wazuhapp/wazuhapp-3.10.1_7.3.1.zip
Transferring 24401947 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Plugin installation complete

system log
======================================================================
tail -f /var/log/elasticsearch/elasticsearch.log
[2019-10-02T16:42:54,754][WARN ][c.f.s.a.BackendRegistry ] [srv-siem]
Authentication finally failed for admin from 172.16.249.184:33140
[2019-10-02T16:43:00,082][WARN ][c.f.s.a.BackendRegistry ] [srv-siem]
Authentication finally failed for admin from 172.16.249.184:33140
[2019-10-02T16:43:05,368][WARN ][c.f.s.a.BackendRegistry ] [srv-siem]
Authentication finally failed for admin from 172.16.249.184:33140
[2019-10-02T16:43:10,643][WARN ][c.f.s.a.BackendRegistry ] [srv-siem]
Authentication finally failed for admin from 172.16.249.184:33140

tail -f /var/log/logstash/logstash-plain.log
[2019-10-02T16:46:13,321][WARN ][logstash.outputs.elasticsearch]
Attempted to resurrect connection to dead ES instance, but got an
error. {:url=>"http://admin:xxx...@172.16.249.184:9200/",
:error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError,
:error=>"Got response code '401' contacting Elasticsearch at URL
'http://172.16.249.184:9200/'"}
[2019-10-02T16:46:18,714][WARN ][logstash.outputs.elasticsearch]
Attempted to resurrect connection to dead ES instance, but got an
error. {:url=>"http://admin:xxx...@172.16.249.184:9200/",
:error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError,
:error=>"Got response code '401' contacting Elasticsearch at URL
'http://172.16.249.184:9200/'"}
[2019-10-02T16:46:24,004][WARN ][logstash.outputs.elasticsearch]
Attempted to resurrect connection to dead ES instance, but got an
error. {:url=>"http://admin:xxx...@172.16.249.184:9200/",
:error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError,
:error=>"Got response code '401' contacting Elasticsearch at URL
'http://172.16.249.184:9200/'"}


I'm not sure here if I'm using the right json.

01-wazuh-local.conf
===================================================================

# Wazuh - Logstash configuration file
## Local Wazuh Manager - JSON file input
input {
file {
type => "wazuh-alerts"
path => "/var/ossec/logs/alerts/alerts.json"
codec => "json"
}
}

filter {
if [data][srcip] {
mutate {
add_field => [ "@src_ip", "%{[data][srcip]}" ]
}
}
if [data][aws][sourceIPAddress] {
mutate {
add_field => [ "@src_ip", "%{[data][aws][sourceIPAddress]}" ]
}
}
}

filter {
geoip {
source => "@src_ip"
target => "GeoLocation"
fields => ["city_name", "continent_code", "country_code2",
"country_name", "region_name", "location"]
}
date {
match => ["timestamp", "ISO8601"]
target => "@timestamp"
}
mutate {
remove_field => [ "timestamp", "beat", "input_type", "tags",
"count", "@version", "log", "offset", "type", "@src_ip"]
}
}

output {
elasticsearch {
hosts => ["172.16.249.184:9200"]
user => admin
password => Secret
ssl => false
ssl_certificate_verification => true
cacert => "/usr/share/elasticsearch/certs/root-ca.pem"
index => "wazuh-alerts-3.x-%{+YYYY.MM.dd}"
document_type => "wazuh"
}
}


Any ideas or help is welcome, thanks ;)


--
rickygm

http://gnuforever.homelinux.com

[Rick Gutierrez]

Hi , any help?

El mié., 2 oct. 2019 a las 16:50, Rick Gutierrez
(<xserve...@gmail.com>) escribió:

--
rickygm

http://gnuforever.homelinux.com

[José Manuel López del Río]

Hello Rick,

I see that you are trying to configure logstash. In the newest Elastic stack  versions (starting from 7.x), Filebeat can ingest directly to elasticsearch and logtash is not mandatory. If you want to use it for some reason, we can help you with the installation process if needed.

When installing Wazuh-manager+Elastic stack I recommend following our documentation. It is all well explained step by step and could help to avoid installation issues.

Manager installation: https://documentation.wazuh.com/3.10/installation-guide/installing-wazuh-manager/linux/centos/wazuh_server_packages_centos.html#centos-from-packages. By default, it is going to install the latest version of the wazuh-manager (3.10.2). To install v3.10.1 you would have to use yum install wazuh-manager-3.10.1

Elastic stack installation (including wazuh app plugin for Kibana): https://documentation.wazuh.com/3.10/installation-guide/installing-elastic-stack/elastic_server_rpm.html. 

To configure Search Guard you can follow the steps of our Search Guard documentation: https://documentation.wazuh.com/3.10/installation-guide/installing-elastic-stack/protect-installation/searchguard.html?highlight=guard#search-guard. Note that this installation guide is for version 7.1.1, you just need to change the versions for the compatible ones for your environment. This would be 7.3.1 --> 37.0.0 for Search Guard. You can have information about the compatible versions of search guard and elastic here: https://docs.search-guard.com/latest/search-guard-versions.

On another note, if the reason for using Search Guard is RBAC, that becomes part of the basic license for Xpack, so you may just need to configure it as mentioned here: https://documentation.wazuh.com/3.10/installation-guide/installing-elastic-stack/protect-installation/xpack.html#xpack-security, and no additional software is needed.

I hope it helps. Let me know if you need further help.

Regards,

Jose Manuel Lopez

> error. {:url=>"http://admin:xxxxxx@172.16.249.184:9200/",

> :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError,
> :error=>"Got response code '401' contacting Elasticsearch at URL
> 'http://172.16.249.184:9200/'"}
> [2019-10-02T16:46:18,714][WARN ][logstash.outputs.elasticsearch]
> Attempted to resurrect connection to dead ES instance, but got an

> error. {:url=>"http://admin:xxxxxx@172.16.249.184:9200/",

> :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError,
> :error=>"Got response code '401' contacting Elasticsearch at URL
> 'http://172.16.249.184:9200/'"}
> [2019-10-02T16:46:24,004][WARN ][logstash.outputs.elasticsearch]
> Attempted to resurrect connection to dead ES instance, but got an

> error. {:url=>"http://admin:xxxxxx@172.16.249.184:9200/",

[José Manuel López del Río]

Hello Rick,

The main problem of using logstash in version 3.10.1 of the wazuh-manager is that we would have to add some lines in the pipeline of filebeat to get it to work. That is the reason for my suggestions in the previous answer.

Related to the error you are facing:

[2019-10-02T16:46:13,321][WARN ][logstash.outputs.elasticsearch]
Attempted to resurrect connection to dead ES instance,

but got an
error. {:url=>"http://admin:xxx...@172.16.249.184:9200/",

:error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError,
:error=>"Got response code '401' contacting Elasticsearch at URL
'http://172.16.249.184:9200/'"}

It seems that elasticsearch is rejecting the connection with logstash, which may be related to the credentials of the user for Search Guard. In case you want to continue with your current configuration I will still investigate it and will get back to you as soon as possible.

Best Regards,

Jose Manuel Lopez

[Rick Gutierrez]

El vie., 4 oct. 2019 a las 8:01, José Manuel López del Río
(<josemanu...@wazuh.com>) escribió:
>
> Hello Rick,

Thanks Jose for answering my email, it's a light start in this darkness;)

>
> I see that you are trying to configure logstash. In the newest Elastic stack versions (starting from 7.x), Filebeat can ingest directly to elasticsearch and logtash is not mandatory. If you want to use it for some reason, we can help you with the installation process if needed.
>
> When installing Wazuh-manager+Elastic stack I recommend following our documentation. It is all well explained step by step and could help to avoid installation issues.

ok, I'm going to follow the documentation you sent me, I can't
necessarily use logstash, I'll install filebeat

>
> Manager installation: https://documentation.wazuh.com/3.10/installation-guide/installing-wazuh-manager/linux/centos/wazuh_server_packages_centos.html#centos-from-packages. By default, it is going to install the latest version of the wazuh-manager (3.10.2). To install v3.10.1 you would have to use yum install wazuh-manager-3.10.1

I am planning to use version 7.3.2 of elasticsearch, wazuh manager
3.10.2 work with this version? , the idea is to have the wazuh app
inside kibana

>
> Elastic stack installation (including wazuh app plugin for Kibana): https://documentation.wazuh.com/3.10/installation-guide/installing-elastic-stack/elastic_server_rpm.html.
>
> To configure Search Guard you can follow the steps of our Search Guard documentation: https://documentation.wazuh.com/3.10/installation-guide/installing-elastic-stack/protect-installation/searchguard.html?highlight=guard#search-guard. Note that this installation guide is for version 7.1.1, you just need to change the versions for the compatible ones for your environment. This would be 7.3.1 --> 37.0.0 for Search Guard. You can have information about the compatible versions of search guard and elastic here: https://docs.search-guard.com/latest/search-guard-versions.

I have honestly done my lab with the latest version of elastiscsearch
6 and I have no problems installing searchguard, but with version 7 I
can't install it, this is my third attempt, and the reason I want to
install it is to ensure access to kibana , but I will follow the
documentation that you attach to me in this email, sometimes I feel
that searchguard is somewhat complex to install , I'm just saying it,
I'm not an expert

>
> On another note, if the reason for using Search Guard is RBAC, that becomes part of the basic license for Xpack, so you may just need to configure it as mentioned here: https://documentation.wazuh.com/3.10/installation-guide/installing-elastic-stack/protect-installation/xpack.html#xpack-security, and no additional software is needed.

if the main reason is to have RBAC , If the main reason is to have
RBAC, I will try this.

>
>
> I hope it helps. Let me know if you need further help.
>

I would appreciate your help in case I get involved in the installation

> Regards,
> Jose Manuel Lopez
>

regards Jose



--
rickygm

http://gnuforever.homelinux.com

[José Manuel López del Río]

Hello Rick,

For sure the Search Guard installation is much trickier than Xpack. Xpack is already integrated with elasticsearch and can be configured much easier.

To sum up, you want to have a full wazuh-manager v3.9.2 + elasticsearch and kibana v7.3.2 implementation. To achieve that, you can follow the documentation provided previously. The installation and configuration of the wazuh-app into Kibana is already considered in the previous guides. 

I will be glad to walk you through the whole process until getting everything up and running, so please keep me posted.

Best Regards,

Jose Manuel Lopez

[Rick Gutierrez]

El lun., 7 oct. 2019 a las 7:53, José Manuel López del Río

(<josemanu...@wazuh.com>) escribió:
>
> Hello Rick,

Hi Jose

>
> For sure the Search Guard installation is much trickier than Xpack. Xpack is already integrated with elasticsearch and can be configured much easier.
>

I already have it ready, the authentication works perfectly, although
here I have a doubt, you can specify a user different from the ones
you bring by default.

> To sum up, you want to have a full wazuh-manager v3.9.2 + elasticsearch and kibana v7.3.2 implementation. To achieve that, you can follow the documentation provided previously. The installation and configuration of the wazuh-app into Kibana is already considered in the previous guides.

I am using these versions, I have a question on the page I do not see
how to configure the wazuh API credentials , I'm using the following
command, but I can't authenticate it

===== version ===
wazuh-api-3.10.2-1.x86_64
wazuh-manager-3.10.2-1.x86_64

=== command for api Wazuh

cd /var/ossec/api/configuration/auth

$> sudo node htpasswd -c user admin

>
> I will be glad to walk you through the whole process until getting everything up and running, so please keep me posted.
>
> Best Regards,
> Jose Manuel Lopez
>

thank , I add a screen of error api Wazuh


--
rickygm

http://gnuforever.homelinux.com

[José Manuel López del Río]

Hello Rick,

You can add new users directly from Kibana. For that you can follow these steps:

1. In Kibana, go to Management.

2. Click on Users at the Security section.

3. Create a new user.

To change the API credentials:

1. Use the following commands

cd /var/ossec/api/configuration/auth

node htpasswd -Bc -C 10 user insertuserhere

And then type your password.

2. Restart wazuh-api and wazuh-manager --> systemctl restart wazuh-api   and   systemctl restart wazuh-manager

In order to use HTTPS at the host URL of the API, you need to secure the wazuh-api. For that, you can use a script at /var/ossec/api/scripts/configure_api.sh. You can check the script documentation here: /var/ossec/api/scripts/configure_api.sh 

I hope it helps. Let me know if you need anything else.

Best Regards,

Jose Manuel Lopez

[Rick Gutierrez]

El mar., 8 oct. 2019 a las 10:26, José Manuel López del Río (<josemanu...@wazuh.com>) escribió:

Hello Rick,

Hi Jose

 

You can add new users directly from Kibana. For that you can follow these steps:

1. In Kibana, go to Management.

2. Click on Users at the Security section.

3. Create a new user.

Excellent

 

To change the API credentials:

1. Use the following commands

cd /var/ossec/api/configuration/auth

node htpasswd -Bc -C 10 user insertuserhere

And then type your password.

2. Restart wazuh-api and wazuh-manager --> systemctl restart wazuh-api   and   systemctl restart wazuh-manager

That was the problem, I had not restarted

 

In order to use HTTPS at the host URL of the API, you need to secure the wazuh-api. For that, you can use a script at /var/ossec/api/scripts/configure_api.sh. You can check the script documentation here: /var/ossec/api/scripts/configure_api.sh 

Ok, I didn't know this, I'm going to try.

 

I hope it helps. Let me know if you need anything else.

I have a problem loading the wazu app in kibana, I get this error: wazuhapp Check Elasticsearch template error , I attach the image

 

Best Regards,

Jose Manuel Lopez

regards! 

--

rickygm

http://gnuforever.homelinux.com

[Rick Gutierrez]

El mar., 8 oct. 2019 a las 10:39, Rick Gutierrez (<xserve...@gmail.com>) escribió:

Ok, I didn't know this, I'm going to try.

 

I hope it helps. Let me know if you need anything else.

I have a problem loading the wazu app in kibana, I get this error: wazuhapp Check Elasticsearch template error , I attach the image

 

I found the error, when I download the template for wazuh it overwrites filebeat.yml and I lose the previous configuration, we have to add it again

 

output.elasticsearch.protocol: https

output.elasticsearch.ssl.certificate: "/etc/filebeat/certs/wazuh-manager.crt"

output.elasticsearch.ssl.key: "/etc/filebeat/certs/wazuh-manager.key"

output.elasticsearch.ssl.certificate_authorities: ["/etc/filebeat/certs/ca/ca.crt"]

output.elasticsearch.username: "elastic"

output.elasticsearch.password: "xxdsdsdsd"

--

rickygm

http://gnuforever.homelinux.com

[José Manuel López del Río]

Hello Rick,

Glad to hear that. Let me know if you need anything else.

Best Regards,

Jose Manuel Lopez