Wazuh - Filebeat Multiple output

[Luke Lee]

Hi all, I am trying to insert the "Suricata" module in Filebeat. But currently I have Wazuh in place using the output to elasticsearch. 

output.elasticsearch:

  hosts: ['http://ip:9200']

  pipeline: geoip-info

  indices:

    - index: 'wazuh-alerts-3.x-%{+yyyy.MM.dd}'

    - index: 'wazuh-monitoring-3.x-%{+yyyy.MM.dd}'

      username: 

      password: 

In this case, can I also export Suricata index here too ?? Please advise. 

[José Sánchez Robles]

Hi Luke Lee,

Y ahora?

Hi, Luke Lee,

Your Filebeat configuration does not look like our configuration. I recommend you to use our Filebeat configuration and add our Wazuh module for Filebeat, you can check how to configure it in our documentation.

When you update your Filebeat configuration change this in your /etc/filebeat/filebeat.yml to enable the Suricata module:

# Wazuh - Filebeat configuration file
filebeat.modules:
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: false
  - module: suricata
setup.template.json.enabled: true
setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.template.json.name: 'wazuh'
setup.template.overwrite: false
setup.ilm.enabled: false

output.elasticsearch.hosts: ['http://[ELASTICSEARCH-IP]:9200']

I hope it helps. Please, don't hesitate to ask again if you have more doubts

Best regards,

Jose.

[Elwali Karkoub]

Hello Luke & Jose,


Sorry for hijacking into your thread. Just wanted to add up that Filebeat does not allow multiple output as Logstash, but you can use multiple filebeat services which will output into different destination.

To achieve above following are the steps :

1. Stop Filebeat service :
   
   
   systemctl stop filebeat
   
   
   
   
2. Create the new service file /usr/lib/systemd/system/filebeat2.service and add :
   
   [Unit] Description=filebeat2 Documentation=https://www.elastic.co/guide/en/beats/filebeat/current/index.html Wants=network-online.target After=network-online.target [Service] ExecStart=/usr/share/filebeat/bin/filebeat -c /etc/filebeat2/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat2 -path.data /var/lib/filebeat -path.logs /var/log/filebeat/filebeat2 Restart=always [Install] WantedBy=multi-user.target
   
   
3. Create the folder /etc/filebeat2 and copy all the configuration files from /etc/filebeat :
   
   
   cp /etc/filebeat/* /etc/filebeat2/
   
   
   
4. Configure the new output under /etc/filebeat2/filebeat.yml.
   
   
   
5. Reload and restart Filebeat :
   
   
   systemctl daemon-reload
systemctl restart filebeat
   
   
   
6. Then you will have two Filebeat processes running and reading Wazuh alerts :
   
   
   
   [root@localhost vagrant]# ps aux | grep filebeat
root      6237  0.0  0.6 480092 18120 ?        Ssl  14:48   0:00 /usr/share/filebeat/bin/filebeat -c /etc/filebeat2/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat2 -path.data /var/lib/filebeat -path.logs /var/log/filebeat/filebeat2
root      6264  0.0  0.6 414556 18016 ?        Ssl  14:48   0:00 /usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
root      6332  0.0  0.0 112712   980 pts/0    R+   14:51   0:00 grep --color=auto filebeat

[root@localhost vagrant]# lsof /var/ossec/logs/alerts/alerts.json
COMMAND    PID  USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
ossec-ana 2942 ossec    9w   REG    8,1   254156 67369809 /var/ossec/logs/alerts/alerts.json
filebeat  6237  root    5r   REG    8,1   254156 67369809 /var/ossec/logs/alerts/alerts.json
filebeat  6264  root    3r   REG    8,1   254156 67369809 /var/ossec/logs/alerts/alerts.json
   
   
   Hope this helps,
   
   Regards,
   Wali
   

[Luke Lee]

Hi Jose & Wali, 

Thanks for both of your replies. Indeed useful to me. 

Wali, may I know when you say filebeat only support one output, means that for every different module, e.g Wazuh, Suricat, Syslog, I will have to use different filebeat directory with separate configuration to load different template? 

Do you mean by that? Thanks 

Secondly, what if I output the results to logstash, instead of Elasticsearch and let the processing done on logstash side then only will pass to ES, will that work? Have any of you tried that ? If not we might have to create multiple Filebeat ? 

[Elwali Karkoub]

Hello Luke,

You can indeed you may use several modules (wazuh, suricata...) with one output. The provided solution would be ideal if you want to index/forward into separated elasticsearch/logstash output and you want to use a custom configuration (custom index name for instance)  for each service.


If you are willing to use logstash in the data flow, following are the steps to achieve it :

• Filebeat configuration would be : https://github.com/wazuh/wazuh/blob/issue-3834/extensions/filebeat/7.x/filebeat_to_logstash.yml
  
  
• Logstash pipeline : https://github.com/wazuh/wazuh/blob/issue-3834/extensions/logstash/7.x/01-wazuh-remote.conf
  
  
• The template will be managed by logstash in this case, make sure to download it and use it :
  
  
  curl -so /etc/logstash/wazuh-template.json https://raw.githubusercontent.com/wazuh/wazuh/v3.10.2/extensions/elasticsearch/7.x/wazuh-template.json
  

  chmod go+r /etc/logstash/wazuh-template.json
  
  

As you may find out here https://github.com/wazuh/wazuh/issues/3834 we are working on improving our documenation to include the steps to use logstash 7.x.

Hope this helps,

Regards,
Wali

[Luke Lee]

Hi, please advise: 

Turn out there are these errors: 

[2020-02-07T15:05:51,783][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

[2020-02-07T15:06:10,479][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified

[2020-02-07T15:06:10,536][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/config/pipeline.yml"}

[2020-02-07T15:06:10,542][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

[2020-02-07T15:06:10,549][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

[2020-02-07T15:06:30,069][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified

[2020-02-07T15:06:30,129][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/config/pipeline.yml"}

[2020-02-07T15:06:30,139][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

[2020-02-07T15:06:30,149][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

My logstash.yml file: 

path.data: /usr/share/logstash/data/

path.logs: /var/log/logstash/

path.config: /etc/logstash/conf.d/

The rest of the files and directories: 

Directories:

Path Dir : /var/lib/logstash

Config Dir : /usr/share/logstash/logstash.conf

Pipeline : /usr/share/logstash/config/pipelines.yml

Other Config : /etc/logstash/conf.d/01-wazuh.conf

Duplicate Config : /usr/share/logstash/conf.d/01-wazuh.conf

[Elwali Karkoub]

Hello Luke,

All the configuration files are placed under /etc/logstash/ and you should not have duplicate config. Following are the location of my confg files :

[root@localhost vagrant]# ls -ll /etc/logstash/
total 84
drwxrwxr-x. 2 root root    27 Feb  7 17:01 conf.d
-rw-r--r--. 1 root root  2019 Jan 15 13:39 jvm.options
-rw-r--r--. 1 root root  7482 Jan 15 13:39 log4j2.properties
-rw-r--r--. 1 root root   342 Jan 15 13:39 logstash-sample.conf
-rw-r--r--. 1 root root  8404 Feb  7 16:53 logstash.yml
-rw-r--r--. 1 root root   285 Jan 15 13:39 pipelines.yml
-rw-------. 1 root root  1696 Jan 15 13:39 startup.options
-rw-r--r--. 1 root root 46821 Feb  7 16:55 wazuh-template.json
[root@localhost vagrant]# ls -ll /etc/logstash/conf.d/
total 4
-rw-r--r--. 1 root root 1156 Feb  7 17:01 01-wazuh.conf

Following is my logstash.yml :

[root@localhost vagrant]# cat /etc/logstash/logstash.yml | grep -v "#"
path.data: /var/lib/logstash

path.logs: /var/log/logstash

It seems that your logstash is not able to locate pipeline.yml as it is not under /etc/logstash.

Hope this helps,

Regards,
Wali

[Luke Lee]

Hi thanks for your kind reply. 

When I go to this directory which you have declare for the 'path.data' I sees this files, not sure is it correct. 

root@:/usr/share/logstash# cd /var/lib/logstash

root@:/var/lib/logstash# ls

dead_letter_queue  plugins  queue  uuid

root@:/var/lib/logstash#

[Elwali Karkoub]

Hello Luke,

In my case i do not have the plugins  (maybe becasue i'm not using any) folder as you can see :

[root@localhost vagrant]# ls /var/lib/logstash/
dead_letter_queue  queue  uuid

Regards,
Wali

[Luke Lee]

Hi I need some help,

After making some changes, I kind of lost. 

Logstash ERROR: 

[WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified

[INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/share/logstash/config/pipeline.yml"}

[ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

[INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

Logstash.yml located in /etc/logstash: 

path.data: /var/lib/logstash

path.config: /etc/logstash/conf.d/logstash.conf

path.logs: /var/log/logstash/

[Elwali Karkoub]

Hello Luke,

You may try deleting the path.config (path.config: /etc/logstash/conf.d/logstash.) entry from your logstash,yml then restart Logstash.

Hope it helps,

Regards,
Wali

[Luke Lee]

Hi good day! 

May I know the /conf.d/logstash refers to logstash.conf ?? 

[Elwali Karkoub]

Hello Luke,

Not sure if i understand you correctly. But following are the configuration files in use :

[root@localhost vagrant]# grep -v "#"  /etc/logstash/logstash.yml
path.data: /var/lib/logstash


path.logs: /var/log/logstash

[root@localhost vagrant]# grep -v "#"  /etc/logstash/conf.d/01-wazuh.conf
input {
    beats {
        port => 5000
    }
}

filter {
    json {
      source => "message"
    }
}

filter {
    if [data][srcip] {
        mutate {
            add_field => [ "@src_ip", "%{[data][srcip]}" ]
        }
    }
    if [data][aws][sourceIPAddress] {
        mutate {
            add_field => [ "@src_ip", "%{[data][aws][sourceIPAddress]}" ]
        }
    }
    if [data][win][eventdata][ipAddress] {
        mutate {
            add_field => [ "@src_ip", "%{[data][win][eventdata][ipAddress]}" ]
        }
    }
}

filter {
    date {
        match => ["timestamp", "ISO8601"]
        target => "timestamp"
    }
    mutate {
        remove_field => [ "message", "beat", "input_type", "tags", "count", "@version", "ecs", "log", "offset", "type", "@src_ip", "host" ]
    }
}
output {
    elasticsearch {
        hosts => ["x.x.x.x:9200"]
        manage_template => true
        template_overwrite => true
        template => "/etc/logstash/wazuh-template.json"
        template_name => "wazuh"
        index => "wazuh-alerts-3.x-%{+YYYY.MM.dd}"
    }
}

Hope this helps,

Regards,
Wali

[Luke Lee]

Hi, thanks for replying. 

I think our configuration are similar. Now my logstash keep restarting and there are no logging for it even i have specify the logs location. 

Do you have any idea? 

[Elwali Karkoub]

Hello Luke,

That may be a sign of a misconfiguration. Would you be able to share with me all your configuration files ?

And just to be sure. Download the template as logstash will try to read it as peer the configuration :

curl -so /etc/logstash/wazuh-template.json https://raw.githubusercontent.com/wazuh/wazuh/v3.10.2/extensions/elasticsearch/7.x/wazuh-template.json

chmod go+r /etc/logstash/wazuh-template.json

Regards,
Wali

[Luke Lee]

Hi, below are the conf files located under /etc/logstash/conf/d/ folder. 

Today I notice my logstash cannot start at all. Please advise. 

/etc/logstash/logstash.yml

node.name: ETA10_test
path.data: /var/lib/logstash
pipeline.ordered: auto
path.config: /etc/logstash/conf.d/
config.debug: true
path.queue: /usr/share/logstash/data/queue
queue.max_events: 0

http.host: "10.0.106.144"
http.port: 9610
log.level: trace
path.logs: /var/log/logstash/
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: *LARCadmin2011*
xpack.monitoring.elasticsearch.hosts: ["http://10.0.106.144:9200"]

/etc/logstash/conf.d/01-wazuh.conf

# Wazuh - Logstash configuration file
## Local Wazuh Manager - JSON file input


input {
   file {
       type => "wazuh-alerts"
       path => "/var/ossec/logs/alerts/alerts.json"
       codec => "json"

  }
}


input {
    beats {
        port => 5000

        codec => "json_lines"
        # ssl => true
        # ssl_certificate => "/etc/logstash/logstash.crt"
        # ssl_key => "/etc/logstash/logstash.key"

    }
}


filter {
    if [data][srcip] {
        mutate {
            add_field => [ "@src_ip", "%{[data][srcip]}" ]
        }
    }
    if [data][aws][sourceIPAddress] {
        mutate {
            add_field => [ "@src_ip", "%{[data][aws][sourceIPAddress]}" ]
        }
    }
}

filter {
    geoip {
        source => "@src_ip"
        target => "GeoLocation"
        fields => ["city_name", "country_name", "region_name", "location"]

    }
    date {
        match => ["timestamp", "ISO8601"]

        target => "@timestamp"
    }
    mutate {
        remove_field => [ "timestamp", "beat", "input_type", "tags", "count", "@version", "log", "offset", "type", "@src_ip", "host"]
    }
}


output {
  elasticsearch {
    hosts => [ "xxx" ]
    index => "wazuh-alerts-3.x-%{+YYYY.MM.dd}"
    user => "xxx"
    password => "*xxxxx*"
  }
}

03-filebeat-output.conf

#Output


output {
  if [fileset][module] == "system" {
    elasticsearch {
      hosts => [ "xxxx:9200" ]
      manage_template => false
      index => "%{[@metadata][beat]}-system-%{+YYYY.MM}"
      pipeline => "%{[@metadata][pipeline]}"
      user => "xxxx"
      password => "*xxxxx*"
    }
  }
}


output {
  if [fileset][module] == "nginx" {
    elasticsearch {
      hosts => [ "xxxxxxx:9200" ]
      manage_template => false
      index => "%{[@metadata][beat]}-nginx-%{+YYYY.MM}"
      pipeline => "%{[@metadata][pipeline]}"
      user => "xxx"
      password => "*xxxx*"
    }
  }
}

30-elasticsearch-output.conf

#Output to Elasticsearch


output {
  elasticsearch {
    hosts => ["xxxxx:9200"]
    user => "xxxxx"
    password => "*xxxxx*"
    manage_template => false
    index => "nginx-%{+YYYY.MM.dd}"
  }
  # stout { code => rubydebug }
}

[Elwali Karkoub]

Hello Luke,


In that case it will be helpful to check the logs :

cat /var/log/logstash/logstash-plain.log | grep -E "(ERROR|WARN|CRITICAL|FATAL)"

Note that in my case. I'm not using any output in the logstash,yml :

[root@localhost vagrant]# grep -v "#" /etc/logstash/logstash.yml
path.data: /var/lib/logstash

path.logs: /var/log/logstash

[root@localhost vagrant]# grep -v "#" /etc/logstash/pipelines.yml

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

Regards,
Wali

[Luke Lee]

Hi, the logs was yesterday. Then after that there are no logs. 

cat /var/log/logstash/logstash-plain.log | grep -E "(ERROR|WARN|CRITICAL|FATAL)"

[2020-02-11T10:34:06,271][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

[2020-02-11T10:34:26,843][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified

[2020-02-11T10:34:26,902][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

[Elwali Karkoub]

Hello Luke,

Can you check if your configurations files have the right owner and permissions :

[root@localhost vagrant]# ls -ll /etc/logstash/
total 84
drwxrwxr-x. 2 root root    27 Feb  7 17:01 conf.d
-rw-r--r--. 1 root root  2019 Jan 15 13:39 jvm.options
-rw-r--r--. 1 root root  7482 Jan 15 13:39 log4j2.properties
-rw-r--r--. 1 root root   342 Jan 15 13:39 logstash-sample.conf
-rw-r--r--. 1 root root  8404 Feb  7 16:53 logstash.yml
-rw-r--r--. 1 root root   285 Jan 15 13:39 pipelines.yml
-rw-------. 1 root root  1696 Jan 15 13:39 startup.options
-rw-r--r--. 1 root root 46821 Feb  7 16:55 wazuh-template.json

Hope it helps,
Regards,
Wali

[Luke Lee]

Hi thanks for your reply. 

Oh it need to owned by 'root' ?? Not logstash ?

[Elwali Karkoub]

Hello Luke,

Indeed all logstash configuration files are owned by root. Following is the complete list of my configuration files :

[root@localhost vagrant]# ls -llR /etc/logstash/

/etc/logstash/:
total 84
drwxrwxr-x. 2 root root    27 Feb  7 17:01 conf.d
-rw-r--r--. 1 root root  2019 Jan 15 13:39 jvm.options
-rw-r--r--. 1 root root  7482 Jan 15 13:39 log4j2.properties
-rw-r--r--. 1 root root   342 Jan 15 13:39 logstash-sample.conf
-rw-r--r--. 1 root root  8404 Feb  7 16:53 logstash.yml
-rw-r--r--. 1 root root   285 Jan 15 13:39 pipelines.yml
-rw-------. 1 root root  1696 Jan 15 13:39 startup.options
-rw-r--r--. 1 root root 46821 Feb  7 16:55 wazuh-template.json

/etc/logstash/conf.d:

total 4
-rw-r--r--. 1 root root 1156 Feb  7 17:01 01-wazuh.conf

Hope it helps,
Wali

[Luke Lee]

Hi thanks for your reply. 

Currently all my logstash files are owned by the user 'logstash' and 'logstash' group. Should I change all of these file to root ?

How about the directory above it? 1 level up. 

Thanks 

[Elwali Karkoub]

Hello Luke,

Exactly the configuration files should be owned by root.

The directory of logstash as well :

drwxr-xr-x.  3 root root             183 Feb  7 17:11 logstash

Regards,
Wali

[Luke Lee]

Hi, how about those logstash files in /usr/share? I have no problem running using logstash user before. Now I change back to root and the service can't run. 

If I run as logstash, will there be any issue?

[Elwali Karkoub]

Hello Luke,

All files under `/usr/share` are owner as well by logstash :

[root@localhost vagrant]# ls -ll /usr/share/logstash/
total 848
drwxr-xr-x. 2 logstash logstash   4096 Feb  7 16:53 bin
-rw-r--r--. 1 logstash logstash   2276 Jan 15 13:39 CONTRIBUTORS
drwxrwxr-x. 2 logstash logstash      6 Jan 15 13:39 data
-rw-r--r--. 1 logstash logstash   4097 Jan 15 13:39 Gemfile
-rw-r--r--. 1 logstash logstash  22792 Jan 15 13:39 Gemfile.lock
drwxr-xr-x. 6 logstash logstash     84 Feb  7 16:53 lib
-rw-r--r--. 1 logstash logstash  13675 Jan 15 13:39 LICENSE.txt
drwxr-xr-x. 4 logstash logstash     90 Feb  7 16:53 logstash-core
drwxr-xr-x. 3 logstash logstash     86 Feb  7 16:53 logstash-core-plugin-api
drwxr-xr-x. 4 logstash logstash     55 Feb  7 16:53 modules
-rw-r--r--. 1 logstash logstash 808305 Jan 15 13:39 NOTICE.TXT
drwxr-xr-x. 3 logstash logstash     30 Feb  7 16:53 tools
drwxr-xr-x. 4 logstash logstash     33 Feb  7 16:53 vendor
drwxr-xr-x. 9 logstash logstash    193 Feb  7 16:53 x-pack

As well as the parent :

drwxr-xr-x.  11 logstash logstash   241 Feb  7 16:53 logstash

Hope this helps,
Wali

[Luke Lee]

Hi , thanks for your sharing. I have the same setting as yours. But still I notice the service will restart after a few seconds. 

Not sure how can we troubleshoot it. 

cat </s

[Elwali Karkoub]

Hello Luke,

Can you check if your logstash service ( in case of using systemctl) file similar to below  :

[root@localhost vagrant]# cat /etc/systemd/system/logstash.service
[Unit]
Description=logstash

[Service]
Type=simple
User=logstash
Group=logstash
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/logstash
EnvironmentFile=-/etc/sysconfig/logstash
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384

[Install]
WantedBy=multi-user.target

Note that i have just tested installing it again following steps described in : https://documentation.wazuh.com/3.11/installation-guide/installing-elastic-stack/transform_logstash.html?highlight=logstash (It would be perfect if you try the same in your case) and it is working as expected.

Can you let me know what version of logstash and in what OS ? any more information would be helpful to replicate your issue.

Regards,
Wali

[Luke Lee]

Hi, Thanks for your reply. 

The content of this file (/etc/systemd/system/logstash.service) are the same. 

My Logstash's version: 7.1.1

OS Version: Ubuntu 16.04.6 LTS (xenial)

[Elwali Karkoub]

Hello luke,

Would you please confirm if the steps described in https://documentation.wazuh.com/3.11/installation-guide/installing-elastic-stack/transform_logstash.html?highlight=logstash are the steps you followed to install logstash ?

It is always helpful to share the configuration files.

You may monitor the logs in `grep logstash /var/log/messages` after starting logstash services it may reveal more information.

How about the performance in that instance ? CPU, Memory ?

By sharing more information we can track down the issue.

Thanks for your collaboration.

Regards,
Wali

[Luke Lee]

Hi , My result for  "filebeat test output" 

filebeat test output
elasticsearch: http://ip:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: ip
    dial up... OK
  TLS... WARN secure connection disabled
  talk to server... OK
  version: 7.3.2

grep logstash /var/log/messages >>  No result. 

How can I check the Memory 252 GB & CPU 32 core. 

[Elwali Karkoub]

Hello Luke,

Your filebeat test ouput has elasticsearch as an output, is that expected ?

Let's approach that in a different way. Do you need logstash in your flow as for now it is not mandatory to send Wazuh's alerts to Kibana ?

If the answer is yes then again : Would you please confirm if the steps described in https://documentation.wazuh.com/3.11/installation-guide/installing-elastic-stack/transform_logstash.html?highlight=logstash are the steps you followed to install logstash ?

Thank you.

Regards,
Wali

cat /var/log</spa

[Luke Lee]

Hi, thanks for replying. 

I was thinking all data has to send to ES. From there then we can view the data/visualization from kibana dashboard. 

As for the LOGSTASH installation , I have followed the guide but its for 7.1.1. 

cat /var<sp

[Elwali Karkoub]

Hello Luke,

In that case, Logstash is not mandatory you will will the data in Kibana Just by going through Filebeat --> Elasticsearch --> Kibana Flow.

Regards,
Wali

[Luke Lee]

Hi thanks for replying. So do I need to have different Filebeat for diff input or output? 

E.g 1 is for wazuh, the other one will be for suricata etc. 

Thanks 

[Elwali Karkoub]

Hello Luke,


If you are referring to output to be output's indices from different inputs you may achieve it as below :

# Wazuh - Filebeat configuration file
filebeat.modules:

#  Normal module, path:  /var/ossec/logs/alerts/alerts.json, index: wazuh-alerts-3.x-yyyy.mm.dd

- module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: false

# Custom input, path: /var/custom/alerts/suricata.json, index: wazuh-suricata-yyyy.mm.dd
- module: suricata
  alerts:
    enabled: true
    input:
      paths:
        - "/var/custom/alerts/suricata.json"
      fields:
        index_prefix: "wazuh-suricata-"

setup.template.json.enabled: true
setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.template.json.name: 'wazuh'

setup.template.overwrite: true
setup.ilm.enabled: false
output.elasticsearch.hosts: ['http://localhost:9200']

If you want to send to different output services ( logstash and elasticsearch as an example), Please refere to my first response in this thread.

Regards,
Wali

-rw-------. 1 root root  <span style="color:#06

[Luke Lee]

Hi, I have edited as you have suggested. But when I restart the Filebeat it gave me error. 

-rw-r--r--.</

[Elwali Karkoub]

Hello Luke,

Would be able to share the error ?

Please run following and share the output :

filebeat test config

filebeat test output

I have tested with below configuration and had no error :

# Wazuh - Filebeat configuration file
filebeat.modules:

  - module: kibana
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: true

# Custom input, path: /var/custom/alerts/suricata.json, index: wazuh-suricata-yyyy.mm.dd

  - module: nginx
    access:
      enabled: true

  - module: suricata
    alerts:
      enabled: true
      input:
        paths:
          - "/var/custom/alerts/suricata.json"
        fields:
          index_prefix: "wazuh-suricata-"

Make sure you do not add any indentations.

Regards,
Wali

[Luke Lee]

Hi, when I do the following, it gave me these results: 

filebeat test config:

Exiting: Fileset suricata/alerts is configured but doesn't exist

filebeat test output:

elasticsearch: http://10.0.106.144:9200...

  parse url... OK

  connection...

    parse host... OK

    dns lookup... OK

    addresses: 10.0.106.144

    dial up... OK

  TLS... WARN secure connection disabled

  talk to server... OK

  version: 7.3.2

Secondly, may I know the path: /var/custom/alerts/suricata.json, is it refers to the suricata .json file? Mine is only located at /var/log/suricata/eve.json. 

Setting that as the path is it correct ?

Please guide, thx. 

/etc<span

[Elwali Karkoub]

Hello Luke,

The path was just an example, you need to change it by the path where you have your suricata logs.

Your setup looks good and filebeat is able to reach elasticsearch. so I'm not sure what error you were referring to ?

Regards,
Wali

[Luke Lee]

Hi, 

Previously I have shared the screenshot that the Filebeat service cannot be started. Yea. 

[Elwali Karkoub]

Hello Luke,

The only screenshot i received was for your filebeat configuration :



But anyway, if the service is still failing please share with the error that you are referring to.

Regards,
Wali

<div style="background-color:rgb(250,250,250);border-color:rgb(187,187,187);border-s

[Luke Lee]

Hi this is the error. After I have made the changes and restart filebeat service. 

[Elwali Karkoub]

Hello Luke,

Sorry but I am not able to access the shared screenshot. Anyway can you run filebeat on debug mode :

filebeat -e -d "publish"

 Then copy and paste the output logs.

Regards,
wali

drwxr-xr-x.  3 root root             183 Feb  7 17<span style="color:

[Luke Lee]

filebeat -e -d "publish"
2020-03-24T15:41:52.074+0800    INFO    instance/beat.go:571    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2020-03-24T15:41:52.075+0800    INFO    instance/beat.go:579    Beat ID: 9e68d952-0a9c-42da-9aae-78ad191170e2
2020-03-24T15:41:52.075+0800    INFO    [index-management.ilm]  ilm/ilm.go:129  Policy name: filebeat-7.1.1
2020-03-24T15:41:55.079+0800    INFO    add_cloud_metadata/add_cloud_metadata.go:346    add_cloud_metadata: hosting provider type not detected.
2020-03-24T15:41:55.079+0800    INFO    [seccomp]       seccomp/seccomp.go:116  Syscall filter successfully installed
2020-03-24T15:41:55.079+0800    INFO    [beat]  instance/beat.go:827    Beat info       {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "9e68d952-0a9c-42da-9aae-78ad191170e2"}}}
2020-03-24T15:41:55.079+0800    INFO    [beat]  instance/beat.go:836    Build info      {"system_info": {"build": {"commit": "3358d9a5a09e3c6709a2d3aaafde628ea34e8419", "libbeat": "7.1.1", "time": "2019-05-23T13:21:33.000Z", "version": "7.1.1"}}}
2020-03-24T15:41:55.079+0800    INFO    [beat]  instance/beat.go:839    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":32,"version":"go1.11.5"}}}
2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64","fe80::7c38:c4ff:fee9:732/64","172.28.0.1/16","fe80::42:3ff:fe07:b1a6/64","172.29.0.1/16","fe80::42:ceff:fe01:37cb/64","fe80::1c18:2dff:fe05:c8c0/64","fe80::b055:29ff:fe3b:6398/64","fe80::6ce3:2fff:fe1b:c12a/64","fe80::a0ab:22ff:feb5:41a9/64","fe80::5840:17ff:fe78:7f2e/64"],"kernel_version":"4.4.0-154-generic","mac":["f8:bc:12:3b:3a:dc","f8:bc:12:3b:3a:dd","f8:bc:12:3b:3a:de","f8:bc:12:3b:3a:df","52:54:00:4e:f6:54","52:54:00:4e:f6:54","02:42:62:23:53:2f","02:42:cc:db:f7:40","02:42:d9:1a:fe:bd","2a:31:74:70:75:ec","32:09:83:e1:19:56","46:43:c2:d6:4d:18","7e:38:c4:e9:07:32","02:42:03:07:b1:a6","02:42:ce:01:37:cb","1e:18:2d:05:c8:c0","b2:55:29:3b:63:98","6e:e3:2f:1b:c1:2a","a2:ab:22:b5:41:a9","5a:40:17:78:7f:2e"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"16.04.6 LTS (Xenial Xerus)","major":16,"minor":4,"patch":6,"codename":"xenial"},"timezone":"+08","timezone_offset_sec":28800,"id":"b4f8b9056e219987a75e0c1a5b729cdc"}}}
2020-03-24T15:41:55.083+0800    INFO    [beat]  instance/beat.go:872    Process info    {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/etc/filebeat", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 21066, "ppid": 34069, "seccomp": {"mode":"filter"}, "start_time": "2020-03-24T15:41:51.780+0800"}}}
2020-03-24T15:41:55.083+0800    INFO    instance/beat.go:280    Setup Beat: filebeat; Version: 7.1.1
2020-03-24T15:41:55.083+0800    INFO    [index-management]      idxmgmt/std.go:165      Set output.elasticsearch.index to 'filebeat-7.1.1' as ILM is enabled.
2020-03-24T15:41:55.083+0800    INFO    elasticsearch/client.go:165     Elasticsearch url: http://10.0.106.144:9200
2020-03-24T15:41:55.084+0800    INFO    [publisher]     pipeline/module.go:97   Beat name: eta10
2020-03-24T15:41:55.103+0800    INFO    instance/beat.go:361    filebeat stopped.
2020-03-24T15:41:55.103+0800    ERROR   instance/beat.go:802    Exiting: Fileset suricata/alerts is configured but doesn't exist

Exiting: Fileset suricata/alerts is configured but doesn'

t exist

[Elwali Karkoub]

Hello Luke,

As peer the error message you may have configured the wrong path to suricata logs or it does not exist :

ERROR   instance/beat.go:802    Exiting: Fileset suricata/alerts is configured but doesn't exist


Filebeat will always fail starting if you configure a wrong or unexisting path.

Please make sure you have the right path configured.

Regards,
Wali

On Tuesday, March 24, 2020 at 8:42:52 AM UTC+1, Luke Lee wrote:

filebeat -e -d "publish"
2020-03-24T15:41:52.074+0800    INFO    instance/beat.go:571    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2020-03-24T15:41:52.075+0800    INFO    instance/beat.go:579    Beat ID: 9e68d952-0a9c-42da-9aae-78ad191170e2
2020-03-24T15:41:52.075+0800    INFO    [index-management.ilm]  ilm/ilm.go:129  Policy name: filebeat-7.1.1
2020-03-24T15:41:55.079+0800    INFO    add_cloud_metadata/add_cloud_metadata.go:346    add_cloud_metadata: hosting provider type not detected.
2020-03-24T15:41:55.079+0800    INFO    [seccomp]       seccomp/seccomp.go:116  Syscall filter successfully installed
2020-03-24T15:41:55.079+0800    INFO    [beat]  instance/beat.go:827    Beat info       {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "9e68d952-0a9c-42da-9aae-78ad191170e2"}}}
2020-03-24T15:41:55.079+0800    INFO    [beat]  instance/beat.go:836    Build info      {"system_info": {"build": {"commit": "3358d9a5a09e3c6709a2d3aaafde628ea34e8419", "libbeat": "7.1.1", "time": "2019-05-23T13:21:33.000Z", "version": "7.1.1"}}}
2020-03-24T15:41:55.079+0800    INFO    [beat]  instance/beat.go:839    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":32,"version":"go1.11.5"}}}
2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64","fe80::7c38:c4ff:fee9:732/64","172.28.0.1/16","fe80::42:3ff:fe07:b1a6/64","172.29.0.1/16","fe80::42:ceff:fe01:37cb/64","fe80::1c18:2dff:fe05:c8c0/64","fe80::b055:29ff:fe3b:6398/64","fe80::6ce3:2fff:fe1b:c12a/64","fe80::a0ab:22ff:feb5:41a9/64","fe80::5840:17ff:fe78:7f2e/64"],"kernel_version":"4.4.0-154-generic","mac":["f8:bc:12:3b:3a:dc","f8:bc:12:3b:3a:dd","f8:bc:12:3b:3a:de","f8:bc:12:3b:3a:df","52:54:00:4e:f6:54","52:54:00:4e:f6:54","02:42:62:23:53:2f","02:42:cc:db:f7:40","02:42:d9:1a:fe:bd","2a:31:74:70:75:ec","32:09:83:e1:19:56","46:43:c2:d6:4d:18","7e:38:c4:e9:07:32","02:42:03:07:b1:a6","02:42:ce:01:37:cb","1e:18:2d:05:c8:c0","b2:55:29:3b:63:98","6e:e3:2f:1b:c1:2a","a2:ab:22:b5:41:a9","5a:40:17:78:7f:2e"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"16.04.6 LTS (Xenial Xerus)","major":16,"minor":4,"patch":6,"codename":"xenial"},"timezone":"+08","timezone_offset_sec":28800,"id":"b4f8b9056e219987a75e0c1a5b729cdc"}}}

2020-03-24T15:41:55.083+0800    INFO    [beat]  instance/beat.go<spa

[Luke Lee]

Hi this is the location I set the path to. Looks like its updated daily and I opened the file there are info .... too 

[Elwali Karkoub]

Hey,

I'm not able to see the attached image. But make sure that you have the exact path configured in your `/etc/filebeat/filebeat.yml` otherwise filebeat won't start.

ERROR   instance/beat.go:802    Exiting: Fileset suricata/alerts is configured but doesn't exist


Regards,
Wali

[Luke Lee]

Hi, this is the exact path to the file: "/var/log/suricata/eve.json"

And I have checked the file exist and working, but still no data on the dashboard. 

Is there anything I need to check? 

[Luke Lee]

Hi, 

This is the filebeat config, 

    - module: suricata

      alerts:

        enabled: true

        input:

          paths:

            - "/var/log/suricata/eve.json"

when restart the filebeat service, the error again and unable to start..... 

[Luke Lee]

Besides that, I tried to run "filebeat setup -e"  and there are errors. The dashboard cannot be loaded properly. 

Is it due to the rights of the .json files? need to give rights to filebeat:filebeat ?

-rw-r--r-- 1 root root  6704 May 23  2019 Filebeat-nginx-logs.json

-rw-r--r-- 1 root root 13680 May 23  2019 Filebeat-mysql.json

-rw-r--r-- 1 root root  4713 May 23  2019 Filebeat-Mongodb-overview.json

-rw-r--r-- 1 root root 10460 May 23  2019 Filebeat-logstash-slowlog.json

-rw-r--r-- 1 root root 18145 May 23  2019 Filebeat-Iptables-Ubiquiti-Firewall-Overview.json

-rw-r--r-- 1 root root 12401 May 23  2019 Filebeat-iis.json

-rw-r--r-- 1 root root  3571 May 23  2019 Filebeat-icinga-startup-errors.json

-rw-r--r-- 1 root root  5988 May 23  2019 Filebeat-icinga-main-log.json

-rw-r--r-- 1 root root  6049 May 23  2019 Filebeat-icinga-debug-log.json

-rw-r--r-- 1 root root  7855 May 23  2019 Filebeat-auth-sudo-commands.json

-rw-r--r-- 1 root root  9273 May 23  2019 Filebeat-auditd.json

-rw-r--r-- 1 root root 14141 May 23  2019 osquery-compliance.json

-rw-r--r-- 1 root root 10918 May 23  2019 ml-traefik-remote-ip-url-explorer.json

-rw-r--r-- 1 root root 10950 May 23  2019 Filebeat-redis.json

-rw-r--r-- 1 root root  6257 May 23  2019 Filebeat-Postgresql-slowlogs.json

-rw-r--r-- 1 root root 14319 May 23  2019 Filebeat-new-users-and-groups.json

-rw-r--r-- 1 root root 16786 May 23  2019 filebeat-network-flows-top-n.json

-rw-r--r-- 1 root root  6348 May 23  2019 Filebeat-logstash-log.json

-rw-r--r-- 1 root root  8561 May 23  2019 Filebeat-Kafka-overview.json

-rw-r--r-- 1 root root 15745 May 23  2019 Filebeat-Iptables-Overview.json

-rw-r--r-- 1 root root  8784 May 23  2019 Filebeat-haproxy-overview.json

-rw-r--r-- 1 root root 13155 May 23  2019 Filebeat-apache.json

[Elwali Karkoub]

Hello Luke,

I have went ahead and installed Suricata and just tested with using the suricata module and following are the steps, PLEASE follow the same :

• Add the default modules in the filebeat yml, as below :
  
  

  #default modules 
  filebeat.config.modules.path: '${path.config}/modules.d/*.yml'

• 
  # Wazuh - Filebeat configuration file
  filebeat.modules:

• 
    - module: wazuh
      alerts:
        enabled: true
      archives:
        enabled: false

• 
  
  setup.template.json.enabled: true

• 
  setup.template.json.path: '/etc/filebeat/wazuh-template.json'
  setup.template.json.name: 'wazuh'
  setup.template.overwrite: true
  setup.ilm.enabled: false

• 
  
  output.elasticsearch.hosts: ['http://YOUR_ELASTIC_SERVER_IP:9200']

  
  
  
  
  
   
  
• Load  and enable the suricata module :
  
  
  filebeat modules enable suricata
  
  
  
  
• Restar filebeat :
  
  
  systemctl restart filebeat.service
  
  
  
  
• The module will take care of reading the logs from eve.json and dump them under the filebeat index, as well as Wazuh module will be working fine:
  
  

Hope this helps,

Regards,
Wali

[Luke Lee]

Hi thanks for testing it out. But mine is doing the same, but there are other things might be causing the complication. 

The filebeat logs mainly showing data from /var/log/secure and /var/log/messages. 

May I know is there any settings in /modules.d/suricata.yml ? 

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64","fe80::7c38:c4ff:fee9:732/64","172.28.0.1/16","fe80::42:3ff:fe07:b1a6/64","172.29.0.1/16","fe80::42:ceff:fe01:37cb/64","fe80::1c18:2dff:fe05:c8c0/64","fe80::b055:29ff:fe3b:6398/64","fe80::6ce3:2fff:fe1b:c12a/64","fe80::a0ab:22ff:feb5:41a9/64","fe80::5840:17ff:fe78:7f2e/64"],"kernel_version":"4.4.0-154-generic","mac":["f8:bc:12:3b:3a:dc","f8:bc:12:3b:3a:dd","f8:bc:12:3b:3a:de","f8:bc:12:3b:3a:df","52:54:00:4e:f6:54","52:54:00:4e:f6:54","02:42:62:23:53:2f","02:42:cc:db:f7:40","02:42:d9:1a:fe:bd","2a:31:74:70:75:ec","32:09:83:e1:19:56","46:43:c2:d6:4d:18","7e:38:c4:e9:07:32","02:42:03:07:b1:a6","02:42:ce:01:37:cb","1e:18:2d:05:c8:c0","b2:55:29:3b:63:98","6e:e3:2f:1b:c1:2a","a2:ab:22:b5:41:a9","5a:40:17:78:7f:2e"],"os":{"family"<span style="color:#660"

[Luke Lee]

Hi when I run this

filebeat -e -M "suricata.var.paths=[/var/log/suricata/eve.json]"

Error shown: 

2020-03-31T16:56:08.466+0800    ERROR   instance/beat.go:802    Exiting: 1 error: invalid config: yaml: line 9: mapping values are not allowed in this context
Exiting: 1 error: invalid config: yaml: line 9: mapping values are not allowed in this context

[Elwali Karkoub]

Hello Luke,

The content of my suricata yml is as below  :

[root@localhost vagrant]# grep -v "#" /etc/filebeat/modules.d/suricata.yml

- module: suricata
  eve:
    enabled: true

I'm not sure what are you referring to by  The filebeat logs mainly showing data from /var/log/secure and /var/log/messages.

Please kindly follow the same steps shared to enable the module.

Regards,
Wali

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64","fe80::7c38:c4ff:fee9:732/64","172.28.0.1/16","fe80::42:3ff:fe07:b1a6/64","<a href="http://172.29.0.1/16" rel="nofollow" target="_blank" onmousedown="this.href='http://www.google.com/url?q\x3d

[Luke Lee]

Hi, My suricata.yml in /etc/filebeat/modules.d/ same as yours. 

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64","fe80::7c38:c4ff:fee9:732/64","172.28.0.1/16","fe80::42:3ff:fe07:b1a6/64","<a href="<a hre

[Elwali Karkoub]

Hey,

Please try enabling the module as described previously and share logs/screenshots if you encounter any issues.

Regards,
Wali

[Luke Lee]

Erm. Sorry which one are you referring ? 

I think I should have enabled that too.

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64","fe80::7c38:c4ff:fee9:732/64","172.28.0.1/16",&qu

[Elwali Karkoub]

Referring to the steps here : https://groups.google.com/forum/#!msg/wazuh/dQ7B2cL43aw/VfyBZaMJBAAJ.

Have tested them and working as excepted.

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64","fe80::7c38:c4ff:fee9:732/64","<a href="http://172.28.0.1/16" rel="nofollow" target="_blank" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2F172.28.0.1%2F16\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG2ZkLW7maM79

[Luke Lee]

Ok thanks. Let me do an upgrade to ELKS 7.7 to see if these problem resolves. Thank you

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","172.19.0.1/16","fe80::42:d9ff:fe1a:febd/64","fe80::2831:74ff:fe70:75ec/64","fe80::3009:83ff:fee1:1956/64","fe80::4443:c2ff:fed6:4d18/64",</span

[Luke Lee]

Oh no after upgrade, my Wazuh API cannot connect. Please help. 

During the Test connection :

2001 - unidentified alias "Mypassword" at line 131, column 31: password: Mypassword ^

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","172.17.0.1/16","fe80::42:ccff:fedb:f740/64","<a href="http://172.19.0.1/16" rel="nofollow" target="_blank" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2F172.19.0.1%2F16\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH0UqP49dQypL5fDhiVL6KwbTb_Gw';return true;" onclick="this.href='http://www.go

[Elwali Karkoub]

Hello Luke,

The Wazuh-api credentials in newer versions are introduced in the file /usr/share/kibana/optimize/wazuh/config/wazuh.yml and it seems the ^ character is causing issue, Can you make the password quoted, you should have in your wazuh.yml (line 131) something like :

hosts:
  - default:
     url: https://localhost
     port: 55000
     user: wazapi
     password: 'Mypassword^'

Hope it helps,
Regards,

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["127.0.0.1/8","::1/128","10.0.106.144/24","fe80::fabc:12ff:fe3b:3ade/64","192.168.122.1/24","172.18.0.1/16","fe80::42:62ff:fe23:532f/64","<a href="http://172.17.0.1/16" rel="nofollow" target="_blank" onmo

[Luke Lee]

Thanks, now I am able to go in. But now, my Xpack still cannot work. I was wondering some alerts features requires this module? 

How to upgrade it ? Any idea

2020-03-24T15:41:55.082+0800    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-12-30T13:38:47+08:00","containerized":false,"name":"eta10","ip":["<a href="http://127.0.0.1/8" rel

[elias...@wazuh.com]

Sorry for the late response.
What do you mean X-Pack doesn't work? Are you able to connect to Kibana's user interface? It would be helpful if you could specify at what step you are encountering these inconveniences.

On the other hand, regarding Xpack, this is an Elastic Stack extension that comes installed by default along with Elasticsearch, so no further action is needed to upgrade Xpack. Also, it is worth mentioning that Xpack is not necessary for the performance of Wazuh or Elastic Stack. The main function of Xpack is to secure the Kibana APP to require authentication and encryption among the instances you had defined (Kibana, Elasticsearch, Filebeat). So it does not provide features or performance improvements in Wazuh alerts.

You may find interesting this link about X-Pack at our documentation:
https://documentation.wazuh.com/3.13/installation-guide/installing-elastic-stack/protect-installation/xpack.html#xpack-security

I hope it helps.
Regards,
Elias