kibana cannot connect to elasticsearch and more..

[Master Yoda]

Hi, i am unable to look at data from kibana, says that he cannot connect to elasticsearch.
When i open direct elasticsearch i cannot do anything, just have json answer...

Also, i cannot see data from sguil...

This is clean install of new version SO...

[Doug Burks]

Hi Master Yoda,

Please provide a fresh copy of sostat output:

sudo sostat-redacted

There will be a lot of output, so you may need to increase your
terminal's scroll buffer OR redirect the output of the command to a
file:

sudo sostat-redacted > sostat-redacted.txt 2>&1

sostat-redacted will automatically redact any IPv4/IPv6/MAC addresses,
but there may be additional sensitive info that you still need to
redact manually.

Attach the output to your email in plain text format (.txt) OR use a
service like http://pastebin.com.

> --
> Follow Security Onion on Twitter!
> https://twitter.com/securityonion
> ---
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at https://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks

[Master Yoda]

hi, here it is:
https://pastebin.com/vKDsg99v

[Doug Burks]

It looks like you're monitoring lots of traffic. How much traffic are
you monitoring?

What kind of traffic are you monitoring? Is it encapsulated in any
way? Are you receiving traffic from a tap or span port?

You'll most likely want to disable http_agent to avoid duplication of
http logs since they're already being stored in Elasticsearch:
https://github.com/Security-Onion-Solutions/security-onion/wiki/DisablingProcesses

You may need to tune the Elastic stack to handle your traffic. Please
see the Elastic pages on our Wiki:
https://github.com/Security-Onion-Solutions/security-onion/wiki/elastic

On Tue, Sep 19, 2017 at 7:30 AM, Master Yoda <m45t3...@gmail.com> wrote:
> hi, here it is:
> https://pastebin.com/vKDsg99v
>

[Master Yoda]

it is SPAN port and MPLS data monitoring..
around 160Mbps...

can you give me more info where and what to look?

[Doug Burks]

The MPLS tagging may be related to your Sguil data issue. Have you
monitored this traffic using Security Onion previously (before the
Elastic stack)? Are you able to receive the traffic without MPLS
tags?

You'll most likely want to disable http_agent to avoid duplication of
http logs since they're already being stored in Elasticsearch:
https://github.com/Security-Onion-Solutions/security-onion/wiki/DisablingProcesses

You may need to tune the Elastic stack to handle your traffic. Please
see the Elastic pages on our Wiki:
https://github.com/Security-Onion-Solutions/security-onion/wiki/elastic

This page then links to the following:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Elasticsearch
https://github.com/Security-Onion-Solutions/security-onion/wiki/Logstash
https://github.com/Security-Onion-Solutions/security-onion/wiki/Kibana

These 3 pages have more information on tuning and log files.

[Master Yoda]

yes, seems it`s MPLS issue (before him, when we push data from vlan, data was visible), is there option to use SO with MPLS tagging?

[Doug Burks]

You can get a second opinion on the MPLS data by looking at the raw
Bro logs in /nsm/bro/logs/current/. If things look strange there,
then there might be an issue with your span or MPLS tagging. If the
Bro logs look correct, then we need to figure out why Snort alerts are
not showing the proper data for your MPLS traffic. Snort *should* be
compiled with MPLS support. You could try switching to Suricata to
see if that makes any difference:
https://github.com/Security-Onion-Solutions/security-onion/wiki/faq#im-currently-running-snort--how-do-i-switch-to-suricata

On Tue, Sep 19, 2017 at 8:40 AM, Master Yoda <m45t3...@gmail.com> wrote:
> yes, seems it`s MPLS issue (before him, when we push data from vlan, data was visible), is there option to use SO with MPLS tagging?
>

[Master Yoda]

ok, with suricata i see data, great!
now to find cause why snort ignore it...
what can be the issue with MPLS configuration?

and what to do with elasticsearch? how to connect him with kibana?

[Doug Burks]

Since Suricata is working for you, let's focus on the Elasticsearch
issue. Per my previous emails, have you reviewed the Elasticsearch
page on our wiki?
https://github.com/Security-Onion-Solutions/security-onion/wiki/Elasticsearch

Specifically, have you reviewed the logs in /var/log/elasticsearch/?

[Master Yoda]

i am, but did not found any help related to this problem? :)

[Doug Burks]

Can you attach the elasticsearch log?

On Tue, Sep 19, 2017 at 10:08 AM, Master Yoda <m45t3...@gmail.com> wrote:
> i am, but did not found any help related to this problem? :)
>

[Master Yoda]

this one?

[Wes Lambert]

Master Yoda, 

What is the output of the following?

(From localhost) curl localhost:9200

(From localhost) sudo docker exec -it so-kibana /bin/bash, then

(From container) curl elasticsearch:9200

Thanks,

Wes

On Tue, Sep 19, 2017 at 10:25 AM, Master Yoda <m45t3...@gmail.com> wrote:

this one?

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.

[Master Yoda]

1. https://pastebin.com/uThkDvPq
2. root@SecurityOnion:/etc/elasticsearch# sudo docker exec -it so-kibana /bin/bash
bash-4.2$ curl elasticsearch:9200

curl: (6) Could not resolve host: elasticsearch; Unknown error

hm, this SO box is behind proxy, maybe he need internet connection to elasticsearch work? :s

[Wes Lambert]

You shouldn't need to use a proxy to connect between the local Docker containers.

What is the output of the following?

sudo docker network inspect so-elastic-net

Thanks,

Wes

[Master Yoda]

[]
Error: No such network: so-elastic-net

[Wes Lambert]

I would imagine the following would not show so-elastic-net?

sudo docker network ls

It looks like the network for the Docker containers didn't get created.  /usr/sbin/so-elastic-start (script that starts the Docker containers uses the value of $DOCKERNET in /etc/nsm/securtyonion.conf to connect the containers to the appropriate network, so that only those containers reside on that particular network.  The network itself is created during setup when /usr/sbin/so-elastic-configure runs.

What is the output of the following?

grep DOCKERNET /etc/nsm/securityonion.conf

If there is nothing in there, then you may want to do the following (although, it may be best to try re-installing altogether):

Create the network:

sudo docker network create so-elastic-net --driver=bridge

Add network name to securityonion.conf:

Add "so-elastic-net' as the value for DOCKERNET in /etc/nsm/securityonion.conf.

Restart services:

sudo so-elastic-restart (or sudo so-elastic-stop && sudo so-elastic-start)

 

On Tue, Sep 19, 2017 at 10:49 AM, Master Yoda <m45t3...@gmail.com> wrote:

[]
Error: No such network: so-elastic-net

[Master Yoda]

that was it, issue solved by executing: sudo docker network create so-elastic-net --driver=bridge