RC1 and X-Pack

[Peter Keenan]

Has anyone been able to get x-pack working with RC1? I had it going with Beta 3, by adding "xpack.security.enabled: false" to /etc/elasticsearch/elasticsearch.yaml but now when I enable it it just crashes and reloads the docker without the module.

[Wes Lambert]

Peter,

I haven't tried it (installing X-Pack over Security Onion RC1), but you may want to look at the following to see if it helps at all:

https://github.com/Security-Onion-Solutions/security-onion/wiki/Kibana#plugins

Thanks,

Wes

On Mon, Feb 5, 2018 at 5:00 PM, Peter Keenan <pe...@keenan.net> wrote:

Has anyone been able to get x-pack working with RC1? I had it going with Beta 3, by adding "xpack.security.enabled: false" to /etc/elasticsearch/elasticsearch.yaml but now when I enable it it just crashes and reloads the docker without the module.

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

[Josh Silvestro]

Peter,

I'd be interested to know if you get this to work. I know I had tried in past beta releases and was unsuccessful.

[Peter Keenan]

I did get it working for the free license steps below

vi /etc/elasticsearch/elasticsearch.yml
Add "xpack.security.enabled: false"
Add "xpack.license.self_generated.type: basic"
vi /etc/kibana/kibana.yml
Add "xpack.security.enabled: false"
sudo docker ps
docker exec -it ELASTICSEARCHCONTAINERID /bin/bash
bin/elasticsearch-plugin install x-pack
bin/elasticsearch-plugin list (and it should respond "x-pack")
exit
sudo docker ps
docker exec -it KIBANACONTAINERID /bin/bash
cd /usr/share/kibana
bin/kibana-plugin install x-pack
(This takes a while, its a large file and the Optimizing and caching thing takes forever)when done you should get "Plugin installation complete""
bash-4.2$ bin/kibana-plugin list (and it should respond "x-p...@6.1.2")

docker restart so-elasticsearch
(wait 60 seconds)
docker restart so-kibana

Done!

[Josh Silvestro]

Peter that worked great, thanks! Yea, I'll be using the free license as well. Would be great to find an alternative for machine learning though. 

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---

You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/UCep_oQ6pjo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onion+unsubscribe@googlegroups.com.

To post to this group, send email to security-onion@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

--

Thank You,

Joshua Silvestro

[Peter Keenan]

Glad it worked!

If you email them at Elastic and explain it's for test/development they will give you a trial license that you can run ML on.

What kind of ML Jobs are you thinking about? I was trying to build a job that looked at bytes transfer, pr packets sent by src IP or the classic number of DNS requests to look for individual anomalies or anomolies vs. the fleet. but I apparently suck at ML and can't get it working.

I use a top talkers (Bytes, Packets, firewall rejects, IDS/AV events) to manually find trouble so an ML job that did the same would be really interesting.

I.E john is a top talker but he is three or 4 times a week so no flag, Paul has never been before and he is now so raise a flag, or dest A is spotify always a top talker so ignore etc..

[Doug Burks]

Hi Peter,

Have you considered implementing this in ElastAlert, which is already
included in Security Onion? You might be able to do top talkers and
possibly combine that with new_term to look for "new" top talkers.
Just a thought.

> --
> Follow Security Onion on Twitter!
> https://twitter.com/securityonion
> ---

> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.

> Visit this group at https://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--

Doug Burks

[Josh Silvestro]

Doug,

I have been using elastalert and have about 20 alerts at this point. I'm only fascinated with the ML for some additional insight and the fact I see quite a few "cookbooks" out there that I can't seem to get to work for elastalert.

[Josh Silvestro]

I'm sure things like this can be done in elastalert, but currently outside my skillset, still learning.

https://www.elastic.co/blog/elasticsearch-and-siem-implementing-host-portscan-detection

[Josh Silvestro]

Actually messing around with the cardinality I got close, but it ended up firing on EVERYTHING 100 alerts, and shut that down quick.

*snippet from my yaml*

type: cardinality
timeframe:
minutes: 10
cardinality_field: "destination_port"
max_cardinality: 20
query_key: "destination_ip"
filter:
- term:
event_type: "bro_conn"

Not sure what I need to change.

[Peter Keenan]

Is there a good learning source for Elastialert? I searched and found the same link @Josh posted and had the same reaction he did, it looked like the most convoluted way of detecting a port scan I have ever seen. anyone know if there is something more practical out there with the standard stuff? 5 failed logons followed by a success, hits against threat intel list, AV/IPS hits, etc..

@Josh - can you give us some insight into what alerts you came up with?

[Philip Robson]

@josh. 

If this for external port scans you would need to filter on source IP ignoring any internal IPs.

I've never used yaml but this is my experience from setting up rules in solarwinds.

On 10 Feb 2018 14:07, "Peter Keenan" <pe...@keenan.net> wrote:

Is there a good learning source for Elastialert? I searched and found the same link @Josh posted and had the same reaction he did, it looked like the most convoluted way of detecting a port scan I have ever seen. anyone know if  there is something more practical out there with the standard stuff? 5 failed logons followed by a success, hits against threat intel list, AV/IPS hits, etc..

@Josh - can you give us some insight into what alerts you came up with?

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.

[Josh Silvestro]

The best learning source for Elastalert, to my knowledge, is their guide:

http://elastalert.readthedocs.io/en/latest/recipes/writing_filters.html

http://elastalert.readthedocs.io/en/latest/ruletypes.html

I understand the filtering and exclusion of IPs, but in this case I'd like to also detect internal hosts. Had a client host get infected a few months back and it was scanning the network slowly and wasn't really picked up by any alerting. Reviewing logs you could tell, but again they were smart and didn't trigger an alert.

[Peter Keenan]

I just rebooted my RC1 server and X-pack did not survive the reboot. Is that expected behavior that plugins would get wiped by a reboot? Josh have you rebooted since you loaded x-pack?

[Josh Silvestro]

Peter - a restart of the elastic stack may have pulled down newer docker images overwriting changes. Also, I'm 99% sure the dockers don't have persistence which would wipe those changes. After weighing the cost benefits, I'm just going to stick with elastalert . 

On Thu, Feb 15, 2018 at 9:11 AM, Peter Keenan <pe...@keenan.net> wrote:

I just rebooted my RC1 server and X-pack did not survive the reboot. Is that expected behavior that plugins would get wiped by a reboot? Josh have you rebooted since you loaded x-pack?

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---

You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/UCep_oQ6pjo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onion+unsubscribe@googlegroups.com.

To post to this group, send email to security-onion@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

[Yylio]

Hi Peter / All.

I have installed X-Pack via this method and its working great. thanks.

Is there a way, or what is the best way to have X-Pack survive a server reboot?

------------
Is there a way to hard code the Container IDs or Docker info so it doesn´t affect X-pack after a reboot? (would that even work?)

or is it better to write a script to uninstall - reboot - reinstall?


Any advice would be much appreciated!

Kyle

[Wes Lambert]

You may want to see:

https://github.com/Security-Onion-Solutions/security-onion/wiki/Kibana#plugins

Note that we don't officially support X-Pack, however,

Thanks,

Wes

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.

Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

--

https://twitter.com/therealwlambert

https://securityonion.net/

[George Frazier]

Hi Wes,

What is the supported/recommended method for exporting csv from Kibana Discovery.

Thanks,
George

[Gelli Ravikumar]

On Monday, 5 February 2018 16:00:11 UTC-6, Peter Keenan wrote:
> Has anyone been able to get x-pack working with RC1? I had it going with Beta 3, by adding "xpack.security.enabled: false" to /etc/elasticsearch/elasticsearch.yaml but now when I enable it it just crashes and reloads the docker without the module.

Hello Peter and All,

I try to follow the steps by peter for x-pack installation. But, when I try to install the x-pack through docker, I have come across "x-pack is not available with the oss distribution to use x-pack features use the default distribution." I would appreciate if you can suggest me how to resolve this.

Thanks and Best regards,
Gelli

[nick...@iastate.edu]

I am also having the same issue when attempting to install x-packs with the basic license in security onion. See attached file.

[Doug Burks]

Hi Nicklaus,

As the error message says, you can't install the x-pack plugin in the current oss distribution.  We hope to have a supported method of using x-pack (now called Elastic Stack Features) in the near future.  Stay tuned for more information!

On Thu, Apr 4, 2019 at 5:43 PM <nick...@iastate.edu> wrote:

I am also having the same issue when attempting to install x-packs with the basic license in security onion. See attached file.

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.

Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

--

Doug Burks
CEO
Security Onion Solutions, LLC

[nick...@iastate.edu]

I am disappointed to hear that but thank you for responding. Best of luck with Elastic Stack Features; I look forward to deploying the hopefully soon. Take care.