FreqServer and Domainstats in a standalone environment

[Ben]

Hello!

I have enabled FreqServer and DomainStats in my standalone test environment and from what I see in sostat everything looks fine, but all related Kibana dashboards are empty. Ideas? Or just not possible in a standalone setting? Thanks in advance!

Cheers, Ben.

[Wes Lambert]

You may want to check the Logstash log to see if the pipeline is getting blocked.

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/7705d452-fdb2-4de6-8fe5-266319c0cea6n%40googlegroups.com.

--

https://twitter.com/therealwlambert

https://securityonion.net/

[Ben]

Everything looks fine ...

[Wes Lambert]

Do you actually have data in Elasticsearch, regardless of if you can see if from within Kibana?

curl -s localhost:9200/_cat/indices 

Thanks,

Wes

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/b7d9db06-3674-4d4a-82be-3196f20c640dn%40googlegroups.com.

[Ben]

There is a lot of data comming in:

However, nothing that points to FreqServer or Domainstats if you are looking for these kind of entries. Do I need specific open ports on my perimeter firewall to the Internet?

[Ben]

Dear all! Any other ideas? Thanks in advance! Cheers, Ben!

[Wes Lambert]

Where have you enabled Domainstats and Freqserver?  On what node?

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/fbbdebba-5e35-4b8a-9e6e-696e5b2c30b8n%40googlegroups.com.

[Ben]

Hi Wes, everything is running on one server (just as a proof of concept ...). Cheers, Ben

[Ben]

Does anybody has an idea what I could try to get  FreqServer and Domainstats working? Thanks in advance! 

[Wes Lambert]

Have you tried using docker logs to check the containers themselves?

Thanks,

Wes

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/2d884093-c2a8-4abd-8a89-cadac6f76317n%40googlegroups.com.

[edgar...@gmail.com]

 Enable on the /etc/nsm/securityonion.conf file (or something like that)

Do a soup to download the latest containers

so-status to verify they are running

Program the daily cron job, with a caveat, there is a change on the 1m list from talos, need to change that.

Also, on the cron job, be sure to verify the command that is a one-liner, even ,try it outside of chron first to seed the list.

Make sure that your port 43 is open, whois needs to work.

That should be it.

Edgar

[Ben]

Hello Edgar,

I did the update and according to so-status the containers are up and running. However, both did only receive 12 packets on eth1, that's it. No Docker logs, no logs in /var/log/freq_server or domain_stats (port 43 is open for my SO host running freq_server and domain_stats and whois requests are working). What cron job are you referring to?

Cheers, Ben

[Edgar M. Toro]

There is a daily cron job that downloads a csv file from azure / talos witn a 1m baby urls that is what feeds the domain stats.

Check the so docs.

Like I said, the docs still refer to an old url that has to be corrected.

-------------------

Edgar M. Toro

You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/0ISXMXJmXDU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/72109ba0-8d4c-48aa-a579-ba1788ebb147n%40googlegroups.com.

[Edgar M. Toro]

https://docs.securityonion.net/en/latest/domainstats.html

Its there.

This is the cron.daily

#/etc/cron.d/domainstats
#
#crontab entry to grab new Top 1m CSV for DomainStats Docker image
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/localbin:/sbin:/bin/usr/sbin:/usr/bin
1 07 * * *   root ( wget -q http://s3.amazonaws.com/alexa-static/top-1m.csv.zip -O /tmp/top-1m.csv.zip && unzip -o
/tmp/top-1m.csv.zip -d /tmp && docker cp /tmp/top-1m.csv so-domainstats:/opt/domain_stats/top-1m.csv && docker restart
so-domainstats && rm -f /tmp/top-1m.csv* ) > /dev/null 2>&1

-------------------

Edgar M. Toro

You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/0ISXMXJmXDU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/72109ba0-8d4c-48aa-a579-ba1788ebb147n%40googlegroups.com.

[Ben]

Thanks Edgar! Successfully ran the cron job (manually ...), but domain_stats and freq_server seems to be stuck somewhere ...