RC1 Sensor node errors

[mdi...@gmail.com]

I'm seeing the following errors on my sensor node after a clean install of RC1:

Strelka-frontend, container not running, errors in stopped pod:

failed to create file /var/log/strelka/strelka.log: open /var/log/strelka/strelka.log: permission denied

strelka-filestream, container not running, errors in stopped pod:

2020/07/21 18:11:42 failed to connect to 10.85.190.49:57314: context deadline exceeded

sensoroni, container not running, errors in stopped pod:

2020/07/21 18:07:38 error Syntax error reading json object error=invalid character ',' looking for beginning of value offset=114

Error: Unable to read configuration file 'sensoroni.json' [invalid character ',' looking for beginning of value]

Any thoughts?  Thanks!

[mdi...@gmail.com]

Also having issues with Zeek - it's not in an error state, but doing an so-zeek-stats returns saying there are no workers. Restarting the Zeek service doesn't help.  Reapplying the high state doesn't help.

##############

# Zeek Stats #

##############

Average throughput:

Interface             kpps       mbps       (10s average)

----------------------------------------

localhost/af_packet::bond0 1.0        4.0

Average packet loss:

 worker-1-1: <error: no running instances of Zeek>

telegraf on the sensor is also giving an error:

2020-07-21T18:44:30Z E! [inputs.exec]: Error in plugin: exec: exit status 1 for command '/scripts/broloss.sh': /scripts/broloss.sh: line 7: rcvd: - : syntax error in expression (error token is ": - ")...

[Wes Lambert]

HI Max,

In what kind of environment, and with what specs are you trying to install Security Onion 2.0 RC1?

Thanks,

Wes

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/e0a99e5c-d254-4eff-8389-f6c538231f11n%40googlegroups.com.

--

https://twitter.com/therealwlambert

https://securityonion.net/

[mdi...@gmail.com]

I fixed the Zeek issue by manually running zeekctl deploy inside the docker container.

[mdi...@gmail.com]

Hi Wes,

It's in a VMWare VM, 8vCPU's, 32GB of RAM, 2.7TB of total disk space, 2 vmxnet3 NIC's, one for management, one for monitoring.  1.4.1 ran flawlessly in this exact same VM container.  Just reinstalled from the ISO same way I did with the previous version. 

Max

[mdi...@gmail.com]

In Grafana on the sensor dashboard, I'm only seeing brief stats for Zeek packet loss, and not showing any data for the monitoring NIC.

[mdi...@gmail.com]

I re-ran the setup on the sensor node - again no errors, but again, errors in sensoroni, Strelka file stream and Strelka front end.

Again, Zeek-stats complaining about no running instances.

[mdi...@gmail.com]

Sorry for the stream of consciousness, looking in SOC under Sensors, nothing is listed there.

[mdi...@gmail.com]

Not a single thing on the page about removing sensors is accurate.  I was going to wipe and reload the node but...  https://docs.securityonion.net/en/2.0/removing-a-sensor.html?highlight=remove%20sensor

[Doug Burks]

Right, not all of the documentation pages have been updated for 2.0 yet. We’ll update it soon. 

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/606cbd38-e91b-4e10-8527-5ee212976bd4n%40googlegroups.com.

--

Doug Burks
Founder and CEO
Security Onion Solutions, LLC

[mdi...@gmail.com]

Removed the data from the sensor node.  Removed the node from salt, removed the salt key.  Reinstalled the sensor node and it's slightly better.  All docker containers are up except for sensoroni.

telegraf is still giving: 2020-07-22T00:40:00Z E! [inputs.exec]: Error in plugin: exec: exit status 1 for command '/scripts/broloss.sh': /scripts/broloss.sh: line 7: 12716 - : syntax error: operand expected (error token is "- ")...

SOC Sensors page still shows no sensors.    However I am getting events.  PCAP's are still stuck at Pending indefinitely.  

The Grafana sensor traffic graph shows nothing.  When looking at the config, this is the data it's pulling from, which is wrong, host = hunter-sensor1 is fine, but interface=none is not:

 "query": "SELECT 8 * derivative(mean(\"bytes_recv\"),1s) FROM \"net\" WHERE \"host\" = 'JumpHost' AND \"interface\" = 'eth0' AND $timeFilter GROUP BY time($interval) fill(null)",

      "tags": [

        {

          "key": "host",

          "operator": "=",

          "value": "hunter-sensor1"

        },

        {

          "condition": "AND",

          "key": "interface",

          "operator": "=",

          "value": "None"

        }

And I'm STILL not getting any NIDS alerts populating in The Hive.  

Navigator is working though :). 

[mdi...@gmail.com]

If I look at the sensorstab.sls file, monint is blank.  Added the interface bond0 and reapplied the state, and it's working now.  

[mdi...@gmail.com]

Fixed sensoroni.  In the salt pillar, it was missing 

pcap:

  sensor_checkin_interval_ms: 10000

Replaced it and restarted sensoroni - that's working.   This also fixed the issue with the sensor not showing up in SOC, and not being able to retrieve PCAP's.  

I think this only leaves the NIDS alerts not showing up in The Hive still.  The cortex docker logs show:

[error] o.e.c.Authenticated - Authentication failure:

session: AuthenticationError User session not found

pki: AuthenticationError Certificate authentication is not configured

key: AuthenticationError Authentication failure

init: AuthenticationError Use of initial user is forbidden because users exist in database

[error] o.e.s.a.MultiAuthSrv - Authentication failure

org.elastic4play.AuthenticationError: Authentication using API key is not supported

at org.elastic4play.services.AuthSrv.authenticate(UserSrv.scala:50)

at org.elastic4play.services.AuthSrv.authenticate$(UserSrv.scala:49)

at org.thp.cortex.services.LocalAuthSrv.authenticate(LocalAuthSrv.scala:19)

at org.elastic4play.services.auth.MultiAuthSrv.$anonfun$authenticate$3(MultiAuthSrv.scala:58)

at org.elastic4play.services.auth.MultiAuthSrv$$anonfun$$nestedInanonfun$forAllAuthProvider$1$1.applyOrElse(MultiAuthSrv.scala:43)

at org.elastic4play.services.auth.MultiAuthSrv$$anonfun$$nestedInanonfun$forAllAuthProvider$1$1.applyOrElse(MultiAuthSrv.scala:41)

at scala.concurrent.Future.$anonfun$recoverWith$1(Future.scala:417)

at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)

at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)

at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)

By the way - you may want to mention that the admin password that is set during the install is stored in plain text on the master.   

[Doug Burks]

Hi Max,

Thanks for letting us know! We are working on fixes for both sensoroni and the admin password.

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/80233fa5-e1dd-4f89-a3e0-167c0b1d49b8n%40googlegroups.com.

[Doug Burks]

Hi Max,

We just published fixes for the sensoroni and admin password issues:

https://blog.securityonion.net/2020/07/security-update-for-security-onion-20.html

Thanks again for letting us know!

[mdi...@gmail.com]

Awesome.  Thanks Doug.  Upgraded and everything went very smoothly.

I'm still digging into the containers and config files to try to figure out why TheHive can't connect to Cortex.  Not getting too far though.

[Doug Burks]

Thanks, Max. Glad that your upgrade went smoothly. We've got to get another release out the door today but once that's done we'll see if we can duplicate TheHive issues.

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/efd8b353-654f-4189-a55e-207ebe20221cn%40googlegroups.com.

[mdi...@gmail.com]

Let me know if you need any more logs.  2.0 is really starting to come together.

[Doug Burks]

Hi Max,

Now that we've got 2.0.2 out the door, we're looking into TheHive/Cortex issue. We're definitely seeing it on our lab systems, so we're working on determining root cause.

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/e385a6b9-ef9c-47cf-83bc-3af09573ebf5n%40googlegroups.com.

[Doug Burks]

Hi Max,

Here are the issues we're working on for TheHive:

https://github.com/Security-Onion-Solutions/securityonion/issues/1094

https://github.com/Security-Onion-Solutions/securityonion/issues/1095

[mdi...@gmail.com]

Thanks Doug - I'm sure it got buried in the thread, but I don't see an issue for the Grafana Sensor dashboard not showing data for monitoring NIC in a distributed setup.  I traced that one down to the sensorstab.sls file, monint is blank.  Added the interface bond0 and reapplied the state, and it's working now.

[Doug Burks]

Hi Max,

I've created the following issue for sensorstab.sls monint being blank:

https://github.com/Security-Onion-Solutions/securityonion/issues/1102

To view this discussion on the web visit https://groups.google.com/d/msgid/security-onion/9ea3e89e-fc04-4898-aa0b-dedaf36d7ff9n%40googlegroups.com.