packet loss

471 views
Skip to first unread message

packetsmacker

unread,
Nov 9, 2017, 3:20:25 PM11/9/17
to security-onion
Looking at the snort statistics tab in sguil I see most all of the interfaces have packet loss. They range from 2% to 75%. I have a lot of rules and I am trying to disable them but other then that am not sure where to look. I have looked at pf_ring settings but I am not sure what to change it to.

I have this for my pf_ring.conf.
options pf_ring transparent_mode=0 min_num_slots=4096


Not sure where to go from here.

Wes Lambert

unread,
Nov 9, 2017, 4:21:52 PM11/9/17
to securit...@googlegroups.com
You'll want to take a look at your sostat output to see if it is actually the IDS itself dropping packets, or PF_RING.

If it appears to be more PF_RING related, then you could try going through some of the steps here:


Otherwise, you may also want to start taking a look at drops at the NIC itself to see if is causing the IDS to have a harder time processing/analyzing packets.

Thanks,
Wes


--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

packetsmacker

unread,
Nov 10, 2017, 5:00:32 PM11/10/17
to security-onion
On Thursday, November 9, 2017 at 4:21:52 PM UTC-5, Wes wrote:
> You'll want to take a look at your sostat output to see if it is actually the IDS itself dropping packets, or PF_RING.
>
>
> If it appears to be more PF_RING related, then you could try going through some of the steps here:
>
>
> https://github.com/Security-Onion-Solutions/security-onion/wiki/PF_RING
>
>
> Otherwise, you may also want to start taking a look at drops at the NIC itself to see if is causing the IDS to have a harder time processing/analyzing packets.
>
>
>
> Thanks,
> Wes
>
>
> On Thu, Nov 9, 2017 at 3:20 PM, packetsmacker <ott....@gmail.com> wrote:
> Looking at the snort statistics tab in sguil I see most all of the interfaces have packet loss. They range from 2% to 75%. I have a lot of rules and I am trying to disable them but other then that am not sure where to look. I have looked at pf_ring settings but I am not sure what to change it to.
>
>
>
> I have this for my pf_ring.conf.
>
> options pf_ring transparent_mode=0 min_num_slots=4096
>
>
>
>
>
> Not sure where to go from here.
>
>
>
> --
>
> Follow Security Onion on Twitter!
>
> https://twitter.com/securityonion
>
> ---
>
> You received this message because you are subscribed to the Google Groups "security-onion" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
>
> To post to this group, send email to securit...@googlegroups.com.

>
> Visit this group at https://groups.google.com/group/security-onion.
>
> For more options, visit https://groups.google.com/d/optout.

Here is part of the sostat.


=========================================================================
Packets received during last monitoring interval (600 seconds)
=========================================================================

eth3: 140323627

=========================================================================
Packet Loss Stats
=========================================================================

NIC:

eth3:

RX packets:149384126564 dropped:0 TX packets:0 dropped:0

-------------------------------------------------------------------------

pf_ring:

Appl. Name : bro-eth3
Tot Packets : 14297243681
Tot Pkt Lost : 125178039


Appl. Name : bro-eth3
Tot Packets : 10899320747
Tot Pkt Lost : 159882181


Appl. Name : bro-eth3
Tot Packets : 11963336505
Tot Pkt Lost : 144211671


Appl. Name : bro-eth3
Tot Packets : 14930500274
Tot Pkt Lost : 14660252


Appl. Name : bro-eth3
Tot Packets : 14196692698
Tot Pkt Lost : 118210305


Appl. Name : bro-eth3
Tot Packets : 8511677265
Tot Pkt Lost : 12087324


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 516242071
Tot Pkt Lost : 133342329


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 370428861
Tot Pkt Lost : 23572166


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 606359502
Tot Pkt Lost : 257447136


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 1224008058
Tot Pkt Lost : 268491840


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 1042296750
Tot Pkt Lost : 483009081


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 634764867
Tot Pkt Lost : 33946881


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 541830153
Tot Pkt Lost : 28791480


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 511643680
Tot Pkt Lost : 20670602


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 356231541
Tot Pkt Lost : 57386105


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 581819495
Tot Pkt Lost : 35849649


Appl. Name : snort-cluster-54-socket-0
Tot Packets : 2699965098
Tot Pkt Lost : 1295727701

-------------------------------------------------------------------------

IDS Engine (snort) packet drops:

/nsm/sensor_data/SO-server-eth3/snort-10.stats last reported pkt_drop_percent as 5.190
/nsm/sensor_data/SO-server-eth3/snort-11.stats last reported pkt_drop_percent as 11.752
/nsm/sensor_data/SO-server-eth3/snort-1.stats last reported pkt_drop_percent as 30.975
/nsm/sensor_data/SO-server-eth3/snort-2.stats last reported pkt_drop_percent as 21.113
/nsm/sensor_data/SO-server-eth3/snort-3.stats last reported pkt_drop_percent as 62.719
/nsm/sensor_data/SO-server-eth3/snort-4.stats last reported pkt_drop_percent as 14.706
/nsm/sensor_data/SO-server-eth3/snort-5.stats last reported pkt_drop_percent as 76.196
/nsm/sensor_data/SO-server-eth3/snort-6.stats last reported pkt_drop_percent as 19.894
/nsm/sensor_data/SO-server-eth3/snort-7.stats last reported pkt_drop_percent as 9.429
/nsm/sensor_data/SO-server-eth3/snort-8.stats last reported pkt_drop_percent as 11.014
/nsm/sensor_data/SO-server-eth3/snort-9.stats last reported pkt_drop_percent as 13.543
-------------------------------------------------------------------------

Bro:

Average packet loss as percent across all Bro workers: 4.073341

SO-server-eth3-1: 1510341755.896128 recvd=1193794892 dropped=118210305 link=1193794892
SO-server-eth3-2: 1510341756.059889 recvd=3229563913 dropped=144211671 link=3229563913
SO-server-eth3-3: 1510341756.300011 recvd=1287444350 dropped=125178039 link=1287444350
SO-server-eth3-4: 1510341756.325784 recvd=2031573617 dropped=14660252 link=2031573617
SO-server-eth3-5: 1510341756.661793 recvd=2149904059 dropped=159882181 link=2149904059
SO-server-eth3-6: 1510341756.861916 recvd=4204987786 dropped=12087324 link=4204987786

Capture Loss:

SO-server-eth3-1 3.380711
SO-server-eth3-1 4.645657
SO-server-eth3-2 1.994477
SO-server-eth3-2 3.036864
SO-server-eth3-3 3.982415
SO-server-eth3-3 6.190086
SO-server-eth3-4 2.666092
SO-server-eth3-4 3.981761
SO-server-eth3-5 1.780005
SO-server-eth3-5 1.828798
SO-server-eth3-6 2.759352
SO-server-eth3-6 4.134621

If you are seeing capture loss without dropped packets, this
may indicate that an upstream device is dropping packets (tap or SPAN port).

-------------------------------------------------------------------------

Netsniff-NG:


Percentage of packets dropped:

/var/log/nsm/SO-server-eth3/netsniff-ng.log --


=========================================================================
PF_RING
=========================================================================
PF_RING Version : 6.6.0 (unknown)
Total rings : 17

Standard (non ZC) Options
Ring slots : 4096
Slot version : 16
Capture TX : Yes [RX+TX]
IP Defragment : No
Socket Mode : Standard
Cluster Fragment Queue : 118
Cluster Fragment Discard : 0

If I am reading this right I would say its the IDS. Is that correct? My load has been down. It's below 24 right now. I would like to think it due to me disabling rules. I can add some more processing units to see if that helps.

packetsmacker

unread,
Nov 10, 2017, 5:21:58 PM11/10/17
to security-onion

Maybe I am confused but I feel like i have adjusted the processing units before. If I did I can't remember how. I might be getting that confused with the number IDS process.

Wes Lambert

unread,
Nov 11, 2017, 8:07:43 AM11/11/17
to securit...@googlegroups.com
You may want to take a look at the following to get an idea of how to adjust the worker processes:


After tuning, if you don't feel as though you still aren't getting the performance you're looking for, try increasing the amount of RAM/CPU, or switch to Suricata and see if that works better for you.

Thanks,
Wes

To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages