pf_ring slot count

[Scott Ellis]

I am building a new server and am trying to find some documentation somewhere that will tell me how large a single pf_ring slot is...my understanding is that it acts as a buffer to processing actions. In order to size it, I know what my network traffic max capabilities are and i know how many MB/s my cpu can process, so I'd like to come up with some formula of some sort that will let me adjust and ensure the slot count is high enough, and also that I don't over size it and take up too much RAM.

Thanks

[Wes]

On Monday, December 21, 2015 at 7:01:23 PM UTC-5, Scott Ellis wrote:
> I am building a new server and am trying to find some documentation somewhere that will tell me how large a single pf_ring slot is...my understanding is that it acts as a buffer to processing actions. In order to size it, I know what my network traffic max capabilities are and i know how many MB/s my cpu can process, so I'd like to come up with some formula of some sort that will let me adjust and ensure the slot count is high enough, and also that I don't over size it and take up too much RAM.
>
> Thanks

Scott, you could try having a look here:

https://github.com/ntop/PF_RING/blob/dev/doc/UsersGuide.pdf
http://luca.ntop.org/IM2009_Tutorial.pdf

The default number of PF_RING slots is set at 4096, however for a high traffic network, you may want to increase the number of slots up to 65534 (or less, by increments of 4096) (min_num_slots in /etc/modprobe.d/pf_ring.conf), then reboot. I'm not sure exactly how large a single slot may be but, the documentation (above) suggests it could hold several packets/flows. I also have not been able to find much documentation in the way of a formula. I would try adjusting it by however much you think may be required, based on the amount of packets you may be dropping. You may want to leave it at the default, and if you do not begin dropping a lot of packets, I would not think you would need to change it (unless the amount of traffic would change, of course).

Thanks,
Wes

[Scott Ellis]

Thanks Wes. Where would I see that I am dropping packets? Would that show up on an ifconfig output?

I spent some time today digging through the internet to pf_ring try and uncover what min_num_slots means in terms of the ring size, no luck. I found one article that said "Set it to a reasonable size..." but provided no further information :(

I will likely just find it experimentally at this point, thanks for your help.

[Wes]

Scott,

You should be able to see this from the "pf_ring stats" and "Netsniff-NG" section of the sostat output for the sensor in question.

Thanks,
Wes

[Doug Burks]

netsniff-ng doesn't use PF_RING at all, so any drops in the
netsniff-ng section of sostat are unrelated to PF_RING.

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at https://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[Scott Ellis]

Doug - my understanding was that Netsniff-NG performed packet capture, but as I dug into the PF_Ring source, it became clear that PF_Ring is a packet capture tool.  Is snort using PF_Ring, and Bro using Netsniff-NG?  

Also, regarding the number of slots - Is SO setting the min_num_slots value for pf_ring?  I was looking at the code, and I am sure I must be missing something, but it looks like this value is not even being used.  SO I circle back again (no pun intended) to my original question, how big is a pf_ring and what impact does the slots count have on its size? 

here is the questionable code from the PF_Ring code base.  I could find no other instances of implementation of th emin_num_slots variable, with the exception of user stuff:

inline int pfring_there_is_pkt_available(pfring *ring) {
	#if 1
	return(ring->slots_info->tot_insert != ring->slots_info->tot_read);
	#else
	/* stronger check: */
	return ((ring->slots_info->remove_off != ring->slots_info->insert_off &&
	(ring->slots_info->tot_insert != ring->slots_info->tot_read)) ||
	((ring->slots_info->remove_off == ring->slots_info->insert_off) &&
	((ring->slots_info->tot_insert - ring->slots_info->tot_read) >= ring->slots_info->min_num_slots)));
	#endif
	}

thanks! 

--
You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/zu7U7U9pBT8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.

[Doug Burks]

Replies inline.

On Wed, Dec 23, 2015 at 4:27 PM, Scott Ellis <scor...@gmail.com> wrote:
>
> Doug - my understanding was that Netsniff-NG performed packet capture, but as I dug into the PF_Ring source, it became clear that PF_Ring is a packet capture tool.  Is snort using PF_Ring, and Bro using Netsniff-NG?  

Snort, Suricata, and Bro use PF_RING.  Netsniff-ng is totally separate and does not use PF_RING at all.

> Also, regarding the number of slots - Is SO setting the min_num_slots value for pf_ring?  

Depending on what options you choose in Setup, it may ask if you want to change the default value of 4096.

> I was looking at the code, and I am sure I must be missing something, but it looks like this value is not even being used.  SO I circle back again (no pun intended) to my original question, how big is a pf_ring and what impact does the slots count have on its size?

The default is 4096.  If the amount of traffic you're monitoring fills all available slots, then you drop packets.

[Scott Ellis]

Replies inline.

On Wednesday, December 23, 2015, Doug Burks <doug....@gmail.com> wrote:

Replies inline.

On Wed, Dec 23, 2015 at 4:27 PM, Scott Ellis <scor...@gmail.com> wrote: 
>
> Doug - my understanding was that Netsniff-NG performed packet capture, but as I dug into the PF_Ring source, it became clear that PF_Ring is a packet capture tool.  Is snort using PF_Ring, and Bro using Netsniff-NG?  

Snort, Suricata, and Bro use PF_RING.  Netsniff-ng is totally separate and does not use PF_RING at all.



> Also, regarding the number of slots - Is SO setting the min_num_slots value for pf_ring?  

  

So then netsniff's only job in life is to provide packet capture storage in each sensor, and then for real time analysis, pf_ring is the ring of choice? (True or false?)

 

Depending on what options you choose in Setup, it may ask if you want to change the default value of 4096.

Understood. But how "big", in bytes, is a single slot?  This is important. I don't want to pllay whack-a-mole with my production servers :) 

> I was looking at the code, and I am sure I must be missing something, but it looks like this value is not even being used.  SO I circle back again (no pun intended) to my original question, how big is a pf_ring and what impact does the slots count have on its size?

The default is 4096.  If the amount of traffic you're monitoring fills all available slots, then you drop packets.

 How big is the buffer in bytes is my final question.

 

http://securityonionsolutions.com -- 

[Chris White]

Hey Scott,

I'll take a stab at answering and maybe filling in some gaps, hopefully I don't mis-speak, if I do please correct.

The answer to your T/F is True.

"How big is the buffer in bytes is my final question."

See the description in pf_ring tuning below.

Tool: Bro IDS
Purpose: Protocol Detection, Logging, Alerting, etc
Packet Source: pf_ring
LB conf: /opt/bro/etc/node.cfg
Potential loss from source to tool:
- at the tap/span
- at the aggregation point
- at the sensor interface
- at pf_ring
- at bro
Note: Bro has a capture_loss.log which will show you a measurement of desyncs, this is a good indicator of loss upstream of Bro itself.

Tool: Snort/Suricata
Purpose: Alert IDS
Packet Source: pf_ring
LB conf: /etc/nsm/<sensorname>/sensor.conf
Potential loss from source to tool:
- at the tap/span
- at the aggregation point
- at the sensor interface
- at pf_ring
- at snort/Suricata

Tool: Netsniff_ng
Purpose: Full packet capture
Packet Source: no pf_ring
LB conf: n/a
Potential loss from source to tool:
- at the tap/span
- at the aggregation point
- at the sensor interface
- at Netsniff ng

PF_Ring Tuning:
Config: min_num_slots
Default: 4096
Recommended: 65534
Location: /etc/modprobe.d/pf_ring.conf
Description: a slot is used to store a packet, the size of a slot is defined by the caplen. The total potential size of the ring is caplen*min_num_slots, actual ring size is caplen*used slots. Lastly your total pf_ring memory footprint is the sum of each ring's cap*used. With a standard MTU and default min_num_slots that is a max memory footprint of 1518*4096=6,217,728 bytes per ring instance. Note if you're doing jumbo frames in any feed and you want actual FPC, be sure to adjust the MTU on the nic and each capture/reading tool and realize your ring size can change significantly (ex. 9000*4096=36,864,000 bytes or 9000*65543=589MB). Lastly, a slot should really only be used when your tool is overwhelmed as it is a buffer mechanism.

[Scott Ellis]

This is a very good answer, thank you.  Just a couple more questions:

1.  I'm not sure that the PF_Ring acts as an overflow.  My understanding is that each instance of snort gets its own ring and that it reads packets from the ring.  

2.  Where did you get "1518" from, and is that in bits or bytes?

3. Why are there two tools being used for packet capture?  Why don't Bro and Snort just use Netsniff_NG?  I imagine it has either to do with load or extensibility....?  

Thank you all so much, and hope everyone is having happy holidays!  

s

[Chris White]

1) The rings are dedicated to the snort or bro instance, but essentially the ring is a queue for the tool to grab packets. By having plenty of available slots in the ring, by increasing that value, you increase the number of packets that can be held in the ring waiting for the tool to take the next packet. I'll use a poor analogy. The ring is like a cup with a hole in the bottom. It isn't a tube because the tool represented by the hole has to do work on the packets flowing through and it simply can't do that at the rate that you can fill the cup. The buffering comes in when you increase the slot count, effectively replacing the cup with a bucket. You have the same size hole but the bucket can hold much more during the ebb and flow of bursty network traffic. When too many packets get tossed in the bucket it overflows and those packets are dropped.

2) The standard MTU or Max Transmission Unit was historically set to a payload of 1500 bytes, add 14 bytes for the Ethernet header and 4 byte FCS and you get a frame size of 1518. This is the default of most if not all O/S's. There are other headers that would effect that value of course, vlan tagging being a common example puts the frame at 1522.

3) The simple answer is there aren't two tools doing packet capture, there is only one tool for packet capture, netsniff-ng. There are multiple tools to analyze packets, in this case bro and snort. I think you're confusing netsniff-ng for being at the same layer as pf_ring. pf_ring is an alternative to libpcap, the old defacto promiscuous nic driver. pf_ring provides crazy better performance and the ability to use multiple instances. netsniff-ng on the other hand is a tool just like snort and bro. Instead of using pf_ring it uses a zero-copy driver. It's purpose is much simpler in SO's use, it just wants to write packets to disk. Since it doesn't need to do tons of cpu intensive operations the work necessary to bring those packets to user land is unnecessary, it only needs to get from nic to disk. It's driver is optimized to make that travel distance as direct as possible, avoiding expensive computational gates, freeing up those resources for the things that need it, i.e. bro and snort. This should be enough to answer the question, bro and snort don't use netsniff-ng because it isn't a driver, and because the copy of packets used by netsniff-ng never get to user land where bro and snort need it to do their work.

I'll add one more performance tip I like to call out often as I think it doesn't get the ink it deserves. Changing snorts pattern loading strategy from ac-split to ac can drastically increase Snorts performance. The default setting balances memory and performance, the ac option favors performance at the cost of memory by loading all rule patterns into memory, preventing snort from needing to call up patterns from slow disk while processing packets. The warning here is that memory usage can get high. I run 36,000 snort rules and each instance of snort consumes 7.6GB.

Finally I'll end with one of my favorite quotes:

Premature optimization is the root of all evil. (D. Knuth)

It is very easy to over engineer and over optimize, when you stop dropping an unacceptable amount of packets stop optimizing :-).

Thanks,
Chris White

--

[Rhydham Joshi]

Hi, 

For your suggestion on increasing Snort's performance: "Changing snorts pattern loading strategy from ac-split to ac can drastically increase Snorts performance", can you please provide the steps on how to do that?

Thanks,

Rhythm

To unsubscribe from this group and all its topics, send an email to securit...@googlegroups.com.