Puppet Packet Rate?

[Peter Berghold]

Has anybody out there done any sort of study on what Puppet produces in terms of I/O packet rate?  I'm being asked to fill in a spreadsheet with that information....

[Christopher Wood]

The answer is that it depends on your catalogs and what you're ramming through them. You might get a squinty rule of thumb measurement from figuring out the size of uploaded facts/report, downloaded catalog/files, add some more for random discussion traffic, divide by the number of seconds an agent run allegedly takes.

That said, you sound like you're living an even more corporate life than I am, and they might not take an answer of "it depends what you make puppet do".

You might also do a packet capture or some hypervisor i/o monitoring during an agent run, see how much that is and divide by the agent run time.

And then your numbers may turn out fictional when somebody decides they're going to ram tarballs in via file resource.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [1]puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> [2]https://groups.google.com/d/msgid/puppet-users/CAArvnv3G8Lbe8GL11SGC251YYn-SjUs-KEPR2NfDjKGqzgZhEw%40mail.gmail.com.
> For more options, visit [3]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:puppet-users...@googlegroups.com
> 2. https://groups.google.com/d/msgid/puppet-users/CAArvnv3G8Lbe8GL11SGC251YYn-SjUs-KEPR2NfDjKGqzgZhEw%40mail.gmail.com?utm_medium=email&utm_source=footer
> 3. https://groups.google.com/d/optout

[Ramin K]

I can share some rough numbers. We do a lot of files so the numbers are
weird and inefficient. Puppet 3.8.7/Passenger 5.1.5/Ruby 2.0/Apache 2.4
on the server side.

avg file metadata per puppetserver: 25k/s
avg hosts per puppetserver: 1200

avg catalog size: 15MB (lots and lots of files)

Peak TX rate: 4MB/s
Peak RX rate: 1.5MB/s

I'd suspect the average Puppetserver to be a fraction of that bandwidth
for the same number of hosts.

Ramin

[Soham Chakraborty]

Ramin,

I am interested to know how did you do the tests? I want to run a simulation myself.

[Ramin K]

Nothing fancy here. Parse the Apache log for /catalog/ to get catalog
sizes though you may need to tweak what you're logging, I'm using the
following, but only %b is needed to get bytes of the request.
http://httpd.apache.org/docs/current/mod/mod_log_config.html

LogFormat "%v %V %{X-Forwarded-For}i %l %u %t \"%r\" %>s %b
\"%{Referer}i\" \"%{User-Agent}i\" %D" combined
CustomLog /var/log/httpd/<%= @vhostname %>-access_log combined

File service metadata requests is parsing for "GET
/$some_env/file_metadata/". BTW in my stats that was 23k req/s if that
wasn't clear.

tx/rx is just from the interface. Nothing else on the box so it's all
Puppet plus general system stats collection, etc.

Ramin

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/61e5dc0f-a791-4eb9-89e8-714014404cb1%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/61e5dc0f-a791-4eb9-89e8-714014404cb1%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.