Configure Puppet master as a universal caching proxy server

[Patryk Bęza]

I'm Puppet's new user and I have a simple question regarding Puppet design: is it possible to configure Puppet master as a universal, OS agnostic caching proxy server for packages served for clients? I know that some GNU/Linux distributions have such proxies – eg. Debian has apt-cacher.

Let me explain my point of view: for simplicity's sake, let's assume that:

• I have set of N packages: {P[1], P[2], ..., P[N]} that I want to be installed on group of K clients {C[1], C[2], ..., C[K]}.
  
• All of the K clients have identical instruction set architecture (ISA) and the same GNU/Linux version installed, so that all clients need exactly the same packages (ignore packages dependencies – they can be downloaded by clients if needed).
  
• All of the K clients have none of the N packages installed.
  

From my understanding, after client update poll, every of the N clients will download K identical packages (ignore their dependencies). Isn't it potentially huge waste of network bandwidth? N*K downloads instead K packages downloads. Wouldn't be sane to download packages on server machine and than multicast them to clients?

[James Pryor]

Hi,

> is it possible to configure Puppet master as a universal, OS agnostic caching proxy server for packages served for clients?

To my knowledge No. Puppet is not a server method of unicasting or multicasting binary software packages to clients. That is left to a package repository server and it's clients.

Isn't this statement:

    >"From my understanding, after client update poll, every of the N clients will download K identical
    >packages (ignore their dependencies). Isn't it potentially huge waste of network bandwidth? N*K
  >downloads instead K packages downloads. Wouldn't be sane to download packages on server
    >machine and than multicast them to clients?"

pretty much orthogonal to what puppet is used for and how puppet works?

Any package download (unicast, multicast, push, or pull) is typically the function of the client's package management system (apt, yum/dnf, zypper, emerge ) which may have client-side caching, and it is all sourced against a centrally managed package repository system. A centrally managed package repository system may or may not have server-side caching and may or may not allow for multicasting the packages out to clients.

And none of that involves puppet. I might get this wrong as I am going directly from memory, but the puppet master delivers a catalog, a set of instructions, to a client so that the client can reach a desired configuration state. The puppet master can send files too, but those files are typically plain-text files or templates, and are not the software packages. Those come typically come from a centrally managed package repository system running on a server.

> "Isn't it potentially huge waste of network bandwidth?"

Yes it can be. A centrally managed package repository system running on a server (or tiers of servers) often needs large amounts of bandwidth, especially in cases of thundering-herd problems.

I hope this helps.

James

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/38909e38-8819-4b2e-97cf-242ae9c255e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Gelnaw]

On Monday, March 13, 2017 at 10:23:28 AM UTC-4, Patryk Bęza wrote:

I'm Puppet's new user and I have a simple question regarding Puppet design: is it possible to configure Puppet master as a universal, OS agnostic caching proxy server for packages served for clients? I know that some GNU/Linux distributions have such proxies – eg. Debian has apt-cacher.

No.  However, squid can be configured to do such, and I use it as a proxy for rpm/deb downloads.  I'm even running it on the same server as my puppet master, because the puppet server is the one box our firewalled systems are allowed to talk to.