What is the intention of thin_storeconfigs?

[Bernd Adamowicz]

I started doing some experiments with the configuration option 'thin_storeconfigs=true' by adding this option to one of my Puppet masters. However, I could not determine any change in behavior. I expected to have the resources collected faster, but Puppet still takes some 15min to do the job. So maybe I misunderstood something. Should this option instead be placed in the client's configuration to make them export only the @@-resources?

Couldn't find any documentation. So any help is appreciated.

Thanks
Bernd

[Denmat]

Hi,

Thin_storeconfig stores less detail in the database, that won't necessarily speed it up but will reduce its footprint.

The actual differences I'm a bit fuzzy on.

Den

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>

[Walter Heck]

Thin stored configs only store exported resources in the database Instead of all resources. There may be more differences though.

Sent from my galaxy tab

[jcbollinger]

On Thursday, July 12, 2012 3:29:15 AM UTC-5, badamowicz wrote:

I started doing some experiments with the configuration option 'thin_storeconfigs=true' by adding this option to one of my Puppet masters. However, I could not determine any change in behavior. I expected to have the resources collected faster, but Puppet still takes some 15min to do the job. So maybe I misunderstood something. Should this option instead be placed in the client's configuration to make them export only the @@-resources?

Couldn't find any documentation. So any help is appreciated.

Google gave me this as my number one hit: http://projects.puppetlabs.com/projects/1/wiki/Using_Stored_Configuration.  It's a bit dated, but it nevertheless contains the key information: "[thin storeconfigs] only collects and stores to the database exported resources, tags and host facts."

'Thick' storeconfigs additionally stores non-exported resources, and possibly other things -- overall, a relational representation of nodes' entire catalogs.

I would expect thin storeconfigs to accelerate resource collection relative to thick only if the overall database size were slowing the DBMS.  If the problem is specifically with the number and/or complexity of exported resources, then thin vs. thick should not make a difference in that area.


John

[jcbollinger]

On Thursday, July 12, 2012 3:29:15 AM UTC-5, badamowicz wrote:

Should [thin_storeconfigs] instead be placed in the client's configuration to make them export only the @@-resources?

No, (thin_)storeconfigs is a function of the master, at catalog compilation time.

John

[Bernd Adamowicz]

Thanks to all who gave feedback on this topic. I will have a deeper look at this issue throughout the next days and weeks and will share my experience then.

Bernd

> -----Ursprüngliche Nachricht-----
> Von: Bernd Adamowicz
> Gesendet: Donnerstag, 12. Juli 2012 10:29
> An: 'puppet...@googlegroups.com'
> Betreff: What is the intention of thin_storeconfigs?

[Brice Figureau]

On 12/07/12 10:29, Bernd Adamowicz wrote:
> I started doing some experiments with the configuration option
> 'thin_storeconfigs=true' by adding this option to one of my Puppet
> masters. However, I could not determine any change in behavior.

As others already have explained, with thin_storeconfigs, only exported
resources, facts and nodes are persisted to the DB. With regular (thick)
storeconfigs every resources are persisted to the database.

> I expected to have the resources collected faster, but Puppet still
> takes some 15min to do the job.

The thing is that if you had run with regular storeconfigs before
activating the thin_storeconfigs option, then your database is already
populated with all the resources definitions and parameters. So the
first time you run with thin_storeconfigs you end up collecting as if
thick was activated, then after the first catalog run (for a given
node), puppet should remove all the un-needed resources (ie the non
exported ones) from the database.
If that doesn't happen, I would suggest you to cleanup the database for
your nodes so that only exported resources are persisted and collected.

> So maybe I misunderstood something.
> Should this option instead be placed in the client's configuration to
> make them export only the @@-resources?

No, it's a master option.
--
Brice Figureau
My Blog: http://www.masterzen.fr/

[Bernd Adamowicz]

Brice,

Thanks for this detailed answer. As mentioned, I will try this throughout the next weeks (will not find the time before.)

Thanks
Bernd

> -----Ursprüngliche Nachricht-----
> Von: puppet...@googlegroups.com [mailto:puppet-
> us...@googlegroups.com] Im Auftrag von Brice Figureau
> Gesendet: Freitag, 13. Juli 2012 11:08
> An: puppet...@googlegroups.com
> Betreff: Re: [Puppet Users] What is the intention of thin_storeconfigs?

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-

> users+un...@googlegroups.com.

[Daniel Pittman]

On Fri, Jul 13, 2012 at 2:08 AM, Brice Figureau
<brice-...@daysofwonder.com> wrote:
> On 12/07/12 10:29, Bernd Adamowicz wrote:
>> I started doing some experiments with the configuration option
>> 'thin_storeconfigs=true' by adding this option to one of my Puppet
>> masters. However, I could not determine any change in behavior.
>
> As others already have explained, with thin_storeconfigs, only exported
> resources, facts and nodes are persisted to the DB. With regular (thick)
> storeconfigs every resources are persisted to the database.

...and to follow up on this late: when you use PuppetDB, not only is
performance better than full storeconfigs, it is usually better than
thin storeconfigs. PuppetDB delivers this *without* giving up any
information.

I can't recommend strongly enough that you look to PuppetDB before you
look to thin storeconfigs in production.

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons