Actual diffs in puppetdb?

[Bruce]

Any plans to get the actual diffs of file changes into puppetdb?  Right now you get the hashes of the buckets on the host, but if diffs could get into puppetdb, it could be extremely useful.

[Ryan Senior]

I've recently added a feature to catalog hashes, similar to what you are describing as a sort of debug mode option.  It's in master now and should be in the next release (1.6).  More information is available in the ticket here: http://projects.puppetlabs.com/issues/22977. It has the potential to be slow and grow very large, so it's something you would enable for short periods of time to help diagnose problems.

Does that cover your use case?

-Ryan

On Tue, Nov 19, 2013 at 10:58 AM, Bruce <bly...@yahoo.com> wrote:

Any plans to get the actual diffs of file changes into puppetdb?  Right now you get the hashes of the buckets on the host, but if diffs could get into puppetdb, it could be extremely useful.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/6e3a0868-89ea-42cc-a3d4-1464be0b938b%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Bruce]

On Tuesday, November 19, 2013 9:05:05 AM UTC-8, Ryan Senior wrote:

I've recently added a feature to catalog hashes, similar to what you are describing as a sort of debug mode option.  It's in master now and should be in the next release (1.6).  More information is available in the ticket here: http://projects.puppetlabs.com/issues/22977. It has the potential to be slow and grow very large, so it's something you would enable for short periods of time to help diagnose problems.

Does that cover your use case?

Maybe.  But usually I don't know I want this information, until I need it.  So having to turn on some debugging ahead of time doesn't help.  This is more for a situation like: "Huh, this node's behavior has changed.  Let me look, aha, this file was changed by puppet, how did it change?"

Currently I'm using the excellent puppetboard tool. (https://github.com/nedap/puppetboard)  It's good at showing me the changes going on, and if I look at a report for a node, that there was a file changed. It shows md5 changed from, and  md5 changed too. In a perfect world, I could click on that, and see the actual diff of what changed.

I was actually thinking of writing a simple daemon to run on all my nodes, something like puppet-bucket-viewer.  You give it an md5, and it returns it if it's present on the node's bucket list.  This would allow modifications to tools like puppetboard to get this information, and display it to the user.

[R.I.Pienaar]

----- Original Message -----
> From: "Bruce" <bly...@yahoo.com>
> To: puppet...@googlegroups.com
> Sent: Tuesday, November 19, 2013 5:25:32 PM
> Subject: Re: [Puppet Users] Actual diffs in puppetdb?
>
>
>
> On Tuesday, November 19, 2013 9:05:05 AM UTC-8, Ryan Senior wrote:
> >
> > I've recently added a feature to catalog hashes, similar to what you are
> > describing as a sort of debug mode option. It's in master now and should
> > be in the next release (1.6). More information is available in the ticket
> > here: http://projects.puppetlabs.com/issues/22977. It has the potential
> > to be slow and grow very large, so it's something you would enable for
> > short periods of time to help diagnose problems.
> >
> > Does that cover your use case?
> >
>
> Maybe. But usually I don't know I want this information, until I need it.
> So having to turn on some debugging ahead of time doesn't help. This is
> more for a situation like: "Huh, this node's behavior has changed. Let me
> look, aha, this file was changed by puppet, how did it change?"
>
> Currently I'm using the excellent puppetboard tool.
> (https://github.com/nedap/puppetboard) It's good at showing me the changes
> going on, and if I look at a report for a node, that there was a file
> changed. It shows md5 changed from, and md5 changed too. In a perfect
> world, I could click on that, and see the actual diff of what changed.

the other side of the coin is that often diffs contain sensitive information.

I think the default behaviour used to be that these were in reports but it
just burned many people.

>
> I was actually thinking of writing a simple daemon to run on all my nodes,
> something like puppet-bucket-viewer. You give it an md5, and it returns it
> if it's present on the node's bucket list. This would allow modifications
> to tools like puppetboard to get this information, and display it to the
> user.

could just use the centralised file bucket stuff and check on the master?

[Bruce]

On Tuesday, November 19, 2013 9:29:16 AM UTC-8, R.I. Pienaar wrote:

the other side of the coin is that often diffs contain sensitive information.

I think the default behaviour used to be that these were in reports but it
just burned many people.

That's true.  Maybe it could be an attribute on a file?  sensitive => false, default to true.  false will show diffs in reports.

 

> I was actually thinking of writing a simple daemon to run on all my nodes,
> something like puppet-bucket-viewer.  You give it an md5, and it returns it
> if it's present on the node's bucket list.  This would allow modifications
> to tools like puppetboard to get this information, and display it to the
> user.

could just use the centralised file bucket stuff and check on the master?

I see how to set the central buckets up.  Is there a programatic way to access that data? 

[Bruce]

On Tuesday, November 19, 2013 9:59:23 AM UTC-8, Bruce wrote:

I see how to set the central buckets up.  Is there a programatic way to access that data? 

Nevermind.  Found http://docs.puppetlabs.com/man/filebucket.html 

[Martin Langhoff]

On Tue, Nov 19, 2013 at 12:25 PM, Bruce <bly...@yahoo.com> wrote:
> Maybe. But usually I don't know I want this information, until I need it.
> So having to turn on some debugging ahead of time doesn't help.

I am using etckeeper in conjunction with Puppet for exactly this use
case. Additionally, my puppet configuration itself is under git.

With those two things, it is trivial to walk back to any change. As
etckeeper is not part of puppet, it tracks _any_ change on /etc, and
this adds very valuable coverage.

For example: if you install an rpm, and one of its dependencies
installs a new cronjob (with some unexpected side-effect), Puppet will
not give you visibility on that unexpected cronjob; etckeeper will.

etckeeper is a keeper ;-)

Aside: If puppet would replace its "file bucket" functionality with a
git-based store I would be the happiest man on earth. The contents of
files changed would be available by path as well as hash. git tags
would also allow for a clean migration (preserving md5-based lookups
for preexisting files).

I am familiar with git internals -- I've authored parts of early git,
and various importers still in use -- so I'd be glad to help flesh
this out, if anyone is ever interested.

Current file bucket is pretty useless to me in practice :-/




m
--
martin....@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff