introducing puppetboard 0.0.1

[Daniele Sluijters]

Hello everyone,

It’s a lovely grey and rainy day here in the Dutch summer, as good a day as any to release a new little project.

Its name is Puppetboard and has as aim to replace Puppet Dashboard’s reporting functionality. It does not nor will it include ENC features. It does all this without storing any data itself but querying PuppetDB instead.

The whole thing is built in Python and relies on Flask and WTForms. The communication logic has been split of in its own library called pypuppetdb which makes heavy use of the requests library. The interface is powered by Twitter Bootstrap with the Flatly theme.

Though I’ve pushed all the code out and made it public it’s all very young but it works fairly well. However, I’ve committed numerous barbarities in the code just to get things working and to figure out how to handle certain things. For the foreseeable time in the future I’ll be working on cleaning all this up and figuring out what I can do on my side and on PuppetDB’s side to make all this work a little better. Especially when it comes to dealing with big responses from PuppetDB...

This is the first time I’m open sourcing a project so that too is all new to me. I’d welcome the feedback and if someone feels brave enough even commits on the projects but try and be gentle about it :-). I’ll also be at PuppetConf including the Developer Day so feel free to reach out to me in person.

To the code:

 * puppetboard: https://github.com/nedap/puppetboard

 * pypuppetdb:  https://github.com/nedap/pypuppetdb

I realise that puppetboard doesn't have a test suite right now but it will soon. In order to do so I have to restructure a few things about it first. The installation documentation will improve with it.

Pypuppetdb's test suite will be expanding the coming days once I'm done mocking the HTTP requests _query() makes and manage to get a decent and big enough set of test data to feed into PuppetDB. This will allow me to run integration tests and benchmark certain changes I have in mind.

I’m hoping to be able to get a release out every month with improvements to both projects, perhaps even faster in the beginning but it remains to be seen how much time I’ll be able to spend on it.

A special thanks goes out to Ken Barber for helping out with all things PuppetDB and coming up with a way to run PuppetDB on Travis so we can run integration tests. Hunter, thank you for being so interested in this project and pushing me to release it.

— 

Daniele Sluijters

Nedap | Steppingstone

[Klavs Klavsen]

It seems very cool. Thank you for sharing.

One thing that I would be missing, before being able to switch (as gathered from screenshots - I haven't tested it yet :) - is the overview of failed, unreported and unresponsive hosts.
Also - there would need to be a way to query this - just as I currently do from puppet-dashboard (for my nagios monitoring) - so that I can alert when I started getting failed runs etc.

[Daniele Sluijters]

Hi Klavs,

Thanks, I'll think about that. An API that could be used for notifications and such was already on my mind. Unfortunately I don't think I can currently ask PuppetDB for the statistics you mention but perhaps that can be added to PuppetDB.

As far as notifying about failed runs through Nagios, we actually use a plugin that checks the report age and its content on every node.

-- 

Daniele Sluijters

[Ellison Marks]

If it's something that means I can finally ditch puppet-dashboard's reporting functionality, I will be on that like jam on toast. Screenshots look very cool, I'll probably try to install it later this week.

[Erik Dalén]

Cool stuff will try it out. I think we might have some Python projects at work that would be interested in using the pypuppetdb library as well.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To post to this group, send email to puppet...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Erik Dalén

[Daniele Sluijters]

Hey,

That's awesome, let me know how it goes. I'm not sure how well it will be able to handle itself at the scale of your infrastructure but only one way to find out!

-- 

Daniele Sluijters

[Ellison Marks]

So I finally got the time to get this going, and it is indeed, very cool. One hitch though: I grabbed the source from github, and most things seemed to work, but the overview page errored out. It seems that it was using python 3 syntax for it's format strings on line 86 and 88.

86: 'avg_resources_node': "{:10.6f}".format(avg_resources_node['Value']),
87: 'mean_failed_commands': mean_failed_commands['MeanRate'],
88: 'mean_command_time': "{:10.6f}".format(mean_command_time['MeanRate']),

I added a 0 to the format specification and it worked  fine

86: 'avg_resources_node': "{0:10.6f}".format(avg_resources_node['Value']),
87: 'mean_failed_commands': mean_failed_commands['MeanRate'],
88: 'mean_command_time': "{0:10.6f}".format(mean_command_time['MeanRate']),

On Wednesday, August 7, 2013 5:47:23 AM UTC-7, Daniele Sluijters wrote:

[Ellison Marks]

Er, excuse me, I misspoke. As it is it relies on having python 2.7+, not 3 for the formatting. Still, the Redhat family, by default, is still on 2.6.

[Daniele Sluijters]

Hey,

Good catch, I'll get that fixed asap.

-- 

Daniele Sluijters

[Niels Abspoel]

Hi Daniele Sluijters,

I've build some archlinux packages to test this out,

and it works very well on Archlinux.

https://aur.archlinux.org/packages/python2-puppetboard/

https://aur.archlinux.org/packages/python2-pypuppetdb/

Op woensdag 7 augustus 2013 14:47:23 UTC+2 schreef Daniele Sluijters:

[José Luis Ledesma]

Hey, I lost this instruction ( was not in the group in august)

Really nice software, being using it for a couple of months, although I think it lacks some things to be even more useful( at least for me):
- in the main page nodes are in unchanged, changed or error state, would be nice to have the outofsync state( although I don't know if you can get this state without analyzing the full report)
- in the nodes page be able to filter to show last changed reports, last error reports and so on, showing only last ten reports I think is not useful enough.

And that it's :)

Thanks,

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/6e95c3f7-3825-4bfc-a912-88b1ef08337e%40googlegroups.com.

[Daniele Sluijters]

Niels: Thank you for the Arch packages, I'll add them to the README's.

Jose: I'm not sure what you're getting at, what do you understand as outofsync? Nodes that haven't checked in in a while show up, if that timeframe is too big for you can modify the settings.

-- 

Daniele Sluijters

[José Luis Ledesma]

Out of sync means that there are configurations in the node that doesn't meet the configuration it should have. It is useful if you work in noop mode by default.

I'm the yaml report files when a resource doesn't meet the configuration established in puppet master it appears as out of sync.

Regards,

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1b7c31c2-f073-4390-9d74-addfd8e863ed%40googlegroups.com.

[Niels Abspoel]

Hi Daniele,

Thanks for the update in the Readme with the Archlinux Packages.

Would you like to have opensuse/suse support packages as well?

I'm building these on opensuse build server: https://build.opensuse.org/project/show/home:aboe76

Please do not link from that repository, it's my own rpm build environment. 

If you want I can push these into the devel:languages:python repository of opensuse/suse, that way more people

can test this software, and from that repository it is even possible to add them to the next version of opensuse in the default repositories.

with regards,

Niels

[Darin Perusich]

You should SR it for the OpenSUSE systems:management:puppet repo so its available along with puppetdb and puppet.

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a0e4ac7d-f470-48ad-8c0e-0ac3817aee40%40googlegroups.com.

[Niels Abspoel]

Darin no problem, will do