Reading Puppet reports with Python

[Douglas Garstang]

Has anyone got/seen Python code to read puppet reports?

I added a bunch of these:

class PuppetReport(yaml.YAMLObject):

  yaml_tag = u'!ruby/object:Puppet::Transaction::Report'

  def __init__(self, host, logs, metrics, records, time):

    self.host = host

    self.logs = logs

    self.metrics = metrics

    self.records = records

    self.time = time

However, the Python YAML parser fails with:

"yaml.constructor.ConstructorError: expected a mapping node, but found scalar"

On the ruby/sym line:

    !ruby/sym config_retrieval: 0.853000164031982

I don't know how to define the classes to accept a scalar data type rather than a mapping node data type, and Mr Google is giving me ZERO answers.

Doug.

[Eduardo S. Scarpellini]

Douglas,
I've started my tests with Python + Puppet Yaml yesterday, based on http://www.mailinglistarchive.com/html/puppet...@googlegroups.com/2010-05/msg00539.html, and I'm having the same difficulties.
I shall keep working on this script today and I'll post the solution, if I can find it.

2010/9/28 Douglas Garstang <doug.g...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--
 Eduardo S. Scarpellini
<scarp...@gmail.com>

[Nigel Kersten]

Huh. This must have changed, as I was doing it in exactly the same
way. Is that actually from an old list message of mine?

I think you've misread the error message. It basically doesn't know
what to do with !ruby/sym at all.

It may turn out nowadays to be simpler to do the parsing in Ruby and
loading the puppet libraries so you get all this for free.

I'll have a poke around and see if I can work out how to coerce
!ruby/sym to something Python can understand.

[Douglas Garstang]

On Tue, Sep 28, 2010 at 9:50 AM, Eduardo S. Scarpellini <scarp...@gmail.com> wrote:

Douglas,
I've started my tests with Python + Puppet Yaml yesterday, based on http://www.mailinglistarchive.com/html/puppet...@googlegroups.com/2010-05/msg00539.html, and I'm having the same difficulties.
I shall keep working on this script today and I'll post the solution, if I can find it.

Eduardo,

That's about the only document I found on the subject too. I don't know why the report document has to be so complicated. The !ruby tags are completely unnecessary. I'm not about to go and learn ruby, as I dislike it immensely. I guess it might be possible to pre-process the reports and strip the !ruby tags out before trying to load the yaml document with python.

Doug.

 

[Douglas Garstang]

On Tue, Sep 28, 2010 at 9:50 AM, Eduardo S. Scarpellini <scarp...@gmail.com> wrote:

Douglas,
I've started my tests with Python + Puppet Yaml yesterday, based on http://www.mailinglistarchive.com/html/puppet...@googlegroups.com/2010-05/msg00539.html, and I'm having the same difficulties.
I shall keep working on this script today and I'll post the solution, if I can find it.

[Douglas Garstang]

Nigel, yep. It's an old message post of yours. I tried it too, and had the same issue as Eduardo. 

It seems that it fails on !ruby/sym because:

yaml.constructor.ConstructorError: expected a mapping node, but found scalar

  in "/var/lib/puppet/reports/foo.com/201009281644.yaml", line 3, column 5

where the code has:

class PuppetObj1(yaml.YAMLObject):

    yaml_tag = u'!ruby/sym'

    def __init__(self, attr):

        self.attr = attr

I really don't know how a scalar node is supposed to differ from a mapping node in the python code.

Doug.

Doug.

 

[Douglas Garstang]

On Tue, Sep 28, 2010 at 10:13 AM, Nigel Kersten <ni...@explanatorygap.net> wrote:

Upon what basis are you saying that the Ruby tags are completely unnecessary?

On the basis that they don't actually contain any useful data.

 

It's not true and you're taking a particularly shitty attitude for
someone who is asking for help Doug.

It is true, and you're taking a particularly sensitive attitude for someone that doesn't know me Nigel.

 

Have you ever dumped Python objects to YAML and tried to read them from Ruby?

No, I don't like Ruby.

 

I've just worked out how to solve your problem but haven't finished it.

I have no desire to do so now.

Good grief.

 

Here is what I've worked out so far. If you have issues, go read the
pyYAML documentation and YAML spec like I just did.


def sym_representer(dumper, data):
 return dumper.represent_scalar(u'!ruby/sym', u'%sd%s' % data)

def sym_constructor(loader, node):
 value = loader.construct_scalar(node)
 return "foo" # you're going to have to work out how to make your
constructor work.

yaml.add_constructor(u'!ruby/sym', sym_constructor)

I already read it, but didn't understand it within the context of the problem I was trying to solve.

Doug.

[Nigel Kersten]

On Tue, Sep 28, 2010 at 10:08 AM, Douglas Garstang
<doug.g...@gmail.com> wrote:
>

Upon what basis are you saying that the Ruby tags are completely unnecessary?

It's not true and you're taking a particularly shitty attitude for

someone who is asking for help Doug.

Have you ever dumped Python objects to YAML and tried to read them from Ruby?

I've just worked out how to solve your problem but haven't finished it.

I have no desire to do so now.

Here is what I've worked out so far. If you have issues, go read the
pyYAML documentation and YAML spec like I just did.

def sym_representer(dumper, data):
return dumper.represent_scalar(u'!ruby/sym', u'%sd%s' % data)

def sym_constructor(loader, node):
value = loader.construct_scalar(node)
return "foo" # you're going to have to work out how to make your
constructor work.

yaml.add_constructor(u'!ruby/sym', sym_constructor)

http://pyyaml.org/wiki/PyYAMLDocumentation#Constructorsrepresentersresolvers

[Rob McBroom]

On Sep 28, 2010, at 1:25 PM, Douglas Garstang wrote:

It's not true and you're taking a particularly shitty attitude for
someone who is asking for help Doug.

It is true, and you're taking a particularly sensitive attitude for someone that doesn't know me Nigel.

If this was your first post, maybe. But you could fill an encyclopedia with examples of you being a dick to people trying to help you on this list.

-- 
Rob McBroom
<http://www.skurfer.com/>

[Peter Meier]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>> It's not true and you're taking a particularly shitty attitude for
>> someone who is asking for help Doug.
>>
>>
>> It is true, and you're taking a particularly sensitive attitude for
>> someone that doesn't know me Nigel.
>
>
> If this was your first post, maybe. But you could fill an
> encyclopedia with examples of you being a dick to people trying to
> help you on this list.
>

http://groups.google.com/groups/profile?hl=en&enc_user=Wszv3BcAAADOr23LoVoQQ1XoTfpZUyf8HqZiDvCVswhrZ6TQxKj0ww

obviously a lot of people don't know him...

~pete
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkyiRA4ACgkQbwltcAfKi3+9rACaA1dTntp/pEj5f5iTLwU/63st
1ZUAmgPf3pEzSJBYE70hQGGqiK3a2hnk
=r+x/
-----END PGP SIGNATURE-----

[Douglas Garstang]

We are a sensitive lot aren't we. All I said was that the !ruby tags were pointless and somehow that was interpreted as a personal attack.

Doug.

[Eduardo S. Scarpellini]

Douglas,
just an update: Now Im testing The Foreman (theforeman.org) and I think it covers my main needs (including a ''kind of'' [not so rich] URL API).
Alternatively to YAML, you can use storedconfigs and some database, like MySQL or sqlite and, then, use any language you want :)

2010/9/28 Douglas Garstang <doug.g...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Douglas Garstang]

On Tue, Sep 28, 2010 at 10:13 AM, Nigel Kersten <ni...@explanatorygap.net> wrote:

On Tue, Sep 28, 2010 at 10:08 AM, Douglas Garstang
<doug.g...@gmail.com> wrote:
>
> On Tue, Sep 28, 2010 at 9:50 AM, Eduardo S. Scarpellini
> <scarp...@gmail.com> wrote:
>>
>> Douglas,
>> I've started my tests with Python + Puppet Yaml yesterday, based on
>> http://www.mailinglistarchive.com/html/puppet...@googlegroups.com/2010-05/msg00539.html,
>> and I'm having the same difficulties.
>> I shall keep working on this script today and I'll post the solution, if I
>> can find it.
>>
>>
>
> Eduardo,
> That's about the only document I found on the subject too. I don't know why
> the report document has to be so complicated. The !ruby tags are completely
> unnecessary. I'm not about to go and learn ruby, as I dislike it immensely.
> I guess it might be possible to pre-process the re

Upon what basis are you saying that the Ruby tags are completely unnecessary?

On the basis that I've been able to strip them out, read in the yaml files with Python just fine, and get the data I need.

Current date/time: 2010-10-04 17:41:21

###  Node                             Last Run(UTC)    Delta Syntax Success Fail

--------------------------------------------------------------------------------

001  app06.pax.xxxxxxxxx.com         2010-10-04 17:40     1   Ok         1    0

002  prov01.den.xxxxxxxx.com         2010-10-04 17:40     1   Ok         2    0

003  name02.den.xxxxxxxx.com         2010-10-04 17:39     2   Ok         1    0

004  rmon01.pax.xxxxxxxx.com         2010-10-04 17:39     2   Ok         1    0

005  jump01.den.xxxxxxxx.com         2010-10-04 17:39     2   Ok         1    0

Doug.

[Ohad Levy]

On Mon, Oct 4, 2010 at 6:54 PM, Eduardo S. Scarpellini <scarp...@gmail.com> wrote:

Douglas,

just an update: Now Im testing The Foreman (theforeman.org) and I think it covers my main needs (including a ''kind of'' [not so rich] URL API).

Just FYI, the new upcoming release have JSON based API, 

if you have any specific needs, now is a good time to raise them at theforeman.org (or #theforeman irc chnanel).

Ohad