Re: [Puppet-dev] Yaml output not quite right from puppet api

[Markus Roberts]

Corey --

I mentioned this before in a previous post but I can't find that post.

Anyways, I am using the puppet api rest interface and my python yaml
parser doesn't like the output given.  Exactly what parser are you
guys using to encode your objects into Yaml?  I don't know why my
parser is choking you the yaml from puppet.
I am using puppet enterprise 1.0 which is version puppet 2.6.4 I
believe.  If I need to patch let me know how to patch with puppet
enterprise.

I am using the following yaml parser

http://pyyaml.org/wiki/PyYAML


This is the part where my yaml parse chokes on

 File "/usr/lib64/python2.4/site-packages/yaml/constructor.py", line
419, in construct_undefined
   node.start_mark)
yaml.constructor.ConstructorError: could not determine a constructor
for the tag '!ruby/object:Puppet::Node::Facts'
 in "<string>", line 1, column 5:
   --- !ruby/object:Puppet::Node::Facts
       ^

This is the yaml output from the test application I built which should
correspond to line 1, column 5
http://pastebin.com/0SgHerVM

That is valid (but inconvenient) yaml; the format specifically allows the serialization of language/application specific objects.  But yeah, we wish we'd not used that feature.

There are a few things you can do here: 1) most yaml parsers should have hooks for you to define mapping to a "local" equivalent type; 2) a sed/awk/python preprocessor could "generisize" the yaml by either replacing the object reference with hashes (which are very similar) or by the equivalent python construct (what does a python object look like serialized?), 3) ... and I forgot what my third idea was...oh.  Some yaml parsers have a (nonstandard) facility to morph unknown objects to something "standard" (e.g. a dictionary).

Do either of these seem feasible?

-- M

-----------------------------------------------------------
When in trouble or in doubt, run in circles,
scream and shout. -- 1920's parody of the 
maritime general prudential rule
------------------------------------------------------------

[Markus Roberts]

On Tue, Mar 15, 2011 at 3:45 PM, Corey Osman <co...@logicminds.biz> wrote:

I am very new to yaml so I have to research your suggestions.  It
sounds like I'll need to create an object for the yaml output to map
to though.

I'll respond back in the thread once I figure this out.

It looks likr pyyaml does support loading foreign types (by mapping them).  Look for add_constructor on this page: http://pyyaml.org/wiki/PyYAMLDocumentation for some examples of how to do this.  You can probably get away with just making all ruby objects into dictionaries for a start.

-- M
-----------------------------------------------------------

[Nigel Kersten]

On Tue, Mar 15, 2011 at 4:01 PM, Markus Roberts <mar...@puppetlabs.com> wrote:
>
>
> On Tue, Mar 15, 2011 at 3:45 PM, Corey Osman <co...@logicminds.biz> wrote:
>>
>> I am very new to yaml so I have to research your suggestions.  It
>> sounds like I'll need to create an object for the yaml output to map
>> to though.
>>
>> I'll respond back in the thread once I figure this out.
>
> It looks likr pyyaml does support loading foreign types (by mapping them).
> Look for add_constructor on this page:
> http://pyyaml.org/wiki/PyYAMLDocumentation for some examples of how to do
> this.  You can probably get away with just making all ruby objects into
> dictionaries for a start.

I've always done this in Python by specifying yaml tags,

http://groups.google.com/group/puppet-users/browse_thread/thread/ededa6a1da32cb00

class PuppetReport(yaml.YAMLObject):
yaml_tag = u'!ruby/object:Puppet::Transaction::Report'
def __init__(self, host, logs, metrics, records, time):
self.host = host
self.logs = logs
self.metrics = metrics
self.records = records
self.time = time

but there are a few options here depending on what you're doing.

[Markus Roberts]

>> I am very new to yaml so I have to research your suggestions.  It
>> sounds like I'll need to create an object for the yaml output to map
>> to though.
>>
>> I'll respond back in the thread once I figure this out.
>
> It looks likr pyyaml does support loading foreign types (by mapping them).
> Look for add_constructor on this page:
> http://pyyaml.org/wiki/PyYAMLDocumentation for some examples of how to do
> this.  You can probably get away with just making all ruby objects into
> dictionaries for a start.

I've always done this in Python by specifying yaml tags,

http://groups.google.com/group/puppet-users/browse_thread/thread/ededa6a1da32cb00

class PuppetReport(yaml.YAMLObject):
 yaml_tag = u'!ruby/object:Puppet::Transaction::Report'
 def __init__(self, host, logs, metrics, records, time):
   self.host = host
   self.logs = logs
   self.metrics = metrics
   self.records = records
   self.time = time

but there are a few options here depending on what you're doing.

I like that way!  Nicely coupled to the intent.

[Corey Osman]

I just removed all the lines that contained !ruby. It seems to work
fine after that. Well I am only getting facts right now, I am sure
the catalog may require additional effort.

tempstring = re.sub('.*\!ruby\/.*','',tempstring)


If anyone is interested my git repo is here:

git://github.com/logicminds/devops.git

On Mar 16, 11:28 am, Markus Roberts <mar...@puppetlabs.com> wrote:
> >> I am very new to yaml so I have to research your suggestions.  It
> > >> sounds like I'll need to create an object for the yaml output to map
> > >> to though.
>
> > >> I'll respond back in the thread once I figure this out.
>
> > > It looks likr pyyaml does support loading foreign types (by mapping
> > them).
> > > Look for add_constructor on this page:

> > >http://pyyaml.org/wiki/PyYAMLDocumentationfor some examples of how to

> > do
> > > this.  You can probably get away with just making all ruby objects into
> > > dictionaries for a start.
>
> > I've always done this in Python by specifying yaml tags,
>

> >http://groups.google.com/group/puppet-users/browse_thread/thread/eded...

[Luke Kanies]

Fortunately, the catalog can be pretty easily formed as json. In fact, in both of these cases, a yaml2json converter would be pretty easy in Puppet, to save you this effort.

> --
> You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
> To post to this group, send email to puppe...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
>

--
The time to repair the roof is when the sun is shining.
-- John F. Kennedy
---------------------------------------------------------------------
Luke Kanies -|- http://puppetlabs.com -|- +1(615)594-8199

[Corey Osman]

I am very new to yaml so I have to research your suggestions. It
sounds like I'll need to create an object for the yaml output to map
to though.

I'll respond back in the thread once I figure this out.

[Corey Osman]