Suggestion: A [QUIZ] in the subject of emails about the problem helps everyone on Ruby Talk follow the discussion. Please reply to the original quiz message, if you can.
More than a few times I've wished I could get a nice nested OpenStruct out of YAML data, instead of the more unwieldy nested hashes. It's mostly a matter of style. It's a straightforward task to convert a nested hash structure into a nested OpenStruct, but it's the sort of task that you can do a lot of ways, and I'll bet some of you can come up with more elegant and/or more efficient ways than I have so far.
Here's a sample YAML document to get you started:
--- foo: 1 bar: baz: [1, 2, 3] quux: 42 doctors: - William Hartnell - Patrick Troughton - Jon Pertwee - Tom Baker - Peter Davison - Colin Baker - Sylvester McCoy - Paul McGann - Christopher Eccleston - David Tennant a: {x: 1, y: 2, z: 3}
> Suggestion: A [QUIZ] in the subject of emails about the problem helps everyone > on Ruby Talk follow the discussion. Please reply to the original quiz message, > if you can.
> More than a few times I've wished I could get a nice nested OpenStruct out of > YAML data, instead of the more unwieldy nested hashes. It's mostly a matter of > style. It's a straightforward task to convert a nested hash structure into a > nested OpenStruct, but it's the sort of task that you can do a lot of ways, and > I'll bet some of you can come up with more elegant and/or more efficient ways > than I have so far.
> Here's a sample YAML document to get you started:
> --- > foo: 1 > bar: > baz: [1, 2, 3] > quux: 42 > doctors: > - William Hartnell > - Patrick Troughton > - Jon Pertwee > - Tom Baker > - Peter Davison > - Colin Baker > - Sylvester McCoy > - Paul McGann > - Christopher Eccleston > - David Tennant > a: {x: 1, y: 2, z: 3}
can we make it more realistic? how bout this a sample data
--- foo: 1 bar: baz: [1, 2, 3] quux: 42 doctors: - William Hartnell - Patrick Troughton - Jon Pertwee - Tom Baker - Peter Davison - Colin Baker - Sylvester McCoy - Paul McGann - Christopher Eccleston - David Tennant a: {x: 1, y: 2, z: 3} table: walnut method: linseed oil type: contemporary id: 1234 send: fedex
??
-a -- be kind whenever possible... it is always possible. - h.h. the 14th dali lama
On 6/2/06, Ruby Quiz <ja...@grayproductions.net> wrote:
> More than a few times I've wished I could get a nice nested OpenStruct out of > YAML data, instead of the more unwieldy nested hashes. It's mostly a matter of > style. It's a straightforward task to convert a nested hash structure into a > nested OpenStruct, but it's the sort of task that you can do a lot of ways, and > I'll bet some of you can come up with more elegant and/or more efficient ways > than I have so far.
First p0st!
Ok, I cheated... Hans pointed me at the rubyquiz site before this showed up in the list, so I got a head start. But in only 15 minutes, I've got it solved, with only 18 lines (only 5 of those are in method bodies), sans whitespace. Golfers... go!
I'd like to thank Hans for a really straightforward (such that I can do it in my limited time), yet still interesting quiz!
> Here's a sample YAML document to get you started:
> --- > foo: 1 > bar: > baz: [1, 2, 3] > quux: 42 > doctors: > - William Hartnell > - Patrick Troughton > - Jon Pertwee > - Tom Baker > - Peter Davison > - Colin Baker > - Sylvester McCoy > - Paul McGann > - Christopher Eccleston > - David Tennant > a: {x: 1, y: 2, z: 3}
Ahh, a nice, quick one.
I had a 7 line method, that became 6 lines, that became 4 lines... became 2 really ugly lines if I can include and not count a helper method to turn mapped Hashes back into Hashes.
My output, reformatted for prettiness (it also handles Ara's additions):
> > Suggestion: A [QUIZ] in the subject of emails about the problem helps > everyone > > on Ruby Talk follow the discussion. Please reply to the original quiz > message, > > if you can.
> > More than a few times I've wished I could get a nice nested OpenStruct > out of > > YAML data, instead of the more unwieldy nested hashes. It's mostly a > matter of > > style. It's a straightforward task to convert a nested hash structure > into a > > nested OpenStruct, but it's the sort of task that you can do a lot of > ways, and > > I'll bet some of you can come up with more elegant and/or more efficient > ways > > than I have so far.
> > Here's a sample YAML document to get you started:
> > --- > > foo: 1 > > bar: > > baz: [1, 2, 3] > > quux: 42 > > doctors: > > - William Hartnell > > - Patrick Troughton > > - Jon Pertwee > > - Tom Baker > > - Peter Davison > > - Colin Baker > > - Sylvester McCoy > > - Paul McGann > > - Christopher Eccleston > > - David Tennant > > a: {x: 1, y: 2, z: 3}
> can we make it more realistic? how bout this a sample data
> --- > foo: 1 > bar: > baz: [1, 2, 3] > quux: 42 > doctors: > - William Hartnell > - Patrick Troughton > - Jon Pertwee > - Tom Baker > - Peter Davison > - Colin Baker > - Sylvester McCoy > - Paul McGann > - Christopher Eccleston > - David Tennant > a: {x: 1, y: 2, z: 3} > table: walnut > method: linseed oil > type: contemporary > id: 1234 > send: fedex
> ??
> -a > -- > be kind whenever possible... it is always possible. > - h.h. the 14th dali lama
> On 6/2/06, Ruby Quiz <ja...@grayproductions.net> wrote: >> More than a few times I've wished I could get a nice nested >> OpenStruct out of >> YAML data, instead of the more unwieldy nested hashes. It's mostly >> a matter of >> style. It's a straightforward task to convert a nested hash >> structure into a >> nested OpenStruct, but it's the sort of task that you can do a lot >> of ways, and >> I'll bet some of you can come up with more elegant and/or more >> efficient ways >> than I have so far.
> First p0st!
> Ok, I cheated... Hans pointed me at the rubyquiz site before this > showed up in the list, so I got a head start. But in only 15 minutes, > I've got it solved, with only 18 lines (only 5 of those are in method > bodies), sans whitespace. Golfers... go!
> I'd like to thank Hans for a really straightforward (such that I can > do it in my limited time), yet still interesting quiz!
> Jacob Fugal
I thought we were waiting to announce completion. I did it in 17 lines, 9 in the body (whitespace included). Plus a unit test and a benchmark. Short and sweet! Thanks Hans! -Mat
> I was throwing different yaml files at my solution and I came > across this > sample of valid YAML which doesn't easily fit into an OpenStruct
> --- > 1: for the money > 2: for the show > 3: to get ready > 4: go go go
> Is this a valid testcase?
I set up my test case to work with strange keys (numbers, OpenStruct methods), but I'm not sure what the right behavior is. I see 3 possible behaviors: 1. accept the data and let the client figure out how to get keys like "methods" back out again. (my choice) 2. Throw an exception when trying to store a key that doesn't map to a legal, free function name - then what if they define the same key twice? 3. Try to remove the functions that are already defined, then redefine - can you undefine core ruby functions like 'methods'? -Mat
On Sat, 3 Jun 2006 03:31:05 +0900, "Jacob Fugal" <lukf...@gmail.com> wrote: > Ok, I cheated... Hans pointed me at the rubyquiz site before this > showed up in the list, so I got a head start. But in only 15 minutes, > I've got it solved, with only 18 lines (only 5 of those are in method > bodies), sans whitespace. Golfers... go!
Hold on. Golfers, can your solutions handle ... this?
> On Sat, 3 Jun 2006 03:31:05 +0900, "Jacob Fugal" > <lukf...@gmail.com> wrote: >> Ok, I cheated... Hans pointed me at the rubyquiz site before this >> showed up in the list, so I got a head start. But in only 15 minutes, >> I've got it solved, with only 18 lines (only 5 of those are in method >> bodies), sans whitespace. Golfers... go!
> Hold on. Golfers, can your solutions handle ... this?
My non-golf solution can't handle that. You sir, are very very sick (I mean that in the nicest way possible). Also I didn't know YAML could have cycles (thought it had to be a tree). This gives me a sinking feeling in the pit of my stomach. If YAML is meant to be used for serialization, then of course it must support cycles... but this makes me worry about all the projects that use YAML as a config file.
> -----Original Message----- > From: MenTaLguY [mailto:men...@rydia.net] > Sent: Friday, June 02, 2006 2:00 PM > To: ruby-talk ML > Subject: Re: [QUIZ] Hash to OpenStruct (#81)
> On Sat, 3 Jun 2006 03:31:05 +0900, "Jacob Fugal" > <lukf...@gmail.com> wrote: > > Ok, I cheated... Hans pointed me at the rubyquiz site before this > > showed up in the list, so I got a head start. But in only > 15 minutes, > > I've got it solved, with only 18 lines (only 5 of those are > in method > > bodies), sans whitespace. Golfers... go!
> Hold on. Golfers, can your solutions handle ... this?
No, and I don't care, because in the real world people don't randomly parse recursive config files.
KYFD.
Regards,
Dan
This communication is the property of Qwest and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
On Sat, 3 Jun 2006 06:25:13 +0900, "Berger, Daniel" <Daniel.Ber...@qwest.com> wrote: > No, and I don't care, because in the real world people don't randomly > parse recursive config files.
Well, just consider it an extra challenge for the folks with extra time who got bored with the original problem, as an alternative to golfing it into the ground. (Though if you'd rather golf that's great too.)
> On Sat, 3 Jun 2006 03:31:05 +0900, "Jacob Fugal" > <lukf...@gmail.com> wrote: >> Ok, I cheated... Hans pointed me at the rubyquiz site before this >> showed up in the list, so I got a head start. But in only 15 minutes, >> I've got it solved, with only 18 lines (only 5 of those are in method >> bodies), sans whitespace. Golfers... go!
> Hold on. Golfers, can your solutions handle ... this?
Wicked test case, MenTaLguY. I had to think that one through a couple times to get it right. I'm up to 21 lines now (whitespace included, no comments), but I can handle it. Still not sure how I can condense it (short of inserting ; anyway). -Mat
On Sat, 3 Jun 2006 05:49:30 +0900, Logan Capaldo <logancapa...@gmail.com> wrote: > If YAML is meant to be used for serialization, then of course it must support > cycles... but this makes me worry about all the projects that use YAML as a config file.
For most of the configuration files I've seen, I think it'd mostly only be an issue if you're doing blind recursive transformations of the tree (as in this case).
Otherwise, in most cases I've seen, there simply isn't any room for an arbitrarily deep set of nested hashes in the schema -- either you'd get an error from a hash being in an unexpected place, or that recursive subtree would simply get ignored.
You might have to think about these sorts of things on rare occasions, but it's not the end of the world.
On Sat, 2006-06-03 at 04:59 +0900, MenTaLguY wrote: > On Sat, 3 Jun 2006 03:31:05 +0900, "Jacob Fugal" <lukf...@gmail.com> wrote: > > Ok, I cheated... Hans pointed me at the rubyquiz site before this > > showed up in the list, so I got a head start. But in only 15 minutes, > > I've got it solved, with only 18 lines (only 5 of those are in method > > bodies), sans whitespace. Golfers... go!
> Hold on. Golfers, can your solutions handle ... this?
My solution started off from the most basic hash to openstruct conversion I could think of: OpenStruct.new(some_hash). Those pesky nested hashes still needed to be dealt with, so I came up with:
class Hash def to_ostruct(clz = OpenStruct) clz.new Hash[*inject([]){|ar,(k,v)|ar<<k<<(v.to_ostruct(clz) rescue v)}] end end
This works, but it's very inefficient, it doesn't pass the case Mentalguy posted, and it doesn't fail well with invalid keys or other errors. To handle those things, I had to go a bit longer:
class Hash def to_ostruct(clz = OpenStruct, cch = {}) cch[self] = (os = clz.new) each do |k,v| raise "Invalid key: #{k}" unless k =~ /[a-z_][a-zA-Z0-9_]*/ os.__send__("#{k}=", v.is_a?(Hash)? cch[v] || v.to_ostruct(clz,cch) : v) end os end end
I chose to fail for invalid keys, rather than introducing potentially confusing renaming rules or similar. It's still not as efficient as it might be, but a bit better than the first one.
Neither solution takes into consideration the problems Ara pointed out - this is the reason for the optional 'clz' parameter to both methods. Undef'ing methods from OpenStruct turned out to be a non-starter, since it uses them itself, so I just implemented a simple, naive DumbStruct that can be used with the to_ostruct methods above:
class DumbStruct alias :__iv_set__ :instance_variable_set alias :__class__ :class instance_methods.each do |m| undef_method(m) unless m =~ /^(__|method_missing|inspect|to_s)|\?$/ end
def method_missing(name, *args, &blk) if (name = name.to_s) =~ /[^=]=$/ name = name[0..-2] __iv_set__("@#{name}", args.first) (class << self; self; end).class_eval { attr_accessor name } else super end end end
Attached are the full files including testcases and a basic benchmark. Thanks for another fun and interesting quiz :)
My solution was very simple: require 'yaml';require 'ostruct';def h(h)h.map{|k,v|h[k]=Hash\ ===v ?h(v):v};OpenStruct.new(h)end;puts h(YAML.load($<.read))
This can't deal with the recursion that MentalGuy(sorry for the wrong capitalisation!) posted. The following is basically the same code: class Hash def to_ostruct copy = {} each do |(key,value)| if value.class == Hash copy[key] = value.to_ostruct else copy[key] = value end end OpenStruct.new(copy) end end I tried to look into changing YAML.load to make OpenStruct's instead of hashes, but I soon gave up on that :)
The main idea here was to build the OpenStruct all at once, rather than resorting to a bunch of Object#send( "#{name}=", value ) calls. To do this, however, we end up going from a hash, to a sequence of pairs, to a flat array of alternating keys and values, back to a hash, and then finally to an OpenStruct.
ATTEMPT #2:
Then I got bored and decided I'd deal with the case of a self-recursive hash. My first attempt used lazy.rb ( http://moonbase.rydia.net/software/lazy.rb ); I simply made the above hashes_to_openstructs() function lazy (by wrapping its innards in promise {}), then memoized it using a hash:
While that's sort of clever, I couldn't help but think there was a simpler solution. It involved building the OpenStruct incrementally, using the same Object#send calls I'd been hoping to avoid.
def hashes_to_openstructs( obj, memo={} ) return obj unless Hash === obj os = memo[obj] = OpenStruct.new obj.each do |k, v| os.send( "#{k}=", memo[v] || hashes_to_openstructs( v, memo ) ) end os end
Ross Bamford wrote: > My solution started off from the most basic hash to openstruct > conversion I could think of: OpenStruct.new(some_hash). Those pesky > nested hashes still needed to be dealt with, so I came up with:
> class Hash > def to_ostruct(clz = OpenStruct) > clz.new Hash[*inject([]){|ar,(k,v)|ar<<k<<(v.to_ostruct(clz) rescue v)}] > end > end
class OpenStruct alias :old_init :initialize def initialize(hash=nil) old_init(hash.each{ |k,v| hash[k] = self.class.new(v) if v.is_a?(Hash) }) end end
To handle parameters that are the same as existant method names (i.e. Ara's sample) requires removal of the 'unless' from new_ostruct_member:
def new_ostruct_member(name) name = name.to_sym meta = class << self; self; end meta.send(:define_method, name) { @table[name] } meta.send(:define_method, "#{name}=""#{name}=") { |x| @table[name] = x } end
% cat hash_to_open_struct2.rb require 'yaml' require 'ostruct' class Object def hash_to_ostruct(visited = []) self end end
class Array def hash_to_ostruct(visited = []) map { |x| x.hash_to_ostruct(visited) } end end
class Hash def hash_to_ostruct(visited = []) os = OpenStruct.new each do |k, v| item = visited.find { |x| x.first.object_id == v.object_id } if item os.send("#{k}=", item.last) else os.send("#{k}=", v.hash_to_ostruct(visited + [ [self, os] ])) end end os end end
yaml_source = <<YAML --- foo: 1 bar: baz: [1, 2, 3] quux: 42 doctors: - William Hartnell - Patrick Troughton - Jon Pertwee - Tom Baker - Peter Davison - Colin Baker - Sylvester McCoy - Paul McGann - Christopher Eccleston - David Tennant - {w: 1, t: 7} a: {x: 1, y: 2, z: 3} YAML evil_yaml = <<EVIL --- &verily lemurs: unite: *verily beneath: - patagonian - bread - products thusly: [1, 2, 3, 4] EVIL
loaded = YAML.load(yaml_source).hash_to_ostruct p loaded.bar.doctors.last.w
evil_loaded = YAML.load(evil_yaml).hash_to_ostruct p evil_loaded.lemurs.beneath p evil_loaded.lemurs.unite.thusly
I manged a very small solution -- practically one line. Only problem is, it doesn't work ;-) But honestly, it's not my fault! No really. Let me explain.
When I first read the quiz my thoughts intitally went to the usual concepts and I considered the Hash#traverse method I wrote some time ago (BTW this quiz helped me improve that method. Many thanks!) But I have good bit of experience with YAML and I immediately had a second thought which would allow me to solve the quiz very quickily and easily. The solution is as follows (were s containes the yaml sample).
YAML.add_builtin_type('map'){ |t,v| OpenStruct.new(v) }; o = YAML.load(s)
But like I said, as clever as it may be, it doesn't work. For whatever reason Syck doesn't handle it properly. Perhaps YAML's 'map' type is too fundamental that it can't comply, or perhaps it's a bug. I don't know. But it just end up returning the same old Hash.
Okay I thought. There's more than one way to skin a cat. And I came up with this close to one-liner that works around the above problem in a most clever way.
i = YAML::load(s) def Hash.def to_yaml_type "!ruby/object:OpenStruct" end o = YAML::load(i.to_yaml)
The nice thing about this soluiton is that it uses a built-in library (YAML/Syck) to do all the hard work --since Syck already understands graphs it takes care of all those messy issues. Cool.
Joey wrote: > The following is basically the same code: > class Hash > def to_ostruct > copy = {} > each do |(key,value)| > if value.class == Hash > copy[key] = value.to_ostruct > else > copy[key] = value > end > end > OpenStruct.new(copy) > end > end
This would be cleaner
class Hash def to_ostruct copy = dup copy.each do |key, value| copy[key] = value.to_ostruct if value.respond_to? :to_ostruct end return copy end end
On Mon, Jun 05, 2006 at 02:55:42AM +0900, Joey wrote: > I tried to look into changing YAML.load to make OpenStruct's instead of > hashes, but I soon gave up on that :)
Your curiousity shouldn't go unmet.
require 'yaml' require 'ostruct'
class << YAML::DefaultResolver alias_method :_node_import, :node_import def node_import(node) o = _node_import(node) o.is_a?(Hash) ? OpenStruct.new(o) : o end end
Gotta post this before I look at other solutions. This caused me to look up what an OpenStruct was so that was benefit #1. Since it was a simple one, I worked through it with my son who is going through Chris Pine's _Learning_to_Program_ right now so that was benefit #2.
I don't know if this will handle the crazier recursive YAML files, but it seems to be fine for normal ones. My son actually struck on the OpenStruct#send being a problem with the presence of the 'send' key in the YAML.
-Rob
# RubyQuiz81: Hash to OpenStruct # 2006-06-02
require 'ostruct' require 'yaml'
class HashToOpenStruct def self.from_yaml(yamlfile) self.to_ostruct(YAML.load(File.open(yamlfile))) end
def self.to_ostruct(h) c = OpenStruct.new h.each { |k,v| c.__send__("#{k}=".to_sym, v.kind_of?(Hash) ? to_ostruct(v) : v) } c end end