IRC log for 2008-04-27

[David Leal]

--- Log opened Sun Apr 27 00:00:03 2008
--- Day changed Sun Apr 27 2008
00:00 < wycats> I think I fucked up the checkout
00:00 < benburkert> it should be DataMapper.auto_migrate!, if your not using that
00:00 < wycats> how do I checkout the branch?
00:00 < benburkert> git checkout branch automigrations
00:00 < benburkert> err, git checkout automigrations
00:00 < wycats> "pathspec 'automigrations' did not match any file(s) known to git"
00:01 < antares> wycats: did you add a remote to clone? that is, is it in your repo or a separate one?
00:01 < wycats> It's your repo
00:01 < wycats> do you keep your branch up to date?
00:02 < wycats> why is it not in core yet?
00:02 < benburkert> yes, it' up to date
00:02 < mcolyer> try a git pull
00:02 < mcolyer> to refresh?
00:02 < benburkert> looks like you didn't add my repository or you haven't refreshed
00:02 < antares> wycats: git fetch <remote name> or just git fetch if it is a separate clone will fetch branches information
00:03 < wycats> /Library/Ruby/Gems/1.8/gems/dm-core-0.9.0/lib/data_mapper/adapters/data_objects_adapter.rb:243:in `destroy_model_storage': undefined method `debug' for nil:NilClass (NoMethodError)
00:04 < benburkert> ooh, i might have left a debug statement in there or somehting
00:05 < benburkert> lemme check
00:06 < benburkert> wycats: your logger is not set
00:06 < wycats> benburkert: coo
00:06 < benburkert> DataMapper.logger = Merb.logger
00:06 < wycats> /Library/Ruby/Gems/1.8/gems/data_objects-0.9.0/lib/connection.rb:34:in `initialize': unable to open database file (Sqlite3Error)
00:06 < wycats> I thought this was fixed
00:07 < benburkert> i've never seen that
00:07 < benburkert> rm your db file
00:07 < wycats> yeah
00:07 < wycats> I gotta touch it
00:08 < wycats> relative URLs don't work
00:08 < wycats> which was fixed a while back in trunk
00:10 < benburkert> wycats: did you see my pastie on a sql sexp?
00:11 < wycats> yeh
00:11 < wycats> it rocked
00:11 < benburkert> i 'refreshed' it a little, lemme pastie it again
00:13 < benburkert> http://pastie.org/187387
00:15 < benburkert> wycats_: did you see my last message?
00:15 < wycats_> benburkert: yes
00:16 < wycats_> that's crazy
00:16 < wycats_> I love it
00:17 < benburkert> lemme know if i'm doing anything wrong, i'm basing of what (little) i know about rubinius
00:25 < mcolyer> I get a "undefined method `instance_variable_name' for nil:NilClass" when I try to set an assignment on a belongs_to
00:26 < mcolyer> can anyone give me a hand?
00:27 < benburkert> mcolyer: are you trying to set an association defined in a super class?
00:28 < mcolyer> yes
00:29 < mcolyer> benburkert: I take it you can't do that?
00:29 < benburkert> not yet :)
00:29 < benburkert> that's broken in DM right now
00:29 < benburkert> but it's a known issue
00:30 < mcolyer> benburkert: so the workaround is just to create the relationship in all of the subclasses?
00:30 < benburkert> yes
00:36 < mcolyer> benburkert: I am still getting that error and I fixed my models that were subclasses, anything else it could be?
00:37 < benburkert> yea, try using many_to_one instead of belongs_to
00:37 < benburkert> and manually add the foreign key property
00:38 < mcolyer> benburkert: is that a part of 0.3.0
00:38 < benburkert> mcolyer: i thought you were on 0.9
00:39 < benburkert> i'm not sure about any of that stuff on 0.3, sorry :(
00:39 < mcolyer> benburkert: any idea of when 0.9 is going to be released?
00:39 < benburkert> mcolyer: no, there's still a few big pieces left to be done
00:44 < mcolyer> benburkert: so it turns out if I set the class explicitly (using :class) it works. The name has an underscore in it. Does dm know how to camelcase?
00:45 < benburkert> DM 0.9 does
00:45 < benburkert> and snakecase
01:19 < postmodern> how does one perform a join query in datamapper
01:19 < postmodern> can't find it listed in any of the documentation
01:24 < benburkert> postmodern: you'll probably have to write the sql query and execute it
01:44 < benburkert> anyone around with root on the ci box?
03:05 < afrench> how does one ask a class if it's resource (a db table) exists already?
03:05 < afrench> I'm looking for the equivalent to Class.table.exists? but in the newest forma
03:05 * afrench working on merb_datamapper a little bit
03:19 < BrianTheCoder> I can't wait for dm 0.9. Then I can knock out my postgres fulltext search adapter and postgis adapter :D
03:23 < afrench> BrianTheCoder: which fulltext index did you end up writing for? tsearch, by chance?
03:23 < BrianTheCoder> yeah
03:23 < BrianTheCoder> I just wrote a simple module
03:23 < afrench> cool
03:24 < BrianTheCoder> but right now it only can index and search on a single field
03:24 < afrench> pooo
03:24 < BrianTheCoder> but it does index and and trigger an autoupdate
03:24 < BrianTheCoder> well doing multiple fields gets complicated cause there are many ways of doing it
03:25 < BrianTheCoder> you could create an index for each field, or just one
03:25 < BrianTheCoder> and then even in how you structure the search query with coalesce's to handle null values, it will take more time
03:25 < BrianTheCoder> plus its more than what I needed for now
03:27 < afrench> def Post.search(query); Post.all(:conditions => ["body @@ plainto_tsquery(?)", query]); end;
03:28 < afrench> that one line method gets you 3/4s of the functionality you need
03:28 < afrench> it doesn't rank by relevancy, though, which is a little harder
03:34 < postmodern> how would one query models based on some variable that's through a belongs_to
03:34 < afrench> postmodern: which version of DM are you using?
03:35 < postmodern> afrench, dm 0.3.1
03:35 < postmodern> afrench, looking forward to 0.9 though
03:37 < afrench> could you pastie some code so I can see what your trying to do?
03:45 < postmodern> afrench, http://pastie.caboo.se/187473
03:47 < afrench> postmodern: I dunno if you can do that with the quicky key => value sql syntax-sugar.....may be better off doing something like first(:include => [:algorithms], :conditions => ["algorithms.name = ?", algo_name])
03:48 < afrench> you don't need the self.first....the class is understood at that level, I'm pretty sure
03:48 < postmodern> sounds good
03:49 < afrench> in the newest DM, this stuff is very very easy to do with the key => value syntax stuff
03:49 < postmodern> sweet!
03:50 < afrench> I think it's something like: first(self.algorithm.name => algo_name) or something like that
03:50 < afrench> I don't have the specs in front of me for the query path stuff that guyvdb wrote
03:53 < postmodern> when you do the conditions how do you do the named params
03:53 < postmodern> ['SQL', {:name => value}] ?
03:53 < postmodern> 'SQL' being 'field = :name'
03:55 < afrench> I'm not sure you can
03:58 < postmodern> wow reading guyvdb's blog
03:58 < postmodern> very happy to see to_xml functionality
03:59 < afrench> yeah, he's very "context-aware"
03:59 < afrench> ;-)
04:17 < postmodern> can't wait till 0.9, :class => 'Bla' is totally borked
04:23 < BrianTheCoder> afrench: is there going to some version of from?
04:23 < afrench> dunno
04:23 < afrench> I know sam's itching to push 0.9 here real soon
04:23 < BrianTheCoder> so you can pull in a model from xml,json,yaml,etc.
04:23 < afrench> oh, well, there'll be adapters and such that can do that
04:24 < afrench> a model can belong to more than one repository, so you can have a Post class that pulls from a DB and a Yaml file repo if you want
04:24 < BrianTheCoder> cause I had a prob in 0.3 where I stored a models attribs but when I did new(attrs) it returned nil for all the associations
04:24 < BrianTheCoder> well this is model for memcache and queueing
04:25 < afrench> hopefully, with 0.9, people won't need memcached so soon in the project lifecycle
04:25 < afrench> cause most of that stuff'll be handled by the IdentityMap
04:25 < BrianTheCoder> well we're using it to cache complex actions
04:26 < BrianTheCoder> like one where where we had to count recent activity in the past 24hrs
04:26 < afrench> yeah, that's the right way to use it
04:27 < afrench> stuffing an object instance into memcache so it can be retrieved later without a DB hit is the _wrong_ reason to use it
04:27 < BrianTheCoder> gotcha
04:27 < BrianTheCoder> yeah right now I'm caching fragments and slow actions
04:28 < BrianTheCoder> caching the slow actions brought the page load time from 4secs to < 1
04:28 < BrianTheCoder> what's guy's blog url?
04:28 < afrench> http://www.guyvdb.info/
04:29 < BrianTheCoder> afrench: when I try and subscribe to your blog it tries to open the .rss file
04:30 < afrench> yeah, webby's fucked up on my site
04:30 < afrench> dunno what's up with that
04:30 < BrianTheCoder> mmk
04:30 < BrianTheCoder> seen feather?
04:30 < afrench> I got datamapper.org's rss feed to work fine...same config on my site and nothing-doing
04:30 < afrench> fubar
04:30 < afrench> no, what's feather?
04:31 < BrianTheCoder> it's a new blog using merb + dm
04:31 < afrench> link?
04:31 < BrianTheCoder> http://github.com/mleung/feather/tree/master
04:33 < afrench> looks cool
04:34 < BrianTheCoder> yeah
04:35 < BrianTheCoder> plus there's a plugin repo
04:35 < BrianTheCoder> http://github.com/eldiablo/feather-plugins/tree/master
05:51 < wycats_> anyone in here know about the Query object?
05:52 < dkubb> wycats: I do
05:55 < dkubb> benburkert: check this out: http://pastie.org/private/9vaqesnqrgslqlaw2xvg
05:56 < dkubb> this is a slightly different approach to yours, but the idea is to dispatch off to the method named in the first element of the array, and hand the other elements into the method as arguments
05:56 < dkubb> benburkert: this makes the usage of a giant case statement unecessary
05:56 < benburkert> dkubb: cool, is this used by Query?
05:56 < dkubb> benburkert: no, I was just messing around one night, and your pastie reminded me about it
05:57 < dkubb> benburkert: when I created Query originally, I was thinking of using something similar to this to generate the SQL, but since I didn't do that actual query generation (sam beat me to it in DOA) I forgot about it.
05:59 < benburkert> dkubb: i think everyone can agree we need a better way to generate the SQL. The method for generating SELECT statements is almost 100 lines, and is just plain ugly
06:00 < benburkert> also, it was too hard to customize the sql to postgres
06:01 < benburkert> there was a few minor changes to the sql string, but it ended up being easier to rewrite the entire sql generating method
06:02 < dkubb> benburkert: if you can come up with a generic way to model SQL, then I'm for it. I wouldn't necessarily just focus on Sexp or ASTs as the only solution to the problem tho. A set of objects for each type of query may very well be a simpler solution.
06:03 < benburkert> dkubb: i definitly agree. In my head, i was thinking the sexp would be the data structure, then i'd add helper classes or methods to actually manage/generate the sexp
06:03 < benburkert> but maybe ASTs would be more appropriate
06:03 < benburkert> looks more concise
06:04 < dkubb> benburkert: yeah, maybe underneath the hood of a nice object a sexp may be the best way to handle the data side of the model, but it would still be nice to have say a table object, and ask it for its list of columns, and then find out what the names are, or what the actual column's SQL will be for example
06:04 < dkubb> i dunno, just throwing out an example
06:04 < benburkert> i agree 100%
06:05 < dkubb> I'd imagine the SQL generation has alot of similarities, but then each DB (especially for stuff like column types) would have vastly different syntaxes
06:17 < wycats_> hey guys
06:17 < wycats_> hey dkubb
06:17 < dkubb> wycats: hey man
06:17 < wycats_> so tell me about Query
06:17 < wycats_> and links
06:20 < dkubb> wycats: the idea with links is that it will act sort of like :join does in AR's finder. at some point they'll have a property that describes the type of join (inner, left outer, etc), but for not its assumed they are all left outer joines
06:20 < dkubb> joins
06:22 < wycats_> dkubb: "will"?
06:22 < wycats_> how does it work today?
06:22 < dkubb> wycats: will meaning it "will act like this if you used it" :) or at least that was the intention, it could've been altered by sam or people writing the DOA code
06:23 < wycats_> so if I want to build up a Query that represents effectively {|x| x.assoc.bar && x.prop == 17 } what does that look like?
06:23 < dkubb> I questioned sam about this early on, and he wanted it to be a left join operation. I'm giving him the benefit of the doubt, but I think even to support the basic use case ( Zoo.all(Zoo.animals.name => "Lion") ), its going to have to support inner joins
06:24 < wycats_> so x.prop == 17 becomes a condition
06:24 < wycats_> [:eql, <property:prop>, 17]
06:24 < wycats_> what does the assoc become?
06:25 < dkubb> I would write this: X.all(:X.assoc.bar.not => nil, :prop => 17), except I think guyvdb is currently working on making the .not operator work on a Query::Path object
06:26 < dkubb> currently Query::Path doesn't handle any of the symbol Operators
06:26 < dkubb> oops
06:26 < dkubb> forget the colon before the X.assoc.bar.not :)
06:26 < wycats_> dkubb: I know how you'd write it
06:26 < wycats_> what I'm trying to determine is how to build the Query object
06:26 < wycats_> OUTSIDE of .all
06:26 < wycats_> (I'm writing the ambition adapter)
06:27 < dkubb> oh, sweet.
06:27 < wycats_> exactly ;)
06:28 < dkubb> why not translate the ambition query into something acceptable for X.all() to handle, and let it transform it into the stuff Query needs? that would probably be the simplest approach.
06:28 < dkubb> one sec, I'll work this out and pastie you what I think it would be
06:28 < wycats_> w00t
06:28 < wycats_> dkubb: good call
06:28 < wycats_> I was kinda wanting to build the query object but you're right that this is simpler
06:29 < dkubb> in general, I would make it the simplest structure possible, and only build the query object if there wasn't any other choice.. I'll see what I can get here
06:40 < benburkert> wycats_: so will this ambition adapter support complex joins
06:40 < benburkert> that's my only real beef with ambition
06:41 < wycats_> benburkert: why not?
06:41 < dkubb> wycats: there's no easy way to translate this until Query::Path gets the ability to work with not() and other operators
06:41 < wycats_> what does the DM code look like
06:41 < wycats_> dkubb: :(
06:41 < dkubb> wycats: and currently only symbol operators can work with those
06:41 < dkubb> wycats: and symbol denotes a property in the current class, not a property of an association, if that makes any sense
06:41 < benburkert> wycats_: the original ambition, that only worked with AR, could only do a single join in a block
06:42 < wycats_> no reason why that would be
06:42 < benburkert> and it was generating SQL, not AR queries
06:43 < dkubb> wow, its been a while since I looked at the query code. Its gotten alot more complex since I last looked at it, with the addition of Query::Path and some of the other things. probably time for a refactor
06:44 < benburkert> wycats_: so will you allow nested selects: X.select { |x| x.y.select {|y| y.z.name == "joe"} || x.w.select {|w| w.age > 20}}
06:45 < benburkert> or, maybe find() would be more appropriate
06:45 < wycats_> benburkert: it was generating AR hashes
06:45 < wycats_> not SQL
06:45 < benburkert> i just want to use ambition for complex joins
06:45 < dkubb> wycats: in general though, Query is a private class so I wouldn't want ambition to generate queries using it
06:45 < wycats_> dkubb: you up for it?
06:45 < postmodern> weird, i keep getting a DataMapper::Adapters::Sql::Commands::LoadCommand::ConditionsError: DataMapper::Adapters::Sql::Commands::LoadCommand::ConditionsError
06:45 < wycats_> benburkert: those nested joins seem crazy
06:47 < benburkert> this probably makes more sense: X.select { |x| x.y.any? {|y| y.z.name == "joe"} || x.w.any? {|w| w.age > 20}}
06:47 < dkubb> wycats: up for refactoring query? yeah, I can do it, or at least work with you on the ambition adapter to get it functional. I kinda wanted to start work on DM-ambition once DM 0.9's outstanding stuff was squared away
06:48 < mattetti> guys, what was the conclusion of the use_orm :datamapper vs use_orm :dm-core discussion?
06:48 < dkubb> If Query supported OR syntax that would be simpler written as: X.all({ :name => 'join' }.or { :age.gt => 20 })
06:48 < dkubb> mattetti: use_orm :datamapper won :)
06:49 < mattetti> cool
06:49 < dkubb> oh, wait, I see the differences in my query to yours. nvm
06:49 < wycats_> dkubb: awesome
06:49 < mattetti> so to update my system i need to install dm-core + more and use_orm :datamapper, right?
06:50 < dkubb> maybe: X.all({ X.y.name => 'joe' }.or { X.w.age.gt => 20 })
06:50 < benburkert> dkubb: that would be amazing
06:50 < benburkert> i'd kill for something like that :)
06:51 < dkubb> mattetti: yes, if you use edge dm-core, do, and dm-more/merb_datamapper you can use_orm :datamapper
06:51 < mattetti> dkubb: thanks, time to update my merb + dm app
06:52 < dkubb> wycats: where are you working on dm-ambition?
06:54 < wycats_> I didn't REALLY start yet because I realized I didn't understand enough about Query to make a go of it
06:54 < dkubb> wycats: my plan would be to try to get it working with DM, and then refactor/update Query so that ambition will send the arguments to Model.all() using the same public interface that anyone else would use, making it so its not so tightly coupled to how Query works under the hood
06:54 < wycats_> dkubb: good call
06:54 < wycats_> as long as everything can be done through that interface
06:54 < wycats_> benburkert's query might be difficult to represent
06:55 < wycats_> you might need a query object inside the query objet
06:55 < wycats_> object
06:55 < dkubb> Query is one of those things we made a really good go of the first time out of the gate, but there's a few niggling issues. I don't want to be scared to revamp its internals or interface to fix those issues, so I kind of wanted to make it an internal class to DM
06:55 < wycats_> absolutely
06:56 < dkubb> yeah true. I'll start with supporting all the basics, and then get more and more complex with it to see where the limits are. in theory we should be able to integrate very nicely with ambition due to the fact that query objects can be nested
06:57 < dkubb> do ambition's specs give examples of queries that it supports? that would be a good place for me to start work on
06:57 < wycats_> ambition's scaffold generates useful starting specs
06:57 < benburkert> dkubb: afaik, ambition does not support nested blocks
06:57 < wycats_> benburkert: no reason it couldn't
06:57 < wycats_> no reason at all
06:57 < wycats_> you know what just occured to me
06:57 < wycats_> instead of .or
06:57 < wycats_> why not just two hashes
06:58 < wycats_> and is inside one hash
06:58 < benburkert> what about .and?
06:58 < wycats_> or is separate hashes
06:58 < wycats_> so {:foo => "bar", :baz => "bar"}, {:bam => "boom"}
06:58 < wycats_> means foo is bar and baz is bar OR bam is boom
06:59 < dkubb> yeah, that could work. wasn't too fond of adding Hash#or for that alternate syntax
06:59 < benburkert> {{:foo => 'bar'}, {:baz => 'bar'}, {:bam => 'boom'}
06:59 < wycats_> agreed
06:59 < wycats_> benburkert: syntax error
06:59 < benburkert> {{:foo => 'bar'}, {:baz => 'bar'}}, {:bam => 'boom'}
06:59 < benburkert> wouldn't work
07:00 < wycats_> what do you want that to do?
07:00 < benburkert> (foo is bar OR baz is bar) AND bam is boom
07:00 < benburkert> is what i meant
07:01 < wycats_> agreed
07:01 < wycats_> hmmm
07:01 < wycats_> that's tricky
07:01 < Heimidal> [{:foo => 'bar', {:baz => 'bar'}, {:bam => 'boom'}]
07:01 < Heimidal> erm
07:01 < Heimidal> [{:foo => 'bar', :baz => 'bar'}, {:bam => 'boom'}]
07:02 < wycats_> we could use a sexp
07:02 < Heimidal> or is that already an and?
07:02 < benburkert> wycats_: sexp's are best left to the underlying internals, don't you think?
07:03 < wycats_> benburkert: maybe
07:03 < wycats_> the real solution is ambition
07:03 < dkubb> Even I'd prefer Hash#or and Hash#and to a sexp as an interface
07:03 < benburkert> yes, maybe converting ruby's sexp to a sql sexp
07:03 < wycats_> benburkert: can't be sql sexp
07:03 < wycats_> has to be Query sexp
07:03 < benburkert> or general query sexp
07:04 < benburkert> yea
07:04 < wycats_> maybe we should come up with a query sexp
07:04 < wycats_> Query atm is suffering from inadequate API planning
07:04 < benburkert> and give Helper classes/methods for common folk
07:04 < wycats_> if I have to ask people how it works :
07:04 < wycats_> P
07:05 < dkubb> it had a very simple API in the beginning, and then it was extended alot in weird directions to support random features
07:05 < wycats_> the conditions API is trivial
07:05 < benburkert> dkubb: i'm just diving into sexp's, do you know of any good resources on AST's?
07:05 < dkubb> benburkert: no, sorry I don't
07:05 < benburkert> b/c i'm trying to wrap my head around the diffrence
07:06 < wycats_> between?
07:07 < benburkert> sexp's and AST's, they seem to be two different approaches to similar problems
07:07 < benburkert> mixing commands & data
07:11 < wycats_> sexps are much simpler structures
07:11 < wycats_> Lisp is pure-sexp
07:11 < wycats_> AST is in-memory nodes
07:11 < mattetti> newbie question time ;) how do I convert has_many :episodes, :through => :shows to a DM association? (can't find a spec for that)
07:12 < dkubb> mattetti: there's no support for :through yet. ior3k is working on this one right now afaik
07:13 < mattetti> arg, ok :(
07:14 < mattetti> what's the ETA? any idea?
07:14 < wycats_> I thought we had :through
07:15 < dkubb> mattetti: unsure, but I would think its coming this week sometime
07:16 < dkubb> wycats: no, unfortunately not. its a TODO item in "what needs to be done" tho
07:16 < wycats_> heh
07:16 < wycats_> time for bed
07:16 < mattetti> ok, well I guess that means it should be usable in 2 weeks :p pretty cool I'll get started and wait for ior3k ;)
07:18 < mattetti> yay, ior3k770 !
07:18 < mattetti> ahaha
07:18 < mattetti> ior3k770, are you done with :through yet? :p
07:18 < ior3k770> Hey
07:18 < ior3k770> Not even started yet
07:19 < mattetti> I was just asking about :through and I as told you were working on it
07:19 < mattetti> any idea of the ETA?
07:19 < ior3k770> Tomorrow work starts
07:19 < dkubb> mattetti: do you want to take it on?
07:19 < ior3k770> It shouldn't take long, I hope
07:19 < mattetti> dkubb: nah, no time and I'm still learning DM
07:20 < mattetti> I can test it though
07:20 < mattetti> I'm trying to convert small apps of mine from AR to DM
07:20 < benburkert> mattetti: i would implement it in pure ruby for the time being
07:20 < benburkert> *_to_one :show
07:20 < benburkert> def episodes; show.episodes; end
07:21 < mattetti> good point
07:22 < dkubb> def episodes; shows.map { |show| show.episodes }.flatten end
07:23 < dkubb> that is likely closer ^^^^
07:23 < dkubb> the code underscores why :through is so neat
07:23 < benburkert> ahh, it would also be *_to_many then
07:24 < mattetti> yeah, one to many
07:24 < dkubb> because instead of issuing one query for each show, you'd issue single query joining shows and episodes
07:25 < dkubb> when it :through works the syntax will be has n, :episodes, :through => :shows
07:32 < benburkert> dkubb: do you have root access on the ci box?
07:33 < dkubb> benburkert: nope, I wish. those failing CI tasks are bugging me
07:33 < dkubb> 90% of them are due to authentication problems with the DBs
07:34 < benburkert> are they failing b/c the databases haven't been setup, or are there bugs/problems with the adapters?
07:34 < dkubb> they are failing because they can't connect to the databases
07:35 < dkubb> benburkert: do you have access to those boxes?
07:35 < benburkert> dkubb: yes, but not root access
07:35 < dkubb> benburkert: I wonder tho if you have enough access to fix this failure: http://ci.datamapper.org/builds/do_postgres/18e1590cb378b0d...
07:36 < dkubb> (the ...) should be hyperlinked too
07:36 < benburkert> yes, that's annoying
07:37 < dkubb> the errors say "no password supplied". I don't know postgres at all really, but is it possible to make the DB user this runs as have no password?
07:37 < benburkert> hmm, i can get to the postgres command shell without a password: psql do_test
07:38 < postmodern> totally weird, i setup a has_many / belongs_to relation, but the join table isn't being created
07:38 < benburkert> postmodern: you shouldn't need a join table on a has_many / belongs_to
07:38 < ior3k770> dkubb: the specs failing are the DO ones?
07:39 < dkubb> benburkert: the specs are connecting using the postgres user to the do_test database. is this wrong?
07:39 < benburkert> yes, i think the username should be left off
07:39 < dkubb> ior3k770: basically any spec that uses postgres is failing, which includes dm-core, dm-more, and do_postgres
07:39 < benburkert> that way, they it will connect as dmweb, which has read/write
07:40 < ior3k770> I added an environment variable to the postgres adapter specs
07:40 < ior3k770> What OS is it?
07:42 < dkubb> benburkert: I'm going to fix that spec so it doesn't specify a username. let's see if this fixes it
07:42 < dkubb> (for do_postgres)
07:42 < ior3k770> The postgres default user is postgres
07:43 < benburkert> ior3k770: it's not the current user?
07:43 < dkubb> I always thought it was the current logged in user
07:43 < benburkert> what about: "postgres://`whoami`@localhost/do_test"
07:43 < benburkert> err, "postgres://#{`whoami`.strip}@localhost/do_test"
07:44 < ior3k770> benburkert: Yes, it's the current user, I thought you were looking for the root user
07:45 < ior3k770> benburkert: can you su -u postgres?
07:45 < benburkert> i don't know postgres' password
07:45 < dkubb> if I click build now, does it pull down the latest from git and build it, or run with the last checkout?
07:45 < ior3k770> benburkert: how about sudo?
07:46 < benburkert> dmweb is not in sudoers
07:46 < dkubb> nope, that didn't fix it. same error
07:46 < benburkert> dkubb: it might
07:47 < dkubb> benburkert: is it possible for you to run the specs locally (as in on the CI server) to see if they pass for you?
07:47 < benburkert> yes
07:47 < benburkert> i'll try it with dmweb
07:47 < benburkert> same error
07:48 < ior3k770> dkubb: the switch to randomize the tests was a good idea
07:48 < dkubb> ior3k770: it won't work until a future rspec release tho :)
07:49 < ior3k770> dkubb: can't we do it by ourselves? At least at the file level
07:49 < benburkert> this could be a problem with ruby 1.8.5
07:49 < dkubb> ior3k770: yeah, the main reason is just to smack the hands of anyone who tries to write specs that are dependent on other specs.. when they write them and run then, they will fail, thus training them to make then correctly
07:49 < benburkert> hmm, there's two connection string in the spec
07:49 < dkubb> ior3k770: yeah, but I don't think it'll do much. most of the dependencies are between "it" blocks in the same file
07:49 < benburkert> line 11 & 32
07:51 < dkubb> benburkert: I see no problems with the usage of that
07:51 < benburkert> eww, ruby 1.8.5 on 64bit linux
07:51 < benburkert> probably not the best setup for CI :P
07:51 < dkubb> benburkert: the one on line 32 is speccing the connection behavior. The one on line 11 is used in a method that sets up the connection for speccing other behaviours
07:52 < dkubb> benburkert: I always tried to argue that you should use the latest stable versions of stuff
07:52 < benburkert> dkubb: yea, i tried changing it on both, no change
07:52 < dkubb> benburkert: at the very least ruby 1.8.6
07:53 < dkubb> benburkert: can you connect using the dmweb user account, without specifying a password?
07:53 < benburkert> dkubb: i agree, hopefully we can use multiruby to run both
07:53 < benburkert> yes
07:53 < dkubb> wow, that's weird
07:53 < benburkert> psql do_test
07:53 < benburkert> that works fine
07:54 < benburkert> dkubb: the reason i think it's a problem with ruby 1.8.5 is the error message is rather cryptic, which unlike most db connection errors
07:55 < postmodern> benburkert, don't you need a join table to store the corresponding ids?
07:55 < benburkert> postmodern: no, only a foreign key property on the model with the belongs_to
07:55 < dkubb> benburkert: can you run this command from the command-line as the dmweb user and let me know what the output is?
07:55 < dkubb> ruby -e 'require "uri"; puts URI.parse("postgres://localhost/do_test").password.inspect'
07:55 < benburkert> nil
07:55 < dkubb> damn
07:55 < postmodern> benburkert, wouldn't that be a has_one ?
07:56 < dkubb> I was hoping it was something else, pointing to a difference between 1.8.5 and 1.8.6
07:56 < benburkert> postmodern: yes
07:57 < benburkert> URI.parse() works
07:57 < benburkert> dkubb: should the password be "", not nil ?
07:57 < dkubb> benburkert: I think it needs to be nil
07:58 < postmodern> benburkert, ah i see
07:59 < postmodern> benburkert, but after looking at my schema it's dont creating the foreign key
07:59 < postmodern> benburkert, so there's no way the rows can relate to eachother
07:59 < benburkert> postmodern: yea, i think you have to create it manually right now
08:00 < benburkert> postmodern: try adding a property :blah_id, Fixnum
08:00 < dkubb> benburkert: can you pastie the contents of the pg_hba.conf file?
08:00 < benburkert> is that in etc or var somewhere?
08:01 < postmodern> ahh this makes sense now
08:01 < dkubb> benburkert: I have no idea (try locate) I've just found an explanation to the error we're seeing, and it may be due to misconfiguration within that file
08:02 < benburkert> btw, looks like this is postgres 8.2
08:02 < postmodern> benburkert, so belongs_to wont make the blah_id property anymore?
08:02 < postmodern> benburkert, i guess that's to allow more exotic relations
08:02 < benburkert> i get a permission denied when i tried to cat it
08:03 < dkubb> benburkert: ok
08:03 < benburkert> postmodern: i think it just hasn't been finished yet ;)
08:04 < postmodern> haha
08:04 < dkubb> benburkert: this looks like the fix: http://forums.matrix.squiz.net/index.php?showtopic=3902
08:04 < benburkert> yep, that looks like it
08:05 < dkubb> postmodern: could you enter that as a ticket?
08:05 < postmodern> dkubb, will do
08:06 < dkubb> postmodern: I'm pretty sure I know how to make it so belongs_to sets up the blah_id property, but if there's no ticket I'll probably forget to fix it by the time I get to seriously work on DM bugs next week
08:07 < dkubb> benburkert: it shows the solution is to add the "local all all trust" line to the pg_hba.conf. If you talk to ssmoot before I do, would you mind mentioning this to him?
08:08 < benburkert> dkubb: sure, np
08:08 < benburkert> i'm out, later
08:09 < dkubb> I'm out too, later guys
08:21 < postmodern> interesting, when i do another example belongs_to creates the foreign-key property
08:21 < postmodern> but in my code it doesn't
08:28 < postmodern> ok i found it
08:29 < postmodern> when i post my models in a namespace, belongs_to doesn't create the foriegn key
08:34 < postmodern> ticket posted
10:23 < mattetti> guys, does DM automatically update created_at and updated_at on a record update/creation?
10:38 < postmodern> mattetti, it should
10:40 < mattetti> any idea why property :description, Text doesn't work and I have to do: property :description, DataMapper::Types::Text
10:53 < postmodern> mattetti, what version are you using?
10:53 < postmodern> mattetti, im using 0.3.1 and i have to specify :datatype not DataType
10:53 < mattetti> 0.9x edge
10:53 < mattetti> :p
10:53 < postmodern> looks like there's still stuff up with resolving constants
10:54 < mattetti> apparently, that's one of the only types I had to use like that, no big deal though
10:54 < mattetti> btw do you know of an easy way to check if a model is using DM or AR?
10:57 < postmodern> perhaps using kind_of?
10:57 < mattetti> Show.included_modules.select{|m| m == DataMapper::Resource}
10:58 < postmodern> kind_of?(ActiveRecord::Base)
10:58 < mattetti> aha NameError: uninitialized constant ActiveRecord
10:58 < mattetti> I first need to check if it's defined
10:59 < mattetti> anyway, Dm 0.9 uses a module
10:59 < postmodern> var.class.ancestors.map { |c| c.name }.include?('ActiveRecord::Base')
11:00 < mattetti> that works for AR but not DM :(
11:01 < postmodern> you could do an if/else block with those
11:01 < postmodern> i suppose i should install 0.9.x
11:02 < postmodern> and start moving to it
11:02 < postmodern> how stable is it currently?
11:02 < postmodern> would you be able to write a blogging database with it
11:02 < mattetti> I never used 0.3.x so I can't compare
11:03 < mattetti> I think you would, but I just started using it recently
11:03 < mattetti> 0.3 is not really supported anymore
11:03 < mattetti> and 0.9 release is getting really close
11:03 < mattetti> Show.included_modules.select{|m| m.to_s == 'DataMapper::Resource'} works fine
11:04 < postmodern> could make an extension to class Object
11:04 < postmodern> def self.is_datamapped?
11:05 < mattetti> good point
11:07 < mattetti> how do you do something like: channel.shows << Show.new()
11:09 < postmodern> i guess setup a has_many :shows
11:09 < postmodern> and belongs_to :channel
11:09 < postmodern> then do channel.save
11:09 < postmodern> or maybe channel.shows << Show.create()
11:09 < postmodern> idk
11:09 < mattetti> let me try
11:10 < postmodern> brb
11:14 < mattetti> yep that worked, for some reason I didn't expect it to work :p
12:14 < neaf> Hi, automigrate doesn't work for me in dm0.9 beacouse of
12:14 < neaf> uninitialized constant DataMapper::Persistence, so how can I create model's table?
12:17 < slurry> neaf: didn't DataMapper::Persistence change to DataMapper::Resource or something?
12:19 < neaf> slurry: Resource I think.
12:21 < neaf> But as I see automigrations are removed from dm ( http://github.com/sam/dm-core/commit/4337a3a138b6912e15018d1b4102c913f6455ccf ) and what is right way to do it now?
12:44 < candyco> neaf: new automigrations are being worked on and should be ready soon I believe
12:57 < neaf> candyco: Thanks :)
15:01 < Somebee> is EmbeddedValue-stuff working in 0.9.0 yet?
15:12 < ssmoot> Anyone familiar with Ruby/C? I need to get at ENV['USER'] in do_postgres.c, but I'm not sure if there's a short-cut to it, or a standard C function that gets me the same thing?
15:13 < ssmoot> zond: ? wayneeseguin? ^^^
15:13 < wayneeseguin> hmm
15:14 < zond_hme0> ssmoot: pong
15:15 < ssmoot> I mean, like there's rb_mKernel for example. I forget where to find all those declarations without looking at the source tho'.
15:16 * wayneeseguin reaches back in his memory
15:17 < ssmoot> :)
15:19 < wayneeseguin> try this
15:21 < wayneeseguin> and there IS a faster way to this I'm just rusty
15:21 < wayneeseguin> first grab the env constant: ID env_class_id = rb_intern("ENV");
15:21 < wayneeseguin> VALUE env = rb_const_get(rb_cObject, env_class_id);
15:21 < wayneeseguin> then grab the string
15:21 < wayneeseguin> VALUE env_user = rb_hash_....
15:21 * wayneeseguin goes to look up the hash methods
15:22 < ssmoot> yeah, I know I could do that, and normally I would, but it seems wrong to take the shortcut just because I don't know how to lookup the already defined interface. ;)
15:22 < ssmoot> at least I'm guessing there's an already defined interface... :/
15:22 < wayneeseguin> http://www.ruby-doc.org/doxygen/1.8.4/group__ruby__hash.html
15:22 < wayneeseguin> did I misunderstand your question?
15:22 < zond_hme0> ssmoot: try getenv
15:22 < ssmoot> Is there any class that might use an ENV var? maybe a platform?
15:23 < zond_hme0> stdlib.h
15:23 < zond_hme0> has it
15:23 < zond_hme0> ssmoot: but why do you need it in do_postgres.c?
15:23 < ssmoot> wayneeseguin: no, you understood perfectly. I'm just trying to figure out the "right" way to do it to buff up my poor C skills. ;)
15:24 < wayneeseguin> ahh ko :)
15:24 < wayneeseguin> I have the same question as zond
15:24 < zond_hme0> ssmoot: the postgres client lib does it by default with the current user
15:24 < ssmoot> zond_hme0: I figured that the default behaviour for psql is to connect as the current user, so it makes the most sense for do_postgres to connect as ENV['USER'] when no username is specified, as in: 'postgres://localhost/do_test'
15:25 < zond_hme0> ssmoot: but that is already the case
15:25 < ssmoot> d'oh :o
15:25 < zond_hme0> ssmoot: if you give NULL as the username, it will use the current user
15:25 < wayneeseguin> LOL
15:25 < ssmoot> :D
15:25 < zond_hme0> and that is why i patched it like that :)
15:25 < wayneeseguin> learning FTW!
15:25 < ssmoot> well then... thanks. ;)
15:25 < zond_hme0> np :D
15:26 < ssmoot> I'm trying to figure out why http://ci.datamapper.org/builds/do_postgres/7d6de257c419970... is failing :/ does it need an empty String for the password instead of NULL maybe?
15:26 < zond_hme0> nope
15:26 < zond_hme0> its probably because the pg_hba.conf on the ci server isnt configured like it should
15:27 < zond_hme0> you need to trust users connecting from 127.0.0.1 as the test user implicitly as being the test user even without password
15:27 < ssmoot> but "psql -U dmweb do_test" works fine in the shell?
15:27 < zond_hme0> but try
15:27 < zond_hme0> psql -h 127.0.0.1 -U dmweb do_test
15:27 < zond_hme0> it will probably fail
15:27 < zond_hme0> unix socket vs tcp socket
15:27 < zond_hme0> the test uses tcp socket to be platform independent
15:28 < ssmoot> It asked for a password :/
15:28 < zond_hme0> exactly
15:28 < zond_hme0> could you pastie the pg_hba.conf on that machine?
15:28 < zond_hme0> i can edit it ftw
15:29 < ssmoot> So how do we make do_postgres use the socket if the host is localhost?
15:29 < ssmoot> zond_hme0: sure.
15:30 < candyco> i get about 5 people a day asking why automigrate isn't working in the merb plugin and I'm getting kinda bored explaining that it's on the way. I was going to change the rake task to print a message about it rather than throwing the very unhelpful exception that it does now. does that sound reasonable?
15:30 * ssmoot pasted http://pastie.textmate.org/private/jxjgl03mawtre9znamktxw
15:30 < zond_hme0> use the socket? you mean unix socket? is that wanted? i find it problematic if we have a feature requirement on unix but not windows (like being able to use unix socket) - its not really necessary in unix either, since unix too support tcp :) but i dunno, perhaps there are usecases for unix socket that i dont knot about
15:30 < ssmoot> zond_hme0: I think it's on the bottom ^^^
15:31 < zond_hme0> candyco: i feel it sounds very constructive - each question not asked gives you more time to fix the actual problem :)
15:31 < zond_hme0> ssmoot: k
15:31 < ssmoot> zond_hme0: ah, well, socket access is *a lot* faster for mysql. dunno about postgres, but I'd assume it's the same. I guess it's a tweak that can be made later.
15:32 < ssmoot> candyco: yeah that sounds good to me.
15:32 < ssmoot> candyco: btw... I think benburket was working on it as well? Are you guys talkin?
15:33 < zond_hme0> ssmoot: ah, well, then perhaps we should make it behave like default 'psql-binary-behaviour' there as well - which is 'localhost' => unix socket, '127.0.0.1' => tcp socket
15:33 < candyco> ssmoot: I'm not doing any work on automigrations. just the merb_datamapper plugin part
15:34 < ssmoot> zond_hme0: yeah, that'd be nice... but on second thought, you've made me rethink it... it can be a tweak for later. :) just want to get the ci passing...
15:34 < ssmoot> candyco: ah, ok.
15:34 < zond_hme0> ssmoot: i would guess that http://pastie.textmate.org/private/wfhtp0s19ztogvufiqrmlw could work
15:39 < zond_hme0> ssmoot: it would be very simple to make it connect through socket - just give NULL as host in the PQsetdbLogin
15:39 < zond_hme0> (unix socket, that is)
15:39 < ssmoot> zond_hme0: ah, sweet.
15:40 < ssmoot> zond_hme0: the dmweb/trust thing seems to have worked for psql, but not the build, so I'm restarting the ci...
15:41 < zond_hme0> ssmoot: weird - could you connect using irb? just c = DataObjects::Postgres::Connection.new("postgres:///do_test") (run as dmweb)?
15:41 * ssmoot pasted http://pastie.textmate.org/private/dc0hpbbikypwufkukxoaa
15:42 < ssmoot> zond_hme0: so change ^^^ to host = NULL and it'll "just work"?
15:42 < ssmoot> zond_hme0: actually, be back in 15. my wife just brought me breakfast. :D
15:56 < candyco> zond_hme0: I have a failing mysql spec in dm-core. it appears to be trying to connect to mysql as the currently logged in user. has something changed or is that correct?
15:57 < zond_hme0> candyco: no, i (and perhaps others) have made the mysql specs act like the postgres, namely trying to connect as current user
15:58 < zond_hme0> candyco: perhaps that is not the wanted behaviour? but it make the db setup quite uncomplicated
16:00 < candyco> zond_hme0: OK, that's fine I'll try setting up a user now. I think it makes a lot of sense. (but mysql devs tend to be lazy and just stick with root users on their local machines so that is likely to be a common point of failure)
16:01 < zond_hme0> candyco: yes, well, perhaps we need some kind of policy decision about that?
16:01 < candyco> zond_hme0: it would be nice if you could run the specs just limited to one adapter as not many end users will have all 3 DB's installed.
16:02 < dkubb> zon_hme0: last night benburkert and I were trying to get the do_postgres specs passing. I made it so they were connecting as the current user, rather than "postgres". Since the fix didn't work, I reverted it back to "postgres". Do you think the specs should always connect using the same username as the logged-in user?
16:02 < zond_hme0> candyco: that was the case a few weeks back - if a do_* gem was missing it just skipped that set of specs
16:03 < zond_hme0> candyco: i made the mistake of trying to build all three do_* gems, and noticed that all hell broke loose if i did, and fixed a few errors, then others fixed even more errors with multiple repos, so maybe that is why the specs now dont just skip running certain adapter specs
16:03 < dkubb> zond_hme0: ^^^^ I got your username wrong above :)
16:03 < zond_hme0> dkubb: i saw it, just havent had time to read it yet :)
16:04 < zond_hme0> dkubb: the thing is - from the beginning they connected as postgres
16:04 < zond_hme0> dkubb: then i made them connect as current user
16:04 < dkubb> right now if the do driver isn't installed, it will fail the associated dm-core integrations spec. I added that, although now I think maybe it should just warn and skip it
16:04 < zond_hme0> dkubb: then someone (dont remember who, check the logs) changed it back again!
16:04 < zond_hme0> dkubb: then you changed it to current user again...
16:04 < dkubb> zond_hme0: haha
16:05 < dkubb> zond_hme0: we should just decide, and then put a comment above them saying "DO NOT CHANGE THIS"
16:05 < zond_hme0> yup
16:05 < zond_hme0> dkubb: i prefer current user
16:05 < candyco> zond_hme0: they might still be working I have all 3 adapters installed anyway. just i couldn't see any way of explicitly telling it what adapters to test against so assumed it wasn't possible. Thanks for the explanation though I think thats fine.
16:06 < dkubb> one thing we could do is make it so the URL uses ENV['USER']. That way if someone wanted to run it as a specific user, they could set the USER env variable and run the script
16:06 < candyco> dkubb: I think it makes sense to standardize them all - current user probably makes most sense
16:07 < zond_hme0> yes, doing it as postgres with or without a password has some bad implications
16:07 < zond_hme0> you will, for example, never be able to run the tests properly on a production machine before deployment
16:08 < zond_hme0> but yes, allowing those that dont want to use current user without password to connect with environment-set user and password could be a good solution
16:08 < zond_hme0> like
16:08 < dkubb> would a properly configured production machine require a password for the postgres user?
16:08 < zond_hme0> ENV["SPEC_DB_USER"] and ENV["SPEC_DB_PASSWORD"]
16:08 < zond_hme0> dkubb: probably! since the postgres user is the superuser of the db
16:13 < dkubb> zond_hme0: BTW: I typically will use unix sockets for all applications that support it, in cases where I have services that aren't used by outside resources, and configure the app not to listen on a TCP socket at all
16:13 < dkubb> zond_hme0: I try to have the very minimum of services running on TCP ports by default when setting up a server
16:14 < zond_hme0> dkubb: sounds reasonable
16:15 < zond_hme0> if you did like ssmoot suggested, setting host = NULL instead of "localhost" in the initialize method, then it would be unix socket if you typed postgres:///do_test, and tcp socket if you typed postgres://localhost/do_test - if that is the desired behaviour, then its a simple fix :)
16:15 < dkubb> zond_hme0: that means a simpler firewall configuration, since I can safely block all traffic in/out from all ports, aside from a few specific ones (80, 443, etc)
16:15 < zond_hme0> dkubb: yes, but can still block everything not on the lo device!
16:15 < zond_hme0> :)
16:15 < zond_hme0> i usually allow everything on dev lo
16:16 < dkubb> zond_hme0: yeah, that's true. I also do allow everything on the loopback usually too.
16:16 < edas> hi
16:16 < zond_hme0> edas: hello
16:16 < zond_hme0> dkubb: anyway, i have to leave now
16:16 < zond_hme0> seeya guys
16:16 < zond_hme0> :)
16:17 < dkubb> see ya
16:18 < edas> I have one question I didn't found the answer on doc : when I don't use an object, is it still in memory because of the identity map ? or does the identity map is using weak references ?
16:19 < dkubb> edas: are you using DM 0.3 or 0.9?
16:19 < ior3k> dkubb: do you have any idea why Query::Path.new takes a property name as its last argument? I searched through the code and didn't find it used anywhere
16:20 < dkubb> ior3k: I don't know. I asked guyvdb about it. I think its probably a bug and should be changed to accept the model as the last argument
16:21 < ior3k> dkubb: the model is the penultimate argument, so I'm tempted to just take it away. What do you think?
16:21 < edas> dkubb: nothing for the moment, I am studying if it can fit my needs/wishes
16:21 < edas> dkubb: wich way des the last version use ?
16:21 < ior3k> dkubb: I need to change Query::Path for use with :through
16:25 < dkubb> edas: in DM 0.9, if you wrap the call in a repository block, the Identity Map will remain in scope for as long as the block. Once the block ends the DM::Repository object goes out of scope and takes the Identity Map with it
16:26 < dkubb> ior3k: I say go for any modifications you need, as long as the specs pass
16:26 < edas> dkubb: not as good as weak references but it should be ok for me, thank you
16:27 < ior3k> dkubb: yeah, but I don't need to remove that property_name from Query::Path. It just seems to be sitting there doing nothing
16:27 < dkubb> edas: we tested out weak hashes, but weren't happy with how any of them performed.. not from a performance point of
16:27 < dkubb> view, but many of them didn't GC when we thought they would
16:28 < dkubb> edas: with long running stuff like a merb app, we were pretty worried about stuff not getting GC'd properly
16:28 < edas> ok
16:28 < dkubb> ior3k: if you remove it, and the places that pass it in, do all the specs still pass? If so, I'd say keep it out. The simpler the implementation the better
16:30 < ior3k> dkubb: there are no places that pass it in, at least in dm-core
16:30 < ior3k> dkubb: hmmm, maybe there's something using it in dm-more...
16:30 < dkubb> ior3k: yeah, now in general when you make changes to dm-core, you need to install it and run the specs for dm-more now
16:33 < dkubb> ssmoot: hey
16:33 < ior3k> dkubb: oh, one other thing. Don't take this the wrong way, but don't you think those type checks are a bit useless with proper speccing?
16:33 < ssmoot> heyas
16:33 < dkubb> ior3k: yes I do
16:33 < ssmoot> so 15 was a bit of an exagerration. ;)
16:33 < ssmoot> type checks?
16:34 < dkubb> ior3k: they are more like bootstrapping until the spec coverage was up
16:34 < zond_hme0> dkubb: how do you even create weak refs in ruby? :)
16:34 < zond_hme0> :O even
16:34 < ssmoot> zond_hme0: so host = NULL?
16:34 < dkubb> ior3k: although, on public interfaces I still think they serve a purpose
16:34 < zond_hme0> i didnt know it was possible?
16:34 < zond_hme0> ssmoot: well, if you change the specs at the same time, so they use //localhost/ instead of ///, otherwise they will change behaviour
16:34 < dkubb> zond_hme0: WeakRef from ruby stdlib is one way
16:34 < ssmoot> er... weakrefs are mostly useless in Ruby.
16:34 < zond_hme0> dkubb: oh, didnt know about that :)
16:35 < dkubb> zond_hme0: yeah, I wouldn't rely on them, heard lots of problems with them
16:35 < ssmoot> If I had properly imported the history of DM into github.com/sam/dm-core, you could explore the sordid history of WeakRefs and WeakHash and DM. ;)
16:35 < zond_hme0> haha
16:36 < ior3k> dkubb: for the public interface, maybe (although I believe in duck typing all the way), but internally they sometimes get in the way
16:37 < ior3k> I see a few places where I'd like to tidy up things a bit, but I'm going to leave that until after :through is done
16:37 < ior3k> and many to many
16:37 < zond_hme0> i just found a post about weakrefs
16:37 < ssmoot> Ruby's GC doesn't appear to be time-boxed *at all*. So without memory thresholds being passed, objects will *never* be collected. So WeakRefs basically just become really expensive, and don't really do anything for you. You almost have to manually manage dereferencing objects the way Repository goes out of scope to ensure you don't have stale objects.
16:37 < dkubb> ior3k: yeah, we could go with duck typing once the code is more or less complete. As of right now we don't know all the methods that the objects need to support internally, so I went with the simplest approach, just to check the types that were passed in
16:37 < zond_hme0> where the writer told about his success in using object ids instead of weakrefs
16:38 < ssmoot> zond_hme0: that's what WeakHash does, but the real hole is in Ruby's GC, so the finalizers never get called.
16:38 < zond_hme0> ssmoot: but if letting something go out of scope gc's it, why wont a weakref do the same thing? i thought weakrefs were just objects that were out of scope from the start?
16:40 < ssmoot> zond_hme0: it doesn't GC it directly. It just ensures that you won't have concurrency issues with stale objects across processes because you ensure that you are frequently getting fresh references from a synchronized resource (the database).
16:41 < ssmoot> This reminds me I need to respond to the Repository patch on the ML...
16:41 < zond_hme0> ssmoot: but i mean, if you had memory problems with weakreffed objects not being gc'd, wouldnt you now have the same problems with the identity maps being a memory problem?
16:42 < zond_hme0> and if you DO have the same problems with IM now, wouldnt it be a nice improvement to use weakreffs instead? (im not objecting to your not using weak refs, i just dont fully understand why :)
16:42 < ssmoot> zond_hme0: it's not memory problems that we care about exactly. Ruby will GC the stuff eventually. It just won't do it until it passes thresholds.
16:42 < zond_hme0> ssmoot: so what exactly was the problem with weak refs?
16:42 < ssmoot> zond_hme0: So the real problem is stale objects. So that's why we let the IM go out of scope for every repository { } block.
16:43 < zond_hme0> ssmoot: ok, so its if you run several processes and they make each others IMs stale
16:43 < zond_hme0> ssmoot: but that problem will STILL exist as it is now (albeit in lesser form)?
16:43 < ssmoot> zond_hme0: the problem was that if the active memory didn't get pushed up, the objects wouldn't be finalized. So we'd have stale objects in different processes.
16:43 < zond_hme0> (if you run dm in several processes)
16:44 < ssmoot> zond_hme0: no... here, I have a sad tale to tell. ;)
16:44 < ssmoot> zond_hme0: Once upon a time there was a blog: http://substantiality.net running on SimpleLog blog engine.
16:44 < zond_hme0> ssmoot: pushed up? i dont understand :O
16:45 < ssmoot> zond_hme0: I replaced AR with DM on the blog.
16:45 < ssmoot> zond_hme0: I was running 2 mongrels for the blog.
16:46 < ssmoot> zond_hme0: Request was made for "/" to list the posts on mongrel1. I posted a comment on an article. The comment count incremented.
16:46 < ssmoot> zond_hme0: But that POST request had gone to mongrel2. The comment count incremented on mongrel2 when "/" was hit.
16:47 < ssmoot> zond_hme0: But on mongrel1, the Article objects were already in the Adapter#identity_map (that never went out of scope, and was WeakHash based).
16:48 < zond_hme0> ssmoot: problematic eh :) but you can still in get that problem with the current implementation (that you get stale objects in IM, if two requests happen simultaneously...), albeit they wont manifest as badly
16:48 < ssmoot> zond_hme0: So the comment-count was inaccurate on one. You couldn't view the new comments on mongrel1 because Article[1].comments was already in the IM and the only thing that would get it to show up was before_filter { database.adapter.identity_map.clear! }
16:49 < ssmoot> zond_hme0: no, because the IM now is tied to the Repository, which by default goes out of scope immediately. If you do something like: around_filter { |action| repository { action.call } } then the widest scope is per-request.
16:49 < zond_hme0> yup
16:50 < ssmoot> zond_hme0: So you wouldn't have stale objects. By default, or explicitly per-request, either way, you're going to get the latest objects for every request.
16:50 < zond_hme0> and if two requests happen simultaneously, you would have two articles, one in each IM, with the same article count. then you would post two articles at the same time in each request, and both would increment the count, and it would be saved, and you would have missed one article in the article count
16:50 < Somebee> Have you thought about making a DataMapper::Types::Boolean? TrueClass/FalseClass is a little unintuitive...
16:51 < ssmoot> zond_hme0: and since multiple requests will stack up dereferenced IMs and model instances in the ObjectSpace, the memory will grow from 30MB to 35MB after a few requests, passing a threshold, and causing Ruby to GC/cleanup the old dereferenced objects.
16:52 < zond_hme0> ssmoot: yes, but during this time, when the two requests happen, wouldnt you have two IMs with a copy each of the article?
16:52 < zond_hme0> ssmoot: and wouldnt they both increment their article count with one, and then save?
16:52 < dkubb> ssmoot: what do you think about moving DataMapper::Types to dm-more/dm-types ? If they were in a separate project I'd submit a ton of custom data-types, like Email, URI, Money, etc. Since they are in dm-core, I don't want to clutter it up with custom data-types.
16:52 < ssmoot> zond_hme0: that's only a hole if you use a counter_cache or something.
16:53 < ssmoot> dkubb: that sounds like a good idea to me.
16:53 < zond_hme0> ssmoot: ok, so stale objects within a request context is acceptable
16:54 < zond_hme0> ssmoot: i actually tend to agree
16:54 < ssmoot> Somebee: we could yeah. Just a custom-type that aliases TrueClass basically. class Boolean < DM::Type; primitive TrueClass; end. That's it.
16:54 < dkubb> ssmoot: ok, will do
16:54 < zond_hme0> ssmoot: simultaneous requests overwriting each other is kind of a known and accepted problem
16:54 < ssmoot> zond_hme0: Well they wouldn't be stale. You'd just have a classic concurrency issue trying to do something like a counter_cache. Which should employ optimistic-locking to prevent.
16:55 < zond_hme0> ssmoot: just doing it within a transaction (go me!) block would be sufficient actually
16:55 < zond_hme0> ssmoot: and since at least postgres has serializable (which acts a lot like optimistic) isolation, that is basically what you said :)
16:55 < Somebee> ssmoot: yep, I think a Boolean type like that would be good to have (in dm-core). And I agree with dkubb, it would be very cool to have dm-types in dm-more, with tons of different types
16:55 < ssmoot> zond_hme0: It's not really an issue with stale objects. It's more an issue of pulling from the DB, incrementing a value, and saving the new count out. You can have that sort of issue without an O/RM using raw database drivers, and it's why we need optimistic locking.
16:56 < Somebee> ssmoot: do you plan on implement EmbeddedValue in 0.9.0?
16:56 < dkubb> Somebee: I would think something like that would be perfect for a dm-more plugin. Are you interested in taking it on?
16:57 < zond_hme0> ssmoot: do you think it would be possible to make the identity map implementation pluggable?
16:57 < ssmoot> Somebee: I think not. 0.9.0 needs to get out ASAP. The whole plan is quick iterations, and if we don't start releasing we'll never get on that track. So I think as soon as has_many:through is done, we should release.
16:57 < ssmoot> 0.9.0 isn't meant to be perfect. It's more like a series of ReleaseCandidates.
16:57 < Somebee> ssmoot: I see
16:58 < zond_hme0> ssmoot: because if it was, it would be possible to create a weak-ref IM for those users that want to run just one huge DM instance on one huge machine, with lots of threads (yey, take advantage of thread safety!), and get a very very fast data access with a wholly caching IM
16:58 < Somebee> dkubb: I'd be more than happy to contribute something like that. Just have to mess around the source a little more, and see if I can get to know your baby :-)
16:58 < dkubb> ssmoot: there's an implementation of Transactions now in dm-core (thanks zond_hme0!), but I think there's a bit of overlap with what was done alot ealier in DO. when you get a second, would you be able to look at the implementations and see if we can remove the duplication and/or unify the efforts. My guess is you'll be ripping it out from the DO side, and using what's in dm-core
16:58 < ssmoot> zond_hme0: Well... It would be possible. I don't think it's something that we want to do really. I think SecondLevelCaches will suit 99% of use-cases.
16:59 < zond_hme0> dkubb: there is no duplication in do, afaik
16:59 < zond_hme0> dkubb: i removed a few transaction things from do when i build my transactions
16:59 < Somebee> dkubb: but so far I love most of what I see, and I'm only guessing it will get even better as documentation (merbstyle(TM)) improves. AR-source vs DM-source is like night and day :D
16:59 < zond_hme0> ssmoot: problems with second level caches is usually that the objects dont
16:59 < zond_hme0> 'live', in them
16:59 < ssmoot> zond_hme0: but you're right... no harm I suppose in have the IM instantiated through Adapter#new_identity_map, and then you could just overwrite it to do whatever...
16:59 < zond_hme0> a lot serialization going on
17:00 < ssmoot> dkubb: k. yeah, I saw that the DO transactions went MIA.
17:00 < zond_hme0> ssmoot: cool, perhaps you could even have a default class that it instantiates, that you can change in init or sth
17:01 < zond_hme0> dkubb: the do transactions were basically only implemented in mysql (perhaps sqlite too), and werent used anywhere at all
17:01 < dkubb> zond_hme0: there's stuff in dataobjects related to transactions that should either be removed, plus there was some old C code related to transactions. I'd prefer to have no duplication between dm-core and do
17:01 < ssmoot> zond_hme0: serialization: right. That's why I've been against some of the suggestions, like using MemCache for an IM. It'd kill performance. It needs to be an SLC.
17:02 < zond_hme0> SLC?
17:02 < ssmoot> zond_hme0: SecondLevelCache.
17:02 < ssmoot> zond_hme0: or SaltLakeCity I guess. Take your pick. :)
17:02 < zond_hme0> dkubb: it was only the c code i removed, what other code regarding transactions are there?
17:02 < zond_hme0> ssmoot: ok, i feel a misunderstanding, what is a second level cache then?
17:03 < zond_hme0> zond: i thought memcache was one
17:03 < zond_hme0> haha
17:03 < dkubb> zond_hme0: do into do/dataobjects and search for "transaction" case insensitively, you'll find at least one reference
17:03 < zond_hme0> ssmoot: ^^^ wrong target there :)
17:03 < ssmoot> zond_hme0: http://github.com/sam/dm-core/tree/master/spec/unit/identity_map_spec.rb#L66
17:03 < zond_hme0> dkubb: oooh, i just looked in the c part. ill look into it
17:03 < dkubb> zond_hme0: a second-level cache is a cache the IdentityMap checks if there is a cache miss
17:04 < zond_hme0> dkubb: aye, but is it typically containing serialized objects? or is it some other magic?
17:04 < dkubb> zond_hme0: so if the object isn't cached in memory, it can check something else to see if it contains it.
17:04 < zond_hme0> dkubb: so you mean use memcache for SLC then?
17:04 < ssmoot> zond_hme0: dkubb: I think transactions should still be a part of DO generally.
17:05 < dkubb> zond_hme0: yes, memcache would be perfect for a SLC
17:05 < ssmoot> zond_hme0: dkubb: So we have our own in DM. Maybe even 2PC eventually. That's great. But DO is general database drivers, and I'd like to see them grow in a way that makes sense for them, not necessarily ruled by DM concerns.
17:05 < zond_hme0> dkubb, ssmoot: well, then i understood perfectly - what i meant then was that if you had a single-instance-multi-thread server for DM, you could use IM as the SLC, and get blazing performance
17:06 < dkubb> ssmoot: yeah, that's what I figured. so if it can be done in DO, then we should do it there, and just provide the interface
17:06 < zond_hme0> ssmoot: it seems logical to have the actual *_transaction methods in do, right now they are in the dm adapters
17:06 < dkubb> ssmoot: to our dm-core stuff. I'm ok with that
17:06 < ssmoot> zond_hme0: right, that makes sense (IM as SLC).
17:08 < ssmoot> cool. so back to getting CI working... ;) zond_hme0: host=NULL?
17:08 < zond_hme0> ssmoot: but some parts of the transaction logic needs to be in dm, to make the transactions aware of what a Resource is, etc
17:08 < zond_hme0> ssmoot: ill look it over, perhaps we can have a seriously smaller transaction class in dm-core, and have most of the muscle in do
17:09 < dkubb> ssmoot: I simplified the IdentityMap a bit, so making a memcache SLC would be trivial. The interface is actually drop-in compatible with the memcached ruby gem
17:09 < zond_hme0> ssmoot: well yes, if you change the specs so that they use postgres://localhost/ instead of just postgres:///, otherwise the specs would suddenly start using unix sockets, which would make luislavena write contracts on our heads
17:09 < zond_hme0> dkubb: you should look at the caffeine-gem
17:10 < zond_hme0> dkubb: it is a 100% c ruby client to memcache
17:10 < ssmoot> zond_hme0: right. Well to me, DO exposes database level transaction interfaces. DM has another. They both probably end up with their own *::Transaction classes. DM's might eventually support 2PC. DO's probably won't. DM's will wrap DO's in DOA to a certain extent, but they aren't mixins or anything. In DO you deal with DO::Transaction with a specific interface. In DM you deal with DM::Transaction.
17:10 < zond_hme0> dkubb: its a hell of a lot faster than the memcache gem
17:10 < dkubb> zond_hme0: last I looked caffeine is slower than the memcached gem by evan weaver
17:10 < zond_hme0> ssmoot: DM
17:10 < zond_hme0> ssmoot: asdf
17:10 < ssmoot> dkubb: spiffy.
17:10 < zond_hme0> ssmoot: DM's actually supports 2PC _now_
17:10 < dkubb> zond_hme0: that's memcached (with a "d").. a newer gem
17:10 < zond_hme0> dkubb: ah, the new memcache gem in c?
17:11 < zond_hme0> dkubb: i have been away from memcache for too long :)
17:11 < zond_hme0> dkubb: well, i wonder if the memcached gem does 'namespace' invalidations, i suggested that earlier as an improvement to steal from caffeine
17:11 < ssmoot> zond_hme0: DM+2PC? srsly???
17:11 < zond_hme0> dkubb: with a single request you can invalidate a gazillion memcache entries
17:11 < ssmoot> that's kinda crazy...
17:11 < zond_hme0> ssmoot: yes, i built it this friday
17:12 < ssmoot> wow...
17:12 < ssmoot> zond_hme0: so about postgres:///. I don't understand... I thought the specs were using postgres://localhost/do_test ? why would they use postgres:/// ?
17:13 < dkubb> zond_hme0: http://blog.evanweaver.com/articles/2008/01/21/b-the-fastest-u-can-b-memcached/
17:13 < ssmoot> oh noes...
17:13 < ssmoot> they're using: "postgres://postgres@localhost/do_test"
17:13 < zond_hme0> ssmoot: i think some use postgres:///, which (as the extension works now, connects to localhost)
17:13 < ssmoot> that's no beuno.
17:13 < zond_hme0> ssmoot: they have been changed back and forward a lots of times
17:14 < ssmoot> so probably just take out the user and our CI will start to "just work".
17:14 * ssmoot goes to try it...
17:14 < zond_hme0> ssmoot: perhaps :)
17:14 < dkubb> ssmoot: last night I removed the postgres user from that, but it didn't fix CI, so I moved it back because I thought it was done that way on purpose
17:14 < dkubb> ssmoot: have you configured postgres so it allows trusted access from localhost?
17:15 < ssmoot> dkubb: for the dmweb user yes. just earlier.
17:15 < dkubb> ssmoot: sweet, then do_postgres and dm-core should both pass
17:15 < ssmoot> zond_hme0: so it's safe to make the host=NULL change as well right? that's not going to break anything?
17:16 < zond_hme0> dkubb: yup, thats me commenting in that thread :)
17:16 < zond_hme0> ssmoot: well, it will change the behaviour of the extension, as i said
17:16 < zond_hme0> ssmoot: but not in a mind-dazzling manner
17:17 < zond_hme0> ssmoot: just that not giving any host will mean unix socket instead of tcp socket
17:17 * ssmoot pasted http://pastie.textmate.org/private/c85qahwupi9xpeidmblzwq
17:17 < ssmoot> zond_hme0: that look right? ^^^
17:17 < edas> Is there some work done somewhere to integrate DM in Rails ? (deactivate AR initialisation and put DM one instead) ?
17:17 < dkubb> zond_hme0: I do agree with his approach of staying close to the metal of what memcache does, and then allowing libraries to be built on top to allow branch invalidation
17:17 < ssmoot> edas: DM 0.9.0 should "just work". You require it. You add a DataMapper.setup for your connection, and that's it...
17:18 < ssmoot> edas: but I haven't tried it personally so no guarantees.
17:18 < dkubb> ssmoot: you wouldn't be able to specify repositories in database.yaml tho, so someone should make a rails DM plugin to make this part dead simple
17:19 < dkubb> ssmoot: merb_datamapper allows you to parse a slightly different database.yaml file, where you can specify repositories, and nest the database info underneath each repo block.
17:19 < dkubb> ssmoot: although I personally prefer to setup DM using the environment ruby files, I can see how some people would want to use the yaml files
17:19 < ssmoot> dkubb: that's true. Personally, I don't like database.yml tho', so probably not me. ;-) I think people should just use the config/environments/*.rb files to setup their connections... it's just one-liners. And then adding a new environment is adding one new file. Not also modifying database.yml.
17:20 < ssmoot> Which is just what you said... ;)
17:21 < dkubb> ssmoot: heh
17:21 < ssmoot> zond_hme0: doth my pastie look right to thou?
17:21 < Somebee> Is STI-support in the pipes, or do you want/need someone to make it (and do you have some ideas on how you'd like to see it implemented)?
17:23 < zond_hme0> ssmoot: the only change was host = NULL instead of "localhost", right?
17:23 < zond_hme0> ssmoot: in which case, yase it looks right
17:23 < zond_hme0> :)
17:23 < ssmoot> Somebee: it is in the pipes... it's not hard, but I've been trying to think of a clean way to do it.
17:23 < dkubb> Somebee: if you want to take on STI, that would be awesome. What I would like to see is similar behavior to what AR does, where if you go ParentClass.all it will return all instances of the Parent *and* all Subclasses, whereas SubClass.all should return just the Subclass (and any classes that inherit from it)
17:23 < ssmoot> zond_hme0: yeah, and not setting host to r_host if r_host was localhost.
17:23 < zond_hme0> dkubb: yes, but i fear that those libraries will not get built :/
17:23 < dkubb> ssmoot: is anyone working on it right now? I don't know
17:24 < ssmoot> dkubb: no, but here's an idea I had to make a lot of things easier: :query option on property declarations.
17:24 < dkubb> zond_hme0: people will build them if they need them. I've independently come up with the same solution before too
17:24 < zond_hme0> ssmoot: ah that as well. is it necessary? it does, of course, conform to some other tools to connect to databases, but it would be nice in a minimal way to make "no host" equal "unix socket"
17:25 < Somebee> aah, I LOVE having the actual properties in my models.. I can't see any reason why people would find it easier to use AR in this respect :/ You have to define them somewhere right
17:25 < dkubb> ssmoot: we also talked about when you define the "type" column, the :default option would be a lambda that returns the current class in string form
17:26 < ssmoot> dkubb: Somebee: property :deleted_at, DateTime, :default => lambda { Time::now }, :query => { :deleted_at => nil }
17:26 < zond_hme0> dkubb: well, true. maybe im just lazy and want my memcache client lib to solve that problem for me in a simpler manner :)
17:26 < dkubb> property :type, Class, :default => lambda { |resource| resource.class.to_s }
17:27 < ssmoot> w00t! http://ci.datamapper.org/builds/do_postgres/28180cde67ce082...
17:27 < dkubb> Somebee: agreed. often I end up with duplication between the migrations and my models too, since I do alot of strict validation.. so I end up saying the length, nullability, etc of each property twice anyway. Might as well have it all in the model from the start IMHO
17:28 < zond_hme0> ssmoot: w00t!
17:28 < dkubb> ssmoot: those ci url's are annoying, since the "..." isn't hyperlinked
17:28 < dkubb> ssmoot: what about dm-core
17:28 < ssmoot> dkubb: right... yeah, probably the :default lambda should take the instance?
17:28 < zond_hme0> ill be off having a rest from teh intarblag, see ya guys
17:29 < ssmoot> zond_hme0: cyas
17:29 < dkubb> ssmoot: yeah, I think it should accept the resource instance, and the property
17:29 < Somebee> dkubb: yep, no reason not to.. when introduced to rails I bought into the whole "whoooa it knows your columns automagically" but before long I had all fields with description in comments at the top of the model-files anyways, hehe
17:29 < ssmoot> dkubb: I think Luis' changes might've broke the build on dm-core. I'll patch it up to use his stuffs.
17:29 < dkubb> ssmoot: you don't have to use them if you don't want to, as your deleted_at example shows, but having them would be useful
17:30 < dkubb> ssmoot: it looks like the dm_core_test db needs to be created for dm-core to pass
17:30 < ssmoot> dkubb: oh right... resource == instance... I'm not used the that yet... doesn't really make sense to me. ;)
17:31 < dkubb> Somebee: yeah I ended up using the annotate_models plugin anyway, just so I could see what the table column specs were when designing my validations.. so in effect I had the same information in 3 separate places, yuk
17:31 < dkubb> ssmoot: hehe
17:32 < ssmoot> so STI: property :name_doesnt_matter, Class, :default => lambda { |r| r.class.to_s }
17:32 < dkubb> ssmoot: FYI: resource.class == model
17:32 < dkubb> ssmoot: yeah, that is perfect
17:33 < ssmoot> so before it matched on :type && Class. But I think moving to just Class is probably best. You shouldn't really be using Class types for anything else anyways.
17:33 < dkubb> ssmoot: agreed
17:33 < ssmoot> and we should probably raise an error if you try to define more than one Class property in a model in #validate_resource! or something.
17:34 < dkubb> ssmoot: yeah, it could be done when assigning to PropertySet too, i.e. blow up when the same property is added to the hash
17:34 < ssmoot> so with those, STI is pretty stupidly simple straight-forward.
17:34 < dkubb> ssmoot: exactly. Just the searching part is where it gets interesting
17:35 < dkubb> ssmoot: a model needs to know its descendants in order to query using its own name, and the name of its descendents
17:35 < ssmoot> and it lets us do things like optimistic-locking, or paranoid, or STI, or whatever without any (or very little) code actually specifically for those features.
17:35 < ssmoot> dkubb: well, no, not really.
17:35 < ssmoot> dkubb: so get this:
17:36 < ssmoot> dkubb: base-class: property :foo, Class, :default => lambda { |r| r.class.to_s }
17:36 < dkubb> let's go Party < Person < User for these examples. Party being the base class
17:37 < ssmoot> dkubb: descendants: property :foo, Class, :default => lambda { |r| r.class.name.to_s }, :query => { :foo => self.name.to_s }
17:37 < dkubb> and for the sake of keeping things simple, the property is named "type"
17:37 < Somebee> theres a minor problem with the resource-generator in merb_datamapper.. fixed it yesterday: http://github.com/somebee/dm-more/commit/dcaead34844defae25e0c43ea63030decae1ed13 but when I come to think about it.. what is the right thing to do here? Model[id] will raise a ObjectNotFoundError, so it should either use that (and remove the raise NotFound unless @user to), or change it to Model.get(id).. whats your pick?
17:38 < ssmoot> dkubb: ok, so :foo == :type then. So just overwriting :type and also specifying the :query option makes it all work.
17:38 < dkubb> that's fine if I go User.all.. but if I go Party.all, its going to need to go: WHERE type IN('Party', 'Person', 'User')
17:38 < ssmoot> Somebee: model.get(id)
17:39 < Somebee> ssmoot: ok
17:39 < ssmoot> Somebee: just since I'm guessing NotFound is some Merb error, so that preserves Merb's standard conventions?
17:40 < Somebee> ssmoot: well, yes, but all of these can be defined themselves in exceptions-controller. No problem for merb_datamapper to mixin an def object_not_found_error there. Then the generated resources would be even cleaner
17:40 < ssmoot> dkubb: well, no, because they all will be. But you're still right because Person.all needs IN('Person', 'User') excluding 'Party'.
17:40 < Somebee> one shouldn't have to check if id was not found twice imho
17:40 < ssmoot> Somebee: oh, ok. sure. Well, whatever seems cleanest to you. :)
17:41 < dkubb> ssmoot: AR does it so that a model knows what its descendants are, so that when querying for the parent class it includes al l the descendants in the results. That's the behavior I would expect from the system too.
17:41 < dkubb> Somebee: I think using Model.get(id) is best, and keeping true with merb's conventions is important
17:41 < ssmoot> dkubb: so yeah... I guess a :query => lambda { |model| :type => [model] + model.decendents }
17:42 < ssmoot> dkubb: pretty sure I saw someone add a model.descendents recently, so we should already have that.
17:43 < dkubb> ssmoot: I think that was related to DM::is::Tree, which I moved to dm-more/dm-is-tree. I don't see any descendents method right now, but shouldn't be hard to add. We just have to "register" the child model with it's parent with a self.inherited method in Resource
17:44 < dkubb> ssmoot: did you get a chance to create a dm_core_test database on the CI server?
17:45 < ssmoot> dkubb: is that the problem? I think the rakefile needs an update now that DO/make_release.sh is gone. so that's what I'm looking at.
17:47 < dkubb> ssmoot: this is totally on a different tangent, but I think we should make LoadedSet lazy by default, and remove LazyLoadedSet. I was also wondering what you thought about renaming it Collection, and then having Repository#all return an instance of Collection, rather than calling LoadedSet#entries. I think it would be helpful if we got people used to using our Array-like object now vs. expecting an Array, would make it easier to make change
17:47 < dkubb> ssmoot: dm-core's failing specs say it can't find the DB. I think if the DB was there they would pass
17:48 < dkubb> ssmoot: the reason for Collection (Rando said he thinks this is a good name, and doesn't need it 'reserved' any longer for him), is that we can then think about doing ambition-like kicker methods at some point. If we control the object that is returned, we can do that, but if its an Array we basically give up that control
17:49 < ssmoot> dkubb: lazy by default and LazyLoadedSet => Collection sounds good to me. Repository#all returning the collection directly... haven't thought about it, lemme get my mull on. It sounds like a good idea, but it's a big change, so don't want to be rash about it. :)
17:49 < dkubb> ssmoot: plus having it lazy by default we can have it return the object from #all, and if its not used at all we have no penalty
17:49 < dkubb> ssmoot: actually its not a big change. all specs pass when I remove the .entries call :)
17:50 < dkubb> ssmoot: on line 47 of repository.rb :)
17:50 < dkubb> or at least they did when I tested it last time. let me try again. any failures would likely be due to expecting an Array rather than an Array-like enumerable object
17:50 < ssmoot> dkubb: well... not specs I mean. :) But general "this is how DM works". There's already serialization issues we need to think through, so I just want to make sure we have a long-term gameplan.
17:51 < ssmoot> dkubb: not saying I'm against it... I'm for it... just want to make sure we think it through... wanna get dm-core working on CI first tho, so gimmie a few. :)
17:52 < dkubb> ssmoot; no problem
17:52 < dkubb> ssmoot: I've got to run for a bit anyway. you can mull it over and we'll talk about it later today or tomorrow.
17:53 < ssmoot> dkubb: coolbeans.
18:02 < dkubb> ssmoot: sorry, just one last thing. you're cool with the LoadedSet => Collection rename, and making it lazy by default (meaning I can make those changes today/tomorrow); but you want to think about the ramifications of returning a Collection instead directly from Repository#all instead of an Array like we do now?
18:02 < ssmoot> dkubb: right
18:02 < dkubb> ssmoot: for what it's worth, I think we could make a Collection stable enough that there are no issues with its serialization, while at the same time giving us alot of room for future growth and potential kicker methods
18:02 < dkubb> ssmoot: ok, cool, will make those changes shortly
18:04 < ssmoot> dkubb: I think so. It's a change I'm looking forward to... I just don't want to have to back-pedal if we missed something. So a little grokking of the ramifications is all...
18:08 < ssmoot> dkubb: zond_hme0: w00t! http://ci.datamapper.org/builds/dm-core/4a0b9a0c492134e....5
18:10 < ssmoot> so gonna go see a play with my wife... be back in a 3 or 4 hours probably.
18:10 < ssmoot> peace.
18:10 < Somebee> who is working on automigrate for dm 0.9.0? I have 14 models now with a total of 120ish properties... not looking forward to adding them in mysql :|
18:14 < dkubb> Somebee: benburkert is working on automigrate. Anytime you want to see what is being worked on and by whom, check this page out: http://wiki.datamapper.org/doku.php?id=what_needs_to_be_done
18:14 < Somebee> dkubb: ah there it is. thanks!
18:37 < hubbub> [dm-core/master] Renamed LoadedSet as Collection
18:38 < ior3k> o.O
18:40 < dkubb> ior3k: see convo above. sam and I spoke about renaming it a few weeks ago, but he wanted me to hold off until I talked to Rand.o because he had dibs on the name Collection
18:40 < ior3k> dkubb: :) It's very Javaish
18:40 < ior3k> dkubb: but it's a great idea
18:41 < ior3k> dkubb: so Collections are now persistent too?
18:41 < dkubb> ior3k: our naming convention uses RESTful terms, and what's a group of Resources in rest terms? a collection
18:41 < ior3k> dkubb: yeah, I understand, I'm just messing with you ;)
18:41 < dkubb> it allows you to provide a block so you can lazily load the collections
18:41 < ior3k> dkubb: I think it's a good name
18:41 < dkubb> ior3k: technically we could go into DAO and make it so all that DB work is done inside a block handed to Collection.new
18:42 < dkubb> ior3k: that way the DB work would be deferred until you called .each or .entries on the collection
18:42 < dkubb> ior3k: which means lazy collections, which is awesome
18:42 < ior3k> dkubb: I agree completely
18:42 < ior3k> dkubb: Hibernate has a few optimizations based on that
18:43 < ior3k> dkubb: they also do stuff like only selecting the ids and then lazy loading individually as needed
18:43 < dkubb> ior3k: I can't wait to go into DAO and try tweaking this, but today is family day (only reason I'm working is cuz they are all asleep) so I probably won't get to it
18:43 < ior3k> dkubb: I started on :through, meanwhile
18:43 < ior3k> dkubb: earlier than what I expected
18:45 < dkubb> ior3k: the other thing above I was asking sam about was returning a collection from Repository#all. as of right now it uses Collection#entries to do the DB work, and then return the results as an Array. I wanted to return a collection object, so we can make kicker methods and only run the DB queries if the collection info is asked for
18:45 < dkubb> ior3k: excellent
18:45 < ior3k> dkubb: totally agree, that can give way to a lot more optimizations
18:46 < dkubb> ior3k: are the with_parent and with_child methods in Relationship used any longer?
18:47 < dkubb> ior3k: one thing I want to do with the Relationship code is make it so the child key property is defined immediately for belongs_to associations (i.e. during Relationship#initialize)
18:48 < dkubb> ior3k: I don't like the fact that the properties are lazily added once Relationship#child_key is called; since its sort of a side effect that isn't clear from the outside
18:48 < ior3k> dkubb: I haven't removed them from many to one, yet. I'm waiting until I finish my current work on associations to start cleaning it up
18:48 < ior3k> dkubb: you may have a problem there, because of circular dependencies
18:48 < dkubb> ior3k: in has() relationships we don't know if the child class has loaded yet, so we still need to lazily add it tho
18:49 < dkubb> ior3k: yeah, I know
18:49 < ior3k> dkubb: yeah, that happens with belongs_to, either
18:49 < dkubb> ior3k: it shouldn't happen with belongs_to, since the child model is the current model you define belongs_to in.. so you know its been loaded
18:50 < ior3k> dkubb: oh, of course, I was thinking about both child and parent now
18:50 < dkubb> ior3k: but with has() you have no way of knowing. I'll probably add something to see if the constant has been defined, and add the child key properties if they have
18:50 < dkubb> ior3k: otherwise I'll defer it
18:51 < dkubb> ior3k: I wish there was a way with ruby to say "when this class is loaded, run this code".. sort of queue up stuff to run once it finds and loads the parent model
18:51 < ior3k> dkubb: I wonder if there is a way around it... maybe with a proxy. Or we could do a require exactly after defining the relationship. There, both classes would be loaded
18:52 < ior3k> dkubb: and we would have the required info because the relationship would be defined already
18:53 < dkubb> ior3k: I was thinking if I had a queue in Relationship for "to be added" properties; and when a model loads it checks the queue to see if it needs to add any properties.
18:53 < dkubb> the problem is I don't know if there is a method that executes when a class is initially defined
18:54 < ior3k> we can use Resource#included
18:54 < ior3k> ...except... I think that is called after the class is defined, but before doing anything with it
18:55 < dkubb> ior3k: that would actually be perfect
18:56 < ior3k> dkubb: no wait, that's inherited. I think included is called at the point you call it in the including class
18:56 < ior3k> dkubb: but Resource is usually the first thing to be included, shouldn't be a problem, should it?
18:56 < dkubb> ior3k: which is fine I think, because you have to include Resource into any classes you want to have DM persistence
18:57 < ior3k> dkubb: oh, of course
18:57 < dkubb> ior3k: I can kick off something to put the properties into the current class
18:57 < ior3k> dkubb: I definitely need to get more sleep :)
18:57 < dkubb> ior3k: which works out very well I think
18:57 < dkubb> ior3k: yeah I'm going to grab a few too. Thanks for thinking this through with me
18:57 < ior3k> dkubb: no problem, thanks for sharing
19:18 < zond_hme0> does anyone know why the spec_helper.rb in do_mysql uses $:.unshift?
19:18 < zond_hme0> and not just require "./lib/do_mysql"?
19:19 < Somebee> is dm-more commit/push-access limited?
19:19 < zond_hme0> Somebee: the same way dm-core is, i would suppose
19:19 < Somebee> zond_hme0: and how is dm-core?
19:20 < zond_hme0> Somebee: ssmoot assigns commit access
19:20 < Somebee> zond_hme0: ok
19:26 < zond_hme0> i need a logger for do
19:26 < zond_hme0> but datamappers logger is just a ripped version of merbs logger
19:27 < zond_hme0> i cant find a DRY way of doing this :/
20:48 < uzytkownik> Hello. How to create UNIQUE key in datamapper on some property?
20:52 < zond_hme0> uzytkownik: i think you just do :unique => true in the property definition (but this is just a guess)
20:52 < uzytkownik> zond_hme0: It was exactly what I wanted (at least I would be mostly surprised if not)
20:53 < zond_hme0> :)
20:56 < uzytkownik> "ArgumentError: :unique is not a supported option in DataMapper::Property::PROPERTY_OPTIONS". I have the newest gem. Isn't it 0.9 feature?
20:59 < zond_hme0> ill check
21:02 < zond_hme0> check PROPERTY_OPTIONS in your lib/data_mapper/property.rb file in your data_mapper gem
21:02 < zond_hme0> in the git edge version :unique is in there...
21:02 < zond_hme0> i dont know where you got your gem, so i cant say anything about that :)
21:04 < uzytkownik> rubygems I think
21:04 < uzytkownik> version 0.3.1
21:05 < uzytkownik> This string appear neighter in this file nor in official docs
21:06 < zond_hme0> ah
21:06 < zond_hme0> 0.3.1 i dont know about
21:07 < zond_hme0> then it is only in git edge vertsion
21:07 < zond_hme0> version
21:07 < zond_hme0> i have to go now, but good luck
21:07 < uzytkownik> Ok
21:07 < uzytkownik> As far I'm using :index => true & validates_unqueness_of
21:08 < uzytkownik> and may be I'll move when it will be next release of
21:08 < uzytkownik> dm
21:14 < luislavena> hello guys
21:16 < luislavena> dkubb: ping?
21:17 < dkubb> luislavena: hey man. I made some Rakefile simplifications according to some of the ones you started, hope you're cool with them
21:17 < luislavena> dkubb: yeah, no issues with that, I'm getting a warning from one change in go_posgres
21:17 < luislavena> do, Imean :-P
21:17 < dkubb> luislavena: what sort of warning?
21:18 < luislavena> do_postgres.c(204) : warning C4047: '!=' : 'unsigned long ' differs in levels of indirection from 'char [10]'
21:18 < dkubb> luislavena: I liked the way you use OpenStruct, but then I was like.. hey we're creating this object just underneath that acts the same was as the OpenStruct, so I figured why not use that instead
21:18 < luislavena> that's due the comparison between a VALUE and a string.
21:19 < luislavena> yeah, the OpenStruct was just there to hold the CONSTANTs way... I don't like neither if you asked me :-P
21:19 < luislavena> besides, the version was hardcoded, no usage of ENV or something that made it worth... so was good you ripped it :-D
21:20 < luislavena> oh, and it seems postgres don't behave the same under windows than linux... or maybe I should add current user (luis) to the postgres user table :-D
21:22 < luislavena> dkubb: I'm getting this: http://pastie.caboo.se/187713
21:22 < dkubb> I think using the current logged in user was going to be the standard for the postgres related specs
21:23 < dkubb> luislavena: here's a pointer to what's wrong: http://forums.matrix.squiz.net/index.php?showtopic=3902
21:23 < luislavena> dkubb: good then, we should add a note into the readme or the top of the specs, to avoid dumbass like me change them :-D,
21:24 < dkubb> luislavena: you need to set your pg_hba.conf to trust people logging in from localhost, so it won't require passwords
21:25 < dkubb> luislavena: yeah, there was some debate this morning, and sam and a couple of other people decided to make it use the current logged in user for the specs.
21:25 < luislavena> dkubb: a good pointer to add to the specs, since I think will not be the only one...
21:25 < luislavena> dkubb: I'll check what user is using...
21:37 < luislavena> dkubb: I'm not familiar with pgsql.... can you tell me the exact configuration line?
21:37 * luislavena is more a mysql and sqlite kind of user.
21:37 < dkubb> luislavena: neither am I, I would suggest opening the file mentioned in that link I sent you, and adding the line shown in the example
21:37 < dkubb> luislavena: I've used postgres for maybe a grand total of 2 hours in my life, although I plan to use it alot more in the future
21:40 < luislavena> dkubb: even adding the trust for host, still fails (restarted the server)... need to check what I'm doing wrong :-P
21:41 < dkubb> maybe stop and start the server explicitly. I know that works for other DB's sometimes
21:42 < luislavena> dkubb: doing it again...
21:44 < luislavena> have this in my pg_hba.conf file:
21:44 < luislavena> http://pastie.caboo.se/private/mi0j7mla3bg3hpfwvc4bqw
21:44 < luislavena> and still don't accept the connection.
21:44 < luislavena> oh, wait... silly me...
21:44 < luislavena> the user is Luis, not luis ... damn case sensitive stuff..
21:48 < luislavena> gret, it worked, I think dm-core should follow the same pattern...
21:58 < luislavena> dkubb: what do you think about this?
21:58 < luislavena> dkubb: http://pastie.caboo.se/private/6s7befxmfpyku2tf8a863w
21:58 < luislavena> this is to remove the char[10] != unsigned long comparison
22:03 < mayo> is acts_as_* the de-facto standard for plugin changing behavior of the model?
22:27 < afrench> what's the equivalent to Model.table.create! in the new 0.9 API?
22:28 < afrench> mayo: I thought is_a_* was more common
22:28 < afrench> not sure though
22:29 < mayo> afrench: not sure, sorry. i went back to using 0.3 for past couple of weks cause of some others problems
22:29 < afrench> hmph
22:29 < afrench> I don't like hearing that...I just spent some time bumping a project up to 0.9
22:30 < mayo> might work for you, there are other people using 0.9. i just don't have the time to deal with devel stuff right now, too busy trying to deliver the actual projects
22:32 < mayo> although I do need features from 0.9 like composite PKs :(
22:33 < afrench> blah, DM is stuck between a rock and a hardplace
22:33 < afrench> 0.3 has regressions in it, and 0.9 is far from ready for prime-time
22:37 < afrench> bbiab
22:52 < hubbub> [dm-core/master] Add 1 and '1' to TrueClass typecast - Foy Savas (http://github.com/sam/dm-core/commit/f9cac37b4058284dd180ff59c8499811815ebd70)
--- Log closed Mon Apr 28 00:00:25 2008