belongs_to, has_many counter_cache

[adamc]

Hi,

I've started looking at the counter_cache implementation with the
original intention to have the increment, decrement update the model
in memory (without relying on a reload) to be called.

You can see a sample of the bug here: http://gist.github.com/30580.

While starting to look at this issue (my first real look at the rails
code base) I'm wondering if anyone has already done any initial
thinking of this issue or has an idea of what they would like to see
happen here. In my initial thinking (I haven't looked at the
association proxy/collection enough) there could be a bi-directional
link for the has_many/belongs_to reflections/assocations.

Looking around I noticed a small bug with custom counter cache names
not be utilized in the record with the has_many association. There is
currently no way for this association/reflection object to know about
the belongs_to settings where the counter_cache name is set.

An easy way to quickly solve this is to add a :counter_cache =>
'custom_name' to the has_many method call. What are your thoughts on
this proposed change for the has_many options?

Thanks,
Adam

[Frederick Cheung]

On 1 Dec 2008, at 02:15, adamc wrote:

>
> Hi,
>
> I've started looking at the counter_cache implementation with the
> original intention to have the increment, decrement update the model
> in memory (without relying on a reload) to be called.
>

Be aware of the subtleties involved if someone else has also created a
record (which is why the counter code doesn't just do foo.bars_counter
+= 1; foo.save)

Fred

[acechase]

Keep in mind that multiple instances of rails will be running on all
but the simplest applications, so relying on in-application
information will be broken from the start since each instance of the
application will have it's own in-memory objects.

I agree it would be nice to not need to reload the object after an
increment/decrement call is made, but I think the proper solution
would be to retrieve the new value from the database after the UPDATE
has occurred. One way I think this could be done is by issuing a query
similar to this:
UPDATE users SET score = score + 1 WHERE users.id = 1; SELECT
users.score FROM users WHERE users.id = 1;

Then the return value from that query could be used to update the in-
memory object. You still have two separate db operations, but at least
they come through in one back and forth trip to the db, which is where
the wost of the performance penalty occurs in this type of sequence
(at least, that's what my profiling has shown). Also note that this
new in-memory value is again immediately stale upon retrieval and
you'd to do a select for update inside a transaction if you wanted to
use that value to write back to the db.

As for the bi-directionality of in-memory AR objects, you should check
out the parental-control plugin, which while incomplete, covers quite
a bit of what is needed to support bi-directional associations in
rails:
http://github.com/h-lame/parental_control/tree/master

Cheers,
Andrew

[Michael Koziarski]

> As for the bi-directionality of in-memory AR objects, you should check
> out the parental-control plugin, which while incomplete, covers quite
> a bit of what is needed to support bi-directional associations in
> rails:
> http://github.com/h-lame/parental_control/tree/master

Nice! I'd love to see something along these lines baked right in to 2.3

--
Cheers

Koz

[adamc]

> One way I think this could be done is by issuing a query
> similar to this:
> UPDATE users SET score = score + 1 WHERE users.id = 1; SELECT
> users.score FROM users WHERE users.id = 1;

Yes, that was my thought too, after the first comment.

I've come to the conclusion that the second query is unnecessary if we
just update the in memory copy directly (without the DB update) and
use the Base.update_counters to "safely" update the counters. If the
in memory copy is stale, even after the counter cache has updated it,
then it would be stale anyways and there is nothing we can do. As
long as the
database has the correct value then all should be good.

One question here, is there any particular reason, why the has_many
associations build method only associates the parent object by id
rather than by object? This is a potential issue when updating the in
memory copy. Although the bi-directional associations may nullify
this question.

blog = Blog.create
puts blog.object_id # => 103230460
post = blog.posts.build
puts post.blog.object # => 103511790

> As for the bi-directionality of in-memory AR objects, you should check
> out the parental-control plugin, which while incomplete, covers quite
> a bit of what is needed to support bi-directional associations in
> rails:http://github.com/h-lame/parental_control/tree/master

Great! A good place to start looking at. Thanks for the link.

Thanks,
Adam

[acechase]

Three cheers to that! h-lame has done some really good work getting
the parental_control plugin working, but I think it would make
hammering out the trickier bits much easier if it was part of core. As
it stands now, without something like parental_control, if you're
using eager includes to avoid 1+N query problems, you have to write
your code to only ever use associations through the direction in which
they were loaded. Which, IMO, is one nasty leak of encapsulation in
the ORM.

-ac

[Murray Steele]

2008/12/2 acechase <acec...@gmail.com>

On Dec 1, 12:14 pm, "Michael Koziarski" <mich...@koziarski.com> wrote:
> > As for the bi-directionality of in-memory AR objects, you should check
> > out the parental-control plugin, which while incomplete, covers quite
> > a bit of what is needed to support bi-directional associations in
> > rails:
> >http://github.com/h-lame/parental_control/tree/master
>
> Nice!  I'd love to see something along these lines baked right in to 2.3
>
> --
> Cheers
>
> Koz

Three cheers to that! h-lame has done some really good work getting
the parental_control plugin working, but I think it would make
hammering out the trickier bits much easier if it was part of core.

Ah-ha! Now I understand why my little un-announced plugin suddenly got some watchers on github ;)

So, here seems like as good a place as any to thrash it out.  What do we think I/we'd need to do to get it into core?  As someone pointed out, it's pretty incomplete so I'm sure I've not covered all the edge cases, although I think it does a pretty good job of giving up if it can't find something that it thinks is the right inverse / reciprocal relationship (and we're using it in a production app here and it's yet to fall over). 

The first natural extension to me would be to allow for :inverse options on association definitions that would allow for getting round trying to work it out, e.g. something like:

class FlamingStick < ActiveRecord::Base
  belongs_to :extreme_juggler, :inverse => :flaming_sticks
end

class ExtremeJuggler < ActiveRecord::Base
  has_many :flaming_sticks, :inverse => :extreme_juggler
end

Although, for these simple cases it does seem like extra work that the framework should be able to work it out for you, so I don't think I'm hugely in favour of dropping the "magic" auto-detection.

Anything else / anyone else used it and spotted stuff I haven't?

Cheers,

Muz

[Chris Cruft]

I don't have a lot to offer to this subject except to say that in my
experience of building declarative bidirectional relationships in
Javascript (to proxy AR records!) I found that "promoting" the
associations to a full-blown model was quite clean. If done directly
in AR, I could imagine something like this:

in app/models/dangerous_circus_tricks.rb

class DangerousCircusTrick < ActiveRecord::AssociationModel
associate ExtremeJuggler, [FlamingSticks]

def adequately_insured?
flaming_sticks < 3 || extreme_juggler.experience == :expert
end
end

Complex relationships (particularly with bidirectionality) sometimes
warrant a dedicated model -even if the particulars of the ORM layer
don't require a dedicated table. I suspect that such an approach
would extend well into ActiveResource and other non-RDBMS stores. It
could even be used to model mixed associations (an AR record
associated with an ActiveRes model).

On Dec 3, 10:59 am, "Murray Steele" <murray.ste...@gmail.com> wrote:
> 2008/12/2 acechase <acech...@gmail.com>

[Chris Cruft]

One other thing I should have mentioned: the ActiveRecord aggregations
interface is, IMHO, ripe for being incorporated into this same type of
modeling. With the latest enhancements to constructors and
converters, Aggregations can start to be viewed as a flexible
interface to in-memory associations.