Is there an "before_commit" hook somewhere in Rails? after_save does not help (example code attached)

[Greg Hauptmann]

Hi, (no luck on the user forum so I'm hoping I can ask here)

I'm trying to get a simple cross-model business rule working.  In this case the rule is (see below for models overview):
  * Rule = Sum(allocations amount, for a book) = Book Amount

ISSUE: The issue is in using after_create is that either the book or allocation is saved before the other.  The only way I can see to make this work is to have a check just prior to COMMIT, where all records are visible within the DB and your final checks can be run (and rolled back if there is problem).  Hence my question:

QUESTION: Is there an "before_commit" hook somewhere in Rails?  (or how else would I satisfy my requirement)


----------------------------------------------------------------------------------
Macintosh-2:after_create_test greg$ spec spec/model/all_in_one_test_spec.rb
============= NEW TEST ======================
BOOK: after_save
F============= NEW TEST ======================
CHAPTER: after_save
.============= NEW TEST ======================
BOOK: after_save
.

1)
RuntimeError in 'Book should save if allocation amount == book amount'
amounts do NOT match
./spec/model/all_in_one_test_spec.rb:26:in `after_save_check'
./spec/model/all_in_one_test_spec.rb:51:

Finished in 0.061092 seconds

3 examples, 1 failure
----------------------------------------------------------------------------------

Macintosh-2:after_create_test greg$ cat -n spec/model/all_in_one_test_spec.rb
     1  require File.expand_path(File.dirname(__FILE__) + '/../spec_helper')
     2
     3  # ------------ ALLOCATION -------------
     4  class Allocation < ActiveRecord::Base
     5    belongs_to :book
     6    belongs_to :chapter
     7
     8    after_save :after_save_check
     9    def after_save_check
    10      puts "ALLOCATION: after_save"
    11      b = self.book
    12      sum = b.allocations.collect{|i| i.amount}.inject(0){|sum, n| sum + n }
    13      raise("amounts do NOT match") if !(b.amount == sum)
    14    end
    15  end
    16
    17  # ----------- BOOK ---------------
    18  class Book < ActiveRecord::Base
    19    has_many :allocations
    20    has_many :chapters, :through => :allocations
    21
    22    after_save :after_save_check
    23    def after_save_check
    24      puts "BOOK: after_save"
    25      sum = self.allocations.collect{|i| i.amount}.inject(0){|sum, n| sum + n }
    26      raise "amounts do NOT match" if !(self.amount == sum)
    27    end
    28
    29  end
    30  # ----------- CHAPTER ---------------
    31  class Chapter < ActiveRecord::Base
    32    has_many :allocations
    33    has_many :books, :through => :allocations
    34
    35    after_save :after_save_check
    36    def after_save_check
    37      puts "CHAPTER: after_save"
    38    end
    39
    40  end
    41
    42  # --------- RSPEC (BOOK) ------------
    43  describe Book do
    44    before(:each) do
    45      puts "============= NEW TEST ======================"
    46      @b = Book.new(:amount => 100)
    47      @c = Chapter.new()
    48    end
    49
    50    it "should save if allocation amount == book amount" do
    51      @b.save!
    52      @c.save!
    53      Allocation.create!(:book_id => @b.id, :chapter_id => @c.id, :amount => 100)
    54    end
    55
    56    it "should raise database exception if try to save allocation prior to book" do
    57      lambda {
    58        @c.save!
    59        Allocation.create!(:book_id => @b.id, :chapter_id => @c.id, :amount => 100)
    60        @b.save!
    61      }.should raise_error
    62    end
    63
    64    it "should raise error if allocation amount != book amount" do
    65      lambda {
    66        @b.save!
    67        @c.save!
    68        Allocation.create!(:book_id => @b.id, :chapter_id => @c.id, :amount => 90)
    69      }.should raise_error
    70    end
    71
    72
    73  end
    74
----------------------------------------------------------------------------------
ActiveRecord::Schema.define(:version => 20090123000614) do

  create_table "allocations", :force => true do |t|
    t.integer  "book_id",    :null => false
    t.integer  "chapter_id", :null => false
    t.integer  "amount",     :null => false
    t.datetime "created_at"
    t.datetime "updated_at"
  end

  create_table "books", :force => true do |t|
    t.integer  "amount",     :null => false
    t.datetime "created_at"
    t.datetime "updated_at"
  end

  create_table "chapters", :force => true do |t|
    t.datetime "created_at"
    t.datetime "updated_at"
  end

end
----------------------------------------------------------------------------------












--
Greg
http://blog.gregnet.org/

[Greg Hauptmann]

PS.  Some additional clarification:

1. Goal is to understand whether I can/should attempt to model a business_rule that cuts across multiple models in Rails
2. I think the given is that to carry out a user scenario (e.g. adding more chapters to a book) would result in having to carry out multiple steps at a per model level (e.g. adjust allocation of amount across chapter for their authors), and that during these model changes the Business Rule would have be violated, however by the time you've finished the Business Rule should have been adhered to. 
   
3. I was really hoping to have a way to do this that didn't break the normal usage of Rails models, but at the same time if you did jump in and try to make one isolated change on one model (e.g. add a new chapter for a book without ensuring all the chapter costs were adjusted to equal the book cost), the Business Rule code would kick in and pull you up with an exception.
   
4. The only way I can see to handle this in the KISS (keep it simple stupid) fashion would be to be able to get some sort of "before_commit" hook from Rails, where I ideally it would give you the models that have changed in this hook, so you do cross-business rule sanity check. So the check here would ultimately be database focused (i.e. check against what is in the database)
   
5. I've thought about doing the check just at object level at "before_save" point, however there seem to be gotchas here.  

Hope that makes sense.  Perhaps this is just not-possible in Rails currently and I should just assume I have to be very careful with all my code, because there won't be that cross-model validation check there to save me.  One reason to have it in place too by the way is that I could leverage a front-end frame work like ActiveScaffold and not have to worry about the fact it would give a user the ability to change one particular row without that cross-model business logic check kicking in.

--
Greg
http://blog.gregnet.org/

[Mike Mangino]

I think I'm a little confused about this. Can you explain again what
you are trying to do?

So the rule is that the sum of allocations for a book must equal a
total. That sounds like a validation. My code would look like

class Book
validate :sum_of_allocation_equals_amount

def sum_of_allocation_equals_amount
error.add(:amount,"Does not match sum of allocations") unless
allocations.sum_of_price == amount
end
end

Then, you can create a book

book = Book.new(:amount=>20)

#and add allocations

book.allocations.build(:amount=>10)
book.allocations.build(:amount=>5)
book.allocations.build(:amount=>5)

# and then save the book, which I believe will save the allocations

book.save

Does that not work?

Mike

[Greg Hauptmann]

The trouble I had with cross-model validation using validates was:

(a) Object Level (i.e. using Rails objects)
* During the carrying out of a scenario (say increasing book value, and then re-deciding what each of the chapter value allocation to this should be) will imply during the process of making these changes the business rule will break, but it's only at the end when you're finished the business rule needs to be applied.  For example the sequence might be:
  - change book value
  - change chapter 1 value
  - change chapter 2 value
  - change chapter 3 value
  - <only at this point should the cross model business rule be checked>
* Hence any validation method getting called at any of the interim steps I don't think will correctly apply the cross-model business rule
* Also at Rails object level another gottcha is just because you assign book a chapter (without saving) doesn't imply you can then see that chapter from the book instance end (or perhaps it was the other way around). 

(b) At Database Level - If the validation code always looks database records to do the comparison then you also get business rule exceptions being through during the use case scenario whereas you really want to only apply it at the end.

So without going to a database solution of some sort (triggers etc) the only way I could think of getting this working at the Rails level was to get access to an overall "just_before_commit" hook, hence my question.  (Someone suggested observers, so I'll have to read up on these, however if they apply at the per model level then I don't think this would probably work either)

Comments?

PS Don't read too much into my example as it is only there for the purpose of trying to highlight a simple cross-model business rule.

--
Greg
http://blog.gregnet.org/

[Mike Mangino]

Personally, I normally solve the problem by making one source of data
the correct record. For instance, I would make the allocation the
source, and then put a callback on allocation that notifies Book of a
change. Then, Book can recalculate itself from the allocations. This
type of things is often very specific to the domain. If I was doing a
percentage breakdown, for instance, allocating a portion of the profit
to a number of people, I would have a single method the balances all
of the allocations and includes the validation logic instead of
allowing people to update a single allocation at a time. For instance,

class Book
def set_allocations(allocations)
raise InvalidAllocations unless valid_allocations?(allocations)
...
end
end

That keeps the logic in a single place and makes the validation a part
of the business rule that it is tied to.

Mike

[Greg Hauptmann]

Hi Mike, all

Understood.  To help align my fictitious example to the cross-model validation question I've asked consider that:
(a) the book value is fixed [e.g. perhaps think of this as a bank account transaction amount, being allocated out to different tax categories & then the user wants to adjust the tax categories]
(b) the user manually adjusts the chapter value (i.e. there is no programmatic approach to calculating the distribution)

So this brings it back to my scenario I'm not sure how to solve in Rails whereby the sequence of events here would be:

  - change chapter 1 value
  - change chapter 2 value
  - change chapter 3 value

  - <only at this point should the cross model business rule be checked, i.e. Book.amount.should == Sum(chapter values)>

My assumption here (correct me if I'm wrong) is that any Rails validation/after_save/observer kicks in at such of the sequence points, whereas what is actually required here is a cross_model business logic check at the end. 

Does this make sense?   Is there a ways in Rails to get access to a "before_commit" type hook that would align with the point I want the business logic check to kick in?

Thanks

--
Greg
http://blog.gregnet.org/

[Matt Jones]

There's no explicit hook, but you can pretty much do what you've
described using transactions.
If you're updating the chapters in a single controller action, you can
use a transaction block
(see http://api.rubyonrails.org/classes/ActiveRecord/Transactions/ClassMethods.html)
to wrap all
the changes. Then, either use an after_save on Book, or just call a
method directly to validate the
combination.

You'll want to use save! and friends within the block, and catch
exceptions (ActiveRecord::RecordInvalid and
ActiveRecord::RecordNotSaved) to display errors.

--Matt Jones

[Greg Hauptmann]

Hi Matt/all,

Actually from what I can see Rails does not hold off on trigger it's "after_save" callbacks until just before commit in the case they are wrapped in a specific transaction (unfortunately).   So there's still no "before_commit" equivalent yet anyone has identified.  Let me know if I'm wrong however here's the test I've run.

------------------- test output -------------------------------------------------

Macintosh-2:after_create_test greg$ spec spec/model/with_transaction_block.rb 

BOOK: before_save

F

1)

RuntimeError in 'Book should allow creation of book-allocation-chapter if costs match'

amounts do NOT match

./spec/model/with_transaction_block.rb:27:in `after_save_check'

./spec/model/with_transaction_block.rb:57:

./spec/model/with_transaction_block.rb:56:

Finished in 0.048817 seconds

1 example, 1 failure

Macintosh-2:after_create_test greg$ 

--------------- spec ------------------------------------------------------------------

require File.expand_path(File.dirname(__FILE__) + '/../spec_helper')

# ------------ ALLOCATION -------------

class Allocation < ActiveRecord::Base

  belongs_to :book

  belongs_to :chapter

  before_save :after_save_check

  def after_save_check

    puts "ALLOCATION: before_save"

    b = self.book

    sum = b.allocations.sum(:amount)

    raise("amounts do NOT match") if !(b.amount == sum)

  end

end

# ----------- BOOK ---------------

class Book < ActiveRecord::Base

  has_many :allocations

  has_many :chapters, :through => :allocations

  before_save :after_save_check

  def after_save_check

    puts "BOOK: before_save"

    sum = self.allocations.sum(:amount)

    raise "amounts do NOT match" if !(self.amount == sum)

  end

end

# ----------- CHAPTER ---------------

class Chapter < ActiveRecord::Base

  has_many :allocations

  has_many :books, :through => :allocations

  before_save :after_save_check

  def after_save_check

    puts "CHAPTER: before_save"

  end

end

# --------- RSPEC (BOOK) ------------

describe Book do

  before(:each) do

    @b = Book.new(:amount => 100)

    @c = Chapter.new()

  end

  it "should allow creation of book-allocation-chapter if costs match" do

    Book.transaction do

      @b.save!  # SEEMS TO TRIGGER BOOK AFTER_SAVE HERE RATHER THAN HOLDING OFF

      @c.save!

      @a1 = Allocation.create!(:book_id => @b.id, :chapter_id => @c.id, :amount => 100)

    end

  end

end

-------mysql log -----------------------------------------------------------------

090125 15:31:46    1534 Connect     root@localhost on after_create_test_test

                   1534 Query       SET SQL_AUTO_IS_NULL=0

                   1534 Statistics 

                   1534 Query       SHOW FIELDS FROM `books`

                   1534 Query       SHOW FIELDS FROM `chapters`

                   1534 Query       BEGIN

                   1534 Query       SHOW FIELDS FROM `allocations`

                   1534 Query       SELECT sum(`allocations`.amount) AS sum_amount FROM `allocations` WHERE (`allocations`.book_id = NULL)

                   1534 Query       ROLLBACK

                   1534 Quit       

Is my analysis correct?

Cheers

Greg

--
Greg
http://blog.gregnet.org/

[stephen paul suarez]

hmmm, i have an idea for you but i think it's a bit hacky.. i've put the check on validation since i think it mostly relates to validating your models..

# ----------- Allocation ---------------

class Allocation < ActiveRecord::Base

   belongs_to :book

   belongs_to :chapter

   validate :check

   

   def check

     #this will always be true.. this just ensures that

     return @check if @check     

@check = true

     self.book.valid?

   end

end

# ----------- BOOK ---------------

class Book < ActiveRecord::Base

  has_many :allocations

  has_many :chapters, :through => :allocations

  

  validates_associated :allocations  

  validate :check

  

  def check

    puts "BOOK: after_save"

#reload associated allocations always

    sum = self.allocations(true).map(&:amount).sum

errors.add("amount","do NOT match") unless self.amount == sum

  end

end

this is a bit tricky, since Allocation#save will try to call on Book#valid?, the validates_associated will in turn try to call Allocation#valid? once again, but the second time the code executes on Allocation#check, the validation will just return true.. hth

--stephen

[Greg Hauptmann]

thanks stephen - I think (haven't tested it) the problem here would still be called a line 1 (see below) of my use case.  So whilst it would trigger validation for chapter in a non-looping sense, the problem is that until Chapter & the Allocation are saved to DB the validation will fail.  Make sense?  Hence why I was interested in an a "before_final_commit" type hook.

---------------------------

1  @b.save!  # SEEMS TO TRIGGER BOOK AFTER_SAVE HERE RATHER THAN HOLDING OFF

2  @c.save!

3  @a1 = Allocation.create!(:book_id => @b.id, :chapter_id => @c.id, :amount => 100)

---------------------------

--
Greg
http://blog.gregnet.org/

[Greg Hauptmann]

BTW - I've had an indication after posting on a mysql forum site that MySql does not provide a "deferred constraints" feature, whilst apparently Oracle and Postgres do.   From what I gather this would have been useful as I could have put the business logic check in the database I think, to be only run after all statements had occurred, but before the final commit.    Implication is that MySql won't provide the solution so it would have to be in Rails if at all.  

At this stage I'd be happy to assume I can not get a fully robust solution to cross-model validation checks in Rails.  What at least would be good is to be able to get to the concept that:

1. Data Access Layer - That is model classes and base ActiveRecord methods for models:  These do not provide any protection themselves.  If a developer uses the normal ActiveRecord calls (e.g. save, update etc) then it's up to them to get it right, however they would be encouraged not to use this layer directly but use the "Service Layer"
   
2. Service Layer - Provides methods to use the models/tables that have cross-model business rules.  Basically these are the "trusted" methods that will respect the business rules (noting it's not possible it seems to have Rails provide the robust solution).  For example:
• update_chapters - would update all chapters first (no model validation calls would be firing), then at the end perform the business logic check
3. Controller Layer - calls the service layer

Implication here is that one may not be able to use the normal tools like ActiveScaffold which automatically gives you maintenance pages for all the models, as it would be hooking into the Data Access Layer directly and therefore wouldn't adhere to Business Logic check..

How does this sound?  Probably the best I can do?

thanks

--
Greg
http://blog.gregnet.org/

[Matt Jones]

In your previous examples, the validation was getting called twice
because you *defined* it twice;
once in Allocation and once in Book. The preferred idiom in Rails is
to define it once, and be careful
how you use the objects. Since your rule won't allow Allocations or
Books to be updated independently,
it seems logical that there will be (roughly) one place in the code
that does the update.

AR doesn't automatically save associated records, but it's easy enough
to do it yourself.

Your models would end up looking like this:

# ------------ ALLOCATION -------------
class Allocation < ActiveRecord::Base
belongs_to :book
belongs_to :chapter

# note no validation here
end

# ----------- BOOK ---------------
class Book < ActiveRecord::Base
has_many :allocations
has_many :chapters, :through => :allocations

before_save :check_totals
def check_totals
sum = self.allocations.sum(:amount)
if !(self.amount == sum)
self.errors.add :amount, 'amounts do not match'
false
end
end

end
# ----------- CHAPTER ---------------
class Chapter < ActiveRecord::Base
has_many :allocations
has_many :books, :through => :allocations

end

With a controller action like this:

def update_stuff
@book = ... # get book instance
# do stuff to @book.allocations - don't use update_attributes, as
it will save the allocations
if @book.save
# success
else
# something went wrong
end
end

The false return value from check_totals will rollback the whole
implicit transaction that book.save is enclosed in
if the totals don't match up.

I'm not sure what the concern about ActiveScaffold is about - I
haven't looked at it in a lot of depth, but I doubt that
it supports the kind of multi-model form you'll need to update the
records the way you plan to. As it stands, it wouldn't
be possible to update either a Book or an Allocation independently.

The concept of "trusted" access methods is somewhat useless; if you
don't trust the code in your controllers, you have
a whole other problem. Even if ironclad validations could be set up,
all it takes is a call to save(false) to get past them...

--Matt

[Ryan Angilly]

That's why I was so confused.  There's no reason to expect @b.save! to hold off validations.  That's the whole point calling save.  If you want, you can build out the objects a la @b.allocations.build(options) and then the save will also save the new allocations.  So you may be able to do something like:

@b = Book.new(book_options)
@c = Chapter.new(chap_options)
@c.save!
@a = @b.allocations.build(alloc_options)

@b.save  # which will save the allocations too

You could also simplify things by creating a workflow where the user could not change both models in one action.  If you're trying to be railsful and restful, changing the book value in one action, and then subsequently changing the chapter values in successive actions may be the way to go.

[Adam]

Greg,

If I understand the problem correctly, you want to update each of the
Allocations for a Book in one request, making sure that by the end of
the request the value sums are still valid. Correct? Based on that
assumption, I'm guessing that you're trying to do something like this
(in, I'm also guessing, your BooksController):

def update
# explicitly open a transaction
# (maybe) update values for specified Book
# save Book
# for each Attribute
# update values
# save
#end
# check Attribute sum validations and rollback transaction if
validation fails, commit otherwise
# end transaction
end

The problem with this is that you're doing a lot of unnecessary
database work in the case where validation fails. And Rails doesn't
really support this approach easily. However, you have all the
information you need in order to do validation before any database
saves. You can build your HTML form to generate params such that the
modifications to Allocations get passed to the Book object on
update_attributes. The Book model then updates its Allocations,
checks the sum validation, and then (in an after_save callback) saves
all of its associated Allocations. If validation fails, you get a run-
of-the-mill Rails validation error response without touching the
database.

On a related note, this validation strikes me as something that you
might want to consider doing on the client side, since you have all
the necessary information available there, and it could end up being
much simpler there.

[Greg Hauptmann]

Matt/Ryan/Adam - thanks.

I think I'd need to ponder further your suggestions and do some tests to let things sink in.  In the meantime to answer some of your questions:

My Goal - Clarify whether Rails could be used (via validation/before_save/after_save type hooks) to provide a solid protection for a cross-model business rule.  That is to protect the developer for making a mistake and for example accidentally making changes (in his/her code) that could lead to business rule violation (i.e. assuming they didn't change the validation/before_create/after_create checks themselves).     

I think in summary what I'm hearing is:

(a) It's not really possible, however

(b) It is possible to achieve protection for the business rule if the developer follows an appropriate approach to making updates/changes, i.e. such that an appropriate validation would kick in if necessary.  

Am I correct here?  

Thanks

--
Greg
http://blog.gregnet.org/