but I can't get this to work properly for the life of me. I have several call like this
scheduler.every '15m' do Url.update_xxxxx end
that just go out to an rss feed and populate my db with new stories. When I run it *without* :blocking => true, I my mysql connections slowly get used up until they hit my limit (even if I keep increasing it)
if I run it like this
scheduler.every '15m' do Url.update_xxxxx ActiveRecord::Base.remove_connection end
then I get a ''ActiveRecord::ConnectionNotEstablished'" error, which persists even if I do this
scheduler.every '15m' do ActiveRecord::Base.verify_active_connections! Url.update_xxxxx ActiveRecord::Base.remove_connection end
The only way to get this to run properly is with :blocking set to true, but then I have a problem with jobs stacked on top of each other that just hogs my cpu Any ideas?
i'm using ruby 1.8.7 with rails 2.3.11, and rufus-scheduler (2.0.10)
> but I can't get this to work properly for the life of me.
> I have several call like this
> scheduler.every '15m' do
> Url.update_xxxxx
> end
> that just go out to an rss feed and populate my db with new stories.
> When I run it *without* :blocking => true, I my mysql connections slowly
> get used up until they hit my limit (even if I keep increasing it)
> if I run it like this
> scheduler.every '15m' do
> Url.update_xxxxx
> ActiveRecord::Base.remove_connection
> end
> then I get a ''ActiveRecord::ConnectionNotEstablished'" error, which
> persists even if I do this
> scheduler.every '15m' do
> ActiveRecord::Base.verify_active_connections!
> Url.update_xxxxx
> ActiveRecord::Base.remove_connection
> end
> The only way to get this to run properly is with :blocking set to true, but
> then I have a problem with jobs stacked on top of each other that just hogs
> my cpu
> Any ideas?
> i'm using ruby 1.8.7 with rails 2.3.11, and rufus-scheduler (2.0.10)
Hello,
could the new :mutex attribute help?
---8<---
scheduler.every '15m', :mutex => 'ar_mutex' do
ActiveRecord::Base.verify_active_connections!
Url.update_xxxxx
#ActiveRecord::Base.remove_connection
end
--->8---
Making sure that all the rufus jobs dealing with active record share the same
mutex [name]...
> Now, thinking a bit more about, I'd say none, unless you have jobs that > don't > deal with ActiveRecord (and wouldn't have to wait on the mutex).
Ah right, so there nothing we can do, except (possibly) explicitly ActiveRecord::Base.establish_connection and ActiveRecord::Base.remove_connection before/after the ActiveRecord call?
On Mon, May 07, 2012 at 10:49:22PM -0700, concept47 wrote:
> Ah right, so there nothing we can do, except (possibly)
> explicitly ActiveRecord::Base.establish_connection
> and ActiveRecord::Base.remove_connection before/after the ActiveRecord
> call?
Yes. Probably.
---8<---
def ActiveRecord.connect(&block)
ActiveRecord::Base.establish_connection
block.call
ActiveRecord::Base.remove_connection
end
--->8---
On Wed, May 09, 2012 at 03:14:04PM -0500, Ike Ofili wrote:
> > Now, thinking a bit more about, I'd say none, unless you have jobs that don't
> > deal with ActiveRecord (and wouldn't have to wait on the mutex).
> Another question for you, do jobs create new threads each time they
> run? Or do they run in the same thread all the time?
Hello Ike,
yes, by default each job executes in a new, dedicated thread, unless you use
:blocking => true.
Aaaaaah this explains my problem! :D
Each of my jobs starts a thread, opens up a new ActiveRecord
connection and then doesn't close it, so my connection pool gets maxed
out within a few hours.
I've tried using
ActiveRecord::Base.connection_pool.with_connection do |connection|
do something
end
but then I have to use the connection which means I can't call a class
method on my ActiveRecord model ... so close ... but yet so far :\
should check out a connection from the pool set aside for Rails in your database.yml pool setting, let your active record call use it and then check it back in, solving our problem. **But this only works in rails 3+** ... you can see the code change here
