scheduler not running under detached mongrel

[Stringfellow Hawke]

Not sure what I did to break this, the scheduler has been running for
over a year flawlessly. But now, it's not firing when I start the
server with the detached option (-d) either in dev or prod. Works
fine if it runs in the shell. Haven't updated any gems.

Ubuntu 10.04
ruby 1.8.7 (2010-01-10 patchlevel 249)
rails 3.0.10
mongrel 1.1.5
rufus-scheduler 2.0.10

I'm guessing it's something trivial, not sure how to go about
troubleshooting. Here's the sample code for initializers
\task_schedular i'm using now to troubleshoot:

require 'rubygems'
require 'rufus/scheduler'

Notifier.send_test.deliver

scheduler = Rufus::Scheduler.start_new
scheduler.every '2m' do
Notifier.send_test.deliver
end


Notifier.send_test is a simple email that gets fired. I get the first
email, so the initializer file is loading, but that's it.

Any help would be much appreciated.

[John Mettraux]

On Sat, Sep 03, 2011 at 12:52:17PM -0700, Stringfellow Hawke wrote:
>
> Not sure what I did to break this, the scheduler has been running for
> over a year flawlessly. But now, it's not firing when I start the
> server with the detached option (-d) either in dev or prod. Works
> fine if it runs in the shell. Haven't updated any gems.

Hello,

sorry for the late reply, I was travelling.

Are you telling me that previously you were not using the -d option ?

--
John Mettraux - http://lambda.io/processi

[Stringfellow Hawke]

no worries thnx for getting back to me

I was always using the -d option, now the scheduler only runs when I
don't use the -d option

[John Mettraux]

On Sun, Sep 04, 2011 at 03:01:11PM -0700, Stringfellow Hawke wrote:
>
> I was always using the -d option, now the scheduler only runs when I
> don't use the -d option

Hello,

what about trying something like

---8<---
loop do
p Time.now
Notifier.send_test.deliver
sleep 2 * 60
end
--->8---

to see if it's really a rufus-scheduler issue ?

Best regards,

[Stringfellow Hawke]

tried that out, the loop was working but it wouldn't daemonize I guess
since it's stuck in that loop, that app didn't finish starting either

anyway, i've been meaning to switch to one of the newer servers so I
tried it on unicorn and the problem is gone, so let's just blame it on
mongrel ;)

thanks for your help, if i figure out what the problem was, i'll post
it

[James Lopez]

Hello,

I've got exactly the same issue... However, I couldn't change my
current production machine configuration to switch from using
Mongrel...

The application versions I'm running are:

Ubuntu 10.04
Ruby 1.9.2
Rails 3.0.10
Mongrel 1.2.0.pre2-x86-mingw32
rufus-scheduler 2.0.10

When I run rails with the -d option the scheduler is working. When
using the -d option the code inside the task is never called and
nothing is logged.

Any advise would be much appreciated.

[John Mettraux]

On Tue, Sep 27, 2011 at 03:27:40AM -0700, James Lopez wrote:
>
> When I run rails with the -d option the scheduler is working. When
> using the -d option the code inside the task is never called and
> nothing is logged.

Hello,

I don't understand this paragraph. Could you please rephrase it (with more details as I haven't used mongrel in while) ?

Thanks in advance,

[James Lopez]

Hi,

If I execute the following in the command line, in order to run Rails
in my production environment:

rails server -e production -p 8000 -d

The task supposed to be scheduled is never executed, and therefore,
never runs.

However, it does run, if I execute the following:

rails server -e production -p 8000

The number of threads is 3, according to the output of ps aux -L

Mongrel is booted just after starting the the rails server, and I have
not changed any specific configuration in Mongrel, apart from the port
it listens to.

Hope this helps.

[John Mettraux]

On Tue, Sep 27, 2011 at 05:30:36AM -0700, James Lopez wrote:
>
> If I execute the following in the command line, in order to run Rails
> in my production environment:
>
> rails server -e production -p 8000 -d
>
> The task supposed to be scheduled is never executed, and therefore,
> never runs.

What do you mean by "never executed" ? Do you mean that the initializer code where the scheduler is initialized is never interpreted ?

"the task is never executed and therefore it never runs" what's the difference between to execute and to run ?

> However, it does run, if I execute the following:
>
> rails server -e production -p 8000
>
> The number of threads is 3, according to the output of ps aux -L

In which case -d or no -d ?

> Mongrel is booted just after starting the rails server, and I have

> not changed any specific configuration in Mongrel, apart from the port
> it listens to.

I don't understand the "mongrel is booted after the rails server" part.

Have you try using mongrel_rails [-d] like explained in

https://github.com/fauna/mongrel

?

Best regards,

[James Lopez]

On Sep 27, 1:43 pm, John Mettraux <jmettr...@gmail.com> wrote:

> What do you mean by "never executed" ? Do you mean that the initializer code where the scheduler is initialized is never interpreted ?

Yes, I mean that the following peace of code inside the initialiser is
never interpreted:

scheduler = Rufus::Scheduler.start_new
scheduler.in("50s") do
Rails.logger.info "test"
end

> "the task is never executed and therefore it never runs" what's the difference between to execute and to run ?

Roughly, if a method is executed means that it has been called. Once
it's been called it should be "running".

> In which case -d or no -d ?

Using -d

> I don't understand the "mongrel is booted after the rails server" part.
>
> Have you try using mongrel_rails [-d] like explained in
>
>  https://github.com/fauna/mongrel
>
> ?

The output after 'rails server -e production -p 8000 -d' is the
following:

rails server -e production -p 8000 -d
=> Booting Mongrel
=> Rails 3.0.10 application starting in production on http://0.0.0.0:8000

I don't need to run Mongrel again as this has been started already.

Thanks in advance.

[John Mettraux]

On Tue, Sep 27, 2011 at 05:56:06AM -0700, James Lopez wrote:
> On Sep 27, 1:43�pm, John Mettraux <jmettr...@gmail.com> wrote:
>
> > What do you mean by "never executed" ? Do you mean that the initializer code where the scheduler is initialized is never interpreted ?
>
> Yes, I mean that the following peace of code inside the initialiser is
> never interpreted:
>
> scheduler = Rufus::Scheduler.start_new
> scheduler.in("50s") do
> Rails.logger.info "test"
> end

So if I understand correctly this code is not interpreted. Where did you place it ? In config/initializers/ ?

If in /config/initializers/ the next question is: are the other initializers not run as well when -d is used ?

Thanks in advance,

[James Lopez]

> If in /config/initializers/ the next question is: are the other initializers not run as well when -d is used ?

Yes, I use an .rb file in config/initializers. The other initilizers
run properly, I've got quite a few of them there like devise,
formtastic, etc. and all of them are working correctly.

Another thing I realised is that using -d only 3 threads are used as I
stated, but not using -d 4 threads are running, I assume the fourth
thread is the one from the scheduler...

[John Mettraux]

On Wed, Sep 28, 2011 at 01:10:06AM -0700, James Lopez wrote:
>
> > If in /config/initializers/ the next question is: are the other initializers not run as well when -d is used ?
>
> Yes, I use an .rb file in config/initializers. The other initilizers
> run properly, I've got quite a few of them there like devise,
> formtastic, etc. and all of them are working correctly.

Hello,

try putting a puts "in" in config/initializers/whatever_the_name.rb before the scheduler instantation and a puts "out" after the scheduler instantiation. This would tell us if the initialization code gets executed or not (if "in" and "out" somehow appear in some log, then the scheduler initialization code got executed and we're left determining why it doesn't trigger).

Try also with Thread.abort_on_exception = true, it might yield something interesting in some log.

> Another thing I realised is that using -d only 3 threads are used as I
> stated, but not using -d 4 threads are running, I assume the fourth
> thread is the one from the scheduler...

That's an excellent piece of information. Would those three threads be mongrel workers ? Maybe some googling would help.

By the way, you haven't told me what happened when starting with mongrel_rails instead of rails.

I will try spending an hour setting up a test environment similar to yours and running tests by myself.

Best regards,

[John Mettraux]

2011/9/28 John Mettraux <jmet...@gmail.com>:

>
> I will try spending an hour setting up a test environment similar to yours and running tests by myself.

James,

by the way, which patch level of Ruby 1.9.2 are you using ? 290 ?

Thanks in advance,

[James Lopez]

Thank you very much John, your help is much appreciated!

I'm using ruby 1.9.2p180

I will get back to you once I gather more information. It seems that
I'm not able to run mongrel_rails instead of running it through the
rails server... as I have the same issue as stated here:

http://stackoverflow.com/questions/3851960/rails-3-0-mongrel-issue

On Sep 28, 10:31 am, John Mettraux <jmettr...@gmail.com> wrote:
> 2011/9/28 John Mettraux <jmettr...@gmail.com>:

[John Mettraux]

On Wed, Sep 28, 2011 at 06:31:29PM +0900, John Mettraux wrote:
> 2011/9/28 John Mettraux <jmet...@gmail.com>:
> >
> > I will try spending an hour setting up a test environment similar to yours and running tests by myself.
>
> James,
>
> by the way, which patch level of Ruby 1.9.2 are you using ? 290 ?

OK, whatever, trying with 290 on Ubuntu 11.04 (sorry I have not 10.04 around).

I've created this mini-project

https://github.com/jmettraux/james_lopez

Sticking to, hopefully, the versions you indicated to me:

https://github.com/jmettraux/james_lopez/blob/master/Gemfile

My initializer looks like:

https://github.com/jmettraux/james_lopez/blob/master/config/initializers/scheduler.rb

Please note that I'm trying both with a thread and both with rufus-scheduler.

The result ? It works without -d, it doesn't work with -d.

I have observed, like you did that the number of [green] threads per process is one less for -d. But I had 5 threads for -d.

I decided to look a bit deeper and came up with this controller:

https://github.com/jmettraux/james_lopez/blob/master/app/controllers/status_controller.rb

I curled it for normal operations:

https://github.com/jmettraux/james_lopez/blob/master/out/status_non_d.txt

and for backgrounded operations:

https://github.com/jmettraux/james_lopez/blob/master/out/status_d.txt

The threads seem to be present.

So the initializer is executed, contradicting your affirmation:

> Yes, I mean that the following peace of code inside the initialiser is
> never interpreted:
>
> scheduler = Rufus::Scheduler.start_new
> scheduler.in("50s") do
> Rails.logger.info "test"
> end

Now why the thread seem not to run... I don't know. If I were you, I'd dig in the mongrel documentation. I'm sure you're not the first one to have that issue.

I could tell you to use Thin or another server, but you already told me you're stuck with Mongrel. Maybe you could try with Mongrel 1.1.5 (the stable release according to https://rubygems.org/search?utf8=%E2%9C%93&query=mongrel)

I've tried googling for things like "mongrel initializer thread" but had no success, it seems all centered on Rails and thread-safety.

I don't think I'll take the time to investigate that further.

Maybe you could try Clockwork (https://github.com/adamwiggins/clockwork), but it also uses a thread like rufus-scheduler. So as long as threads in an initializer seem to get lost, it's probably not going to work.

All the best.