Message notification mails don't adhere to the configuration

[Benjamin Podszun]

Hi.

Finally made my copy of gitorious to work just fine. I had to do one
modification though: My setup cannot use the local sendmail for all
emails. I changed config/environments/production.rb to define

config.action_mailer.delivery_method = :smtp

config.action_mailer.smtp_settings = {
:address => "myhost",
:port => "25",
:domain => "myhost",
:authentication => :plain,
:user_name => "theAccount",
:password => "thePassword"
}

which worked fine for most cases (registration mail, forgot password
mail etc.), but failed to work for messages sent from user a to user
b. I cannot read ruby at all, but it seems that this happens in a
different way, submitting an event to the active messaging queue,
which results in the poller script sending out the message
asynchronously.

The problem? The poller script doesn't adhere to the rails
configuration above. I "fixed" it, being completely ignorant about the
language, by changing script/poller to include

require File.dirname(__FILE__) + '/../config/boot'

at the start, which among others seems to load the rails
environment...
Now - it obviously works now, but I'm sure I did something very nasty
here. What would be the _right_ way to configure the smtp settings for
the poller script/the message_forwarding_processor.rb?

Thanks in advance,
Ben

[Marius Mårnes Mathiesen]

On Wed, Dec 30, 2009 at 5:03 PM, Benjamin Podszun <benjamin...@gmail.com> wrote:

Now - it obviously works now, but I'm sure I did something very nasty
here. What would be the _right_ way to configure the smtp settings for
the poller script/the message_forwarding_processor.rb?

Ben,
For being "ignorant about Ruby" I must say you did some good work :-)

Actually, the config/boot file should be loaded through quite a few steps by the script/poller file (this loads another file which loads another file which ... you get the idea.) Before changing this: what happened? Did you get any errors? And do you supply a RAILS_ENV when running the poller script? 

Regards,

- Marius

[Benjamin Podszun]

On Mon, Jan 4, 2010 at 10:22 AM, Marius Mårnes Mathiesen
<marius.m...@gmail.com> wrote:
> On Wed, Dec 30, 2009 at 5:03 PM, Benjamin Podszun
> <benjamin...@gmail.com> wrote:
>>
>> Now - it obviously works now, but I'm sure I did something very nasty
>> here. What would be the _right_ way to configure the smtp settings for
>> the poller script/the message_forwarding_processor.rb?

Hi Marius.

> Ben,
> For being "ignorant about Ruby" I must say you did some good work :-)

Just "messing around". More dangerous than useful :)

> Actually, the config/boot file should be loaded through quite a few steps by
> the script/poller file (this loads another file which loads another file
> which ... you get the idea.) Before changing this: what happened? Did you
> get any errors? And do you supply a RAILS_ENV when running the poller
> script?

Before changing this setting I got an error from the sendmail
configuration: It basically just tried to send it out using my local
sendmail. This doesn't work (for now), for several reasons, one is
that I don't have a FQDN for that internal machine and the mail will
be rejected by any target MTA.
Yes, I can (and should!) fix that with a working smarthost
configuration or something, but so far I didn't bother.

The environments/production.rb configuration (see original mail) works
for now, but obviously wasn't used by the poller script: It didn't
connect directly and use the settings there dropped the notification
in my sendmail outbox and it bounced.

For reference: This is my /etc/init.d/git-poller init script:
/bin/su - git -c "cd /var/www/git.devtools/gitorious;
RAILS_ENV=production script/poller $@"

Just to be safe: This is my complete poller script:

#!/usr/bin/env ruby
# encoding: utf-8

if RUBY_VERSION < '1.9'
$KCODE = 'u'
else
Encoding.default_internal = Encoding::UTF_8
Encoding.default_external = Encoding::UTF_8
end

require File.dirname(__FILE__) + '/../config/boot'
require 'rubygems' if RUBY_VERSION < '1.9'
require 'daemons'

APP_ROOT = File.expand_path(File.dirname(__FILE__) + '/..')
script_file = File.join(File.expand_path(APP_ROOT),'vendor','plugins','activemessaging','poller.rb')
tmp_dir = File.join(File.expand_path(APP_ROOT), 'tmp/pids')

options = {
:app_name => "poller",
:dir_mode => :normal,
:dir => tmp_dir,
:multiple => true,
:ontop => false,
:mode => :load,
:backtrace => true,
:monitor => false,
:log_output => true
}

Daemons.run(script_file,options)

Again, I'm still ignorant, but ruby gems is afaik the "CPAN of ruby",
right? Is this connected to rails? That's the only "require" that was
in the file, other than "daemons". Which script should include the
environment configuration or the rails stuff? If you give me a hint, a
starting point, I might be able to track this further down. For now
I'm clueless.

Thanks for the help so far. Much appreciated!

Regards,
Ben

[Benjamin Podszun]

On Mon, Jan 4, 2010 at 2:59 PM, Benjamin Podszun
<benjamin...@gmail.com> wrote:
> On Mon, Jan 4, 2010 at 10:22 AM, Marius Mårnes Mathiesen

> Again, I'm still ignorant, but ruby gems is afaik the "CPAN of ruby",
> right? Is this connected to rails? That's the only "require" that was
> in the file, other than "daemons". Which script should include the
> environment configuration or the rails stuff? If you give me a hint, a
> starting point, I might be able to track this further down. For now
> I'm clueless.

Sorry for replying to myself, but I understand now that the poller
script loads/runs vendor/plugins/activemessaging/poller.rb, which in
turn includes

# Load Rails
RAILS_ROOT=File.expand_path(File.join(File.dirname(__FILE__), '..','..','..'))
load File.join(RAILS_ROOT, 'config', 'environment.rb')

So that _looks_ good, but again: It seems that this doesn't use the
settings defined in environments/production.rb for me. I can see that
boot.rb includes environment.rb as well.

Possible ideas (can you shoot them down?):
- boot.rb does something to the ActionMailer class (creating an
instance that reads the configuration, whatever)
- boot.rb predefines something that leads to the parsing of
environments/production.rb

Thanks for the help and regards,
Ben

[Marius Mårnes Mathiesen]

On Mon, Jan 4, 2010 at 3:18 PM, Benjamin Podszun <benjamin...@gmail.com> wrote:

On Mon, Jan 4, 2010 at 2:59 PM, Benjamin Podszun

<benjamin...@gmail.com> wrote:

So that _looks_ good, but again: It seems that this doesn't use the

settings defined in environments/production.rb for me. I can see that
boot.rb includes environment.rb as well.

environment.rb is the file normally included/required in order to load the Rails environment; boot.rb is loaded from this.

 

Possible ideas (can you shoot them down?):
- boot.rb does something to the ActionMailer class (creating an
instance that reads the configuration, whatever)
- boot.rb predefines something that leads to the parsing of
environments/production.rb

My first suspicion would be that your RAILS_ENV doesn't get picked up by script/poller; however requiring boot.rb shouldn't affect this. Does a git diff show that this is the *only* change you made? In that case, does it work if you require config/environment instead of config/boot?

Regards,

Marius

[Benjamin Podszun]

On Tue, Jan 5, 2010 at 10:51 AM, Marius Mårnes Mathiesen
<marius.m...@gmail.com> wrote:
> On Mon, Jan 4, 2010 at 3:18 PM, Benjamin Podszun
> <benjamin...@gmail.com> wrote:
>>
>> On Mon, Jan 4, 2010 at 2:59 PM, Benjamin Podszun
>> <benjamin...@gmail.com> wrote:
>> So that _looks_ good, but again: It seems that this doesn't use the
>> settings defined in environments/production.rb for me. I can see that
>> boot.rb includes environment.rb as well.
>
> environment.rb is the file normally included/required in order to load the
> Rails environment; boot.rb is loaded from this.

The other way around, no? boot.rb starts with
# Configure your app in config/environment.rb and config/environments/*.rb

and later defines a method "read_environment_rb"

>>
>> Possible ideas (can you shoot them down?):
>> - boot.rb does something to the ActionMailer class (creating an
>> instance that reads the configuration, whatever)
>> - boot.rb predefines something that leads to the parsing of
>> environments/production.rb
>
> My first suspicion would be that your RAILS_ENV doesn't get picked up by
> script/poller; however requiring boot.rb shouldn't affect this. Does a git
> diff show that this is the *only* change you made? In that case, does it
> work if you require config/environment instead of config/boot?

bpodszun@tis-gitorious:/var/www/git.devtools/gitorious$ git diff script/poller
diff --git a/script/poller b/script/poller
index 9a4ee3f..7881cf1 100755
--- a/script/poller
+++ b/script/poller
@@ -8,6 +8,7 @@ else

Encoding.default_external = Encoding::UTF_8
end

+require File.dirname(__FILE__) + '/../config/boot'

require 'rubygems' if RUBY_VERSION < '1.9'
require 'daemons'

I'm pretty sure I did that (including environment.rb) first (because I
knew I touched the environment stuff and didn't even know that a
boot.rb script exists or for what it is anyway) and it didn't work. I
can test it again though, later today.

Thanks for the help,
Ben

[Marius Mårnes Mathiesen]

On Tue, Jan 5, 2010 at 11:14 AM, Benjamin Podszun <benjamin...@gmail.com> wrote:

On Tue, Jan 5, 2010 at 10:51 AM, Marius Mårnes Mathiesen

> environment.rb is the file normally included/required in order to load the
> Rails environment; boot.rb is loaded from this.

The other way around, no?

Actually, no - Rails takes care of loading the files for you provided you load environment.rb

If you open an IRB session and "require config/environment" everything needed is loaded, which is not the case if you "require config/boot":

<RAILS_ROOT> irb

irb(main):001:0> require "config/boot"

=> true

irb(main):002:0> User

NameError: uninitialized constant User

from (irb):2

from /usr/local/bin/irb:12:in `<main>'

<RAILS_ROOT> irb

irb(main):001:0> require "config/environment"

=> true

irb(main):002:0> User

=> User(id: integer, login: string, email: string, crypted_password: string, salt: string, created_at: datetime, updated_at: datetime, remember_token: string, remember_token_expires_at: datetime, activation_code: string, activated_at: datetime, ssh_key_id: integer, fullname: string, url: text, identity_url: text, is_admin: boolean, suspended_at: datetime, aasm_state: string, public_email: boolean, wants_email_notifications: boolean, password_key: string, avatar_file_name: string, avatar_content_type: string, avatar_file_size: integer, avatar_updated_at: datetime)

In my understanding, including environment.rb should be enough to get a working Rails environment. 

Side note: The poller script is basically part of the activemessaging Rails plugin (http://code.google.com/p/activemessaging/), with only slight modifications for Gitorious. 

- Marius

[Johan Sørensen]

On Wed, Dec 30, 2009 at 5:03 PM, Benjamin Podszun
<benjamin...@gmail.com> wrote:
> emails. I changed config/environments/production.rb to define

[...]

> The problem? The poller script doesn't adhere to the rails
> configuration above. I "fixed" it, being completely ignorant about the
> language, by changing script/poller to include

Make sure you start it in the correct environment, eg:
$ env RAILS_ENV=production script/poller ...

I think the poller defaults to the development environment, whereas
the others default to production..

Cheers,
JS

[Benjamin Podszun]

Hi Johan.

Thanks again, but I think I'm doing exactly that. Quoting from my
previous mails:

For reference: This is my /etc/init.d/git-poller init script:
/bin/su - git -c "cd /var/www/git.devtools/gitorious;
RAILS_ENV=production script/poller $@"

Am I missing a typo in there? In addition: It works if I include the
(global, residing in config/) boot.rb.
I'm not directly including any environments/*.rb file, so somehow it
seems to pick up my environments/production.rb on its own (which I
assume means that the right environment is selected?).

Regards,
Ben

[Benjamin Podszun]

Thank you for the ruby quickstart. Poking around in irb I created this
series of "tests". Are they useful?

Reminder: This is what I have now:

bpodszun@tis-gitorious:/var/www/git.devtools/gitorious$ git diff script/poller
diff --git a/script/poller b/script/poller
index 9a4ee3f..7881cf1 100755
--- a/script/poller
+++ b/script/poller
@@ -8,6 +8,7 @@ else
Encoding.default_external = Encoding::UTF_8
end

+require File.dirname(__FILE__) + '/../config/boot'
require 'rubygems' if RUBY_VERSION < '1.9'
require 'daemons'

I created a copy of the poller script that just sources everything and
outputs the settings I need:

bpodszun@tis-gitorious:/var/www/git.devtools/gitorious$ diff
script/poller script/poller_test
11d10
< require File.dirname(__FILE__) + '/../config/boot'
31c30,31
< Daemons.run(script_file,options)
---
> puts ActionMailer::Base.smtp_settings
> #Daemons.run(script_file,options)

bpodszun@tis-gitorious:/var/www/git.devtools/gitorious$
RAILS_ENV=production script/poller_test
script/poller_test:30:in `<main>': uninitialized constant ActionMailer
(NameError)

Modified the test script to include environment:

bpodszun@tis-gitorious:/var/www/git.devtools/gitorious$ diff
script/poller script/poller_test
11c11
< require File.dirname(__FILE__) + '/../config/boot'
---
> require 'config/environment'
31c31,32
< Daemons.run(script_file,options)
---
> puts ActionMailer::Base.smtp_settings
> #Daemons.run(script_file,options)
bpodszun@tis-gitorious:/var/www/git.devtools/gitorious$
RAILS_ENV=production script/poller_test
{:address=>"MyHost", :port=>"25", :domain=>"MyHost",
:authentication=>:plain, :user_name=>"Foo@MyHost",
:password=>"MyPassword"}

So that works. Going back to 'config/boot' instead doesn't work, as
you said, and results in the same error as above (uninitialized
constant).

Two questions at this point:

1) Why does the initial change that I made work at all (it does, just
verified it again). I'll play around some after work/tonight..

2) Should these settings be available at that point of the script at all?

Thanks,
Ben

[Marius Mårnes Mathiesen]

On Tue, Jan 5, 2010 at 1:06 PM, Benjamin Podszun <benjamin...@gmail.com> wrote:

1) Why does the initial change that I made work at all (it does, just
verified it again). I'll play around some after work/tonight..

In my opinion, it should work without needing to require boot/environment; the poller script needs to run a full Gitorious environment to function properly. What's a little strange is that almost everything seems to work for you (?) - apart from the ActionMailer settings. 

 

2) Should these settings be available at that point of the script at all?

If you're referring to ActionMailer::Base's smtp_settings: absolutely!  If requiring config/environment in script/poller helps, something that should be loaded isn't being loaded. 

So long :-) 

- Marius

[Benjamin Podszun]

On Tue, Jan 5, 2010 at 1:31 PM, Marius Mårnes Mathiesen
<marius.m...@gmail.com> wrote:
> On Tue, Jan 5, 2010 at 1:06 PM, Benjamin Podszun
> <benjamin...@gmail.com> wrote:
>>
>> 1) Why does the initial change that I made work at all (it does, just
>> verified it again). I'll play around some after work/tonight..
>
> In my opinion, it should work without needing to require boot/environment;
> the poller script needs to run a full Gitorious environment to function
> properly. What's a little strange is that almost everything seems to work
> for you (?) - apart from the ActionMailer settings.

Let me shoot back: What else from the configuration is used in that
script? What does it do? It _does_ create projects/add keys/send
notifications - the latter after messing around.

>> 2) Should these settings be available at that point of the script at all?
>
> If you're referring to ActionMailer::Base's smtp_settings: absolutely!  If
> requiring config/environment in script/poller helps, something that should
> be loaded isn't being loaded.

Yes, that's what I was referring to. Can you reproduce this behaviour?

Thanks for the fast replies,
Ben

[Marius Mårnes Mathiesen]

On Tue, Jan 5, 2010 at 1:48 PM, Benjamin Podszun <benjamin...@gmail.com> wrote:

Let me shoot back: What else from the configuration is used in that
script? What does it do? It _does_ create projects/add keys/send
notifications - the latter after messing around.

What it does is process/consume messages from the message queue, typically actions that take too long to be done synchronously while the user is waiting for a web request to finish/a git push to complete. The app/processors directory contains processors that consume messages sent from other parts of the application; one processor per message queue. The wiki has some more information:

  http://gitorious.org/gitorious/pages/GitoriousComponents

Basically, the poller will call scripts that can do "anything" - and it does in fact do things like copy git repositories on disk, add events when a series of commits is pushed etc.

 

> If you're referring to ActionMailer::Base's smtp_settings: absolutely!  If
> requiring config/environment in script/poller helps, something that should
> be loaded isn't being loaded.

Yes, that's what I was referring to. Can you reproduce this behaviour?

I'm sorry, but I lost track of what worked and what didn't in your installation (did requiring environment help?)

 

Thanks for the fast replies,

Not always as fast as it could though...

Cheers,

- Marius