[AMaViS-user] Using RelayCountry with amavis?

[Per olof Ljungmark]

Hi,

I am fiddling with Mail::SpamAssassin::Plugin::RelayCountry in our
Spamassassin config, and it seems to load ok from init.pre:
spamassain -D --lint:
[85215] dbg: plugin: loading Mail::SpamAssassin::Plugin::RelayCountry
from @INC

However, it does not show up when I start amavisd:

amavis[85275]: extra modules loaded after daemonizing:
Mail/DomainKeys/Header.pm, Mail/DomainKeys/Key.pm,
Mail/DomainKeys/Key/Public.pm, Mail/DomainKeys/Message.pm,
Mail/DomainKeys/Policy.pm, Mail/DomainKeys/Signature.pm,
Mail/SpamAssassin/Plugin/DomainKeys.pm, Mail/SpamAssassin/Plugin/PDFInfo.pm

Anyone on the list using this plugin that can give me a hint here? Using
latest stable release.

Thanks,

--per

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
AMaViS-user mailing list
AMaVi...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Bill Landry]

Per olof Ljungmark wrote:
> Hi,
>
> I am fiddling with Mail::SpamAssassin::Plugin::RelayCountry in our
> Spamassassin config, and it seems to load ok from init.pre:
> spamassain -D --lint:
> [85215] dbg: plugin: loading Mail::SpamAssassin::Plugin::RelayCountry
> from @INC
>
> However, it does not show up when I start amavisd:
>
> amavis[85275]: extra modules loaded after daemonizing:
> Mail/DomainKeys/Header.pm, Mail/DomainKeys/Key.pm,
> Mail/DomainKeys/Key/Public.pm, Mail/DomainKeys/Message.pm,
> Mail/DomainKeys/Policy.pm, Mail/DomainKeys/Signature.pm,
> Mail/SpamAssassin/Plugin/DomainKeys.pm, Mail/SpamAssassin/Plugin/PDFInfo.pm
>
> Anyone on the list using this plugin that can give me a hint here? Using
> latest stable release.

Maybe the RelayCountry module has not been included in
"additional_perl_modules" in amavisd? I think you can add it to your
amavisd.conf. From the RELEASE_NOTES for the amavisd-new-2.4.3 section:

- added a global configuration variable @additional_perl_modules, which
is a list of additional Perl module names or absolute file names that
should be compiled/executed (by calling 'require') at a program startup
time by a master parent process, before chroot-ing and before changing
UID takes place. Its purpose is to pre-load additional non-standard
SpamAssassin plugins and similar modules that a standard SpamAssassin
initialization would miss, causing them to be loaded later by each
child process, which is inefficient and may not work in a chrooted
process. Example:
@additional_perl_modules = qw(
/usr/local/etc/mail/spamassassin/FuzzyOcr.pm
/usr/local/etc/mail/spamassassin/ImageInfo.pm
/usr/local/etc/mail/spamassassin/WebRedirect.pm
String::Approx Net::HTTP Net::HTTP::Methods
URI URI::http URI::_generic URI::_query URI::_server
HTTP::Date HTTP::Headers HTTP::Message HTML::HeadParser
HTTP::Request HTTP::Response HTTP::Status
LWP LWP::Protocol LWP::Protocol::http
LWP::UserAgent LWP::MemberMixin LWP::Debug
);
Make sure these files are owned by root and not writable by unprivileged
users such as amavis!

Try setting up a .cf file for RelayCountry, for example:
=====
ifplugin Mail::SpamAssassin::Plugin::RelayCountry

header RELAY_BR X-Relay-Countries =~ /\bBR\b/
describe RELAY_BR Relayed through Brazil
score RELAY_BR 1.0

header RELAY_CN X-Relay-Countries =~ /\bCN\b/
describe RELAY_CN Relayed through China
score RELAY_CN 1.0

header RELAY_HK X-Relay-Countries =~ /\bHK\b/
describe RELAY_HK Relayed through Hong Kong
score RELAY_HK 1.0

endif
=====

and see if SA reports entries like:

RELAY_CN=1, RELAY_CN Relayed through China

Bill

[Per olof Ljungmark]

Brad White wrote:
> It doesn't show up in the amavis generated log entries on my system, but it
> is definitely working. I does show up when I run a spamassasin -D --lint:
>
> [14432] dbg: plugin: loading Mail::SpamAssassin::Plugin::RelayCountry from
> @INC
>
> I have the plugin enabled via the init.pre file in my spamassasin directory:
>
> # RelayCountry - add metadata for Bayes learning, marking the countries
> # a message was relayed through
> #
> # Note: This requires the IP::Country::Fast Perl module
> #
> loadplugin Mail::SpamAssassin::Plugin::RelayCountry
>
>
> You also need the extra configuration in your spamassassin local.cf file,
> something like:

>
> ifplugin Mail::SpamAssassin::Plugin::RelayCountry

> header RELAY_CN X-Relay-Countries =~ /\bCN\b/
> describe RELAY_CN Relayed through China
> score RELAY_CN 1.0

> header RELAY_KR X-Relay-Countries =~ /\bKR\b/
> describe RELAY_KR Relayed through Korea
> score RELAY_KR 1.0
> header RELAY_RU X-Relay-Countries =~ /\bRU\b/
> describe RELAY_RU Relayed through Russia
> score RELAY_RU 1.0
> header RELAY_US X-Relay-Countries =~ /\bUS\b/
> describe RELAY_US Relayed through United States
> score RELAY_US 0.001
> header RELAY_PL X-Relay-Countries =~ /\bPL\b/
> describe RELAY_PL Relayed through Poland
> score RELAY_PL 0.001
> endif # Mail::SpamAssassin::Plugin::RelayCountry
>
> This definitely works for me on my installation.

Yes you're right, it does not show up until you turn on a higher debug
level, thanks!

Also, the docs say "Also for 3.1.0, you can apply a patch [WWW]
http://bugzilla.spamassassin.org/show_bug.cgi?id=3815 which will allow
you to add a separate MIME header that shows all the message's relay
countries, independent of the rules."

add_header all Relay-Country _RELAYCOUNTRY_

this again I don't see, is there amavisd-tweaking involved here?

> On 10/3/08 5:08 AM, "Per olof Ljungmark" <p...@intersonic.se> wrote:
>
>> Hi,
>>
>> I am fiddling with Mail::SpamAssassin::Plugin::RelayCountry in our
>> Spamassassin config, and it seems to load ok from init.pre:
>> spamassain -D --lint:
>> [85215] dbg: plugin: loading Mail::SpamAssassin::Plugin::RelayCountry
>> from @INC
>>
>> However, it does not show up when I start amavisd:
>>
>> amavis[85275]: extra modules loaded after daemonizing:
>> Mail/DomainKeys/Header.pm, Mail/DomainKeys/Key.pm,
>> Mail/DomainKeys/Key/Public.pm, Mail/DomainKeys/Message.pm,
>> Mail/DomainKeys/Policy.pm, Mail/DomainKeys/Signature.pm,
>> Mail/SpamAssassin/Plugin/DomainKeys.pm, Mail/SpamAssassin/Plugin/PDFInfo.pm
>>
>> Anyone on the list using this plugin that can give me a hint here? Using
>> latest stable release.
>>

>> Thanks,
>>
>> --per

[mouss]

Per olof Ljungmark wrote:
> [snip]

>
> Also, the docs say "Also for 3.1.0, you can apply a patch [WWW]
> http://bugzilla.spamassassin.org/show_bug.cgi?id=3815 which will allow
> you to add a separate MIME header that shows all the message's relay
> countries, independent of the rules."
>
> add_header all Relay-Country _RELAYCOUNTRY_
>
> this again I don't see, is there amavisd-tweaking involved here?
>

there's an example in amavisd-custom.conf.

for example, you could add this at the end of your amavisd.conf:

#######
package Amavis::Custom;

BEGIN {

import Amavis::Conf qw(:platform :confvars c cr ca $myhostname);

import Amavis::Util qw(do_log untaint safe_encode safe_decode);

import Amavis::rfc2821_2822_Tools;

import Amavis::Notify qw(build_mime_entity);

}

sub new {

my($class,$conn,$msginfo) = @_;

my($self) = bless {}, $class;

$self; # returning an object activates further callbacks,

# returning undef disables them

}

sub before_send {

my($self,$conn,$msginfo) = @_;

my($all_local) = !grep { !$_->recip_is_local }
@{$msginfo->per_recip_data};

if ($all_local) {

my($hdr_edits) = $msginfo->header_edits;

my ($rly_country) =
$msginfo->supplementary_info('RELAYCOUNTRY');

$hdr_edits->add_header('X-Relay-Countries', $rly_country)

if defined $rly_country && $rly_country ne '';

my($languages) = $msginfo->supplementary_info('LANGUAGES');

$hdr_edits->add_header('X-Spam-Languages', $languages)

if defined $languages && $languages ne '';

}

}

#
1; # insure a defined return

[mouss]

Per olof Ljungmark wrote:
>
> OK, thanks. I've gotten so far now that amavisd reports the header but
> it is not inserted into the actual message headers.
>

I tested the snippet I sent you and it worked.

> amavis[1968]: (01968-01) header: X-Relay-Countries: VN\n
>
> --per

[Brad White]

I think that amavis handles all header re-writing and ignores any such
directives in the spamassassin config files. This info is in the amavis FAQ
btw. So, if you ran your message through spamassassin by hand you would
likely see the "X-Spam-Relay-Countries" header added to your message, but it
would not be there if the email passed through amavis. This is normal
behavior I believe.

[Per olof Ljungmark]

mouss wrote:
> Per olof Ljungmark wrote:
>>
>> OK, thanks. I've gotten so far now that amavisd reports the header but
>> it is not inserted into the actual message headers.
>>
>
> I tested the snippet I sent you and it worked.
>
>
>> amavis[1968]: (01968-01) header: X-Relay-Countries: VN\n

Yup, I'm getting there now. I had the (wrong) impression that the actual
country was looked up separately but reading the code again proved the
opposite, it is dug out from the RELAYCOUNTRY variable in SA.

Thanks a lot for your kind help.

[Per olof Ljungmark]

mouss wrote:
> Per olof Ljungmark wrote:
>> OK, thanks. I've gotten so far now that amavisd reports the header but
>> it is not inserted into the actual message headers.
>>
>
> I tested the snippet I sent you and it worked.
>
>
>> amavis[1968]: (01968-01) header: X-Relay-Countries: VN\n

The X-Relay-Countries header is present in ham mail but not in messages
marked as spam. What is the additional parameter then for including it
with spams as well?

[mouss]

Per olof Ljungmark wrote:
> mouss wrote:
>> Per olof Ljungmark wrote:
>>> OK, thanks. I've gotten so far now that amavisd reports the header
>>> but it is not inserted into the actual message headers.
>>>
>>
>> I tested the snippet I sent you and it worked.
>>
>>
>>> amavis[1968]: (01968-01) header: X-Relay-Countries: VN\n
>
> The X-Relay-Countries header is present in ham mail but not in messages
> marked as spam. What is the additional parameter then for including it
> with spams as well?

same reply: I tested the snippet I sent you and it worked, for both ham
and spam.

so please provide more information. In particular:

- show the headers of a message where the relay info isn't added

- show what you actually did to add the header (did you use the snippet
as I sent it or did you modify it? if you modified it, show the result)

PS. the header won't appear if the recipient is not "local" (same as for
X-Spam-* headers).

[Per olof Ljungmark]

mouss wrote:
> Per olof Ljungmark wrote:
>> mouss wrote:
>>> Per olof Ljungmark wrote:
>>>> OK, thanks. I've gotten so far now that amavisd reports the header
>>>> but it is not inserted into the actual message headers.
>>>>
>>> I tested the snippet I sent you and it worked.
>>>
>>>
>>>> amavis[1968]: (01968-01) header: X-Relay-Countries: VN\n
>> The X-Relay-Countries header is present in ham mail but not in messages
>> marked as spam. What is the additional parameter then for including it
>> with spams as well?
>
> same reply: I tested the snippet I sent you and it worked, for both ham
> and spam.
>
> so please provide more information. In particular:
>
> - show the headers of a message where the relay info isn't added
>
> - show what you actually did to add the header (did you use the snippet
> as I sent it or did you modify it? if you modified it, show the result)
>
>
> PS. the header won't appear if the recipient is not "local" (same as for
> X-Spam-* headers).

Yes, I suspect the recipient is not regarded as local in this case.
I use the unmodified snippet you provided (thanks).

All spams end up in spam...@inter-sonic.com which I presumed was a
local recipient because inter-sonic.com is in the local domain list and
inter-sonic.com is $MYDOMAIN.

I will turn on debug and see what that gives.

Here are the headers from a spam:

Return-Path: <>
Received: from imapserver.example.com ([unix socket])
by imapserver.example.com (Cyrus v2.3.12p2) with LMTPA;
Sun, 05 Oct 2008 11:16:26 +0200
X-Sieve: CMU Sieve 2.3
Received: from mail-router.example.com [IP here])
by imapserver.example.com (Postfix) with ESMTPSA id 04D196EB011
for <spam...@imapserver.example.com>; Sun, 5 Oct 2008 11:16:26 +0200
(CEST)
Received: from localhost (localhost [127.0.0.1])
by mail-router.example.com (Postfix) with ESMTP id CBF9A2EB824
for <spam...@inter-sonic.com>; Sun, 5 Oct 2008 11:16:25 +0200 (CEST)
X-Envelope-From: <forexd...@live.com>
X-Envelope-To: <us...@a-local-domain.tld>
X-Envelope-To-Blocked: <us...@a-local-domain.tld>
X-Quarantine-ID: <zTMoKYKJANeW>
X-Spam-Flag: YES
X-Spam-Score: 28.37
X-Spam-Level: ****************************
X-Spam-Status: Yes, score=28.37 tag=-999 tag2=4.8 kill=5.1
tests=[ADVANCE_FEE_2=1.234, ADVANCE_FEE_3=1.432, ADVANCE_FEE_4=0.639,
BAYES_99=5, DCC_CHECK=3, FORGED_MUA_OUTLOOK=3.116,
J_CHICKENPOX_37=0.6, MILLION_USD=1.528, MSOE_MID_WRONG_CASE=0.82,
RELAY_ES=0.1, RELAY_FR=-0.01, SARE_FRAUD_X3=3.5, SARE_FRAUD_X4=1.667,
SARE_FRAUD_X5=1.667, SUBJ_ALL_CAPS=2.077, US_DOLLARS_3=2]

<local received headers hidden>

Received: from smtp20.orange.fr (smtp20.orange.fr [193.252.22.31])
by station.inter-sonic.com (Postfix) with ESMTP id AAFAB13C41E
for <us...@a-local-domain.tld>; Sun, 5 Oct 2008 11:16:21 +0200 (CEST)
Received: from User (unknown [84.77.136.111])
by mwinf2021.orange.fr (SMTP Server) with ESMTP id 2D1EF1C0009D;
Sun, 5 Oct 2008 11:16:06 +0200 (CEST)
X-ME-UUID: 200810050916071...@mwinf2021.orange.fr
Reply-To: <llyodst...@hotmail.com>
From: "Lloyds TSB Pacific Bank" <forexd...@live.com>
Subject: **LLOYDS TSB PACIFIC LIMITED/ URGENT ATTENTION NEEDED**
Date: Sun, 5 Oct 2008 11:16:14 +0200
MIME-Version: 1.0
Content-Type: text/plain;
charset="Windows-1251"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
Message-Id: <2008100509160...@mwinf2021.orange.fr>
To: undisclosed-recipients:;

[mouss]

Per olof Ljungmark wrote:
>
> Yes, I suspect the recipient is not regarded as local in this case.
> I use the unmodified snippet you provided (thanks).
>
> All spams end up in spam...@inter-sonic.com which I presumed was a
> local recipient because inter-sonic.com is in the local domain list and
> inter-sonic.com is $MYDOMAIN.
>

The sample you show has X-Spam headers, so the domain is local.

> I will turn on debug and see what that gives.
>
> Here are the headers from a spam:

it should have an "FR ES".

run the message through spamassassin -t and see if it gets these. if it
doesn't, this is an SA issue, so re-run it with -D. For example
(assuming Bourne shell, not C shell):

spamassassin -D -t 2>&1 < sample.eml | tee /tmp/sa.out

[Per olof Ljungmark]

mouss wrote:
> Per olof Ljungmark wrote:
>> Yes, I suspect the recipient is not regarded as local in this case.
>> I use the unmodified snippet you provided (thanks).
>>
>> All spams end up in spam...@inter-sonic.com which I presumed was a
>> local recipient because inter-sonic.com is in the local domain list and
>> inter-sonic.com is $MYDOMAIN.
>>
>
> The sample you show has X-Spam headers, so the domain is local.
>
>> I will turn on debug and see what that gives.
>>
>> Here are the headers from a spam:
>
> it should have an "FR ES".
>
> run the message through spamassassin -t and see if it gets these. if it
> doesn't, this is an SA issue, so re-run it with -D. For example
> (assuming Bourne shell, not C shell):
>
> spamassassin -D -t 2>&1 < sample.eml | tee /tmp/sa.out

SA returns proper headings...
X-Spam-Relay-Countries: FR ES

what string should I look for in the amavisd debug output? I've dug
through quite a lote but cannot see anything relevant, sorry.

--per

[mouss]

Per olof Ljungmark wrote:
> mouss wrote:
>> Per olof Ljungmark wrote:
>>> Yes, I suspect the recipient is not regarded as local in this case.
>>> I use the unmodified snippet you provided (thanks).
>>>
>>> All spams end up in spam...@inter-sonic.com which I presumed was a
>>> local recipient because inter-sonic.com is in the local domain list
>>> and inter-sonic.com is $MYDOMAIN.
>>>
>>
>> The sample you show has X-Spam headers, so the domain is local.
>>
>>> I will turn on debug and see what that gives.
>>>
>>> Here are the headers from a spam:
>>
>> it should have an "FR ES".
>>
>> run the message through spamassassin -t and see if it gets these. if
>> it doesn't, this is an SA issue, so re-run it with -D. For example
>> (assuming Bourne shell, not C shell):
>>
>> spamassassin -D -t 2>&1 < sample.eml | tee /tmp/sa.out
>
> SA returns proper headings...
> X-Spam-Relay-Countries: FR ES
>
> what string should I look for in the amavisd debug output? I've dug
> through quite a lote but cannot see anything relevant, sorry.
>

For some reason, your messages don't appear on the list.

This happened to me not long ago (Thread "please remove CR
moron:From:...", my post stayed in the sourceforce queue from 27 Sept to
4 Oct.

anyway, you can add log statements in before_send() and see which part
of the flow is missed, if any. Maybe someone else has better ideas.

[Mark Martinec]

Per,

> The X-Relay-Countries header is present in ham mail but not in messages
> marked as spam. What is the additional parameter then for including it
> with spams as well?

Perhaps you are confusing quarantined mail with mail passed on
to recipients (either ham or spam). The patch from mouss
adds a X-Relay-Countries header field to passed mail,
not to quarantined mail.

Mark