Configure ONCE_RECEIVED

[Fabian Stanke]

Hi group!

Essentially: How is the once_received module configured correctly?

My use-case: automated mails from my local server get scores based on ONCE_RECEIVED and _STRICT rules that I would like to avoid.

I’m using rspamd as a transport filter in „rspamc —mime“ mode that all messages have to pass.

I created /var/lib/rspamd/dynamic/once_received.conf and created good_host entries for local and the local hostname. However there is no difference and emails still get scored with the above rules.

 

Thank you for any hints. Please ask if I need to provide more information.

Best regards,

Fabian

[Vsevolod Stakhov]

I'd recommend to reconfigure this module completely in
rspamd.conf.override, for example:

once_received {
good_host = "mail";
bad_host = "static";
symbol_strict = "ONCE_RECEIVED_STRICT";
symbol = "ONCE_RECEIVED";
}

Another issue is that if this module cannot resolve host, then it always
insert 'symbol_strict'. I could change that behaviour but I'm not sure
if it is a correct solution. Anyway, it turns out that once_received
module needs some refactoring and additional attention.

--
Vsevolod Stakhov

[Fabian Stanke]

Hi Vsevolod,

Thank you for your reply!

Am 12.02.2016 um 23:42 schrieb Vsevolod Stakhov <vsev...@highsecure.ru>:

I'd recommend to reconfigure this module completely in
rspamd.conf.override, for example:

once_received {
   good_host = "mail";
   bad_host = "static";
   symbol_strict = "ONCE_RECEIVED_STRICT";
   symbol = "ONCE_RECEIVED";
}

Could you please explain, which part of the header is matched against „good_host“ and „bad_host“? My example are messages from log check that have a Received header like this:

Received: from logcheck by stanke.wien with local (Exim 4.86) (envelope-from <logc...@stanke.wien>) id 1aUPX9-0001CI-Nn for logc...@stanke.wien; Sat, 13 Feb 2016 03:02:03 +0100

Now what should I use as good_host? „logcheck“ or „local“ or „stanke.wien“? Or something else?

Also what is the point of „bad_host“, if every message having only one Received header is scored by default? Does this relate to the „STRICT“ score?

Another issue is that if this module cannot resolve host, then it always
insert 'symbol_strict'. I could change that behaviour but I'm not sure
if it is a correct solution. Anyway, it turns out that once_received
module needs some refactoring and additional attention.

Should I insert one or more of the above names in /etc/hosts so that they can be resolved? Which one(s)?

I think once this is answered, the documentation of the module is complete and comprehensive ;-)

Thank you for your effort in providing and supporting rspamd!

Cheers,

Fabian

[Vsevolod Stakhov]

On 13/02/2016 06:26, Fabian Stanke wrote:
> Hi Vsevolod,
> Thank you for your reply!
>
>> Am 12.02.2016 um 23:42 schrieb Vsevolod Stakhov

>> <vsev...@highsecure.ru <mailto:vsev...@highsecure.ru>>:

>>
>> I'd recommend to reconfigure this module completely in
>> rspamd.conf.override, for example:
>>
>> once_received {
>> good_host = "mail";
>> bad_host = "static";
>> symbol_strict = "ONCE_RECEIVED_STRICT";
>> symbol = "ONCE_RECEIVED";
>> }
>
> Could you please explain, which part of the header is matched against
> „good_host“ and „bad_host“? My example are messages from log check that
> have a Received header like this:
>
> Received: from logcheck by stanke.wien with local (Exim 4.86)

> (envelope-from <logc...@stanke.wien <mailto:logc...@stanke.wien>>) id
> 1aUPX9-0001CI-Nn for logc...@stanke.wien <mailto:logc...@stanke.wien>;

> Sat, 13 Feb 2016 03:02:03 +0100
>
> Now what should I use as good_host? „logcheck“ or „local“ or
> „stanke.wien“? Or something else?
>
> Also what is the point of „bad_host“, if every message having only one
> Received header is scored by default? Does this relate to the „STRICT“
> score?
>
>>
>> Another issue is that if this module cannot resolve host, then it always
>> insert 'symbol_strict'. I could change that behaviour but I'm not sure
>> if it is a correct solution. Anyway, it turns out that once_received
>> module needs some refactoring and additional attention.
>>
>
> Should I insert one or more of the above names in /etc/hosts so that
> they can be resolved? Which one(s)?
>
> I think once this is answered, the documentation of the module is
> complete and comprehensive ;-)
> Thank you for your effort in providing and supporting rspamd!

Well, I've reworked this module so `good_host` patterns have always
priority over everything else. I've also added `whitelist` option to
allow specifying networks for which this module should be disabled.
Finally, I've added some documentation about this module:
https://rspamd.com/doc/modules/once_received.html

Experimental packages rebuild has been started.

--
Vsevolod Stakhov

[Fabian Stanke]

> Am 13.02.2016 um 15:49 schrieb Vsevolod Stakhov <vsev...@highsecure.ru>:
>
> Well, I've reworked this module so `good_host` patterns have always
> priority over everything else. I've also added `whitelist` option to
> allow specifying networks for which this module should be disabled.
> Finally, I've added some documentation about this module:
> https://rspamd.com/doc/modules/once_received.html
>
> Experimental packages rebuild has been started.

Wow, thanks for following this up that quickly! Whitelisting sound like the perfect means for dealing with local deliveries.

Presently I’m not on the experimental branch, because I’m using rspamd in „production“ for the family. However it seems that I can’t use just the new lua module, because new facilities for accessing the IP address have been introduced in the C/Lua interface.

Cheers, Fabian

[Vsevolod Stakhov]

Well, I've backported these changes to 1.1 stable branch as they seems
to be quite straightforward.

BTW, experimental packages are actually also used in many production
environments (and bugs are fixed much quicker in that branch). The only
warning is 'test before install'. Though, it applies for stable branch
as well...

--
Vsevolod Stakhov

[Fabian Stanke]

Am 13.02.2016 um 22:43 schrieb Vsevolod Stakhov <vsev...@highsecure.ru>:
> Well, I've backported these changes to 1.1 stable branch as they seems
> to be quite straightforward.

Cool, I have received the updated packages about a week ago.

There seems to be a problem parsing the „real_hostname“ value. Here is the log excerpt when one message is processed:

Mar 2 20:02:04 stanke rspamd[24676]: (normal) <3d7acd>; task; accept_socket: accepted connection from 127.0.0.1 port 56824
Mar 2 20:02:04 stanke rspamd[24676]: (normal) <3d7acd>; task; rspamd_message_parse: loaded message; id: <E1abC28-...@stanke.wien>; queue-id: <undef>
Mar 2 20:02:04 stanke rspamd[24676]: (normal) lua_radix_get_key: rspamd{ip} is not valid
Mar 2 20:02:04 stanke rspamd[24676]: (normal) <3d7acd>; task; lua_metric_symbol_callback: call to (ONCE_RECEIVED) failed: /usr/share/rspamd/lua/once_received.lua:82: bad argument #1 to 'lower' (string expected, got nil); trace: [1]:{[C]:-1 - lower [C]}; [2]:{/usr/share/rspamd/lua/once_received.lua:82 - <unknown> [Lua]};
Mar 2 20:02:04 stanke rspamd[24676]: (normal) <3d7acd>; task; rspamd_task_write_log: id: <E1abC28-...@stanke.wien>, from: <logc...@stanke.wien>, (default: F (no action): [-3.09/15.00] [BAYES_HAM,MIME_GOOD]), len: 895, time: 213.790ms real, 3.174ms virtual, dns req: 5

The full message that has been processed above is this:
_____

Return-path: <logc...@stanke.wien>
Envelope-to: logc...@stanke.wien
Delivery-date: Wed, 02 Mar 2016 20:02:04 +0100

Received: from logcheck by stanke.wien with local (Exim 4.86)

(envelope-from <logc...@stanke.wien>)
id 1abC28-0006NE-1y
for logc...@stanke.wien; Wed, 02 Mar 2016 20:02:04 +0100
To: logc...@stanke.wien
Subject: stanke.wien 2016-03-02 20:02 +0100 System Events
Auto-Submitted: auto-generated
MIME-Version: 1.0 (mime-construct 1.11)
Message-Id: <E1abC28-...@stanke.wien>
From: logcheck system account <logc...@stanke.wien>
Date: Wed, 02 Mar 2016 20:02:04 +0100
X-Spam-Scanner: rspamc 1.1.4
X-Spam-Scan-Time: 0.214
X-Spam-Action: no action
X-Spam-Score: -3.09 / 15.00
X-Spam-Level:
X-Spam-Symbols: BAYES_HAM,MIME_GOOD

This email is sent by logcheck. If you no longer wish to receive
such mail, you can either deinstall the logcheck package or modify
its configuration file (/etc/logcheck/logcheck.conf).

System Events
=-=-=-=-=-=-=
Mar 2 19:02:03 stanke rspamd[24676]: (normal) lua_radix_get_key: rspamd{ip} is not valid
_____

As usual, if there is any additional information that I can contribute to help follow this up, I’ll do my best.

Regards,
Fabian