greylisting: actions and thresholds

[Felix Schwarz]

Hi,

I'm trying to understand how rspamd's greylisting is expected to work in
general. I think I understood most of the mechanics now but I'd like to
validate that my understanding is correct.

- rspamd can used integrated greylisting (with redis) or external greylisting
(if no redis is configured, returns action "greylist").
However without redis rspamd does not maintain any history so another
software must take care of whitelisting the server in case of repeated
delivery.

- If redis greylisting is enabled, rspamd will always return one of three
actions when a mail is received the first time: "no action", "reject" and
"soft reject".
"soft reject" is most likely the greylisting (could also be some other
internal rspamd error).

- On subsequent deliveries (after "timeout") rspamd will not return soft
reject but also other error codes like "add_header".

Is that mostly correct?


Also I noticed that rspamd always saves the greylisting token as triplet
"from:to:ip". Is that really necessary? I guess if the remote site is a real
mailserver it will always retry regardless of which sender/recipient we are
talking about...

thanks,
Felix

[Vsevolod Stakhov]

On 02.12.2017 16:54, Felix Schwarz wrote:
> Hi,
>
> I'm trying to understand how rspamd's greylisting is expected to work in
> general. I think I understood most of the mechanics now but I'd like to
> validate that my understanding is correct.
>
> - rspamd can used integrated greylisting (with redis) or external greylisting
> (if no redis is configured, returns action "greylist").
> However without redis rspamd does not maintain any history so another
> software must take care of whitelisting the server in case of repeated
> delivery.

No, it cannot work with external greylisting anymore.

> - If redis greylisting is enabled, rspamd will always return one of three
> actions when a mail is received the first time: "no action", "reject" and
> "soft reject".
> "soft reject" is most likely the greylisting (could also be some other
> internal rspamd error).

Yes, `soft reject` is basically `try again later` with some potentially
customized MTA message.

> - On subsequent deliveries (after "timeout") rspamd will not return soft
> reject but also other error codes like "add_header".

Yes (but not reject btw).

> Is that mostly correct?
>
>
> Also I noticed that rspamd always saves the greylisting token as triplet
> "from:to:ip". Is that really necessary? I guess if the remote site is a real
> mailserver it will always retry regardless of which sender/recipient we are
> talking about...

No, that's the whole point of greylisting: we delay all email we unsure
about to let other sources of intelligence (e.g. DNS or URL black lists)
to react on that particular spam spike. Even trusted senders can send
spam outbreaks...

Rspamd also saves data tokens for greylisting.

[Felix Schwarz]

Am 02.12.2017 um 18:35 schrieb Vsevolod Stakhov:
> No, it cannot work with external greylisting anymore.

Ok - maybe it would be less confusing for users if rspamd would never return
the "greylist" action anymore? That was the part which surprised me most when
I enabled the greylisting module (that rspamd now returned "soft reject"
instead of "greylist" as I expected).

>> - On subsequent deliveries (after "timeout") rspamd will not return soft
>> reject but also other error codes like "add_header".
>
> Yes (but not reject btw).

So you are saying rspamd will never use the "reject" action for retried
deliveries? (Probably I misunderstood what you tried to say.)

>> Also I noticed that rspamd always saves the greylisting token as triplet
>> "from:to:ip". Is that really necessary? I guess if the remote site is a real
>> mailserver it will always retry regardless of which sender/recipient we are
>> talking about...
>
> No, that's the whole point of greylisting: we delay all email we unsure
> about to let other sources of intelligence (e.g. DNS or URL black lists)
> to react on that particular spam spike. Even trusted senders can send
> spam outbreaks...

Ah, I just noticed one difference of rspamd's greylisting to other common
greylisting implementations: In the early days of greylisting the idea was to
get rid of dial up spam bots which did not bother implementing a real/queuing
SMTP server. That means hosts/IPs were whitelisted after initial greylisting.

If I understand rspamd's greylisting correctly it is purely message-based and
the idea is mostly to get a few more minutes just in case a real-time
blacklist catches similar messages and raises alert levels.
Getting rid of dial-up bots without proper SMTP servers is only a side effect
here (and from my observation these low-quality bots make so many mistakes
that we are rejecting their messages anyway).
If greylisting is message-based and not host-based then you need a triplet
check of course.

> Rspamd also saves data tokens for greylisting.

That is actually an interesting point: How strict is rspamd about
retransmissions? Say for example a sender has multiple mxout IPs and they get
greylisted by rspamd. After 10 minutes the second system retries the delivery
and the IP is from a completely different subnet. However sender/recipient and
body hash are the same.
Will rspamd accept the message in that case or must all conditions match?



I have two more questions about the greylisting module:

The docs mention:
> ipv4_mask: mask to apply for IPv4 addresses (19 by default)
> ipv6_mask: mask to apply for IPv6 addresses (64 by default)

So it means you are pretty leniant about IPv4 retried deliveries but very
restrictive for IPv6 deliveries. Any particular reason for this?
(I guess currently it doesn't matter as very few providers use IPv6 for their
mailflow.)

> whitelist_domains_url: map of hostnames and/or eSLDs of hostnames to skip
> greylisting for

So this refers to the hostname of the sender, right?
What confused me slightly is the "domains" part in the option name. (e.g. it
might also refer to the sender's domain as present in the SMTP envelope)


Thank you very much for your help, much appreciated.
Felix