R_BAD_CTE_7BIT when using Exim/spamd (latest 1.2.x branch)

[Felix Schwarz]

Hi,

I'm using rspamd with Exim (at SMTP time) and yesterday I upgraded to the
latest git (1211bc8) from the 1.2.x branch. (otherwise my setup is still the
same as described in https://groups.google.com/forum/#!topic/rspamd/JwKgXEQSNpw ).

Now I noticed that rspamd often (but not always) detects R_BAD_CTE_7BIT even
though the text body seems to be perfectly valid.

Just for my understanding:
- R_BAD_CTE_7BIT only looks at the actual mail "body", right?
- If there is no "Content-Type:" header but the actual message is just
trivial (ASCII) text, should that trigger R_BAD_CTE_7BIT?

Any other pointers where I should look? I'd like to avoid digging through the
whole code :-)
Felix

[Felix Schwarz]

Am 24.05.2016 um 09:37 schrieb Felix Schwarz:
> Now I noticed that rspamd often (but not always) detects R_BAD_CTE_7BIT even
> though the text body seems to be perfectly valid.

I forgot to mention that "rspamc" doesn't return the symbol when running it on
the plain message (stored by Exim+dovecot, I don't think either one of them
adds invalid bytes).

[Felix Schwarz]

one more thing: I can reproduce the problem now with an "empty" Thunderbird
text message.

[Vsevolod Stakhov]

That should be fixed in master and rspamd-1.2 branches. Thank you for
the report.

--
Vsevolod Stakhov

[Felix Schwarz]

Am 24.05.2016 um 14:03 schrieb Vsevolod Stakhov:
> That should be fixed in master and rspamd-1.2 branches.

I upgraded my test machine to git version 61da7f7c but unfortunately rspamd
still reports R_BAD_CTE_7BIT.

Interestingly this doesn't always happen. I sent an empty message via two
different providers to my machine and one of them got marked R_BAD_CTE_7BIT
(and BAYES_SPAM btw) while it worked just fine for the other (which also got
BAYES_HAM)...

Both messages were submitted with Thunderbird so the headers are very much the
same. The only difference I'm seeing is that the "bad" messages has a DKIM
signature while the "good" message got two anti-spam headers ("X-Provags-ID",
"X-UI-Out-Filterresults").

Felix

[Felix Schwarz]

Am 24.05.2016 um 15:09 schrieb Felix Schwarz:
> Both messages were submitted with Thunderbird so the headers are very much the
> same. The only difference I'm seeing is that the "bad" messages has a DKIM
> signature while the "good" message got two anti-spam headers ("X-Provags-ID",
> "X-UI-Out-Filterresults").

ah, I just observed something *very* interesting:
- server to tries to deliver message to my rspamd host
- R_BAD_CTE_7BIT detected
- greylisting
- I force a new delivery from the source server (so the mail was not altered
in any way, just queued by exim)
- IS NOT DETECTED
(I disabled DKIM signing for this test so I think we can rule out the DKIM
header.)

However I noticed one thing in the logs:
The first delivery was using ipv6 (all my hosts run with the usual dual stack
setup). The second delivery used ipv4.

Also that matches my observation with the other provider I mentioned in my
previous mail - they only use ipv4. Could ipv6 be a problem here?

Felix

[Felix Schwarz]

Am 24.05.2016 um 15:17 schrieb Felix Schwarz:
> - I force a new delivery from the source server (so the mail was not altered
> in any way, just queued by exim)
> - IS NOT DETECTED

I guess it's obvious that I messed up the line above. I wanted to mention that
the symbol R_BAD_CTE_7BIT was not emitted by rspamd for the second delivery.

Felix

[Vsevolod Stakhov]

I don't understand your issue at all. Do you have reproducible samples?

--
Vsevolod Stakhov

[Felix Schwarz]

After some debugging I believe this is a bug in rspamd related to the recently
added Exim workaround.

I built a minimal reproducer script and filed a bug:
https://github.com/vstakhov/rspamd/issues/637

Thank you very much,
Felix

[Felix Schwarz]

Am 24.05.2016 um 09:37 schrieb Felix Schwarz:

> Any other pointers where I should look? I'd like to avoid digging through the
> whole code :-)

maybe this is helpful to others searching with Google:
In the default configuration all messages are delivered to rspamd via plain
TCP. This means you can inspect the data rspamd sees using plain
tcpdump+wireshark.

Based on that I build a minimal Python test case which contained the test
message. This script is available via
https://github.com/vstakhov/rspamd/issues/637 so feel free to use it as you like.

fs