Bug#1006283: milter fails on utf-8 text

[Matus UHLAR - fantomas]

Package: pyspf-milter
Version: 2.9.2-1

Hello,

it seems that pyspf-milter fails if the incoming message contains unicode
content.

Looks like it fails even in case when it could continue successfully:

Feb 17 22:25:57 fantomas sm-mta[13913]: STARTTLS=server, relay=lists.sourceforge.net [216.105.38.7], version=TLSv1.2, verify=NOT, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256/256
Feb 17 22:25:57 fantomas pyspf-milter[1573]: prepend Authentication-Results: fantomas.fantomas.sk; spf=pass (sender SPF authorized) smtp.mailfrom=lists.sourceforge.net (client-ip=216.105.38.7; helo=lists.sourceforge.net; envelope-from=courier-us...@lists.sourceforge.net; receiver=<UNKNOWN>)
Feb 17 22:25:57 fantomas sm-mta[13913]: 21HLPo98013913: from=<courier-us...@lists.sourceforge.net>, size=6029, class=-30, nrcpts=2, msgid=<5946cd31-34c2-b6f7...@dmj.nu>, bodytype=8BITMIME, proto=ESMTPS, daemon=MTA-v4, relay=lists.sourceforge.net [216.105.38.7]
Feb 17 22:25:57 fantomas pyspf-milter[1573]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 542: invalid continuation byte
Feb 17 22:25:57 fantomas pyspf-milter[1573]: pyspf-filter: milter claimed not to reply in state 7 but did anyway 4
Feb 17 22:26:57 fantomas sm-mta[13913]: 21HLPo98013913: Milter (pyspf-milter): timeout before data read, where=eom

Jan 9 06:59:33 fantomas sm-mta[5692]: STARTTLS=server, relay=lists.sourceforge.net [216.105.38.7], version=TLSv1.2, verify=NOT, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256/256
Jan 9 06:59:33 fantomas pyspf-milter[3257]: prepend Authentication-Results: fantomas.fantomas.sk; spf=pass (sender SPF authorized) smtp.mailfrom=lists.sourceforge.net (client-ip=216.105.38.7; helo=lists.sourceforge.net; envelope-from=courier-us...@lists.sourceforge.net; receiver=<UNKNOWN>)
Jan 9 06:59:33 fantomas sm-mta[5692]: 2095xQbw005692: from=<courier-us...@lists.sourceforge.net>, size=6862, class=-30, nrcpts=2, msgid=<2982fa42cb1d0a63fd9f170...@mariochiari.net>, bodytype=8BITMIME, proto=ESMTPS, daemon=MTA-v4, relay=lists.sourceforge.net [216.105.38.7]
Jan 9 06:59:33 fantomas pyspf-milter[3257]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc1 in position 370: invalid start byte
Jan 9 06:59:33 fantomas pyspf-milter[3257]: pyspf-filter: milter claimed not to reply in state 7 but did anyway 4
Jan 9 07:00:33 fantomas sm-mta[5692]: 2095xQbw005692: Milter (pyspf-milter): timeout before data read, where=eom

I have set mailserver to accept mail for these cases and I may provide
example of data if needed

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
My mind is like a steel trap - rusty and illegal in 37 states.

[Matus UHLAR - fantomas]

I just found example of such message in amavis' quarantine:

I'm copy&pasting from quarantine, the Tom\x{E1}\x{9A} is encoded by amavis
and "Tom<E1><9A>" as shown by less, both are supposed to be encoded in windows-1250

converted to utf8: Tomáš


Return-Path: <steveba...@gmail.com>
Delivered-To: spam-quarantine
X-Envelope-From: <steveba...@gmail.com>
X-Envelope-To: <tomas.r...@example.com>
X-Envelope-To-Blocked: <tomas.r...@example.com>
X-Quarantine-ID: <8oKuq2xdQFz5>
X-Amavis-Alert: BAD HEADER SECTION, Non-encoded non-ASCII data (and not UTF-8)
(char E1 hex): To: tomas.r...@example.com Tom\x{E1}\x{9A} Redacted
X-Spam-Flag: YES
X-Spam-Score: 10.989
X-Spam-Level: **********
X-Spam-Status: Yes, score=10.989 tag=x tag2=6.31 kill=10 tests=[BAYES_50=0.8,
DCC_CHECK=1.1, DKIM_ADSP_CUSTOM_MED=0.001, FORGED_GMAIL_RCVD=1,
FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001,
FREEMAIL_REPLYTO=1, FREEMAIL_REPLYTO_END_DIGIT=0.25,
GB_FREEMAIL_DISPTO=0.498, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.1,
NML_ADSP_CUSTOM_MED=0.9, NO_ENV_AND_HDR_FROM_MATCH=0.001,
SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665, SPOOFED_FREEMAIL=1.469,
SPOOFED_FREEM_REPTO=1.964, SPOOF_GMAIL_MID=0.988,
T_GB_FREEM_FROM_NOT_REPLY=0.01, T_SCC_BODY_TEXT_LINE=-0.01]
autolearn=no autolearn_force=no
Received: from unknown by localhost (amavisd-new, unix socket) id 8oKuq2xdQFz5
for <tomas.r...@example.com>; Fri, 18 Mar 2022 04:32:27 +0100 (CET)
Received: from hp0.27.ebvinxm.sbs (hp0.27.ebvinxm.sbs [164.92.89.141])
by example.com (amavisd-milter);
Fri, 18 Mar 2022 04:27:26 +0100 (CET)
(envelope-from <steveba...@gmail.com>)
Authentication-Results: example.com; dmarc=fail (p=none dis=none) header.from=gmail.com
Reply-To: steveba...@gmail.com
From: Steve Barnes <steveba...@gmail.com>
To: tomas.r...@example.com Tom<E1><9A> Redacted

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.

Linux - It's now safe to turn on your computer.
Linux - Teraz mozete pocitac bez obav zapnut.