Outlook 2007 generates illegal Message-ID

[Cedders]

Outlook 2007 will sometimes create a Message-ID header that is illegal
under RFC 2822, and in a format that has previously been mostly
associated with spam. I noticed this when Outlook users with email
addresses in the form xxx...@example.org complained their outgoing
messages were being marked as spam, and the bug only seems to trigger
when the local part of the From address contains a dot. We then see
things like:

From: "Mary Jones" <mary....@example.org>
To: som...@example.net
Date: Mon, 27 Jul 2009 10:17:06 +0100
Message-ID: <001601ca0e9b$e4b53100$ae1f9300$@jo...@example.org>
MIME-Version: 1.0

RFC 2822 specifies:
msg-id = [CFWS] "<" id-left "@" id-right ">" [CFWS]
id-left = dot-atom-text / no-fold-quote / obs-id-left
id-right = dot-atom-text / no-fold-literal / obs-id-
right
and "@" is not a permitted character in an atom. It appears Outlook
2007 is misidentifying the domain part from the From address, which
Outlook version 11 never did.

There is discussion about a workaround planned for SpamAssassin at
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5707
but this pattern may also be a spam cue in MS's own junk mail filters.

[neo [mvp outlook]]

If you can supply additional details like what mail server you are using on
what platform and such, I can check to see if the same thing happens under
the beta Outlook 2010. However keep in mind that RFC2822 applies to server
to server communications and not server to MUA (e-mail clients), therefore
there is no guarantees that Microsoft will address in Outlook 2007/2010.

"Cedders" <cedric...@gmail.com> wrote in message
news:45e57752-d10a-4ba0...@k26g2000vbp.googlegroups.com...

[Cedders]

On Aug 9, 3:16 pm, "neo [mvp outlook]"

<n...@discussions.microsoft.com> wrote:
> If you can supply additional details like what mail server you are using on
> what platform and such, I can check to see if the same thing happens under
> the beta Outlook 2010.  

Thanks - anything you can do to draw the attention of the developers
to this would be useful. Would it be worth resubmitting this as
'Suggestion for Microsoft' if I can through the Community Discussion
Groups portal (seems to not accept messages at the moment)? The
problem has been pointed out a few times before, e.g.:

http://www.pcreview.co.uk/forums/thread-3454981.php
http://techrepublic.com.com/5208-6230-0.html?forumID=101&threadID=236063&start=0

I don't think it would be that hard to make and distribute a patch,
but although the issue has apparently been around since Outlook 2007
was released, the Office developers have probably never received the
bug report (since it's so hard to work out where to send them!).

The MTA (mail server) I use is Postfix 2.3.8, but I really don't think
that's relevant. I've just tested via the SMTP servers of Yahoo and
O2 (my ISP) and the illegal Message-ID header is passed through there
just the same, as described in the links above. It is an issue with
the MUA, which creates a unique ID based on its hostname or sender
domain, and nothing to do with the MTA. The MUA I used in this case is
Outlook 2007 (12.0.6504.5000) SP2 MSO (12.0.6425.1000); I ran
Microsoft Update first just in case it had been recently patched. On
the other hand, Gmail's outgoing SMTP does completely rewrite the
message-ID (and of course it is valid), but the vast majority of smart
hosts I am sure will not: it would seem to make sense to keep the same
Message-ID from recipient to sender for diagnostic purposes and
identification.

So the steps to reproduce the problem are: (1) make sure you are using
a sender address of the form firstname...@domain.tld; (2) use
any standard outgoing mail server (not Gmail) in the account settings;
(3) examine the headers of the received message for the message-ID;
(4) if it contains multiple @ signs, the problem exists.

> However keep in mind that RFC2822 applies to server
> to server communications and not server to MUA (e-mail clients), therefore

Do you have a reference for that? SMTP is the standard way of sending
from a MUA *to* a server, as well as server-to-server.

Or are you suggesting that ISPs should rewrite the illegal Message-IDs
in user-submitted outgoing messages? It's not such a bad workaround
for our users, although a bit tricky, and how many ISPs are going to
be aware of the problem? IMHO it would definitely be best for the
issue to be fixed at source.

[neo [mvp outlook]]

RFC 2476

8.3. Add 'Message-ID'

The MSA MAY add or replace the 'Message-ID' field, if it lacks it, or
it is not valid syntax (as defined by [MESSAGE-FORMAT]).

If I'm not mistaken, the Message ID field became an issue under Outlook 2003
as Microsoft decided not to have one generated. This caused a large outcry
as well as it caused many issues for antispam solutions as well.

PS - I have posted on this in the past, so it should turn up in search
engines.

"Cedders" <cedric...@gmail.com> wrote in message

news:37c2037e-955b-4928...@l31g2000vbp.googlegroups.com...

[Cedders]

On 10 Aug, 12:50, "neo [mvp outlook]" <n...@discussions.microsoft.com>
wrote:

> RFC 2476
>
> 8.3.  Add 'Message-ID'
>
>    The MSA MAY add or replace the 'Message-ID' field, if it lacks it, or
>    it is not valid syntax (as defined by [MESSAGE-FORMAT]).

Sure. I wasn't suggesting Gmail was doing anything wrong, nor on the
other hand, O2 or Yahoo for _not_ rewriting. And most MTAs (or
message submission) agents in my experience will add a Message-Id if
it is missing, but it looks like most don't test its validity. RFC
2821 and 2822 still apply, and 2822 defines a valid Message-ID.

And I could set up header rewriting, although I've suggested to the
users that they just use aliases of firs...@example.org. One of
them seems to be using Apple Mail now anyway (which correctly parses
the domain from the From line). But this doesn't help operators of
MSAs who aren't aware of the Outlook bug.

>
> If I'm not mistaken, the Message ID field became an issue under Outlook 2003
> as Microsoft decided not to have one generated.  This caused a large outcry
> as well as it caused many issues for antispam solutions as well.

Yes, there's an account of this at http://www.slipstick.com/emo/2003/up031211.htm.
It looks like the history was that until an SP of Outlook 2003, it
used code based on the machine name, similar to the way Windows Mail
still operates, and then users complained about 'leaking' of the
machine name in the Message-ID (which is daft, because Outlook uses
the machine name for the HELO too). The response was to remove the
Message-ID, merely breaking a SHOULD in RFC 2822. The thing was
recoded for Outlook 2007 to fix that, and works for the most part
except in the firstname.lastname example which is wrongly parsed by
Outlook, and then it's breaking a MUST NOT in 3.1.

>
> PS - I have posted on this in the past, so it should turn up in search
> engines.

Yes, I mentioned a report here in microsoft.public.outlook.general in
the link above, back in 2007, and then there was another one in
m.p.o.i in 2008:
http://groups.google.co.uk/group/microsoft.public.outlook.installation/browse_thread/thread/7f86acd3d8d382fb/889b76444ef10147

So given that it's been posted here for over two years, the questions
is how do we ensure that Microsoft fixes it, or at least is aware of
it?

Did you get anywhere with Outlook 2010 beta?

All best wishes

C

[Cedders]

"Cedders" wrote:
> Yes, I mentioned a report here in microsoft.public.outlook.general in
> the link above, back in 2007, and then there was another one in
> m.p.o.i in 2008:
> http://groups.google.co.uk/group/microsoft.public.outlook.installation/browse_thread/thread/7f86acd3d8d382fb/889b76444ef10147
>
> So given that it's been posted here for over two years, the questions
> is how do we ensure that Microsoft fixes it, or at least is aware of
> it?

Anyone? There have been two service packs since it was first reported, and
the fix is probably a one-liner. Does anyone from Microsoft read these
newsgroups?

[F.H. Muffman]

Well, you could call technical support to report a bug. I wouldn't trust
MS to read the newsgroups, these are primarily a user to user support venue.

--
f.h.
Microsoft Outlook MVP

[neo [mvp outlook]]

Generally no as Brian pointed out that newsgroups are peer-to-peer
activities. I did pass this thread on to someone on the Outlook team in
Microsoft. As I mentioned before, there is no way for any of the MVPs to
promise anything because we are just like you that we don't work for
Microsoft, but try to help out someone that has a question/issue the best we
can.

"Cedders" <Ced...@discussions.microsoft.com> wrote in message
news:A7539A8A-C34F-4EE1...@microsoft.com...

[Cedders]

"neo [mvp outlook]" wrote:

> Generally no as Brian pointed out that newsgroups are peer-to-peer
> activities. I did pass this thread on to someone on the Outlook team in
> Microsoft.

Many thanks. All I can do now is hope they've added it to their internal
issue tracker. I also posted to the Outlook blog, but don't know if it will
ever be seen in the moderation queue. The next step was going to be writing
a letter... I had passed on the MS support number to the user about other
false positive issues (on _incoming_ email), but I think they were worried it
would get too technical, and as it doesn't affect me directly, I'm not on
site with the user and read stories of MS charging up front to report a bug,
it's a bit off-putting. I've had a decent response from MS on security
issues, but there's nothing similar for general bugs.

I think it's going to be a while before most anti-spam software and embedded
firewalls get a workaround because it's an MUA issue.

C