Google "multiple destination domains" issue

[Vincent Fox]

Hi,

It's been 9 years since Google did whatever backend change causes this:
451-4.3.0 Multiple destination domains per transaction is unsupported.
Please
451 4.3.0 try again. fa9si788651wic.42 - gsmtp

MTA like sendmail efficiently look up the MX for the recipient list and batch them up, but Google doesn't allow it, leading to a logjam. I have a mailing list server and also a bulkmail server on which I both notice stuff queueing up for retries quite often.

Is there any workaround for this in Sendmail yet?

It has.become a VEXING problem as more and more people MX Google for their domains. Exim I could set "multi_domains = false" to workaround it. I am however using LDAP mail-routing and certificate authentication, I have to figure out how to do those in Exim. I'd rather not have to toss out decades of Sendmail experience over one feature.

Thanks!

[b...@ripco.com]

Vincent Fox <vince...@gmail.com> wrote:

> MTA like sendmail efficiently look up the MX for the recipient list and
> batch them up, but Google doesn't allow it, leading to a logjam. I have a
> mailing list server and also a bulkmail server on which I both notice
> stuff queueing up for retries quite often.
>
> Is there any workaround for this in Sendmail yet?

Not sure about this, Claus will probably know off the top of his head, but
we ran into a similar problem and there was sort of a fix.

Check out the doc/op/op.me and specifically section 3 "Queue Groups and
Queue Directories".

From memory there was a way to change delivery based on MX, size, priority
and other things. Again not sure but I think you can group the queue by
domain, so that anything to bi...@dom1.com and sa...@dom1.com would be
delivered as a group even if fr...@dom2.com and be...@dom2.com are also on
a gmail MX. Point is, email to dom1.com and dom2.com are not mixed together
on one delivery even though they end up at the same place.

This I think was with yahoo or hotmail (google wasn't around yet) but keep
in mind they sort of count IP addresses these days. No matter who you are
emailing, if they see the same IP coming in over a period of time, they'll
still block you.

It's a different error message, something to do with detection of an
abnormal rate so you might be trading off one problem for another.

-bruce
b...@ripco.com

[Grant Taylor]

On 6/10/21 8:58 AM, Vincent Fox wrote:
> Hi,

Hi,

> It's been 9 years since Google did whatever backend change causes this:
> 451-4.3.0 Multiple destination domains per transaction is unsupported.
> Please
> 451 4.3.0 try again. fa9si788651wic.42 - gsmtp

I don't recall running into that. Maybe I've been lucky.

> MTA like sendmail efficiently look up the MX for the recipient list
> and batch them up, but Google doesn't allow it, leading to a logjam.
> I have a mailing list server and also a bulkmail server on which I
> both notice stuff queueing up for retries quite often.
>
> Is there any workaround for this in Sendmail yet?

I don't know if there is a setting to prevent such batching or not. I
feel like there probably is.

The other thing that you might consider - which I use - that might help
is to try "Interactive" delivery instead of "Queued" delivery. The idea
being that Sendmail will try to deliver the individual message when it's
quite likely only the single recipient domain at original inbound SMTP
time as opposed to queuing and getting a batch that may be multi-domain.



--
Grant. . . .
unix || die

[Claus Aßmann]

Vincent Fox wrote:

> 451-4.3.0 Multiple destination domains per transaction is unsupported.

It's fantastic that these big companies can make up whatever
rules they like...

> Is there any workaround for this in Sendmail yet?

Maybe turn off the m flag for the mailer?

m This mailer can send to multiple users on the same
host in one transaction.

There are other options, but this should be the easiest.

--
Note: please read the netiquette before posting. I will almost never
reply to top-postings which include a full copy of the previous
article(s) at the end because it's annoying, shows that the poster
is too lazy to trim his article, and it's wasting the time of all readers.

[Vincent Fox]

On Thursday, June 10, 2021 at 10:50:01 AM UTC-7, Claus Aßmann wrote:
> Vincent Fox wrote:
>
> > 451-4.3.0 Multiple destination domains per transaction is unsupported.
> It's fantastic that these big companies can make up whatever
> rules they like...

I agree, but they are a 900 lb gorilla and will not change. Most infuriating
is how Google doesn't even respond to questions about why they do it.
Non-Google apologists claim it is "anti-spam" which makes no sense at all.

But evidently there is a way to work around this which works in Exim.
I was hoping something similar had come along for Sendmail.

> > Is there any workaround for this in Sendmail yet?
> Maybe turn off the m flag for the mailer?
>
> m This mailer can send to multiple users on the same
> host in one transaction.
>
> There are other options, but this should be the easiest.

Is this the same as max_rcpts 1?

Simple case I'm submitting an email:
67 *@gmail.com
1 *@foo.com. (MX is google)
12 *@bar.edu. ( ditto)
3 *@baz.org. (ditto)

Google accepts 67 recipients and tempfail the others. So they go for another
few spins in the queue. The 4th spin finishes off the last recipents after
the set retry interval, which I have shortened but it's still noticeable.

If I move to splitting per.single recipient.... well I can do that but it's
more to the inefficient end of things than I'd like.

Unfortunately my recipient list is too varied and frankly there are
too many domains now MX thru google, to use queue groups. I'd be
chasing my own tail forever manually managing queue groups.

Thanks!

[Claus Aßmann]

Vincent Fox wrote:

> I was hoping something similar had come along for Sendmail.

Not yet - maybe you can write a patch? Unfortunately I don't have
much time to loook into this, hopefully someone does!

> > m This mailer can send to multiple users on the same
> > host in one transaction.

> Is this the same as max_rcpts 1?

Not sure which option you mean here.

If you mean this:

5.10. Q -- Queue Group Declaration

recipients
The maximum number of recipients per enve-
lope. Envelopes with more than this number
of recipients will be split into multiple
envelopes in the same queue directory. The
default value 0 means no limit.

then the answer is: "No" (don't use that).

> Simple case I'm submitting an email:
> 67 *@gmail.com
> 1 *@foo.com. (MX is google)
> 12 *@bar.edu. ( ditto)
> 3 *@baz.org. (ditto)

> If I move to splitting per.single recipient.... well I can do that but it's
> more to the inefficient end of things than I'd like.

Not really, it's all done in one session - of course it depends on
the size of the mail - so you don't have to wait for multiple queue
runs.

[Andrzej Adam Filip]

Vincent Fox <vince...@gmail.com> wrote:
> On Thursday, June 10, 2021 at 10:50:01 AM UTC-7, Claus Aßmann wrote:
>> Vincent Fox wrote:
>>
>> > 451-4.3.0 Multiple destination domains per transaction is unsupported.
>> It's fantastic that these big companies can make up whatever
>> rules they like...
>
> I agree, but they are a 900 lb gorilla and will not change. Most infuriating
> is how Google doesn't even respond to questions about why they do it.
> Non-Google apologists claim it is "anti-spam" which makes no sense at all.

> […]

IMHO it simplifies a lot "inbound SMTP proxy"
*without* "store now, forward later".

--
[Andrew] Andrzej A. Filip

[Kelsey Cummings]

On Thursday, June 10, 2021 at 11:45:01 AM UTC-7, Claus Aßmann wrote:
> Not yet - maybe you can write a patch? Unfortunately I don't have
> much time to loook into this, hopefully someone does!

Claus, pulling up an old thread here but would having a sponsor provide you the opportunity to implement a solution? This has finally reached my attention too and the ability to split envelopes by domain (something which Brandon Long thinks sendmail can do, but I can't figure out how) or perhaps being able to assign a queue group by wildcard matching on an MX record which would allow the cheesy fix of just setting a single recipient per transaction on a queue assigned to gmail's mx servers.

In any case, savvy users have started to notice and complain about delays into gmail hosted domains because of this.

[b...@ripco.com]

Kelsey Cummings <k...@corp.sonic.net> wrote:

> In any case, savvy users have started to notice and complain about delays
> into gmail hosted domains because of this.

I kind of lost the original thread but I think you are looking for this
solution.

In your sendmail.mc stick these in:

define(`SMTP_MAILER_MAXRCPTS', `1')dnl
define(`RELAY_MAILER_MAXRCPTS', `1')dnl

Those will make sendmail queue them out one at a time rather than bundle them
together and get that multi-domain isn't allowed error from gmail.

Been using that for a while and haven't seen a single rejection or error
dealing with them. Also works well for the idiots at protection.outlook.com
because they randomly set max nrcpts to 2.

-bruce
b...@ripco.com

[Kelsey Cummings]

On Wednesday, November 2, 2022 at 6:18:32 AM UTC-7, b...@ripco.com wrote:
> In your sendmail.mc stick these in:
>
> define(`SMTP_MAILER_MAXRCPTS', `1')dnl
> define(`RELAY_MAILER_MAXRCPTS', `1')dnl

I'm reluctant to break a fundamental optimization in email delivery to
work around google's behavior. Although perhaps on today's
infrastructure it wouldn't really make all that much of difference.

-K

[Claus Aßmann]

Kelsey Cummings wrote:

> attention too and the ability to split envelopes by domain (something which
> Brandon Long thinks sendmail can do, but I can't figure out how) or perhaps

Try the following:
Add this to your mc file in the proper places:

QUEUE_GROUP(`one', `Path=/var/spool/mqueue/one, r=1')
LOCAL_CONFIG
Kbestmx bestmx

LOCAL_RULESETS
Squeuegroup
R< $+ > $1
R $+ @ $+ $: $(bestmx $2 $)
R $+ $: $>SearchList <! qgrpmx> $| <D:$1> <>
R<?> $@
R<$+> $# $1

Add something like this to access_db:
qgrpmx:google.com. one

Rebuild the cf file and the map.
mkdir /var/spool/mqueue/one
with proper owner/group/permissions

Now domains which have a "best MX" pointing to google.com
should end up in queuegroup one which should have this
option:

recipients
The maximum number of recipients per envelope.

This might be a good start - but I haven't tested it!

It would be nice if someone gives it a try and actually
reports back (not like with the IPv4 patch where people
ask for it but don't give feedback...)


Note: an enhanced version to send all recipients with the
same domain to google instead of just 1 per transaction
probably requires some code changes -- which would take
much more time and effort.

[Kelsey Cummings]

On Thursday, November 3, 2022 at 12:01:44 AM UTC-7, Claus Aßmann wrote:
> Try the following:
> Add this to your mc file in the proper places:

...

Claus, this seems to work as expected provided you don't get stuck thinking that existing FEATURE(`queuegroup') configs are still going to work as well. Perhaps it's possible to do both the bestmx lookup and then override the result with a non-empty qgrp lookup. But, that's not an issue for our use case and arguably the other groups we have defined would benefit from being MX based and not domain based in the first place. With two qgrpmx's defined, I was able to split envelopes across both of them and the default mqueue with them all respecting the queues' recipient limit. I'd call this a success!