Unable to run qmail-remote
Ian Eiloart
I have a personal domain hosted on a virtual server which uses qmail in a
plesk environment.
I don't know much about qmail, but I do also manage an Exim mail server for
my University.
A couple of times a month, I have reason to send an email to about 50
people - agendas and minutes of a local group meeting. I choose to use my
personal domain (and therefore qmail for this).
Unfortunately, the virtual server resources are fairly tight, and about one
in four mailings I see that many of the intended recipients don't get my
message. For example, this last time 42 of the 53 recipients didn't get the
message. Usually, there isn't a problem, and the list of recipients doesn't
vary.
The bounce message says this (I've obscured the domain names, since they'r
e not relevant exce:
> Hi. This is the qmail-send program at svr.example.co.uk.
> I'm afraid I wasn't able to deliver your message to the following
> addresses. This is a permanent error; I've given up. Sorry it didn't work
> out.
>
> <t...@example.org>:
> Unable to run qmail-remote.
Then it lists all the other recipients, with the same error. All the
recipients before tim did get the message, and all the recipients after tim
(except me, but I'm the only local recipient) bounced.
Am I right in supposing that this is what's happening?:
1. qmail accepts my message and queues it without attempting any
deliveries. (Exim would attempt to make all the deliveries and only queue
messages with temporary failures).
2. Some other stuff happens here. Maybe multiple copies of the message get
queued, local mail gets separated from remote. Dunno, but somehow the
message (or copies) get into an outgoing mail queue.
3. Separate qmail-remote processes are launched by a queue runner for each
remote delivery address, even when they're in the same domain.
4. If the queue runner can't launch qmail-remote first time for a given (or
maybe it makes a few attempts), then it bounces the email.
My main concern is (4). If this is happening without retries, can I
configure qmail to hang on to the messages and retry?
--
Ian Eiloart
IT Services, University of Sussex
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
Maurice Lucas
> Hi,
>
> I have a personal domain hosted on a virtual server which uses qmail in a
> plesk environment.
First. Plesk 1mail is not qmail it is a patched version and we/nobody
knows what is changed.
> A couple of times a month, I have reason to send an email to about 50
> people - agendas and minutes of a local group meeting. I choose to use my
> personal domain (and therefore qmail for this).
>
> Unfortunately, the virtual server resources are fairly tight, and about one
> in four mailings I see that many of the intended recipients don't get my
> message. For example, this last time 42 of the 53 recipients didn't get the
> message. Usually, there isn't a problem, and the list of recipients doesn't
> vary.
>
> The bounce message says this (I've obscured the domain names, since they'r
> e not relevant exce:
>
>
> > Hi. This is the qmail-send program at svr.example.co.uk.
> > I'm afraid I wasn't able to deliver your message to the following
> > addresses. This is a permanent error; I've given up. Sorry it didn't work
> > out.
> >
> > <t...@example.org>:
> > Unable to run qmail-remote.
Somehow your qmail-remote process crashed.
could you post the relevant portion of /var/log/qmail/current
> Then it lists all the other recipients, with the same error. All the
> recipients before tim did get the message, and all the recipients after tim
> (except me, but I'm the only local recipient) bounced.
Some of the remote users gets the message?
> Am I right in supposing that this is what's happening?:
>
> 1. qmail accepts my message and queues it without attempting any
> deliveries. (Exim would attempt to make all the deliveries and only queue
> messages with temporary failures).
qmail always queue's a mail
> 2. Some other stuff happens here. Maybe multiple copies of the message get
> queued,
1 message in the local queue and 1 in the remote queue
> local mail gets separated from remote.
Correct
> Dunno, but somehow the
> message (or copies) get into an outgoing mail queue.
>
> 3. Separate qmail-remote processes are launched by a queue runner for each
> remote delivery address, even when they're in the same domain.
correct
> 4. If the queue runner can't launch qmail-remote first time for a given (or
> maybe it makes a few attempts), then it bounces the email.
>
> My main concern is (4). If this is happening without retries, can I
> configure qmail to hang on to the messages and retry?
I think your qmail-send/log process doesn't run so the pipe to the log
gets full.
With kind regards,
Maurice Lucas
Ian Eiloart
--On 5 August 2006 13:27:44 +0200 Maurice Lucas <msl...@taos-it.nl> wrote:
> On Sat, 2006-08-05 at 11:52 +0100, Ian Eiloart wrote:
>> Hi,
>>
>> I have a personal domain hosted on a virtual server which uses qmail in
>> a plesk environment.
>
> First. Plesk 1mail is not qmail it is a patched version and we/nobody
> knows what is changed.
Noted
>> A ...>> >
>> > <t...@example.org>:
>> > Unable to run qmail-remote.
>
> Somehow your qmail-remote process crashed.
>
> could you post the relevant portion of /var/log/qmail/current
Ah, well there's one difference. Plesk qmail logs are at
/usr/local/psa/var/log/maillog
These files are present:
maillog
maillog.processed
maillog.processed.1
maillog.processed.1.gz
maillog.processed.2.gz
maillog.processed.3.gz
smtp_pendings.log
xferlog
xferlog.processed
I've pasted an extract from maillog.processed below.
>
>> Then it lists all the other recipients, with the same error. All the
>> recipients before tim did get the message, and all the recipients after
>> tim (except me, but I'm the only local recipient) bounced.
>
> Some of the remote users gets the message?
Ah, well perhaps I should say that the addresses in the bounce message
aren't all the addresses in the recipient list. I'm assuming that the rest
of the recipients got the message. maillog.processed contains this entry:
> Aug 2 12:20:53 svr qmail: 1154517653.809002 starting delivery 990: msg
> 102383715 to remote ***********@cix.co.uk Aug 2 12:20:56 svr qmail:
> 1154517656.536518 delivery 990: success:
> 212.241.168.133_accepted_message./Remote_host_said:_250_2.0.0_k72BKulR013
> 100_Message_accepted_for_delivery/
where I've hidden the local part of the recipient address. This is the only
message my server handled for that recipient on that day.
Hey, don't we get unique message ids in the logs? How do I find log entries
for that message only?
>> Am I right in supposing that this is what's happening?:
...
Thanks for your responses here.
>
>> 4. If the queue runner can't launch qmail-remote first time for a given
>> (or maybe it makes a few attempts), then it bounces the email.
>>
>> My main concern is (4). If this is happening without retries, can I
>> configure qmail to hang on to the messages and retry?
>
> I think your qmail-send/log process doesn't run so the pipe to the log
> gets full.
Well, it does run. However, it seems that qmail is trying to run 50 odd
processes separately. Only the first few ran. My virtual server has pretty
limited reserved RAM, so I'm not too surprised at this. Given that I'm only
trying to deliver into about a dozen domains, I'm amazed that qmail wants a
process for each recipient. Is there reason that qmail wants separate smtp
sessions when it could just as well specify several recipients in one smtp
session?
> With kind regards,
> Maurice Lucas
Thanks, Maurice. That's very helpful of you.
PS. This is my first day on the qmail list, and it's pretty wild out there!
>
Aug 2 12:20:53 svr qmail: 1154517653.775667 new msg 102383715
Aug 2 12:20:53 svr qmail: 1154517653.775967 info msg 102383715: bytes 6265
from <i...@eiloart.com> qp 11401 uid 2020
Aug 2 12:20:53 svr qmail: 1154517653.808767 starting delivery 989: msg
102383715 to local 8-...@eiloart.com
Aug 2 12:20:53 svr qmail: 1154517653.808933 status: local 1/10 remote 0/20
Aug 2 12:20:53 svr qmail: 1154517653.809002 starting delivery 990: msg
102383715 to remote nor******@*****.***k
Aug 2 12:20:53 svr qmail: 1154517653.809073 status: local 1/10 remote 1/20
Aug 2 12:20:53 svr qmail: 1154517653.811005 starting delivery 991: msg
102383715 to remote r******@*****.***com
Aug 2 12:20:53 svr qmail: 1154517653.811199 status: local 1/10 remote 2/20
Aug 2 12:20:53 svr qmail: 1154517653.811269 starting delivery 992: msg
102383715 to remote FN******@*****.***uk
Aug 2 12:20:53 svr qmail: 1154517653.811330 status: local 1/10 remote 3/20
Aug 2 12:20:53 svr qmail: 1154517653.817226 starting delivery 993: msg
102383715 to remote grac******@*****.***.uk
Aug 2 12:20:53 svr qmail: 1154517653.817376 status: local 1/10 remote 4/20
Aug 2 12:20:53 svr qmail: 1154517653.821306 starting delivery 994: msg
102383715 to remote kar******@*****.***ne.com
Aug 2 12:20:53 svr qmail: 1154517653.821455 status: local 1/10 remote 5/20
Aug 2 12:20:53 svr qmail: 1154517653.831752 starting delivery 995: msg
102383715 to remote simo******@*****.***.uk
Aug 2 12:20:53 svr qmail: 1154517653.831911 status: local 1/10 remote 6/20
Aug 2 12:20:53 svr qmail: 1154517653.837143 starting delivery 996: msg
102383715 to remote james_******@*****.***il.co.uk
Aug 2 12:20:53 svr qmail: 1154517653.837331 status: local 1/10 remote 7/20
Aug 2 12:20:53 svr qmail: 1154517653.854277 starting delivery 997: msg
102383715 to remote car******@*****.***.uk
Aug 2 12:20:53 svr qmail: 1154517653.854454 status: local 1/10 remote 8/20
Aug 2 12:20:53 svr qmail: 1154517653.859162 starting delivery 998: msg
102383715 to remote mich******@*****.***ov.uk
Aug 2 12:20:53 svr qmail: 1154517653.859348 status: local 1/10 remote 9/20
Aug 2 12:20:53 svr qmail: 1154517653.867149 starting delivery 999: msg
102383715 to remote tim******@*****.***y.org
Aug 2 12:20:53 svr qmail: 1154517653.867308 status: local 1/10 remote 10/20
Aug 2 12:20:53 svr qmail: 1154517653.871146 starting delivery 1000: msg
102383715 to remote rod******@*****.***ov.uk
Aug 2 12:20:53 svr qmail: 1154517653.871329 status: local 1/10 remote 11/20
Aug 2 12:20:53 svr qmail: 1154517653.874614 starting delivery 1001: msg
102383715 to remote yvon******@*****.***uk
Aug 2 12:20:53 svr qmail: 1154517653.874758 status: local 1/10 remote 12/20
Aug 2 12:20:53 svr spamd[13455]: got connection over /tmp/spamd_full.sock
Aug 2 12:20:53 svr qmail: 1154517653.893177 starting delivery 1002: msg
102383715 to remote jam******@*****.***k
Aug 2 12:20:53 svr qmail: 1154517653.893365 status: local 1/10 remote 13/20
Aug 2 12:20:53 svr qmail: 1154517653.895476 starting delivery 1003: msg
102383715 to remote tina******@*****.***com
Aug 2 12:20:53 svr qmail: 1154517653.895650 status: local 1/10 remote 14/20
Aug 2 12:20:53 svr qmail: 1154517653.901733 starting delivery 1004: msg
102383715 to remote guy******@*****.***net
Aug 2 12:20:53 svr qmail: 1154517653.901923 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.901988 delivery 999: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.902419 status: local 1/10 remote 14/20
Aug 2 12:20:53 svr qmail: 1154517653.902586 starting delivery 1005: msg
102383715 to remote dore******@*****.***com
Aug 2 12:20:53 svr qmail: 1154517653.902646 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.906891 delivery 1000: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.908515 status: local 1/10 remote 14/20
Aug 2 12:20:53 svr qmail: 1154517653.910181 starting delivery 1006: msg
102383715 to remote ******@*****.***.com
Aug 2 12:20:53 svr qmail: 1154517653.911784 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.911957 starting delivery 1007: msg
102383715 to remote Pebb******@*****.***om
Aug 2 12:20:53 svr qmail: 1154517653.912018 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.912081 delivery 1001: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.912308 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.912389 starting delivery 1008: msg
102383715 to remote hur******@*****.***k
Aug 2 12:20:53 svr qmail: 1154517653.912449 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.915096 delivery 1002: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.915254 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.916217 starting delivery 1009: msg
102383715 to remote stem******@*****.***t
Aug 2 12:20:53 svr qmail: 1154517653.917143 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.925363 starting delivery 1010: msg
102383715 to remote der******@*****.***.uk
Aug 2 12:20:53 svr qmail: 1154517653.928176 status: local 1/10 remote 17/20
Aug 2 12:20:53 svr qmail: 1154517653.929160 delivery 1003: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.933366 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.933808 delivery 1004: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.933874 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.933942 starting delivery 1011: msg
102383715 to remote cll******@*****.***m
Aug 2 12:20:53 svr qmail: 1154517653.934005 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.934073 delivery 1005: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.936002 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.936077 starting delivery 1012: msg
102383715 to remote lew******@*****.***.com
Aug 2 12:20:53 svr qmail: 1154517653.936522 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.936585 delivery 1006: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.936647 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.936714 starting delivery 1013: msg
102383715 to remote merl******@*****.***com
Aug 2 12:20:53 svr qmail: 1154517653.936775 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.939760 delivery 1007: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr spamd[11417]: Using default config for i...@eiloart.com:
/var/qmail/mailnames/eiloart.com/ian/user_prefs
Aug 2 12:20:53 svr qmail: 1154517653.940973 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.941699 starting delivery 1014: msg
102383715 to remote tonypa******@*****.***com
Aug 2 12:20:53 svr qmail: 1154517653.943131 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.944360 delivery 1008: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.945098 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.949964 starting delivery 1015: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.950845 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.952289 delivery 1009: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.953038 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.953484 delivery 1010: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.957338 status: local 1/10 remote 14/20
Aug 2 12:20:53 svr qmail: 1154517653.962551 starting delivery 1016: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.964166 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.965738 starting delivery 1017: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.966863 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.967040 delivery 1011: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.967103 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.967426 starting delivery 1018: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.967487 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.968230 delivery 1012: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.968981 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.969446 delivery 1013: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.969644 status: local 1/10 remote 14/20
Aug 2 12:20:53 svr qmail: 1154517653.970584 starting delivery 1019: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.970830 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.975137 starting delivery 1020: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.976133 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.979898 delivery 1014: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.980084 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.980355 starting delivery 1021: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.980444 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.982171 delivery 1015: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.982442 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.982537 delivery 1016: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.982606 status: local 1/10 remote 14/20
Aug 2 12:20:53 svr qmail: 1154517653.982676 starting delivery 1022: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.982742 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.988396 starting delivery 1023: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.988533 status: local 1/10 remote 16/20
Aug 2 12:20:53 svr qmail: 1154517653.998231 delivery 1017: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:53 svr qmail: 1154517653.998401 status: local 1/10 remote 15/20
Aug 2 12:20:53 svr qmail: 1154517653.998465 starting delivery 1024: msg
102383715 to remote ******@*****.***
Aug 2 12:20:53 svr qmail: 1154517653.998532 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.000955 delivery 1018: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.002941 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.004013 starting delivery 1025: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.005846 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.011427 delivery 1019: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.011517 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.011580 starting delivery 1026: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.011639 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.011697 delivery 1020: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.011755 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.011815 delivery 1021: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.011872 status: local 1/10 remote 14/20
Aug 2 12:20:54 svr qmail: 1154517654.011934 starting delivery 1027: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.011992 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.020125 delivery 1022: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.020258 status: local 1/10 remote 14/20
Aug 2 12:20:54 svr qmail: 1154517654.020319 starting delivery 1028: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.024220 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.024309 delivery 1023: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.024367 status: local 1/10 remote 14/20
Aug 2 12:20:54 svr qmail: 1154517654.024428 starting delivery 1029: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.024487 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.029161 starting delivery 1030: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.029328 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.032007 delivery 1024: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.032624 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.033532 starting delivery 1031: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.033594 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.037209 delivery 1025: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.037282 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.037344 starting delivery 1032: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.037401 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr spamd[11417]: processing message
<52E560D92CF434B46594B91E@Vigor10> for i...@eiloart.com:110.
Aug 2 12:20:54 svr qmail: 1154517654.039566 delivery 1026: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.039675 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.039748 starting delivery 1033: msg
102383715 to remote ******@*****.***k
Aug 2 12:20:54 svr qmail: 1154517654.039814 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.044012 delivery 1028: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.044671 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.044733 starting delivery 1034: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.044789 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.046158 delivery 1027: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.046322 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.050539 starting delivery 1035: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.050671 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.050748 delivery 1029: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.052031 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.052206 delivery 1030: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.052310 status: local 1/10 remote 14/20
Aug 2 12:20:54 svr qmail: 1154517654.052375 starting delivery 1036: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.052436 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.056689 starting delivery 1037: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.056795 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.056861 delivery 1032: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.056922 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.057855 delivery 1031: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.057921 status: local 1/10 remote 14/20
Aug 2 12:20:54 svr qmail: 1154517654.057994 starting delivery 1038: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.058055 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.059568 starting delivery 1039: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.059723 status: local 1/10 remote 16/20
Aug 2 12:20:54 svr qmail: 1154517654.060718 delivery 1033: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.060820 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.060882 delivery 1034: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.061580 status: local 1/10 remote 14/20
Aug 2 12:20:54 svr qmail: 1154517654.061651 starting delivery 1040: msg
102383715 to remote ******@*****.***
Aug 2 12:20:54 svr qmail: 1154517654.061712 status: local 1/10 remote 15/20
Aug 2 12:20:54 svr qmail: 1154517654.062635 delivery 1035: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.062724 status: local 1/10 remote 14/20
Aug 2 12:20:54 svr qmail: 1154517654.064979 delivery 1036: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.066087 status: local 1/10 remote 13/20
Aug 2 12:20:54 svr qmail: 1154517654.066209 delivery 1037: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.066282 status: local 1/10 remote 12/20
Aug 2 12:20:54 svr qmail: 1154517654.066349 delivery 1038: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.066714 status: local 1/10 remote 11/20
Aug 2 12:20:54 svr qmail: 1154517654.066783 delivery 1039: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.066858 status: local 1/10 remote 10/20
Aug 2 12:20:54 svr qmail: 1154517654.067075 delivery 1040: failure:
Unable_to_run_qmail-remote./
Aug 2 12:20:54 svr qmail: 1154517654.067557 status: local 1/10 remote 9/20
Aug 2 12:20:54 svr qmail: 1154517654.190181 delivery 989: deferral:
/var/qmail/bin/qmail-local:_error_while_loading_shared_libraries:_libc.so.6:_failed_to_map_segment_from_shared_object:_Cannot_allocate_memory/rm:_error_while_loading_shared_libraries:_libc.so.6:_failed_to_map_segment_from_shared_object:_Cannot_allocate_memory/
Aug 2 12:20:54 svr qmail: 1154517654.190366 status: local 0/10 remote 9/20
Aug 2 12:20:54 svr qmail: 1154517654.269816 delivery 991: success:
217.146.188.189_accepted_message./Remote_host_said:_250_ok_dirdel/
Aug 2 12:20:54 svr qmail: 1154517654.269960 status: local 0/10 remote 8/20
Aug 2 12:20:54 svr qmail: 1154517654.583470 delivery 993: success:
193.252.22.141_accepted_message./Remote_host_said:_250_Ok:_queued_as_E3BAF140062C/
Aug 2 12:20:54 svr qmail: 1154517654.583975 status: local 0/10 remote 7/20
Aug 2 12:20:54 svr qmail: 1154517654.640855 delivery 997: deferral:
212.74.100.147_failed_after_I_sent_the_message./Remote_host_said:_451_transient_Brightmail_processing_error/
Aug 2 12:20:54 svr qmail: 1154517654.640957 status: local 0/10 remote 6/20
Aug 2 12:20:55 svr qmail: 1154517655.046589 delivery 998: success:
212.42.162.18_accepted_message./Remote_host_said:_250_2.0.0_k72BKi61035932_Message_accepted_for_delivery/
Aug 2 12:20:55 svr qmail: 1154517655.047274 status: local 0/10 remote 5/20
Aug 2 12:20:55 svr qmail: 1154517655.314022 delivery 996: success:
65.54.247.8_accepted_message./Remote_host_said:_250__<52E560D92CF434B46594B91E@Vigor10>_Queued_mail_for_delivery/
Aug 2 12:20:55 svr qmail: 1154517655.314359 status: local 0/10 remote 4/20
Aug 2 12:20:55 svr qmail: 1154517655.384495 delivery 995: success:
217.77.187.15_accepted_message./Remote_host_said:_250_2.6.0_6402_bytes_received_in_00:00:00;_Message_accepted_for_delivery/
Aug 2 12:20:55 svr qmail: 1154517655.384933 status: local 0/10 remote 3/20
Aug 2 12:20:56 svr qmail: 1154517656.536518 delivery 990: success:
212.241.168.133_accepted_message./Remote_host_said:_250_2.0.0_k72BKulR013100_Message_accepted_for_delivery/
Aug 2 12:20:56 svr qmail: 1154517656.537317 status: local 0/10 remote 2/20
Aug 2 12:20:57 svr qmail: 1154517657.039700 delivery 994: success:
212.188.138.195_accepted_message./Remote_host_said:_250_2.6.0__<52E560D92CF434B46594B91E@Vigor10>_Queued_mail_for_delivery/
Aug 2 12:20:57 svr qmail: 1154517657.040396 status: local 0/10 remote 1/20
Aug 2 12:20:59 svr qmail: 1154517659.816620 delivery 992: success:
212.42.162.23_accepted_message./Remote_host_said:_250_2.0.0_k72BKnMD032135_Message_accepted_for_delivery/
Aug 2 12:20:59 svr qmail: 1154517659.816968 status: local 0/10 remote 0/20
Aug 2 12:21:40 svr qmail: 1154517700.839142 starting delivery 1041: msg
102383765 to remote *@zenspider.ca
Aug 2 12:21:40 svr qmail: 1154517700.839288 status: local 0/10 remote 1/20
Aug 2 12:22:34 svr qmail: 1154517754.842223 starting delivery 1042: msg
102383715 to local 8-*@eiloart.com
Kai Bolay
> Well, it does run. However, it seems that qmail is trying to run 50 odd
> processes separately. Only the first few ran. My virtual server has
> pretty limited reserved RAM, so I'm not too surprised at this. Given
> that I'm only trying to deliver into about a dozen domains, I'm amazed
> that qmail wants a process for each recipient. Is there reason that
> qmail wants separate smtp sessions when it could just as well specify
> several recipients in one smtp session?
Yes. It's a design decision. "Unbundling" makes implementation easier
and VERP possible.
> Aug 2 12:20:53 svr qmail: 1154517653.901923 status: local 1/10
remote 15/20
> Aug 2 12:20:53 svr qmail: 1154517653.901988 delivery 999: failure:
Unable_to_run_qmail-remote./
Wow. Only 15 out of 20 possible qmail-remote sessions and you're running
out of RAM. Check out
http://cr.yp.to/qmail/faq/efficiency.html#concurrency - you'll want to
reduce concurrencyremote significantly. Maybe down to 5 just to be sure.
Probably reduce concurrencylocal, too.
Kai
Kyle Wheeler
> Ian Eiloart wrote:
>
>> Well, it does run. However, it seems that qmail is trying to run 50 odd
>> processes separately. Only the first few ran. My virtual server has
>> pretty limited reserved RAM, so I'm not too surprised at this. Given
>> that I'm only trying to deliver into about a dozen domains, I'm amazed
>> that qmail wants a process for each recipient. Is there reason that
>> qmail wants separate smtp sessions when it could just as well specify
>> several recipients in one smtp session?
>
> and VERP possible.
There's more to it than that. If that was all it is, those would be
stupid reasons. There's a good discussion/justification of the method
and comparison to the alternatives here:
http://www.lifewithqmail.org/lwq.html#multi-rcpt
It boils down to: it's *faster* as well as simpler.
~Kyle
--
Ten percent of people can think, another ten percent of people think
that they think, and eighty percent of people would rather die than be
made to think.
-- Ralph Waldo Emerson
Ian Eiloart
--On 7 August 2006 11:55:21 -0400 Kyle Wheeler <kyle-...@memoryhole.net>
wrote:
>
> There's more to it than that. If that was all it is, those would be=20
> stupid reasons. There's a good discussion/justification of the method=20
> and comparison to the alternatives here:=20
> http://www.lifewithqmail.org/lwq.html#multi-rcpt
>
> It boils down to: it's *faster* as well as simpler.
Really? Its faster to make six connections to a remote host to send the
same email to six different people, than it is to make one connection and
say "here's an email for these six people"?
It's faster to wait for the remote host to virus scan the same message six
times than once? To wait for the remote host to spam scan the message six
times? To validate the sender address six times? To validate the sending
host six times?
I guess that in some cases, where the remote host can parallelise the
incoming streams, there may be a benefit. But usually not. The case is made
in this paragraph:
> The second method uses multiple connections and sends multiple copies of
> the message. That "wastes" bandwidth, but due to the nature of the SMTP
> protocol, requires fewer round-trip delays, and is faster than the third
> method. It's also simpler than the third method, so the MTA can be coded
> in a more straightforward manner. And finally, because each recipient
> gets their own copy of the message, it's possible for the MTA to
> implement VERPs (see next section).
>
The comment on VERP is irrelevant, if the sender is different, then it's a
different message. I simply don't believe that "due to the nature of the
SMTP protocol, [making a connection for each recipient] requires fewer
round-trip delays,". That might be true *per message*, but there are many
more messages required. In fact, for multiple recipients, it's one round
trip per extra recipient - but sending a separate message is a connection,
greeting, MAIL, RCPT, DATA and message.
For large mailing lists, "messages have, at most, a couple recipients, and
they're usually on separate hosts," is not true either. Many of my large
mailing lists have dozens of hotmail subscribers, for example. I guess that
that's not "Most messages".
Still, given that it's embedded into the architecture, I'm not suggesting
that qmail change this behaviour.
Ian Eiloart
--On 7 August 2006 11:15:07 -0400 Kai Bolay <k...@bolay.de> wrote:
> Ian Eiloart wrote:
>
> > Well, it does run. However, it seems that qmail is trying to run 50
> odd
> > processes separately. Only the first few ran. My virtual server has
> > pretty limited reserved RAM, so I'm not too surprised at this. Given
> > that I'm only trying to deliver into about a dozen domains, I'm amazed
> > that qmail wants a process for each recipient. Is there reason that
> > qmail wants separate smtp sessions when it could just as well specify
> > several recipients in one smtp session?
>
> Yes. It's a design decision. "Unbundling" makes implementation easier and
> VERP possible.
Actually, VERP is possible without this. For example, Exim can be
configured to do VERP, but doesn't require multiple processes - the VERP
address is calculated by Exim. It is perfectly natural for an SMTP session
to deliver a message, say "RSET", and deliver another message from a new
VERP sender address. Even when delivering to multiple domains, the SMTP
sessions could be handled serially with a performance hit, or in threads.
I guess the implementation may not be so easy.
> > Aug 2 12:20:53 svr qmail: 1154517653.901923 status: local 1/10
> remote 15/20
> > Aug 2 12:20:53 svr qmail: 1154517653.901988 delivery 999: failure:
> Unable_to_run_qmail-remote./
>
> Wow. Only 15 out of 20 possible qmail-remote sessions and you're running
> out of RAM.
It was the cheapest virtual server I could find! There is some guaranteed
available RAM, but not much.
> Check out
> http://cr.yp.to/qmail/faq/efficiency.html#concurrency - you'll want to
> reduce concurrencyremote significantly. Maybe down to 5 just to be sure.
Ah, that could do the trick. Thanks!
> Probably reduce concurrencylocal, too.
Well, there's just me that's local at the moment. But I guess it should
probably be the same.
> Kai
Richard Lyons
> For large mailing lists, "messages have, at most, a couple recipients, and
> they're usually on separate hosts," is not true either. Many of my large
> mailing lists have dozens of hotmail subscribers, for example. I guess that
> that's not "Most messages".
Even with large lists, the savings may not be great:
http://marc.theaimsgroup.com/?l=qmail&m=104337391928112&w=2
Rick.
Kyle Wheeler
>> There's more to it than that. If that was all it is, those would be
>> http://www.lifewithqmail.org/lwq.html#multi-rcpt
>>
>> It boils down to: it's *faster* as well as simpler.
>
> Really? Its faster to make six connections to a remote host to send the
> same email to six different people, than it is to make one connection and
> say "here's an email for these six people"?
Yes. And that webpage explains why:
>> That "wastes" bandwidth, but due to the nature of the SMTP
>> protocol, requires fewer round-trip delays, and is faster than the
>> third method.
Since you aren't paying attention, let me lay it out for you: as an
SMTP client, you MAY NOT send the next recipient until the server has
approved of the first. An SMTP conversation goes like this:
> HELO friend
...wait...
< 250 memoryhole.net
> MAIL FROM: <st.c...@north.pole>
...wait...
< 250 ok
> RCPT TO: <f...@bar.com>
...wait...
< 250 ok
> RCPT TO: <fo...@bar.com>
...wait...
< 250 ok
> DATA
...wait...
< 354 go ahead
...and so forth. Notice all those "...wait..." statements. You're NOT
allowed to proceed until the server responds. (This is somewhat helped
if the server supports the PIPELINING extension.)
If you send them as separate connections, the waiting can overlap.
Like so:
> HELO friend > HELO friend
...wait... ...wait...
< 250 memoryhole.net < 250 memoryhole.net
> MAIL FROM:<st.c...@north.pole> > MAIL FROM:<st.c...@north.pole>
...wait... ...wait...
< 250 ok < 250 ok
> RCPT TO: <f...@bar.com> > RCPT TO: <fo...@bar.com>
...wait... ...wait...
< 250 ok < 250 ok
> DATA > DATA
...wait... ...wait...
< 354 go ahead < 354 go ahead
Tada! The same information communicated with only four waits rather
than five.
> I guess that in some cases, where the remote host can parallelise
> the incoming streams, there may be a benefit. But usually not. The
> case is made in this paragraph:
If your host cannot parallelize incoming streams, then you deserve
what you get. As a *server*, you MUST be able to handle multiple
clients at the same time. Because we're dealing in I/O, there's going
to be a lot of lag-time for a single connection, so you basically have
no excuse for refusing to handle other connections at the same time. I
have no idea why you say "But usually not." about the prospects of a
*server* being able to handle multiple connections at the same time.
Go examine Apache, sendmail, postfix, openssh, and then come back and
tell me how many of them are incapable of handling multiple incoming
streams at the same time. (Hint: none of them.)
> I simply don't believe that "due to the nature of the SMTP protocol,
> [making a connection for each recipient] requires fewer round-trip
> delays,". That might be true *per message*, but there are many more
> messages required. In fact, for multiple recipients, it's one round
> trip per extra recipient - but sending a separate message is a
> connection, greeting, MAIL, RCPT, DATA and message.
Yes, sending a separate message is an extra connection, but it is not
extra WAITING. Instead you are using extra BANDWIDTH to decrease
LATENCY by doing your communication in PARALLEL. Please look up these
basic networking concepts before making an ass of yourself on a public
mailing list.
> For large mailing lists, "messages have, at most, a couple
> recipients, and they're usually on separate hosts," is not true
> either. Many of my large mailing lists have dozens of hotmail
> subscribers, for example. I guess that that's not "Most messages".
Okay, let's talk basic large mailing list policy. From
http://www.faqs.org/faqs/mail/list-admin/software-faq/ section 2.05:
Majordomo (a good example of the "leave-it-to-Sendmail" model),
once it decides to forward a message to a list, passes it to a
single Sendmail process along with the addresses for the entire
list. Sendmail then does what it can to optimize delivery (i.e.
sorting by MX record), and starts connecting to each machine in
series. The result on a 200-subscriber list, with everyone in the
U.S. and most with different mail exchangers, is that there's
about an hour's delay between the first delivery and the last.
Yikes!
ListProc speeds up the delivery process for large lists by passing
each message off to your MTA (usually Sendmail, but ListProc
doesn't care, it just connects to port 25 with SMTP) in chunks of
N addresses, where N is defineable. Depending on how the MTA is
configured, delivery will usually be faster because deliveries are
parallelized -- the longest list is of size N. However, Sendmail
can't optimize deliveries as much, because no particular Sendmail
process has the entire list to work with. Thus network efficiency
can go down (not actually a big problem, because ListProc produces
"chunks" sorted by domain).
In qmail, this is not a problem, because each recipient is treated
independently (excluding the host-backoff scheme). It is not
*efficient*, but the throughput is very high, and that's what really
matters.
The question is: how optimized can your mailing list be for your
"dozens of Hotmail subscribers"? Do you use VERP? If you do use VERP
(a *very* good idea) then you get no benefit from multi-recipient
messages to the same host. If not, how do you track which users are
reachable and which are not---parsing bounce messages? How much do you
pay, in terms of CPU time, to handle those bounce messages? If you do
not parse bounce messages, what sort of a penalty do you pay for
sending messages that you could have known would not succeed?
~Kyle
--
Disobedience, in the eyes of any one who has read history, is man's
original virtue. It is through disobedience that progress has been
made, through disobedience and through rebellion.
-- Oscar Wilde
Kyle Wheeler
> In qmail, this is not a problem, because each recipient is treated
> independently (excluding the host-backoff scheme). It is not
> *efficient*, but the throughput is very high, and that's what really
> matters.
I should add that it depends on what you're optimizing, and what your
constraints are. If you are bandwidth-limited, then you may get higher
throughput by condensing your messages to the same host, if only
because you can get more messages out per unit of time that way.
However, on most networks, SMTP represents less than half of the
network traffic (I don't have the exact number handy, but I believe
it's usually something like 8-15%). As such, using a little extra
bandwidth to reduce latency is a solid win.
~Kyle
--
We must not confuse dissent with disloyalty. When the loyal opposition
dies, I think the soul of America dies with it.
-- Edward R. Murrow
Ian Eiloart
--On 8 August 2006 11:12:29 -0400 Kyle Wheeler <kyle-...@memoryhole.net>
wrote:
> On Tuesday, August 8 at 12:17 PM, quoth Ian Eiloart:
>>> There's more to it than that. If that was all it is, those would be
>>> stupid reasons. There's a good discussion/justification of the
>>> method and comparison to the alternatives here:
>>> http://www.lifewithqmail.org/lwq.html#multi-rcpt
>>>
>>> It boils down to: it's *faster* as well as simpler.
>>
>> Really? Its faster to make six connections to a remote host to send the
>> same email to six different people, than it is to make one connection
>> and say "here's an email for these six people"?
>
> Yes. And that webpage explains why:
>
>>> That "wastes" bandwidth, but due to the nature of the SMTP
>>> protocol, requires fewer round-trip delays, and is faster than the
>>> third method.
>
> Since you aren't paying attention,
I am paying attention. Please don't be rude. I count 8 waits, not four.
Parallelised, maybe, depending on whether the remote server lets you do
that. Mine does, to a limit of a few connections, in order to prevent
denial of service attacks. Beyond that, you have to wait just to get the
connection.
And, more to the point, qmail screws up badly when it can't launch enough
processes - bouncing my mail instead of delivering it. In the end, it took
days, not minutes to get my ONE message to fifty recipients. And it took me
fifteen minutes of real to resend the message without duplication.
>> I guess that in some cases, where the remote host can parallelise
>> the incoming streams, there may be a benefit. But usually not. The
>> case is made in this paragraph:
>
> If your host cannot parallelize incoming streams, then you deserve what
> you get. As a *server*, you MUST be able to handle multiple clients at
> the same time. Because we're dealing in I/O, there's going to be a lot of
> lag-time for a single connection, so you basically have no excuse for
> refusing to handle other connections at the same time. I have no idea why
> you say "But usually not." about the prospects of a *server* being able
> to handle multiple connections at the same time. Go examine Apache,
> sendmail, postfix, openssh, and then come back and tell me how many of
> them are incapable of handling multiple incoming streams at the same
> time. (Hint: none of them.)
>
>> I simply don't believe that "due to the nature of the SMTP protocol,
>> [making a connection for each recipient] requires fewer round-trip
>> delays,". That might be true *per message*, but there are many more
>> messages required. In fact, for multiple recipients, it's one round
>> trip per extra recipient - but sending a separate message is a
>> connection, greeting, MAIL, RCPT, DATA and message.
>
> Yes, sending a separate message is an extra connection, but it is not
> extra WAITING. Instead you are using extra BANDWIDTH to decrease LATENCY
> by doing your communication in PARALLEL. Please look up these basic
> networking concepts before making an ass of yourself on a public mailing
> list.
Please look up basic courtesy concepts before doing likewise.
.....
> that's what really matters.
>
> The question is: how optimized can your mailing list be for your "dozens
> of Hotmail subscribers"? Do you use VERP? If you do use VERP (a *very*
> good idea) then you get no benefit from multi-recipient messages to the
> same host.
Yes, actually I do use VERP. I have a choice of doing that in Mailman or
Exim. Currently I do it in Mailman, but I don't like that because it means
handing putting a message per recipient into the outgoing Mailman queue.
I'm planning on doing it in the Exim config instead. But that's off topic.
> If not, how do you track which users are reachable and which
> are not---parsing bounce messages? How much do you pay, in terms of CPU
> time, to handle those bounce messages? If you do not parse bounce
> messages, what sort of a penalty do you pay for sending messages that you
> could have known would not succeed?
>
> ~Kyle
--
Kyle Wheeler
> I am paying attention. Please don't be rude. I count 8 waits, not
> four.
Total time spent waiting divided by the round-trip-time equals: 4
waits. What matters is the total time spent sending message X to both
recipients.
> Parallelised, maybe, depending on whether the remote server lets you
> do that.
Other than intentional limits (and the limits, if you're only trying
to prevent denial of service, can be pretty high-20 or more), what
mail server cannot handle multiple connections? You keep suggesting
that parallel network connections are some bizarre, rare event. Why?
> Mine does, to a limit of a few connections, in order to prevent
> denial of service attacks. Beyond that, you have to wait just to get
> the connection.
I know, I know, draconian connection limiting is a misguided anti-spam
policy that has been spreading relatively recently at a few
universities and small companies (usually based, as best I can tell,
on a recommendation by Barracuda Spam Firewalls), but that makes it
neither a good idea nor something we should optimize for.
Put it this way: qmail's been a part of the scene (in its current
incarnation, with its current policies and behaviors) since at least
1998. Depending on what survey you read, it's either the #2 mail
server in the world, or the #3. Multiple connections from a single
host, therefore, is not only common, but to be expected. Designing a
mail server that cannot handle or permit that is *BROKEN*. This is
going to become more and more the case as NAT installations become
more common for companies as part of a way of reducing costs (no need
to purchase an entire IP range), increasing security (harder to meddle
with your unsecured windows machines if you can't address them
directly), and dealing with the exhaustion of the IPv4 address space.
If I send you two SYN packets from different source ports at roughly
the same time, I expect that those ACK packets will come back at
roughly the same time. I don't have to wait extra long to open two
connections, unless you've set up some sort of filter that refuses to
respond to the second SYN. The time to set up twenty connections,
assuming your system isn't broken (or overloaded), should be
approximately the same time necessary to set up a single connection.
> And, more to the point, qmail screws up badly when it can't launch enough
> processes - bouncing my mail instead of delivering it. In the end, it took
> days, not minutes to get my ONE message to fifty recipients. And it took me
> fifteen minutes of real to resend the message without duplication.
?
I have *never* had that kind of trouble with qmail. On an average day,
my queue mostly consists of mailing list messages that are 99%
complete and are waiting for that last mistyped email address (e.g.
hotmailk.com) to time-out. For direct comparison to your experience,
the average qtime for my (small) 400-recipient mailing lists is
549.284 seconds per message (lists without incorrect addresses
complete much faster, on average). My concurrencyremote is the
default, 20 processes. Something very odd must have happened to your
setup.
>> The question is: how optimized can your mailing list be for your
>> "dozens of Hotmail subscribers"? Do you use VERP? If you do use
>> VERP (a *very* good idea) then you get no benefit from
>> multi-recipient messages to the same host.
>
> Yes, actually I do use VERP. I have a choice of doing that in Mailman or
> Exim. Currently I do it in Mailman, but I don't like that because it means
> handing putting a message per recipient into the outgoing Mailman queue.
> I'm planning on doing it in the Exim config instead. But that's off topic.
So, in other words, you're an exim user badmouthing qmail on the qmail
list? HEH. I believe there's a common term for that sort of behavior.
Because you're using VERP, you're not even benefiting from any
advantages condensing messages with multiple recipients might net you.
What's your beef here?
~Kyle
--
Every American expects and deserves clean air, and then we act on that
belief, then we will set an example for the rest of the world to
follow.
-- George H. W. Bush
Ian Eiloart
--On 8 August 2006 12:55:23 -0400 Kyle Wheeler <kyle-...@memoryhole.net>
wrote:
> Put it this way: qmail's been a part of the scene (in its current
> incarnation, with its current policies and behaviors) since at least
> 1998. Depending on what survey you read, it's either the #2 mail server
> in the world, or the #3.
Or 17th
<http://www.falkotimme.com/projects/survey_smtp_032004.php>
But I'd not put any value in such surveys. Witness the recent decline in
Apache "popularity" as a result of one hosting provider deciding to switch.
Feizhou
>> And, more to the point, qmail screws up badly when it can't launch
>> enough processes - bouncing my mail instead of delivering it. In the
>> end, it took days, not minutes to get my ONE message to fifty
>> recipients. And it took me fifteen minutes of real to resend the
>> message without duplication.
>
> ?
>
> I have *never* had that kind of trouble with qmail. On an average day,
> my queue mostly consists of mailing list messages that are 99% complete
> and are waiting for that last mistyped email address (e.g. hotmailk.com)
> to time-out. For direct comparison to your experience, the average qtime
> for my (small) 400-recipient mailing lists is 549.284 seconds per
> message (lists without incorrect addresses complete much faster, on
> average). My concurrencyremote is the default, 20 processes. Something
> very odd must have happened to your setup.
You forgot that he is not using qmail but something that was originally
qmail. Most likely patched/modified beyond recognition.
Ian Eiloart
--On 10 August 2006 11:32:37 +0800 Feizhou <fei...@graffiti.net> wrote:
>
>>> And, more to the point, qmail screws up badly when it can't launch
>>> enough processes - bouncing my mail instead of delivering it. In the
>>> end, it took days, not minutes to get my ONE message to fifty
>>> recipients. And it took me fifteen minutes of real to resend the
>>> message without duplication.
>>
>> ?
>>
>> I have *never* had that kind of trouble with qmail. On an average day,
>> my queue mostly consists of mailing list messages that are 99% complete
>> and are waiting for that last mistyped email address (e.g. hotmailk.com)
>> to time-out. For direct comparison to your experience, the average qtime
>> for my (small) 400-recipient mailing lists is 549.284 seconds per
>> message (lists without incorrect addresses complete much faster, on
>> average). My concurrencyremote is the default, 20 processes. Something
>> very odd must have happened to your setup.
>
> You forgot that he is not using qmail but something that was originally
> qmail. Most likely patched/modified beyond recognition.
So, what does vanilla qmail do with an email when it can't launch
qmail-send?
Matthew R. Dempsky
> So, what does vanilla qmail do with an email when it can't launch
> qmail-send?
qmail-send is the master daemon that processes the queue and runs
qmail-lspawn and qmail-rspawn. If it can't run, then the queue simply
never gets processed.
If qmail-remote dies when qmail-rspawn forks it, then the delivery
attempt is treated the same as any other temporary error. If you
continue having concurrency issues, the messages may eventually bounce.
Ian Eiloart
--On 10 August 2006 08:34:07 -0500 "Matthew R. Dempsky" <m...@alkemio.org>
wrote:
> On Thu, Aug 10, 2006 at 11:02:44AM +0100, Ian Eiloart wrote:
>> So, what does vanilla qmail do with an email when it can't launch
>> qmail-send?
>
> qmail-send is the master daemon that processes the queue and runs
> qmail-lspawn and qmail-rspawn. If it can't run, then the queue simply
> never gets processed.
>
> If qmail-remote dies when qmail-rspawn forks it, then the delivery
> attempt is treated the same as any other temporary error. If you
> continue having concurrency issues, the messages may eventually bounce.
Ah, yes it's qmail-remote that doesn't launch. My messages get bounced
immediately as if a permanent error had occurred, so maybe that's one thing
that's been changed with a patch. Very strange thing to be changing, though.