Large scale modular Qmail cluster and the Spam Problem
The Random Posting Network
I run the central mailcore for a medium-size business-class ISP.
The system is entirely qmail based, and, simplified, consists of:
A primary inbound cluster which accepts inbound SMTP traffic.
A set of anti-virus and anti-spam servers which can be switched in and
out of the mail path for any given domain name.
A cluster of final-delivery servers which accept mail from the other
systems and deliver it to end-users POP3 (or Serialmail SMTP) mailboxes,
using Vpopmail.
A relay cluster for customer use which handles outbound mail.
The internal routing between these servers is handled via smtproutes
files, which are assembled by an automated control system.
So far so good, we handle upwards of 10,000 domains, some with local
POP3 delivery, some with SMTP delivery to end-users own mailservers (MS
Exchange, etc), and a few with .qmail forwarding to various mailboxes at
other ISPs.
The only problem is forged-sender spam-bouncing. As currently
configured, our inbound receivers have no way of knowing whether any
given address at a domain we handle mail for actually exists or not, so
in the event that it doesn't we're left bouncing the mail to whatever
happened to be in the envelope sender. With real mail this is of course
correct behaviour but with spam and viruses the envelope sender is
invariably forged so such bounces are either undeliverable, and hence
sit in the queue for a week before expiring, or they are real users
elsewhere, in which case they get an unwanted bounce message.
Now, for domains where final delivery is an Exchange server outwith our
systems, there's not a lot we can do, however I'd like, if possible, to
enable the inbound receivers to issue 55x rejection messages at the
initial SMTP session where the following conditions are *all* met:
Domain's mail is handled by our final delivery system.
<user>'s mailbox does not exist.
There is no .qmail-<user> file present.
.qmail-default for <domain> is set to vpopmail's "bounce-no-mailbox"
option.
Is there any easy way to achieve this? I can if necessary write scripts
to generate a list of relevant domains and then lists of permitted users
at those domains, though note the system still needs to accept and
forward any mail for domains which are in rcpthosts but not in this
list.
Thoughts?
--
The Random Posting Network
Bryan D. Black
I ran into this awhile ago.
Heres what I did:
Create a cdb for postmasters (your Exchange customers, etc.) and a cdb
(maybe in your case per domain) of legit users. Store them in tmpfs/ramfs.
Write a efficient socket based server (i used pthreads) that listens and
reads each cdb per connection from a patched qmail-smtpd (creates a socket
to the pthreaded server and sends addr.s) have that return 1 or 0
based on a simple cdb_seek().
Reject with the your error message if the rcpt user doesn't exist. I used
libcdb.a from tinycdb.
--
Bryan D. Black
Vice President - Information Systems
Triton Technologies, Inc.
The Random Posting Network
<m...@triton.net> writes
>I ran into this awhile ago.
>
>Heres what I did:
>
>Create a cdb for postmasters (your Exchange customers, etc.) and a cdb
>(maybe in your case per domain) of legit users. Store them in tmpfs/ramfs.
>
>Write a efficient socket based server (i used pthreads) that listens and
>reads each cdb per connection from a patched qmail-smtpd (creates a socket
>to the pthreaded server and sends addr.s) have that return 1 or 0
>based on a simple cdb_seek().
>Reject with the your error message if the rcpt user doesn't exist. I used
>libcdb.a from tinycdb.
Thanks for that - assuming I get the go-ahead to build a devel system,
I'll try that. Thanks!
nxtconsulting
The Random Posting Network wrote:
>
> The system is entirely qmail based, and, simplified, consists of:
>
> A primary inbound cluster which accepts inbound SMTP traffic.
>
> A set of anti-virus and anti-spam servers which can be switched in
> and
> out of the mail path for any given domain name.
>
> A cluster of final-delivery servers which accept mail from the other
> systems and deliver it to end-users POP3 (or Serialmail SMTP)
> mailboxes,
> using Vpopmail.
>
> A relay cluster for customer use which handles outbound mail.
>
> The internal routing between these servers is handled via smtproutes
> files, which are assembled by an automated control system.
>
> So far so good, we handle upwards of 10,000 domains, some with local
> POP3 delivery, some with SMTP delivery to end-users own mailservers
> (MS
> Exchange, etc), and a few with .qmail forwarding to various mailboxes
> at
> other ISPs.
>
> configured, our inbound receivers have no way of knowing whether any
> given address at a domain we handle mail for actually exists or not,
> so
> in the event that it doesn't we're left bouncing the mail to
> whatever
> happened to be in the envelope sender. With real mail this is of
> course
> correct behaviour but with spam and viruses the envelope sender is
> invariably forged so such bounces are either undeliverable, and
> hence
> sit in the queue for a week before expiring, or they are real users
> elsewhere, in which case they get an unwanted bounce message.
>
> Now, for domains where final delivery is an Exchange server outwith
> our
> systems, there's not a lot we can do, however I'd like, if possible,
> to
> enable the inbound receivers to issue 55x rejection messages at the
> initial SMTP session where the following conditions are *all* met:
>
> Domain's mail is handled by our final delivery system.
> <user>'s mailbox does not exist.
> There is no .qmail-<user> file present.
> "bounce-no-mailbox"
> option.
>
> Is there any easy way to achieve this? I can if necessary write
> scripts
> to generate a list of relevant domains and then lists of permitted
> users
> at those domains, though note the system still needs to accept and
> forward any mail for domains which are in rcpthosts but not in this
> list.
>
> Thoughts?
>
> The Random Posting Network *
What you're looking for already exists in the spamcontrol patch for
qmail. It's called RECIPIENTS Extension, read more about it here in
section 6.
http://tinyurl.com/2grdz
Later,
Jonathan Cutrer
NXT Network Consulting
http://www.nxtnc.com/
nxtconsulting
------------------------------------------------------------------------
Posted via http://www.webservertalk.com
------------------------------------------------------------------------
View this thread: http://www.webservertalk.com/message109557.html