Stress-dependent server personality
Wietse Venema
SMTP server personality. Most of the material needed for this
already exists in Postfix.
Here is how it could work:
- The master(8) daemon passes a new command-line option to the
smtpd(8) child processes; the option indicates whether or not
all the SMTP service ports are busy. This is a simple change.
- In the smtpd(8) child process, switch to an alternate set of
main.cf configuration parameters, depending on command-line
options received from the master daemon. This is not new: it's
how the combined SMTP/LMTP client chooses between SMTP and LMTP.
- Of course, stress information would be made available in the
policy protocol, and perhaps to Milters.
Each smtpd_mumble configuration parameter would then get its own
doppelgaenger, for example stress_smtpd_mumble (a fixed prefix is
easily implemented with a shell script, and is less work than having
to define a lot of new parameter names in the Postfix source code).
So we would have
smtpd_timeout = 45s
stress_smtpd_timeout = 10s
smtpd_hard_error_limit = 10
stress_smtpd_hard_error_limit = 1
The more courageous sysadmins would make smtpd_delay_reject dependend
on stress level and provide alternate sets of smtpd_mumble_restrictions.
How would one test the alternate personality? Switching to a
different set of main.cf parameters is not possible after a Postfix
daemon has already read main.cf, so the switch must happen before
main.cf is read (*).
Testing would require an extra master.cf entry
127.0.0.1:12345 .. .. .. .. .. smtp -p stress
(or whatever) to connect to an smtpd(8) process that is always in
the stress mode personality.
Thus, stress-dependent server behavior can be implemented with
minor Postfix modifications, but it has a few obvious limitations.
- Testing is possible only by connecting to the test port and using
the XCLIENT protocol. Not a big deal, because stress-dependent
behavior is for advanced system administrators. Hopefully some
day someone will finally implement a utility that uses the XCLIENT
protocol to automate Postfix tests.
- stress_smtpd_mumble parameters would default to $smtpd_mumble.
This may cause some anomalies when the $smtpd_mumble parameter's
actual value contains "... $smtpd_mumble ..."; the non-stress
reference won't be replaced by " ... $stress_smtpd_mumble ...".
- No client dependent switching of main.cf parameters: the parameters
are chosen before smtpd knows what client has connected (*). When
the stress hits the fan, legitimate clients and strangers both
experience the same reduced level of service.
Making the Postfix SMTP server change personality depending on
client identity (or client history) requires invasive changes (*).
If client-dependent personalities are needed. it would make more
sense to have a front end (kernel or user-land) that forwards
preferred clients to a nicer smtpd(8) instance, and that forwards
strangers to a more hostile one.
Wietse
(*) Changing this would violate the constraint that I am spending
most of my time not working on Postfix for the rest of the year.
http://marc.info/?l=php-dev&m=116621380305497&w=2
Sactifices must be made.
Tony Earnshaw
[...]
> The dynamic "-o stress=yes" setting is simple enough that it could
> be implemented as an emergency patch for Postfix 2.3 and earlier.
Sounds good. Especially on the ISC/SANS dshield list there is a bunch of
people reporting repeated increased hammering from diverse IPs, leading
to the MTA striking. They generally don't report what MTA they're using
(to them that fact is immaterial, that's not their problem) and the only
solution till now is a script to parse the logs and adapt the firewall.
For them this might be a good reason for looking at Postfix.
My Postfix 2.4.5 site only gets occasional bursts that soon stop.
--Tonni
--
Tony Earnshaw
Email: tonni at hetnet dot nl
Noel Jones
> >
> > - The master(8) daemon passes a new command-line option to the
> > smtpd(8) child processes; the option indicates whether or not
> > all the SMTP service ports are busy. This is a simple change.
>
>sets of main.cf parameters. If the master would set a command-line
>option like "-o stress=yes" when a type "inet" service is utilized
>for 80%, that would be sufficient.
>
>For example, if the number of smtpd processes reaches 80% of
>the process limit, the master would specify "-o stress=yes"
>on the smtpd command lines, and this could change the meaning
>of smtpd_timeout or smtpd_peername_lookup:
>
> # Use 10s timeouts when under stress.
> smtpd_timeout = ${stress?10}${stress:300}
>
> # Don't look up the client hostnames when under stress.
> # Caution: this pretends that lookups fail with a hard errors.
> smtpd_peername_lookup = ${stress?no}${stress:yes}
>
>The dynamic "-o stress=yes" setting is simple enough that it could
>be implemented as an emergency patch for Postfix 2.3 and earlier.
>
This looks interesting.
I'm making the assumption that the above settings are examples (and
not proposed defaults).
In which case it looks as if pretty much any parameter used by smtpd
would be able to use the
smtpd_foo = ${stress?emergency_foo}${stress:normal_foo}
style syntax to specify alternate settings. Correct?
Cool.
--
Noel Jones
Noel Jones
Never mind. I see you have a patch and instructions out already.
Thanks, I'll try it.
--
Noel Jones
Christian Winter
> Thus, stress-dependent server behavior can be implemented with
> minor Postfix modifications, but it has a few obvious limitations.
>
> - Testing is possible only by connecting to the test port and using
> the XCLIENT protocol. Not a big deal, because stress-dependent
> behavior is for advanced system administrators. Hopefully some
> day someone will finally implement a utility that uses the XCLIENT
> protocol to automate Postfix tests.
>
plugged together a view lines of perl to perform checks on our company
mail filters with XCLIENT extensions (a class deriving from Net::SMTP
and a command line perl script) and already thought about bundling it
together and putting it at CPAN. Any suggestions are welcome.
That's what the script does so far:
-----------------------------------------------------------------------------
NAME
xclienttest.pl - Command line tool to test XCLIENT SMTP
DESCRIPTION
xclienttest.pl -f FROM -r RECIPIENT [-a] [-s SERVER] [-p PORT] [-c
CREDENTIALS] [-N XCLIENT_NAME] [-A XCLIENT_ADDRESS] [-P
XCLIENT_PROTOCOL] [-H XCLIENT_HELO] [-F CFGFILE] [-d] [-h]
Command line tool for testing of SMTP servers that support the XCLIENT
protocol. All neccessary paramters can be set via the command line, the
mail headers and body can either be supplied via STDIN (sendmail style)
or a standard test message can be generated via the *-a* switch.
COMMAND LINE ARGUMENTS
-h Show this help screen.
-a Create a test mail with standard contents, don't listen on *STDIN*
for a user-supplied message.
-f FROM
Set the sender of the mail. Must be a syntactically valid SMTP
address.
-r RECIPIENT
Recipient of the mail. Must be a syntactically valid SMTP address.
-s SERVER
SMTP server to connect to, defaults to 127.0.0.1.
-p PORT
Listen port of the SMTP server to connect to, defaults to 25.
-c CREDENTIALS
If the server requires a login, the credentials can be passed as a
string in the form "USERNAME:PASSWORD".
-N XCLIENT_NAME
The alternative client name that is to be supplied via the XCLIENT
command.
-A XCLIENT_ADDRESS
The IPV4 or IPV6 client address that is to be supplied via the
XCLIENT command.
-P XCLIENT_PROTOCOL
The SMTP client protocol (SMTP or ESMTP) to be supplied via the
XCLIENT command, this can differ from the result of a previous
HELO/EHLO command.
-H XCLIENT_HELO
The alternativ HELO/EHLO string to be passed to the server via the
XCLIENT command.
-F CFGFILE
Instead of passing the XCLIENT values via the command line, you can
also give a config file that contains some or all of them. Note that
command line options override config file options.
The contents of the config file are the XCLIENT verbs followed by an
equal sign followed by the plain values. Spaces are allowed before
and after the equal sign.
Example:
ADDR = 172.30.0.2
NAME = bad_client.from.local
PROTO = ESMTP
HELO = bad_client
-d Show debug output on STDOUT.
---------------------------------------------------------------------------------
-Chris