XCLIENT, XFORWARD: widely used, standardisation?
Richard Dawe
Some background:
Before mail reaches MessageLabs' servers it may go through a number of
administrative domains (e.g.: customer -> service provider ->
MessageLabs). On transit through the administrative domains, the
originating server's IP address and HELO string are lost. Clearly this
information loss can impact the effectiveness of spam detection (e.g.:
blocklists).
XCLIENT seems like it will help in this case for service providers we
trust (and vice-versa).
So I have some questions:
(1) How widely used is XCLIENT? Do you know of any ISPs or other
organisations using XCLIENT to pass information about the originating
server around?
(2) I saw that some SMTP proxies support XCLIENT. Do any MTAs support
XCLIENT and/or XFORWARD?
(3) Do you have any plans to standardise XCLIENT and XFORWARD?
I actually started writing an Internet draft on a similar mechanism to
XCLIENT, but based on extra parameters the MAIL FROM. But rather than
creating another SMTP extension, it would seem sensible to use a
pre-existing extension like XCLIENT.
Thanks, regards,
--
Richard Dawe
Software Engineer - Mail Engine
MessageLabs - Be certain
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Victor Duchovni
> (1) How widely used is XCLIENT? Do you know of any ISPs or other
> organisations using XCLIENT to pass information about the originating
> server around?
In the Postfix architecture XCLIENT is primarily intented for testing.
While it also supports fetchmail and similar re-injection scenarios, where
the sender wants to be treated as though it were the client specified in
the XCLIENT command, such use is not recommended. Downstream rejection
of mail creates backscatter.
What is used more frequently is XFORWARD, which makes the original client
information available (for logging and content filters), but does not
impersonate the original client directly. This allows proxy filters
to apply appropriate content policies/scores, while the real client's
identity is used for SMTP acceess control, ...
So if Message-Labs is to harmonize with trusted Postfix clients, you
want to implement XFORWARD, not XCLIENT.
> (2) I saw that some SMTP proxies support XCLIENT. Do any MTAs support
> XCLIENT and/or XFORWARD?
The need for these commands emerged with the introduction of the pre-queue
proxy filter in Postfix 2.1:
http://www.postfix.org/SMTPD_PROXY_README.html
I am not aware of support for these in any other MTA.
> (3) Do you have any plans to standardise XCLIENT and XFORWARD?
This would be nice, because SMTP proxy filter vendors would be able to
implement MTA-neutral filters that would not need to parse the multitude
of Received header formats from various MTAs or know which Received
header represents the entry-point into the organization's network.
This said, most of the gain is in the logs of single-host multi-stage
Postfix configurations, where XFORWARD ensures correct logging by
secondary stages. And so the requirement is mostly that proxies not
remove, reject or damage the XFORWARD command (i.e. just transparently
forward what they don't understand). There is no need for the proxy
to implement support for XFORWARD (the administrator can configure
a regexp to identify the first point-of-entry Received header).
Things get more complex when relaying occurs accross organization
boundaries, and one (say MessageLabs) has to determine the point of
origin in email converging from multiple organizations MTAs, ...
One can see in this situation how it would be useful for you to have a
standard. Such a standard has not been a priority here, because we have
not focused the needs of externally hosted content-filter providers.
This said it would not be unwelcome.
> I actually started writing an Internet draft on a similar mechanism to
> XCLIENT, but based on extra parameters the MAIL FROM. But rather than
> creating another SMTP extension, it would seem sensible to use a
> pre-existing extension like XCLIENT.
>
By all means propose something useful and see whether the IETF will
back a standard in this space. XCLIENT was chosen over MAIL FROM in part
because the size of the data being sent would make the already bloated
MAIL FROM substantially bigger, and potentially create interoperability
issues with firewalls that limit SMTP commands to 1000 bytes or similar.
The IETF on the other hand may be disinclined to expand the set of
SMTP verbs, or simply disinclined to standardize protocols for private
relay arrangements that are not relevant for the general case of domain
to other domain mail. Good luck.
What I really would like to see standardize by the way (but have not
taken the time to propose properly) is my TLS client session key
generation trick that makes TLS session caching work for most
split-brain server caches (logical MX hosts with multiple IPs that
are physically separate hosts, or single IP addresses that map onto
multiple hosts via load-balancers).
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majo...@postfix.org?body=unsubscribe%20postfix-users>
Wietse Venema
> Hello.
>
> Some background:
>
> Before mail reaches MessageLabs' servers it may go through a number of
> administrative domains (e.g.: customer -> service provider ->
> MessageLabs). On transit through the administrative domains, the
> originating server's IP address and HELO string are lost. Clearly this
> information loss can impact the effectiveness of spam detection (e.g.:
> blocklists).
>
> XCLIENT seems like it will help in this case for service providers we
> trust (and vice-versa).
XCLIENT etc. are Postfix-specific extensions. They are not
standardized, and they need further development. In particular,
XFORWARD needs to be made more extensible so that it can pass more
hints to SMTP-based content filters, as I was asked two weeks ago.
XCLIENT etc. are not widely used as far as I know. Neither are
ready for standardization at this point and need further development;
however, my limited cycles are currently more focused on making
Postfix complete with respect to existing RFCs.
Proxy support does not involve much; all Postfix requires is that
proxies copy commands and responses verbatim; it's only when the
proxy actually wants to know the meaning of XFORWARD attributes
that it needs to understand the commands.
> (3) Do you have any plans to standardise XCLIENT and XFORWARD?
>
> I actually started writing an Internet draft on a similar mechanism to
> XCLIENT, but based on extra parameters the MAIL FROM. But rather than
> creating another SMTP extension, it would seem sensible to use a
> pre-existing extension like XCLIENT.
I am not married to the way things are currently implemented, but
I was concerned that things would eventually break as more and more
attributes add to the length of the MAIL FROM command.
I'm certainly interested in standardizing some or all of the
functionality covered with XCLIENT and/or XFORWARD, in a manner
that is not tied to a specific MTA implementation.
Wietse
Richard Dawe
On Wed, 2005-06-01 at 17:55, Wietse Venema wrote:
[snip]
> XCLIENT etc. are not widely used as far as I know. Neither are
> ready for standardization at this point and need further development;
> however, my limited cycles are currently more focused on making
> Postfix complete with respect to existing RFCs.
Do you have a summary of what further development is needed?
(I searched the postfix-users archive, but could not see anything.)
[snip]
> I am not married to the way things are currently implemented, but
> I was concerned that things would eventually break as more and more
> attributes add to the length of the MAIL FROM command.
I think that is a good decision.
> I'm certainly interested in standardizing some or all of the
> functionality covered with XCLIENT and/or XFORWARD, in a manner
> that is not tied to a specific MTA implementation.
For what it's worth, I couldn't see anything in the documentation that
ties it to postfix. I think it could be implemented fine with our MTA,
but I haven't tried it.
Richard Dawe
On Wed, 2005-06-01 at 17:53, Victor Duchovni wrote:
> On Wed, Jun 01, 2005 at 05:25:45PM +0100, Richard Dawe wrote:
[snip]
> What is used more frequently is XFORWARD, which makes the original client
> information available (for logging and content filters), but does not
> impersonate the original client directly. This allows proxy filters
> to apply appropriate content policies/scores, while the real client's
> identity is used for SMTP acceess control, ...
>
> So if Message-Labs is to harmonize with trusted Postfix clients, you
> want to implement XFORWARD, not XCLIENT.
Thanks, I see what you mean.
[snip]
> > (3) Do you have any plans to standardise XCLIENT and XFORWARD?
>
> This would be nice, because SMTP proxy filter vendors would be able to
> implement MTA-neutral filters that would not need to parse the multitude
> of Received header formats from various MTAs or know which Received
> header represents the entry-point into the organization's network.
>
> This said, most of the gain is in the logs of single-host multi-stage
> Postfix configurations, where XFORWARD ensures correct logging by
> secondary stages. And so the requirement is mostly that proxies not
> remove, reject or damage the XFORWARD command (i.e. just transparently
> forward what they don't understand). There is no need for the proxy
> to implement support for XFORWARD (the administrator can configure
> a regexp to identify the first point-of-entry Received header).
>
> Things get more complex when relaying occurs accross organization
> boundaries, and one (say MessageLabs) has to determine the point of
> origin in email converging from multiple organizations MTAs, ...
How you express the data seems relatively easy (even if standardisation
is painful). The trust aspect of this is the hardest part, I think.
I'm not sure how well the ACLs for use of XFORWARD would scale across
adminstrative boundaries. Perhaps the ACLs could be exchanged using DNS?
Organisation A would provide organisation B with a public key.
Organisation B would then sign some records with the public key and
publish them under some well-defined prefix for organisation A to query.
> One can see in this situation how it would be useful for you to have a
> standard. Such a standard has not been a priority here, because we have
> not focused the needs of externally hosted content-filter providers.
> This said it would not be unwelcome.
That's completely understandable.
> > I actually started writing an Internet draft on a similar mechanism to
> > XCLIENT, but based on extra parameters the MAIL FROM. But rather than
> > creating another SMTP extension, it would seem sensible to use a
> > pre-existing extension like XCLIENT.
> >
>
> By all means propose something useful and see whether the IETF will
> back a standard in this space. XCLIENT was chosen over MAIL FROM in part
> because the size of the data being sent would make the already bloated
> MAIL FROM substantially bigger, and potentially create interoperability
> issues with firewalls that limit SMTP commands to 1000 bytes or similar.
>
> The IETF on the other hand may be disinclined to expand the set of
> SMTP verbs, or simply disinclined to standardize protocols for private
> relay arrangements that are not relevant for the general case of domain
> to other domain mail. Good luck.
I wasn't expecting standarisation to be easy. ;)
> What I really would like to see standardize by the way (but have not
> taken the time to propose properly) is my TLS client session key
> generation trick that makes TLS session caching work for most
> split-brain server caches (logical MX hosts with multiple IPs that
> are physically separate hosts, or single IP addresses that map onto
> multiple hosts via load-balancers).
That sounds interesting. I'll take a look at the postfix sources.
Wietse Venema
> Hello.
>
> On Wed, 2005-06-01 at 17:55, Wietse Venema wrote:
> [snip]
> > XCLIENT etc. are not widely used as far as I know. Neither are
> > ready for standardization at this point and need further development;
> > however, my limited cycles are currently more focused on making
> > Postfix complete with respect to existing RFCs.
>
> Do you have a summary of what further development is needed?
>
> (I searched the postfix-users archive, but could not see anything.)
The two extensions are similar in appearance but serve different needs:
1 - XCLIENT is a client impersonation mechanism to override access
controls. It overrides audit trail information.
2 - XFORWARD is a mechanism to propagate additional audit trail
information.
Right now the server will accept only the attributes that it has
support for, and none of the existing attributes happens to
require XTEXT encoding. There is a need for more XFORWARD attributes
to provide more hints to content filters. If we add more attributes
then the implementation will be easier if all attributes are XTEXT
encoded, instead of just those attributes that need it.
> [snip]
> > I am not married to the way things are currently implemented, but
> > I was concerned that things would eventually break as more and more
> > attributes add to the length of the MAIL FROM command.
>
> I think that is a good decision.
With the drawback that it adds to the SMTP command set; this may
fail with some stateful firewalls. Of course very long MAIL FROM
commands may trip up firewalls in a different manner.
> > I'm certainly interested in standardizing some or all of the
> > functionality covered with XCLIENT and/or XFORWARD, in a manner
> > that is not tied to a specific MTA implementation.
>
> For what it's worth, I couldn't see anything in the documentation that
> ties it to postfix. I think it could be implemented fine with our MTA,
> but I haven't tried it.
What I meant to say was that the mechanism is MTA specific because
it is not standardized. I'd love to see something standardized so
that it will be adopted by multiple MTAs.
It does not have to be XCLIENT but if it is almost sufficient
the we might just fix the limitations and use that.
Wietse
