serious bug with check_client_access
Vincent Lefevre
It seems that I've found a serious bug in check_client_access
(or something is missing in the documentation).
A message was blocked with the following in the log:
Nov 3 21:16:55 ioooi postfix/smtpd[15423]: NOQUEUE: reject: RCPT from mx003.twitter.com[128.121.146.152]: 554 5.7.1 Service unavailable; Client host [128.121.146.152] blocked using bl.spamcop.net; Blocked - see http://www.spamcop.net/bl.shtml?128.121.146.152; from=<twitter-follow-xxxxxxxxxx=xxxxx...@postmaster.twitter.com> to=<xxxxx...@xxxxxx.xxx> proto=ESMTP helo=<mx003.twitter.com>
In my /etc/postfix/main.cf file, I have:
smtpd_client_restrictions =
permit_mynetworks,
check_client_access hash:/etc/postfix/rbl_override,
reject_rbl_client bl.spamcop.net,
reject_rbl_client sbl.spamhaus.org
and /etc/postfix/rbl_override contains in particular:
.twitter.com OK
/etc/postfix/rbl_override.db had been rebuilt with
"postmap /etc/postfix/rbl_override".
As .twitter.com matches subdomains, it should have matched
mx003.twitter.com, and due to the ".twitter.com OK", the RBL's
shouldn't have been checked.
The machine is an Ubuntu 9.04 (jaunty), with the postfix 2.5.5-1.1
package. The machine has been rebooted since the latest config
changes.
--
Vincent Lefèvre <vin...@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Wietse Venema
> As .twitter.com matches subdomains, it should have matched
What documentation supports this?
Vincent Lefevre
The access(5) man page says:
domain.tld
Matches domain.tld.
The pattern domain.tld also matches subdomains, but only
when the string smtpd_access_maps is listed in the Postfix
parent_domain_matches_subdomains configuration setting.
Otherwise, specify .domain.tld (note the initial dot) in
order to match subdomains.
See the last sentence.
/dev/rob0
> On 2010-11-03 22:00:21 -0400, Wietse Venema wrote:
> > Vincent Lefevre:
> > > As .twitter.com matches subdomains, it should have matched
> >
> > What documentation supports this?
>
> The access(5) man page says:
>
> domain.tld
> Matches domain.tld.
>
> The pattern domain.tld also matches subdomains, but only
> when the string smtpd_access_maps is listed in the Postfix
> parent_domain_matches_subdomains configuration setting.
> Otherwise, specify .domain.tld (note the initial dot) in
> order to match subdomains.
>
> See the last sentence.
See the one before that. :) What is your setting of
parent_domain_matches_subdomains?
Repeat after me: "Oops, sorry Wietse." :)
--
Offlist mail to this address is discarded unless
"/dev/rob0" or "not-spam" is in Subject: header
Vincent Lefevre
> On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote:
> > On 2010-11-03 22:00:21 -0400, Wietse Venema wrote:
> > > Vincent Lefevre:
> > > > As .twitter.com matches subdomains, it should have matched
> > >
> > > What documentation supports this?
> >
> > The access(5) man page says:
> >
> > domain.tld
> > Matches domain.tld.
> >
> > The pattern domain.tld also matches subdomains, but only
> > when the string smtpd_access_maps is listed in the Postfix
> > parent_domain_matches_subdomains configuration setting.
> > Otherwise, specify .domain.tld (note the initial dot) in
> > order to match subdomains.
> >
> > See the last sentence.
>
> See the one before that. :) What is your setting of
> parent_domain_matches_subdomains?
You should read my message again. The setting of
parent_domain_matches_subdomains matters only for domain.tld.
But my config file has the form ".domain.tld".
Noel Jones
> On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote:
>> On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote:
>>> On 2010-11-03 22:00:21 -0400, Wietse Venema wrote:
>>>> Vincent Lefevre:
>>>>> As .twitter.com matches subdomains, it should have matched
>>>>
>>>> What documentation supports this?
>>>
>>> The access(5) man page says:
>>>
>>> domain.tld
>>> Matches domain.tld.
>>>
>>> The pattern domain.tld also matches subdomains, but only
>>> when the string smtpd_access_maps is listed in the Postfix
>>> parent_domain_matches_subdomains configuration setting.
>>> Otherwise, specify .domain.tld (note the initial dot) in
>>> order to match subdomains.
>>>
>>> See the last sentence.
>>
>> See the one before that. :) What is your setting of
>> parent_domain_matches_subdomains?
>
> You should read my message again. The setting of
> parent_domain_matches_subdomains matters only for domain.tld.
> But my config file has the form ".domain.tld".
>
Your interpretation is incorrect.
".domain.tld" only works if parent_domain_matches_subdomains
does NOT include smtpd_access maps.
/dev/rob0
> On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote:
> > On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote:
> > > On 2010-11-03 22:00:21 -0400, Wietse Venema wrote:
> > > > Vincent Lefevre:
> > > > > As .twitter.com matches subdomains, it should have matched
> > > >
> > > > What documentation supports this?
> > >
> > > The access(5) man page says:
> > >
> > > domain.tld
> > > Matches domain.tld.
> > >
> > > The pattern domain.tld also matches subdomains, but only
The pattern domain.tld also matches subdomains ...
You don't have that in your map.
> > > when the string smtpd_access_maps is listed in the Postfix
> > > parent_domain_matches_subdomains configuration setting.
You failed to provide postconf -n in the OP, and again failed to
answer my question about it.
> > > Otherwise, specify .domain.tld (note the initial dot) in
----------------^^^^^^^^^
> > > order to match subdomains.
You're not in the "otherwise" category, I bet.
> > >
> > > See the last sentence.
> >
> > See the one before that. :) What is your setting of
> > parent_domain_matches_subdomains?
>
> You should read my message again. The setting of
> parent_domain_matches_subdomains matters only for domain.tld.
> But my config file has the form ".domain.tld".
http://www.postfix.org/postconf.5.html#parent_domain_matches_subdomains
Sahil Tandon
> On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote:
> > On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote:
> > > On 2010-11-03 22:00:21 -0400, Wietse Venema wrote:
> > > > Vincent Lefevre:
> > > > > As .twitter.com matches subdomains, it should have matched
> > > >
> > > > What documentation supports this?
> > >
> > > The access(5) man page says:
> > >
> > > domain.tld Matches domain.tld.
> > >
> > > The pattern domain.tld also matches subdomains, but
> > > only when the string smtpd_access_maps is listed in the
> > > Postfix parent_domain_matches_subdomains configuration
> > > setting. Otherwise, specify .domain.tld (note the
> > > initial dot) in order to match subdomains.
> > >
> > > See the last sentence.
> >
> > See the one before that. :) What is your setting of
> > parent_domain_matches_subdomains?
>
> You should read my message again. The setting of
> parent_domain_matches_subdomains matters only for domain.tld. But my
> config file has the form ".domain.tld".
No, *you* should re-read the documentation.
--
Sahil Tandon <sa...@FreeBSD.org>
Vincent Lefevre
> On Thu, Nov 04, 2010 at 03:36:30AM +0100, Vincent Lefevre wrote:
> > On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote:
> > > On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote:
> > > > On 2010-11-03 22:00:21 -0400, Wietse Venema wrote:
> > > > > Vincent Lefevre:
> > > > > > As .twitter.com matches subdomains, it should have matched
> > > > >
> > > > > What documentation supports this?
> > > >
> > > > The access(5) man page says:
> > > >
> > > > domain.tld
> > > > Matches domain.tld.
> > > >
> > > > The pattern domain.tld also matches subdomains, but only
>
>
> You don't have that in your map.
>
> > > > parent_domain_matches_subdomains configuration setting.
>
> answer my question about it.
>
> > > > order to match subdomains.
>
> You're not in the "otherwise" category, I bet.
So what? There's some logic error. "Otherwise" is a replacement for
"if smtpd_access_maps is not listed in parent_domain_matches_subdomains"
but nowhere it is said that domain.tld is not supported if this
pattern starts with a dot.
Vincent Lefevre
> ".domain.tld" only works if parent_domain_matches_subdomains does NOT
> include smtpd_access maps.
The man page says nothing like that. So, the documentation should be
fixed.
Noel Jones
> On 2010-11-03 21:40:54 -0500, Noel Jones wrote:
>> ".domain.tld" only works if parent_domain_matches_subdomains does NOT
>> include smtpd_access maps.
>
> The man page says nothing like that. So, the documentation should be
> fixed.
>
The vast majority of readers seem to interpret that section as
intended. You're welcome to post a documentation patch in a
new thread, but I don't think the behavior or its
documentation has changed in ~10 years, so don't hold your breath.
And next time behavior doesn't match your expectations, you
might get more sympathy if your message starts with "please
clarify this for me" rather than "serious bug".
-- Noel Jones
Vincent Lefevre
> On 11/3/2010 10:00 PM, Vincent Lefevre wrote:
> >On 2010-11-03 21:40:54 -0500, Noel Jones wrote:
> >>".domain.tld" only works if parent_domain_matches_subdomains does NOT
> >>include smtpd_access maps.
> >
> >The man page says nothing like that. So, the documentation should be
> >fixed.
>
> The vast majority of readers seem to interpret that section as intended.
I'd say that the vast majority of readers probably haven't tried
".domain.tld" (with an initial dot) or haven't even read the
documentation, so that they didn't have to interpret the sentence
on this point. And many people interpret "If A then B" as also
meaning that "If not A then not B" (against mathematical logic),
and this doesn't mean that they are correct.
Also, perhaps other users use this form and don't know that it doesn't
work, because AFAIK, one doesn't get any feedback (I haven't seen any
warning) until the sender says that some mail has been rejected by
some later rule.
> You're welcome to post a documentation patch in a new thread, but I don't
> think the behavior or its documentation has changed in ~10 years, so don't
> hold your breath.
Something difficult for users who don't know postfix internals.
What happens if the ".domain.tld" form is used? Is it simply ignored?
Or does anything else occur?
> And next time behavior doesn't match your expectations, you might get more
> sympathy if your message starts with "please clarify this for me" rather
> than "serious bug".
Actually if a documentation is incorrect/incomplete, it is a bug in
the documentation. And FYI, the consequence was a lost mail. So, this
is quite serious.
Noel Jones
> Actually if a documentation is incorrect/incomplete, it is a bug in
> the documentation. And FYI, the consequence was a lost mail. So, this
> is quite serious.
>
I'm so sorry you lost your twitter post. The access map
format you're looking for is
twitter.com OK
Vincent Lefevre
> I'm so sorry you lost your twitter post.
Actually I might have lost other mail (though this is a bit unlikely)
since I was generally using an initial dot.
> The access map format you're looking for is
> twitter.com OK
Thanks for the information. I've corrected the whole access file.
BTW, so, there is no way to match only subdomains (by that, I mean
all possible subdomains, but not the domain itself) without changing
parent_domain_matches_subdomains?
I don't currently need such a feature, but I'm asking just in case...
--
Vincent Lef�vre <vin...@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Ar�naire project (LIP, ENS-Lyon)
Noel Jones
> BTW, so, there is no way to match only subdomains (by that, I mean
> all possible subdomains, but not the domain itself) without changing
> parent_domain_matches_subdomains?
That's correct with indexed tables. With regexp or pcre
tables there is no automatic subdomain search; you control the
scope of the search with your expression.
To answer your other question, when
parent_domain_matches_subdomains includes smtpd_access_maps
(the default), the form ".domain.tld" is never searched for.
As a result, the entry is silently ignored.
-- Noel Jones
Покотиленко Костик
> On 11/3/2010 10:00 PM, Vincent Lefevre wrote:
> > On 2010-11-03 21:40:54 -0500, Noel Jones wrote:
> >> ".domain.tld" only works if parent_domain_matches_subdomains does NOT
> >> include smtpd_access maps.
> >
> > The man page says nothing like that. So, the documentation should be
> > fixed.
> >
>
> The vast majority of readers seem to interpret that section as
> documentation has changed in ~10 years, so don't hold your breath.
Vast majority even believe in strange things...
Actually I read that section exactly like Vincent Lefevre.
I'll comment:
> The access(5) man page says:
>
> domain.tld
> Matches domain.tld.
>
> The pattern domain.tld also matches subdomains, but only
> when the string smtpd_access_maps is listed in the Postfix
> parent_domain_matches_subdomains configuration setting.
> Otherwise, specify .domain.tld (note the initial dot) in
> order to match subdomains.
I read "Otherwise" as "If you don't like to depend on the value of
parent_domain_matches_subdomains"
"specify .domain.tld (note the initial dot) in order to match
subdomains" means exactly that using ".domain.tld" form WILL match
subdomains with no other condition.
Also regarding to what you are telling there is no difference in those
two sentences, and it's completelly unclear in what those two notation
forms differ and why author has written one idea in two sentences.
Check your English ;)
> And next time behavior doesn't match your expectations, you
> might get more sympathy if your message starts with "please
> clarify this for me" rather than "serious bug".
If behavior doesn't match your expectations and also doesn't match docs
it's a bug, either in soft or in docs.
--
Покотиленко Костик <cas...@meteor.dp.ua>
Emmanuel Fusté
Hopefully, I generaly clear parent_domain_matches_subdomains in my
configurations.
Emmanuel.
Покотиленко Костик
> Zitat von О©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©Ґ О©ҐО©ҐО©ҐО©ҐО©ҐО©Ґ <cas...@meteor.dp.ua>:
>
> > О©Ґ О©ҐО©ҐО©Ґ, 03/11/2010 О©Ґ 22:16 -0500, Noel Jones О©ҐО©ҐО©ҐО©ҐО©Ґ:
> >> On 11/3/2010 10:00 PM, Vincent Lefevre wrote:
> >> > On 2010-11-03 21:40:54 -0500, Noel Jones wrote:
> >> >> ".domain.tld" only works if parent_domain_matches_subdomains does NOT
> >> >> include smtpd_access maps.
> >> >
> >> > The man page says nothing like that. So, the documentation should be
> >> > fixed.
> >> >
> >>
> >> The vast majority of readers seem to interpret that section as
> >> intended. You're welcome to post a documentation patch in a
> >> new thread, but I don't think the behavior or its
> >> documentation has changed in ~10 years, so don't hold your breath.
> >
> > Vast majority even believe in strange things...
> >
> > Actually I read that section exactly like Vincent Lefevre.
> >
> > I'll comment:
> >
> >> The access(5) man page says:
> >>
> >> domain.tld
> >> Matches domain.tld.
> >>
> >> The pattern domain.tld also matches subdomains, but only
> >> when the string smtpd_access_maps is listed in the Postfix
> >> parent_domain_matches_subdomains configuration setting.
> >> Otherwise, specify .domain.tld (note the initial dot) in
> >> order to match subdomains.
> >
> > I read "Otherwise" as "If you don't like to depend on the value of
> > parent_domain_matches_subdomains"
>
> statement. If it is unclear, ask for clarification.
Actually it's not clear to what condition the "Otherwise" is a conter
part. So it's being decided by a reader's logic wich differs sometimes.
I'm myself understood the actual meaning only after clarification on
this list. And, pleople would ask for clarification only if they can't
understand what is being ment, and not in case they think they
unsterstand.
Also, it's completelly unstated that ".domain.tld" notation doesn't work
if smtpd_access_maps is listed in parent_domain_matches_subdomains.
> Crying loud
> "serious bug" because your language interpretation is different from
> others is not helpful at all.
List subscriber classified this situation as a "serious bug" from it's
point of view, and I think this is correct. If you don't agree, just
reclassify it.
BTW, clear docs save much time for both, the user, the list.
So, I suggest rewritiing this paragraph as following:
The pattern domain.tld also matches subdomains, but only
when the string smtpd_access_maps is listed in the Postfix
parent_domain_matches_subdomains configuration setting.
The pattern .domain.tld (note the initial dot) matches
only subdomains and only when the string
smtpd_access_maps is NOT listed in the Postfix
parent_domain_matches_subdomains configuration setting,
otherwise it's ignored.
Is this correct?
--
О©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©ҐО©Ґ О©ҐО©ҐО©ҐО©ҐО©ҐО©Ґ <cas...@meteor.dp.ua>
Vincent Lefevre
> >>The access(5) man page says:
> >>
> >> domain.tld
> >> Matches domain.tld.
> >>
> >> The pattern domain.tld also matches subdomains, but only
> >> when the string smtpd_access_maps is listed in the Postfix
> >> parent_domain_matches_subdomains configuration setting.
> >> Otherwise, specify .domain.tld (note the initial dot) in
> >> order to match subdomains.
> >
> >I read "Otherwise" as "If you don't like to depend on the value of
> >parent_domain_matches_subdomains"
>
> "Otherwise" is clearly related to the last part of the previous statement.
Yes, but it adds information in the case where smtpd_access_maps IS NOT
listed in the Postfix parent_domain_matches_subdomains configuration
setting.
The problem occurs when smtpd_access_maps IS listed in the Postfix
parent_domain_matches_subdomains configuration setting. What the
man page says in THIS case is: "The pattern domain.tld also matches
subdomains" where the pattern domain.tld can be ".twitter.com" for
instance.
I don't think there is anything wrong with my reasoning.
> If it is unclear, ask for clarification. Crying loud "serious bug"
> because your language interpretation is different from others is not
> helpful at all.
As for me, the documentation was clear and didn't match the observed
behavior. So, I couldn't say that it was unclear.
mouss
> On 2010-11-03 22:55:59 -0500, Noel Jones wrote:
>> I'm so sorry you lost your twitter post.
> Actually I might have lost other mail (though this is a bit unlikely)
> since I was generally using an initial dot.
a good idea is to include both dotted and undotted entries:
example.com OK
.example.com OK
unless you have a reason not to do so.
if you search the archives for posts I've sent, you'll see that I always
include both.
also, parent_domain_... is deprecated. See
http://www.postfix.org/postconf.5.html#parent_domain_matches_subdomains
The recommendation is to empty it and use dots explicitely. here:
# postconf parent_domain_matches_subdomains
parent_domain_matches_subdomains =
>> The access map format you're looking for is
>> twitter.com OK
> Thanks for the information. I've corrected the whole access file.
>
> BTW, so, there is no way to match only subdomains (by that, I mean
> all possible subdomains, but not the domain itself) without changing
> parent_domain_matches_subdomains?
>
see above: it is recommended not to rely on parent_domain_...
otherwise, you can do whatever you want with pcre:
/\.example\.com$/ OK
or with sql or ldap.
Vincent Lefevre
> otherwise, you can do whatever you want with pcre:
> /\.example\.com$/ OK
> or with sql or ldap.
For pcre, the man page is not clear. It says:
Each pattern is a regular expression that is applied to the entire
string being looked up. Depending on the application, that string is an
entire client hostname, an entire client IP address, or an entire mail
address.
But where is it described whether the string is an entire client
hostname, an entire client IP address, or an entire mail address?
According to your example, the string is an entire client hostname.
But then, this means that one cannot match IP addresses.
--
Vincent Lefèvre <vin...@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Stan Hoeppner
> On 2010-11-04 17:18:17 +0100, mouss wrote:
>> otherwise, you can do whatever you want with pcre:
>> /\.example\.com$/ OK
>> or with sql or ldap.
>
> For pcre, the man page is not clear. It says:
>
> Each pattern is a regular expression that is applied to the entire
> string being looked up. Depending on the application, that string is an
> entire client hostname, an entire client IP address, or an entire mail
> address.
>
> But where is it described whether the string is an entire client
> hostname, an entire client IP address, or an entire mail address?
check_client_access pcre:/etc/postfix/filter.pcre
check_sender_access pcre:/etc/postfix/filter.pcre
check_recipient_access pcre:/etc/postfix/filter.pcre
As you can see, this is defined by the smtpd_foo_restriction you target
the PCRE table with. What is checked against the table is dependent on
the restriction used. Read the documentation for each check_*_access
restriction above at: http://www.postfix.org/postconf.5.html
--
Stan
Vincent Lefevre
> check_client_access pcre:/etc/postfix/filter.pcre
> check_sender_access pcre:/etc/postfix/filter.pcre
> check_recipient_access pcre:/etc/postfix/filter.pcre
>
> As you can see, this is defined by the smtpd_foo_restriction you target
> the PCRE table with. What is checked against the table is dependent on
> the restriction used. Read the documentation for each check_*_access
> restriction above at: http://www.postfix.org/postconf.5.html
On this page, it is said:
check_client_access type:table
Search the specified access database for the client hostname,
^^^^^^^^^^^^^^^
parent domains, client IP address, or networks obtained by
^^^^^^^^^^^^^^^^^
stripping least significant octets. See the access(5) manual page
for details.
And in the access(5) manual page:
Depending on the application, that string is an entire client
hostname, an entire client IP address, or an entire mail address.
So, which string is checked when a pcre table is used with
check_client_access? The client hostname or the client IP address?
Wietse Venema
> On 2010-11-04 19:06:57 -0500, Stan Hoeppner wrote:
> > check_client_access pcre:/etc/postfix/filter.pcre
> > check_sender_access pcre:/etc/postfix/filter.pcre
> > check_recipient_access pcre:/etc/postfix/filter.pcre
> >
> > As you can see, this is defined by the smtpd_foo_restriction you target
> > the PCRE table with. What is checked against the table is dependent on
> > the restriction used. Read the documentation for each check_*_access
> > restriction above at: http://www.postfix.org/postconf.5.html
>
> On this page, it is said:
>
> check_client_access type:table
> Search the specified access database for the client hostname,
> ^^^^^^^^^^^^^^^
> parent domains, client IP address, or networks obtained by
> ^^^^^^^^^^^^^^^^^
> stripping least significant octets. See the access(5) manual page
> for details.
>
> And in the access(5) manual page:
>
> Depending on the application, that string is an entire client
> hostname, an entire client IP address, or an entire mail address.
>
> So, which string is checked when a pcre table is used with
> check_client_access? The client hostname or the client IP address?
check_client_access searches the address and domain with ALL lookup
table types. It just doesn't do the substring lookups with PCRE,
REGEXP and CIDR.
Wietse
Jeroen Geilman
On 2010-11-04 19:06:57 -0500, Stan Hoeppner wrote:check_client_access pcre:/etc/postfix/filter.pcre check_sender_access pcre:/etc/postfix/filter.pcre check_recipient_access pcre:/etc/postfix/filter.pcre As you can see, this is defined by the smtpd_foo_restriction you target the PCRE table with. What is checked against the table is dependent on the restriction used. Read the documentation for each check_*_access restriction above at: http://www.postfix.org/postconf.5.htmlOn this page, it is said: check_client_access type:table Search the specified access database for the client hostname, ^^^^^^^^^^^^^^^ parent domains, client IP address, or networks obtained by ^^^^^^^^^^^^^^^^^ stripping least significant octets. See the access(5) manual page for details. And in the access(5) manual page: Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. So, which string is checked when a pcre table is used with check_client_access? The client hostname or the client IP address?
REGULAR EXPRESSION TABLES
This section describes how the table lookups change when
the table is given in the form of regular expressions. For
a description of regular expression lookup table syntax,
see regexp_table(5) or pcre_table(5).
Each pattern is a regular expression that is applied to
the entire string being looked up. Depending on the appli-
cation, that string is an entire client hostname, an
entire client IP address, or an entire mail address. Thus,
no parent domain or parent network search is done,
user@domain mail addresses are not broken up into their
user@ and domain constituent parts, nor is user+foo broken
up into user and foo.
Patterns are applied in the order as specified in the ta-
ble, until a pattern is found that matches the search
string.
Actions are the same as with indexed file lookups, with
the additional feature that parenthesized substrings from
the pattern can be interpolated as $1, $2 and so on.
I copied the entire section detailing PCRE access matches for you, since you seem unable to find it.
How many domain names look like IP addresses to you ?
If check_client_access matches against both IPs and hostnames, then your regex table will match against both IPs and hostnames.
Also read http://www.postfix.org/pcre_table.5.html for more detail on PCRE maps.
-- J.
Vincent Lefevre
> check_client_access searches the address and domain with ALL lookup
> table types. It just doesn't do the substring lookups with PCRE,
> REGEXP and CIDR.
If I understand correctly, there's another difference: in the default
table format, the string to be checked depends on the pattern form
(e.g. hostname for domain.tld, IP address for net.work.addr.ess), but
for pcre, both strings are checked against all patterns?
So, with pcre, if I want to check whether the IP address starts with
1.2.3, I need something like:
/^1\.2\.3\.[0-9]+$/
because /^1\.2\.3\./ could also match hostnames (I've noticed in my
mail archives that hostnames of this form occur in practice).
Vincent Lefevre
> *REGULAR EXPRESSION TABLES*
> This section describes how the table lookups change when
> the table is given in the form of regular expressions. For
> a description of regular expression lookup table syntax,
>
> Each pattern is a regular expression that is applied to
> the entire string being looked up. Depending on the appli-
> cation, that string is an entire client hostname, an
> entire client IP address, or an entire mail address. Thus,
> no parent domain or parent network search is done,
> /user@/ and/domain/ constituent parts, nor is/user+foo/ broken
> up into/user/ and/foo/.
>
> Patterns are applied in the order as specified in the ta-
> ble, until a pattern is found that matches the search
> string.
>
> Actions are the same as with indexed file lookups, with
> the additional feature that parenthesized substrings from
>
>
> I copied the entire section detailing PCRE access matches for you,
> since you seem unable to find it.
Useless answer. If you had read my message, you would have seen that
I quoted from it.
> How many domain names look like IP addresses to you ?
>
> If check_client_access matches against both IPs and hostnames, then your
> regex table will match against both IPs and hostnames.
This is not what the documentation says:
Depending on the application, that string is an entire client
hostname, an entire client IP address, or an entire mail address.
^^
It is said "or", and "or" doesn't mean "both". Quite the opposite.
Jeroen Geilman
It bears repeating.
How many domain names look like IP addresses to you ? If check_client_access matches against both IPs and hostnames, then your regex table will match against both IPs and hostnames.This is not what the documentation says: Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. ^^ It is said "or", and "or" doesn't mean "both". Quite the opposite.
Each pattern is a regular expression that is applied to the entire string being looked up.
check_client_access type:table
- Search the specified access database for the client hostname, parent domains, client IP address, or networks obtained by stripping least significant octets. See the access(5) manual page for details.
the result is as explained.
Nowhere in the entire documentation is it mentioned that a regex table will ONLY match a domain OR an IP address.
If it's not in the manual, then it's not supported.
-- J.
Reinaldo de Carvalho
> On 2010-11-04 17:18:17 +0100, mouss wrote:
>> otherwise, you can do whatever you want with pcre:
>> /\.example\.com$/ OK
>> or with sql or ldap.
>
> For pcre, the man page is not clear. It says:
>
check_client_access type:table
Search the specified access database for the client hostname,
parent domains, client IP address, or networks obtained by stripping
least significant octets. See the access(5) manual page for details.
This means that not import the table type, each check_client_access
entry in restriction, will generate some lookups.
I'll change my tombstone words for you: "While not fully understand a
documentation, don't try to adapt this documentation to the way you
work, but rather yourself to the way the documentation works".
--
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net
"While not fully understand a software, don't try to adapt this
software to the way you work, but rather yourself to the way the
software works" (myself)
Vincent Lefevre
> If you combine
>
> Each pattern is a regular expression that is applied to the entire string being looked up.
>
>
> with
> check_client_access /type:table
> <http://www.postfix.org/DATABASE_README.html>/*
> Search the specified access database for the client hostname, parent
> domains, client IP address, or networks obtained by stripping least
> significant octets. See the access(5)
>
> the result is as explained.
It is said "or", not "and".
> Nowhere in the entire documentation is it mentioned that a regex table will
> ONLY match a domain OR an IP address.
Read again what you quoted above.
Reinaldo de Carvalho
<rein...@gmail.com> wrote:
>
> check_client_access type:table
> Search the specified access database for the client hostname,
> parent domains, client IP address, or networks obtained by stripping
>
> This means that not import the table type, each check_client_access
> entry in restriction, will generate some lookups.
>
This means that not matter the table type.
Stan Hoeppner
> On 2010-11-04 20:33:11 -0400, Wietse Venema wrote:
>> check_client_access searches the address and domain with ALL lookup
>> table types. It just doesn't do the substring lookups with PCRE,
>> REGEXP and CIDR.
>
> If I understand correctly, there's another difference: in the default
> table format, the string to be checked depends on the pattern form
> (e.g. hostname for domain.tld, IP address for net.work.addr.ess), but
> for pcre, both strings are checked against all patterns?
>
> So, with pcre, if I want to check whether the IP address starts with
> 1.2.3, I need something like:
>
> /^1\.2\.3\.[0-9]+$/
>
> because /^1\.2\.3\./ could also match hostnames (I've noticed in my
> mail archives that hostnames of this form occur in practice).
This is why you need to use "fully qualified" patterns when matching
forward/reverse hostnames. For example:
/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.dyn\.isp\.net$/ REJECT
In practice, most ISPs don't have a /8 worth of dynamically assigned
addresses, usually a /16 or less. So for a specific ISP dynamic range
it would look my like this:
/^201\.33\.[0-9]{1,3}\.[0-9]{1,3}\.dyn\.isp\.net$/ REJECT
That will match a /16 of rDNS patterns only at the ISP "isp.net"
--
Stan
Vincent Lefevre
> On Thu, Nov 4, 2010 at 10:42 PM, Reinaldo de Carvalho
> <rein...@gmail.com> wrote:
> >
> > check_client_access type:table
> > Search the specified access database for the client hostname,
> > parent domains, client IP address, or networks obtained by stripping
> > least significant octets. See the access(5) manual page for details.
> >
> > This means that not import the table type, each check_client_access
> > entry in restriction, will generate some lookups.
>
> This means that not matter the table type.
Yes, it will generate *some* lookups, but it doesn't say exactly
*which* lookups. That was precisely my question.
Reinaldo de Carvalho
> On 2010-11-04 23:06:17 -0300, Reinaldo de Carvalho wrote:
>> On Thu, Nov 4, 2010 at 10:42 PM, Reinaldo de Carvalho
>> <rein...@gmail.com> wrote:
>> >
>> > check_client_access type:table
>> > Search the specified access database for the client hostname,
>> > parent domains, client IP address, or networks obtained by stripping
>> > least significant octets. See the access(5) manual page for details.
>> >
>> > This means that not import the table type, each check_client_access
>> > entry in restriction, will generate some lookups.
>>
>> This means that not matter the table type.
>
> Yes, it will generate *some* lookups, but it doesn't say exactly
> *which* lookups. That was precisely my question.
>
- client hostname (reverse dns hostname)
- client IP address.
if smtpd_access_maps in parent_domain_matches_subdomains.
- compare client hostname without the first part at left by dot
- compare client hostname without the first and the second part at
left by dot (and recursively at the TDL)
Stan Hoeppner
> This is not what the documentation says:
>
> Depending on the application, that string is an entire client
> hostname, an entire client IP address, or an entire mail address.
_Application_ in this sentence refers to things like
smtpd_foo_restrictions.
For instance, check_client_access is going to pass both a hostname and
an IP address to your pcre table, but not an email address.
check_sender_access is going to pass only an email address to your pcre.
Please don't get frustrated. I had difficulty with some of the
documentation myself not all that long ago, and I still have problems
digesting some of the docs now and then. The Postfix documentation is
some of the most comprehensive and complete for any piece of software
I've ever worked with. However, as you've noticed, sometimes it tends
to be written in a manner that seems geared more toward a college
computer science graduate than the average Joe sysadmin, and that may be
true.
Postfix is a very powerful MTA. As a result the documentation is very
technical. Postfix isn't for everyone. I'm guessing some folks might
read two Postfix man pages and burn holes in their shoes running out to
pay $2000 for MS Exchange and a handful of CALs.
Be patient, be polite (even when it seems others aren't), and we'll help
you get where you want to go. Learning Postfix literally is a journey.
I've been using it for 5 years and I've probably only touched on 1% of
it's capabilities, probably less than that now that I think of the
length of the parameters page. :)
--
Stan
mouss
> On 2010-11-04 17:18:17 +0100, mouss wrote:
>> otherwise, you can do whatever you want with pcre:
>> /\.example\.com$/ OK
>> or with sql or ldap.
> For pcre, the man page is not clear. It says:
>
> entire client hostname, an entire client IP address, or an entire mail
> address.
>
> hostname, an entire client IP address, or an entire mail address?
>
> According to your example, the string is an entire client hostname.
> But then, this means that one cannot match IP addresses.
>
You need to read BOTH the doc of the map type AND the doc of what it is
used for (access in this case).
in short, for each map, you have multiple parameters:
- the map type
- the search context (check_client_access, check_sender_acces, ...
transport, virtual_alias_maps, ... etc)
- the list of search keys
for each combination, a "search list" is derived: for each key, sub-keys
are derived (whether this occurs and how depends on the map type &
context).
and it is this search list that you need to grasp. so here is an example.
for check_client_access, the search keys are: the hostname and the IP
(in that order).
the sub-keys depend on the map type, so let's look at a few.
we assume the hostname is lab1.lab2.lab3.example.com and the IP is 1.2.3.4
[hash/cdb/...]
- if parent_domain_matches_subdomains contains smtpd_access: here, the
search list is
S = ( lab1.lab2.lab3.example.com, lab2.lab3.example.com,
lab3.example.com ..., com, 1.2.3.4, 1.2.3, 1.2, 1 )
so postfix will search for each element of this set and stops as soon as
a match is found.
- if parent_domain_matches_subdomains does not contains smtpd_access,
then the search list becomes
S = ( lab1.lab2.lab3.example.com, .lab2.lab3.example.com,
.lab3.example.com ..., .com, 1.2.3.4, 1.2.3, 1.2, 1 )
note the leading dot before lab2, lab3, ...
[pcre/regexp]
with such maps, no subkeys are used. this means the search list is
S = { lab1.lab2.lab3.example.com, 1.2.3.4 }
[cidr]
with cidr, only the IP is meaningful, so the set becomes
S = { 1.2.3.4}
now if we were using check_helo_access, then it's as above except that
there is no IP.
and if we were about check_sender_access, then we only have one key (the
email address) but may have many sub-keys.
Vincent Lefevre
> On Thu, Nov 4, 2010 at 11:13 PM, Vincent Lefevre <vin...@vinc17.net> wrote:
> > Yes, it will generate *some* lookups, but it doesn't say exactly
> > *which* lookups. That was precisely my question.
>
> - client hostname (reverse dns hostname)
> - client IP address.
OK, and mous said in that order (but maybe that's just the current
implementation, and the user shouldn't rely on the order for the
future).
> if smtpd_access_maps in parent_domain_matches_subdomains.
> - compare client hostname without the first part at left by dot
> - compare client hostname without the first and the second part at
> left by dot (and recursively at the TDL)
but not with a regular expression table.
Vincent Lefevre
> in short, for each map, you have multiple parameters:
> - the map type
> - the search context (check_client_access, check_sender_acces, ...
> transport, virtual_alias_maps, ... etc)
> - the list of search keys
Thanks a lot for this very detailed answer. This was exactly the kind
of description I was looking for. I have only one comment:
> [hash/cdb/...]
>
> - if parent_domain_matches_subdomains contains smtpd_access: here, the
> search list is
> S = ( lab1.lab2.lab3.example.com, lab2.lab3.example.com,
> lab3.example.com ..., com, 1.2.3.4, 1.2.3, 1.2, 1 )
> so postfix will search for each element of this set and stops as soon as a
> match is found.
Testing the tld alone seems to be excluded by the access(5) man page,
which only documents "domain.tld", i.e. the pattern must contain
at least one dot. Is it an error in the man page (which could say
"domain" instead, like in Section "Email address extension") or is
it intentional?
Stan Hoeppner
> Testing the tld alone seems to be excluded by the access(5) man page,
> which only documents "domain.tld", i.e. the pattern must contain
> at least one dot. Is it an error in the man page (which could say
> "domain" instead, like in Section "Email address extension") or is
> it intentional?
If you want to block rDNS TLDs this PCRE works with check_client_access:
/^.*?(info|kr|jp|sg|qa)$/i 550 We do not accept mail from .$1 domains
You could also use this for check_sender_access, check_helo_access,
etc--it should work with any check that passes a string with .tld in it.
--
Stan
mouss
>
>> [hash/cdb/...]
>>
>> - if parent_domain_matches_subdomains contains smtpd_access: here, the
>> search list is
>> S = ( lab1.lab2.lab3.example.com, lab2.lab3.example.com,
>> lab3.example.com ..., com, 1.2.3.4, 1.2.3, 1.2, 1 )
>> so postfix will search for each element of this set and stops as soon as a
>> match is found.
> which only documents "domain.tld", i.e. the pattern must contain
> at least one dot.
No dot required. so "com" is not excluded from the search list.
otherwise, hash/cdb and the like can't be used to block mail from
*@*.invalid for example.
> Is it an error in the man page (which could say
> "domain" instead, like in Section "Email address extension") or is
> it intentional?
yes, it should be "domain" instead of "domain.tld". you can submit a
documentation patch;-p
mouss
> On 2010-11-04 23:36:04 -0300, Reinaldo de Carvalho wrote:
>> On Thu, Nov 4, 2010 at 11:13 PM, Vincent Lefevre<vin...@vinc17.net> wrote:
>>> Yes, it will generate *some* lookups, but it doesn't say exactly
>>> *which* lookups. That was precisely my question.
>> - client hostname (reverse dns hostname)
>> - client IP address.
> OK, and mous said in that order (but maybe that's just the current
> implementation, and the user shouldn't rely on the order for the
> future).
>
Given all the compatibility efforts invested in postfix (thanks to
Wietse and others), I doubt this will change.
That said, I believe it's better not to rely on, be it just for clarity.
having things like
example.com restriction_class_1
192.0.2 restriction_class_2
(with different restriction classes or results) in the same map is a
call for trouble!
also, for IP addresses, better use cidr (I mean the map type, not a
bottle from Normandy;-p).