selective greylisting?
Folkert van Heusden
I'm using sqlgrey together with postfix.
Would it be possible to do selective greylisting triggered by an
rbl-list?
e.g. something like:
smtpd_recipient_restrictions = reject_rbl_client spamcop.org check_greylist
where check_greylist is:
check_greylist = check_policy_service inet:127.0.0.1:2501
So: if a connection was made from a host blacklisted by spamcop, do
greylisting on him. That way I can also use rbl_lists that sometimes
block also false positives.
Folkert van Heusden
sqlgrey: http://sqlgrey.sourceforge.net/
--
Looking for a cheap but fast webhoster with an excellent helpdesk?
http://keetweej.vanheusden.com/redir.php?id=1001
----------------------------------------------------------------------
Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com
Julian Pawlowski (lists)
Folkert van Heusden wrote:
> Would it be possible to do selective greylisting triggered by
> an rbl-list?
Yes. Eather use the function included with sqlgrey or if you want postfix to handle it have a look to check_client_access (if you want it per smtp-client host), check_recipient_access (if you want it per recipient) and check_sender_access (if you want it per sender).
http://www.postfix.org/postconf.5.html#check_client_access
http://www.postfix.org/postconf.5.html#check_sender_access
http://www.postfix.org/postconf.5.html#check_recipient_access
I use the following:
smtpd_recipient_restrictions =
...
check_client_access mysql:$config_directory/maps/client_access.mysql
check_client_access pcre:/etc/postfix/rules/dialups.pcre
...
check_recipient_access mysql:$config_directory/maps/recipient_greylist.mysql
permit
Additionally you have to define a new restriction class:
smtpd_restriction_classes = check_greylist
check_greylist = check_policy_service inet:127.0.0.1:2501
After it you can define the corresponding entries in your lookup tables für check_*_access and forward it to check_greylist as destination.
My dialups.pcre includes the following:
----------------------------
# well-known names do not need to be greylisted (one line!)
/^(.+[._-])*(apache|bounce|bulk|delay|d?ns|external|extranet|filter|firewall|forward|gateway|gw|m?liste?s?|(bulk|dead|mass|send|[eqw])?mail(er)?|e?mail(agent|host|hub|scan(ner)?)|messagerie|mta|v?mx|out(bound)?|pop|postfix|w?proxy|rela(is|y)|serveu?r|smarthost|vsmtp|web|www)(gate|mail|mx|pool|out|server)?[0-9]*[._-]/ DUNNO
# everything with 4 or more dots/hyphens in the hostname
/(\-.+){4}$/ check_greylist
/(\..+){4}$/ check_greylist
# every host with no reverse DNS
/unknown/ check_greylist
# grelist hosts with potential dynamic IP character in reverse DNS
# stolen from http://sqlgrey.bouton.name/dyn_fqdn.regexp (one line!)
/(^|[0-9.x_-])(abo|br(e|oa)dband|cabel|(hk)?cablep?|catv|cbl|cidr|d?client2?|cust(omer)?s?|dhcp|dial?(in|up)?|d[iu]p|[asx]?dsld?|dyn(a(dsl|mic)?)?|home|in-addr|modem(cable)?|(di)?pool|pppool|ppp|ptr|rev|static|user|YahooBB[0-9]{12}|c[[:alnum:]]{6,}(\.[a-z]{3})?\.virtua|[1-9]Cust[0-9]+|AC[A-Z][0-9A-F]{5}\.ipt|pcp[0-9]{6,}pcs|S0106[[:alnum:]]{12,}\.[a-z]{2})[0-9.x_-]/ check_greylist
----------------------------
Greetz,
Julian
Folkert van Heusden
Thank you for your elaborate reply.
> > Would it be possible to do selective greylisting triggered by
> > an rbl-list?
> Yes. Eather use the function included with sqlgrey or if you want postfix to handle it have a look to check_client_access (if you want it per smtp-client host), check_recipient_access (if you want it per recipient) and check_sender_access (if you want it per sender).
> http://www.postfix.org/postconf.5.html#check_client_access
> http://www.postfix.org/postconf.5.html#check_sender_access
> http://www.postfix.org/postconf.5.html#check_recipient_access
> I use the following:
...
> Julian
Thank you, but that is not entirely what I would like to do: I would
like postfix to check an rbl-list and if the host is in that list, then
execute the greylisting-part. So ONLY do greylisting if it is in an
RBL-list.
Folkert van Heusden
--
Temperature outside: 7.500000, temperature livingroom: 18.3
Ralf Hildebrandt
> Thank you, but that is not entirely what I would like to do: I would
> like postfix to check an rbl-list and if the host is in that list, then
> execute the greylisting-part. So ONLY do greylisting if it is in an
> RBL-list.
That's only possible in the policy daemon itself, so patch the daemon
that implements the greylisting.
--
Ralf Hildebrandt (Ralf.Hil...@charite.de) pl...@charite.de
Postfix - Einrichtung, Betrieb und Wartung Tel. +49 (0)30-450 570-155
http://www.postfix-buch.com
Why you can't find your system administrators:
they have found the exit.
Noel Jones
>Thank you, but that is not entirely what I would like to
>do: I would
>like postfix to check an rbl-list and if the host is in
>that list, then
>execute the greylisting-part. So ONLY do greylisting if it
>is in an
>RBL-list.
This can only be done via a policy service. Either modify
the greylist service you are using, or use policyd-weight
to determine which clients to greylist.
--
Noel Jones
Julian Pawlowski (lists)
> [...] or use policyd-weight to
How would you do this?
Greetz,
Julian
Noel Jones
>Noel Jones wrote:
> > [...] or use policyd-weight to
> > determine which clients to greylist.
>
>How would you do this?
Configure policyd-weight to return a result of "greylist"
for suspicious clients, then define a
smtpd_restriction_classes for greylist that contains
greylist = check_policy_service type:servicename
--
Noel Jones
Folkert van Heusden
> > Thank you, but that is not entirely what I would like to do: I would
> > like postfix to check an rbl-list and if the host is in that list, then
> > execute the greylisting-part. So ONLY do greylisting if it is in an
> > RBL-list.
> That's only possible in the policy daemon itself, so patch the daemon
> that implements the greylisting.
I did some further googling and found this:
http://www.orangegroove.net/code/marbl/
which is exactly what you describe!
Seems to work...
Folkert van Heusden
--
www.vanheusden.com/multitail - multitail is tail on steroids. multiple
windows, filtering, coloring, anything you can think of
Julian Pawlowski (lists)
> Configure policyd-weight to return a result of "greylist"
> for suspicious clients, then define a
> smtpd_restriction_classes for greylist that contains greylist
> = check_policy_service type:servicename
Yes, sure. Already have this (read me posting before *g*).
I was wondering if policyd-weight now has greylist support directly
included (means it handles greylisting _additionally_ to what it already
does).
Greetz
Julian
Michael J Wise
> So ONLY do greylisting if it is in an RBL-list.
I'd suggest only NOT greylist people who are on a local whitelist.
It's a smaller set, and should have better results.
And remember, watch the logs, guyz....
Aloha mai Nai`a!
--
"Please have your Internet License http://kapu.net/~mjwise/
and Usenet Registration handy..."
Julian Pawlowski (lists)
> I did some further googling and found this:
> http://www.orangegroove.net/code/marbl/
> which is exactly what you describe!
Looks interesting. Should be included to policyd-weight I think :-)
I don't want to have too much policyd's running that work for the same
category...
Regards,
Julian
Julian Pawlowski (lists)
> And remember, watch the logs, guyz....
Permanently :-)
It's sometimes better than every thriller EVER! ;-D
Julian
Noel Jones
Not that I know of - I don't use policyd-weight.
But using the method I described, you can have
policyd-weight call some other greylist service selectively
based on what you define as suspicious, such as RBL hits,
rDNS hostname (or lack of one), HELO hostname, etc.
--
Noel Jones
Julian Pawlowski (lists)
> But using the method I described, you can have policyd-weight
> call some other greylist service selectively based on what
> you define as suspicious, such as RBL hits, rDNS hostname (or
> lack of one), HELO hostname, etc.
I didn't have a closer look to the configuration but I think that's not
possible without modifying the code itself as the configuration is not
that flexible.
Michael J Wise
> Michael J Wise wrote:
>> And remember, watch the logs, guyz....
>
> Permanently :-)
" Constant Vigilance!
> It's sometimes better than every thriller EVER! ;-D
Eh, I'll still take the Trials and Tribulations of the Boy Wizard....
Got "Exceeds Expectations" on both my WOMBATs!
:)
I have better things to do than watch logs.
But watching logs is important.
mouss
You can use defer_if_reject and then have a script to parse the logs and
add the clients to a whitelist or to a blocklist.
Noel Jones
I didn't look that closely either, but it should be pretty
simple either way - just convince policyd-weight to return
a restriction class name such as "greylist" rather than
"554 foo".
--
Noel Jones
SATOH Kiyoshi
I have the same idea, too.
In order to use greylisting and DNSBL more safely.
I wanted to do greylisting only for dynamic IP.
Ex.1
/etc/postfix/main.cf
---
smtpd_restriction_classes =
check_greylist
check_greylist = check_policy_service inet:60000
smtpd_recipient_restrictions =
…
check_client_access regexp:/etc/postfix/check_client_fqdn
---
/etc/postfix/check_client_fqdn
---
/^unknown$/ check_greylist
/^[^\.]*[0-9][^0-9\.]+[0-9]/ check_greylist
/^[^\.]*[0-9]{5}/ check_greylist
/^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]/ check_greylist
/^[^\.]*[0-9]\.[^\.]*[0-9]-[0-9]/ check_greylist
/^[^\.]*[0-9]\.[^\.]*[0-9]\.[^\.]+\..+\./ check_greylist
/^(dhcp|dialup|ppp|adsl)[^\.]*[0-9]/ check_greylist
---
About a pattern of this dynamic IP, please refer to this site.
Study Report of an Anti-spam System with a 99% Block Rate
-- TheSelective SMTP Rejection (S25R) System --
http://www.gabacho-net.jp/en/anti-spam/anti-spam-system.html
And, I wanted to use this method by DNSBL.
Ex.1 can be replaced like this.
Ex.2
/etc/postfix/main.cf
---
smtpd_recipient_restrictions =
…
check_client_access regexp:/etc/postfix/check_client_fqdn
check_policy_service inet:60000
---
/etc/postfix/check_client_fqdn
---
!/(^unknown$)|(^[^\.]*[0-9][^0-9\.]+[0-9])|(^[^\.]*[0-9]{5})|(^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z])|(^[^\.]*[0-9]\.[^\.]*[0-9]-[0-9])|(^[^\.]*[0-9]\.[^\.]*[0-9]\.[^\.]+\..+\.)|(^(dhcp|dialup|ppp|adsl)[^\.]*[0-9])/
OK
---
Ex.3 - Only dynamic IP is checked by DNSBL.
/etc/postfix/main.cf
---
smtpd_recipient_restrictions =
…
check_client_access regexp:/etc/postfix/check_client_fqdn
reject_rbl_client spamcop.org
---
--
SATOH Kiyoshi <sa...@hakuba.jp> http://d.hatena.ne.jp/stealthinu/
Robert Felber
> Hi,
>
> I'm using sqlgrey together with postfix.
> Would it be possible to do selective greylisting triggered by an
> rbl-list?
I fail to see the reason behind this. You can do selective greylisting based
on regular expression: either greylist host which have common dns names used
for dynamic clients (such as (cable|pool|dyn|\d.\d\.d\.d)\..*\..* or those
who do NOT match a regex like (smtp|mail|mx).*\..*\..*
If you want to use RBLs which list dynamic clients only then you may use for
instance policyd-weight with following configuration:
file /etc/policyd-weight.conf:
$dnsbl_checks_only = 1;
$MAXDNSBLHITS = 0;
$MAXDNSBLMSG = 'check_greylist';
@dnsbl_score = (
'dynablock.njabl.org', 3.25, 0, 'DYN_NJABL',
'otherdyn.donain.tld', 0, 0, 'OTHER_RBL'
);
In postfix' main.cf you must the configure a restriction class for
'check_greylist' which utilizes sqlgrey or whatever you are using for
greylisting.
However, this approach is very ugly and using regular expression based
greylisting sounds more appropriate.
--
Robert Felber (PGP: 896CF30B)
Munich, Germany
Ralf Hildebrandt
> > That's only possible in the policy daemon itself, so patch
> > the daemon that implements the greylisting.
>
> That doesn't make sense as we have the query to the RBL before he wants to
> use the greylist daemon.
Of course it makes sense.
> Did you mean to patch sqlgrey in an way to let sqlgrey do the RBL checks
> instead of postfix? That would be the only way maybe, yes...
Yes, that's what I said.
--
Ralf Hildebrandt (Ralf.Hil...@charite.de) pl...@charite.de
Postfix - Einrichtung, Betrieb und Wartung Tel. +49 (0)30-450 570-155
http://www.postfix-buch.com
Chaos, panic, and disorder - my work here is done.
Robert Felber
> On Wed, Nov 01, 2006 at 10:00:55PM +0100, Folkert van Heusden wrote:
> > Hi,
> >
> > I'm using sqlgrey together with postfix.
> > Would it be possible to do selective greylisting triggered by an
> > rbl-list?
>
> I fail to see the reason behind this. You can do selective greylisting based
> on regular expression: either greylist host which have common dns names used
> for dynamic clients (such as (cable|pool|dyn|\d.\d\.d\.d)\..*\..* or those
> who do NOT match a regex like (smtp|mail|mx).*\..*\..*
>
> If you want to use RBLs which list dynamic clients only then you may use for
> instance policyd-weight with following configuration:
>
> file /etc/policyd-weight.conf:
>
> $dnsbl_checks_only = 1;
> $MAXDNSBLHITS = 0;
> $MAXDNSBLMSG = 'check_greylist';
This should probably be
$MAXDNSBLMSG = 'check_greylist ';
Policyd-weight returns then the string
"check_greylist ; check http://rbls.org/?q=$ip"
Usually it returns
"550 Your MTA is listed in too many DNSBLs; check http://rbls.org/?q=$ip"
I am not certain how postfix treats garbage in case of restriction classes.
Robert Felber
> > $MAXDNSBLMSG = 'check_greylist';
> ^^^^^^^^^^^^^^^^
>
> This should probably be
> $MAXDNSBLMSG = 'check_greylist ';
>
> Policyd-weight returns then the string
> "check_greylist ; check http://rbls.org/?q=$ip"
>
> Usually it returns
> "550 Your MTA is listed in too many DNSBLs; check http://rbls.org/?q=$ip"
>
> I am not certain how postfix treats garbage in case of restriction classes.
Ok, postfix fails to accept such return message:
Nov 2 09:07:35 fpsvr1z150 postfix/smtpd[7961]: warning: access table inet:127.0.0.1:12525 has entry with lookup table: greylist ; MTA helo: foo.domain.de, MTA hostname: p54994f8b.dip.t-dialin.net[84.153.79.139] (helo/hostname mismatch)
Nov 2 09:07:35 fpsvr1z150 postfix/smtpd[7961]: warning: do not specify lookup tables inside SMTPD access maps
Nov 2 09:07:35 fpsvr1z150 postfix/smtpd[7961]: warning: define a restriction class and specify its name instead.
(postconf -n smtpd_restriction_classes
smtpd_restriction_classes =
has_our_domain_as_sender
permit_valid_relayer
check_helo
check_backscatter
check_policyd_weight
greylist
)
I'll see whether I can design it such, that restriction classes may be defined
and if so, that no explanatory garbage is being appended.
Julian Pawlowski (lists)
>
> Yes, that's what I said.
:-)
mouss
> Hi.
>
> I have the same idea, too.
> In order to use greylisting and DNSBL more safely.
> I wanted to do greylisting only for dynamic IP.
>
> Ex.1
> /etc/postfix/main.cf
> ---
> smtpd_restriction_classes =
> check_greylist
> check_greylist = check_policy_service inet:60000
> smtpd_recipient_restrictions =
> …
> check_client_access regexp:/etc/postfix/check_client_fqdn
> ---
> /etc/postfix/check_client_fqdn
> ---
> /^unknown$/ check_greylist
> /^[^\.]*[0-9][^0-9\.]+[0-9]/ check_greylist
>
> /^[^\.]*[0-9]{5}/ check_greylist
> /^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]/ check_greylist
> /^[^\.]*[0-9]\.[^\.]*[0-9]-[0-9]/ check_greylist
> /^[^\.]*[0-9]\.[^\.]*[0-9]\.[^\.]+\..+\./ check_greylist
> /^(dhcp|dialup|ppp|adsl)[^\.]*[0-9]/ check_greylist
>
you can add 'dynamic', 'pool'...
you can also include these when not at start (for instance
/\.dynamic\./), though some care is needed.
Jorey Bump
> SATOH Kiyoshi wrote:
>> /^unknown$/ check_greylist
>> /^[^\.]*[0-9][^0-9\.]+[0-9]/ check_greylist
>>
>> /^[^\.]*[0-9]{5}/ check_greylist
>> /^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]/ check_greylist
>> /^[^\.]*[0-9]\.[^\.]*[0-9]-[0-9]/ check_greylist
>> /^[^\.]*[0-9]\.[^\.]*[0-9]\.[^\.]+\..+\./ check_greylist
>> /^(dhcp|dialup|ppp|adsl)[^\.]*[0-9]/ check_greylist
>>
>
> you can add 'dynamic', 'pool'...
> you can also include these when not at start (for instance
> /\.dynamic\./), though some care is needed.
This is actually a place where the SORBS DUL would come in handy. It
casts too wide a net to use as a blacklist, but it does a lot of this
kind of categorization already (and more). It would be suitable for
flagging hosts to greylist.
mouss
yes. but having been listed for almost a year on sorbs-dul, I don't go
sorbs that easily;-p. This is why I prefer mixing both dyn-like regexps
and lists.
- if hosts matches "loose" patterns and is listed (sorbs-dul or
njabl-dul), then grey
- if host matches "strong" patterns, GL. (because lists will never
include _all_ dynamic hosts).
example of "loose" would be /dynamic/ (which may match a static host),
while /\.dynamic\./ would rarely match a static host (and since we are
only doing GL, consequences are minimized).
Victor Duchovni
> yes. but having been listed for almost a year on sorbs-dul, I don't go
> sorbs that easily;-p.
It will be interesting to see whether you will be listed by the SpamHaus
PBL when it goes live and if so whether you will be able to get unlisted...
We'll find out soon enough, not that many weeks left in Q4 of '06,
though it could take time for the PBL data content to ramp up after
initial launch.
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majo...@postfix.org?body=unsubscribe%20postfix-users>
If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Jorey Bump
> yes. but having been listed for almost a year on sorbs-dul, I don't go
> sorbs that easily;-p. This is why I prefer mixing both dyn-like regexps
> and lists.
> - if hosts matches "loose" patterns and is listed (sorbs-dul or
> njabl-dul), then grey
> - if host matches "strong" patterns, GL. (because lists will never
> include _all_ dynamic hosts).
>
> example of "loose" would be /dynamic/ (which may match a static host),
> while /\.dynamic\./ would rarely match a static host (and since we are
> only doing GL, consequences are minimized).
Because this is for greylisting, the consequences are almost eliminated.
So why maintain regex lists at all? If an RBL can be consulted without
an outright block, it's safe for legitimate hosts to be caught in a net
they can easily escape.
I've also had problems with the SORBS DUL (but was delisted quickly),
and dislike denying access due to fuzzy name matching. It reminds me of
a certain 2000 Florida presidential election... :)
mouss
> mouss wrote:
>
>> yes. but having been listed for almost a year on sorbs-dul, I don't
>> go sorbs that easily;-p. This is why I prefer mixing both dyn-like
>> regexps and lists.
>> - if hosts matches "loose" patterns and is listed (sorbs-dul or
>> njabl-dul), then grey
>> - if host matches "strong" patterns, GL. (because lists will never
>> include _all_ dynamic hosts).
>>
>> example of "loose" would be /dynamic/ (which may match a static
>> host), while /\.dynamic\./ would rarely match a static host (and
>> since we are only doing GL, consequences are minimized).
>
> Because this is for greylisting, the consequences are almost
> eliminated. So why maintain regex lists at all? If an RBL can be
> consulted without an outright block, it's safe for legitimate hosts to
> be caught in a net they can easily escape.
I'd say this is recipient dependent. for people who want "quick mail",
it is easier to tell them "seen his hostname? hope your business doesn't
depend on people who can't buy a fancy name..." rather than "At the time
of posting, he was listed on something called a DUL. I can post you
links about this....".
>
> I've also had problems with the SORBS DUL (but was delisted quickly),
> and dislike denying access due to fuzzy name matching. It reminds me
> of a certain 2000 Florida presidential election... :)
:)
SATOH Kiyoshi
On Thu, 02 Nov 2006 17:07:55 -0500
Jorey Bump <li...@joreybump.com> wrote:
Subject: Re: selective (greylisting|DNSBL|...) (Re: selective greylisting?)
The aim of my idea is to check selectively clients which seem to have a
dynamic IP address by greylisting or DNSBL. Then, hopefully false
positives will get fewer and resource load will get lower.
Of course SORBS DUL can be used to check out clients which seem to have
a dynamic IP address. However, pattern matching of client's FQDNs must
need far less resource load than referring to DNS does. Also resource
load of DNSBL will get lower.
SATOH Kiyoshi
On Thu, 02 Nov 2006 23:02:23 +0100
mouss <use...@free.fr> wrote:
Subject: Re: selective (greylisting|DNSBL|...) (Re: selective greylisting?)
> SATOH Kiyoshi wrote:
> > /^unknown$/ check_greylist
> > /^[^\.]*[0-9][^0-9\.]+[0-9]/ check_greylist
> > /^[^\.]*[0-9]{5}/ check_greylist
> > /^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]/ check_greylist
> > /^[^\.]*[0-9]\.[^\.]*[0-9]-[0-9]/ check_greylist
> > /^[^\.]*[0-9]\.[^\.]*[0-9]\.[^\.]+\..+\./ check_greylist
> > /^(dhcp|dialup|ppp|adsl)[^\.]*[0-9]/ check_greylist
>
> you can add 'dynamic', 'pool'...
> you can also include these when not at start (for instance
> /\.dynamic\./), though some care is needed.
Yes, you can add "dynamic", "pool", "xdsl" etc. to the bottom-level
hostname pattern rule. However, it is not so effective to do so because
almost all DHCP clients are checked out by the former rules.
The rule /\.dynamic\./ may increase false positives. The domains
dynamic.com and dynamic.net exist. You should qualify this rule with
some other condition if you use it.
For your information, here is a message from ASAMI Hideo, who invented
the S25R pattern rule set.
------------------------------------------------------------------------
The Selective SMTP Rejection (S25R) anti-spam system checks out SMTP
accesses from end-user's computers (mainly dynamic IP addresses). It
uses the reject_unknown_client specification and six regexps.
[Rule 0] Reverse lookup falure (including paranoid check falure).
[Rule 1] The bottom-level name (at left-end) of a reverse FQDN includes
two or more numeral strings devided by a string of any characters but
numerals.
example: 220-139-165-188.dynamic.hinet.net
regexp: /^[^\.]*[0-9][^0-9\.]+[0-9]/
[Rule 2] The bottom-level name of a reverse FQDN includes a sequence of
five or more numerals.
example: YahooBB220030220074.bbtec.net
regexp: /^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]/
[Rule 3] Excluding the top three layers, the bottom-level or second
bottom-level name of a reverse FQDN begins with a numeral.
examples: 398pkj.cm.chello.no, host.101.169.23.62.rev.coltfrance.com
regexp: /^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]/
[Rule 4] The bottom-level name of a reverse FQDN ends with a numeral,
and the second bottom-level name includes two or more numeral strings
devided by a hyphen.
example: m226.net81-66-158.noos.fr
regexp: /^[^\.]*[0-9]\.[^\.]*[0-9]-[0-9]/
[Rule 5] Both of the bottom-level and second bottom-level names of a
reverse FQDN end with a numeral, provided that the FQDN is five or more
layered.
example: m500.union01.nj.comcast.net
regexp: /^[^\.]*[0-9]\.[^\.]*[0-9]\.[^\.]+\..+\./
[Rule 6] The bottom-level name of a reverse FQDN begins with "dhcp",
"dialup", "ppp" or "adsl" and includes a numeral.
examples: dhcp0339.vpm.resnet.group.upenn.edu,
dialupM107.ptld.uswest.net, PPPbf708.tokyo-ip.dti.ne.jp,
adsl-1415.camtel.net
regexp: /^(dhcp|dialup|ppp|adsl)[^\.]*[0-9]/
This rule set can be rewritten as one regexp as follows:
/^(unknown$|[^\.]*[0-9][^0-9\.]+[0-9]|[^\.]*[0-9]{5}|([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]|[^\.]*[0-9]\.[^\.]*[0-9]-[0-9]|[^\.]*[0-9]\.[^\.]*[0-9]\.[^\.]+\..+\.|(dhcp|dialup|ppp|adsl)[^\.]*[0-9])/
I also invented a simplified regexp:
/^(unknown$|[0-9]|[^\.]*([0-9]{5}|[0-9]([^0-9\.]+|\.)[0-9]))/
SATOH Kiyoshi developed a selective greylisting system named Rgrey. It
greylists clients which match the S25R rule set. I have found that
SQLgrey has the almost same concept as Rgrey. SQLgrey uses a long
regexp:
(^|[0-9.x_-])(abo|br(e|oa)dband|cabel|(hk)?cablep?|catv|cbl|cidr|d?client2?|cust(omer)?s?|dhcp|dial?(in|up)?|d[iu]p|[asx]?dsld?|dyn(a(dsl|mic)?)?|home|in-addr|modem(cable)?|(di)?pool|ppp|ptr|rev|static|user|YahooBB[0-9]{12}|c:alnum:?{6,}(\.[a-z]{3})?\.virtua|[1-9]Cust[0-9]+|AC[A-Z][0-9A-F]{5}\.ipt|pcp[0-9]{6,}pcs|S0106:alnum:?{12,}\.[a-z]{2})[0-9.x_-]
When I investigated match rates using data of 4,423 hosts which
attempted to send abuse mail to my personal mail server in July 2006, I
found that the match rates of the S25R rule set and the simplified egexp
are better than that of the SQLgrey regexp.
SQLgrey regexp: matched 3,781 hosts (85.5%)
S25R rule set: matched 4,344 hosts (98.2%)
simplified regexp: matched 4,331 hosts (97.9%)
I recommend you to use the S25R rule set or the simplified regexp for
SQLgrey.
If you want to know detailed information about the S25R anti-spam system,
please refer to:
http://www.gabacho-net.jp/en/anti-spam/
------------------------------------------------------------------------
SATOH Kiyoshi
> [Rule 2] The bottom-level name of a reverse FQDN includes a sequence of
> five or more numerals.
> example: YahooBB220030220074.bbtec.net
> regexp: /^([^\.]+\.)?[0-9][^\.]*\.[^\.]+\..+\.[a-z]/
[Rule 2] The bottom-level name of a reverse FQDN includes a sequence of
five or more numerals.
example: YahooBB220030220074.bbtec.net
regexp: /^[^\.]*[0-9]{5}/
Jorey Bump
That may or may not be true. Pattern matching may in fact be far more
resource intensive, as it must be performed on every connection, while a
DNSBL will enjoy the advantage of cached lookups.
The reason I suggest the SORBS DUL is because it seems to already use
such pattern matching and additional techniques that they believe are
strong indicators of dynamic addresses. Note that I don't agree with all
of their methods, especially where it involves marking a host as dynamic
if it doesn't conform to their Internet Draft for Generic DNS Naming
Schemes:
<http://tools.ietf.org/wg/dnsop/draft-msullivan-dnsop-generic-naming-schemes-00.txt>
In the context you've described, however, it would be mostly harmless
and spare you from the chore of maintaining patterns.
Mark Martinec
> > > Thank you, but that is not entirely what I would like to do: I would
> > > like postfix to check an rbl-list and if the host is in that list, then
> > > execute the greylisting-part. So ONLY do greylisting if it is in an
> > > RBL-list.
> >
> > That's only possible in the policy daemon itself, so patch the daemon
> > that implements the greylisting.
>
> http://www.orangegroove.net/code/marbl/
> which is exactly what you describe!
Some time ago I suggested to Cami to incorporate remote fingerprinting
intelligence into his policy daemon. Somehow this didn't catch,
perhaps because modifying a C program is not as easy as doing it
in Perl.
The maRBL by Ian Campbell is just such a cute programming pearl
and perfect for this job, so I added a bit of code to it to let it
trigger greylisting not only on RBL hits, but also if the remote
host is running Windows (more than 95% of mail coming from Windows
is spam from zombies, so this information by itself is worth
triggering greylisting).
Patched maRBL queries a daemon running p0f-analyzer.pm, which comes
with amavisd-new package, but is a standalone service, covered by
a BSD license. If p0f-analyzer.pm is already in use with amavisd,
the marbl may query the same daemon. Queries/responses are almost
instantaneous, cost is almost nil. The hard work is done by a p0f
service, which inspects IP packets and feeds its information to
p0f-analyzer.pm, which just caches it for a couple of minutes and
answers queries from this cache. Detail in RELEASE_NOTES of
amavisd-new ( http://www.ijs.si/software/amavisd/release-notes.txt ),
search for "p0f" (it was introduced with amavisd-new-2.4.0).
Below is a patch to be applied to marbl. I hope Ian will consider
incorporating it into his code. Happy selective greylisting!
Mark
--- maRBL-1.0.pl.orig Mon Nov 6 00:48:54 2006
+++ marbl Fri Nov 10 02:32:05 2006
@@ -7,4 +7,6 @@
use Net::Server::Multiplex;
use Net::RBLClient;
+use IO::Socket::INET;
+use Time::HiRes ();
use vars qw(@ISA);
@ISA = qw(Net::Server::Multiplex);
@@ -52,4 +54,9 @@
$config{'127.0.0.1:2552'} = {
+ p0f => { # queries daemon p0f-analyzer.pl (comes with amavisd-new)
+ p0f_service => 'inet:127.0.0.1:2345', # where is p0f-analyzer.pl?
+ matching_regexp => qr{^Windows\b},
+ # matching_regexp => qr{^Windows XP},
+ },
rbls => [
'sbl-xbl.spamhaus.org',
@@ -71,4 +78,52 @@
};
+# Prepares an UDP socket and sends a query with SMTP client IP address
+# information to a daemon p0f-analyzer.pl, which is keeping a cache of
+# gathered intelligence on recent incoming TCP connections to our MTA.
+sub p0f_init($$$$$) {
+ my($self, $hostport,$timeout,$query,$nonce) = @_;
+ local($1,$2,$3);
+ $hostport =~ /^(?: inet: )? (?: \[ ([^\]]*) \] | ([^:]*) ) : ([^:]*) /six
+ or die "Bad p0f method syntax: $hostport";
+ my($host,$port) = ($1.$2, $3); my($sock);
+ $self->{net_server}->log(2, "p0f query: %s port=%s %s %s",
+ $host,$port,$query,$nonce);
+ $sock = IO::Socket::INET->new(Type=>SOCK_DGRAM, Proto=>'udp')
+ or die "Can't create INET socket: $!";
+ my($hisiaddr) = inet_aton($host) or die "p0f - unknown host: $host";
+ my($hispaddr) = scalar(sockaddr_in($port, $hisiaddr));
+ defined($sock->send("$query $nonce", 0, $hispaddr)) or die "p0f - send: $!";
+ { sock=>$sock, wait_until=>(Time::HiRes::time + $timeout),
+ query=>$query, nonce=>$nonce };
+}
+
+# Collect a response from p0f-analyzer.pl which provides best guess about
+# remote host operating system, based on passive OS fingerprinting (p0f);
+# The p0f-analyzer.pl comes with amavisd-new, but is a standalone daemon and
+# is covered by a liberal BSD license (see RELEASE_NOTES of amavisd-new for
+# details on p0f usage).
+sub p0f_collect_response($$) {
+ my($self, $obj) = @_;
+ my($timeout) = $obj->{wait_until} - Time::HiRes::time;
+ if ($timeout < 0) { $timeout = 0 };
+ my($sock) = $obj->{sock};
+ my($resp,$nfound); my($rin,$rout);
+ $rin = ''; vec($rin,fileno($sock),1) = 1;
+ while ($nfound=select($rout=$rin, undef,undef,$timeout)) {
+ my($inbuf); my($rv) = $sock->recv($inbuf,1024,0);
+ defined $rv or die "p0f - error receiving from socket: $!";
+ if ($inbuf =~ /^([^ ]*) ([^ ]*) (.*)\015\012\z/) {
+ my($r_query,$r_nonce,$r_resp) = ($1,$2,$3);
+ if ($r_query eq $obj->{query} && $r_nonce eq $obj->{nonce})
+ { $resp = $r_resp };
+ }
+ $self->{net_server}->log(2, "p0f collect: max_wait=%.3f, %.35s... => %s",
+ $timeout,$inbuf,$resp);
+ $timeout = 0;
+ }
+ defined $nfound or die "p0f - select on socket failed: $!";
+ $resp;
+}
+
# Save the ip/port the connection came in on. This gets used to decide
# which RBLs to use, and which actions to take.
@@ -122,8 +177,26 @@
my ($self, $attr) = @_;
my $hostport = $self->{addr} . ':' . $self->{port};
+ my $cl_ip = $attr->{client_address};
my $cfg = $config{$hostport};
- my $rbl = Net::RBLClient->new('lists' => $cfg->{rbls});
- $rbl->lookup($attr->{client_address});
+ my $p0f = $cfg->{p0f};
+ my $p0f_service = !ref($p0f) ? undef : $p0f->{p0f_service};
+ my $p0f_regexp = !ref($p0f) ? undef : $p0f->{matching_regexp};
+ if (defined($p0f_service) && defined($p0f_regexp) &&
+ $cl_ip ne '' && $cl_ip ne '0.0.0.0' && $cl_ip ne '::') {
+ my $nonce = int(rand(1000000000)); # not too clever, but good enough
+ my $os_fingerprint_obj = $self->p0f_init($p0f_service, 0.050,
+ $cl_ip, $nonce);
+ if (defined($os_fingerprint_obj)) {
+ my $os_fingerprint = $self->p0f_collect_response($os_fingerprint_obj);
+ # >95% of mail from remote Windows hosts is spam coming from zombized
+ # machines, so it is worth to greylist
+ return $cfg->{hit} if $os_fingerprint ne '' &&
+ $os_fingerprint =~ /$p0f_regexp/;
+ }
+ }
+
+ my $rbl = Net::RBLClient->new('lists' => $cfg->{rbls});
+ $rbl->lookup($cl_ip);
my @listed_by = $rbl->listed_by;
if (@listed_by != 0) {
@@ -158,3 +231,3 @@
}
-main;
+main();
Mark
Geert Hendrickx
> I did some further googling and found this:
> http://www.orangegroove.net/code/marbl/
> which is exactly what you describe!
> Seems to work...
This one is similar: http://www.tahina.priv.at/~cm/spam/
It greylists hosts that:
- have no reverse DNS
- have an IP or non-FQDN as HELO
- have otherwise non-resolvable HELO
- are on some RBL
- fail SPF check
It's a Perl script and uses SQLite as the backend, but it can easily be
modified to use PgSQL for example (as it uses DBI).
Geert
mouss
> On Wednesday November 1 2006 22:49, Folkert van Heusden wrote:
>
>>>> Thank you, but that is not entirely what I would like to do: I would
>>>> like postfix to check an rbl-list and if the host is in that list, then
>>>> execute the greylisting-part. So ONLY do greylisting if it is in an
>>>> RBL-list.
>>>>
>>> That's only possible in the policy daemon itself, so patch the daemon
>>> that implements the greylisting.
>>>
>> I did some further googling and found this:
>> http://www.orangegroove.net/code/marbl/
>> which is exactly what you describe!
>> Seems to work...
>>
>
> Some time ago I suggested to Cami to incorporate remote fingerprinting
> intelligence into his policy daemon. Somehow this didn't catch,
> perhaps because modifying a C program is not as easy as doing it
> in Perl.
>
Cami's policyd is "asynchronous", so needs an asynchronous resolver.
There aren't much async resolver implementations out there, and the
available ones haven't been used enough. also, people are used to their
system resolver (integration with nsswitch and the like)...
Mark Martinec
> Cami's policyd is "asynchronous", so needs an asynchronous resolver.
> There aren't much async resolver implementations out there, and the
> available ones haven't been used enough. also, people are used to their
> system resolver (integration with nsswitch and the like)...
Not sure what you are trying to say here.
p0f has nothing to do with DNS, it just passively examines packets
of normal SMTP sessions, just like tcpdump, trying to guess what
type of network stack is running on remote side.
Mark