Procmail tips page pointer
Jari Aalto+mail.procmail
Posting-Frequency: 2 times a month
Maintainer: Jari Aalto <jari....@poboxes.com>
Last-Modified: $Docid: 2002-01-01 Jari Aalto $
Announcement: "Procmail tips page: all you wanted to know about procmail"
Faq archive is at:
The Procmail documentation project is at
http://pm-doc.sourceforge.net/
Other procmail documents
Era's exellent procmail pages (including procmail faq) are at:
http://www.iki.fi/~era/procmail/links.html
http://www.iki.fi/~era/procmail/mini-faq.html
Another exellent email filtering tips and recipes (procmail)
Page by Prof. Timo Salmi at
http://www.uwasa.fi/~ts/info/proctips.html
What is Procmail?
[faq] Procmail is a mail processing utility, which can help you
filter your mail; sort incoming mail according to sender, Subject
line, length of message, keywords in the message, etc; implement an
ftp-by-mail server, and much more. Procmail is also a complete
drop-in replacement for your MDA. (If this doesn't mean anything to
you, you don't want to know.)
Procmail runs under Unix. See Infinite Ink's Mail Filtering and
Robots page for information about related utilities for various other
platforms, and competing Unix programs, too (there aren't that many
of either).
Description
The procmail tips page: a collection of procmail recipes,
instructions, howtos. The document also contains URL pointers to
the procmail mailing list and sites that fight against Internet
UBE. You will also find many other interesting subjects that
discuss about internet email: haeders and RFCs.
The tips are compiled from the procmail discussion list,
from comp.mail.misc and from the author's own experiences with
procmail. The document is actively maintained and new sections
appear every 2 or 3 weeks.
This document does not teach you basics of procmail; instead you
have to be familiar with the procmail man pages already before this
document's tips are useful to you.
If you want to have automatic notification whenever page changes,
please visit below and register this page to to your reminder list:
http://www.netmind.com/URL-minder/new/register.html
To get nicely formatted netmind messages, see procmail module
pm-janetmind.rc
Table of contents as of 2002-01-01
1.0 Document id
1.1 General
1.2 What is Procmail?
1.3 Abbreviations and thanks
1.4 Version information
1.5 Document layout and maintenance
1.5.1 Sending improvements
1.6 About presented recipes
1.7 Variables used in recipes
1.8 About "useless use of cat award"
2.0 Procmail pointers
2.1 Where is procmail developed
2.2 About procmail's Y2K compliance
2.3 Procmail resources
2.4 Procmail mode for Emacs
2.5 Procmail module library project
2.5.1 Where to get the modules
2.5.2 Terminology
2.5.3 Foreword to using modules
2.5.4 Header file modules
2.5.5 General modules
2.5.6 Low-level Date and time handling
2.5.7 Higher-level Date and time handling
2.5.8 Forwarding and account modules
2.5.9 Vacation modules
2.5.10 Message-id based modules
2.5.11 Cron modules
2.5.12 Backup modules
2.5.13 Confirmation modules
2.5.14 File Servers
2.5.15 Mime modules
2.5.16 Filtering message body or headers
2.5.17 Miscellaneus modules
2.5.18 Mailing list modules
2.6 Procmail code to filter UBE
3.0 Dry run testing
3.1 What is dry run testing
3.2 Why the From field is not okay after dry run
3.3 Getting default value of a procmail variable
4.0 Things to remember
4.1 Get the newest procmail
4.2 Csh's tilde is not supported
4.3 Be sure to write the recipe starting right
4.4 Always set SHELL
4.4.1 If system has no /bin/sh and you're forced to use csh/tcsh
4.4.2 Procmail won't work well with SHELL set to csh derivate
4.5 Check and set PATH
4.6 Keep the log on all the time
4.7 Never add a trailing slash for directories
4.8 Remember what term DELIVERED means
4.9 Beware putting comment in wrong place
4.10 Brace placement
4.11 Local lockfile usage
4.12 Global lockfile
4.13 Gee, where do I put all those ! * $ ??
4.14 Sending automatic reply, use X-loop header
4.15 Avoid extra shell layer (check command for SHELLMETAS)
4.16 Think what shell commands you use
4.17 Using absolute paths when calling a shell program
4.18 Disabling a recipe temporarily
4.19 Keep message backup, no matter what
4.20 Order of the procmail recipes
5.0 Procmail flags
5.1 The order of the flags
5.2 Flag w and recipe with |
5.3 Flag w, lockfile and recipe with |
5.4 Flag f and w together
5.5 Flags h and b
5.6 Flag h and sinking to /dev/null
5.7 Flag i and pipe flag f
5.8 Flag r
5.9 Flag c's background
5.10 Flag c before nested block forks a child
5.11 Flag c and understanding possible forking penalty
5.12 Flags before nested block
5.13 Flags aAeE tutorial
6.0 Matching and regexps (regular expressions)
6.1 Philosophy of abstraction in regexps
6.2 Matches are not case-sensitive
6.3 Procmail uses multiline matches
6.4 Headers are folded before matching
6.5 Improving Space-Tab syndrome
6.6 Handling exclamation character
6.7 Rules for generating a character class
6.8 Matching space at the end of condition
6.9 Beware leading backslash
6.10 Correct use of TO Macro
6.11 Procmail's regexp engine
6.12 Procmail and egrep differences
6.13 Understanding procmail's minimal matching (stingy vs. greedy)
6.14 Explaining \/ and ()\/
6.15 Explaining ^^ and ^
6.16 ANDing traditionally
6.17 ORing traditionally
6.18 ORing and score recipe
6.19 ORing by using De Morgan rules
7.0 Variables
7.1 Setting and unsetting variables
7.2 Variable initialisation and sh syntax
7.3 Testing variables
7.4 What does $\VAR mean?
7.5 Common pitfalls when using variables
7.6 Quoting: Using single or double quotes
7.7 Quoting: Passing values to an external program
7.8 Passing values from an external program
7.9 Incrementing a variable by a value N
7.10 Comparing values
7.11 Strings: How many characters are there in a given string?
7.12 Strings: How to strip trailing newline.
7.13 Strings: deriving the last N characters of a string.
7.14 Strings: Getting partial matches from a string.
7.15 Strings: Procmail string manipulation example
7.16 How to raise a flag if the message was filed
7.17 Dollar sign in condition lines.
7.18 Finding mysterious foo variable
7.19 Storing code to variable
7.20 Getting headers into a variable.
7.21 Converting value to lowercase
8.0 Suggestions and miscellaneous
8.1 Speeding up procmail
8.2 See the procmail installation's examples
8.3 Printing statistics of your incoming mail
8.4 Storing UBE mailboxes outside of quota
8.5 Using first 5-30 lines from the message
8.6 Using cat or echo in scripts?
8.7 How to run an extra shell command as a side effect?
8.8 Forcing "ok" return status from shell script
8.9 Make your own .procmailrc available to others
8.10 Using dates efficiently
8.11 Keep simple header log
8.12 Gzipping messages
8.13 Emergency stop for your .procmailrc
9.0 Scoring
9.1 Using scores by an example
9.2 Brief Score tutorial
9.3 Score's scope
9.4 Counting length of a string
9.5 Counting lines in a message (Adding Lines: header)
9.6 Determining if body is longer than header
9.7 Matching last Received header
9.8 How to add Content-Length header
9.9 Testing message size or number of lines
9.10 Counting commas with recursive includerc
10.0 Formail usage
10.1 Fetching fields with formail -x
10.2 Always use formail's -rt switch
10.2.1 For procmail versions prior 3.14
10.3 Using -rt and rewriting the From address
10.4 Formail -rt and Resent-From header
10.5 Quoting the message
10.6 Without quoting the message
10.7 How to include headers and body to the reply message
10.8 Adding text to the beginning of message
10.9 Adding text to the end of message
10.10 Adding text before quoted message
10.11 Adding extra headers from file
10.12 Splitting digest
10.13 Mailbox: Splitting to individual files
10.14 Mailbox: Extracting all From addresses from mailbox
10.15 Mailbox: Applying procmail recipe on whole mailbox
10.16 Mailbox: run series of commands for each mail (split mailbox)
10.17 Option -D and cache
10.18 Option -D and message-id in the body
10.19 Reducing formail calls (conditionally adding fields)
10.20 Formail -A -a options
10.21 Formail -e -s options
11.0 Saving mailing list messages
11.1 Using subroutine pm-jalist.rc to detect mailing lists
11.2 Using plus addressing foo...@address.com
11.3 Using RFC comment trick for additional information
11.4 Simple mailing list handling
11.5 Archiving according to TO
11.6 Using Return-Path to detect mailing lists
12.0 Procmail, MIME and HTML
12.1 Mime Bibliography
12.2 Mime notes
12.3 Software to deal with mime or html
12.4 Mime content type application/ms-tnef
12.5 Trapping html mime messages
12.6 Complaining about html messages
12.7 Converting HTML body to plain text
12.8 Getting rid of unwanted mime attachments (html, vcard)
12.9 Sending contents of a html page in plain text to someone
13.0 Simple recipe examples
13.1 Saving: MH folders -- numbered messages
13.2 Saving: to monthly folders
13.3 Modifying: Filtering basics
13.4 Modifying: Squeezing empty lines around message body
13.5 Modifying: shuffling headers always to same order
13.6 Service: Auto answerer to empty messages
13.7 Service: File server -- send fileas as attachments upon request
13.8 Service: Ping responder
13.9 Service: simple vacation with procmail
13.10 Service: vacation code example
13.11 Service: Auto-forwarding
13.12 Service: forward only specific messages
13.13 Service: Making digests
13.14 Kill: killing advertisement headers and footers
13.15 Kill: simple killfile recipe with procmail
13.16 Kill: duplicate messages
13.17 Kill: spam filter with simple recipes
13.18 Kill: (un)subscribe messages
13.19 Time: Once a day cron-like job
13.20 Time: Running a recipe at a given time
13.21 Time: Triggering mail and using cron
13.22 Decoding: Uudecode
13.23 Decoding: MIME
13.24 How to send commands in the message's body
13.25 Matching two words on a line, but not one
13.26 How to define personal XX macros?
13.27 How to change subject by body match
13.28 How to change Subject according to some other header
13.29 How to call program with parameters
14.0 Miscellaneous recipes
14.1 Matching valid Message-Id header
14.2 Sending two files in a message
14.3 Excessive quoting of message
14.4 Sending message to pager in chunks
14.5 Playing particular sound when message arrives
14.6 Combining multiple Original-Cc and Original-To headers
14.7 Forwarding sensitive messages in encrypted format
15.0 Procmail and PGP
15.1 Decrypt pgp messages automatically
15.2 Getkeys from keyserver
15.3 Auto grab incoming pgp keys
16.0 Includerc usage
16.1 Using: multiple rc files
16.2 Using: You can call rc file conditionally
16.3 Autoloading an rc file
16.4 Making: naming of the rc file
16.5 Making: Using namespace when saving procmail variables
16.6 Making: Public and private variables in rc file
16.7 The rules of thumb for constructing general purpose rc file
16.8 An includerc skeleton
17.0 Mailing list server
18.0 Common troubles
18.1 Procmail modes: normal, delivery, and mailfilter.
18.2 Procmail as sendmail Mlocal mail filtering device
18.3 Procmail doesn't pass 8bit characters
18.4 My ISP isn't very interested in installing procmail
18.5 My ISP has systemwide procmailrc; is this a good idea?
18.6 Procmail changes mailbox and directory permissions
18.7 Changing mbox permission during compilation to 660
18.8 The .forward file must be real file
18.9 Using .forward if procmail already is LDA
18.10 Mail should be put in the mailqueue if write fails
18.11 Qmail: how to make it work with procmail
18.12 Qmail: Procmail looks file from /var/spool/mail only
18.13 Qmail: patch to procmail 3.11pre7 to work with Maildirs
18.14 AFS: How to use Procmail when HOME is in AFS cell
18.15 Help, some idiot sent my address to 30 mailing lists
18.16 Help, Procmail beeps and prints to my console
18.17 Help, procmail dumps mail to console
18.18 Help, corrupted From_ line in mailbox
18.19 Directing user's mail to HOME instead of /var/spool/
18.20 NFS mounting /var/mail is a good way to get bad performance
18.21 I can't see the sendmail's response in LOGFILE
18.22 Compiling procmail and choosing locking scheme
18.23 Forwarding lot of mail causes heavy load
18.24 What happens to mail if MDA Procmail fails
18.25 Procmail reads entire 90Mb message into memory
18.26 Help, procmail uses occasionally huge chunk of memory
18.27 Procmail signalled out of memory in my verbose log
18.28 Variables DEFAULT and ORGMAIL
18.29 When DEFAULT cannot be mailed to
18.30 Variable DROPPRIVS
18.31 Variable HOME
18.32 Variable HOST
18.33 Variable LINEBUF
18.34 Variable LOG and LOGFILE
18.35 Variable TRAP
18.36 Variable UMASK
18.37 UMASK and permissions
18.38 Performance difference between backtick and "|" recipe
18.39 Procmail's temporary file names while writing file out
18.40 Parameter $@
18.41 Procmail variables are null terminated (detecting null string)
18.42 FROM_DAEMON TO and TO_ and case-sensitiveness
18.43 TO_ macro deciphered
18.44 TO_ macro and RFC 822
18.45 FROM_DAEMON deciphered
19.0 Technical matters
19.1 List of exit codes
19.2 List of precedence codes
19.3 Sendmail and -t
19.4 RFC822 Reply-To and formail problem with multiple recipients
19.5 Procmail and IMAP server
19.6 Machine which processes mail
19.7 Compiling procmail and MAILSPOOLHOME
20.0 Smartlist
20.1 MLM RFC
20.2 Other mailing list software
20.3 SmartList code (mailing list implementation with procmail)
20.4 Installation trouble: getparams
20.5 Accepting mail only from users in whitelist(s)
21.0 Additional procmail or MUA software
21.1 Comstat to handle multiple mailboxes
21.2 Elm and pgp support (Mutt is the successor to elm.)
21.3 MH sites
22.0 Additional procmail software for Emacs
22.1 What is Emacs
22.2 Emacs and procmail mode and Lint
22.3 Emacs and lining up backslashes
22.4 Emacs and browsing mailbox files
22.5 Emacs and live-mode.el
22.6 Emacs and font-lock.el
23.0 Procmail, Emacs and Gnus
23.1 Gnus pointers
23.2 Why use procmail with Gnus
23.3 Setting up gnus for procmail - Basics
23.4 Gnus for procmail - More gnus
23.5 Emacs and Gnus -- Fiddling with spool files
23.6 Gnus and article snippets
23.7 Emacs GNUS - POP - Procmail
24.0 RFC, Request for comments
24.1 RFCs and their jurisdiction (munged Addresses)
24.2 Comments about addresses munging
24.3 RFC and valid mail address characters
24.4 RFC and login-name@fdqn
24.5 RFCs and messages signature
24.6 RFC and using MIME in usenet newsgroups
24.6.1 in Message-id: <wklni3b...@usenet.nospam.fogey.com>]
24.7 Some RFC Pointers
25.0 Introduction to E-mail Headers
25.1 To find out more about mail (Resources)
25.2 Lecture by Alan Stebbens
25.3 Applied to received messages
25.4 Bcc lecture by Alan Stebbens
25.5 Bcc lecture by Philip Guenther
26.0 Message headers
26.1 What is correct From address syntax
26.2 What's that X-UIDL header?
26.3 What is that first From_ header?
26.4 Message-Id header
26.5 Received header
26.6 Return-Path
26.7 Errors-To
26.8 X-Subscription-Info
26.9 Reply-To header
26.10 Mail-Copies-To header
26.11 Mail-Followup-To and Reply-To-Personal headers
26.12 Content-Length header and From_ specification
26.13 Moral about CC copies in usenet
27.0 Other interesting code
27.1 Misc mail related pointers
27.2 Expire mail pointers
27.3 Usenet News related pointers
27.4 Code: Perl Extract procmail man pages from 3.11pre7.tar.gz
27.5 Code: Sh remove matching lines from file
28.0 UBE in Internet
28.1 Terms used and foreword
28.2 UBE strategies
28.2.1 4g. I asked to be "removed" - guess what? I got another U*E
28.2.2 4h. I asked to be "removed" - guess what? The message bounced
28.3 UBE and bouncing message back
28.4 UBE and "I don't mind" attitude
28.5 We need a law against UBE
29.0 Anti-UBE pointers
29.1 NoCEM, CAUCE and others
29.1.1 http://www.obtuse.com/smtpd.html
29.2 General Filtering pages (more than procmail)
29.3 Junk mail and spam
29.4 Comprehensive list of spammers
29.5 Misc pointers
29.6 Questionable UBE stop services
29.7 UBE related newsgroups or mailing lists
29.8 Software: the net abuse page
29.9 Software: adcomplain -- Perl junk mail rport
29.10 Software: Ricochet -- Perl junk mail rport
29.11 Software: yell -- perl
29.12 Software: RBL lookup tool -- C
29.13 Software: mapSoN
29.14 Software: spamgard
29.15 Software: Spam Be Gone
29.16 Software: TinyGnus - Emacs Gnus plug-in
Jari Aalto+mail.procmail
Announcement: "Anti-UBE pointers"
Availability
Faq archive is at: http://www.faqs.org/faqs/
This message is an excerpt of a bigger procmail document at
available below. The section numbered 3.0 attached to the end of
this mail is part of it.
http://pm-doc.sourceforge.net/
Terms used
[Part of this has been excerpted from the Email Abuse Faq]
._UBE_ = Unsolicited Bulk Email
._UCE_ = (subset of UBE) Unsolicited Commercial Email
_Spam_ = Spam describes a particular kind of Usenet posting (and
canned spiced ham), but is now often used to describe many kinds of
inappropriate activities, including some email-related events. It
is technically incorrect to use "spam" to describe email abuse,
although attempting to correct the practice would amount to tilting
at windmills.
_Spam_ = definition by Erik Beckjord. "Some people decide that Spam
is anything you decide you want to ban if you can't handle the
intellectual load on a list." Remember, not to be confused with
real spam, which is unwanted bulk mail.
People are nowadays seeking a cure which will stop
or handle UBE. That can be easily done with procmail (under your
control) and with sendmail (by your sysadm). In order to select the
right strategy against UBE messages, you should read this section
and then decide how you will be using your procmail to deal with it.
Foreword and recommendation
There are two highly recommended software that you should check if
you're serious about taking actions agains UBE:
o `rblcheck' which has proven to be very efficient, fast and system
load friendly for ISPs that filter mail at MTA level.
o `Ricochet' which is a Perl program that examines
the headers to find out right complaint destinations. You no longer
need to be a Email header expert to understand how the headers have
been forged.
3.0 Anti-UBE pointers
3.1 NoCEM, CAUCE and others
"NoCEM"
http://www.cm.org/
"Dougal's NoCeM-E"
http://advicom.net/~dougal/antispam/
... Dougal is sysadm for an ISP. His page has wealth of information
about Anti-SPAM Tools. You also find his mailing list for NoCeM-E.
"The Coalition Against Unsolicited Commercial Email (CAUCE)"
http://www.cauce.org/faq.html
...The Problem: Unsolicited commercial email, more commonly known as
"spam", is a growing problem on the Internet. If you've used the
Internet for any length of time, you've probably received
solicitations via email to purchase products or services.
A Solution: A group of Internet users who are fed up with spam have
formed a coalition whose purpose is to amend 47 USC 227, the
section of U.S. law that bans "junk faxing", so that it will cover
electronic mail as well.
"Teergrubing against Spam"
http://www.iks-jena.de/mitarb/lutz/usenet/teergrube.en.html
...`Teergrubing' It's German and means Tar-Pit. Once you have been
stuck you can't get out. ...slow down internet connections in order
to stop UBE abuse. Several hundred teergrubes are able to block
spamming worldwide without blocking any e-mail. How do I start: If
you are the admin of a MX host, install a teergrube.
"Obtuse smtpd for UNIX"
http://www.obtuse.com/smtpd.html
Main (configurable) features:
o deny unauthorized relay (no more relay rape!)
o permit selective relay exceptions (eg. UUCP downstream)
o regex() filtering [block those spamming dialins!]
o deny access for no MX, no PTR, etc.
o defeat % hack
o support MAPS, ORBS, DUL, IMRSS, etc RBLs plus your local RBL
o support exception list for domains for which you will accept mail
o support selective tarpit'ing on refused connections
o individually configurable rejection messages
o precedence and override ordering
o informative log summary scripts
"Lot of good articles about spam"
http://www.sun.com/sunworldonline/swol-12-1997/swol-12-spam.html
"(anti-spam Law) US Representative Chris Smith's statement on junk
e-mail"
http://www.sun.com/sunworldonline/swol-08-1997/swol-08-junkemail.html
...considerable variation in the approaches at the federal level,
and state legislation varies widely as well. Professor David Sorkin
of John Marshall Law School, who summarized and provided links to
the major spam-related lawsuits noted above, also provides status
summaries and links to state and federal legislation
"Select email court cases -- Lots of them"
http://www.jmls.edu/cyber/cases/spam.html
America Online, Inc. v. Cyber Promotions, Inc.,
Compuserve Inc. v. Cyber Promotions, Inc., etc.
"Anti-Spam Directory of Information and Resources"
http://www.ao.net/waytosuccess/nospam.html
"FREE - The Forum for Responsible and Ethical E-mail"
http://www.spamfree.org/
"Ethical Marketing Using FREE Resources"
http://www.ao.net/waytosuccess/index.html
3.2 General Filtering pages (more than procmail)
"Nancy McGough - Mail Filtering FAQ"
http://ssil.uoregon.edu/~trenton/autopage/page7547.html
http://www.ii.com/internet/faqs/launchers/mail/filtering-faq/
"Information Filtering Resources"
http://www.ee.umd.edu/medlab/filter/ Doug Oard <oa...@glue.umd.edu>
...This page lists all known internet-accessible information
filtering resources.
3.3 Junk email and spam
"Spam FAQ"
ftp://rtfm.mit.edu/pub/usenet/alt.spam/
http://www.cs.ruu.nl/wais/html/na-dir/net-abuse-faq/spam-faq.html
"The email abuse FAQ"
http://members.aol.com/emailfaq/emailfaq.html
What is UBE, UCE, EMP, MMF, MLM, Spam, it is all explained here.
"Get that spammer -- A VERY GOOD LINK"
http://kryten.eng.monash.edu.au/gspam.html
...All about Spam; traceroute, netabuse etc. Full of links and docs
"Whois"
http://www.networksolutions.com/cgi-bin/whois/whois/
"Advertising on Usenet: How To Do It, How Not To Do It"
ftp://rtfm.mit.edu/pub/usenet/advertising/
"Dealing with Junk Email"
http://www.mcs.com/~jcr/junkemaildeal.html
...What you should do (and not do) when you have been victimized by
a junk emailer. This document teaches you how to read headers in
order to trace the origin of junk email, and includes detailed
examples to show you how it is done. Headers are designed for
computers to read, not people, so they can be a little hard to
follow. Therefore, I hereby grant permission to print or
electronically save a copy of this page on your local machine for
your personal use while tracing junk email. Please check back for
updates and corrections, though.
o What Not To Do: Stuff that doesn't work
o What to do: effective techniques, including how to trace junk
email back to its source
o Stay Calm (take a deep breath...)
o Stay Mad (don't get discouraged)
o How to identify the sender and who gives them Internet access
o Who to complain to, abuse addresses, online services
o What to say and how to say it, effective complaining
"How to fight back."
http://www.oeonline.com/~edog/spamstop.html
o Look at the header of the advertising message. Find the
"Message-ID" line. (You might have to tell your e-mail program to
display this.)
o The words after the @ sign are the sender's real--not
faked--Internet Service Provider, or ISP. (Spammers often try to
disguise their address, but the Message-ID is a good clue.)
o Write a complaint to the postmaster of that ISP, similar to the
one below. (If the ISP is junkmail.com, then let
postm...@junkmail.com hear from you.)
"Practical Tools to Boycott Spam"
http://spam.abuse.net/spam/
...We have been actively engaged in fighting spam for years. Recent
events, including pending court battles, prompt us to present this
page to the public. Fight spam to keep the Internet useful for
everyone.
o Filtering mail to your personal account
o Blocking spam email for an entire site
o Blocking Usenet spam for an entire site
o Blocking IP connectivity from spam sites
o Other tools and techniques for limiting spam
o Sample Acceptable Use Policy statements for ISPs
"Spam -- stop that!"
http://com.primenet.com/spamking/buyerbeware.html
"The Campaign to stop junk email web site"
http://www.mcs.com/~jcr/junkmail.html
...we will attempt to teach victims and potential victims (that's
everyone with an email address) the most effective methods of
prevention and retribution.
"news.admin.net-abuse.* Homepage"
Timothy M. Skirvin <tski...@math.uiuc.edu>
http://www.ews.uiuc.edu/~tskirvin/home/nana/
"The automated spamhandler beta information heap."
http://www.halcyon.com/natew/
"Preventing relaying in Sendmail"
...This package adds two independent features to sendmail,
access control and relay control. They will be described here
simultaneously, but you can elect to include support for only one
of them (either one) on your mail server. Access control lets you
deny access to the server based on the senders envelope address or
his IP address. Relay control lets you decide who gets to relay
email through your server.
ftp://ftp.xyzzy.no/sendmail/access.tar.Z
"Anti-Spam Provisions in Sendmail 8.8"
http://www.sendmail.org/antispam.html
http://mail-abuse.org/
http://www.informatik.uni-kiel.de/%7Eca/email/check.html#check_rcpt
o Preventing relaying through your SMTP port
o Refuse mail from selected hosts
o Restrict mail acceptance from certain users to avoid mailbombing
[1998-06-15 PM-L walter] Somebody's starting to exploit a hole in
sendmail 8.8, where giving a HELO longer than 1024 bytes causes
buffer overflow, and all following "Received:" headers are lost. If
it's done off a relay, we have no clue who sent it. There may be a
more elegant solution, but here's a quick-n-dirty procmail filter
for this stunt...
"Blocking Email"
http://www.nepean.uws.edu.au/users/david/pe/blockmail.html
o Do you or your users, receive "junk email" (aka., "spam")
o Do you have Sendmail R8.8.5 running at your site?
o Would you like to block known "junk email" senders' addresses?
Now you can - and there's no need to patch any source code, either.
Take advantage of Sendmail's check_mail rule, to see if the
sender's address is a member of a nominated "class" - drawn from
the contents of the named file. Additional information and links:
o Prospective Addresses/Domains to Block
o Limiting Unsolicited Commercial Email
o EFF "Net Abuse and Spamming" Archive
o [U.S.] Court Lets AOL Block Email
o Anti-Spam HOWTO
o Net Abuse FAQ
o Figuring out Fake Email & Posts
o Fight Unwanted Email
o Unsolicited Junk Email - Bad for Business
o Fight Unsolicited Email and Mailing
o Yahoo's Junk Email Resources
o jmfilter
o Complaints Addresses at U.S. ISPs
o news.admin.net-abuse.* Homepage
o Processing Mail With ProcMail
o Panix's rc.shared ProcMail Configuration
o ProcMail Workshop
o Email Self Defence
o The SPAM-L mailing list
"Preventing relaying in Netscape Messaging Server"
http://www.tsc.com/~bobp/nms-no-relay.html
...discusses anti-spam configurations for Netscape
Messaging Server (NMS). These include proper anti-relay config,
spam filters, and using blacklists such as MAPS from NMS. I was
compelled to compile this page because of the extremely poor
Netscape documentation which includes anti-relay configurations
that are easily defeated. --Bob Poortinga <bo...@tsc.com>
"US Federal Trade Commission"
http://www.ftc.gov/
...staff publicized the Commission's UCE mailbox, "u...@ftc.gov,"
and invited consumers to forward their UCE to it. spam complaints
<u...@ftc.gov>
"Spam Spade Web based tracking tool"
http://www.blighty.com/
...Figuring out forged headers and verifying IP addresses and
whois information.
"Misc"
http://www.junkbusters.com/
http://www.well.com/~jbremson/spam
http://www.wolfenet.com/~jhardin/procmail-security.html
3.4 Comprehensive list of spammers
"Against Spam -- The garbage collecting."
http://www.spam-archive.org/
To support this archive please forward email spam to
<spam...@toby.han.de>. Everybody is invited to bounce Mail-Spam
he/she has got to this list. This is a mailing list to distribute
actual spam-eMail. All incoming mail will be checked by subject and
from/sender-address wether it has already been distributed or not.
No discussions in this list. To discuss about this list please
subscribe to <spam-...@hiss.org>.
To subscribe to _blacklist-update_ mailing list
TO: <Majo...@hiss.han.de>
BODY: subscribe blacklist-update y...@somewhere.com
Mail <postm...@spam-archive.org> to discuss about blacklist if
your name is on it. (maintained by Axel Zinser <fi...@sis.han.de>)
Get the updated blacklist from
ftp://ftp.spam-archive.org/spam/blacklist/
3.5 Misc pointers
Is there a way to block local users from spamming other sites?
Maybe somehow force sentmail to read a rc file that would maybe
then grab the from field and see if the user exists on the system
or not. Or run it through some sort of filters.
[philip] You can and should do this purely in sendmail. I ended up
crafting a check_from ruleset that verifies that the envelope
sender address is either a) not local; b) a local user; or c) a
local alias. At the time I did this mainly to force people to
configure their Eudora clients so they didn't say "Return Address:
your...@gac.edu" but it also covers the outgoing bogus source
address spam case. For those interested in this kinda thing I've
(just) put it up for FTP:
ftp://ftp.gac.edu/pub/guenther/
"IBM's Secure Mailer -- open source"
http://www.postfix.org/
[1998-12-15 PM-L Matthew McGehrin <mat...@reverse.net>] The
official project is known as 'IBM's Secure Mailer'. The
unofficial codename was Vmailer, but they had to rename that, to
Postfix to agree with the lawyers. I should know, I have been
alpha testing this mailer for the past year, and it so blazing
fast, its amazing. It's faster and simplier to use than sendmail,
and also faster and more secure than qmail. It works fine with
procmail. (look in my headers). set
"mailbox_command=/usr/bin/procmail" in /etc/postfix/main.cf
[1998-12-15 PM-L Liviu Daia <da...@stoilow.imar.ro>] it has
explicit hooks for both procmail and RBL. In fact it's incredibly
easy to setup, I got it compiled and configured (with an actually
usable configuration) in about 15 minutes after downloading it.
Adding masquerading and a virtual domain took another 2 minutes.
:-) You should really give it a try, it's faster than QMail and
_much_ faster than sendmail. So far, I'm quite impressed.
"Qmail"
http://pobox.com/~djb/qmail.html
http://www.qmail.org/
"Sendmail"
http://www.sendmail.org/
"Fetchmail -- old pop3 replacement"
ftp://ftp.ccil.org/pub/esr/
http://www.ccil.org/~esr/
http://www.tuxedo.org/~esr/fetchmail/
"Maildrop filter utility"
http://www.geocities.com/SiliconValley/Peaks/5799/maildrop.README.html
...Alternative to procmail
"Lua"
http://www.tecgraf.puc-rio.br/lua/ <l...@tecgraf.puc-rio.br>
[possible replacement for procmail language] ... *Lua* is a
programming language originally designed for extending
applications, but also frequently used as a general-purpose,
stand-alone language. Lua combines simple procedural syntax
(similar to Pascal) with powerful data description constructs based
on associative arrays and extensible semantics. Lua is dynamically
typed, interpreted from bytecodes, and has automatic memory
management with garbage collection, making it ideal for
configuration, scripting, and rapid prototyping.
3.6 Questionable UBE stop services
"IEMMC: Internet E-Mail Marketing Council Formed 1997-03"
The IEMMC was formed to provide an industry wide trade association
for the purpose of promoting responsible e-mail marketing, and to
establish an industry standard code of procedures and ethics which
will internally regulate and govern the commercial e-mail marketing
industry....Under this system, all e-mail of a commercial,
unsolicited nature must pass through a universal filtration system
which will block the sending of any and all commercial e-mail to the
address on the list. Bulk e-mailers will be required to join the
organization
Others have commented that:
...IEMMC is a joke. you are probably not doing yourself any favors
...Don't take that IEMMC seriously! Many people registered with
them and got as many or even more spam as before. After all,
Cyberpromo (the operator of IEMMC) knows that the registered
addresses will be valid for some time, so they can use and sell
this valuable list to other junk mailers.
"Spammer blacklist"
http://www.netchem.com
...<rem...@netchem.com> Dear Sir/Madam, Your email address may be on
many spammers' lists. We are compiling a *remove* list. Forward the
original junk to <li...@netchem.com>
"No Junk E-Mail database"
http://pages.ripco.com:8080/~glr/nojunk.html
...We will help stop unwanted email to you..the list is submitted to
us, and those addresses that appear in the "do not mail" list are
removed and the "cleaned" list is returned
3.7 UBE related newsgroups or mailing lists
alt.kill.spammers
alt.hackers.malicous
alt.2600
[1997-08-13 alt.privacy.anon-server by anonymous poster] Proper
etiquette demands you contact their ISP. However, if the ISP are
not interested in helping you, you should consider a posting in
alt.kill.spammers (or even alt.hackers.malicous or alt.2600) - give
as many details as you can about the spammer.
A certain spam-provider targeted the alt.hackers.malicious
newsgroup. Not the most sensible thing to do. The ISPs IPs were
found, their MX host was hacked. All their DNS entries was
published on alt.2600 (so that everyone could add filters to ignore
all mail from this company). Oh yeah, their password file also made
it to the group! The ISP then posted a complaint to alt.2600, much
to the enjoyment of everyone who took part. That host basically
died a horrible death. I'm pretty sure that not many people are
going to lose any sleep over this! I might as well mention that the
ISP's complaint mentioned that their "freedom" was being
abused. hehehe. Most of these postings can be seen in dejanews
or altavista archives of usenet.
"SPAM-L mailing list and Doug Muth's Page"
http://www.claws-and-paws.com/spam-l/
... "The SPAM-L FAQ" - A FAQ for SPAM-L, an anti-spam mailing list.
This FAQ discusses how to join the list and what to post there, AND
it also delves into the technical aspects of spam. For instance,
the various kinds of forgeries seen in spams are discussed here,
along with information on how to recognise them. If you hate spam,
this is something worth checking out... "TheGoodsites List" - I
maintain this list, which is part of the Spam Boycott, to show
which Internet providers out there act responsibly when dealing
with spam. If you're looking for an ISP and want to know where they
stand on spam, this is the list for you.
Send an email message to <list...@peach.ease.lsoft.com>
with the words "subscribe SPAM-L <First name> <Last name>" in the
body of the message (no quotes). f you would like to contact the
owner, the convention is the same as with all listserv lists. Just
send e-mail to <spam-l-...@peach.ease.lsoft.com>
3.8 Software: the net abuse page
Scott Hazen Mueller <sc...@zorch.sf-bay.org>
http://spam.abuse.net/spam/tools/
3.9 Software: adcomplain -- Perl junk email rport
<bil...@agora.rdrop.com>
http://www.rdrop.com/users/billmc/adcomplain.html
Adcomplain runs under Unix, Windows-NT, and Windows-95. Adcomplain
is a tool for reporting inappropriate commercial e-mail and usenet
postings, as well as chain letters and "make money fast" postings.
It automatically analyzes the message, composes an abuse report,
and mails the report to the offender's internet service provider.
The report is displayed for your approval prior to mailing.
Adcomplain can be invoked from the command line or automatically
from many news and mail readers.
#todo: url missing
[a user happy user reports] ...About 95% of all cases can be
traced correctly --- unless they come from a known spamhouse;
where complaining to them would not do much good anyway. Mailing
lists with strange Received-Headers also can present problems in
tracing
3.10 Software: Ricochet -- Perl junk email rport
http://www.vipul.net/ricochet/
<rico...@vipul.net> Vipul Ved Prakash
MailingList: <ricochet-ann...@vipul.net> with subject
"subscribe"
A lot of unsolicited email goes unreported because tracing the
origins of a possibly forged mail and finding the right people to
report to is complicated and time-consuming. Ricochet, a smart net
agent, automates this process. It traces the names and add resses
of the systems where the spam originated from along with the
servers that provide domain name resolution services to these
systems (in most cases their ISPs). Then it collects/generates a
list of email addresses of tech/billing/admin/abuse contacts of
these system and mails them a complaint and a copy of the spam.
Detailed description of its workings can be found in the README
file that comes with the package.
3.11 Software: yell -- perl
ftp://ftp.netcom.com/pub/bo/bobmacd/yell (57k)
Bob MacDowell <bobmacd+cm...@netcom.com>
yell - auto-responds to "spam" e-mail. Scans for site names, e-mail
addresses and Web site names and sends appropriate messages to
users, postmasters and Webmasters.
3.12 Software: ifile - Perl
http://www.cs.cmu.edu/~jr6b/ifile/
Jason Daniel Rennie <jr...@andrew.cmu.edu>
...ifile is different from other mail filtering programs in
three major ways: 1) ifile does not require you to generate a set
of rules in order to successfully filter mail 2) ifile uses the
entire content of messages for filtering purposes 3) ifile learns
as you move incorrectly filtered messages to new mailboxes ifile is
not dependent upon any specific mail system and should be adaptable
to any mail system which allows an outside program to perform mail
filtering. Currently, ifile has been adapted to the MH and EXMH
mail systems.
3.13 Software: RBL lookup tool -- C
[1997-12-04 PM-L Edward S. Marshall <emar...@logic.net>]
...rblcheck is a lightweight C program for doing checks against
Paul Vixie's Blackhole List. It works well in conjunction with
Procmail for filtering unwanted bulk email (under QMail, for
example, you can invoke it with the value of the environment
variable TCPREMOTEIP). rblcheck is extremely simple:
% rblcheck 1.2.3.4
where 1.2.3.4 is the IP address you want to check.
This is a quick note to announce the availability of a new tool for
using Paul Vixie's RBL blacklist (see http://mail-abuse.org/ for
more information about the blacklist itself, if you don't already
know). Most tools which use the blacklist block email on a
site-wide basis. For many networks, this treads on both the ideals
of the administration, and on the perceived freedoms of the end
user.
Personally, I don't care either way. :-)
This tool was to fill the need I personally had to reject mail,
since one of the systems I receive mail through cannot, for various
political reasons, implement the available RBL filters on a
site-wide basis.
rblcheck is a simple tool meant to be used from procmail and
other personal filtering systems under UNIX in the absence of a
site-wide filter, as an alternative to imposing site-wide
restrictions, or as a means of imposing restrictions on systems
that cannot support the existing RBL filter patches.
Simply put: you hand it an IP address, and it determines if the IP
is in the RBL filter, providing the caller with a positive or
negative response. With the package, a sample procmail recipe is
provided, and examples of using it under QMail and Sendmail are
given.
.http://mail-abuse.org/
.http://www.isc.org/bind.html The official home page
.http://www.xnet.com/~emarshal/rblcheck/
It has only been tested under Linux 2.x and Solaris 2.5.1. Success
stories, patches, questions, suggestions, and flames can be
directed to me at <emar...@logic.net>.
[PM-L Aaron Schrab <aaron+p...@schrab.com>] Here is my rbl
setup, but, this depends both upon the format of the Received:
lines, and the way that mail passes through your mail system.
I currently grab the IP address from the first Received: header
inserted by my ISP (I'm a sysadmin at the ISP, so I have a good
knowledge of how mail gets passed around internally). Here's the
recipe that I use.
# if there's a Received: header from one of these servers, it's
# (probably) the right one
BACKUPSERVER = "([yz]\.mx\.execpc\.com)"
VIRTSERVER = "(vm[0-9]+\.mx\.execpc\.com)"
LOCALSERVER = "([abc]\.mx\.execpc\.com)"
# Match a header containing:
# Received: <anything> [<ip address>]) by <local server>
:0
* $ $SUPREME^0 ^Received:.*\[\/[0-9.]+\]\)$s+by$s+${BACKUPSERVER}
* $ $SUPREME^0 ^Received:.*\[\/[0-9.]+\]\)$s+by$s+${VIRTSERVER}
* $ $SUPREME^0 ^Received:.*\[\/[0-9.]+\]\)$s+by$s+${LOCALSERVER}
{
IP = $MATCH
# trim it down to just the IP address
:0
* IP ?? ^^\/[0-9.]+
{
IP = $MATCH
:0 W
* ! ? /home/aarons/bin/rblcheck -q $IP
{
SPAM = "$SPAM $IP is rbl'd$NL"
}
}
}
It seems to be a procmail issue with letting the IP info
from sendmail pass through to the rblcheck program. I have not
been able to find anyone using rblcheck successfully with
procmail as a delivery agent...
[1998-03-26 PM-L Edward S. Marshall <emar...@logic.net> ] This is a
standard problem; you should be able to change the invocation of
procmail the same way as the example (run env, which in turn runs
procmail). Make sure that there is a '-p' argument passed to
procmail; this preserves the environment you're constructing with
env (newer sendmail revisions sanitize the environment for you, so
that's not really an issue).
If you're still having troubles, make sure you're using the latest
incarnation of rblcheck, with the latest supplied procmail recipe;
earlier revisions had rather insidious bugs.
[1998-03-26 PM-L Xavier Beaudouin (kiwi) <ki...@oav.net>] Also it
seems that sendmail 8.9.0Beta3 has builtin rules. I use it with
sendmail 8.8.8 and tcpwrapper every day and there is about 80%
spam rejected. Sounds very good. In your /etc/hosts.allow just add
the following lines :
sendmail: ALL: spawn /usr/local/bin/rblcheck -q %a && \
exec /usr/sbin/sendmail -bs || /bin/echo \\
"469 Connection refused. You are in my Black List !!!\r\b\r\n"
&& \
(safe_finger -l @%h 2>&1 | /bin/mail -s "%d-%h %u" root)
In your /etc/inetd.conf just add this line :
smtp stream tcp nowait root /usr/sbin/tcpd \
/usr/sbin/sendmail -bs
And check that your sendmail is _not_ working as a daemon. That's
all. Also if you have huge queue you can add a /usr/sbin/sendmail -q
in the root crontab... This should help to send some waiting
messages. I think we can use this to wait for official 8.9.0
sendmail since there is some cf/feature/rbl.m4 there.
[timothy] ...I think there's a much more efficient way to do
this: you can compile sendmail -DTCPWRAPPERS and let it run as a
daemon
3.14 Software: mapSoN
Note: You can do exactly the same as below with procmail with one
of the listed procmail modules: pm-jacookie.rc. See the code.
"mapSoN (NoSpam backwards) -- The no spam utility"
http://mapson.gmd.de/
ftp://ftp.gmd.de/gmd/mapson/
Most spam filtering tools I've seen so far are based on procmail, or
a similar tool, and use a list of keywords or addresses to drop
unwanted junk mail. While this might be nice to filter mail from
known spam domains like "cyberpromo.com", it won't catch faked
headers.
mapSoN must be installed as filter program for your incoming mail,
usually by adding an appropriate entry to your $HOME/.forward file.
This means that mapSoN will get all your incoming mail and it will
decide whether or not to actually deliver it to your mailbox.
o First of all, an user defined ruleset is checked against the
mail. If any keywords or patterns match, the mail will be dealt
with according to your wishes. This is useful to drop some
sender's mail completely, or to sort mail into different mail
folders.
o If no rule matches the mail, mapSoN will check whether the mail
is a reply to an e-mail you sent, or whether it is a reply to a
USENET posting of yours. If it is, the mail will always be
delivered.
o If no signs of a reply-mail can be found, mapSoN will check
whether the sender stated in the From: header has sent you mail
before. If he has, the mail will pass. If this is the first time
you receive an e-mail from this address, though, mapSoN will
delay the delivery of the mail and spool it in your home
directory. Then it will send a short notice to the address the
mail comes from, which may look like this:
From: Peter Simons <sim...@petidomo.com>
To: never_...@me.before
Subject: [mapSoN] Request for Confirmation
mapSoN-Confirm-Cookie: <some_weird_cryptographic_cookie>
The person who tried to contact you will then reply to this
"request for confirmation", citing the cookie stated in the mail.
When your mapSoN receives this confirmation mail, it will deliver
the spooled mail into your folder. Furthermore, the address will be
added to the database, so that mail from this person will pass
directly in future.
If no confirmation mail arrives within a certain time, mapSoN can
either delete the spooled mails, or send them to a special folder,
or whatever you prefer.
3.15 Software: spamgard
[similar to MapSon]
ftp://ftp.netcom.com/pub/wj/wje/release/sg-howto
...sppamgard(tm) screens from your e-mail unsolicited bulk mail. It
does this in a way that you only have to change things if you have
a new person from whom you _do_ want to receive mail; you don't
have to change things every time a spamster thinks of a new trick
to pull, or a new spamster comes along. And spamgard(tm) is
designed so that those who aren't in your "Good Guys" list can get
mail to you anyway until you put them there. The instructions for
them to get mail to you are simple and newbie-tested, but will
still keep out bulk mail. If you're on a mailing list you _want_ to
be on, there are provisions for accepting all mail from a set of
mailing lists that you specify.
3.16 Software: Spam Be Gone
"Spam Be Gone"
http://www.internz.com/SpamBeGone/
...uses machine learning and artificial intelligence technologies
to examine incoming mail messages and determine their
priority... is more than just a Spam filter, it's a general purpose
mail message prioritiser. You train the system, telling it which
are good, and which are bad messages. As Spam Be Gone! learns it
becomes customised for each individual user.
PM-L W. Wesley Groleau <wwg...@sparc01.fw.hac.com> comments:
.> They only distribute binaries, and I'm paranoid. Anyone able to
.> convince me it's not really a Trojan Horse to collect addresses of
.> spam-haters or something even worse?
I did some sleuthing. I am 95% convinced that SpamBeGone is not
a front or cover for any spammer(s). To protect the author's
privacy, I won't say why I'm convinced or how I got the info.
Sorry. If you're paranoid like me, you'll have to do your own
sleuthing before you use it.
I'm also convinced SpamBeGone's theory is sound. I won't judge
the implementation until I've used it for a while.
PM-L R Lindberg & E Winnie <rlin...@kendaco.telebyte.com> comments:
I have to agree with the recent comments about Spam Be Gone, I
found it tends to be inaccurate. I first set it up about a week
ago, followed the directions and trained it on several (15 to 20)
messages. One from each list we get, and the remainder from my
logs of SPAM messages.
The first day it missed about half the SPAM, and nailed about 1/3
of the real messages. So I tuned the key-words a bit, trained it
on about 100 more SPAMs and trained it on all the good messages
it nailed. Since then it has nailed every SPAM received, however
the second day it nailed about 20% of the good messages, which I
then trained it to like. Since then it has been nailing about
10% of the good messages, despite continual training. I also
added every list to the address book, and it still nails posts
from this list, and my wife's lace list.
I even went through my entire log of SPAM and trained it on every
one that didn't come out a 5 (bad). Being the kind of person I
am, I also checked after I trained it, and found four SPAMs, the
despite my training it that they were bad (5) came out as not so
bad (4). I don't dare kill 4's as far too much of my mail (like
this list) ends up as 4's.
For me, this program is not ready for prime time. If the comments
are correct that it only learns on Subject and From headers, it's
not even worth trying. Since lists use the TO and CC headers to
be identified, and there are several excellent other headers
(X-Advertisement comes to mind) that would be assests for killing
SPAM.
3.17 Software: ClearMail
http://www.clearmail.com/ 1998-08-27
Scott R Carter <sca...@world.std.com>
ClearMail offers individuals some very strong control over spam
through a quite unique concept. The software includes Procmail,
Perl and C code. System Requirements include:
ClearMail helps to control spam by allowing a user to classify
e-mail as high or low priority based on an Address Book or "White
List" of known senders. Unknown senders can also send high priority
mail by including a special Mail Key (token) in their message
(initial message from unknown sender without valid Key results in a
bounceback message with instructions).
What makes ClearMail different from similar concepts is that
spammers are not able to easily obtain the Mail Key to bypass the `
system because it is conveyed as an image.
o Unix operating system
o Shell accounts for users
o Individual .forward, .procmailrc files
o Sendmail
o Procmail
o Perl
o Public Web server
3.18 Software: TinyGnus - Emacs Gnus plug-in
http://tiny-tools.sourceforge.net/
Platform: win32 and Unix Emacs versions.
*TinyGnus* Is Emacs lisp extension package that integrated directly
to Gnus mail/newsreaders. It includes simple but efective UBE
fighting hotkeys that make it possible to complain bunch of UBE
messages a once. Features:
o USER MUST DECIDE WHICH IS *ube* MAIL.
o User selects messages that are ube with Gnus select commands.
o Hotkey C-c ' u examines messages' headers and runs `nslookup'
for each Received header to determine *abuse* *spam* and
*postmaster* addresses where to send the complaint.