Mail.app Junk Filter: Poison Spam?
Barbarossa
the Bayesian junk filter of Mail.app.
Recently, at least two other people have reported Mail.app
crashing when an e-mail is marked as 'Junk.' Only resetting the
Junk database solves this.
I found this extraneous text on one of my received e-mails.
I have never seen anything like it before (well, I didn't look
too hard before.)
Is this what they mean by "Filter Poison?"
<Start Quote>
dementia newton postmark bootes buttercup crossover palm
atlantes handwrite laurent togo launder cast til contagion
manipulable extrinsic correlate contractual trumpery plural
csnet draw seismology congest erlenmeyer precede lac blouse
toward coexist velours mommy embroil effeminate capitulate
bremsstrahlung latex peoria ornament replete protrude tunnel
suppression whence farm goshawk eerie purdue centric comedy
catsup soya teacup malconduct attune extremis cryptographer
hollerith elmhurst stepson syndic unanimity wrongdoing linda
vita saloonkeep congo tabernacle acs flirtation conjecture lens
inset cannot young coarse careful comprise via piccolo bindle
canfield storeroom scoot enormity cohn biddable petroglyph
aldrich spinster rutherford canon deafen ceremonial duckling
epigram heterostructure broccoli tenterhooks cent christlike
mollie adage ghost as antelope elmhurst aiken fragmentation
pillar preponderate mohawk deviate disco astronomic beck
cholinesterase gaberones pip alderman glacier intuition holly
bastion charybdis sex change i'd eddie firehouse nutmeg
brazzaville jackdaw salmonberry hathaway programming cummings
doric deerstalker reb urbanite octogenarian protest clobber ace
credulous ouzo humble caption vincent prefect preside
proprietary door pansy mascara hippy propitiate lord carob
foursome jordan chapel esteem bijection anita andromeda den mph
he'll annie gerundive blackjack rocket actual corsage garnet
daub pentagon bodyguard doberman benson gerundial riga
nantucket pyrometer divorce caviar disneyland communicant
hillbilly bore sandhill bow paraboloid emigrate anaplasmosis
freehold frontiersmen lord williams francisco perturbate
eigenfunction combustion velvet nepal trip timetable parameter
performance ripple handlebar alterate botch bloodstain casebook
wolves parimutuel ulterior antoine sleet care dressmake mcmahon
skat minimal papacy exaggerate cottrell tied avenue adapt
larval tabernacle tungstate extort menlo whistle sequential
langmuir introversion amorphous pasteboard spectrograph
humphrey swizzle russ division crocus dow irreproducible
negligee nuthatch wallow avesta magnify peridotite kudzu
exaltation crevice dodecahedra jerry missouri afghan nicosia
latitudinary hannibal parsnip leapfrog
<End Quote>
--
_____________B_a_r_b_a_r_o_s_s_a____________ ;^{>
Wayne B. Hewitt Encinitas, CA whe...@ucsd.edu
Frédérique & Hervé Sainct
filter (this process adds many "normal" words to the message, that won't
be ranked spam at first) -but my bayesian filter still does well, maybe
because it also processes headers etc. (I don't use Mail's one but
SpamSieve). Now, maybe this spoils the database anyway, by adding
ordinary words to it, but these bases are "pruned" every now and then,
and I doubt the effect will be important. SpamSieve never crashes.
Herve
--
Frédérique & Hervé Sainct, h.sa...@laposte.net
Frédérique's initial is missing in front of the above address
l'initiale de Frédérique manque devant l'adresse email ci-dessus
Gerry
Barbarossa <whe...@ucsd.edu> wrote:
> I have heard of spam being sent with text that will clog up
> the Bayesian junk filter of Mail.app.
Hmm. Where does the word "Bayesian" come from.
> Recently, at least two other people have reported Mail.app
> crashing when an e-mail is marked as 'Junk.' Only resetting the
> Junk database solves this.
I'm one of them, but the only message I received that seem to make
things go to hell was one that was blank in all fields, from, to,
subject, everything. I opened it up and inside there was nothing.
I've received this message once or twice a week for a couple of months,
but have deleted each time rather than flagging it.
Somewhere along the line in the last month, I think I might have
reflexively flagged it as junk, but my mail.app isn't crashing. *IF*
this is what I did, it was unrelated. I'll report back if that's not
the case.
> I found this extraneous text on one of my received e-mails.
> I have never seen anything like it before (well, I didn't look
> too hard before.)
>
> Is this what they mean by "Filter Poison?"
>
> <Start Quote>
Was this text clot just part of the body of the message?
--
First they gerrymander us into one-party fiefs. Then they tell us they only
care about the swing districts. Then they complain about voter apathy.
-- Gail Collins
Simon Fraser
Gerry <222...@spam.really.sucks> wrote:
> In article <whewitt-A04D86...@news3.news.adelphia.net>,
> Barbarossa <whe...@ucsd.edu> wrote:
>
> > I have heard of spam being sent with text that will clog up
> > the Bayesian junk filter of Mail.app.
>
> Hmm. Where does the word "Bayesian" come from.
It comes from Bayesian Probability, the probability theorem on which
many spam filters are based. For details on the math, see
<http://www.cis.hut.fi/harri/thesis/valpola_thesis/node12.html>
The canonical description of how to turn this into a spam filter
is in "A plan for spam" by Paul Graham:
<http://www.paulgraham.com/spam.html>
Essentially, the filter looks at the probabilities with which
words (or "tokens") occur in spam and non-spam messages, and
generates a probability that an incoming message is spam or not
based on its set of tokens. When you mark a message as spam or
not spam, the filter adds the tokens in that message to the
set of "good" or "bad" tokens.
Simon
--
Simon Fraser <mailto:sm...@smfr.org> <http://www.smfr.org/>
(Professional driver on closed road)
Barbarossa
> > I have never seen anything like it before (well, I didn't look
> > too hard before.)
> >
> > Is this what they mean by "Filter Poison?"
> >
Gerry <222...@spam.really.sucks> wrote:
> Was this text clot just part of the body of the message?
Yes, in plain text.
Frédérique & Hervé Sainct
> But what seems to happen for me (using Eudora's Baysian filter) is that
> every couple of weeks the spammers add a new list of words, or change
> their tactics somewhat. This causes a few messages to sneak through the
> filter, and reduces its accuracy to something like 94% in my case
SpamSieve: 98.2% over a little bit more than one year here :-)
But to be honest a part of my non-spam email is in a foreigh language,
which probably helps (99.99% of all spams being in english)
Chris Jones
h.sa...@laposte.net (Frédérique & Hervé Sainct) wrote:
> But to be honest a part of my non-spam email is in a foreigh language,
> which probably helps (99.99% of all spams being in english)
Really? Over half the spam I get is in Korean. Must be that
"globalization" thing I keep hearing about ;)
Chris
--
To reply by email, please remove the initial digit from my address.
Make the world a better place: shoot a spammer today!
Marc Heusser
h.sa...@laposte.net (Frédérique & Hervé Sainct) wrote:
> James Meiss <j...@NOSPAM.invalid> wrote:
>
> > But what seems to happen for me (using Eudora's Baysian filter) is that
> > every couple of weeks the spammers add a new list of words, or change
> > their tactics somewhat. This causes a few messages to sneak through the
> > filter, and reduces its accuracy to something like 94% in my case
>
> SpamSieve: 98.2% over a little bit more than one year here :-)
The newest trick from spammers is to include habeas-headers
(www.habeas.com). By default SpamSieve is set to accept these messages
without further examination. This was a good idea up to now - for the
time being just uncheck that option in Preferences and all is well again.
SpamSieve is not really bothered by all this "junk poetry".
Marc
--
Marc Heusser
(remove the obvious: CHEERS and MERICAL...until end to reply via email)
Frédérique & Hervé Sainct
wrote:
> The newest trick from spammers is to include habeas-headers
> (www.habeas.com). By default SpamSieve is set to accept these messages
> without further examination. This was a good idea up to now - for the
> time being just uncheck that option in Preferences and all is well again.
> SpamSieve is not really bothered by all this "junk poetry".
True -the author of SpamSieve warned registered users to uncheck this a
couple of day ago.
Tom Harrington
Marc Heusser <marc.h...@CHEERSheusser.comMERCIALSPAMMERS.invalid>
wrote:
> SpamSieve is not really bothered by all this "junk poetry".
Really? That's interesting to hear. I've never bothered with spam
filters beyond Mail.app's Bayesian approach. But if SpamSieve would
deal with the newer random-text spams I might have to get it.
--
Tom "Tom" Harrington
Macaroni, Automated System Maintenance for Mac OS X.
Version 1.4: Best cleanup yet, gets files other tools miss.
See http://www.atomicbird.com/
BK
> I have heard of spam being sent with text that will clog up
> the Bayesian junk filter of Mail.app.
>
> <Start Quote>
>
> dementia newton postmark bootes buttercup crossover palm
> atlantes handwrite laurent togo launder cast til contagion
[SNIP]
> <End Quote>
indeed, this is becoming more and more common and it's designed to
trick the spam filters.
However, in my experience, most of those messages aim to hide this
between bogus HTML tags so that the random text will not show up in
your mail client.
I have therefore set my SMTP server to ignore all incoming mail that
has HTML for those email accounts which are exposed to address
harvesting robots.
As a result, I only get spam once or twice a month now on those email
accounts while the SMTP server logs show that there are thousands of
messages being rejected because they contain HTML.
I have set the SMTP server to respond with an error message "HTML
email not allowed here" so if there are any false positives the sender
will be able to resend the message in plain text, which I prefer
anyway.
For me this arrangement works well because on those exposed email
addresses I can afford not to accept any HTML nor any attachments
(SMTP server also set to reject all multipart mime messages).
On my regular email addresses where I can't afford to do this I hardly
get any spam anyway because I never use them such that they would show
up on the web where the robots would see them.
Another way to fight spam, which I really like is to use a spamtrap
email address. CommuniGate Pro (CGP) has a feature by which all mail
from a sender is rejected if any TO address matches the spamtrap email
address.
All you have to do is make sure that you always quote the spamtrap
address together with your exposed address, so that robots will pick
them both up together, like this ...
m...@mydomain.com (hey folks send mail to me ONLY on this address)
SPAM...@mydomain.com (hey folks, NEVER send me any mail on this
address)
Any spam send to you will then be send to both addresses (typically in
the same SMTP session) and the mere existence of the spamtrap address
in the TO list will cause all mail from that sender to be rejected
right away. No matter how much care the spammer has taken to confuse
your spam filters, all their effort will have been in vain, because
they fell into the spamtrap ;-)
Of course you may not want to use the word "spamtrap" since if this
becomes a popular method of fighting spam and the spammers are not
getting any spam through anymore, then they will for sure filter out
all the addresses that contain the word "spamtrap". CGP allows you to
define any address you like as a spamtrap address.
I am sure this has been around for a while but I only just stumbled
into it when we recently migrated to CGP. Anyway, I really like the
idea and I believe it is going to be very difficult for spammers to
work around the trap. I am looking forward to see the results in the
server logs once the robots have picked up our trap addresses ;-)
rgds
bk
Frédérique & Hervé Sainct
even though I fear the spamtrap principle causes more mail traffic (this
must be in fact negligible) it opens a lot of perspectives.
On my side I try to convince my smtp providers to run Bayesian filtering
on their side (a very little number of them seem to announce it, but
none allows serious testing before paying).
Also, let's mention the free Tmicha service, that offers an email
hosting (and even redirection "on the fly" I think) which is filtered
with a variety of criterions including spamtraps.
Its only problem is, one of these strategies consists in barring *all*
email coming from suspect IPs, and if one of your correspondents
actually uses the same service he too will be filtered...
Herve
BK
> even though I fear the spamtrap principle causes more mail traffic (this
> must be in fact negligible) it opens a lot of perspectives.
Actually, no, it doesn't cause more mail traffic, not if used the way
I described. That's the beauty of it ;-)
This is because you are exposing the spamtrap address together with
your actual address and they both must be within the same domain or
else it won't work anyway. The fact that they are both within the same
domain means that any spam coming your SMTP servers way will now
contain both addresses in the TO field instead of just the one
address. It is the same SMTP session, there is no additional mail
sent, only an additional addressee for the same mail which you would
have been sent anyway.
> On my side I try to convince my smtp providers to run Bayesian filtering
> on their side (a very little number of them seem to announce it, but
> none allows serious testing before paying).
Why not run your own smtp server? If you have got a Mac, you've got
one built-in already, it's only lying dormant. If you have got
Panther, you get Postfix with it, which is an excellent server, secure
design, small resource footprint and easy to handle.
check out Postfix Enabler from Cutegde systems at
http://www.macupdate.com/info.php/id/13237
it's an Aqua GUI interface for - well - enabling and configuring
postfix, very easy. You'll have it running within less than 30 seconds
and it's only $9.95 shareware fee. If you rather fancy sendmail,
they've also got a Sendmail Enabler.
Anyway, once you run your own SMTP server, you can extend it with
filters and tricks as you please. OSX gives you all the tools, use
them to your advantage ;-)
rgds
bk
Frédérique & Hervé Sainct
> Actually, no, it doesn't cause more mail traffic, not if used the way
> I described. That's the beauty of it ;-)
>
> This is because you are exposing the spamtrap address together with
> your actual address and they both must be within the same domain or
> else it won't work anyway. The fact that they are both within the same
> domain means that any spam coming your SMTP servers way will now
> contain both addresses in the TO field instead of just the one
> address. It is the same SMTP session, there is no additional mail
> sent, only an additional addressee for the same mail which you would
> have been sent anyway.
I understand that if I only retrieve my "good" address and then filter
out everything containing also the spamtrap address in the "to" field I
won't increase my own traffic, but aren't there any chances that lots of
spams are sent to the spamtrap address in parallel? Even it that address
indeed does not exist, I presume there will be traffic linked to it, at
least the "recipient did not exist" messages
> Why not run your own smtp server? If you have got a Mac, you've got
> one built-in already, it's only lying dormant. If you have got
> Panther, you get Postfix with it, which is an excellent server, secure
> design, small resource footprint and easy to handle.
Even though I have a reasonable permanent connection, on some of the
oldest addresses I must keep I get by now almost one new spam every
minute: the download duration indeed is sensible, and uses bandwidth. I
am fully satisfied with my local SpamSieve bayesian filter, but I would
prefer this filtering being done earlier in the process, e. g. in my
mail provider machines, without reaching me. I can't see how running my
own SMTP could help this, but I am quite new to SMTPs, so don't be vexed
:-)
Gerry
Hervé Sainct <h.sa...@laposte.net> wrote:
My name is Gerry and I spell it with a "G" and I have no interest in
changing it to make somebody else's life easier; screw 'em. So this
may or may not interest you and if it doesn't I can sympathize.
On my newsreader, without a lot of fussing I won't do, and on many
other newsreaders as well (with or without fussing) your name as it
appears in message lists is
=?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?=
Thought you might like to know how your name appears.
Tom Harrington
Gerry <222...@spam.really.sucks> wrote:
> In article <1g7pwvw.19fssji1xp5oagN%h.sa...@laposte.net>, Frédérique &
> Hervé Sainct <h.sa...@laposte.net> wrote:
>
> My name is Gerry and I spell it with a "G" and I have no interest in
> changing it to make somebody else's life easier; screw 'em. So this
> may or may not interest you and if it doesn't I can sympathize.
>
> On my newsreader, without a lot of fussing I won't do, and on many
> other newsreaders as well (with or without fussing) your name as it
> appears in message lists is
> =?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?=
You're using an inadequate newsreader. His name looks fine from here,
and I certainly haven't "fussed" with it. :-)
James L. Ryan
From: h.sa...@laposte.net
(=?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?= =?ISO-8859-1?Q?nct?=)
I'll agree with Gerry that munged addresses are a big time annoyance. It's
like sending someone a self-addressed return envelope with the caveat that
the address on the envelope is incorrect and needs to be changed.
-- James L. Ryan -- TaliesinSoft
Tom Harrington
But that address is not munged, at least not by the sender.
James L. Ryan
(in message <tph-B741CC.21121518012004@localhost>):
> In article <0001HW.BC307ECA...@news-server.austin.rr.com>,
> James L. Ryan <talies...@mac.com> wrote:
>
>> On my newsreader, Hogwasher, Frédérique's from address appears as follows:
>>
>> From: h.sa...@laposte.net
>> (=?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?= =?ISO-8859-1?Q?nct?=)
>>
>> I'll agree with Gerry that munged addresses are a big time annoyance. It's
>> like sending someone a self-addressed return envelope with the caveat that
>> the address on the envelope is incorrect and needs to be changed.
>
> But that address is not munged, at least not by the sender.
Now I'm a bit confused! Frédérique himself states (and I'm copying directly
from his prior posting in this thread):
Frédérique's initial is missing in front of the above address
If that isn't "munging" what is?
Tom Harrington
James L. Ryan <talies...@mac.com> wrote:
> On Sun, 18 Jan 2004 22:12:15 -0600, Tom Harrington wrote
> (in message <tph-B741CC.21121518012004@localhost>):
>
> > In article <0001HW.BC307ECA...@news-server.austin.rr.com>,
> > James L. Ryan <talies...@mac.com> wrote:
> >
> >> On my newsreader, Hogwasher, Frédérique's from address appears as follows:
> >>
> >> From: h.sa...@laposte.net
> >> (=?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?= =?ISO-8859-1?Q?nct?=)
> >>
> >> I'll agree with Gerry that munged addresses are a big time annoyance. It's
> >> like sending someone a self-addressed return envelope with the caveat that
> >> the address on the envelope is incorrect and needs to be changed.
> >
> > But that address is not munged, at least not by the sender.
>
> Now I'm a bit confused! Frédérique himself states (and I'm copying directly
> from his prior posting in this thread):
>
> Frédérique's initial is missing in front of the above address
In that case I guess I'm the one who's confused. Never mind my previous
message.
> If that isn't "munging" what is?
I tend to agree with you that munging return addresses is a big-time
annoyance. For a couple of years I refused to do it. Eventually the
email address I was using at the time became so inundated with spam as
to be entirely useless. So when I switched to a new one I started
munging. I know it's inconvenient for anyone who might want to email
me, but I think it's an unfortunate necessity.
Gerry
<t...@pcisys.no.spam.dammit.net> wrote:
> > On my newsreader, without a lot of fussing I won't do, and on many
> > other newsreaders as well (with or without fussing) your name as it
> > appears in message lists is
> > =?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?=
>
> You're using an inadequate newsreader.
All newsreaders are inadequate in one way or other. We pick which
features we want. This is not an inadequacy of my newsreader.
Curiously, just the opposite.
> His name looks fine from here, and I certainly haven't "fussed" with
> it. :-)
I read the asian-language newsgroups, and English-language newsgroups.
As such, I need to state explicitly which coding each is to receive and
where or one or other will look wrong, just as it is with with your
newsreader. It seems likely that the defaults worked for you but they
do not for everyone.
I assume I can continue on into the uneding side-topic "your newsreader
stinks" further but my point was about the OP's name display not how
MT-Newswatcher doesn't suck. In some newsreaders his name is munged. I
thought he might want to know this.
Frédérique & Hervé Sainct
> On my newsreader, without a lot of fussing I won't do, and on many
> other newsreaders as well (with or without fussing) your name as it
> appears in message lists is
> =?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?=
>
> Thought you might like to know how your name appears.
Your newsreader doesn't handle accentuated characters.
I would'nt have had the idea to answer agressively, but to quote you
very exactly, my name include accentuated characters, "and I have no
interest in changing them to make somebody else life easier, etc."
Frédérique & Hervé Sainct
> Frédérique's initial is missing in front of the above address
>
> If that isn't "munging" what is?
my three years old address below, the one with an f added, gets 50 to
100 spams a day, because I used it three years ago during *two weeks* to
post in a couple of newsgroups (I never left it on a web site). Because
of this, and like exactly half of the people in this thread, I don't
print it, nowhere in my posts. There is no nickname -anybody sees my
real name. There is not even any insult to spammers (a la "spam.sucks").
> > (=?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?= =?ISO-8859-1?Q?nct?=)
This is something different: My first name contains accentuated
characters, which some barebone newsreaders may just not translate
correctly.
--
Frédérique & Hervé Sainct, h.sa...@laposte.net
Frédérique's initial is missing in front of the above address
Gerry
Hervé Sainct <h.sa...@laposte.net> wrote:
> > > (=?ISO-8859-1?Q?Fr=E9d=E9rique_=26_Herv=E9_Sai?= =?ISO-8859-1?Q?nct?=)
>
> This is something different: My first name contains accentuated
> characters, which some barebone newsreaders may just not translate
> correctly.
You're very welcome.
Benjamin Han
<t...@pcisys.no.spam.dammit.net> said:
> In article <marc.heusser-CDC1...@news.bluewin.ch>,
> Marc Heusser <marc.h...@CHEERSheusser.comMERCIALSPAMMERS.invalid>
>
> wrote:
>
> > SpamSieve is not really bothered by all this "junk poetry".
>
> Really? That's interesting to hear. I've never bothered with spam
> filters beyond Mail.app's Bayesian approach. But if SpamSieve would
> deal with the newer random-text spams I might have to get it.
Mail.app uses LSA (latent semantic analysis), a.k.a. SVD (singular
value decomposition).
Ben