Passive Spam Revocation (PSR)
Yao Ziyuan 2
Currently almost all mail systems (e.g. Hotmail and Gmail) use a spam
filter, which can drop good and important messages.
I propose an optional feature for current mail systems. The main idea
is if a message is considered spam, this spam status can be tracked by
the sender (but not sent to him directly, as the From field can be
faked). The message can be re-marked as "not spam" if the sender can
solve a CAPTCHA.
STEP 1: A is going to send B a message. A's mail client generates a
random code and puts it in a custom field in the outgoing message's
header:
Code: <random code>
STEP 2: A's mail client sends the message, waits 30 seconds, and then
visits:
https://spamstatus.<B's mail domain>/?msgid=<Message-
ID>&code=<Code>
This page displays one of these possible "spam statuses":
* MESSAGE CONSIDERED SPAM. (A CAPTCHA is also presented below.)
* MESSAGE CONSIDERED NOT SPAM.
* PENDING. PLEASE TRY AGAIN LATER.
* All other responses mean B's mail system doesn't support this
feature.
In the first case, A's mail client will report the status and the
CAPTCHA to A. A can choose to solve the CAPTCHA to prove the message
is not spam.
Like the idea? Here is the official Google group for it:
http://groups.google.com/group/passive-spam-revocation
Regards,
Yao Ziyuan
http://sites.google.com/site/yaoziyuan/
David F. Skoll
> STEP 1: A is going to send B a message. A's mail client generates a
> random code and puts it in a custom field in the outgoing message's
> header:
> Code: <random code>
> STEP 2: A's mail client sends the message, waits 30 seconds, and then
> visits:
> https://spamstatus.<B's mail domain>/?msgid=<Message-
> ID>&code=<Code>
> This page displays one of these possible "spam statuses":
> * MESSAGE CONSIDERED SPAM. (A CAPTCHA is also presented below.)
> * MESSAGE CONSIDERED NOT SPAM.
> * PENDING. PLEASE TRY AGAIN LATER.
> * All other responses mean B's mail system doesn't support this
> feature.
> In the first case, A's mail client will report the status and the
> CAPTCHA to A. A can choose to solve the CAPTCHA to prove the message
> is not spam.
It's an interesting idea. However, there are problems:
1) The sender gets to choose the code. So a spammer could choose a
code, solve the CAPTCHA once, and then use the same code and
Message-ID over and over again. Since that's the only data used by the
spamstatus server to present the status, the protocol breaks.
2) You've provided spammers with an Oracle. They can keep blasting
different variants of their message at your domain until the
spamstatus server says MESSAGE CONSIDERED NOT SPAM. Now they know
exactly how to get past your filter.
3) If a code and message ID ever leak out (eg, in a forwarded message),
a spammer can reuse them to get past the filter. This attack is similar
to (1). Alternatively, an attacker can take a look at the Oracle to see
if it considered the original message spam or not, and thereby gain
information about your filter.
IMO, providing timely feedback to spammers about how their mail fared during
mail filtering is a bad idea. It only helps them to improve their spam.
Regards,
David.
D. Stussy
news:d86f6df7-bd63-426d...@q14g2000vbi.googlegroups.com...
> Passive Spam Revocation (PSR)
>
> Currently almost all mail systems (e.g. Hotmail and Gmail) use a spam
> filter, which can drop good and important messages.
>
> I propose an optional feature for current mail systems. The main idea
> is if a message is considered spam, this spam status can be tracked by
> the sender (but not sent to him directly, as the From field can be
> faked). The message can be re-marked as "not spam" if the sender can
> solve a CAPTCHA.
Sorry. Challenge/response is already declared a BAD idea.
David F. Skoll
> Sorry. Challenge/response is already declared a BAD idea.
It's not C/R because no challenge is sent. The challenge is implicit,
so there's no backscatter problem.
I've already posted about the other weaknesses of the scheme, though.
-- David.
David F. Skoll
> Showing a message's spam status to the sender can be bad, if he is
> really a spammer. So the page can also return:
> * SPAM STATUS HIDDEN. (A CAPTCHA is also presented below.)
> This means the sender can solve the CAPTCHA to see the status and
> change it to NOT SPAM.
That's really bad. That means I have to solve a CAPTCHA for every
email I send? That just won't fly; there aren't enough hours in the
day for that. You've just invented Challenge-Response with an implicit
Challenge.
Anyway, we already have Passive Spam Revocation. It works like this:
1) I send a Very Important Email to X.
2) I wait a bit, and don't hear from X.
3) I phone X up and say "Didn't you get my email?"
4) X says "Oops, sorry. It was caught by my filter; I will release it."
My point is that for most emails, the recipient will eventually get around
to releasing false-positives from his/her filter. For Very Important Emails,
I'm willing to invest in an out-of-band solution to get the mail through
quickly.
If we could mark emails as important, and only have the system present
the CAPTCHA for that, maybe I'd go for it. But not for every single
email.
Regards,
David.
D. Stussy
news:73554$4ae5877e$d1d97a75$59...@PRIMUS.CA...
> D. Stussy wrote:
>
> > Sorry. Challenge/response is already declared a BAD idea.
>
> It's not C/R because no challenge is sent. The challenge is implicit,
> so there's no backscatter problem.
(Your subsequent post 6 minutes later says otherwise.... It IS C/R.)
> I've already posted about the other weaknesses of the scheme, though.
It doesn't matter if the "challenge" is delivered via means outside of
SMTP. It still requires a legitimate sender to take an extra step, so it's
NO DIFFERENT from C/R. How is the sender supposed to know that the extra
step of visiting the web page is required if he's not told to do it - and
thus "challenged?" What's to prevent spammers from automatically hitting
such a system with an HTTP followup? As there are now databases of
captchas, only unknown ones need be passed to the spammer to solve.
Legitimate mail should NEVER be hampered. Only malicious mail should be
hampered. C/R has proven that any model based on the premise of the sender
having to do something else besides compose and send doesn't work.
It doesn't matter if the challenge or the response is in band (SMTP) or via
another method (e.g. HTTP). You're still making legitimate senders "jump
through hoops" to have their mail delivered while doing NOTHING to spammers
who won't care.
David F. Skoll
> "David F. Skoll" <d...@roaringpenguin.com> wrote in message
> news:73554$4ae5877e$d1d97a75$59...@PRIMUS.CA...
>> It's not C/R because no challenge is sent. The challenge is implicit,
>> so there's no backscatter problem.
> (Your subsequent post 6 minutes later says otherwise.... It IS C/R.)
OK. It's not traditional C/R.
> It doesn't matter if the "challenge" is delivered via means outside
> of SMTP. It still requires a legitimate sender to take an extra
> step, so it's NO DIFFERENT from C/R.
It's different from traditional C/R in that the *most* annoying side-effect
of C/R (challenges sent to unrelated third parties) is avoided.
As you point out, it's the same in that the *second-most* annoying side
effect (extra work for legitimate senders) is still present.
> How is the sender supposed to know that the extra
> step of visiting the web page is required if he's not told to do it - and
> thus "challenged?"
Presumably, if this is widely adopted, MUAs and other software will
"know" to hit the site just as we "know" how to interpret SPF records, etc.
[...]
> Legitimate mail should NEVER be hampered.
I believe the OP's motivation was to reduce the hampering of
legitimate mail by letting the sender release it from the recipient's
spam filter. The addition of the "SPAM STATUS HIDDEN" option kills
that idea, though. You either have to provide spammers with an
Oracle, or make senders do possibly useless extra work. You can't
have it both ways.
> You're still making legitimate senders "jump
> through hoops" to have their mail delivered while doing NOTHING to
> spammers who won't care.
Read the proposal again. The CAPTCHA is only offered if the mail was
trapped in a filter. A sender can presumably ignore it and wait for
the mail to be released by the recipient in the normal course of events.
(This is assuming you don't use the SPAM STATUS HIDDEN option.)
I don't think it's a good idea. But it's not as harmful or in-your-face
as traditional C/R.
-- David.
D. Stussy
> D. Stussy wrote:
> > "David F. Skoll" <d...@roaringpenguin.com> wrote in message
> > news:73554$4ae5877e$d1d97a75$59...@PRIMUS.CA...
>
> >> It's not C/R because no challenge is sent. The challenge is implicit,
> >> so there's no backscatter problem.
>
> > (Your subsequent post 6 minutes later says otherwise.... It IS C/R.)
>
> OK. It's not traditional C/R.
It's still a variation. We know it doesn't work.
> > It doesn't matter if the "challenge" is delivered via means outside
> > of SMTP. It still requires a legitimate sender to take an extra
> > step, so it's NO DIFFERENT from C/R.
>
> It's different from traditional C/R in that the *most* annoying
side-effect
> of C/R (challenges sent to unrelated third parties) is avoided.
A C/R system placed behind a forgery filter would avoid the same thing.
> As you point out, it's the same in that the *second-most* annoying side
> effect (extra work for legitimate senders) is still present.
>
> > How is the sender supposed to know that the extra
> > step of visiting the web page is required if he's not told to do it -
and
> > thus "challenged?"
>
> Presumably, if this is widely adopted, MUAs and other software will
> "know" to hit the site just as we "know" how to interpret SPF records,
etc.
SPF hasn't penetrated like it should have. Less than 15% of all domains
(per various surveys) even have SPF records*, and even fewer bother to
check for them.
Footnote *: And even when they do, they're still using TXT-RRs, not
SPF-RRs; the latter being supported by DNS servers for the past 3 years.
> > Legitimate mail should NEVER be hampered.
>
> I believe the OP's motivation was to reduce the hampering of
> legitimate mail by letting the sender release it from the recipient's
> spam filter. The addition of the "SPAM STATUS HIDDEN" option kills
> that idea, though. You either have to provide spammers with an
> Oracle, or make senders do possibly useless extra work. You can't
> have it both ways.
It still gets caught in the first place. It's still delayed not for any
reason related to its transport and delivery.
I hadn't even considered the feedback aspect to spammers. I didn't have to
go that far to realize the concept was a bad idea.
> > You're still making legitimate senders "jump
> > through hoops" to have their mail delivered while doing NOTHING to
> > spammers who won't care.
>
> Read the proposal again. The CAPTCHA is only offered if the mail was
> trapped in a filter. A sender can presumably ignore it and wait for
> the mail to be released by the recipient in the normal course of events.
> (This is assuming you don't use the SPAM STATUS HIDDEN option.)
On every mail or just a few, it doesn't matter. It's the principle, not
its frequency, that's the problem.
> I don't think it's a good idea. But it's not as harmful or in-your-face
> as traditional C/R.
Same design flaws and premises. I'll grant you fewer side-effects.
David F. Skoll
>> It's different from traditional C/R in that the *most* annoying
>> side-effect of C/R (challenges sent to unrelated third parties) is
>> avoided.
> A C/R system placed behind a forgery filter would avoid the same thing.
Ah yes? You know of an 100% accurate "forgery filter"? How come you're
not vacationing in your million-dollar beach house by now? :-)
[...]
> It still gets caught in the first place. It's still delayed not for any
> reason related to its transport and delivery.
The OP is recognizing reality: Spam filters will make mistakes and you
will have FPs. His proposal is a (flawed) attempt to mitigate FPs and
is interesting for that reason.
-- David.
D. Stussy
> D. Stussy wrote:
>
> >> It's different from traditional C/R in that the *most* annoying
> >> side-effect of C/R (challenges sent to unrelated third parties) is
> >> avoided.
>
> > A C/R system placed behind a forgery filter would avoid the same thing.
>
> Ah yes? You know of an 100% accurate "forgery filter"? How come you're
> not vacationing in your million-dollar beach house by now? :-)
Uisng SPF and DK on the same message comes pretty close. However, I didn't
invent them.
(PS: I already live in a $1M+ house, although it's not beachfront.)
> > It still gets caught in the first place. It's still delayed not for
any
> > reason related to its transport and delivery.
>
> The OP is recognizing reality: Spam filters will make mistakes and you
> will have FPs. His proposal is a (flawed) attempt to mitigate FPs and
> is interesting for that reason.
How do you get from forgery filter to spam filter? They are NOT the same
thing. Forgery status and spam status are DISJOINT characteristics.
David F. Skoll
> Uisng SPF and DK on the same message comes pretty close. However, I
> didn't invent them.
SPF is mildly useful to prevent forgeries if you trust the domain's
owner and you trust DNS. And all that DK does is prove that a message
travelled through a specific server.
And you (as a recipient) have no control over whether the sender
uses SPF/DK or not.
>> The OP is recognizing reality: Spam filters will make mistakes and you
>> will have FPs. His proposal is a (flawed) attempt to mitigate FPs and
>> is interesting for that reason.
> How do you get from forgery filter to spam filter?
You are losing track. You brought in "forgery filter". The OP was
proposing a method to mitigate spam filter FPs.
-- David.
D. Stussy
> D. Stussy wrote:
>
> > Uisng SPF and DK on the same message comes pretty close. However, I
> > didn't invent them.
>
> SPF is mildly useful to prevent forgeries if you trust the domain's
> owner and you trust DNS. And all that DK does is prove that a message
> travelled through a specific server.
That's why I said "close" above. SPF can have intra-server forgeries, and
DK can have its private key stolen. Both can suffer from DNS poisoning.
> And you (as a recipient) have no control over whether the sender
> uses SPF/DK or not.
Not my problem. If they don't, then they PERMIT ANYONE to use their
mailboxes as the sending identity, and are therefore spammer collaborators
when spammers use (or abuse) their resources. As a recipient, if they
haven't told me that a message from them is forged (by deploying a
forgery-detection method, whether these or some future construct, and at
least one marks it as not authentic), the message is NOT forged, and I am
historically required to assume that they did send it (or authorized it to
be sent by a third party) unless I have other information that tells me it
wasn't.
Therefore, since my server checks for both SPF and DK, should the message
be spam, I don't care if the sending identity was forged but wasn't
detected because the mailbox owner didn't deploy these tools. That owner is
nonetheless responsible for the spam, even if that responsibility is based
solely on his failure to protect his mailbox asset(s) from spammer abuse
(cf. "Joe Job").
This behavior is part of what the backscatterer DNSBL enforces - recipients
must check during SMTP for both spam and forgery status, and SMTP reject
messages which are either, else face a potential listing.
> >> The OP is recognizing reality: Spam filters will make mistakes and you
> >> will have FPs. His proposal is a (flawed) attempt to mitigate FPs and
> >> is interesting for that reason.
>
> > How do you get from forgery filter to spam filter?
>
> You are losing track. You brought in "forgery filter". The OP was
> proposing a method to mitigate spam filter FPs.
He brought in "The sender should/will/must do 'X' for message delivery to
continue." Once the sender identity is brought into the issue, so comes
message forgery.
David F. Skoll
[...]
>> You are losing track. You brought in "forgery filter". The OP was
>> proposing a method to mitigate spam filter FPs.
> He brought in "The sender should/will/must do 'X' for message delivery to
> continue."
No. That's not how I read it.
He said: The sender can *choose* to do 'X' to expedite delivery. From
TFA: "A can choose to solve the CAPTCHA to prove the message is not
spam."
If the sender chooses not to do 'X', then the message may be released from
the filter or not, depending on what the recipient does.
> Once the sender identity is brought into the issue, so comes
> message forgery.
The sender identity is never brought into the issue. In the scheme,
the (unidentified) sender merely hits a publicly-known web site and
provides it some information that was in the message. We don't know
who the sender is, but we know that whoever hits the web site probably
is the same person who sent the message, because he knows the secret
information that was embedded in the message.
-- David.
Yao Ziyuan 2
> I see the original poster has added this to the spec:
>
> > really a spammer. So the page can also return:
> > This means the sender can solve the CAPTCHA to see the status and
> > change it to NOTSPAM.
>
> That's really bad. That means I have to solve a CAPTCHA for every
> email I send? That just won't fly; there aren't enough hours in the
No, SPAM STATUS HIDDEN means you can choose to solve the CAPTCHA on
the spam status page, to make your mail no longer marked as spam by
the recipient's mail system (whether it was previously marked as spam
or not). If you don't solve the CAPTCHA, you don't know your mail's
spam status and can't do anything about it, just like dealing with
current systems.
> day for that. You've just invented Challenge-Response with an implicit
> Challenge.
>
> Anyway, we already havePassiveSpamRevocation. It works like this:
David F. Skoll
> No, SPAM STATUS HIDDEN means you can choose to solve the CAPTCHA on
> the spam status page, to make your mail no longer marked as spam by
> the recipient's mail system (whether it was previously marked as spam
> or not). If you don't solve the CAPTCHA, you don't know your mail's
> spam status and can't do anything about it, just like dealing with
> current systems.
Since SPAM STATUS HIDDEN is the only secure mode of operation, then
PSR is useless. It provides no advantage over current systems.
In fact, it has security problems of its own; I'll leave it to the reader
to figure them out.
Regards,
David.