It was a concept proposed to prevent spamming (or could be expanded to
those ends which is what I'd like to do)
I'm racking my brains and can't remember it
~ Mike
> Can anyone remember the name of the proposed email replacement system
> that relied on the receiving mail server downloading the messages from
> the sending mail server (if it wants them) rather than the sending mail
> server "uploading" (if you will) messages to the receiving mail server?
I believe you're referring to DJB's "Internet Mail 2000":
http://cr.yp.to/im2000.html
Regards,
David.
That's it! :-) Thanks,
~ Mike
Okay, so building on this idea (Internet Mail 2000), my suggestion is a
bit like this...
RSA Encryption is mandatory. Each email address must have an RSA
encryption key. Their public key is stored on the mail-server and made
available for retrieval by anyone wishing to send them a mail.
When you wish to send an e-mail, your mail client downloads the public
keys for each of your recipients (or if you've send them an email
recently within 'x' amount of time, a local cached copy of the key).
You generate a random key for the e-mail which must be, say, 128 bytes
long - This we'll refer to as the "e-mail key" for this message.
You then encrypt the "e-mail key" to each one of the recipients public keys.
You then encrypt the e-mail, attachments etc... using the "e-mail key"
using, say, AES encryption (perhaps different encryption systems could
be used, and these could be included somewhere in the message).
This would all result in a XML formatted "header" that you'd then upload
to the mail server, which might look a bit like this:
<msg
id="029916b6-b0a2-4603-94a6-b9a9fcd83a11"
generated="1997-07-16T19:20:30.45+01:00"
from="us...@domain1.local"
>
<rcpts
count="2">
<rcpt
address="us...@domain2.local">
h5dLBjiJCsbzTyfj7hg0NZXsSfEnxZFZoPupwft77wscmEPZsl3Es5mSuQEqQw9FyVceneeEYtVc
2hxldDcbyQe5ymVIk1m8PUGjRyvyUh7SQsjfrBCah8utU9VY37aEY47xSLVIBKyW46m+vAlmommS
obylOdgCEJxScJM5ovJQxB0qya4Zj2jlnKDpM4r0uAshCj+tiLRWp4FskCiQGz8XV40G2rDm210y
XBw8qZMbYk9RucDP4JbpOb10LiWm95xc4lSBZ4rBXQ489xlVSrI4McbjxXcB4k3a652AD6TsdLX4
wqZVNCpslFdfNDP0CPp02xEp2wFUc2XefyjuXw==
</rcpt>
<rcpt
address="us...@domain3.local">
YpRlGIiGSf73Nnwhb0l57lv+/sKxQqetP3O8VVqz3mZ6cmNhQRRjmqF4MA+qRfEqx3ogPxvOXy+0
/pxl4pgQefA34hnCCceHXcnVOa+WH9g09jMvK6xINA4zidKt3igK7pOsfSs7yvnIt1FAEMYcQ5lj
LUt29ueu3j0vZgj26stZWBAyHvK1JYpxV4IM5Y6BMh4bXDB5R54Bs55ZVIIeYEltm0NygzBXjM3n
AoVIneejEjtRbCnEdAObqOey6rFJ3bIGl+XtYEY0pQ+QHB6o7lHc07HuGWHB1kjHDGpT8y9D8Gx7
eV0EoInHxV4DTmGGhidN+CZnRUj/Ruq5prx75g==
</rcpt>
</rcpts>
</msg>
(The "id" fields and the "generated" date would probably be appended by
the server, which would ensure the UUID was indeed unique, and the time
was accurate using NTP).
The client would also upload the encrypted e-mail, attachments etc..
that the server would then place on a publicly accessible mail-server.
The server would then notify all the recipients mail servers that a
e-mail is available for them, along with the "id" for that particular
message.
The recipient would be informed about the message when he next polled
his/her mail-server for new messages and can either ignore the message
of decide to download it.
If they decide to download it then their mail client would connect to
the public webserver to download the message using the "id" it's been
given, along with any additional attachments (which would be referenced
in the message):
https://domain1.local/store/029916b6-b0a2-4603-94a6-b9a9fcd83a11/message
https://domain1.local/store/029916b6-b0a2-4603-94a6-b9a9fcd83a11/<att 1>
https://domain1.local/store/029916b6-b0a2-4603-94a6-b9a9fcd83a11/<att 2>
The mail client can then inform the users mailserver that they've
downloaded the message and inform the server to either notify the
senders mailserver (delivery confirmation), or just remove the message
header and not inform the senders mailserver (if you do not wish them to
know you received the message for some reason).
One advantage of encrypting to each recipient is the time it would take
to do this - You could also add a expiry to the encryption to make sure
messages could not be replayed. It would make spamming very difficult
because of the amount of processing power required to carry out the
encryption for a lot of recipients - And that attachments would be much
smaller because base64 encoding would not be required (they'd be binary
and stored on the webserver) and you wouldn't have to deliver the
attachment to each person allowing for very large attachments.
Just a kind of outline at the moment, with some finer details missing
but anyone have any constructive criticism and/or suggestions on the
concept?
~ Mike
> Just a kind of outline at the moment, with some finer details missing
> but anyone have any constructive criticism and/or suggestions on the
> concept?
http://www.rhyolite.com/anti-spam/you-might-be.html#spammers-are-stupid-3
http://www.rhyolite.com/anti-spam/you-might-be.html#senior-IETF-member-5
http://www.rhyolite.com/anti-spam/you-might-be.html#programmer-11
There are good reasons why IM-2000 went nowhere. What's the incentive
to start using it when SMTP is ubiquitous?
Regards,
David.
Well that's kind of one of the only arguments "FOR" the current e-mail
system. Problems came up that were not anticipated by its designers
(well, it kind of just grew organically so maybe designers isn't the
word for it). It's now universally used and so difficult to change.
The incentive would be lack of spam, end-to-end security, sender
authentication and reduced network overheard for binary files (as base64
isn't used for attachments), as well as realistic support for very large
attachments (gigs perhaps).
The biggest barrier for IM-2000 was probably nobody tried to answer all
the questions to put together a complete specification - Then actual
create some server/client applications for it. Without those
applications to at least demonstrate a working concept, it's not going
to go anywhere fast.
As a developer, the later is within my scope.
I'd agree it's definitely hard to challenge the statu quo but I think
it's an interesting project to peruse even if nothing comes of it. Just
the matter of filling in the blanks. IMO, IM-2000 is a good base to
build on.
~ Mike
>Just a kind of outline at the moment, with some finer details missing
>but anyone have any constructive criticism and/or suggestions on the
>concept?
Hi Mike,
off the top of my head, there are these potential problems that I
think you'd need to address:
1) I currently have some pretty good spam filters and its rare for any
spam to get into my inboxes, (or for good mail to get rejected by the
spam filters). The spam filtering uses a broad range of techniques
including content analysis. If the mail system only sent me a
notification of an email, I'd still have to download it so that the
spam filters can evaluate it, so I don't see that it has much of a
spam reduction benefit from that point of view.
2) Some spammers use read notifications, web bugs etc, to try to
determine which recipients are worth spamming again because they
opened the mail and read it. With the sorts of systems you are talking
about, they wouldn't need such complications because the server
holding the mail copy could log who reads their mail and when. It
would also have privacy implications for non-spam mail as well. Now I
know a lot of businesses who think it would be a benefit to know
exactly the date and time that their email was read by the recipient,
but I'd prefer to keep my privacy.
3) Some malicious mails contain rather large attachments, e.g. if they
are sending an infected file. If the spammer only had to send out a
small notification instead of the whole file, and only see it
downloaded by people who actually open the mail, he'd avoid wasting
bandwidth and might actually be able to send out more spam, not less.
4) ISPs who implement outbound virus scanning etc would no longer be
able to detect anything in the outbound mail. They'd all look the same
as legitimate mails.
5) The idea that generating the encrypted checksum is going to be time
consuming and therefore expensive for the spammer (but not for the
recipient) is flawed. Most spam is sent out using compromised
machines. The spammer is already stealing bandwidth and cpu. How will
it matter to him if a compromised machine runs a little slower still?
Most zombied PCs have plenty of spare processing cycles. The
limitation on spam sending is usually the bandwidth and the existing
filters and blocks, not the processing power of the infected host.
6) How long does the sending server keep email? The other day I sent
someone a document and received an autoreply back to say she is on a
sabbatical and won't be back at her desk until next January. No
problem, the mail will still be there for her when she gets back to
work. Under your outline, is the mail server expected to keep mail for
as long as it is unclaimed? Suppose you send someone a mail to an
obsolete Hotmail account. Would the sending server have to keep the
mail forever, even after Hotmail has closed down the acocunt?
I think we've already got plenty of tools for dealing with spam,
phishing and viruses, and coming up with another technical fix isn't
going to be that productive, unless you are 100% sure that it is a
permanent solution to all problems. Right now, we need to address the
issue of so many people getting their PCs zombied. Maybe it needs
better protection built into Windows etc. Maybe it needs better
firewall software so that more people block direct-to-MX from their
PCs. Maybe it needs more ISPs to block direct-to-MX from consumer
space, more blocking of ports used for zombie remote control, and more
use of community blacklists to cut off the machines being used to send
out the spam. I'm not saying any of those is easy, but I think they
are all easier than trying to come up with yet another alternative to
SMTP.
Yes this is true, but you do have a verifiable encrypted message. The
server system could even take care of validation. It cannot decrypt the
message because of the end-to-end nature, but it could check a signature
and check the mail is valid.
I would envisage the concept relying on a web of trust, you'd deny a
message purely based on it's verifiable sender. Rather like with an
instant messenger client you accept or block new contacts.
They'd have to squeeze their entire spam message into the from address
in order to get some kind of message across to you, and you could put in
place restrictions on acceptable characters and sizes of from addresses
(before the "@").
Now someone "could" attempt to DoS you with these new senders but then
that's always going to be a problem. And they'd at least have to
generate a key for each one, and that key would have to map to a valid
domain and mailserver.
Assuming we carry over the good practice of requiring valid DNS and
reverse lookups for the host it would be easily to block these are soon
as the DoS started.
> 2) Some spammers use read notifications, web bugs etc, to try to
> determine which recipients are worth spamming again because they
> opened the mail and read it. With the sorts of systems you are talking
> about, they wouldn't need such complications because the server
> holding the mail copy could log who reads their mail and when. It
> would also have privacy implications for non-spam mail as well. Now I
> know a lot of businesses who think it would be a benefit to know
> exactly the date and time that their email was read by the recipient,
> but I'd prefer to keep my privacy.
Yes, this is true of legitimate mail - I'd hasten to add that spam and
things like this you'd not download - Well unless you really wanted to
read the spam and accept and download based on their notification.
The sender though would indeed have the exact date/time you accepted the
message for download. But in the case of a web or trust, with an
already established whitelisted sender (remember you guarantee their
identity to a high degree with the RSA key system) I'd imagine in this
scenario you'd automatically accept the message and it would immediately
be downloaded (in the case that you trust someone, and you trust their
key, and the key has already been validated).
So it would not in this case be so much of a read notification, only a
pretty good indication that it made it to the persons mailbox.
If you don't trust someone enough to even want to give them that
information (that you received the e-mail, not that you read it). Maybe
that's the kind of person you shouldn't take e-mail from? :-)
They already know this with SMTP (or have a pretty good idea) if your
SMTP server accepts the e-mail.
> 3) Some malicious mails contain rather large attachments, e.g. if they
> are sending an infected file. If the spammer only had to send out a
> small notification instead of the whole file, and only see it
> downloaded by people who actually open the mail, he'd avoid wasting
> bandwidth and might actually be able to send out more spam, not less.
True, you'd have to accept the message from a stranger though. If it's
the case of someone you know and trust sending a tainted mail then this
is going to always be a problem. Restrictions on file extensions would
still be possible.
> 4) ISPs who implement outbound virus scanning etc would no longer be
> able to detect anything in the outbound mail. They'd all look the same
> as legitimate mails.
Yes this is true. You'd have to restrict the file extensions to prevent
this. But it's always kind of easy to get around, a lot of people send
the "Hey, follow this link" style emails using social engineering to do
their handy work rather than attach the virus directly.
> 5) The idea that generating the encrypted checksum is going to be time
> consuming and therefore expensive for the spammer (but not for the
> recipient) is flawed. Most spam is sent out using compromised
> machines. The spammer is already stealing bandwidth and cpu. How will
> it matter to him if a compromised machine runs a little slower still?
> Most zombied PCs have plenty of spare processing cycles. The
> limitation on spam sending is usually the bandwidth and the existing
> filters and blocks, not the processing power of the infected host.
However, in this case these zombie machines must have a valid email
account that maps to a server. And as above we can apply the same
precautions are currently often used (requiring valid DNS and reverse
DNS for the host).
Now the spammer can't email out directly from the zombie machines as any
sender they like. They must generate the key and upload it to a server
with that correct DNS setup.
Now say this is a hackers server that they've setup to channel the mail,
well that's checkmate - Easily detectable by the receiver and the
hackers ISP and easy to block and shutdown.
And bear in mind they can only transmit information to you in the from
address (their entire advert must be in there).
Just decline the message, perhaps the option to decline and refuse the
domain until further notice - Or perhaps in extreme cases the TLD or
certain IP block. That hackers mailserver is now out the picture.
> 6) How long does the sending server keep email? The other day I sent
> someone a document and received an autoreply back to say she is on a
> sabbatical and won't be back at her desk until next January. No
> problem, the mail will still be there for her when she gets back to
> work. Under your outline, is the mail server expected to keep mail for
> as long as it is unclaimed? Suppose you send someone a mail to an
> obsolete Hotmail account. Would the sending server have to keep the
> mail forever, even after Hotmail has closed down the acocunt?
Yes, this could be a problem. Although I in-visage (as above) that with
a web of trust and whitelisting based on public keys that you they could
also have this email waiting for them in their inbox. Just not reading
it until they get back (as before).
Unknown or unexpected senders would be hindered - But I'd kind of see
that as a plus.
To be completely away from your email and not checking it at all for ~6
months and expecting email from unknown senders in the meantime is
probably quite a rare scenario IMO.
The sender could establish a validity time for the email, after which no
more notifications would be sent and the message can be removed.
> I think we've already got plenty of tools for dealing with spam,
> phishing and viruses, and coming up with another technical fix isn't
> going to be that productive, unless you are 100% sure that it is a
> permanent solution to all problems. Right now, we need to address the
> issue of so many people getting their PCs zombied. Maybe it needs
> better protection built into Windows etc. Maybe it needs better
> firewall software so that more people block direct-to-MX from their
> PCs. Maybe it needs more ISPs to block direct-to-MX from consumer
> space, more blocking of ports used for zombie remote control, and more
> use of community blacklists to cut off the machines being used to send
> out the spam. I'm not saying any of those is easy, but I think they
> are all easier than trying to come up with yet another alternative to
> SMTP.
Yes, zombie PC's are a big problem and there are lots of things that
could be done to SMTP to move it forward - But the problem you'll find
with staying with SMTP is a lot of hosts simply cannot apply a lot of
the tools for dealing with spam.
A lot of people would rather take the hit on spam rather than
accidentally reject a legitimate email. Now that would be rare with the
system I'm proposing, well it should be not possible unless their coming
from a known spamming host or they are on a domain you blacklisted.
How many domains already don't properly check HELO's? Don't check
reverse DNS? Don't have SPF records? You make sure you build it all
into a new system and force compliance from day 1 then you get rid of a
lot of the fear of false positives losing business.
That and the lengths ISP's and mailserver admins have to go to reduce
spam, the shear cost of resources, bandwidth, equipment, man hours is
just epic. We keep trying to patch up the old war-horse SMTP but with
everyone at different stages.
Thanks for the detailed response BTW, it was most appreciated and
definitely what I had in mind for constructive criticism. Although some
of the responses I'm dreaming up on the fly of course and I did not
specify originally, but that's because it's all just a kernel of an idea
based on an idea. It's all still forming! :-) Your post was most
useful in fixing my mind on those issues.
~ Mike
> 4) ISPs who implement outbound virus scanning etc would no longer be
> able to detect anything in the outbound mail.
>
The fact that a "pull" system doesn't exactly support mechanisms
designed specifically for operation on "push" systems is not a
disadvantage, but an error of thinking. You might as well say that HTTP
is at a disadvantage because ISPs cannot scan the contents of the WWW
sites that their customers put up. It's the same sort of category error.
> Maybe it needs better firewall software so that more people block
> direct-to-MX from their PCs. Maybe it needs more ISPs to block
> direct-to-MX from consumer space, more blocking of ports used for
> zombie remote control, and more use of community blacklists to cut off
> the machines being used to send out the spam. I'm not saying any of
> those is easy, but I think they are all easier than trying to come up
> with yet another alternative to SMTP.
>
Ironically, by yet further balkanizing SMTP in exactly the ways that you
propose you make the case for replacing SMTP all the more attractive.
Every such measure, from making yet more groups of third class Internet
citizens to having yet more contradictory, opaque, and arbitrary
criteria for randomly refusing to accept mail from people, makes the
system that much less useful for its purpose of transporting mail as it
fragments into a myriad of non-intercommunicating pieces. So yes, adopt
all of those measures and more.
http://homepage.ntlworld.com./jonathan.deboynepollard/Proposals/IM2000/
> Then actual create some server/client applications for it. Without
> those applications to at least demonstrate a working concept, it's not
> going to go anywhere fast.
>
> As a developer, the later is within my scope.
>
Do you have an ASN.1 library? (-:
Thanks for the link, I had a quick scan through it (I'll read it all a
bit later).
It doesn't appear like the guy has done anything since 2004 though.
From a cursory glance looks reasonably complete, good basis.
According to WIKI (http://en.wikipedia.org/wiki/Internet_Mail_2000) the
closest anyone has come to implement something like this is StubMail in
2006 (http://en.wikipedia.org/wiki/StubMail).
I assume it's not gone anywhere from there although I really wish more
people would clearly date stuff!
>> Then actual create some server/client applications for it. Without
>> those applications to at least demonstrate a working concept, it's not
>> going to go anywhere fast.
>>
>> As a developer, the later is within my scope.
>>
> Do you have an ASN.1 library? (-:
I have done this much coding toward the concept:
______
But I'm an awfully keen guy, I've spawned at least 1 decent Open Source
project into existence by just discussion and doing the leg work to put
everything together.
Regardless. even if it never gets off the ground or even implemented in
a partial form it would still make an interesting development project.
Lots of interesting fields being covered.
~ Mike
> > [...]
>> There are good reasons why IM-2000 went nowhere. What's the incentive
>> to start using it when SMTP is ubiquitous?
> Well that's kind of one of the only arguments "FOR" the current e-mail
> system.
Again: What's the incentive to change? Do you suppose the average ISP
will take kindly to having to store all of its users' outgoing email?
The costs of a system like IM-2000 are very high and the benefits are not
at all clear.
> The incentive would be lack of spam,
That's refreshingly naive. :)
Spammers will subvert the system. You can count on that.
> end-to-end security,
That's not clear to me.
> sender authentication
That's a negative. When I send inquiry emails to people I don't know, I'm
not sure I want or need strong sender authentication.
> and reduced network overheard for binary files (as base64
> isn't used for attachments),
A very small benefit.
> as well as realistic support for very large attachments (gigs perhaps).
A very small benefit again.
> As a developer, the later is within my scope.
The problem with spam is not primarily technical. It's a social problem
and no amount of technology that preserves the great features of email
(the ability to quickly get in touch with someone with whom you've
never communicated before) will get rid of spam.
> I'd agree it's definitely hard to challenge the statu quo but I think
> it's an interesting project to peruse even if nothing comes of it. Just
> the matter of filling in the blanks. IMO, IM-2000 is a good base to
> build on.
Well, go ahead. I think you're tilting at windmills, but who knows... maybe
you'll come up with something amazing.
Regards,
David.
> Unknown or unexpected senders would be hindered - But I'd kind of see
> that as a plus.
You obviously do not use email for business purposes. My business relies
on receiving queries from unknown and unexpected senders. We'd be out of
business in 30 days if this were hindered.
-- David.
Hindered, not blocked. If you wish your business address to receive
email from anyone, you can do this. You can allow or block whatever you
like.
~ Mike
Why doesn't Timo Salmi answer your question, or participate in this
thread?
He's always posting a regular procmail-tips post here. Why doesn't he
participate?