I know this is semi-long but please take the time to read it.
EmmanuelGoldstein
about the potential architecture of Okopipi. I'm a software developer
myself, so I love working on new and novel approaches to complex
problems. Plus, I love the idea of a spam-fighting p2p network... in
theory. However, I think we all need to take a deep breath and take
inventory of the problems and where we are at with potential solutions.
Perhaps it would be helpful to distill the problem domain a bit.
All of the difficult work in this project stems from the fact that we
don't know who to trust. There have been a number of suggestions in
the group ranging from Karma to trusted nodes to PGP key signing, etc.
None of them is a full solution though, and as we get farther and
farther along in the mental exercise that is the Okopipi architectural
design, things keep getting more complicated. In the software world,
complexity = bugs, and we cannot afford those.
Who can you trust? Nobody - unless there is a trusted architecture in
place. Within the current architecture, this means a trusted
authentication subsystem, a trusted command relay subsystem, etc. This
is diametrically opposed to the idea that we want all nodes to be equal
(or at least appear that way in the case of the admin nodes). It
simply cannot happen. If you allow the system to open it's arms and
allow anyone in, you are going to have infiltration. Say that one of
the routing nodes belongs to a spammer, but nobody knows it because it
behaves itself for 6 months or a year. By that time, it would have
gained lots of trust or karma or whatever other metric is used to
determine that kind of thing. Then, when the spammer decides, he
alters the node to start subtly undermining the rest of the system.
Trust and anonymity don't mix. Even with a PGP key, you don't know who
someone is. You may know that so and so computer sent a message, but
that doesn't convey trust in an way. So where does that leave us? It
leaves us with a set of publicly available servers that do all the
trusted stuff. Obviously that won't work because it gives the spammers
somewhere to focus their attacks. You cannot hide the servers in the
manners suggested without opening the flood gates for compromised
nodes.
I don't know of an elegant solution to the problem if we want to stick
to the p2p method. A rogue node has the exact same information at its
disposal as a clean node.
How then do you detect a rogue node? I don't know. If one was well
written, I think it would be extremely difficult. Do you check with
two or three nodes or peers to see if they respond in a manner
consistant with your expectations? No, you cannot do that because as
Poltergeist suggested - we're talking about the scenario where we are
outnumbered 1000 to 1.
So how do you meld the ideas of a resiliant, attack resistant
architecture with an untrusted network? One suggestion that I saw was
to use a moderated newsgroup to disseminate commands. Those commands
would be digitally signed. The clients would need a rudimentary nntp
client built in to talk to the newsgroup(s). New scripts would be
binary(base64) encoded and signed. Script submissions would come in
via other secure methods and would be overseen by a trusted group of
people. Focus on the spam submission problem and the script
distribution problem. These are feasible - easy even (when compared to
the p2p trust situation). We are spinning our collective intellectual
wheels in trying one-up the spammers. The fact is that the spammers
have more at stake, and they will have all the time in the world to
think of new ways to attack our design. Instead of thinking of more
and more complicated logic to detect and subvert and circumnavigate all
the potential attacks, lets instead take away the complexities. Remove
the attack vectors. And better yet, get Okopipi to the masses a lot
quicker.
I know everyone already has their minds set on p2p. It's a sexy
techology - but look what has happened to all the well known p2p
systems out there. They've all been poisoned to some degree by the
MPAA or RIAA or some guy who wants to spread viruses. Those that
thought they were anonymous are being outed all the time and
sued/expelled from college/etc. Look, we can still have our shiny
client application with whatever bells and whistles we want - it just
doesn't need to be build on top of a p2p network.
Like I said, I've seen a bunch of other good ideas out there and even
shot down a few of them. Now is your chance to shoot down mine :)
Spy der Mann
Yes, that's how it will work.
> Focus on the spam submission problem and the script distribution problem.
Script distribution will be just more pubkey-signed stuff.
The "problem" is spam reporting. Each node would keep a copy of their
SPAM, and it would be globally accessible. But I still need Nano to
explain to me how it's *actually* done.
Spy der Mann
Our P2P is an anonymous overlay network. Basically, Kazaa and other
similar networks had these flaws:
* Kazaa, Gnutella, eDonkey depended on "supernodes" or "overpeers".
These can be shut down with a DDOS.
Our network just depends on the nodes to transmit information. There's
no central point of attack.
* Traditional P2P networks weren't truly anonymous, your IP could be
easily revealed when you began downloading a file.
Our network transmits and receives data, but we've designed the routing
algorithm so that each node just transmits data but you really don't
know the true origin. It could be ANY node on the network.
The anonymity for our network is based on 3 things:
1) Sender anonymity. You really don't know who sent the message. You
just receive it and/or pass it around.
2) Recipient anonymity. You really don't know who is going to receive
the message either, you just know what route it will follow.
3) Encryption. The messages are encrypted with the recipient's public
key.
In other words, what you have in our network is not an envelope with
sender and recipient address. Just an envelope with the name of a VERY
LONG street, and everybody on the street would get the message and
check if it didn't belong to him.
However, once the recipient is able to open the package, he knows who
sent it and what the message contains.
So, as you can see, our network is truly revolutionary. What spammers
could know is simply if a node belongs or not to their botnet. But they
don't know ALL the nodes' IP's, because their position in the network
will be fixed. No, they CANNOT choose it.
This is (i suppose, but nano's the real expert here) because nodes
would be asked for their permission to join. Messages coming from
external nodes would just be ignored.
Perhaps you haven't read the papers on resient p2p networks, but they
take ALL of these matters into account. They provide simulations with a
percentage of infiltrators. The ODRI network that we're going to use is
the most robust against infiltrators.
EmmanuelGoldstein
on the resistant p2p networks you mentioned before I open up my mouth
again :)
EmmanuelGoldstein
good links that talk about De Bruijn graphs and anonymous routing.
Should tide some of us over until Nano is finished :)
http://www.cs.berkeley.edu/~nikitab/papers/quals.pdf
Presentation given by the same guy:
http://www.researchchannel.org/prog/displayevent.asp?rid=2991
Spy der Mann
I'll wait for Nano's explanation to come out. ;-)
Anyway I read the thing and our algorithm chooses a different approach.
EmmanuelGoldstein
read a fair amount about fault-tolerant p2p systems in general, but
none of them (including de Bruijn/ODRI) seem to solve the problem of
massive infiltration. de Bruijn graphs seem to present a good
hierarchy for efficient routing (with nodes joining and leaving all the
time), but it doesn't account for massive quantities of malicious nodes
.
Poltergeist
> percentage of infiltrators. The ODRI network that we're going to use is
> the most robust against infiltrators.
Yes, but can they really deal with situations where only 0.1 %
of the nodes are real frogs, and 99.9 % are zombies?
I have really looked, and have not found such solutions
among the papers. Did I miss something?
Spy der Mann
be big enough to overwhelm the zombies. The only sure way to tackle
this would be to limit the number of users or giving them some
authorization code sent over e-mail.
But we don't know if it's feasible (I really hope it is).
The bottom line is that NO system is infallible. What we're working on
is a system that will have much fewer weaknesses than current P2P
implementations.
Poltergeist
> read a fair amount about fault-tolerant p2p systems in general,
This makes me /very/ curious about what you may have to say about
the topology that I've been proposing, which I believe can resist
extreme infiltration without any problem, and will survive whatever
onslaught the spammers may throw at it.
Unfortunately my description of it is far longer than your semi-long
text:
http://groups.google.com/group/okopipi-dev/browse_frm/thread/080fc17b1bf83bcb
EmmanuelGoldstein
the old Byzantine Generals Problem
(http://en.wikipedia.org/wiki/Byzantine_fault_tolerance). Whole areas
of study have been done around the problem, but what it comes down to
is that you can only have so many traitors before the system fails.
Here is the original paper... pretty interesting:
http://www.cs.wisc.edu/~bart/739/papers/byzantine.pdf
EmmanuelGoldstein
Spy der Mann
isolated users connecting with each other. How would the admins connect
to them?
Okay, so maybe you implied that the network is a de Bruijn Graph of de
Bruijn Graphs... so we come to the same issue, to establish the network
you would have to connect to untrusted nodes, so the spammers would
just not form networks and outnumber your clusters.
Poltergeist
Wow, I must have made a really lousy explanation! I thought I
had explained clearly, but apparently not!
It all starts with somebody creating one of these trust networks.
Let's say it's Jim. Jim invites some people he trusts, maybe you
and nano. You in turn invite me and Emmanuel to your trust network.
I invite Tortanick. Tortanick invites wayne and secondwheel.
Spam submitted by secondwheel can reach Jim, because secondwheel
is a member of Tortanick's network, so secondwheel's frog can
send spam to Tortanick's frog, and Tortanick is a member of my
network so his frog can relay the spam to me, and I relay to
you, and you relay to him.
We're all connected. All the small networks together form a big
network of networks. That's what I've called the supernetwork.
To join the Okopipi supernetwork you have to find somebody
who is already a member of the supernetwork. You have to get
an invitation to that person's trust network.
> How would the admins connect
> to them?
No admins.
Admins are examples of the Hierarchy-and-Authority approach.
Hierarchy and Authority must be avoided, because it introduces
several serious vulnerabilities.
If you think of what I've described in my long text, you'll notice
that no admins are needed.
Everything that reeks of Hierarchy-and-Authority must be avoided,
almost at any cost. Almost.
Then, in the end, if we have really done everything we can to
avoid it, and really just can't do without, then, and /only/
then, should we arrange some /extremely/ carefully defined
admin function, or whatever it is we can't do without.
So it may be impossible to do without admins, but it should
really be an absolutely last resort, as pared down as is ever
possible.
> so we come to the same issue, to establish the network
> you would have to connect to untrusted nodes,
If you invite someone you don't trust much you give him a low
trust score. This means he gets very limited influence.
If he then invites 1000 spammers and 1000 zombies, he can play
all he wants with his spammers and zombies, they won't affect
you much, since, at the point of contact with you, everything
that comes from that direction gets a low score.
Mosinu
verification IRL. Make the person go to the Local PD, show his
identification and get that Police Dept to fax someone a letter on the
PD's letter head stating they have verified so and so. The person who
gets the fax then calls the PD via looking up the number through the
local telco to verify they did indeed send the fax.
This way, we have his real name and address (all we want), its
verified and the person didn't have to give out any critical
information to anyone except the local police dept to verify he was who
he said he was.
This doesn't 100% solve the problem of infiltration, though if someone
did and breaks the law we have his information already without having
to look him up. This will also make some of the less than trust worthy
think before they go trying to derail this from the inside.
I know its a lot of work, but it is doable and safer than trying to
rely on the internet solely to verify someone. We could do this for all
the developers, super nodes and leaf nodes, clients would not have to
go through this extra verification since they don't control anything.
Just brain storming ideas.
Spy der Mann
because not everyone is online at all time.
And we DO need admins to send you the scripts etc.
cyberscan
by myself that does something similar to yours. Rather than sending or
forwarding spam to another peer, what my program does is allow the spam
to be saved to a specific folder. My program then parse the spam and
looks for any url that is also contained in an instruction file. If
found, the instruction file tells the program how to submit a
complaint. The only thing that is distributed via the peer to peer
network is the instruction file itself, and that is cryptographically
signed using 1024 bit encryption.
The part I am working on now is the peer to peer instruction
transmitting and receiving. A peer connect to another peer and
divulges the date of the intruction file it has. If the date of the
file is more recent than the remote peer, the file is transmitted to
the remote peer. The remot peer adds the ipaddress of the sending
peer to its peer list. The local peer does the same for the remote.
Each peer ranks other peers based on the files received and sent, the
availability of the peer, and any peer that sends an improperly signed
file is removed from the remote peer's contact list.
If the peer computer is running LimeWire then the peer will save a copy
of the instruction file to the Shared folder so that the instruction
file can be obtained over the GNUTELLA network as well. It is also
very possible for emails or any other file to be dropped into a special
folder where the program can parse said file for new instruction file.
With multiple distribution channels for the instruction files, spammers
will have to basically take doen the entire Internet if they want to
stop the project. Doing that will be very unprofitable.
I think we are going to see some fun times ahead ;-)
nano
that Okopipi has 100,000 users (one fifth of what blue security had).
That would be 100,000,000 zombie nodes. We are definitely looking at a
possibility of ratios in the single digit range. We might have 600,000
zombies and 100,000 good nodes. Even those numbers seem to me to be
unrealistic (especially since I anticipate more than 100,000 users),
because we have designed the system to react negatively to zombie
nodes. Black hole attacks will be excised from the network, and we have
the ability to ban IPs from the network (with a "banned users" DHT and
a ban command signed by an admin). The ban ability could be used if
thousands of zombies all try to submit a fake spam report. An admin
would see the report and investigate it like any other spam report, and
if he found it to be fake, he could ban every IP that submitted it
simultaneously (would be based on a secure IP hash of course, not the
actual IP, so that no one's IP is compromised).
In addition, we're contemplating the use of distributed captchas as a
security measure if we have reason to believe we are outnumbered by
zombies. That would not stop them completely (spammers have ways around
captchas) but it would slow them down a lot, and allow us to keep using
the network.
To sum up, 1000 to 1 ratio is ridiculous. The ratio will much more
likely much closer to 1:1.
EmmanuelGoldstein
going to be tricky to determine the bad nodes from the good. The
Byzantine Fault Tolerance model underscores the problem of nodes that
are only faulty/traitorous on occasion. It's relatively easy to
determine when a node is failing consistantly. I am interested to see
what mechanism you have in mind to deal with the traitors. And once
you come up with that, you can probably do a Ph.D thesis on it :)
Kevin Winter
> determine when a node is failing consistantly. I am interested to see
> what mechanism you have in mind to deal with the traitors. And once
> you come up with that, you can probably do a Ph.D thesis on it :)
Here's a crazy idea. Let's reduce the case a bit. We're dealing with
the possibility that large numbers of spambots will be loading this
program in an attempt to infiltrate our p2p system. I know that people
make a habit of making lists of bad machines (from under certain ISPs
especially). Theoretically, we could use similar lists and only allow
known "good" nets to host large numbers of clients.
(note: now that i think about this, the dynamic nature of many hosts
likely makes this infeasible)
Alternatively, why not make it impossible for a bot to run our client?
Either require that the user actually makes keypresses of the
keyboard (integrate with OS and listen for actual keypresses?)
Better yet, have an challenge question. This may be infeasible on a
per-spam basis, but maybe batch messages together. Then for every
batch of messages "submitted" to the "master" nodes, have them ask the
client to answer the challenge question. This way the client can't
fake an answer, and we have reassurance that the node is behaving
properly.
We already are having a client-server architecture, even if our
routing system is p2p and distributed. A distributed challenge
question for every "batch" of spams make sense. Also, if its a bot,
the only solution os for each bot to forward the message to its
creator to answer. And while we have lots of users willing to answer
captchas to make opt-outs on sites, i doubt that the bot owners will
have the time or dedication to solve them all. This, coupled with a
"trust", "karma" or "integrity" rating based on number of valid spam
reports could solve the bot problem.
Any comments?
~Kevin
--
Open Source, Open Mind
EmmanuelGoldstein
could potentially be put to good use, there are more roadblocks in that
area. Any Turing test can be defeated by simply having a user on some
porn site fill in the required data. So, the only ways that come to
mind to prevent this would be to make it so the request times out very
quickly OR make it something that only a computer can do but which is
computationally expensive and takes awhile to solve.
I don't know how you could implement a short timeout without having
nodes be able to quickly connect to the admin nodes. As per past
discussions, the admin nodes would only be reachable during certain
unknown times, and not directly. You couldn't rely on timestamps
coming from the client - those could easily be forged. You could have
the solved test be sent back through several nodes and verify that the
time stamps match from each node that it passes through. This might
also be a good way to catch traitor nodes (if 4 out of 5 nodes reports
one timestamp and another reports something way different you know you
might have a traitor node).
Relying on computation power rather than human intervention is an
interesting idea, but has its cons as well. Older, slower computers
might take too long to compute something that modern computers could
quickly determine. Also, there would obviously be a lot of resource
usage on the client system so it would have to be done when the system
is idle. Again, that might not be a bad thing. It might tie up the
bot nets.
Anyhow, just some food for thought.
Spy der Mann
question. Botnets will *NOT* run Okopipi. They will *EMULATE* it. Which
means they'll emulate the keyboard presses and mouse moves if they can.
As a webmaster and programmer, I can tell you one thing: You *can't*
trust the "client". Everything must be server validated. This is what
captchas are.
In any case, I think that with enough connections per node, our network
will be fault tolerant even in a 10:1 bot/user ratio.
EmmanuelGoldstein
challenge-response. I think the idea of listening for actual
keypresses is interesting but it misses the point. We have to assume
that the attackers are not going to be using our client application.
The kinds of attacks that we are trying to defend against are those
where the attacker is using his own malicious client to connect to the
network. We cannot rely on anything a client tells us. That's why we
need a system of checks and balances to try and separate the wheat from
the chaff, and to detect traitor nodes.
Also, even though we have a client-server architecture, it is
unorganized and the "server" nodes are not directly accessible from the
client. So, getting a response to a question could take quite awhile.
Enough time that the question could be displayed on some web site for a
user to fill out.
EmmanuelGoldstein
Mosinu wrote:
> If we have to go with an admin approach why don't we do it with
> verification IRL.
> We could do this for all
> the developers, super nodes and leaf nodes, clients would not have to
> go through this extra verification since they don't control anything.
That's an interesting idea. The idea of having a p2p setup means that
the /network/ decides who becomes a "super node" (sorry Spy, I know you
hate that term) and a leaf node. They are all joining/leaving the
network all the time, so it would not be feasible to have them all
predetermined and authenticated by a real person.
Plus, I dont think that more than a couple hundred users would go
through the trouble just to have the pleasure of running a supernode.
Even if they did, that would give the spammers a much smaller target to
aim at.
Poltergeist
> because not everyone is online at all time.
People will change this if it will reduce their spam problem.
The trust networks don't need to be connected day and night,
it's enough that the online times of the computers overlap for
a while once every day. Home computers can be left running from
dinner to bedtime, for example.
Users who pay by the minute for connection time can connect
two or three times during the hours when the broadband users
are connected.
This won't be suitable for people on shift hours and people
in very different time zones. These groups can have one or
more members leaving their computers running day and night
while the others connect a few times every day, or every
member can leave his computer running for more than 12
hours every day, or they can agree on suitable hours.
Okopipi will be used by people who are seriously annoyed
by spam. If leaving their computgers online more than they
used to is what it takes, many will do it.
> And we DO need admins to send you the scripts etc.
It's better to have scripts distributed to lots and lots
of script checkers, with arrangements to let them exchange
comments on anything they find doubtful. You gain bragging
points by spotting problems and by writing scripts that are
free of problems.
In the end they vote. Okopipi will only execute scripts that
have been approved by a sufficiently large number of script
checkers who have a sufficiently large trust ranking.
> Poltergeist: Your approach sounds quite difficult to follow.
No, it doesn't! :-) What else?
Poltergeist
> that Okopipi has 100,000 users (one fifth of what blue security had).
Yes, but do you expect that number from day one? Even with the
high visibility that we have now, that would be a truly fantastic
feat of marketing.
Even if it takes us only a week to reach 100,000, which I find
difficult to believe, during that week the spammers will be
desperate to shoot us down. They know that their days may be
counted if they don't shoot us down while we're small.
Being desperate, they will bring in all the artillery they can
muster.
> We might have 600,000
> zombies and 100,000 good nodes.
Every zombie machine will emulate ten frogs at the very least. More
likely there will be far more frogs emulated by every zombie machine.
> The ban ability could be used if
> thousands of zombies all try to submit a fake spam report.
What is a fake spam report? Why wouldn't they submit convincing
reports?
I should have read closely to learn about this when fake spam
reports were being discussed in other threads, but at the time
I didn't have time to follow the discussions closely. Sorry
about that.
> In addition, we're contemplating the use of distributed captchas as a
> security measure if we have reason to believe we are outnumbered by
> zombies. That would not stop them completely (spammers have ways around
> captchas) but it would slow them down a lot, and allow us to keep using
> the network.
In our thread where captchas were discussed somebody claimed that today
captchas can be parsed efficiently with OCR technology.
> To sum up, 1000 to 1 ratio is ridiculous. The ratio will much more
> likely much closer to 1:1.
Please convince me that this is true. Believe me, on this issue
I would definitely prefer to be wrong. It would make everything
so much easier.
You have not yet convinced me, but please keep trying.
Poltergeist
Some spammers are closely related to organised crime syndicates.
If they can take our service out by threatening and/or attacking
a small number of known people, those people may be in serious
danger.
Kevin Winter
> unorganized and the "server" nodes are not directly accessible from the
> client. So, getting a response to a question could take quite awhile.
> Enough time that the question could be displayed on some web site for a
> user to fill out.
I'd argue that we could count on a maximum travel time of a minute,
no? so, the master sends a challenge question to the node, and
demands that the responder answer it in a minute (or even two). Do
you really think that the byzantine generals will have enough willing
dupes to fill out every challenge answer?
Let's do some numbers. Say we get 100,000 valid users. Say that the
bots match this (any higher just proves my point more). Say that each
client reports one batch of messages a day. thats 100,000 challenge
answers due in from the bots. And unless they are answered within the
timeout period (even a few minutes), then the message fails. Failed
messages could reduce karma and such.
Even going on extremely conservative numbers, you'll see that the
distributed captcha COULD reduce the number of zombies. I admit that
it might be challenging to implement, but this is simpler than the
byzantine generals problem. With the generals, it was a number of
humans and a few were sometimes-traitors. we're talking about a huge
number of automated sometimes-traitors all controlled by a few people.
therefore, a captcha, or similar challenge question, could reduce the
risk of byzantine failure.
An alternative to a captcha would be a silly question, perhaps even
concerning this project. If our captcha had a picture of a little
girl and her doll, with a picture of spam messages around her and the
question "how many spam emails are bothering little susie?", i think
the spammers might not be willing to place these captchas on their
site.
i think we may want to consider some method of guaranteeing that the
client is not a zombie. our "enemy" will be a large number of zombies
controlled by a few people, whereas our "good" clients will be a large
number of humans controlling a single {computer,account} at a time.
we can differentiate to eliminate byzantine failures based on the
information.
The question is how?
HumanChimp
Kevin Winter
> or how about: http://www.kittenauth.com/
OMG PONIES!!!!11!!!!one!
Actually, that's perfect :)
Exactly at odds with a porn site. We could make it assign an id to
each image sent, and the client has to submit back the id's of all the
correct images. fail three times and it counts against you. If we
could also include a single image that advertises okopipi and explains
what its trying to do (with our logo), the spammers would actually be
helping us spread by using porn site visitors to solve them :)
EmmanuelGoldstein
The anti-spam captcha idea is brilliant! We must be careful not to
rely on anything that can be DDOS'd though. So, the captcha answers
would have to make their way back to the trusted authentication servers
in a roundabout way and in a manner that is tamper proof.
We are still at risk for byzantine failures though - even with the
turing tests. The turing tests would hopefully cut down on the ability
for bots to infiltrate the system in the first place. In that way, the
risk level is lower even though the opportunity for failure still
exists.
Also, you mention that a reasonable timespan for challenge-response
might me a minute or two. Unless the initial idea has changed, the
admin servers were only going to be online/avilable for a short
interval and at unknown times. We are talking about asychronous
operations taken to the extreme. The challenge might need to be issued
when the client isnt even online, and vice versa for the response.
Either that, or make the response be some computationally
challenging/pseudo-random problem that doesn't involve a human being.
I think we are heading down the right road to coming up with some
fairly robust botnet countermeasures, but we're not quite there yet.
Kevin Winter
> rely on anything that can be DDOS'd though. So, the captcha answers
> would have to make their way back to the trusted authentication servers
> in a roundabout way and in a manner that is tamper proof.
In another post, i mentioned our own lightweight version of DNS, we
might want to give that at least a thought, even if we ultimately
shoot it down. It would guarentee that a single node will only know
the IP of the next node in a route to a "master" node.
> We are still at risk for byzantine failures though - even with the
> turing tests. The turing tests would hopefully cut down on the ability
> for bots to infiltrate the system in the first place. In that way, the
> risk level is lower even though the opportunity for failure still
> exists.
Hopefully we can reduce the number of byzantine generals (as opposed
to good generals) down to the point where we don't care if they fail
occasionally. I forget the exact number of byzantine generals a
system can tolerate.
> Also, you mention that a reasonable timespan for challenge-response
> might me a minute or two. Unless the initial idea has changed, the
> admin servers were only going to be online/avilable for a short
> interval and at unknown times. We are talking about asychronous
> operations taken to the extreme. The challenge might need to be issued
> when the client isnt even online, and vice versa for the response.
Right, but SOMETHING has to be online in order to receive the spams.
Otherwise you'd get into the situation where you have the clients
never connecting/on at the same time as the master nodes! So, whoever
is a "master" or "handler" or whichever could handle the receipt of a
spam report. make it like so:
1. client c sends spam report
2. network routes it to a receiver of reports r
3. r sends c the challenge and timestamps the sending
4. at some point, c responds to the challenge
5. r checks to see if the response is in time, if so, accept report to
eventually deliver and tell c received, otherwise reject
6. if rejected, or c doesn't hear from r after a timeout, resend the
report (which will go to either a new R or a different R, either of
which works)
This allows anything to fail safely, except for a network failure,
which we can ignore if we use TCP. (or some other end-to-end protocol)
~Kevin
P.S. sorry if i'm crossing threads here, just trying to get the ideas out :)
Spy der Mann
1. Ask 5 random nodes in the network to produce a captcha.
2. Concatenate them (and distort them) and show the captcha to the
client.
3. Split the alphanumeric answer back and send the result to the nodes.
4. The nodes give authorization and the user is allowed into the
network.
EmmanuelGoldstein
What happens when some (or all) of those random nodes are compromised.
What if I am a spammer and somehow get some of my nodes onto the system
and in the position to participate in the authorization you mention in
#4? I could just give authorization all day long to my other rogue
nodes. Better yet, my rogue node could just maintain a list of all the
botnet systems and automatically give them the thumbs up whenever
necessary.
Kevin Winter
I like this, takes the load off the master servers. However, i think
it should be on a per-spam/message/batch instead of simply access.
this means that we cover byzantine failures (in which a node sometimes
behave badly, but not always).
As i said earlier, i like kitten auth because it it entirely opposite
of the spammers wares. also, we could include an image of our logo,
motto, website, etc in place of one of the images. we could use the
random nodes as you suggested to create this captcha, which would cut
down on individual bandwidth usage for the image transfers.
(dynamic image sending is needed, or else the spammer could cache all
the images with filename/checksum and possibly defeat it that way)
~Kevin
EmmanuelGoldstein
apologize if I am off the mark.
Kevin Winter
> apologize if I am off the mark.
See, i never really intended them to be directed to a website. more
that they have to download 9 small gif images (10k each?) from a p2p
client (we could even randomize them and send one image from each
random node to help the bandwidth). We could throttle no more than
one report in a 5 minute period, etc, to remove the DDOS ability.
Here's a way to implement it:
(all nodes have a small library of the images, use say 500 stock images)
1. Master node contacts 9 random nodes to see if they can help
2. each helper node runs a requested image through a filter (so the
checksum will be different, could even use a digital watermark)
3. each helper rename the file to a identifier and sends it to the
message sending machine. the identifier is sent to the master node
4. the requesting node send back a list of the "correct" identifiers
to the master node.
This means that the helper nodes can't byzantinely fail (unless they
mess up the protocol (identifier, etc)) and authorize bad nodes, only
the master knows the correct ones. also, a bandwidth consumption
wouldn't be easy with images in the 10kb range or less. added to
throttling where the master node will drop requests is asked too
often, makes it pretty resilent to DDOS.
EmmanuelGoldstein
Along with the hidden admin nodes we could implement a system of spies.
These would be trusted nodes that nobody but the admins know about.
They would be undetectable in the network because they would have an
identical signature to all other nodes.
But, their purpose would be to reveal faulty/malicious nodes. The idea
is that they contact nodes at random, communicate something with the
node and then compare the action that node takes with some kind of
expectation. If they dont match, you may have a problem. These spy
nodes would be unique because they would be able to communicate with
random nodes but they would also be able to collaborate with the admin
nodes to check results. Think of them as probes that are injected into
the bloodstream of the okopipi network to see where problems are.
Spy der Mann
later we'll have to trust AT LEAST ONE node for the validation.
Who will that be?
nano
threads is covering topics for which we have already found an
acceptable solution, or which fail to take into account the network
design that we are working with. This is no one's fault but mine for
failing to be fast enough in documenting our design decisions, but I
wanted to point at that much of what is being said is fairly moot. I
would like to clarify a few points, though:
Poltergeist wrote:
> > Assume that Okopipi has 100,000 users (one fifth of what blue security had).
>Yes, but do you expect that number from day one? Even with the
>high visibility that we have now, that would be a truly fantastic
>feat of marketing.
>
>Even if it takes us only a week to reach 100,000, which I find
>difficult to believe, during that week the spammers will be
>desperate to shoot us down. They know that their days may be
>counted if they don't shoot us down while we're small.
We expect/hope to launch with a significant userbase, and there are
ways to increase that number (by allowing software download before the
network launch date, for instance), but what you're failing to take
into account is the fact that when we release a client, spammers will
still have to develop zombie versions of the client before they can
take over our network. That will take time.
>Every zombie machine will emulate ten frogs at the very least. More
>likely there will be far more frogs emulated by every zombie machine.
The current idea for verifiable identifiers in the network is a secure
hash of the peer's IP address (without a port). Since we'll always make
sure that a node isn't already in the network before allowing it to
join, only one IP address may join the network at a time. Of course, a
computer with many network interfaces could still join once for each
IP, but that's not a big deal because there is a low upper-bound and a
large cost for that loophole.
> > The ban ability could be used if
> > thousands of zombies all try to submit a fake spam report.
> What is a fake spam report? Why wouldn't they submit convincing
> reports?
Spoofed spam could be used by spammers to simulate a DDoS botnet using
our network, and attack whomever they wanted to--that is, unless we
implement a review process for spam submissions and scripts, which we
are planning to do. If they submitted convincing reports, they'd only
be able to attack sites that look and act like possible spamvertised
sites (porn sites and such). They wouldn't be able to use it to attack,
say, Slashdot, because we'd see right through that attempt when
reviewing spam submissions.
> In our thread where captchas were discussed somebody claimed that today
> captchas can be parsed efficiently with OCR technology.
This is true only for very weak captchas. Strong captchas cannot be
parsed accurately with current OCR technology.
> You have not yet convinced me, but please keep trying.
I don't feel a need to convince you. A thousand to one is clearly
hyperbolic.
Regarding the Byzantine Generals problem, it doesn't really apply here.
All interactions with the network will be *verifiable*. We aren't
dealing with inconsistent votes here, as in the Byzantine problem,
which are completely unverifiable as true or false, because there is no
true or false vote. However, if we tell a node to store data in the
DHT, and it doesn't do it, or it stores it incorrectly, we'll know, and
be able to take action against it. If we tell a node to route a message
for us, and the message never gets there, we'll never receive an
acknowledgement of receipt, and we'll know the node didn't do its job.
There aren't very many possible interactions with the network, and all
of them are verifiable. Nodes which do not behave as we expect them to
will be shunned. This is the same core philosophy of Bittorrent, which
works very well despite the best efforts of half of the internet.
Regarding distributed captchas, Spy's description is the one that we've
been pondering. When a node A receives a request from node B and wants
to perform a turing test on the sender, the node asks ten (or so) nodes
in the network to generate a captcha of a single character. It compiles
the ten images into a single image (which allows it to, for instance,
remove spacing between the characters), and then sends the image to
node B. The response is sent back, and node A then sends the individual
characters to each of the ten nodes that submitted the captchas and
requests verification.
If a bad node consistently rejects verification of captchas that it is
asked to verify, that's good! That makes it easy to identify the bad
node and remove it from the network. If a node consistently gets a low
percentage of captchas completed, that's good! That makes it easy to
identify OCR attempts on the captchas, since they generally have low
success rates.
Here's how we take advantage of that. From here on out is off the top
of my head. When a captcha is failed, node A then tests both sides of
the captcha mechanism with some amount of probability (randomized so
that nodes don't know which captchas are real and which are tests):
1) Test node B. To do this, node A generates a captcha by itself and
sends it to B. If B fails again (consistently), then B is at fault.
Discontinue communications with B.
2) Test the nodes used to build the captcha. To do this, node A
requests captcha images from the same nodes again, and tries to pass
the captcha itself (by asking the local user to fill it out). If the
same nodes fail the captcha again (consistently), discontinute
communications with those nodes.
Notice that discontinuing communications with the nodes is a
distributed greedy heuristic that eventually removes bad nodes from the
network and doesn't allow them to do anything on the network. At the
same time, it isn't abusable, because each node decides on its own who
to stop talking to, and it doesn't require an admin.
Emmanuel: why have special spies, when every node in the network can be
a spy? All nodes should always be suspicious of other nodes and try to
figure out if they're bad or not.
Anyway, maybe once this all makes it into an application-level design
draft (the earliest forms of which are still weeks away), then we'll be
able to discuss it more eloquently.
I am very sorry that discussions like this tend to be circular without
much consensus because of the lack of consistency of thought.
Eventually, we'll converge. Let's hope so, anyway!
-nano
EmmanuelGoldstein
am glad that you are taking your time with the design. There were
rumblings here and there of trying to get a design put together in a
couple of weeks, which of course is laughable to anyone who has ever
worked on a complicated piece of software. Please let me know if I can
be of any assistance.
Poltergeist
> when we release a client, spammers will
> still have to develop zombie versions of the client before they can
> take over our network. That will take time.
I wouldn't be surprised if, when we deploy our network, we find
an already operational Okopipi network up and running, started
just to mock us. Their development will occur in paralell with
ours, and mostly they'll re-use our software. They'll just remove
some stuff they don't want and insert a few functions here and
there.
Simple stuff will be enough for them because their first priority
won't be to take us over as a botnet, their first priority will
be to shoot us down, and one strategy will be to disrupt our
services just enough to create a lot of irritation among our
users. Small random disruptions are in many cases very easy
to code.
> Since we'll always make
> sure that a node isn't already in the network before allowing it to
> join, only one IP address may join the network at a time.
http://en.wikipedia.org/wiki/Network_address_translation
A few years ago I read in several places that all of AOL's
customers were behind a single IP address.
> This is the same core philosophy of Bittorrent, which
> works very well despite the best efforts of half of the internet.
That sounds very reassuring. Except Bittorrent isn't de Bruijn.
And its adversaries can't use botnets. But maybe we can hope
that it's a good sign anyway.
> the node asks ten (or so) nodes
> in the network to generate a captcha
[...]
> If a bad node consistently rejects verification
[...]
> When a captcha is failed, node A then tests both sides of
> the captcha mechanism
[...]
> 1) Test node B. To do this,
[...]
> 2) Test the nodes used to build the captcha. To do this,
[...]
With the large number of different security measures that you
plan to scatter all over the place, I really feel that it gets
/far/ more complicated than my solution. Just look at that
captcha complexity.
The only difficult things in my solution are routing and choosing
which node will take care of an object. And they really seem
very simple compared to all those security arrangements.
Routing isn't really difficult, just different. Each small
network, and usually each frog, keeps a list of all the spam
hashes that it is aware of, along with routing information,
that is, information on which two or three of the nearest
neighbors are good routes to the node that is responsible
for that spam (responsible for DHT storage etc). Just the
closest neighbors are listed, not the entire route.
If there isn't room in the table for all the spam hashes,
we can drop the least important spams, since we have reliable
priority data. Low-priority spam will fall below our radar
anyway, so dropping them from the table has no impact except
on some statistical computations.
And very likely dropping them will only be necessary on old
machines.
As for choosing which node takes care of a spam (takes care
of DHT storage etc), I was hoping we'd flesh that out jointly
in discussion, but it seems things aren't going that way.
Just to show that it isn't all that complicated I'll describe
a solution briefly.
Whenever a node sees a spam hash that isn't in its routing
table, the node broadcasts to all its most highly trusted
neighbors, asking them if they can take or delegate that
spam. The neighbors in turn broadcast the same question
to their most trusted neighbors, and so on. Along with the
question, cumulative trust metrics and hop counts are sent.
The broadcast spreads until either it encounters a node
that knows of a path to a node that already has the spam,
or it crosses its own path, or a predefined hop count is
reached. The answer is then returned, along with cumulative
trust metrics and information on remaining free capacity
in the nodes and small networks. The node and network that
are most suitable according to these metrics get the job.
I really don't think this is seriously complicated. And
I feel pretty sure that it's much simpler than having lots
of different security measures all over the place, stuff
like that captcha arrangement. Especially considering that
this routing and election stuff becomes well-contained.
The surrounding code can treat it just like any ordinary
network.
> each node decides on its own who
> to stop talking to, and it doesn't require an admin.
Wow! A de Bruijn graph where nodes individually make their own
decisions on how to route the data??? I'm surprised, to me this
sounds like a contradiction in terms. I'm very curious to see
what you mean.
> Anyway, maybe once this all makes it into an application-level design
> draft (the earliest forms of which are still weeks away), then we'll be
> able to discuss it more eloquently.
I look forward to this.
> I am very sorry that discussions like this tend to be circular without
> much consensus because of the lack of consistency of thought.
> Eventually, we'll converge. Let's hope so, anyway!
Yes, let's hope'!
-- An important thing occurred to me, just in case someone reading
my argumentation should feel that I might be trying to create my
own pet package or slip in complicated stuff with backdoors or
something.
If in the end my proposal should be used, I would insist on /not/
coding it myself.
That's because I view it as the moat and drawbridge around our
castle. I'm claiming that it can replace other security measures.
This means that it should be known to lots of people, scrutinised,
discussed, turned inside out and upside down.
If I only explain my thoughts on it, without writing even a single
line of code for it, then I must make my descriptions readable,
and even some people who aren't involved in coding it, but are just
interested, will read, get to know it, and help spot weaknesses.
nano
> an already operational Okopipi network up and running, started
> just to mock us. Their development will occur in paralell with
> ours, and mostly they'll re-use our software. They'll just remove
> some stuff they don't want and insert a few functions here and
> there.
attempt to develop in parallel, but the likelihood is that they would
be behind us in development, since they'd have to play catch-up each
time we changed something. Also! Just an idea, might not have any
merit: with each new version of the client, we could break the network
protocol. That would force them to redploy their entire botnet each
time we deployed a new version, but if our users were able to download
over the frognet, we would be nearly instantaneously re-deployed
(assuming Okopipi is a very small program). The difference is that
their system is monolithic and ours is p2p.
> http://en.wikipedia.org/wiki/Network_address_translation
I don't think I need to look up NAT on wikipedia; thanks, though. Could
easily be solved by allowing peers to accept spam reports from other
local machines on the network and submit them by proxy. The importance
of only one peer per IP is huge.
> That sounds very reassuring. Except Bittorrent isn't de Bruijn.
Not quite sure what that has to do with *anything.*
> With the large number of different security measures that you
> plan to scatter all over the place, I really feel that it gets
> /far/ more complicated than my solution. Just look at that
> captcha complexity.
You think your network will magically not require security? Good luck.
:)
Firstly, I was simply explaining the idea that Spy had mentioned
previously. I don't think our network even requires captchas. It's just
a security thing that we *could* implement if we felt the need later.
By showing you how something *could* be done, I am attempting to show
you that there are ways of solving your unsolveable problems. And you
have to admit, the distributed captcha idea as I described above is not
only elegant, but ironclad. I hate to toot my own horn, but sometimes I
strike gold. Might not be necessary, but should we need to go down that
road, there it is.
The reason I don't think it's necessary is for the reason that I stated
in my last message: all actions on the network are verifiable. Bad
nodes *cannot* misbehave without detection. The only attack that I am
aware of that can work in this situation is one that Spy came up with,
which is basically a massive distributed simultaneous black-hole
attack, where a large portion of the network (specifically, greater
than the bisection width of the graph, which is about 30% of total
nodes in de Bruijn graphs) suddenly drops out of the network at once.
Even if they managed to split our network, though, we could simply
reset the network. The next day, it'd be back up and running, and we'd
have the IPs of tens of thousands of nodes that had inexplicably died
simultaneously the day before.
> > each node decides on its own who
> > to stop talking to, and it doesn't require an admin.
> Wow! A de Bruijn graph where nodes individually make their own
> decisions on how to route the data??? I'm surprised, to me this
> sounds like a contradiction in terms. I'm very curious to see
> what you mean.
Hm. Sarcasm?
No, nodes that are found to be faulty will be removed from the network,
because its neighbors will eventually just take its part of the network
by force (this is possible in part precisely because of the structure
of de Bruijn graphs--specifically, their node-disjoint backup paths).
Of course nodes can't arbitrarily decide how to route data. Routing is
verifiable.
If F2F networks are such a good idea, why is there *no existing, large
p2p solution that uses them?* It seems to me that you can't possibly
have a single F2F network. You will have many small ones. And you will
have high centrality. For instance, in an effort to connect your
sub-network to a larger F2F network, you'll find the largest
sub-network you can and connect to it. That will lead to a star
topography with a high-risk single point of failure. In general, a F2F
network will likely have very poor resilience and speed. In addition,
it is highly susceptible to social engineering attacks, whereby
spammers can incur the trust of many sub-networks.
Not to mention the low usability. Users have to be able to identify a
trusted entity, which most users will be unable to do (users are dumb).
You'll have less connectivity, which means less spam shared, less
scripts submitted. You have no ability for moderating the network, nor
for optimizing or load-balancing it. And the network topology is
horribly underspecified atm. You still haven't identified a greedy
routing algorithm. Does your system require massive broadcasts to
establish paths to nodes? Imagine if the internet worked like that.
I just don't think you have thought your idea through enough. What
makes it harder for you is that there is absolutely no study to support
that what you are suggesting might even be a good idea, much less
implementable, much less implementable with high
security/resilience/speed. When I read your decriptions, I feel like
you're talking about some big nebulous idea. I need algorithms. I want
to read how it works. I want proof that it's not just a pipe dream.
With ODRI, I can look and say, "There is a paper that demonstrates its
effectiveness. Not only that, but here's Koorde, another p2p network
designed on a de Bruijn network." I can look at DHTs and say, "These
have been proven to be an effective way to store data in a p2p
network." With your ideas it's just a bunch of random thoughts and you
provide no low-level understanding of how the parts interoperate.
Beyond that, however, I am simply ideologically opposed to your
suggestion of an F2F network. I do not like the idea for reasons of
usability and resilience to attacks.
Now, in addition, responding to these giant posts is costing me about
an hour for each one. I'm not prepared to keep it up for much longer. I
suggest that if you want your design to be seriously considered (and I
do promise to consider it myself), then please create a design draft
that explains the workings of the network and the network of networks
*in detail,* preferably with use cases from a user-standpoint to combat
the usability argument. This is nothing more than what I am already
doing for Audrey, my proposed network. Then we can both present our
designs, and they can compete based on merit. The time spent here
debating issues could be much better spent doing real work on designs.
-nano
Kevin Winter
> time we changed something. Also! Just an idea, might not have any
> merit: with each new version of the client, we could break the network
> protocol. That would force them to redploy their entire botnet each
> time we deployed a new version, but if our users were able to download
> over the frognet, we would be nearly instantaneously re-deployed
> (assuming Okopipi is a very small program). The difference is that
> their system is monolithic and ours is p2p.
I strongly oppose this, if only to avoid becoming AOL. Remember how
AOL kept switching around their protocol in the early years of AIM?
Every other client had to constantly release a new version to keep up.
In the end they(AOL) gave up, because there were always people
willing to reverse engineer it.
Besides, it smacks of security by obscurity. I feel that since we
can't guarantee they use our client, we should design the PROTOCOL to
be the robust system we want/need. With the protocol being our
"system", we could care less what client they use. And if their
client doesn't follow our protocol, it won't exist on our network.
Spy der Mann
The IP blocking lists in anti-spam sites. Obviously the IP's used to
send spam are the same ones used for botnets. We block them, and voila
:D
nano
> I strongly oppose this, if only to avoid becoming AOL. Remember how
> AOL kept switching around their protocol in the early years of AIM?
> Every other client had to constantly release a new version to keep up.
> In the end they(AOL) gave up, because there were always people
> willing to reverse engineer it.
>
> Besides, it smacks of security by obscurity. I feel that since we
> can't guarantee they use our client, we should design the PROTOCOL to
> be the robust system we want/need. With the protocol being our
> "system", we could care less what client they use. And if their
> client doesn't follow our protocol, it won't exist on our network.
I think you miss the point.
We would WANT them to have to keep releasing their zombie
client--that's not a negative aspect of this idea. That means they have
to redeploy it to their entire botnet each time, which will cost them
lots of money/time either in bandwidth, or in developing a bittorrent
solution to redistribute it. The point is that it is easy for us to
make some tiny change to break our network code and redeploy, but
redeploying an entire botnet is hard for them.
It's not security by obscurity at all. It's just a way to lessen the
number of zombie nodes on our network by slowing them down.
Of course the protocol should be robust. Of course the protocol is the
whole system. It certainly isn't the client. What can the client do?
That's why modifying the protocol is effective. Modifying any other
aspect of the client would have effect on them.
Anyway, like I said, it's just a possibility. And I have no doubt that
it *would* be effective. But it might not be worth the utility to
develop and maintain.
Kevin Winter
>
> We would WANT them to have to keep releasing their zombie
> client--that's not a negative aspect of this idea. That means they have
> to redeploy it to their entire botnet each time, which will cost them
> lots of money/time either in bandwidth, or in developing a bittorrent
> solution to redistribute it. The point is that it is easy for us to
> make some tiny change to break our network code and redeploy, but
> redeploying an entire botnet is hard for them.
> Of course the protocol should be robust. Of course the protocol is the
> whole system. It certainly isn't the client. What can the client do?
> That's why modifying the protocol is effective. Modifying any other
> aspect of the client would have effect on them.
And if the protocol is the all-important part of this project, why
change it so often? I personally feel that us changing the protocol to
foil them is childish and saying that our protocol isn't good enough
to begin with.
I really like the idea of beating them with completely open-source and
transparent software. Designed correctly, we should care less what
they do - or don't do - with their botnets.
> Anyway, like I said, it's just a possibility. And I have no doubt that
> it *would* be effective. But it might not be worth the utility to
> develop and maintain.
Effective or not, i really don't see changing the protocol as an
effective use of resources. Why don't we instead design the protocol
to be more robust and resistant to byzantine failures to begin with?
Changing the protocol often is a headache waiting to happen...
Whats the point of taking a good while to create a very careful and
strong design, only to modify it every week or more?
master...@gmail.com
Thing. Did you think about the other clients people might want to use?
Doing this sounds like a form of vendor lock-in - splitting the network
into those who can't/won't/don't use the official client (and therefore
the latest protocol) and those that do.
However, the protocol _will_ have to change, and there will be version
conflicts. I think the best way to handle these is the way MSN
Messenger does. The server (in our case any part of of the overall
client recieving connections from other nodes) supports multiple
protocols and the client (in our case any part of client making
connections to other nodes) supports one (this may have to be expanded
to more in our case - nodes the client wants to talk to may have older
versions of the protocol. The two then negotiate a protocol to use,
normally the latest one supported by the client. If the server doesn't
support any of the protocol versions that the client does, it redirects
the client to another server that does.
Spy der Mann
don't work, and would complain. Oh, there's a new patch! What is this,
Microsoft Windows? :-/
And remember we have no site - it would become nearly impossible to
update the software so often, and this could become a weakness that the
spammers would exploit. We want to confuse the enemy, not OURSELVES!
The protocol should stay FIXED, so that people won't have to update
their clients, and our numbers would increase - protecting the network
because our numbers themselves.
There's one thing that worries me, tho. And that's a bot infecting one
of our froggie machines, who would become a spy itself. We must ensure
our customers have their windows patched, or at least block the
vulnerable ports and disable the services commonly used to infect
machines. (Or even better, detect possible intrusions and block the
intruding IP's)
For me this has more advantages than changing the protocol.
kevincw01
and safer. Also, there is very little need for an admin. The other
approach requires an admin(or set of) to "start" the mass opt-out. If
we have these series of trusted circles who only exchange opt-out
scripts and everybody is constantly opting out as they receive the
spam, it's more of a real-time ebb and flow of opt-outs. There really
is no one to attack that would have any affect on the "network".
I look forward to the meeting in 2 weeks where we can debate further
and come to a final design decision. (saturday 6/10/06 2pm PST IRC
UNDERNET, #okopipi)
-Kevin.
Poltergeist
nano wrote:
> responding to these giant posts is costing me about
> an hour for each one. I'm not prepared to keep it up for much longer.
Hey, for once we agree on something! :-)
> I
> suggest that if you want your design to be seriously considered (and I
> do promise to consider it myself), then please create a design draft
Thanks, but now I'm getting convinced that the chances that you'll
choose my solution over your own are practically zero. Considering
that you don't like F2F, and that you demand proof in academic
papers, and that neither of these two factors is likely to change,
in practice you have already made your decision.
Writing a detailed technical specification for an already doomed
proposal would just be a waste of time.
I may consider your decision a bad one, but sooner or later some
decision has to be taken, and in that perspective a bad decision
is better than no decision. And I don't have any /certainty/
that your frog will die in the coming onslaught, I'm just very
pessimistic. Let's hope I'm wrong!!!
And if my pessimism should turn out to be warranted, it's not
the end of the world, we can then analyse statistics, reconsider
decisions, and improve the frog.
When I posted my proposal I expected constructive discussion,
where people might look more closely at certain details together.
The discussion has turned out to be very different from what
I expected.
> This is nothing more than what I am already
> with each new version of the client, we could break the network
> protocol.
While we're still carefully scrutinising the new protocol against
backdoors and vulnerabilities, and wrapping up the last details,
they are already busy copying the source code for our new protocol,
inserting it in their source, and spreading the protocol among
their bots, to be activated at the right moment.
> And you
> have to admit, the distributed captcha idea as I described above is not
> only elegant, but ironclad. I hate to toot my own horn, but sometimes I
> strike gold.
You seem to assume that the problem of spotting sabotage is much
easier than it actually is. So even though I agree with you that
this captcha arrangement is probably unnecessary, I still think
it's worth explaining why spotting sabotage here would be quite
difficult. The same principle applies elsewhere.
Unless I misunderstood the descriptions, and assuming we have the
1:1 zombie/frog ratio that you have mentioned (which you know I
consider extremely optimistic), then on average five of the ten
image-giving nodes will be zombies. Our adversary can then attack
us by letting each such zombie node falsely fail the test once every
15 times, irregularly, at random with a 1/15 chance. The user then
gets a failure on every third captcha even though he answers correctly.
This, together with other similarly annoying sabotage, is likely
to irritate and frustrate many users enough to make them uninstall
the frog and spread the word that it's crap.
Gathering enough statistics to be certain whether such an image-
giving node is faking failures once every 15 times will take some
time, if it's possible at all, considering how some users have
great difficulties with captchas. And also considering that the
zombie will compensate its statistical anomaly by claiming that
incorrect replies are correct.
Your description assumes, very explicitly, that a saboteur node
will be consistent in its sabotaging behavior. That would be an
extremely clumsy saboteur.
> You think your network will magically not require security? Good luck.
> :)
Sheesh, no! It only gets simpler, it doesn't disappear, obviously.
And ordinary security stuff is still needed.
It's a different approach, I have a topology that is designed
from scratch to withstand extreme onslaughts. You have a topology
originally designed for very different purposes, littered with
patches for the new purpose. I have a moat surrounding my castle.
You let the enemy enter in hordes, and you plan to catch them in
traps on the inside, traps that you base on observing the damage
that they do in your castle, and collecting statistics about this
damage.
You argue that my castle is unproven and may fall to pieces. This
is certainly a valid concern, we can't risk that happening. And
I am now convinced that, no matter how solid my castle is, I can
never convince you that it's solid.
> Hm. Sarcasm?
I was genuinely perplexed and didn't know what to make of your
description.
-- The following details are now of only academic interest, but
personally I find them interesting. Don't answer unless you really
feel like it.
> If F2F networks are such a good idea, why is there *no existing, large
> p2p solution that uses them?*
With different requirements you need different solutions. There's
no reason to put up with the disadvantages of F2F unless you have to.
> For instance, in an effort to connect your
> sub-network to a larger F2F network, you'll find the largest
> sub-network you can and connect to it.
Why on earth would you want to connect to a /large/ sub-network? People
in /close/ relationships get the most influence. If you want influence
you'll look for /small/ sub-networks.
If for some reason you prefer large sub-networks, go ahead, but you'll
get less influence on the calculation of spam priority. You'll still
get our opt-out service, it should work just fine, it'll just rely
more on priorities calculated elsewhere.
> In addition,
> it is highly susceptible to social engineering attacks, whereby
> spammers can incur the trust of many sub-networks.
To get influence you have to get our /volunteers/ to trust you highly,
or someone who is highly trusted by our /volunteers/. If anyone along
this chain gives a low rating, the influence is low.
> Not to mention the low usability. Users have to be able to identify a
> trusted entity, which most users will be unable to do (users are dumb).
We can spread the word that they must only join trusted friends,
trusted acquaintances, employers and ISPs.
> You have no ability for moderating the network,
When our volunteers adjust trust settings, that's moderating.
Other types of moderation can certainly be added.
> nor for optimizing or load-balancing it.
As for load balancing, I did mention that information about remaining
free capacity is used when deciding which node takes care of a new
spam hash.
> Does your system require massive broadcasts to
> establish paths to nodes? Imagine if the internet worked like that.
All the descriptions I've seen so far of your solution require
broadcasts for all messages to and from the admins. That's far
more than my network, which broadcasts only for deciding which
node takes care of a new spam hash (a new spamvertiser website
address).
> I feel like
> you're talking about some big nebulous idea. I need algorithms. I want
> to read how it works. I want proof that it's not just a pipe dream.
[...]
> With your ideas it's just a bunch of random thoughts
"A big nebulous idea," "a pipe dream," "a bunch of random thoughts."
Unless my English is very poor, you're using a rather aggressive
tone.
Aren't you asking rather a lot of me? From the beginning it became
very clear that my description was far too long and detailed. It took
forever until anyone read it. That doesn't seem like an invitation
to write something even longer and even more detailed!
I expected at least some constructive discussion about my proposal,
where people would work on some details together. But every single
comment that I've received so far has been clearly aimed at finding
fault. Basically there's nothing wrong with that, ideas need to
be checked critically. But such single-minded fault-searching is
not exactly an invitation to write something even longer and more
detailed!
I've been busy answering these fault-searches, and now, in the
middle of this, suddenly you're complaining, in a rather aggressive
tone, that I have not yet posted a lengthy and detailed technical
specification?
Look, all I did was present a proposal, expecting that we might
have a constructive discussion, looking more closely at certain
details.
Poltergeist
> Actually, I like poltergeist's approach.
Thanks! :-) Aahh, encouraging words! Exactly what I needed! :-)
Spy der Mann
rejected, too.Yes, I did get flamed a bit, but remember, we're not
enemies, we're allies.
The problem I see with your proposal is that friends - or at least one
of them - would need to be online constantly to connect to the network.
Since the number of friends is variable, I don't think this social
friends network can widthstand a lot of churn (read-as: people
constantly signing in and out - and this happens a lot in P2P
networks). Besides - what happens to the lonely person who doesn't know
anyone - at least anyone related to Okopipi? He's isolated and will
never connect.I know, the case is rare, but not everyone with a
computer has a lot of online friends - my point is that your proposal
is limited by social factors and these are impossible to overcome.
Furthermore, what we're testing is networks which have been tested both
in theory and in simulators against enemy attacks. Your friends network
could be the greatest idea in the world, but we won't know if it's as
safe AND practical as you claim it to be, until we test it. But I don't
think that Okopipi would be a nice guinea pig/frog to try your
approach. Let's suppose it works - congratulations, but if it doesn't,
it would be a horrible fiasco. To put it bluntly, we cannot allow
Okopipi to rely on the unproved theory of one single person - compared
against the ph.D. papers of the deBruijn networks, your solution has no
theoretical defense.
And yes, excuse Nano, we all are humans and sometimes when we're tired
and underslept we can say things that offend other people.
So, take it easy. I'm sure the de Bruijn model will work.
Spy der Mann
TOP of the de Bruijn network. But that would be something to talk about
later, when we have v1.0 working.
nano
Like I said, I'm not married to it, but whether you happen to like it
or not, changing the protocol would be an effective way of combating
zombie nodes. I don't think we really want alternate clients on our
network (because we don't want to be liable for what they might do),
and with a client application that is small (say <200K), and that could
be updated over the network immediately after a change, you wouldn't
have to go to a website and download a patch each time. Additionally,
the change to the protocol doesn't have to be a complete rewrite of the
protocol (of course not!). The protocol would still be based on the
same routing algorithm, etc. We'd just switch some data in the headers
or something, or add garbage data, or move the header to the end of the
packet. An easy change that would immediately break all zombie nodes.
And we could do it over and over again.
Just because protocol changes are bad in other situations doesn't mean
it'd be bad in this situation. Don't worry, it leaves a bad taste in MY
mouth as well. I'm just saying that it could be done.
kevincw01:
>Actually, I like poltergeist's approach. It is simpler to implement
>and safer.
I believe that it is harder to implement, and more dangerous.
> Also, there is very little need for an admin. The other
> approach requires an admin(or set of) to "start" the mass opt-out.
Actually, both require someone to approve the opt-out. But in our case,
the number of people who can approve an opt-out is tiny, whereas in
Poltergeist's system, anyone who has someone's trust can approve
opt-outs.
> If we have these series of trusted circles who only exchange
> opt-out scripts and everybody is constantly opting out as they
> receive the spam, it's more of a real-time ebb and flow of opt-outs.
There may be more opt-outs, but that's because there would be thousands
of duplicate opt-out scripts written by thousands of people. Not only
is that inefficient use of time, but it's also inefficient from a
bandwidth perspective. Wouldn't it be better for admins to choose the
best opt-out script and use that for everyone instead of a thousand
inferior scripts?
> There really is no one to attack that would have any affect on
> the "network".
If they can't find the admins in the network, then they can't attack
them either. Don't set up straw-men and then knock them down.
-nano
Spy der Mann
> and that could
> be updated over the network immediately after a change, you wouldn't
> have to go to a website and download a patch each time.
Yeah but what happens if a client doesn't connect when the protocol is
changed?
Additionally,
> the change to the protocol doesn't have to be a complete rewrite of the
> protocol (of course not!). The protocol would still be based on the
> same routing algorithm, etc. We'd just switch some data in the headers
> or something, or add garbage data, or move the header to the end of the
> packet.
> An easy change that would immediately break all zombie nodes.
> And we could do it over and over again.
If the zombies use our source code, I doubt that'd have any effect on
them anyway... All they need is a genius programmer who will analyse
our source code, and adapt some intelligence into them. So I really
don't see how we could be ahead of them, if we both use the same source
code - unless we use the old security-through-obscurity trick, and we
know that doesn't work. It's like if we implemented the DRM used in
DVD's... the client has the password, so the client can work around it.
After all, remember that they're hackers - they're experts at finding
holes and secret stuff in code.
kevincw01
>
> -nano
But I loooove straw-men!
nano
have a strong a priori *bad mojo* about your design, which I know is
detrimental to rational discussion.
Spy der Mann summed up the main objections to your idea very succinctly
above. Really, it's less about whether it will work, and more about
whether we can *know* it will work. That is why I requested proof in
the form of a detailed design document, which I know is asking a lot.
I feel that you misunderstood much of what I was trying to say in my
last post (which, again, is my fault for not being clear), so instead
of replying section by section, I'll just say that much of the
ridiculousness you saw in my arguments is because I didn't mean them
the way you read them.
Regarding the distributed captchas, you seem to have looked at the
simplest method possible and argued against that. In the case of
inconsistent failure/success, it would be a simple matter to keep
statistical records for each node, along with margin of error based on
a high confidence. You can measure users' success rates in the same
way, and compare them with the rates against a particular node for that
user, and take action if the rates for a particular node fall outside
the margin of error.
Now just a few direct replies:
> And I am now convinced that, no matter how solid my castle is,
> I can never convince you that it's solid.
Not just by writing posts, no. I require evidence, not just prose. I
wouldn't expect less of anything else.
> All the descriptions I've seen so far of your solution require
> broadcasts for all messages to and from the admins. That's far
> more than my network, which broadcasts only for deciding which
> node takes care of a new spam hash (a new spamvertiser website
> address).
Firstly, there's no reason to broadcast a message TO an admin.
Secondly, the only broadcasts necessary FROM an admin are for opt-out
notifications. Maybe on the order of twenty broadcasts a day? If you
have to broadcast each time you have a new spam report, that's N * M
broadcasts per day, where N is the number of nodes in the network and M
is the average number of reports per day.
Lastly, I'm sorry that I name-called your idea at the end, calling it a
pipe dream, etc. I would like nothing more than if it turned out to be
the Holy Grail of secure p2p networks. My little mind just can't
comprehend that. I'm sure many physicists had trouble with Einstein's
theories of relativity as well, until they saw the math and checked the
proofs.
masterdriverz
> I'm against this idea. Suddenly people would wonder why their froggies
> don't work, and would complain. Oh, there's a new patch! What is this,
> Microsoft Windows? :-/
Did you read my post? The whole point was that frogs that supported
different versions *would* work together. To have a working design
software *has* to evolve. That means new versions.
> And remember we have no site - it would become nearly impossible to
> update the software so often, and this could become a weakness that the
> spammers would exploit. We want to confuse the enemy, not OURSELVES!
Aside from the fact that there are tons of proposed ways of
distributing the client on the groups alone, the point of my post was
thatr people *wouldn't* have to update all the time. Simply updating
the hosted version on most sites would ensure that the latest uptakers
have the latest (and most secure) version of the client.
> The protocol should stay FIXED, so that people won't have to update
> their clients, and our numbers would increase - protecting the network
> because our numbers themselves.
Realistically, you can't hope to have a fixed protocol. As I said
before, software evolves. What happens if there's an exploit in
Okopipi? I'm not argueing for constant updating, just for having the
mechanisms in place to allow updating.
> There's one thing that worries me, tho. And that's a bot infecting one
> of our froggie machines, who would become a spy itself. We must ensure
> our customers have their windows patched, or at least block the
> vulnerable ports and disable the services commonly used to infect
> machines. (Or even better, detect possible intrusions and block the
> intruding IP's)
While I commend your attitude, you *can't* do everything for everyone.
Quoting off of the main page of the wiki: "This project aims to become
a distributed replacement of Blue Security's anti-spam software, which
does not depend on a centralized server."
We cannot spoon-feed thousands of users to keep their machines updated,
botnet-free and devoid of virus/malware. There are hundreds of other
services and programs that do. We should stick to a simple goal.
Enforce the rule that spammers should not send unsolicited email to
people on a DNIR.
HumanChimp
> Spy der Mann wrote:
> > The protocol should stay FIXED, so that people won't have to update
> > their clients, and our numbers would increase - protecting the network
> > because our numbers themselves.
>
> Realistically, you can't hope to have a fixed protocol. As I said
> before, software evolves. What happens if there's an exploit in
> Okopipi? I'm not argueing for constant updating, just for having the
> mechanisms in place to allow updating.
I would agree with Spy der Mann that the protocol should remain fixed,
and you have a valid point, masterdriverz, that (although hopefully
not) there could an exploit in the Okopipi protocol which could allow
for something unwanted to happen.
However, when i mean fixed, (and i think this is what Spy der Mann
meant aswell, but correct me if i'm wrong) i mean that it won't be
constantly adjusted and changed just for the sole purpose of causing
the spammers to have to re-write their scripts every week or so. Of
course if there is an exploit it will need to be fixed, and therefore
it would be stupid not to change a faulty protocol, but don't change it
to keep the spammers one step behind. The protocol should be strong
enough so that even if the spammer knows it like the back of his hand,
it shouldn't make it significantly weaker.
Poltergeist
> Poltergeist: I do apologize for my tone.
OK.
> Really, it's less about whether it will work, and more about
> whether we can *know* it will work.
I think we can't know anything beforehand. The onslaught that
we can expect is uncharted territory. But I certainly understand
your reasoning.
> Firstly, there's no reason to broadcast a message TO an admin.
Interesting. I'll read your specifications with great interest.
> I'm sure many physicists had trouble with Einstein's
> theories of relativity as well, until they saw the math and checked the
> proofs.
Wow, what a comparison! Heh, I wish!
Poltergeist
> we're not
> enemies, we're allies.
Yes, and I'm glad we are.
> Besides - what happens to the lonely person who doesn't know
> anyone - at least anyone related to Okopipi?
Just like ISPs can set up customer networks with up to 32,000
nodes, so can anyone, including us, if we want. Members of
such networks get full service, the only difference is they
have weaker votes when choosing which spams should be processed.
> Furthermore, what we're testing is networks which have been tested both
> in theory and in simulators against enemy attacks.
I doubt that any network architecture has been tested under
the extreme onslaught that we are likely to see.
> Your friends network
> could be the greatest idea in the world, but we won't know if it's as
> safe AND practical as you claim it to be, until we test it.
Although I now consider my proposal dead, I'm thinking that
I might whip up some kind of simulation of it, to satisfy
my own curiosity after all these discussions.
> To put it bluntly, we cannot allow
> Okopipi to rely on the unproved theory of one single person
It was never my intent that it be approved on /my/ say-so.
I expected a discussion where opinions, ideas and proposals
would meet and lead to some kind of community opinion.
> And yes, excuse Nano, we all are humans and sometimes when we're tired
> and underslept we can say things that offend other people.
Indeed this can happen to us all.
Poltergeist
> whereas in
> Poltergeist's system, anyone who has someone's trust can approve
> opt-outs.
For sensitive stuff like scripts, lots of trusted people must check
the script, with time enough to point out flaws, and then vote.
> there would be thousands
> of duplicate opt-out scripts written by thousands of people.
Come on, nano, that's ridiculous! Be serious!
Don't set up straw-men and then knock them down.
> Don't set up straw-men and then knock them down.
Hey, once again we agree on something!
Jlew
you need a server of some type (or some way) for the computer to find
another computer on the network? I mean you need some way to get in
the network, like wouldn't you need some server to call up and to get a
list of computers to talk too and find out whats on the network? I
just think that if we have a computer acting as a server, what stops
the spammers from attacking that server? How would that work?
Spy der Mann
> > Specially
> > because not everyone is online at all time.
>
> People will change this if it will reduce their spam problem.
by raising their electricity bills?
>
> The trust networks don't need to be connected day and night,
> it's enough that the online times of the computers overlap for
> a while once every day.
I never said day and night. I said at the same time. What will happen
when many people of a social network go offline? Can you handle the
bandwidth load? Remember there are no supernodes here.
> This won't be suitable for people on shift hours and people
> in very different time zones. These groups can have one or
> more members leaving their computers running day and night
Isn't that asking too much of them? And finally, the Okopipi team can't
have many friends in the world, so this means that the chain of data
would slowly propagate through the small social networks, introducing a
terrible lag.
...
> In the end they vote. Okopipi will only execute scripts that
> have been approved by a sufficiently large number of script
> checkers
Then again, this introduces more difficulties. Not only people will
have to leave their computers on, they'll also have to check the
scripts? I find that MUCH MORE annoying than having to check through
e-mails, plus this voting system would introduce a serious
vulnerability: Again, the botnets.
The botnets could emulate a GIGANTIC network of friends, and each could
emulate the votes and pass them to innocent people, turning Okopipi
into a giant botnet where the scripts could be used to harm third
parties or download even more viruses.
To put it simply, unregulated voting is a fatal mistake.So we have to
discard voting to use only approved scripts.
Another issue you forget is social engineering. Phishers do this well,
and with this, spammers could use (through bots) social engineering to
gain trust and let people connect to them.
Think about this. "The Okopipi fan club of California". A spammer joins
in and gains access to the subnet. Or what if a spammer FOUNDS the
club?
Furthermore, how do you know that the computer you'll connect to is
_REALLY_ your friend Joe and not a bot in disguise? You'll have to
introduce public keys for validation - and even this could be a serious
problem. Imagine a spammer using keyloggers / etc to find info about
your friends, and sending you a mail with the message: "I lost my
private key, here's the new one". Who would you trust? Do you think the
average user has time to visit whois services (or does he even KNOW
those services?) and find out if what his friend said is really true?
So, don't depend on e-mail, but on services like MSN or AIM (which then
would become a target for DOS attacks).
All the realtime social peer to peer networks (i.e. chat) depend on
connecting to a central server or directory to connect to your friends.
Even Jabber, the most recent, depends on supernodes for that.
And remember that one of the things that we want to do is to make it
simple for users to work together. And e-mail is definitely not the
answer, for one SINGLE reason: Anonymity. Blue Security users could be
threatened by spammers, saying: "You are a member of blue security".
But with a social network, a spammer can send you a cease and desist
letter: "Hello, Mr. Anderson. Yesterday you conspired with your friends
Jimmy, John, Steve and together you sent a fake report to the website
www.something.com, causing network problems. We have proof, and
attached is the netlog. Unless you stop using Okopipi, a lawsuit will
follow."
In short, I think the approach you propose is non-feasible because of:
* COMPLETE lack of anonymity
* High susceptibility to social engineering
* Lag and bandwidth problems for modem users (you can't guarantee
optimal network connections with a social network)
* Lack of scripts security
* making first contact with your friends is an unsolved problem
* Having to review scripts manually and raise your electricity / phone
bills
* the need of authentication for your friends' messages if by mail, or
the risk of chat services getting DDOS'ed if by instant messaging
In conclusion, the de Bruijn network solves ALL these problems but one:
Botnet infiltration. And it solves that problem in a much more graceful
way than your proposal, by keeping structural restraints on the network
topology.
Spy der Mann
the world with a program to find currently active nodes in the network.
nano
> Jlew: We plan to use an approach like Gnutella's, have many servers in
> the world with a program to find currently active nodes in the network.
That's the first I've heard of it. I thought we were gonna use a
bootstrap list of nodes. :)
Spy der Mann
have to be many of them to avoid a DDOS attack. I think that what we're
trying to do is a first for the whole history of the internet - how can
we have a resilient network of websites?
kevincw01
the latest list of nodes. There are many web-based usenet viewers to
act as "the websites" such as google groups. Let the spammers DDOS
google groups...hehehe, right....
Spy der Mann
wouldn't like having my IP listed in a usenet post...
Spy der Mann
must be authorized by the admins before taking any action.
Kevin Winter
> However, when i mean fixed, (and i think this is what Spy der Mann
> meant aswell, but correct me if i'm wrong) i mean that it won't be
> constantly adjusted and changed just for the sole purpose of causing
> the spammers to have to re-write their scripts every week or so. Of
> course if there is an exploit it will need to be fixed, and therefore
> it would be stupid not to change a faulty protocol, but don't change it
> to keep the spammers one step behind. The protocol should be strong
> enough so that even if the spammer knows it like the back of his hand,
> it shouldn't make it significantly weaker.
This was exactly what i was thinking. Thanks for taking the words
right out of my mouth and articulating them so beautifully :)
Kevin Winter
>
> There's a problem with that... usenet posts cannot be erased. I
> wouldn't like having my IP listed in a usenet post...
Thats not true. On alt.sysadmin.recovery (to name one) (viewed
through google), MANY users request their messages not be archived,
and are removed after a period of time (something like 7 days. The
harder problem is authenticating a correct list (pgp signing?)
kevincw01
Scott Mansell
ok, then use my IP. It's dynamic :)
Same here, and i've got a linux box setup as a firewall.