Anti DoS patches in next XT release and future research directions

[Mike Hearn]

The next XT release will include a few patches intended to improve resistance to DoS attacks.

It came to my attention that some people have been confused by others about what these patches actually do. So this email explains. Let me know if anything is unclear.

getdata confusion fix

There is a known issue in Core 0.11 that lets attackers confuse the node into never downloading transactions, thus blinding them to a payment. This can be used to make double spending easier.

This patch by Tom Harding fixes the confusion issue.

 

getdata timeout fix

Bitcoin waits too long before giving up on a peer that isn't feeding it a transaction it asked for. Tom did some calculations that show the timeout can be much lower. So this patch lowers the timeout from two minutes to 20 seconds. ~99% of all transactions arrive within 20 seconds so additional load due to retries should be minimal.

IP prioritisation when full

tl;dr - if a node runs out of resources and has to pick, it prefers to service regular peers than peers behind Tor. This reflects that attackers require Tor/similar, whereas regular users may merely prefer it. 

If for some reason you don't like this or it's buggy, you can disable it with the -disableipprio flag. But you shouldn't: the patch encodes common sense. If you don't agree yet, read on .....

Longer explanation:

When Bitcoin was new, it was realised that you could cause a node to run out of memory by just connecting to it over and over again. Each connected peer requires some bookkeeping data in RAM, and unlimited connections + limited RAM = boom.

So in 2011 Gavin added a limit of (by default) 125 inbound connections. Once a node has accepted this many connections it refuses any more.

This helped avoid node crashes but converted the problem into a different one. Now it's possible for someone to literally "fill up" the network by connecting over and over to every node from a wide range of IP addresses. Once every node is full, any wallet that's newly started will find itself unable to connect to the P2P network.

This is obviously a big problem for mobile P2P wallets that only run for a few minutes at a time, but it can affect users of XT/Core too if their computer isn't always on.

One easy way to mount this attack is to use Tor. Tor gives an attacker many open proxies in different IP ranges for free, and there's no way to get caught if you use it for attacks. Ideal! 

It has been said by at least one Bitcoin Core developer that this attack is very hard to pull off, so it's a theoretical problem. Actually I prototyped it as part of working on the fix. It's about 60 lines of code and took me about an hour to write. So the claim that it's difficult is dubious.

Given this, it is no surprise that someone else has done it too. We have seen this attack be mounted against one of Gavin's XT nodes. He woke up one morning to discover it was no longer accepting P2P connections, because every slot was jammed with Tor exits.

This patch fixes the issue. It adds code that only runs when the node is full. As nodes are not supposed to get full unless there's an attack, this code should ideally not run, or hardly ever run. If a node reaches its -maxconnections limit instead of rejecting all new connections, it calculates a priority score for each connection and if there are any lower than the new inbound connection, that lower scoring peer is disconnected to make room. And it adds a starting rule that gives Tor connections a lower score than clearnet connections. Hopefully there will be many other heuristics added over time.

The result is:   if and only if your node has run out of resources, and it has connections via Tor, then it will kick out the Tor connections one at a time to make room for non-Tor users.

This reflects the reality that Tor is much more attractive to attackers than real users:  nobody is going to DoS bitcoin from their home internet connection but people use Bitcoin from such connections all the time.

Network partition risks

My patch is a tiny first step towards fixing a long standing problem with the design of Bitcoin's DoS protection system:  it works by attempting to ban IPs that engage in "misbehaviour". This has two problems:

1. It's possible to DoS a node without triggering the misbehaviour rules
2. It assumes 1 IP = 1 person

The latter assumption isn't valid for lots of users, like users on mobile phones, at hotels, at conferences, at some universities ....... and users behind Tor!

One of the misleading things I've seen a Blockstream employee say is that the Tor prioritisation patch "risks network partition", which is a fancy way of saying users behind Tor might get disconnected entirely from the main network.

I have a problem with this argument for a couple of reasons.

The first is that anyone can already trigger such a partition. All you have to do is connect to each Bitcoin node from every Tor exit, and then "misbehave". The exit will then get banned for 24 hours. New connections are then no longer possible. Existing nodes will keep their connections intact, but eventually people will have to restart their nodes, and then they'll end up banned too. This results in a kind of slow motion network partition. The best fix for this is to replace the notion of banning IPs with something else .... like a more advanced form of priority.

The second is that it suggests Bitcoin should just not care about the attack I outlined above. The guy who has been saying this also believes that non-Bitcoin-Core P2P wallets are a bad idea, which is consistent with Blockstream's vision of Bitcoin as a kind of clearing network between quasi-institutional entities. So, no surprise that he doesn't care about attacks that affect mobile P2P users.

However, the Bitcoin XT project does care about them.

The third reason is that to force Tor users to be disconnected and make a partition unique to this patch, you would have to flood the network from non-Tor IPs. As you are probably breaking the law by doing that, you need to find some other shield. The framework is general so if someone does this via some other proxy network, we can give those networks even lower priority than Tor. Problem fixed. 

Then you'd have to use botnets and the like, but I know from my time fighting hackers at Google that this is much harder to pull off than using something like Tor. Attacks getting harder, riskier and more expensive? That's progress.

Future research ideas

The new framework is very basic. There are many other heuristics we can add to make it work better, and eventually remove the Tor specific logic entirely.

One idea is to raise the priority of nodes that are doing useful stuff, like relaying us data.

Another idea is to extend the P2P protocol so clients that don't have long lived connections and can't provide services to the network (i.e. phones) can gain priority by proving they own bitcoins. It makes sense that a user who has 500 BTC in a 12 month old saving wallet should take priority (if need be) over a user who has no bitcoins at all.

Another quick heuristic is to do dialback on connect. One issue with connection prioritisation is that we have to decide fairly quickly and with low resources whether to service a new connection or not. A quick check if the connecting IP is accepting port 8333 would identify not only Tor but all proxy and botnet services    (running custom software on botnets is much harder/riskier than using their predefined services). Of course it'd also identify mobile/NATd users. So we'd want to combine it with the new protocol above and get wallets to implement it first. 

Obviously, in a mostly anonymous system like Bitcoin, any heuristic can be gamed by an attacker who is willing to spend enough resources to simulate real users. All we can do is raise the costs.

So with those ideas combined we'd have a pretty nice general framework that doesn't need any hints about specific IP ranges anymore. But it'd take a lot more work.

Thanks for reading this far! Any questions, fire away.

[Chris Wheeler]

Could these peer prioritisation rules also be used to help with block propagation for miners. I believe one of the problems is that when a miner finds a block, they have to send it ASAP to all their peers, and currently this is done in parallel (see http://rusty.ozlabs.org/?p=509). If we were to send it sequentially by priority block propagation should be faster, reducing orphan rates, I think? We'd benefit from factoring bandwidth/latency into priority if that is possible.

[Mike Hearn]

Maybe, I haven't thought about it much. Interesting idea though.

I think the next bottleneck after removing the 1mb limit would be propagation speed. It's possible that we would see miners setting their soft limits lower than the hard limit to reduce orphans. Gavin has done a lot of research into this with IBLTs and things.

My inclination is to say, you know, we have this Bloom filtering protocol already at the P2P layer. If you set a 0xFF filter then what you get is every block being relayed as a set of hashes. Transactions you don't have yet are pushed out automatically. Maybe we should just teach Core how to reassemble a merkle block into a CBlock and then see if it helps at all. The nice thing is, you can deploy such an upgrade on the current network incrementally. 

[Andrew LeCody]

Thanks Mike, great info. I have two questions:

Is the prioritization code simply looking up the list of Tor exit nodes?
How is the prioritization handled with regards to Tor hidden services?

--
You received this message because you are subscribed to the Google Groups "bitcoin-xt" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcoin-xt+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Hearn]

Is the prioritization code simply looking up the list of Tor exit nodes?

For now, yes. It has a compiled in list too, though obviously that degrades with time.

 

How is the prioritization handled with regards to Tor hidden services?

If you run a node as a hidden service then at this time you don't have any DoS protection at all via any mechanism: you cannot ban or prioritise IPs in onionland.