CloudFlare moving into Chinese market

[Ox Cart]

http://techcrunch.com/2014/11/24/cloudflare-will-offer-a-local-version-of-its-web-security-service-in-china-in-2015/

I suspect that this will break domain fronting with CloudFlare in China.

As a general thing, international infrastructure providers opening Chinese operations seems bad for collateral freedom.

Cheers,
Ox

[Rishab Nithyanand]

Also: https://en.greatfire.org/blog/2014/nov/china-just-blocked-thousands-websites

Does this mean that meek could be in trouble at some point? I wonder if EdgeCast and other CDNs will change their T&Cs to disallow hosting of freedom projects.

[David Fifield]

On Tue, Nov 25, 2014 at 06:20:39AM -0600, Ox Cart wrote:
> http://techcrunch.com/2014/11/24/
> cloudflare-will-offer-a-local-version-of-its-web-security-service-in-china-in-2015
> /

>
> I suspect that this will break domain fronting with CloudFlare in China.
>
> As a general thing, international infrastructure providers opening Chinese
> operations seems bad for collateral freedom.

It could be bad or maybe not. It depends on how the CDN service works.

If the CDN edge server just forwards uncached requests from itself
directly to the origin server, then yes, fronting breaks because we can
assume that the origin server is IP blocked.

However, there's the possibility that the CDN uses its own faster
private network to forward the request to another edge server, closer to
the origin, and outside the censor's sphere of control. From
conversations I've had with CDN folks, this scenario seems plausible. It
really depends on how the CDN interacts with the firewall, whether it
has some privileged treatment or whether it has to use the same
connection ordinary users use.

It's something we can test, of course. When you front a request through
CloudFront, say, is the edge server used by the client the same host
that makes the request of the origin server?

David Fifield

[David Fifield]

On Tue, Nov 25, 2014 at 11:07:05AM -0800, Rishab Nithyanand wrote:
> Also: https://en.greatfire.org/blog/2014/nov/china-just-blocked-thousands-websites
>
> Does this mean that meek could be in trouble at some point? I wonder if
> EdgeCast and other CDNs will change their T&Cs to disallow hosting of freedom
> projects.

It's not a good sign in general, because it shows that at least one
censor (the GFW) has a pretty large tolerance for collateral damage. But
I don't think it spells the end of domain fronting, for a few reasons.

On collateral damage, see "Why You Can't Read The Atlantic in China":
http://www.theatlantic.com/international/archive/2014/11/why-you-cant-read-the-atlantic-in-china/382883/
"It’s not that The Atlantic or Sony had just uploaded subversive
content .... the Chinese government tried to block certain sites
served up by EdgeCast. But EdgeCast serves up content for tens
of thousands of sites, and the Chinese authorities seem to be
blocking many others as well."
We've learned that the GFW can tolerate breaking a certain number of web
sites in the course of blocking circumvention. However, it doesn't mean
that their tolerance for collateral damage is unlimited.
https://edgecastcdn.net/ is blocked but https://github.com/, which is
also used for circumvention purposes, is still not.

The EdgeCast block was likely in response to the kind of mirrors
GreatFire was cleverly hosting at URLs like https://edgecastcdn.net/00107ED/g/
(there are more at https://github.com/greatfire/wiki). The idea is
similar to domain fronting, in that part of the request (here the path
component of the URL rather than the Host header) is hidden from the
censor under HTTPS, and only the domain is revealed.

The blocking of edgecastcdn.net is by DNS poisoning (click one of the
dates at https://en.greatfire.org/https/edgecastcdn.net for more
details), the mildest of the blocking techniques GFW is known to employ.
If they were really serious about blocking EdgeCast, they would also do
IP blocking and inject TCP RSTs--but they aren't doing that. And I
suggest that they couldn't actually do that. There's a big difference
between blocking the specific DNS name edgecastcdn.net, and blocking the
entire EdgeCast CDN. Blocking just the domain name effectively blocks
the mirror sites, and also any unrelated web sites that happen to
hardcode "edgecastcdn.net". But most, I would guess, sites that use
EdgeCast actually use their own CNAME alias for edgecastcdn.net. Those
sites are unaffected by DNS poisoning, but would be broken if the entire
CDN was blocked. In other words, the collateral damage could have been
greater, but it appears the GFW took an expedient step that also limited
damage.

A very similar event happened at the end of September on Akamai, but it
didn't get as much attention. The special domain a248.e.akamai.net is
roughly the equivalent of edgecastcdn.net, was also used for mirror
hosting, and it was also blocked by DNS poisoning.
https://en.greatfire.org/https/a248.e.akamai.net
The few web sites that happened to hardcode a248.e.akamai.net would have
been broken, but that's not the common way to use Akamai. The common way
is to use your own CNAME, and let Akamai direct the request according to
the Host header (which is exactly what we take advantage of in domain
fronting). GFW didn't block everything on Akamai (which would have truly
enormous collateral damage), they only blocked one specific name.

What makes meek and similar systems more resilient is the domain
fronting trick. We can still direct traffic to our own domain; it just
means that edgecastcdn.net and a248.e.akamai.net no longer work as front
domains--you have to use something else. For the front domain you can
choose any domain that CNAMEs to the CDN, or even a large set of them.
The strategy behind choosing a front is either to choose one that is so
important the censor can't block it without incurring too much damage,
or to rotate among a selection of lesser ones. So for example, we could
choose some other big domain hosted on Akamai, one the GFW is reluctant
to block, rather than the generic a248.e.akamai.net.

In a way, the fact that a domain name was blocked is good news for us.
It means that at least one censor still prefers fairly crude methods of
blocking. The edgecastcdn.net situation was pretty much ideal for
website fingerprinting–style attacks, after all: there's only a small
number of mirrored files, some with exact file sizes known in advance.
The fact that GFW decided to just suck up the collateral damage, rather
than deploy a DPI-type classifier, says something about the relative
attractiveness of different classes of attack to the censor.

I think this episode is a good example of the benefit of a slight change
in thinking regarding circumvention. We should think about systems not
in terms of a binary "can it be blocked?", but rather ask the question
"what does it *cost* to block it?" We can say that blocking domain
fronting costs the blocking of a domain; some censors can afford it,
others cannot. In this way we make blocking resistance at least somewhat
quantifiable. (Of course it's more complicated than that; there are
other potential attacks; but just take the one with the minimum cost as
representative. So far it appears that blocking a domain has the minimum
cost.)

The attack you mention, that of a censor pressuring a CDN to kick out
users like us, is a real one. As we assume that the censor controls
larger and larger sections of the network, it becomes harder and harder
to defeat. It's important to remember that the cost model cuts both
ways, though: CloudFlare needs China and China needs CloudFlare. But as
for which one needs the other *more*, who would prevail in a conflict;
that is an economic and political question, and that's how we should
think about it. Not only that one side could unilaterally deny service
to the other, but what incentives exist on all sides.

David Fifield

[Ox Cart]

David,

Thanks for the detailed and very insightful analysis!

It's important to remember that the cost model cuts both
ways, though: CloudFlare needs China and China needs CloudFlare. But as
for which one needs the other *more*, who would prevail in a conflict;
that is an economic and political question, and that's how we should
think about it. Not only that one side could unilaterally deny service
to the other, but what incentives exist on all sides.

I think this cuts to the heart of it.  Depending on implementation details, CloudFlare moving into China won't necessarily result in domain fronting being immediately broken, but it creates new incentives and worse, legal requirements, for CloudFlare to break it.  Irrespective of whether the Chinese CloudFlare contacts origin servers directly or via its own private network, given that "Regulations require customers to hold various state approvals before they can be on the CloudFlare China network", if CloudFlare's initial implementation happens to provide a loophole that allows unapproved customers to become available in China, I imagine that the Chinese government can and would request that this loophole be closed to stay in compliance with regulations.

What makes the problem of providers like CloudFlare and Amazon moving in-country particularly pernicious is that it provides an escape hatch for "legitimate" sites.  Collateral freedom requires that it be difficult or impossible for censors to segregate desirable (e.g. economically advantageous) traffic from undesirable traffic in order to block the undesirable.  By having a presence in-country, CloudFlare can offer those customers who have a significant economic stake in China the ability to continue to reach the Chinese market even if the regular CloudFlare platform has been blocked in China.  Consequently, this establishes a regime in which China can regulate international sites much like it already regulates Chinese sites.  Ultimately, I fear that this will lead to a balkanized Internet in which China can maintain a Chinese version of the Internet that is cut off from all international links except for those deemed economically desirable.

Assuming that many of CloudFlare's highest paying customers are commercial organizations that would be able to obtain Chinese approval, it's not clear to me that there's any conflict between CloudFlare's interests and China's interests.  In fact, if China becomes more aggressive about blocking international links, that would seem to benefit providers like CloudFlare and Amazon who enjoy the privilege of operating in-country and can now charge extra for offering access to China.

An economic conflict could be manufactured, for example by enough of CloudFlare's paying customers (or their customers) objecting to such practices on principle and boycotting CloudFlare, or by the U.S. or E.U. passing some sort law prohibiting companies that operate in their jurisdiction from actively supporting censorship abroad (conceptually similar to the Foreign Corrupt Practices Act), but both such outcomes would require significant political mobilization and are far from assured.

Sorry if I'm being overly pessimistic here, but I'm just not seeing the upside to these sorts of developments.