Strange GZip behaviour of App Engine and Cloudflare - both

[Kaan Soral]

While I was inspecting the performance of my app, I decided to manually send content as zipped, I quickly discovered the "gzip" encoding, and I was glad to find out that pretty much everything was gzipped naturally, thanks to modern browsers and the app engine architecture

However, while testing my iOS app calls, which are from webkit, I discovered that the content received was gzipped, yet with 1.00x compression, while the same content is always sent with 6.25x compression - so basically there was something wrong on iOS - and my performance concerns was for mobile, so basically things didn't work

Here are my observations: (a 130kb call)

1) Naked app engine calls get gzipped at 6.25x for iOS - good (26kb gzip)

2) Cloudflare wrapped calls get gzipped at 1.00x for iOS - very bad (130kb gzip ?!)

3) Both Cloudflare and naked app engine calls get gzipped at 6.25x for web/chrome - good (26kb gzip)

4) Naked app engine calls are not gzipped with curl with compression flags on - bad (130kb raw)

5) Cloudflare wrapped app engine calls get gzipped at 6.25x from curl - good (26kb gzip)

As you might have noticed, things are pretty chaotic, on two levels

i) I think the 1.00x gzip situation is a bug of cloudflare, yet they couldn't reproduce it, as they test things with a simple curl request - they claim cloudflare re-gzips everything if it's not gzipped already - which conflicts with (2)

ii) I think the responsive/black-box gzip logic of app engine architecture is complicating things a lot, as observed from (4), the data sent to curl isn't a gzip, although curl requests with gzip encoding flags in this trial

Anyway, I just wanted to share my findings with the community, my solution was to use app engine for mobile calls, which is something I was meaning to do, so things worked out for me, yet I couldn't uncover the cause of (2)

[Nick]

Do you have a facility to look at what headers are being sent from each of these user agents?

Accept

Accept-Encoding

Accept-Charset and possibly

Accept-Range

May all affect whats going on at these stages. In particular, how accept is handled (both the order of elements and the quality (q)) is a roll of the dice. My guess is that your ios client library may be sending something unexpected which you can tweak to get cloudflare to work as it normally would (and possibly they can improve themselves). For example, maybe if it doesnt have accept-encoding of gzip or compress, it might just gzip without compression given the stream is already gzipped from destination.

[Kaan Soral]

It seemed only the User-Agent and the Accept */* header was there, as I inspected with the Safari Inspector

At one point, I also tried manually adding Accept-Encoding with gzip, however the behaviour remained the same