Google Cloud Storage TTFB increased with gzip

[Denys Koch]

Hi all!

When I upload assets to cloud storage with `gsutil cp -z js,css,html`,

then their TTFB (Time To First Byte) on serving increasing from ~20ms to 180ms.

This is big performance impact. Why is this happening and how to solve this?

Here more samples (URLs are still valid, if you want to try yourself):

    $ ab -c 5 -n 50 https://storage.googleapis.com/latencytest/test-raw.txt

    

    Concurrency Level:      5

    Time taken for tests:   2.048 seconds

    Complete requests:      50

    Failed requests:        0

    Total transferred:      45710 bytes

    HTML transferred:       8050 bytes

    Requests per second:    24.41 [#/sec] (mean)

    Time per request:       204.846 [ms] (mean)

    Time per request:       40.969 [ms] (mean, across all concurrent requests)

    Transfer rate:          21.79 [Kbytes/sec] received

    

    Connection Times (ms)

                  min  mean[+/-sd] median   max

    Connect:       88  117  16.7    112     160

    Processing:    21   73 112.8     36     487

    Waiting:       21   71 113.3     34     487

    Total:        122  189 118.4    146     613

    

    

    $ ab -c 5 -n 50 https://storage.googleapis.com/latencytest/test.txt

    

    Concurrency Level:      5

    Time taken for tests:   3.374 seconds

    Complete requests:      50

    Failed requests:        0

    Total transferred:      45150 bytes

    HTML transferred:       7250 bytes

    Requests per second:    14.82 [#/sec] (mean)

    Time per request:       337.403 [ms] (mean)

    Time per request:       67.481 [ms] (mean, across all concurrent requests)

    Transfer rate:          13.07 [Kbytes/sec] received

    

    Connection Times (ms)

                  min  mean[+/-sd] median   max

    Connect:       90  109   9.0    107     136

    Processing:   174  206  44.6    190     389

    Waiting:      172  204  44.3    189     384

    Total:        274  315  47.3    299     495

curl output for gzipped file:

    $ curl -s -v https://storage.googleapis.com/latencytest/test.txt > /dev/null

    *   Trying 2a00:1450:400e:805::2010...

    * Connected to storage.googleapis.com (2a00:1450:400e:805::2010) port 443 (#0)

    * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

    * Server certificate: *.storage.googleapis.com

    * Server certificate: Google Internet Authority G2

    * Server certificate: GeoTrust Global CA

    > GET /latencytest/test.txt HTTP/1.1

    > Host: storage.googleapis.com

    > User-Agent: curl/7.43.0

    > Accept: */*

    > 

    < HTTP/1.1 200 OK

    < X-GUploader-UploadID: AEnB2UpBZ1SoG2fiD3_qSOmIWWvL86Kd-r21kXOS08UlvMOc90Eco-vd3ol3NnwDrkJKwKk7zav0ePdp9QYXm7lt4NdV39h-VQ

    < Date: Tue, 17 Jan 2017 20:44:19 GMT

    < Cache-Control: no-transform

    < Expires: Wed, 17 Jan 2018 20:44:19 GMT

    < Last-Modified: Mon, 16 Jan 2017 13:46:54 GMT

    < ETag: "88b49948e59eaad05d60a52001b50d7a"

    < x-goog-generation: 1484574414392000

    < x-goog-metageneration: 2

    < x-goog-stored-content-encoding: gzip

    < x-goog-stored-content-length: 145

    < Content-Type: text/plain

    < Content-Encoding: gzip

    < Content-Language: en

    < x-goog-hash: crc32c=MlL4Uw==

    < x-goog-hash: md5=iLSZSOWeqtBdYKUgAbUNeg==

    < x-goog-storage-class: STANDARD

    < Accept-Ranges: bytes

    < Server: UploadServer

    < Alt-Svc: quic=":443"; ma=2592000; v="35,34"

    < Transfer-Encoding: chunked

    < 

    { [261 bytes data]

    * Connection #0 to host storage.googleapis.com left intact

[Adam (Cloud Platform Support)]

It seems like the first byte is getting delayed until the first window of the gzip-encoded streams is being decoded (but ab shouldn't accept gzip encoding by default). Have you compared the results by using -H “Accept-Encoding: gzip,deflate” to see what's going on?