URLFetch adds gzip,deflate,br to outbound requests' Accept-Encoding?

[Ryan B]

hi all! i had to debug a URLFetch problem recently, and the root cause turned out to be that i was unexpectedly receiving a brotli-compressed response. it took me a while to figure out why: evidently URLFetch is adding gzip,deflate,br to outbound requests' Accept-Encoding HTTP header.

is this expected? it's not documented on https://cloud.google.com/appengine/docs/standard/python/outbound-requests#request_headers , and brotli support isn't built into many common HTTP libraries, e.g. python requests, so this will usually be an unpleasant surprise when talking to a server that supports brotli. is it unintentional? should i file a bug?

here's a snippet that reproduces this on https://shell-hrd.appspot.com/ :

>>> from google.appengine.api import urlfetch
>>> resp = urlfetch.fetch('https://httpbin.org/headers', headers={'Accept-Encoding': 'foo'})
>>> resp.status_code
200
>>> print(resp.content)
{
  "headers": {
    "Accept-Encoding": "foo,gzip,deflate,br",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "AppEngine-Google; (+http://code.google.com/appengine; appid: s~shell-hrd)",
    "X-Cloud-Trace-Context": "795505ff8141cf535fa9cf33afe59daa/1604381323415253803;o=1"
  }
}

[Ryan B]

i've filed https://issuetracker.google.com/issues/112277350 for this.

[Jorge A (Google Cloud Support)]

Since you've filed the issue through the proper channel, we'll continue to look at this through the issue tracker.