why http response's Transfer-Encoding field is chunked

[atao]

I installed apache 2.2.17 as a proxy server,and installed
mod_pagespeed module. I downloaded a html to test.
but result is not correct. testing command is:
curl -o /tmp/test.html -v http://192.168.1.2/publicforum/content/free/1/2086886.shtml
-H "Host: www.tianya.cn"

result is:
* About to connect() to 192.168.1.2 port 80
* Trying 192.168.1.2... connected
* Connected to 192.168.1.2 (192.168.1.2) port 80
> GET /publicforum/content/free/1/2086886.shtml HTTP/1.1
> User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Accept: */*
> Host: www.tianya.cn
>
% Total % Received % Xferd Average Speed Time Time
Time Current
Dload Upload Total Spent
Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02
--:--:-- 0< HTTP/1.1 200 OK
< Date: Fri, 18 Feb 2011 08:16:33 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< Content-Type: text/html
< Cache-control: max-age=0, no-cache, no-store
< X-tianya: 1255507538
< Age: 0
< Via: Tianya Cache
< X-Cache: MISS115
< Via: 1.1 test.primenets.net
< X-Mod-Pagespeed: 0.9.14.6-0
< Vary: Accept-Encoding
< Connection: close
< Transfer-Encoding: chunked
100 121k 0 121k 0 0 27408 0 --:--:-- 0:00:04
--:--:-- 35155* Closing connection #0


I think that response should include Content-Length field, but testing
result does not include this field, but Transfer-Encoding field. I
want to know why. I read source code ,but can not find reason. please
help me, why such result!

[Joshua Marantz]

Does disabling mod_pagespeed change the Content-Length and Transfer-Encoding fields?  Other than with 'curl', did you notice something else wrong?  Do browsers misinterpret the data due to the chunked encoding?

I just tried another mod_pagespeed-enabled site, androidacademy.com and the content-length is set properly, and transfer-encoding is not set.

curl -o - -v http://androidacademy.com|head 

* About to connect() to androidacademy.com port 80 (#0)

*   Trying 212.110.164.236... connected

* Connected to androidacademy.com (212.110.164.236) port 80 (#0)

> GET / HTTP/1.1

> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15

> Host: androidacademy.com

> Accept: */*

> 

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0< HTTP/1.1 200 OK

< Date: Fri, 18 Feb 2011 14:08:16 GMT

< Server: Apache/2.2.14 (Ubuntu)

< X-Powered-By: PHP/5.3.2-1ubuntu4.7

< Set-Cookie: 8d49ce4ad435e5532b6f2b546599ce55=e4pugj92ie7vk79fdk4im3re07; path=/

< P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"

< Cache-Control: max-age=0, no-cache, no-store

< Pragma: no-cache

< X-Mod-Pagespeed: 0.9.15.3-404

< Vary: Accept-Encoding

< Content-Length: 126107

< Content-Type: text/html; charset=utf-8

< 

...

-Josh

[atao]

hi,Josh:
i test your command,it is proper. Thanks. my problem is that content-
length field sometimes exist in response, sometimes does not exist in
response,instead of Transfer-Encoding: chunked,when i test diffrent
html, set apache as a proxy . I dont't understand this .
-- Atao

On 2月18日, 下午10时09分, Joshua Marantz <jmara...@google.com> wrote:
> Does disabling mod_pagespeed change the Content-Length and Transfer-Encoding
> fields? Other than with 'curl', did you notice something else wrong? Do
> browsers misinterpret the data due to the chunked encoding?
>
> I just tried another mod_pagespeed-enabled site, androidacademy.com and the
> content-length is set properly, and transfer-encoding is not set.
>

> curl -o - -vhttp://androidacademy.com|head

[Shawn Ligocki]

Hi Atao,

Please see section 4.4 of the HTTP spec: http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html

Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non- identity transfer-coding, the Content-Length MUST be ignored.

Apache makes the choice of whether or not to send the response out in chunked encoding or in a single chunk (with a Content-Length header). This should not have anythign to do with mod_pagespeed.

Please let us know if you are having problems loading your page through mod_pagespeed. If you want to know more about why apache is choosing to use chunked encoding or not, you should contact the apache groups.

Cheers,

-Shawn