ETag support: please take a look at this change
131 views
Skip to first unread message
Michael Bolin
Nov 29, 2011, 1:25:39 PM11/29/11
to pl...@googlegroups.com
I have been investigating using ETags in the InputFileHandler so it can send 304s when an input is not modified (which is most of the time). I have put together the change here:
Unfortunately, at least for my project, now sometimes the InputFileHandler does not seem to return a response. Of the ~150 or so JS files that I am loading, when a response fails to be served, it is never for the same JS files. What's worse is that it seems to happen in Chrome, but not Firefox. Obviously this smells of a race condition. It could be a bug in Chrome, a race condition in plovr that is only exhibited because Chrome is faster than Firefox, or something else entirely. I have no idea.
Here is a precompiled binary with my patch included if you want to try it out to see if you can reproduce the issue on your own projects:
If anyone has any thoughts on the change or the bug, then please let me know!
--Michael
Ilia Mirkin
Nov 29, 2011, 1:34:06 PM11/29/11
to pl...@googlegroups.com
Hmmm... The ETag's that you return are like "asdf". Most of the time
I've seen ETag usage it uses "weak" tags, i.e. W/"asdf". I forget what
the diff is exactly, but could be related?
I've seen ETag usage it uses "weak" tags, i.e. W/"asdf". I forget what
the diff is exactly, but could be related?
The way that I've found most reliable in the past, BTW, is to use
if-modified-since. Never played around with ETag directly though,
perhaps it works just as well.
-ilia
Michael Bolin
Nov 29, 2011, 1:45:01 PM11/29/11
to pl...@googlegroups.com
From my reading of http://en.wikipedia.org/wiki/HTTP_ETag, I think "strong ETags" are fine here (those without the W/), but I could be wrong. I'll give W/ a shot.
Michael Bolin
Nov 29, 2011, 1:51:36 PM11/29/11
to pl...@googlegroups.com
Hmm, just tried weak ETags (W/) and that made the bug even easier to produce. I'm not sure whether that makes me more or less frustrated.
Michael Bolin
Nov 29, 2011, 1:58:40 PM11/29/11
to pl...@googlegroups.com
Also, I'm a little worried about introducing cache headers in general, as apparently at one time we categorically disallowed caching because of an issue in IE6:
Ilia Mirkin
Nov 29, 2011, 2:13:10 PM11/29/11
to pl...@googlegroups.com
You could try dumping the log from chrome's network inspector (it
creates a json dump of all the request data) to help see what's really
going on.
creates a json dump of all the request data) to help see what's really
going on.
Michael Bolin
Nov 29, 2011, 2:15:06 PM11/29/11
to pl...@googlegroups.com
Maybe part of the problem is with com.sun.net.httpserver.HttpServer?
I believe that I am trying to send:
HTTP/1.1 304 Not Modified
but Chrome claims that it sees:
HTTP/1.1 304 Not Modified
Content-length: 0
Date: Tue, 29 Nov 2011 19:07:19 GMT
Perhaps the Date and Content-length headers are causing a problem?
Michael Bolin
Nov 29, 2011, 2:16:49 PM11/29/11
to pl...@googlegroups.com
This was at least a bug in lighttpd at one time: http://redmine.lighttpd.net/issues/1002
Perhaps it is a bug in Java's server as well and Firefox ignores it but Chrome does not?
Ilia Mirkin
Nov 29, 2011, 2:17:55 PM11/29/11
to pl...@googlegroups.com
Hmmm... yes, that definitely sounds like a problem. You should verify
with tcpdump (tcpdump -s 0 -A port 8090), I've seen Chrome make minor
lies in its display of various things.
Michael Bolin
Nov 29, 2011, 2:23:35 PM11/29/11
to pl...@googlegroups.com
Verified with curl. Yikes.
Michael Bolin
Nov 29, 2011, 3:01:38 PM11/29/11
to pl...@googlegroups.com
I just filed a bug against Sun. I took a lot of time to write it up, but they fail to provide a link after filling out their form, so after all that effort, I can't link to what I submitted (lame). Here is the bulk of my bug report:
In reading the source code at http://www.docjar.com/html/api/sun/net/httpserver/ExchangeImpl.java.html, there is a bug in the implementation of sendResponseHeaders().
When the response code is a 304, the Content-Length header is sent with a value of 0. When a 304 is sent, NO CONTENT-LENGTH HEADER SHOULD BE SENT AT ALL. I believe this is at least causing problems for Google Chrome 15.0.874.121, though it likely also affects other user agents.
Note that this was also an issue in lighttpd at one time: http://redmine.lighttpd.net/issues/1002
I think that this would be a lot simpler if you only wrote the headers that the user explicitly requested. Your API has convoluted things by requiring the response code and the content-length as part of the same method.
Also, your code would be a lot cleaner if at the start of sendResponseHeaders(), you did a check:
if (contentLen < 0) {
contentLen = -1;
}
The specification says that all values of contentLen <= -1 should be treated the same, but it is hard to tell whether this is the case because the logic is so convoluted. At a minimum, this caused problems in the Java 6 implementation:
There, you can see that in lines 176-189, checks are done for contentLen == -1 rather than <= -1. If you normalized the value at the start of the method, then this would be fine.
Michael Bolin
Nov 29, 2011, 4:44:42 PM11/29/11
to pl...@googlegroups.com
Hmm, now I am having trouble reproducing the issue on my simplified test page:
It would be helpful if others were willing to try out the binary and let me know whether they see any issues. If I'm the only one with the problem, then I think this would help speed up plovr even more.
Michael Bolin
Nov 29, 2011, 5:56:52 PM11/29/11
to pl...@googlegroups.com
Here is the dump from the HAR (HTTP archive) file for a file that failed to load:
{
"cache": {},
"pageref": "http://xn--js-wka.com:1337/desktop",
"request": {
"bodySize": 0,
"cookies": [],
"headers": [
{
"name": "User-Agent",
"value": "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2"
},
{
"name": "Accept",
"value": "*/*"
},
{
"name": "Referer",
"value": "http://xn--js-wka.com:1337/desktop"
},
{
"name": "If-None-Match",
"value": "\"a8778765\""
}
],
"headersSize": 295,
"httpVersion": "HTTP/1.1",
"method": "GET",
"queryString": [],
},
"response": {
"bodySize": 0,
"content": {
"compression": 0,
"size": 0
},
"cookies": [],
"headers": [],
"headersSize": 22,
"httpVersion": "HTTP/1.1",
"redirectURL": "",
"status": 0
},
"startedDateTime": "2011-11-29T22:22:59.509Z",
"time": 128,
"timings": {
"blocked": 0,
"connect": -1,
"dns": -1,
"receive": null,
"send": -1,
"ssl": -1,
"wait": -1
}
},
Michael Bolin
Nov 29, 2011, 6:01:24 PM11/29/11
to pl...@googlegroups.com
By comparison, here is the dump of a file that loaded successfully:
{
"cache": {},
"pageref": "http://xn--js-wka.com:1337/desktop",
"request": {
"bodySize": 0,
"cookies": [
{
"expires": null,
"httpOnly": false,
"name": "__utma",
"secure": false,
"value": "223746180.500724665.1301580717.1304710301.1320368923.5"
},
{
"expires": null,
"httpOnly": false,
"name": "__utmz",
"secure": false,
"value": "223746180.1320368923.5.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)"
},
{
"expires": null,
"httpOnly": false,
"name": "_chartbeat2",
"secure": false,
"value": "h8asdnlnn3c5rwmj.1320368923246"
},
{
"expires": null,
"httpOnly": false,
"name": "EventSpoonId",
"secure": false,
"value": "60048b1d4f520103a779e111deada8c8c321b04848052ee5234e25732fdac4b52d9143cac531b1a92db45a28b3870e5a5c85644ba93a8711421f6563124a4f45"
}
],
"headers": [
{
"name": "Accept-Encoding",
"value": "gzip,deflate,sdch"
},
{
"name": "Accept-Language",
"value": "en-US,en;q=0.8"
},
{
"name": "Cookie",
"value": "__utma=223746180.500724665.1301580717.1304710301.1320368923.5; __utmz=223746180.1320368923.5.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _chartbeat2=h8asdnlnn3c5rwmj.1320368923246; EventSpoonId=60048b1d4f520103a779e111deada8c8c321b04848052ee5234e25732fdac4b52d9143cac531b1a92db45a28b3870e5a5c85644ba93a8711421f6563124a4f45"
},
{
"name": "Connection",
"value": "keep-alive"
},
{
"name": "Accept-Charset",
"value": "ISO-8859-1,utf-8;q=0.7,*;q=0.3"
},
{
"name": "Host",
"value": "xn--js-wka.com:1980"
},
{
"name": "User-Agent",
"value": "Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2"
},
{
"name": "Accept",
"value": "*/*"
},
{
"name": "Referer",
"value": "http://xn--js-wka.com:1337/desktop"
},
{
"name": "Cache-Control",
"value": "max-age=0"
},
{
"name": "If-None-Match",
"value": "\"cae1e978\""
}
],
"headersSize": 799,
"httpVersion": "HTTP/1.1",
"method": "GET",
"queryString": [],
},
"response": {
"bodySize": 0,
"content": {
"compression": 1463,
"mimeType": "text/javascript",
"size": 1463
},
"cookies": [],
"headers": [
{
"name": "Date",
"value": "Tue, 29 Nov 2011 22:22:59 GMT"
},
{
"name": "Content-length",
"value": "0"
}
],
"headersSize": 85,
"httpVersion": "HTTP/1.1",
"redirectURL": "",
"status": 304,
"statusText": "Not Modified"
},
"startedDateTime": "2011-11-29T22:22:59.511Z",
"time": 132,
"timings": {
"blocked": 0,
"connect": 43,
"dns": 2,
"receive": 2,
"send": 0,
"ssl": -1,
"wait": 1
}
},
Michael Bolin
Nov 29, 2011, 6:47:59 PM11/29/11
to pl...@googlegroups.com
Here is more detailed information from another failure. This information was taken from chrome://net-internals/#events
t=1322610321672 [st= 0] +REQUEST_ALIVE [dt=250]
t=1322610321672 [st= 0] +URL_REQUEST_START_JOB [dt=249]
--> load_flags = 71368832 (ENABLE_LOAD_TIMING | MAYBE_USER_GESTURE | REPORT_RAW_HEADERS | VERIFY_EV_CERT)
--> method = "GET"
--> priority = 1
--> url = "http://xn--js-wka.com:1980/input/desktop/third_party/closure/goog/mochikit/async/deferred.js"
t=1322610321672 [st= 0] HTTP_CACHE_GET_BACKEND [dt= 0]
t=1322610321672 [st= 0] HTTP_CACHE_OPEN_ENTRY [dt= 0]
t=1322610321672 [st= 0] HTTP_CACHE_ADD_TO_ENTRY [dt= 0]
t=1322610321672 [st= 0] HTTP_CACHE_READ_INFO [dt= 0]
t=1322610321672 [st= 0] +HTTP_STREAM_REQUEST [dt=249]
t=1322610321921 [st=249] HTTP_STREAM_REQUEST_BOUND_TO_JOB
--> source_dependency = {"id":40638,"type":11}
t=1322610321921 [st=249] -HTTP_STREAM_REQUEST
t=1322610321921 [st=249] +HTTP_TRANSACTION_SEND_REQUEST [dt= 0]
t=1322610321921 [st=249] HTTP_TRANSACTION_SEND_REQUEST_HEADERS
--> GET /input/desktop/third_party/closure/goog/mochikit/async/deferred.js HTTP/1.1
Host: xn--js-wka.com:1980
Connection: keep-alive
User-Agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2
Accept: */*
Referer: http://xn--js-wka.com:1337/desktop
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: [value was stripped]
If-None-Match: "843c3404"
t=1322610321921 [st=249] -HTTP_TRANSACTION_SEND_REQUEST
--> net_error = -15 (SOCKET_NOT_CONNECTED)
t=1322610321921 [st=249] -URL_REQUEST_START_JOB
--> net_error = -15 (SOCKET_NOT_CONNECTED)
t=1322610321922 [st=250] -REQUEST_ALIVE
--> net_error = -15 (SOCKET_NOT_CONNECTED)Michael Bolin
Nov 29, 2011, 9:01:16 PM11/29/11
to pl...@googlegroups.com
I have updated the public demos to use the new jar that supports ETags. You should be able to see the error on:
The initial load will be all 200s. The subsequent load should be all 304s, though if you try it a few times, pretty soon you'll probably see some requests that do not load. I cannot reproduce the load failure when loading the file directly from the location bar.
Michael Bolin
Nov 29, 2011, 9:23:54 PM11/29/11
to pl...@googlegroups.com
This now has its own issue filed for Chrome: http://code.google.com/p/chromium/issues/detail?id=105824
I could not reproduce the issue in either Firefox or Safari.
Reply all
Reply to author
Forward
0 new messages