Bug#610199: polipo: "Server ignored conditional request" and downloads repeatedly
Steven Chamberlain
Version: 1.0.4.1-1.1
Hi,
I just experienced some very nasty behaviour from polipo.
It seems a client was viewing
http://www.theaa.com/route-planner/index.jsp which makes some use of
Google Maps. Somehow polipo ended up trying to download two map tiles
from Google's maptile server over, and over again.
I only noticed this 7 hours later, with about 9 GiB of data downloaded.
And this was at least 2 hours after the client's machine had been
powered down.
polipo.log contains the following messages repeating in random order:
> Couldn't get peer name: Transport endpoint is not connected
> Refusing connection from unauthorised net
> Aborting pipeline on mw1.google.com:80.
> Server ignored conditional request.
> Server ignored conditional request.
> Server ignored conditional request.
> Server ignored conditional request.
> Restarting pipeline to cbk0.google.com:80.
> Server ignored conditional request.
> Server ignored conditional request.
> Server ignored conditional request.
> Restarting pipeline to mw1.google.com:80.
> Server ignored conditional request.
> Server ignored conditional request.
> Server ignored conditional request.
> Server ignored conditional request.
I managed to record a short packet capture with tcpdump before I killed
polipo. The HTTP queries generated by polipo looked like this -- they
don't look like conditional requests though:
> GET /vt/lyrs=m@142&hl=en&src=api&x=1018&y=660&z=11&s=Ga HTTP/1.1
> Host: mt0.google.com
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)
> Accept: image/png,image/*;q=0.8,*/*;q=0.5
> Accept-Language: en-gb,en;q=0.5
> Accept-Encoding: gzip,deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Referer: http://www.theaa.com/route-planner/index.jsp
> Cookie: [omitted]
> Connection: keep-alive
The responses from Google's map tileserver were like this, followed by a
stream of PNG data (within the same packet):
> HTTP/1.1 200 OK
> Date: Sat, 15 Jan 2011 08:24:58 GMT
> Content-Type: image/png
> X-Content-Type-Options: nosniff
> Server: maptiles-versatile
> Content-Length: 9722
> X-XSS-Protection: 1; mode=block
> Cache-Control: public, max-age=22222222
Polipo responded to these packets by immediately closing the connection
(TCP packets with FIN or RST flag). This kept happening until I killed
polipo with killall ('/etc/init.d/polipo stop' wasn't enough).
Then I saw in netstat that the proxy server's connections from the
client host were in the CLOSE_WAIT state with 0 bytes in Recv/Send-Q,
but that machine had been offline for at least 2 hours by that time.
That's all I've been able to figure out so far. I'll certainly keep an
eye out in case this happens again.
Regards,
--
Steven Chamberlain
ste...@pyro.eu.org
--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Steven Chamberlain
thanks
Hi,
This bug occasionally still happens. (And no newer version has been
released upstream or added to the Debian archive since squeeze).
It results in prolonged, repeated fetches of a URI until the polipo
process is manually restarted. (Seen by me to do this for 23 hours
continuously, even after the proxy's only client was powered off).
This could seriously bite someone who pays for either
upstream/downstream data transfer, so I think a 'critical' severity may
be justified on that basis. It could annoy the website owner too.
Other log messages seen around the time this problem occurred:
> Unsupported Cache-Control directive no-check -- ignored.
> Server returned weak ETag -- ignored.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Couldn't parse ETag.
> Unsupported Cache-Control directive stale-while-revalidate -- ignored.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> "Not changed" reply with no ETag.
> Server ignored conditional request.
> Unsupported Cache-Control directive stale-while-revalidate -- ignored.
> Read from server failed: Connection reset by peer
Julien Cristau
Hi Steven,
On Sun, Jan 16, 2011 at 04:03:34 +0000, Steven Chamberlain wrote:
> Package: polipo
> Version: 1.0.4.1-1.1
>
> Hi,
>
> I just experienced some very nasty behaviour from polipo.
>
> It seems a client was viewing
> http://www.theaa.com/route-planner/index.jsp which makes some use of
> Google Maps. Somehow polipo ended up trying to download two map tiles
> from Google's maptile server over, and over again.
>
> I only noticed this 7 hours later, with about 9 GiB of data downloaded.
> And this was at least 2 hours after the client's machine had been
> powered down.
>
https://github.com/jech/polipo/commit/12b9179c9271c3d277f130184439516f74be7012
fix it?
Cheers,
Julien