for a while now we've been having issues with our web servers.. and today we may have finally figured out why.
we used to serve directly from apache with mod_wsgi but a few weeks ago we moved to gunicorn running through apache with mod_proxy and pagespeed
anyhow - we've had a number of servers get tied up in very high cpu usage without any explanation.
in the beginning, we thought it was just mod_wsgi, and gunicorn did help but still - web1 wasn't working until this afternoon, again due to high cpu.
and indeed, we saw a large amount of requests in our log files to domains other than oknesset.org
apparently there is an attack, specifically on web1.oknesset.org
that accesses the server directly but then sends requests to http://<some other domain>/<for something else>
mod_pagespeed has/had a vulnerability that makes it actually go get that file -> many such requests kill the server
anyhow - how does all this affect you?
1. as an initial defensive measure we've blocked access on port 80 directly to any web server.
all port 80 requests go through the load balancer (which should only answer to requests for oknesset.org
hopefully this will solve our high cpu problem
amir and meir