Threading Bug Causing Corrupted File Response
Sunny Hirai
the same file twice at the same time, you get an error that flashes in
Firefox before displaying the image that says:
The image "path/to/image.png" cannot be displayed, because it
contains errors.
Since this was working before, I'm guessing the bug was introduced in
1.1 or at least a recent-ish release (not exactly sure when the bug
crawled in).
Although I was using a custom file mapper, this bug still happens in
Rack's file mapper. I can replicate by doing:
app = Rack::Builder.new do
run Rack::File.new( '.' )
end
If I hit the file twice quickly, Firefox will flash the error. It will
retry immediately so the image eventually comes up but there are twice
as many requests made.
The bug can be replicated using both Mongrel and WEBrick.
Sunny Hirai
Sunny Hirai
Actually, the error is hard to reproduce when going directly to the
web service because everything happens fast. We put in a proxy to
simulate slow connections and that's what is causing the problems to
show up more.
Sunny
Sunny Hirai
Sunny
Sunny Hirai
This problem does not seem to happen on Linux and only on Windows.
Note that it is also not a :num_processor limit that it is hitting as
it is set to 950.
I may be giving up on fixing this for now since it won't affect our
production environment, just my personal development environment.
Still, it feels like it should be fixed.
Sunny
James Tucker
On 28 Jan 2010, at 01:22, Sunny Hirai wrote:
> Just an update.
>
> This problem does not seem to happen on Linux and only on Windows.
Platform specific, good clue.
> Note that it is also not a :num_processor limit that it is hitting as
> it is set to 950.
num_processors is not the only limit in effect, there's also socket, select and file handle limits. under 1.8.x on win32 presently, i believe ruby is pretty limited, 256 handles if i remember correctly (this needs checking).
> I may be giving up on fixing this for now since it won't affect our
> production environment, just my personal development environment.
> Still, it feels like it should be fixed.
When you describe the problem, you say "flashed and then reloaded", are you sure it's not just a close connection effect due to a refresh?
Where are you seeing the "error"?
You can quite easily test this by getting a copy of curl[1] and spawning a bunch of curl processes in parallel. Log the results, and save the files downloaded, then hash check all the files, and correlate response data. If some of the files are corrupted, please post your script that demonstrates the error.
HTH