Concurrent long-running file uploads

[Bill Moseley]

I have an existing app that accepts file uploads.   It's Apache/mod_perl perfork with Catalyst sitting behind a hardware load balancer.

Catalyst buffers the uploads to local disk then the app's job is to do some simple processing/checking on the file and then write that file out to permanent storage. 

Of course, concurrent uploads are limited by the forking Apache.   So, I'm looking for suggestions how to handle many more uploads concurrently and more efficiently.

The two approaches I'm looking at are using Twiggy and rewrite some of the code, or leave code as-is and put Perlbal in front to buffer slow uploads (which only solves the slow upload part, not the request processing).

Does anyone have any other suggestions or examples that I should consider?

Thanks,

--

Bill Moseley
mos...@hank.org