Sending file blob as postData + mod_security = dangerous CPU load

[fabrizim]

Hi all-

I am writing a class for the Ext library to implement managed upload
queue using the beta.desktop openFiles function along with the ability
to send file chunks as postData with the beta.httprequest. Everything
was going swimmingly while testing on my local machine, and I thought
that maybe this time, just this one time, I would be able to write
something and it would just work the way I thought it would /
should... So - I uploaded my test page to my production server, tried
to upload a few 2M images, and, lo and behold, my apache process
seized up and I died a bit inside.

So, after a futile day of messing around with code (including an
interesting foray into creating a multipart/form-data request with the
file blob - which isn't possible at this point, but should be
considered), I walked away from my computer, made a cocktail, and
realized what was going on. Ironically, I had already had this problem
while working with swfupload - which I am replacing with this gears
implementation due to those fun security enhancements in Flash 10...

Anywho... the Apache security module, mod_security, scans all
postData. So, when you are uploading a chunk of file, it will be
vigorously processed. This apparently consumes about 99-100% of my CPU
(!) and slows the entire upload process considerably.

If anyone else is having these issues, all you have to is disable the
mod_security post scanning filter or just disable the module
completely. This can be done with an .htaccess file...

# Turn off the module completely...
SecFilterEngine Off

# Or just the Post Scanning
SecFilterScanPost Off

Hope this helps someone else...

Best Regards-
Mark

Oh, and great work on gears - its awesome.

[Michael Nordman]

We have a Gears object that is not yet in the 'official' build that does allow script to compose a multi-part/formdata payload. Not sure when this will make its way into a real build.

interfaqce BlobBuilder {

  void append(string|blob);  // string data will be utf8 encoded in the resulting blob

  Blob getAsBlob();

};

With this interface you still have to manually compose the multipart boundaries, but in the end, you can create a valid multipart payload including binary file parts.

[fabrizim]

That sounds great. Are you planning on exposing the BlobBuilder class
to javascript? An alternate way I was considering was allowing an
Array to be passed to the HttpRequest::send method (indstead of just
Blob or String) that could contain both Strings and Blobs, and then
the new Blob could be built with the BlobBuilder.

On Nov 18, 3:31 pm, Michael Nordman <micha...@google.com> wrote:
> We have a Gears object that is not yet in the 'official' build that does
> allow script to compose a multi-part/formdata payload. Not sure when this
> will make its way into a real build.
> interfaqce BlobBuilder {
>   void append(string|blob);  // string data will be utf8 encoded in the
> resulting blob
>   Blob getAsBlob();
>
> };
>
> With this interface you still have to manually compose the multipart
> boundaries, but in the end, you can create a valid multipart payload
> including binary file parts.
>

[Michael Nordman]

On Wed, Nov 19, 2008 at 11:09 AM, fabrizim <mark.f...@gmail.com> wrote:

That sounds great. Are you planning on exposing the BlobBuilder class
to javascript?

 

I think this is likely, along with javascript sample code that demonstrates how to construct multipart formdata encoded blob.