ATTN: bodyParser changes in 3.4.0

[tjholowaychuk]

 Express 3.4.0 and Connect 2.9.0 have made some small changes to bodyParser(), and more specifically the multipart() middleware used within it. There

 have been concerns regarding temporary-file usage, however to maintain backwards compatibility for now I've added some documentation on 

 http://www.senchalabs.org/connect/multipart.html.

  We've also switched to the "multiparty" library, instead of using formidable, which allows you to stream the parts directly to arbitrary 

  destinations without hitting disk. Keep in mind that the destination streams must properly implement node's backpressure mechanisms

  otherwise you're likely to cause large memory bloat causing the process to fail. The "defer" option let's subsequent middleware listen

  on "part" events to stream accordingly instead of writing to disk, providing the convenient req.files object that you might be used to.

  Another alternative if you're concerned is to simply use express.json(), and express.urlencoded(), and leave out multipart() all together. Use

  `if (req.is('multipart/form-data')` and formidable/multiparty/parted directly.

  The tmpfile used is os.tmpDir()'s value, so if you plan on continuing to use disk it's highly recommended to set up a strategy for dealing

  with unnecessary temporary files, this is good practice for any production environment, much like log rotation it is critical to any large

  deployment. An example tool is reap(1) https://github.com/visionmedia/reap. Tools like this should be used regardless of the cleanup technique,

  as application processes may fail at any point in time, and may never have the chance to unlink() the file.

  The default limits for bodyParser(), urlencoded(), multipart(), and json() have also been adjusted. The default limit for multipart is now 100mb,

  and 1mb for the other two. If you anticipate requests larger than this you may pass {  limit: '200mb' } to either bodyParser() or the others. It's

  recommended to use each one individually, bodyParser() is a legacy convenience aggregate of the others, but applying a global .limit option

  between the three of them is not a great choice, as sending 200mb of JSON could halt the application.

  If node sits behind a reverse proxy such as nginx you may easily tweak this behaviour there as well.

  If you have questions, concerns, or suggestions let me know.

[Hage Yaapa]

The quick response to the concerns is much appreciated, TJ!

--
You received this message because you are subscribed to the Google Groups "Express" group.
To unsubscribe from this group and stop receiving emails from it, send an email to express-js+...@googlegroups.com.
To post to this group, send email to expre...@googlegroups.com.
Visit this group at http://groups.google.com/group/express-js.
For more options, visit https://groups.google.com/groups/opt_out.

[Raul Vieira]

Thanks.

Sent from my iPhone

--

[Joe McCann]

Nice work TJ.

[Camilo Aguilar]

whoa nice work TJ, as always ;D. 

On Sun, Sep 8, 2013 at 8:18 AM, Joe McCann <joseph...@gmail.com> wrote:

Nice work TJ.

--
You received this message because you are subscribed to the Google Groups "Express" group.
To unsubscribe from this group and stop receiving emails from it, send an email to express-js+...@googlegroups.com.
To post to this group, send email to expre...@googlegroups.com.
Visit this group at http://groups.google.com/group/express-js.
For more options, visit https://groups.google.com/groups/opt_out.

--

Camilo Aguilar

Software Engineer

http://github.com/c4milo

[Brian Falk]

awesome work!

[Aaron Heckmann]

I'm curious, since formidable is also configurable to not hit disk, were there any other motivations as well?

Aaron

@aaronheckmann

soundcloud.com/ajhecky

github.com/aheckmann

[Simon Mansfield]

I'm experiencing some very odd behaviour regarding this suggestion...

If I use bodyParser() globally, my route that accepts multipart form data works fine (although all routes are susceptible to a "file spam" attack), if I switch to global use of app.use(express.json()) & app.use(express.urlencoded()) and in my route add express.multipart() it fails entirely.

I've stepped into the function returned by express.multipart() and it fails it's first check that req._body isn't null/undefined.

Any idea why this might be happening?

[Billy Newman]

Any reason why the 'type' property is no longer being set?  Am I to assume that if the Content-Type is set correctly on file upload this should still work as it did in the past?

Thanks,
Billy

[Simon Mansfield]

Billy, I believe it's due to the change in underlying parser; the content type is now under file.headers I think...?

—
Sent from Mailbox for iPhone

--
You received this message because you are subscribed to a topic in the Google Groups "Express" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/express-js/iP2VyhkypHo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to express-js+...@googlegroups.com.

[Billy Newman]

Simon, big thanks you are correct.

file.headers['content-type'] was what I needed.

[lucj06]

Thanks TJ.

Any way to call an authentication middleware before calling bodyparser to avoid unauthentified client to post huge files ?

[greelgorke]

put your authentication middleware before the bodyparser. if you need the bodyparser for authentication, then put an alternative before it, or just a custom middleware function.