I want to parse the POST body twice.

270 views
Skip to first unread message

action

unread,
Mar 2, 2014, 11:18:32 AM3/2/14
to clo...@googlegroups.com
I want to parse the POST body twice,
but in the second place:
org.xml.sax.SAXParseException: Premature end of file.
How should I clone the body render for the second parse?

James Reeves

unread,
Mar 2, 2014, 11:44:09 AM3/2/14
to clo...@googlegroups.com
Depending on how large the input is, you could either read it all into memory (e.g. with slurp or similar), or dump it to a temporary file.

- James


--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Herwig Hochleitner

unread,
Mar 2, 2014, 11:44:44 AM3/2/14
to clo...@googlegroups.com
Copy the body to a byte array or file before parsing (or while parsing it the first time, if you want to save latency).
Your server doesn't do this, so that it can reuse a few small buffer chunks instead of allocating the whole content-length, while streaming the body into your application.

cheers


--

action

unread,
Mar 2, 2014, 12:23:37 PM3/2/14
to clo...@googlegroups.com, ja...@booleanknot.com
how to use slurp?
first:
(slurp body)
and second reuse:
 (slurp body)

在 2014年3月3日星期一UTC+8上午12时44分09秒,James Reeves写道:

James Reeves

unread,
Mar 2, 2014, 12:46:14 PM3/2/14
to action, clo...@googlegroups.com
You can't consume an input-stream twice. You need to store the result somewhere.

For example:

  (defn handler [request]
    (let [body-str (slurp (:body request))]
      (do-something-to body-str)
      (do-something-else-to body-str)))

Be aware that in production you'll likely want to limit the maximum size of the stream to avoid malicious users pushing through large amounts of junk data and using up all the memory in your app. There's a class in the Apache Commons I/O called BoundedInputStream that might be of use to you.

- James

Herwig Hochleitner

unread,
Mar 2, 2014, 12:46:59 PM3/2/14
to clo...@googlegroups.com
Don't slurp twice. Slurp mutates the stream by reading bytes off of it.

Also, slurping an xml stream isn't a good idea, because the xml header can change the character encoding like so 

<?xml version="1.0" encoding="ISO-8859-1"?>

Try

(let [bufstream (doto (java.io.ByteArrayOutputStream. content-length)
                  (clojure.java.io/copy body bufstream))
      buffer (.toByteArray bufstream)]
  (parse-1 (java.io.ByteArrayInputStream. buffer))
  (parse-2 (java.io.ByteArrayInputStream. buffer)))

This way, your parse functions that take an input stream work unchanged.
Be aware, that this allocates memory in the size of the body.
Reply all
Reply to author
Forward
0 new messages