Stream Closed error on POST

[Daniel E. Renfer]

Hello,

I am getting a strange error when I try to send a POST request to my
application that is being handled by a route called from the main route.

All of my GET requests serve fine, as well as if I put the relevant
route fn in the main routes. (forgive me if the terminology is off for
some of these components)

The stack trace follows:

ava.io.IOException: Stream closed
org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:325)
org.apache.catalina.connector.CoyoteInputStream$4.run(CoyoteInputStream.java:177)
java.security.AccessController.doPrivileged(Native Method)
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:172)
sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
java.io.InputStreamReader.read(InputStreamReader.java:184)
java.io.BufferedReader.fill(BufferedReader.java:153)
java.io.BufferedReader.read(BufferedReader.java:174)
clojure.contrib.duck_streams$slurp_STAR___1167.invoke(duck_streams.clj:185)
compojure.http.request$slurp_body__171.invoke(request.clj:53)
compojure.http.request$get_form_params__179.invoke(request.clj:66)
compojure.http.request$with_params__186$fn__188.invoke(request.clj:85)
compojure.http.request$with_cookies__199$fn__201.invoke(request.clj:110)
compojure.http.routes$routes_STAR___442$fn__444$fn__446.invoke(routes.clj:194)
clojure.core$some__3808.invoke(core.clj:1501)
compojure.http.routes$routes_STAR___442$fn__444.invoke(routes.clj:194)
compojure.http.request$with_params__186$fn__188.invoke(request.clj:90)
compojure.http.request$with_cookies__199$fn__201.invoke(request.clj:110)
compojure.http.session$with_session__332$fn__335.invoke(session.clj:165)
compojure.http.servlet$request_handler__504.invoke(servlet.clj:112)
net.mycyclopedia.Servlet$_service__368.invoke(Servlet.clj:56)
net.mycyclopedia.Servlet.service(Unknown Source)
sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:616)
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:244)
java.security.AccessController.doPrivileged(Native Method)
javax.security.auth.Subject.doAsPrivileged(Subject.java:537)
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:276)
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:162)

One note: I am running on this under Apache Tomcat, not Jetty.

On furthur testing, it appears that I am getting this error on any post
request that doesn't match a route in main, not just the ones that I've
put in another defroutes.

From the looks of it, there's something wrong with it
trying to slurp the request body? I don't quite follow all the internals
of compojure yet.

On a related note, I'm just now trying to get into the habit of testing
my clojure programs. If anyone could give me any tips on using test-is
to write tests for my routes, that would be greatly appreciated.

contents of Servlet.clj:
(ns net.mycyclopedia.Servlet
(:use compojure.http
compojure.http.session
compojure.control
;; snip swank ns
net.mycyclopedia.controller)
(:require (net.mycyclopedia [view :as view])
(net.mycyclopedia.view [statement :as view.statement])
(net.mycyclopedia.controller
;; snip other controllers
[statement :as statement]))
(:import (javax.servlet.http HttpServletRequest
HttpServletResponse))
(:gen-class
:extends javax.servlet.http.HttpServlet))

;; snip swank initalization

(defroutes fallback
(ANY "*"
(or (serve-file "/home/duck/mycyclopedia/war/" (params :*))
:next))
(ANY "*"
(page-not-found)))

(defroutes main
;; snip other routes
statement/routes
fallback)

(decorate main (with-session :memory))

(defservice main) ;; line 56 in original program

Thank you for any help. I can give more info if needed.

Daniel E. Renfer
xri: @id*duck

[James Reeves]

On Apr 23, 2:25 pm, d...@kronkltd.net (Daniel E. Renfer) wrote:
> I am getting a strange error when I try to send a POST request to my
> application that is being handled by a route called from the main route.

It was a bug caused by the InputStream of the body being read through
twice. The inner defroutes was trying to parse the parameters from the
request body after the outer defroutes had already parsed them.

I've updated the code to only parse the parameters if they were not
already exist in the request object, and added a unit test to guard
against this issue in future.

Sorry for the bug.

- James

[Daniel E. Renfer]

James Reeves <weave...@googlemail.com> writes:

Thanks for the fix, only now I am getting this slightly different error.

java.io.IOException: Stream closed

org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:325)
org.apache.catalina.connector.CoyoteInputStream$4.run(CoyoteInputStream.java:177)
java.security.AccessController.doPrivileged(Native Method)
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:172)
sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
java.io.InputStreamReader.read(InputStreamReader.java:184)
java.io.BufferedReader.fill(BufferedReader.java:153)
java.io.BufferedReader.read(BufferedReader.java:174)
clojure.contrib.duck_streams$slurp_STAR___1167.invoke(duck_streams.clj:185)

compojure.http.request$slurp_body__453.invoke(request.clj:52)
compojure.http.request$get_form_params__461.invoke(request.clj:64)
compojure.http.routes$match_form_method__705.invoke(routes.clj:134)
compojure.http.routes$match_method__709.invoke(routes.clj:141)
net.mycyclopedia.controller.welcome$fn__775$matcher__731__auto____777.invoke(welcome.clj:9)
net.mycyclopedia.controller.welcome$fn__775$fn__781.invoke(welcome.clj:9)
compojure.http.routes$routes_STAR___739$fn__741$fn__743.invoke(routes.clj:207)
clojure.core$some__3808.invoke(core.clj:1501)
compojure.http.routes$routes_STAR___739$fn__741.invoke(routes.clj:207)
compojure.http.request$with_params__475$fn__477.invoke(request.clj:94)
compojure.http.request$with_cookies__488$fn__490.invoke(request.clj:111)
compojure.http.routes$routes_STAR___739$fn__741$fn__743.invoke(routes.clj:207)
clojure.core$some__3808.invoke(core.clj:1501)
compojure.http.routes$routes_STAR___739$fn__741.invoke(routes.clj:207)
compojure.http.request$with_params__475$fn__477.invoke(request.clj:94)
compojure.http.request$with_cookies__488$fn__490.invoke(request.clj:111)
compojure.http.session$with_session__621$fn__624.invoke(session.clj:165)
compojure.http.servlet$request_handler__801.invoke(servlet.clj:112)
net.mycyclopedia.Servlet$_service__870.invoke(Servlet.clj:54)
net.mycyclopedia.Servlet.service(Unknown Source)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:616)
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:244)
java.security.AccessController.doPrivileged(Native Method)
javax.security.auth.Subject.doAsPrivileged(Subject.java:537)
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:276)
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:162)

controller.welcome is the first set of routes I am using, the url in
question should be handled by the last one.

(defroutes main
welcome/routes
user/routes
entry/routes
auth/routes
statement/routes
fallback)

Aside maybe from the naming scheme, is this the right way to break my
routes up into different sections?

[James Reeves]

On Apr 25, 12:01 am, d...@kronkltd.net (Daniel E. Renfer) wrote:
> Thanks for the fix, only now I am getting this slightly different error.

Hi Daniel,

I'm having trouble reproducing your problem. The code below works on
my system. Does it work correctly on yours?

(ns example
(:use compojure))

(defroutes welcome
(GET "/"
(html
(form-to [:post "/"]
(text-field :name)
(submit-button "Submit"))))
(POST "/"
(html
[:h1 (params :name)])))

(defroutes fallback
(ANY "*"
[404 "Page Not Found"]))

(defroutes main
welcome
fallback)

(run-server {:port 8080}
"/*" (servlet main))

> Aside maybe from the naming scheme, is this the right way to break my
> routes up into different sections?

Yep.

- James

[Daniel Renfer]

James,

Just to give you an update. I installed your code into a fresh version
of compojure, and it worked fine. I tried turning it into a servlet
and ran it under Tomcat, and got the same problem I've been getting.

I've also discovered that with the code I have, if I send a POST
request with parameters, it runs fine, but if I send it with an empty
request, then I get the same error.

I'm going to continue to look into this, and if I can manage to come
up with a reproducible test case (or a patch) I'll make sure to send
it along.

Daniel E. Renfer

[James Reeves]

On Apr 26, 3:19 pm, Daniel Renfer <d...@kronkltd.net> wrote:
> Just to give you an update. I installed your code into a fresh version
> of compojure, and it worked fine. I tried turning it into a servlet
> and ran it under Tomcat, and got the same problem I've been getting.
>
> I've also discovered that with the code I have, if I send a POST
> request with parameters, it runs fine, but if I send it with an empty
> request, then I get the same error.

Odd. Do you still get the same error if you send an empty request
without nested routes, e.g.

(ns example
(:use compojure))

(defroutes main

(GET "/"
(html
(form-to [:post "/"]
(text-field :name)
(submit-button "Submit"))))
(POST "/"
(html
[:h1 (params :name)])))

(run-server {:port 8080}
"/*" (servlet main))

If that doesn't work, then it probably means that Tomcat is passing
Compojure a closed stream. If it does work, then there's something
subtly wrong with the Compojure logic as well.

- James