[akka-http-2.0.2] multipart and fields and fileupload problem

[Marek Żebrowski]

I'm trying to use fileuplad together with form fields in one endpoint. My smallest use case that demonstrates failure

request is 'multipart/form-data' with fields as in example:

if "image" uploaded is large enough (about 1MB) I got error:

MalformedFormFieldRejection(sizes,Substream Source cannot be materialized more than once,None)

with smaller upload sizes it works fine.

probably multipart/form-data must be handle in one run, so this construct is illegal.

val routes = {
  (path("resize") & post) {
          formFields('imageId, 'sizes, 'rotate ? "normal") { (imageId, sizesStr, rotateStr) =>
            uploadedFile("image") {
              case (meta, file) => {
                complete {

// do something with file, sizes, rotate ...
                }
              }
            }
    }
  }

[Rüdiger Klaehn]

Hi Marek,

This seems to happen whenever the request entity is non-strict.

I suppose you should be able to workaround this issue using entity.toStrict. But of course I think this should work out of the box.

If somebody wants to take a deeper look at this, here is a standalone reproducer: https://gist.github.com/rklaehn/d4d3ee43443b0f4741fb

Cheers,

Rüdiger

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Roland Kuhn]

Thanks a lot for the gist, would you mind filing a ticket?

Regards,

Roland

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn

[Rüdiger Klaehn]

Sure.

P.s. @Marek here is a workaround so you can continue working until
this is fixed. It is a directive that transforms the HttpEntity to
strict. Here is the fixed file in the gist:
https://gist.githubusercontent.com/rklaehn/d4d3ee43443b0f4741fb/raw/f1fba061eac5d13dd19523f2df9660500a5a8129/UploadHandlerToStrict.scala

@Roland does a directive like this exist somewhere? It might be useful.

def toStrict(timeout: FiniteDuration): Directive[Unit] = {
def toStrict0(inner: Unit ⇒ Route): Route = {
val result: RequestContext ⇒ Future[RouteResult] = c ⇒ {
// call entity.toStrict (returns a future)
c.request.entity.toStrict(timeout).flatMap { strict ⇒
// modify the context with the strictified entity
val c1 = c.withRequest(c.request.withEntity(strict))
// call the inner route with the modified context
inner()(c1)
}
}
result
}
Directive[Unit](toStrict0)

[Marek Żebrowski]

Thanks!

[Rüdiger Klaehn]

OK, done

https://github.com/akka/akka/issues/19506

I guess the reason I did not find a toStrict directive is that you
don't want to encourage people to do this, since it will of course
blow up with an OOME if you do a really large POST request... :-)

On Mon, Jan 18, 2016 at 2:52 PM, Marek Żebrowski
<marek.z...@gmail.com> wrote:
> Thanks!

[Marek Żebrowski]

for now I took a different approach, with manually parsing parts

formData.parts.map { part =>
  if (part.filename.isDefined) {
    val destination = File.createTempFile("akka-http-upload", ".tmp")
    val fileInfo = FileInfo(part.name, part.filename.get, part.entity.contentType)
    part.entity.dataBytes.runWith(FileIO.toFile(destination)).map(_ => FilePart(fileInfo, destination))
  } else {
    part.entity.toStrict(10 seconds).map(e => ParamPart(part.name, e.data.utf8String))
  }
}.runFold(ReqData(Map.empty, None)) {
  case (acc, el) =>
    el match {
      case f: FilePart     => acc.copy(file = Some(f))
      case ParamPart(k, v) => acc.copy(params = acc.params + (k -> v))
    }
}