forcing akka-http into parsing chunked entity

[A.i.]

Hi, we are creating an http service based on akka, and encountered a problem related to handling an invalid http request, but I hope akka experts can quickly point to a good solution. So:

we need to handle a PUT request from a client which sends chunked entity ~without~ specifying Transfer-Encoding header field but ~with~ Content-Length (hello AWS S3 SDK). This results in akka considering this request a regular one (not chunked) and therefore not attempting to unchunk it and returning raw bytes including chunk delimiters etc. for the entity. Technically akka's behavior is correct - from HTTP point this request is not chunked. Still, we need to parse this chunked body, and its format matches HTTP chunking and we would like to reuse akka's chunking support to handle this data if possible.

And before writing our own chunked entity parser, I am wondering if following options are possible:

- coercing akka into treating an entity as chunked, even though original request does not have Transfer-Encoding and does have Content-Length field.
   looks like decision on whether to use chunked entity is made in HttpRequestParser class. The logic seems to be hardcoded to use chunked entity iff Transfer-Encoding: chunked header field is present. Also the code would emit a failure if Content-Length field is present. Maybe there is a way to inject this field into http request before HttpRequestParser gets it?

- using existing facility from akka to unchunk a chunked http body
  chunked entity parsing is happening in HttpMessageParser class, but it is private. Is it possible to somehow reuse akka's parser for http chunked body?

- any other way of getting unchunked bytes using akka functionality?

Sample request which needs to be handled below (the bytes it is trying to transfer are:
abcdefghijklmnopqrstuvwxyz
01234567890112345678901234
!@#$%^&*()-=[]{};':',.<>/?
01234567890112345678901234
abcdefghijklmnopqrstuvwxyz):

PUT /n-b-n-o/objectname0 HTTP/1.1
Host: 127.0.0.1:1234
x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD
Authorization: ...
X-Amz-Date: 20170616T205827Z
Content-Encoding: text/plain
User-Agent: aws-sdk-java/1.10.67 Linux/2.6.18-308.4.1.el5 Java_HotSpot(TM)_64-Bit_Server_VM/25.73-b02/1.8.0_73
amz-sdk-invocation-id: d1bcc96f-3cd2-421e-8323-6b1dfac9552c
x-amz-decoded-content-length: 135
amz-sdk-retry: 0/0/
Content-Type: application/octet-stream
Content-Length: 308
Connection: Keep-Alive
Expect: 100-continue

87;chunk-signature=4ecb554ecb327a19beba7528233fe6575ea31f1a2d0a8a84a57e1676fb21153b
abcdefghijklmnopqrstuvwxyz
01234567890112345678901234
!@#$%^&*()-=[]{};':',.<>/?
01234567890112345678901234
abcdefghijklmnopqrstuvwxyz

0;chunk-signature=0256bdb04c3ad7de9392781ea8ea8fe3e3bcff89713754e0e1869aeac5adc75f


Thanks!

[Christian Schmitt]

Actually the request doesn't look like a "Transfer-Encoding" chunked. Request. It's a special "encoding" of AWS S3. So you need to create your own parser for that.

[A.i.]

Thanks for looking at this. What incompatibilities with chunked HTTP format do you see? The request body looks fully compatible with https://tools.ietf.org/html/rfc2616#section-3.6.1 to me. To validate this, I modified aforementioned sample request by adding Transfer-Encoding: chunked field and removing Content-Length while leaving request body unchanged and fed it to our service through telnet - and yes, akka was able to handle this request as chunked without complaining.

[Christian Schmitt]

Ah yeah sorry was confused with something different.

Best Regards
Christian Schmitt

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/ktCOXdoe1UM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.