Akka-http: how to deal with incorrect http headers

[Rüdiger Klaehn]

Hi all,

While playing around with akka-http, I wrote a minimal http proxy server:
https://gist.github.com/rklaehn/3aa3215046df2c0a7795

To test it, I configured a browser with this as proxy and browsed some
popular news web pages that load a zillion things from ad servers,
trackers and the like.

It works just fine, but I get a large number of warnings and the
occasional error in the log because apparently a lot of the
"secondary" traffic going over the proxy is not 100% correct according
to the HTTP spec. See some examples below.

Is it possible to tell akka-http to ignore errors such as
IllegalResponseException and just pass through everything? I don't see
a single line of my own code in the stack traces, so I am a bit at a
loss how to approach this.

Cheers,

Rüdiger

---
Opening connection to weltonline01.webtrekk.net
[WARN] [01/31/2015 17:33:42.631]
[MinimalProxy-akka.actor.default-dispatcher-70]
[ActorSystem(MinimalProxy)] Illegal request header: Illegal 'cookie'
header: Invalid input ',', expected tchar, '\r', WSP or '=' (line 1,
column 32): C=1; cid=-6251207954767438390,0,0,0,0;
uid=-6251207954767438390
^
Opening connection to track.adform.net
[WARN] [01/31/2015 17:33:42.679]
[MinimalProxy-akka.actor.default-dispatcher-56]
[ActorSystem(MinimalProxy)] Illegal response header: Illegal
'content-type' header: Invalid input ':', expected tchar, '\r', WSP or
'/' (line 1, column 13): Content-type: image/gif; charset=utf-8
^
[WARN] [01/31/2015 17:33:42.680]
[MinimalProxy-akka.actor.default-dispatcher-52]
[ActorSystem(MinimalProxy)] Explicitly set HTTP header 'content-type:
Content-type: image/gif; charset=utf-8' is ignored, illegal RawHeader
Opening connection to comcluster.cxense.com
[ERROR] [01/31/2015 17:33:42.716]
[MinimalProxy-akka.actor.default-dispatcher-78]
[akka://MinimalProxy/user/$a/flow-15752-broadcast] failure during
processing
akka.http.model.IllegalResponseException: Illegal character ' ' in header name
at akka.http.engine.client.HttpClient$$anonfun$2.applyOrElse(HttpClient.scala:89)
at akka.http.engine.client.HttpClient$$anonfun$2.applyOrElse(HttpClient.scala:86)
at akka.stream.impl.fusing.Collect.onPush(Ops.scala:37)
...

[rklaehn]

Here is an example of an URL that works fine without the proxy, but causes a parse error when going via the proxy:

http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34835.0;attach=633689

curl "http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34835.0;attach=633689"

=> lots of binary data

curl --proxy 127.0.0.1 "http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34835.0;attach=633689"

=> Illegal request-target: Invalid input ';', expected '+', query-char, '%', '&' or 'EOI' (line 1, column 59)

log:

[WARN] [01/31/2015 18:25:05.165] [Proxy-akka.actor.default-dispatcher-23] [ActorSystem(Proxy)] Illegal request, responding with status '400 Bad Request': Illegal request-target: Invalid input ';', expected '+', query-char, '%', '&' or 'EOI' (line 1, column 59): http://forum.nasaspaceflight.com/index.php?action=dlattach;topic=34835.0;attach=633689

   

[Rüdiger Klaehn]

I have looked at the reference.conf of akka-http-core-experimental
(1.0-M2) and found the following settings that should solve some of
these problems. But adding the configuration below as an
application.conf does not seem to make a difference WRT. parsing of
Uris. I still get a "400 Bad Request" for queries containing ";".

Any ideas?

application.conf:

akka.http {

parsing {

# Sets the strictness mode for parsing request target URIs.
# The following values are defined:
#
# `strict`: RFC3986-compliant URIs are required,
# a 400 response is triggered on violations
#
# `relaxed`: all visible 7-Bit ASCII chars are allowed
#
# `relaxed-with-raw-query`: like `relaxed` but additionally
# the URI query is not parsed, but delivered as one raw string
# as the `key` value of a single Query structure element.
#
uri-parsing-mode = relaxed-with-raw-query

# Enables/disables the logging of warning messages in case an incoming
# message (request or response) contains an HTTP header which cannot be
# parsed into its high-level model class due to incompatible syntax.
# Note that, independently of this settings, akka-http will accept messages
# with such headers as long as the message as a whole would still be legal
# under the HTTP specification even without this header.
# If a header cannot be parsed into a high-level model instance it will be
# provided as a `RawHeader`.
illegal-header-warnings = off

> --
>>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>>> Check the FAQ:
>>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+...@googlegroups.com.
> To post to this group, send email to akka...@googlegroups.com.
> Visit this group at http://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.

[driak]

Hello,

I'm running into the same problem, did you ever find out how to resolve this issue?

[Florian Rosenberg]

I'm seeing a similar problem, but not wit the URI but with the Strict-Transport-Header, it seems to be invalid, changing the conf to ignore it has no effect. I'm using a websocket though.

After debugging around for a while, I instrumented some code and see this:

Illegal 'strict-transport-security' header: Invalid input 'EOI', expected OWS, 'i' or token0 (line 1, column 18): max-age=31536000;

It may well be that the endpoint I'm sending it to does not treat this correctly and returns a 400 (Bad request), but I need to be able to ignore this somehow. 

[johannes...@lightbend.com]

Hi Florian,

can you clarify what needs improvement? Is that about client or server side?

Johannes