A way to handle invalid http requests/headers with Vertx?

[javadevmtl]

Using 2.1.1

My verticle...

import org.vertx.java.core.Handler;

import org.vertx.java.core.http.HttpServerRequest;

import org.vertx.java.platform.Verticle;

import org.vertx.java.core.VoidHandler;

import org.vertx.java.core.buffer.Buffer;

public class Server extends Verticle {

public void start() {

vertx.createHttpServer().requestHandler(new Handler<HttpServerRequest>() {

public void handle(final HttpServerRequest req) {

final String clientHost = req.remoteAddress().getHostString();

System.out.println("CONNECTED: " + clientHost);

req.dataHandler(new Handler<Buffer>() {

public void handle(Buffer buffer) {

System.out.println("BUFFER: " + buffer);

}

});

req.endHandler(new VoidHandler() {

protected void handle() {

req.response().end("Hello World!");

System.out.println("END!");

}

});

}

}).listen(80);

}

}

If a client connects and sends invalid headers we get the CONNECTED println above. But non of the handlers fire. Is there a way to throw an exception if invalid headers are received or handle it and reply back with a 400 bad request?

The culprit request is coming from our monitoring software Whats Up Gold which is set up as follows

Send=GET /someFolder/somfile \r\n

Everything after the Send=

It was setup a long time ago and it works by pure fluke.

The management argument is that this works with our IIS servers and our applications and that the Vert.x proxy must be transparent. I kind of agree and disagree. I disagree with management because HTTP is supposed to be a standard. But 15 years in building HTTP applications, even the biggest of biggest clients (we are talking big brand names) can do the stupidest things and not have proper standard HTTP clients.

Also the above works with google.com. So at least it's able to capture the fact and return a response.

With the vertx verticle it connects but how can we handle it? Maybe it's not something in Vert.x but maybe I can somehow catch it in my verticle?

Thanks

[javadevmtl]

I added

System.println("PATH: " + req.path());

And it printed /bad-request

So I guess I could use that. But there's nothing else we can capture? Like what was exactly sent so we can log it etc...

[javadevmtl]

I also added

System.println("METHOD: " + req.method());

And it printed GET

So vertx is doing some parsing...

Maybe the path should not be replaced with /bad-request and it should be thrown as an exception?

[Tim Fox]

What do you mean by "invalid headers"? Do you have a reproducer?

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Randall Richard]

Wouldn't implementing req.exceptionHandler() be called accordingly?

-Randall

[javadevmtl]

Hi Tim and Randall

I supose I mean "malformed"

I posted the culprit header in my original post. Here it is again. I don't think I can provide a reproducer except my verticle since it's the client (monitoring system) sending the header. Someone typed that in 10 years ago and it's been working ever since. But not when fronting our server with the vert-x proxy, monitoring begins to fail. And the argument is it has to have exact functionality as if it was the original IIS server otherwise I can't use vert.x :(

GET /somepath/somefile?SomeQueryString=bla\r\n

If I add the HTTP/1.1 version and CRLFCRLF (\r\n\r\n) vert.x will handle it.

The monitoring system will send exactly what was typed above and \r\n is converted to the actual CRLF bytes. 

- IIS/application will process this.

- google.com replied back with a HTTP 302 so it's handling it somehow. 

- Tomcat with the Request Info sample processed it no exceptions thrown all info displayed.

I guess they are more lax in processing?

I also provided my Verticle below again and yes Randal I thought of exception handler also but nothing got caught. 

req.method() returns GET

req.path() returns /bad-request

req.uri() returns /bad-request

So vertx is doing some parsing. It would be nice to somehow allows us to catch the fact and handle it without having vertx modidy anything. So path and URI should be returned as sent by the client.

The latest verticle...

import org.vertx.java.core.Handler;

import org.vertx.java.core.http.HttpServerRequest;

import org.vertx.java.platform.Verticle;

import org.vertx.java.core.VoidHandler;

import org.vertx.java.core.buffer.Buffer;

import java.util.Map;

public class Server extends Verticle {

 public void start() {

          vertx.createHttpServer().requestHandler(new Handler<HttpServerRequest>() {

                     public void handle(final HttpServerRequest req) {

                              req.exceptionHandler(new Handler<Throwable>() {

                                public void handle(Throwable t) {

                                              t.printStackTrace();

                                   }

                              });

                                final String clientHost = req.remoteAddress().getHostString();

                         System.out.println("CONNECTED: " + clientHost);

                        System.out.println("METHOD: " + req.method());

                         System.out.println("PATH: " + req.path());//This prints /bad-request

                             System.out.println("URI: " + req.uri());

                               

                               //req.response().end("ERR!");

                         

                                for (Map.Entry<String, String> header: req.headers().entries()) {

                                      System.out.println("KEY: " + header.getKey() + ": " + header.getValue());

[Alexander Lehmann]

You can reproduce this with the example server from the web page (add a logging output for file) and telnet

telnet localhost 8080
asdfasdf

vertx run Server.java
Succeeded in deploying verticle
/bad-request

[javadevmtl]

Yes, Alexandlre it can be tested that way! 

Also, I tested some more servers.

Simply put...

- Jetty 9: GET /test-spec/test\r\n works returns expected output

- Tomcat 8: GET /examples/servlets/servlet/RequestInfoExample\r\n works returns expected output

- IIS 7: GET /somepath/MyCustomISAPI.dll\r\n works returns expected ouput

- Vertx 2.1.1: GET /somepath/something\r\n does not work see below

- Vertx 2.1.1: GET /somepath/something HTTP/1.1\r\n\r\n works output is based on what is in endHandler().of posted verticle

I installed the above servers and tested the sample applications shipped with the servers using: GET, no HTTP version and single CRLF and the sample applications returned their expected output.

Based on the verticle I provided in the above posts. The requestHandler will fire, but no exception is caught nor is dataHandler(), endHandler() called. So there no way for my vertx app to process as expected. Is it right, is it wrong? Should vertx replace the path with /bad-request?

It seems that Jetty, Tomcat and IIS are more forgiving with NO HTTP version and single CRLF

[Tim Fox]

On 16/07/14 18:42, javadevmtl wrote:

Hi Tim and Randall

I posted the culprit header in my original post. Here it is again. I don't think I can provide a reproducer except my verticle since it's the client (monitoring system) sending the header. Someone typed that in 10 years ago and it's been working ever since. But not when fronting our server with the vert-x proxy, monitoring begins to fail. And the argument is it has to have exact functionality as if it was the original IIS server otherwise I can't use vert.x :(

GET /somepath/somefile?SomeQueryString=bla\r\n

That's not an invalid header - it's an invalid (or malformed) HTTP request. There are no HTTP headers there.

So the monitoring system will send exactly what was typed above and \r\n is converted to the actual CRLF bytes. IIS will process this. I tried google.com and google replied back with a HTTP 302 so it's handling it somehow. I'm going to try apache also. But they seem to be a bit more lax on the processing of headers.

I also provided my Verticle above and yes Randal I thought of exception handler also but nothing got caught. 

req.method() returns GET

req.path() returns /bad-request

req.uri() returns /bad-request

Why not just check for "bad-request" and return a 400 - which seems to be the appropriate status code?

So vertx is doing some parsing.

I would rather have "malformed" exception thrown and have any original data that could be parsed available. So I could handle as required. like path should return the original path and uril also.

The latest verticle...

import org.vertx.java.core.Handler;

import org.vertx.java.core.http.HttpServerRequest;

import org.vertx.java.platform.Verticle;

import org.vertx.java.core.VoidHandler;

import org.vertx.java.core.buffer.Buffer;

import java.util.Map;

public class Server extends Verticle {

 public void start() {

          vertx.createHttpServer().requestHandler(new Handler<HttpServerRequest>() {

                     public void handle(final HttpServerRequest req) {

                              req.exceptionHandler(new Handler<Throwable>() {

                                public void handle(Throwable t) {

                                              t.printStackTrace();

                                   }

                              });

                                final String clientHost = req.remoteAddress().getHostString();

                         System.out.println("CONNECTED: " + clientHost);

                        System.out.println("METHOD: " + req.method());

                         System.out.println("PATH: " + req.path());//This prints /bad-request

                             System.out.println("URI: " + req.uri());

                               

                               //req.response().end("ERR!");

                         

                                for (Map.Entry<String, String> header: req.headers().entries()) {

                                      System.out.println("KEY: " + header.getKey() + ": " + header.getValue());

                              }

                                req.dataHandler(new Handler<Buffer>() {

                                public void handle(Buffer buffer) {

                                            System.out.println("BUFFER: " + buffer);

                                       }

                              });

                                req.endHandler(new VoidHandler() {

                     

                                       protected void handle() {

                                              req.response().end("Hello World!");

                                            System.out.println("END!");

                                    }

                              });

                    }

              }).listen(80);

 }

}

On Wednesday, 16 July 2014 12:29:05 UTC-4, Randall Richard wrote:

[Alexander Lehmann]

It's not really invalid, its just incredibly old. The HTTP implementation before 1.0 didn't support headers (either in request or in response) and just returned files.

http://www.w3.org/Protocols/HTTP/AsImplemented.html

the current http rfc suggests that servers should recognize http 0.9, 1.0 and 1.1

                                              req.response().end(<span style="color: #080;" class="styled-by-

...

[John Smith]

Yeah sorry for mixing technicall gargon. Its only a GET request (as above posts) with NO headers.

So if Alex is correct and from what I gathered (see above posts) is that all the latest Jetty Tomcat and IIS can handle that request properly. Is there something we can do?

Vertx http requestHandler is firing, though not the dataHandler() or the endHandler() and path is replaced with /bad-request. So it seems that Vertx can handle it partially.

I would much rather be able process the incomming request than return 400. So the path should stay intact? :)

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/EWJQqeQGmRw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

[Tim Fox]

I believe Vert.x only supports HTTP 1.1. I'm not sure how easy it would be to support earlier versions too - but this is really a Netty question.

You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[javadevmtl]

So this is a Norman thing?

I think there is support for 1.0 because req.version() in this case returns HTTP_1_0 so it doing some sort of fallback... But not sure.

[Alexander Lehmann]

I wonder if you could change the test request on whatsup to
Send=GET /someFolder/somefile HTTP/1.0\r\n\r\n

to get a http 1.0 compatible request (I had a similar problem with a BigIP switch at work once, where the monitoring requests were using http 0.9, which didn't recognize redirects since they do not show up without headers).

I can take a look at the vert.x server code, but if that is handled in netty,it may require changes there.

[Alexander Lehmann]

Ok, found the bad-request code, that is in netty (io.netty.handler.codec.http.HttpRequestDecoder), so as long as netty doesn't support http 0.9 requests, it will work in vert.x

[John Smith]

I did it seems to be ok. I managed to convinve pointy haired boss. Been monitoring and everything seems ok. Will track it for a bit and see how it goes. After all http 1.0 is from 1996 our company only since 1999 and service architecture came in 2001 so we should be good!

[Norman Maurer]

Netty only supported 1.0 and 1.1