HTTP2 support in LittleProxy?

[hypatia3.1...@gmail.com]

Hi there, I work on a QA team at Akamai which is interested in BrowserMob Proxy. We'd like to see H2 support added to LittleProxy and BMProxy, and I'm wondering what some of the steps required for that to happen might be. Working on Netty 4.1 support for existing functionality seems like a first step; I've tried compiling and running tests with Beta 5, but get some test failures for EncryptedUDTChainedProxyTest that I haven't been able to figure out.

It looks like Frank Ganske (ganskef) is taking the role of one of the primary maintainers, and I also see a lot of commits by Jason Hoetger (jekh) who also works on BrowserMob Proxy. Do any of the current maintainers have an idea of the roadmap towards H2 support, please? Once we have a better idea of the roadmap and requirements for getting there, we may be willing to contribute towards development and/or testing.

Thanks,
~Tai Dickerson
(Spam-protection-ish: My work email is firstname...@akamai.com)

[Frank Ganske]

Hi Tai,

sorry, I have no primary role :-) .

I don't use UDT, but the test works for me with Netty 4.1 and I use
netty-all for tests in my application. It's not really integrated, but a
plug in replacement of Netty 4.0. Try this dependencies and note the
comments:

<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec-http</artifactId>
<version>${netty.version}</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-handler</artifactId>
<version>${netty.version}</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-example</artifactId>
<version>${netty.version}</version>
<scope>test</scope>
<!--
If you have problems in Eclipse, copy the Maven plugin to
eclipse/dropins/plugins/os-maven-plugin-1.2.3.Final.jar,
restart Eclipse and update project with [alt]+[F5], see:

https://github.com/trustin/os-maven-plugin#issues-with-eclipse-m2e-or-other-ides
-->
<!--If this fails, you could exclude netty-tcnative: -->
<!--
<exclusions>
<exclusion>
<artifactId>netty-tcnative</artifactId>
<groupId>io.netty</groupId>
</exclusion>
</exclusions>
-->
</dependency>


It would be great to integrate Netty 4.1. There are more interesting
features, like the proxy client capability in netty-handler-proxy. But,
in my opinion LttleProxy is not ready to move to such new features. Some
cleanup has to be done before. There are many open issues and pull
requests and it lacks a release.


Next, I'm not familiar with HTTP/2. Is it ready, in a stable state? It
need encryption as far as I know. You would like to intercept this for
test reasons? So it depends on a MITM module. It's an outstanding
refactoring in LittleProxy.

I think HTTP/2 should be evaluated separately at a first step. Keep it
simple as you can. Then lets see how to integrate it.


Regards Frank

> --
> You received this message because you are subscribed to the Google
> Groups "LittleProxy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to littleproxy...@googlegroups.com
> <mailto:littleproxy...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Frank Ganske]

Sorry again, outstanding was wrong. I mean (in German) ausstehend, offen
= overdue, open (?)

[hypatia3.1...@gmail.com]

Hi Frank, thanks for the reply,

Yes, HTTP2 is stable and in the wild. If you're using a recent version of Firefox or Chrome to connect to Twitter or Google, there's a good chance the connection is using H2.

You are also correct that HTTP2 requires a TLS/HTTPS connection. I thought BMProxy already had a working implementation of that, though - just without certificate spoofing. (jekh - if you see this would you be able to clarify, please?)

Netty 4.1 beta 4 and beyond include H2 support - that would be one of the reasons for upgrading to Netty 4.1.

You're suggesting we try a simple implementation and then look at integration?

~Tai

[Frank Ganske]

> Yes, HTTP2 is stable and in the wild. If you're using a recent version
> of Firefox or Chrome to connect to Twitter or Google, there's a good
> chance the connection is using H2.
>

I'm regular using Debian stable (Iceweasel 31.8.0), but with OS X,
Firefox uses HTTP/2.0 with Google. If I switch to a proxy connection via
my application (using Netty 4.1) it is be degraded to HTTP/1.1.

> You are also correct that HTTP2 requires a TLS/HTTPS connection. I
> thought BMProxy already had a working implementation of that, though -
> just without certificate spoofing. (jekh - if you see this would you be
> able to clarify, please?)
>

Yes, LittleProxy supports MITM with self signed certificates out of the
box. I think, this is used with BMProxy. Ok for you? What's the use case?

I'm not sure, whether Jason is reading this list. You could open an
issue or PR in LP or BMP of course.

> Netty 4.1 beta 4 and beyond include H2 support - that would be one of
> the reasons for upgrading to Netty 4.1.
>

I'll dig into the tests and examples, since I'm starting some days in
vacation, but this depends on Strandwetter. :-)

> You're suggesting we try a simple implementation and then look at
> integration?
>

Yes, your welcome. Try to build an independent test. You've seen an
example client using HTTP/2.0 with Netty somewhere? Have a look at this
branch, which is using Netty 4.1 with LittleProxy:

https://github.com/ganskef/LittleProxy-mitm/tree/netty_proxy_client

Regards Frank

[je...@outlook.com]

Hi Tai -- to answer your initial question, we do not have any sort of official roadmap for HTTP/2 support in LittleProxy, but only because of a lack of resources, not a lack of desire to implement it. If you or your colleagues are eager to contribute design, development, and/or testing, that would be fantastic. Since I don't believe the Netty maintainers have any plans to backport HTTP/2 support to 4.0, we'll have to figure out the logistics of supporting two versions of Netty. I'll open a github issue for Netty 4.1 support where we can all discuss Netty 4.1 + LP integration, and we can springboard off of that into the changes needed for HTTP/2 support.

Regarding the test failures you noticed with the Netty 4.1 beta, I've encountered the same issues and I believe I have a fix. I'll submit a PR for it shortly.

> I'm not sure, whether Jason is reading this list. You could open an issue or PR in LP or BMP of course.

@ganskef -- my notifications for this list weren't set up, so I just fixed that :)