Adding an HttpFilter disables proxy communication

[Nir Asis]

I'm trying to record response resources and for that I have add a filter to the proxy. Problem is that once a filter is added no communication is passing through the proxy not the filter is called. Simply removing the filter makes everything work. Here's a sample code using WebDriver and a Browser as the proxy client. Any help will be appreciated.

public class GroupExampleInterceptor {

    public static void main(String[] args) throws InterruptedException {

        HttpResponseFilters responseFilters = new HttpResponseFilters() {

            @Override

            public HttpFilter getFilter(String s) {

                return defaultHttpFilter;

            }

            private DefaultHttpFilter defaultHttpFilter = new DefaultHttpFilter(new HttpResponseFilter() {

                @Override

                public HttpResponse filterResponse(HttpRequest httpRequest, HttpResponse httpResponse) {

                    return httpResponse;

                }

            });

        };

        int port = 9090;

        HttpProxyServer proxyServer = new DefaultHttpProxyServer(port, responseFilters);

        proxyServer.start();

        Proxy proxy = new Proxy();

        proxy.setProxyType(Proxy.ProxyType.MANUAL);

        String proxyStr = String.format("localhost:%d", port);

        proxy.setHttpProxy(proxyStr);

        proxy.setSslProxy(proxyStr);

        DesiredCapabilities capability = DesiredCapabilities.firefox();

        capability.setCapability(CapabilityType.PROXY, proxy);

        String urlString = "http://www.yahoo.com/";

        WebDriver driver = new FirefoxDriver(capability);

        driver.manage().timeouts().pageLoadTimeout(30, TimeUnit.SECONDS);

        driver.get(urlString);

        driver.close();

        System.out.println("Driver closed");

        proxyServer.stop();

        System.out.println("Proxy stopped");

    }

}

 

[Nir Asis]

I'm adding some more information. Attached are two code examples, one without an interceptor and one with. In addition attached are the proxy logs for both cases. As you can see the one with the interceptor experiences an TooLongFrameException. Earlier in the log you might notice that in opposed to the example without the interceptor here it is decided that the response is not chunked.

[Nir Asis]

Well, looks like the problem is in one of the dependencies. I'm using both Selenium and LittleProxy and if Selenium jar is placed first in the classpath the problem appears. If it is placed second the problem is gone. I assume it is something related to different Netty version used by both this tools. I'll try to update when I'll have more details.

[Nir Asis]

More interesting results: even after changing the order of dependencies it still happened on some URLs. I have then discovered that DefaultHttpFilter which I've been using implements getMaxResponseSize() with a value of 1024 * 200. Changing this value to be 1024 * 2000 (one more zero) seems to solve the problem regardless of dependencies.

I'll appreciate an explanation about this number. Isn't it passed on some HTTP header to the server side? Aren't large responses being chunked? How should I choose the right value for this parameter?

[Nir Asis]

Hi,

I've downloaded directly from git.

version littleproxy-0.4-114-g93628b4

On Sun, Sep 9, 2012 at 9:36 AM, Muhil Rajaram <muhi...@gmail.com> wrote:

Hi, 

   Can you tell me which version of littleproxy you are using and where you have downloaded it?

thanks,

Muhil