Read the Original request

261 views
Skip to first unread message

Måñðj Kµmår

unread,
Mar 2, 2016, 6:03:03 AM3/2/16
to LittleProxy
Is it possible to read the original request that we get? For instance if I override with 

public HttpFilters filterRequest(final HttpRequest originalRequest, ChannelHandlerContext ctx) --> I am not able to get the complete details from the originalRequest (ie. URI + headers + body with any content, get/post/any payload).

Is there a way to open the requests and read them at this level (or at any point before forwarding to the server)?

Måñðj Kµmår

unread,
Mar 2, 2016, 9:54:57 AM3/2/16
to LittleProxy
I have added 
@Override
public int getMaximumRequestBufferSizeInBytes() {
return 512 * 1024;
} But I am not getting any content in the request. It is a SOAP call that is made. How can I read a SOAP payload?

Måñðj Kµmår

unread,
Mar 2, 2016, 10:08:07 AM3/2/16
to LittleProxy
Did a little more debugging. The SOAP calls are made from within my web application. When I checked the method from little proxy, am recording only CONNECT methods here. Is this how usually SOAP call are seen in a proxy?
Is there any other way to read the request that I am sending?

Måñðj Kµmår

unread,
Mar 3, 2016, 9:06:02 AM3/3/16
to LittleProxy
Ok. I've used the little-proxy mitm and am now able to see the complete requests that are made with the payload. But, now I am stuck with and error message, 

java.lang.UnsupportedOperationException: unsupported message type: DefaultFullHttpResponse (expected: ByteBuf, FileRegion)

I am a complete newbie to Proxies and netty as well. I was hoping that it would be easier with the filters in littleproxy.
Anyway any ideas on this is really appreciated. It will also be helpful if you even point me in the right direction, to take it up from here, if I have to write pipeline in netty. A zero-point start will be really helpful.

Sivasubramaniam Sivakumar

unread,
Mar 3, 2016, 11:31:36 AM3/3/16
to littl...@googlegroups.com
Hi,

Littleproxy-mitm should work without any changes to the netty pipeline. Can you please post your bootstrap and filter code? I can check.

Thanks

--
You received this message because you are subscribed to the Google Groups "LittleProxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to littleproxy...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Måñðj Kµmår

unread,
Mar 3, 2016, 2:15:33 PM3/3/16
to LittleProxy

I had not added any code. I am using the examples to start.

Anyway, my bootstrap is 

/* using the default MITM params */
DefaultHttpProxyServer.bootstrap()
        .withPort(PORT)
        .withManInTheMiddle(new CertificateSniffingMitmManager(new Authority(new File("the key store path which has the .pem file")
        ,"littleproxy-mitm"
        ,"Be Your Own Lantern".toCharArray()
        ,"LittleProxy-mitm"
        ,"littleproxy-mitm" + ", describe proxy here"
        ,"Certificate Authority"
        ,"littleproxy-mitm"
        ,"littleproxy-mitm"+ ", describe proxy purpose here, since Man-In-The-Middle is bad normally.")))
        .withFiltersSource(filtersSource)
        .withAllowLocalOnly(false)
        .withName("AProxy")
        .start();

The functions overridden in the filters are 

public HttpResponse proxyToServerRequest(HttpObject httpObject) {
                   
                    try {
                    if(httpObject instanceof FullHttpRequest) {
                    System.out.println(((FullHttpRequest) httpObject).getMethod()+":::"+((HttpRequest) httpObject).getUri());
                   
                    if(((FullHttpRequest) httpObject).getUri().contains("EntitlementService")) {
                    /* I do not reach this point at all while executing */
                    System.out.println("Return my custom reponse");
                   
                    //I make a call to a localhost service and return that response here.
                         
                    }
                    }
                    else {
return null; 
}
                    }
                    catch(UnsupportedOperationException e) {
                    return null;
                    }
                    }

and 

public int getMaximumRequestBufferSizeInBytes() {
                return 100*1024;
            }


I do not reach a point where I am try to filter my requests at all. The stack trace is 

22492  2016-03-03 19:48:53,445 ERROR [FilteringProxy-ClientToProxyWorker-3] impl.ClientToProxyConnection (?:?).?() - (AWAITING_INITIAL) [id: 0x4cd1d95a, /0:0:0:0:0:0:0:1:23868 => /0:0:0:0:0:0:0:1:9999]: Caught an exception on ClientToProxyConnection
java.lang.UnsupportedOperationException: unsupported message type: DefaultFullHttpResponse (expected: ByteBuf, FileRegion)
at io.netty.channel.nio.AbstractNioByteChannel.filterOutboundMessage(AbstractNioByteChannel.java:276)
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:654)
at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1054)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:658)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:716)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:651)
at io.netty.channel.PendingWriteQueue.removeAndWrite(PendingWriteQueue.java:186)
at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:447)
at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:432)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:688)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:718)
at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:706)
at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:741)
at io.netty.handler.codec.http.HttpObjectAggregator.decode(HttpObjectAggregator.java:131)
at io.netty.handler.codec.http.HttpObjectAggregator.decode(HttpObjectAggregator.java:54)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at org.littleshoot.proxy.impl.ProxyConnection$RequestReadMonitor.channelRead(ProxyConnection.java:707)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at org.littleshoot.proxy.impl.ProxyConnection$BytesReadMonitor.channelRead(ProxyConnection.java:684)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:971)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:854)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:745)

I am not sure as to what I should be looking here for. If you can let me know what variables or methods I should be looking here specifically for, I can get you more debug logs.

And thanks for your time and help. I really do appreciate it.

Sivasubramaniam Sivakumar

unread,
Mar 4, 2016, 7:46:04 PM3/4/16
to littl...@googlegroups.com
Hi,

I tried setting up a littleproxy instance with the same bootstrap and filter configuration and I was able to see the code hitting the "System.out.println("Return my custom reponse");" line. The only problem I see in your code is your MitmManager - 

.withManInTheMiddle(new CertificateSniffingMitmManager(new Authority(new File("the key store path which has the .pem file")

The keystore path is incorrect. Can you try with - 

.withManInTheMiddle(new CertificateSniffingMitmManager())

Thanks 

Måñðj Kµmår

unread,
Mar 5, 2016, 12:25:01 AM3/5/16
to LittleProxy
Without an Authority with the proper file path I end up with "Received fatal alert: bad_certificate" errors. That I why I created any Authority and passed it onto my CertificateSniffingMitmManager. I am pretty sure the keystore path is correct. Any more over my proxy is able to handle calls to localhost (my webapp running in my localhost), 302s, cookies from an external site seamlessly before running into the exception, It is only when I issue the SOAP call from the app that exception occurs. Even rest calls went through without problems.
I even tried hitting google/yahoo and signed in using my creds. It all went well (with the authority in the bootstrap I had mentioned). It fails only on the SOAP call from inside my webapp.

PS: I running fom eclipse on Win 7 x64.

Måñðj Kµmår

unread,
Mar 5, 2016, 12:27:03 AM3/5/16
to LittleProxy
And also this works when I return 0 in getMaximumRequestBufferSizeInBytes. No errors. No exceptions. When I return something non-zero, it fails.

Sivasubramaniam Sivakumar

unread,
Mar 5, 2016, 1:08:24 AM3/5/16
to littl...@googlegroups.com
Oh okay. I just went by the exact string you shared and thought you hadn't replaced it. Two things I can think of - 

1) If you are using Java 6, then you need to add a dependency for jzlib 1.1.2. This is required since you are using the HttpObjectAggregator (>0 for getMaximumRequestBufferSizeInBytes). But since you say that other requests work, then this might not be the issue.
2) There could be a problem with the way you construct the response (DefaultFullHttpResponse).  Can you share how you construct this without any private information? I am again not sure about this because you say the execution does not reach the point where you return your custom response. Can you try returning your custom HttpResponse for every request and share what happens?

Thanks.

Sivasubramaniam Sivakumar

unread,
Mar 5, 2016, 1:10:33 AM3/5/16
to littl...@googlegroups.com
Another thing to try - can you try using clientToProxyRequest() interceptor instead of proxyToServerRequest() and report what happens?

Thanks
Reply all
Reply to author
Forward
0 new messages