Can't get /hosts or /blacklist to work

[Mad D0g]

I just started using BMP today, I'm on nodejs and using this - https://github.com/zzo/browsermob-node

Creating hars works but not /hosts or /blacklist (didn't try others yet).

these are the errors I'm getting:


getting this error from /blacklist

sending PUT to /proxy/9096/blacklist with {"regex":"http\:\/\/(.*?)\.facebook\.com","status":403}

/proxy/9096/blacklist
java.lang.ArrayIndexOutOfBoundsException: 0
    at com.google.sitebricks.routing.DefaultPageBook$MethodTuple.call(DefaultPageBook.java:678)
    at com.google.sitebricks.routing.DefaultPageBook$MethodTuple.call(DefaultPageBook.java:665)
    at com.google.sitebricks.routing.DefaultPageBook$PageTuple.callAction(DefaultPageBook.java:544)
    at com.google.sitebricks.routing.DefaultPageBook$PageTuple.doMethod(DefaultPageBook.java:528)
    at com.google.sitebricks.routing.WidgetRoutingDispatcher.fireEvent(WidgetRoutingDispatcher.java:137)
    at com.google.sitebricks.routing.WidgetRoutingDispatcher.bindAndReply(WidgetRoutingDispatcher.java:90)
    at com.google.sitebricks.routing.WidgetRoutingDispatcher.dispatch(WidgetRoutingDispatcher.java:73)
    at com.google.sitebricks.DebugModeRoutingDispatcher.dispatch(DebugModeRoutingDispatcher.java:63)
    at com.google.sitebricks.SitebricksFilter.doFilter(SitebricksFilter.java:41)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.sitebricks.HiddenMethodFilter.doFilter(HiddenMethodFilter.java:74)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1323)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:474)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:935)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:404)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:870)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:346)
    at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
    at org.eclipse.jetty.server.HttpConnection$RequestHandler.messageComplete(HttpConnection.java:1084)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:846)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
    at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:520)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:528)
    at java.lang.Thread.run(Thread.java:695)


getting this error from /hosts

sending POST to /proxy/9097/hosts with {"xxx.facebook.com": "127.0.0.0.1"}

WARN 01/19 22:11:10 o.e.j.u.log          - /proxy/9093/hosts
com.google.sitebricks.routing.EventDispatchException: Exception [RuntimeException - "Unable to obtain input stream from servlet request (was it already used or closed elsewhere?). Error:
No content to map to Object due to end of input"] thrown by event method [public com.google.sitebricks.headless.Reply net.lightbody.bmp.proxy.bricks.ProxyResource.remapHosts(int,com.google.sitebricks.headless.Request)]

at com.google.sitebricks.SitebricksInternalModule$1$1.as(SitebricksInternalModule.java:107)
(See below for entire trace.)

    at com.google.sitebricks.routing.DefaultPageBook$MethodTuple.call(DefaultPageBook.java:678)
    at com.google.sitebricks.routing.DefaultPageBook$MethodTuple.call(DefaultPageBook.java:665)
    at com.google.sitebricks.routing.DefaultPageBook$PageTuple.callAction(DefaultPageBook.java:544)
    at com.google.sitebricks.routing.DefaultPageBook$PageTuple.doMethod(DefaultPageBook.java:528)
    at com.google.sitebricks.routing.WidgetRoutingDispatcher.fireEvent(WidgetRoutingDispatcher.java:137)
    at com.google.sitebricks.routing.WidgetRoutingDispatcher.bindAndReply(WidgetRoutingDispatcher.java:90)
    at com.google.sitebricks.routing.WidgetRoutingDispatcher.dispatch(WidgetRoutingDispatcher.java:73)
    at com.google.sitebricks.DebugModeRoutingDispatcher.dispatch(DebugModeRoutingDispatcher.java:63)
    at com.google.sitebricks.SitebricksFilter.doFilter(SitebricksFilter.java:41)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.sitebricks.HiddenMethodFilter.doFilter(HiddenMethodFilter.java:74)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1323)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:474)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:935)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:404)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:870)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:346)
    at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
    at org.eclipse.jetty.server.HttpConnection$RequestHandler.messageComplete(HttpConnection.java:1084)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:846)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
    at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:520)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:528)
    at java.lang.Thread.run(Thread.java:695)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.google.sitebricks.routing.DefaultPageBook$MethodTuple.call(DefaultPageBook.java:671)
    ... 34 more
Caused by: java.lang.RuntimeException: Unable to obtain input stream from servlet request (was it already used or closed elsewhere?). Error:
No content to map to Object due to end of input
    at com.google.sitebricks.SitebricksInternalModule$1$1.as(SitebricksInternalModule.java:107)
    at net.lightbody.bmp.proxy.bricks.ProxyResource.remapHosts(ProxyResource.java:389)
    ... 39 more
Caused by: java.io.EOFException: No content to map to Object due to end of input
    at org.codehaus.jackson.map.ObjectMapper._initForReading(ObjectMapper.java:2173)
    at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2125)
    at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1441)
    at com.google.sitebricks.client.transport.JacksonJsonTransport.in(JacksonJsonTransport.java:105)
    at com.google.sitebricks.SitebricksInternalModule$1$1.as(SitebricksInternalModule.java:104)
    ... 40 more

[Mad D0g]

oh, and I remembered that (according to the docs) one method (/rewrite) takes a parameter called "matchRegex", but another (/blacklist) takes a parameter called "regex" - could there be a mixup there?

[Mad D0g]

I also tried sending the data as key=value&key2=value2 with same result :-(

[Patrick Lightbody]

What's the code you're using to send the commands? Can you try using the straight HTTP REST protocol and see if that works? The problem could be either with the node driver or with BMP so this will help me debug it for you. Any steps to reproduce will be welcome!

--
 
---
You received this message because you are subscribed to the Google Groups "BrowserMob Proxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-pro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Mad D0g]

is there an easy way of sending REST commands from the command line? I'll google around.
I'm using the method `doReq`, which everything else in the library goes through. I switched to a key=value approach after I noticed that's how the other commands do it.

I'm still getting errors, for /blacklist, nothing new. but for /hosts, I'm getting:

sending POST to /proxy/9097/hosts with xxx.facebook.com=127.0.0.0.1

Unexpected character ('x'): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: org.eclipse.jetty.server.HttpInput@715a64e6; line: 1, column: 2]"] thrown by event method [public com.google.sitebricks.headless.Reply net.lightbody.bmp.proxy.bricks.ProxyResource.remapHosts(int,com.google.sitebricks.headless.Request)]

at com.google.sitebricks.SitebricksInternalModule$1$1.as(SitebricksInternalModule.java:107)

I tried surrounding it with quotes and got this funny thing:

Can not deserialize instance of java.util.LinkedHashMap out of VALUE_STRING token

[Mad D0g]

Ok, I'm sorry, /hosts works, I just send a stringified JSON object and that worked with /hosts. I expected the library to stringify but it just checked to see if it was a string.

I tried doing the same with /blacklist but still getting the same error about array being out of bounds - documentation says two objects - regex & status, why array?

[Mad D0g]

Also, do you know why har files created by BMP are not compatible with this viewer? http://www.softwareishard.com/blog/har-viewer/
I'm looking for an embeddable viewer and used this one before.

Thanks

[Mad D0g]

Oh, you asked for the code, I simplified it to this:

proxy.doReq("PUT", 'proxy/[port]/blacklist', JSON.stringify({

    regex: 'http:\/\/(.*?)\.facebook\.com',
    status: 403

});

So to recap - I made /hosts work by adding the stringify command, but it didn't help with /blacklist

Thanks

[Patrick Lightbody]

OK so the problem is that our API is just kind of inconsistent. The /hosts API takes a JSON payload of key/value pairs to do the mapping. But the /blacklist API takes HTTP request parameters of "regex" and "status" and no body should be submitted at all. So change your URL to be something like /proxy/[port]/blacklist?regex=...&status=... and you'll be OK.

Patches welcome to clean up the API of course :)

[Patrick Lightbody]

The problem is your regex. I just checked in a test that shows off the right regex:

https://github.com/lightbody/browsermob-proxy/commit/30a8978dd87205d25e88853fb33a6517b3e01f20

On Tue, Feb 4, 2014 at 9:20 AM, Mad D0g <madd0g...@gmail.com> wrote:

Thanks Patrick,

I do not get errors back from /blacklist anymore, but it doesn't seem to work. I tried these with status 403.

facebook.com
.*facebook.com
\.\*facebook.com

I still see 200 and content length in the har files.

[Mad D0g]

it says message deleted again? I don't feel very welcome :-\