BMP with Selenium in Python (Error after Java update)

367 views

Skip to first unread message

Adrián Bermúdez

unread,

Nov 24, 2017, 11:34:22 AM11/24/17

to BrowserMob Proxy

Hi all,

I was using browsermob-proxy 1.4.2 with selenium (different versions) in Python (browsermob-proxy = 0.8.0) to visit different websites with Firefox (different versions) to generate HAR files. Everything was working fine until I updated Java (OpenJDK) in my Ubuntu 14.02.

After the update, it is unable to connect to some websites (e.g. https://designshack.net/). The sites are loading normally without bmp though. I think there is a conflict between Java and BMP but I can't find out what's the problem.

I downgraded the version of Java and it is working fine. However, due to the some features of my project I must keep Java updated.

Any suggestions about what's the problem?

Below is the output of the bmp server.log using selenium 3.7.0 with Firefox 57.0.

Running BrowserMob Proxy using LittleProxy implementation. To revert to the legacy implementation, run the proxy with the command-line option '--use-littleproxy false'.

[INFO 2017-11-23T22:16:00,414 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4

[INFO 2017-11-23T22:16:00,487 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT

[INFO 2017-11-23T22:16:00,520 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null}

[WARN 2017-11-23T22:16:01,147 org.eclipse.jetty.util.log] (main) FAILED SelectChann...@0.0.0.0:8080: java.net.BindException: Address already in use

[WARN 2017-11-23T22:16:01,147 org.eclipse.jetty.util.log] (main) FAILED org.eclipse.jetty.server.Server@5cc52fc2: java.net.BindException: Address already in use

[ERROR 2017-11-23T22:16:01,148 net.lightbody.bmp.proxy.Main] (main) Failed to start Jetty server. Aborting. java.net.BindException: Address already in use

at sun.nio.ch.Net.bind0(Native Method) ~[?:1.7.0_151]

at sun.nio.ch.Net.bind(Net.java:463) ~[?:1.7.0_151]

at sun.nio.ch.Net.bind(Net.java:455) ~[?:1.7.0_151]

at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) ~[?:1.7.0_151]

at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) ~[?:1.7.0_151]

at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:162) ~[browsermob-dist-2.1.4.jar:?]

at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:356) ~[browsermob-dist-2.1.4.jar:?]

at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:240) ~[browsermob-dist-2.1.4.jar:?]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]

at org.eclipse.jetty.server.Server.doStart(Server.java:269) ~[browsermob-dist-2.1.4.jar:?]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]

at net.lightbody.bmp.proxy.Main.main(Main.java:64) [browsermob-dist-2.1.4.jar:?]

Exception in thread "main" net.lightbody.bmp.exception.JettyException: Unable to start Jetty server

at net.lightbody.bmp.proxy.Main.main(Main.java:68)

Caused by: java.net.BindException: Address already in use

at sun.nio.ch.Net.bind0(Native Method)

at sun.nio.ch.Net.bind(Net.java:463)

at sun.nio.ch.Net.bind(Net.java:455)

at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)

at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)

at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:162)

at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:356)

at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:240)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)

at org.eclipse.jetty.server.Server.doStart(Server.java:269)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)

at net.lightbody.bmp.proxy.Main.main(Main.java:64)

^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^$

[INFO 2017-11-23T22:15:59,240 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp965413474-16) Proxy listening with TCP transport

[INFO 2017-11-23T22:15:59,241 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp965413474-16) Proxy started at address: /0:0:0:0:0:0:0:0:8087

[ERROR 2017-11-23T22:16:01,556 org.littleshoot.proxy.impl.ProxyToServerConnection] (LittleProxy-6-ProxyToServerWorker-3) (HANDSHAKING) [id: 0xdfe9b294, L:/10.0.2.15:48866 - R:designshack.net/104.25.137.104:443]: Caught an exception on ProxyToServerConnection io.n$

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) [browsermob-dist-2.1.4.jar:?]

at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [browsermob-dist-2.1.4.jar:?]

at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [browsermob-dist-2.1.4.jar:?]

at java.lang.Thread.run(Thread.java:748) [?:1.7.0_151]

Caused by: java.lang.RuntimeException: java.security.InvalidKeyException: EC parameters error

at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1393) ~[?:1.7.0_151]

at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:516) ~[?:1.7.0_151]

at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:794) ~[?:1.7.0_151]

at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:762) ~[?:1.7.0_151]

at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.7.0_151]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1097) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:968) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:902) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) ~[browsermob-dist-2.1.4.jar:?]

... 15 more

Caused by: java.security.ProviderException: java.security.InvalidKeyException: EC parameters error

at sun.security.pkcs11.P11Key$P11ECPublicKey.getEncodedInternal(P11Key.java:1024) ~[sunpkcs11.jar:1.7.0_151]

at sun.security.pkcs11.P11Key.equals(P11Key.java:158) ~[sunpkcs11.jar:1.7.0_151]

at java.util.ArrayList.indexOf(ArrayList.java:298) ~[?:1.7.0_151]

at java.util.ArrayList.contains(ArrayList.java:281) ~[?:1.7.0_151]

at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:239) ~[?:1.7.0_151]

at sun.security.validator.Validator.validate(Validator.java:260) ~[?:1.7.0_151]

at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) ~[?:1.7.0_151]

at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) ~[?:1.7.0_151]

at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) ~[?:1.7.0_151]

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1464) ~[?:1.7.0_151]

at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:213) ~[?:1.7.0_151]

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:961) ~[?:1.7.0_151]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:901) ~[?:1.7.0_151]

at sun.security.ssl.Handshaker$1.run(Handshaker.java:899) ~[?:1.7.0_151]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_151]

at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1333) ~[?:1.7.0_151]

at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1123) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1008) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:902) ~[browsermob-dist-2.1.4.jar:?]

at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) ~[browsermob-dist-2.1.4.jar:?]

... 15 more

Caused by: java.security.InvalidKeyException: EC parameters error

at sun.security.ec.ECParameters.getAlgorithmParameters(ECParameters.java:284) ~[?:1.7.0_151]

at sun.security.ec.ECPublicKeyImpl.<init>(ECPublicKeyImpl.java:59) ~[?:1.7.0_151]

at sun.security.pkcs11.P11Key$P11ECPublicKey.getEncodedInternal(P11Key.java:1021) ~[sunpkcs11.jar:1.7.0_151]