UnknownHostException when host name is not in /etc/hosts
1,172 views
Skip to first unread message
André Matheus
Nov 13, 2013, 8:49:57 AM11/13/13
to browserm...@googlegroups.com
Hi,
here at work we're using BrowserMob Proxy to run our selenium tests against different environments.
When running theses tests on a developer or QA machine, we change the /etc/hosts file so that
hostnames get the correct IP for each environment (staging of each team, integration, QA).
To run theses tests on Jenkins we're using BrowserMob Proxy to point these hostnames the same way.
Doing that, I've noticed that when I add a host mapping for a host not reachable nor defined in /etc/hosts,
I get UnknownHostException, like this:
INFO 11/13 13:19:31 n.l.b.p.h.BrowserMo~ - java.net.UnknownHostException: testedehost: Name or service not known when requesting http://testedehost/
The way I've made it work here was change BrowserMobHttpClient.java, creating a DnsResolver that tries
the hostNameResolver (which respects the mappings defined in the proxy) first and after that trying
normal resolution:
diff -r bmghbeta8/browsermob-proxy/src/main/java/net/lightbody/bmp/proxy/http/BrowserMobHttpClient.java browsermob-proxy-browsermob-proxy-2.0-beta-8/src/main/java/net/lightbody/bmp/proxy/http/BrowserMobHttpClient.java
16,18c16
< import org.apache.http.conn.ClientConnectionRequest;
< import org.apache.http.conn.ConnectionPoolTimeoutException;
< import org.apache.http.conn.ManagedClientConnection;
---
> import org.apache.http.conn.*;
27a26,27
> import org.apache.http.impl.conn.PoolingClientConnectionManager;
> import org.apache.http.impl.conn.SystemDefaultDnsResolver;
43a44
> import java.net.InetAddress;
45a47
> import java.net.UnknownHostException;
88c90
< private ThreadSafeClientConnManager httpClientConnMgr;
---
> private PoolingClientConnectionManager httpClientConnMgr;
124c126,138
< httpClientConnMgr = new ThreadSafeClientConnManager(schemeRegistry) {
---
> DnsResolver dnsResolver = new SystemDefaultDnsResolver() {
> @Override public InetAddress[] resolve(String host) throws UnknownHostException {
> try {
> InetAddress a = hostNameResolver.resolve(host);
> return new InetAddress[] {a};
> } catch (IOException e) {
>
> }
> return super.resolve(host);
> }
> };
>
> httpClientConnMgr = new PoolingClientConnectionManager(schemeRegistry, dnsResolver) {
I've commited these changes to a fork I've created at
https://github.com/andrematheus/browsermob-proxy
I would like to know if this line of reasoning is correct and if so, if I can merge these changes to
BrowserMob Proxy. If I haven't made myself clear please ask, I'll be glad to help.
I've made some tests using this script:
------------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash
TESTHOST=testedehost
TESTMAPPING=127.0.0.1
# start browsermob-proxy
sh ./bin/browsermob-proxy --port=25000 &
BMOBPID=$!
sleep 5
# setup proxy
curl -X POST -d "port=25100" http://localhost:25000/proxy
# setup one host
curl -X POST -v --silent --output /dev/stderr --write-out "%{http_code}" \
-H 'Accept: application/json' \
-H 'Content-type: application/json' \
http://localhost:25000/proxy/25100/hosts -d "{ \"${TESTHOST}\" : \"${TESTMAPPING}\" }"
curl --proxy http://localhost:25100 http://$TESTHOST
kill -9 ${BMOBPID}
------------------------------------------------------------------------------------------------------------------------------------------
Just save somewhere then run it from the directory where you unzipped browsermob.
The curl, assuming you don't have "testedehost" in your /etc/hosts file, will return an error page:
------------------------------------------------------------------------------------------------------------------------------------------
200<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Problem loading page</title>
<link rel="stylesheet" href="chrome://global/skin/netError.css" type="text/css" media="all" />
<link rel="icon" type="image/png" id="favicon" href="chrome://global/skin/icons/warning-16.png"/>
</head>
<body>
<!-- PAGE CONTAINER (for styling purposes only) -->
<div id="errorPageContainer">
<!-- Error Title -->
<div id="errorTitle">
<h1 id="errorTitleText">Server not found</h1>
.....
------------------------------------------------------------------------------------------------------------------------------------------
Thanks,
André
here at work we're using BrowserMob Proxy to run our selenium tests against different environments.
When running theses tests on a developer or QA machine, we change the /etc/hosts file so that
hostnames get the correct IP for each environment (staging of each team, integration, QA).
To run theses tests on Jenkins we're using BrowserMob Proxy to point these hostnames the same way.
Doing that, I've noticed that when I add a host mapping for a host not reachable nor defined in /etc/hosts,
I get UnknownHostException, like this:
INFO 11/13 13:19:31 n.l.b.p.h.BrowserMo~ - java.net.UnknownHostException: testedehost: Name or service not known when requesting http://testedehost/
The way I've made it work here was change BrowserMobHttpClient.java, creating a DnsResolver that tries
the hostNameResolver (which respects the mappings defined in the proxy) first and after that trying
normal resolution:
diff -r bmghbeta8/browsermob-proxy/src/main/java/net/lightbody/bmp/proxy/http/BrowserMobHttpClient.java browsermob-proxy-browsermob-proxy-2.0-beta-8/src/main/java/net/lightbody/bmp/proxy/http/BrowserMobHttpClient.java
16,18c16
< import org.apache.http.conn.ClientConnectionRequest;
< import org.apache.http.conn.ConnectionPoolTimeoutException;
< import org.apache.http.conn.ManagedClientConnection;
---
> import org.apache.http.conn.*;
27a26,27
> import org.apache.http.impl.conn.PoolingClientConnectionManager;
> import org.apache.http.impl.conn.SystemDefaultDnsResolver;
43a44
> import java.net.InetAddress;
45a47
> import java.net.UnknownHostException;
88c90
< private ThreadSafeClientConnManager httpClientConnMgr;
---
> private PoolingClientConnectionManager httpClientConnMgr;
124c126,138
< httpClientConnMgr = new ThreadSafeClientConnManager(schemeRegistry) {
---
> DnsResolver dnsResolver = new SystemDefaultDnsResolver() {
> @Override public InetAddress[] resolve(String host) throws UnknownHostException {
> try {
> InetAddress a = hostNameResolver.resolve(host);
> return new InetAddress[] {a};
> } catch (IOException e) {
>
> }
> return super.resolve(host);
> }
> };
>
> httpClientConnMgr = new PoolingClientConnectionManager(schemeRegistry, dnsResolver) {
I've commited these changes to a fork I've created at
https://github.com/andrematheus/browsermob-proxy
I would like to know if this line of reasoning is correct and if so, if I can merge these changes to
BrowserMob Proxy. If I haven't made myself clear please ask, I'll be glad to help.
I've made some tests using this script:
------------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash
TESTHOST=testedehost
TESTMAPPING=127.0.0.1
# start browsermob-proxy
sh ./bin/browsermob-proxy --port=25000 &
BMOBPID=$!
sleep 5
# setup proxy
curl -X POST -d "port=25100" http://localhost:25000/proxy
# setup one host
curl -X POST -v --silent --output /dev/stderr --write-out "%{http_code}" \
-H 'Accept: application/json' \
-H 'Content-type: application/json' \
http://localhost:25000/proxy/25100/hosts -d "{ \"${TESTHOST}\" : \"${TESTMAPPING}\" }"
curl --proxy http://localhost:25100 http://$TESTHOST
kill -9 ${BMOBPID}
------------------------------------------------------------------------------------------------------------------------------------------
Just save somewhere then run it from the directory where you unzipped browsermob.
The curl, assuming you don't have "testedehost" in your /etc/hosts file, will return an error page:
------------------------------------------------------------------------------------------------------------------------------------------
200<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Problem loading page</title>
<link rel="stylesheet" href="chrome://global/skin/netError.css" type="text/css" media="all" />
<link rel="icon" type="image/png" id="favicon" href="chrome://global/skin/icons/warning-16.png"/>
</head>
<body>
<!-- PAGE CONTAINER (for styling purposes only) -->
<div id="errorPageContainer">
<!-- Error Title -->
<div id="errorTitle">
<h1 id="errorTitleText">Server not found</h1>
.....
------------------------------------------------------------------------------------------------------------------------------------------
Thanks,
André
Patrick Lightbody
Nov 16, 2013, 2:24:29 PM11/16/13
to browserm...@googlegroups.com
André,
Really sorry for not commenting to your Github comments yet. I honestly can’t remember why we used XBill DNS vs the regular in-JVM DNS, but I’m open to adding some more flexibility to meet your needs. In browsing through the code, was it obvious to you why I did this? :)
Patrick
--
---
You received this message because you are subscribed to the Google Groups "BrowserMob Proxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-pro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Patrick Lightbody
Nov 17, 2013, 1:27:07 PM11/17/13
to browserm...@googlegroups.com
Apologies -- it turns out two separate people asked for slightly similar fixes, but the approach is different. In my sleep-deprived stupor I assumed you (André) were the same person that opened this pull request:
I think the three of us (you, me, and Alex) should discuss a bit more on the approach. I am still not remembering why I switched to XBill DNS, but I'm certain I had a reason at one point :)
Shall we discuss here or on PR 45?
Patrick
To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-proxy+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages