json-rpc behind a proxy server

109 views
Skip to first unread message

Dominic

unread,
Jan 2, 2008, 7:45:08 PM1/2/08
to json-rpc-client
Hi all,

I've discovered that json-rpc doesn't work too well behind an HTTP
proxy. The issue seems to be related to how that Apache commons
httpclient is instantiated. The HTTPSession class simply instantiates
a HttpClient object and doesn't allow you to modify any of the
parameters. Ideally, we need to do the following:

proxyHost = System.getProperty("deployment.proxy.http.host");
proxyPort = System.getProperty("deployment.proxy.http.port");
httpclient = new Client.HttpClient();
hostConfig = httpclient .getHostConfiguration();
hostConfig.setProxy(proxyHost, proxyPort);

Obviously one should check to see whether the "proxyHost" is set
before updating the host config object. This solution won't deal with
more complex situations (such as if an autoconfig parameter is set for
the proxy in the Java control panel).

Ideally the Apache httpclient authors should provide better support
for any proxy server settings which are configured in the Java control
panel.

Regards,
Dominic.

Sasha Ovsankin

unread,
Jan 6, 2008, 1:51:51 AM1/6/08
to json-rp...@googlegroups.com
Dominic --

I have added the Client.getHostConfiguration() method. You can find the patch against the current jabsorb trunk here:
http://issues.jabsorb.org/attachment.cgi?id=9
Let me know if this can work for you.

Ideally the Apache httpclient authors should provide better support
for any proxy server settings which are configured in the Java control
panel.
I am not sure to what extent this is possible within the scope of commons-httpclient. This setting in the Java control panel may be relevant for applets running within the (IE) browser; implementing proxy autoconfiguration may be much more complicated.

Regards,
-- Sasha

Dominic

unread,
Jan 10, 2008, 12:33:31 AM1/10/08
to json-rpc-client
Hi Sasha,

I have compiled the latest code and the getHostConfiguration()
effectively allows me to configure the proxy server for the Apache
commons HTTP client. The problem is that the
System.getProperty("deployment.proxy.http.host") call is not
sufficient for finding the proxy information.

In fact, after doing some research, there isn't a good technique (the
best option is the undocumented com.sun.java.browser.net.ProxyService
but even this doesn't work with Java WebStart).

Is it not possible to for us to use the standard HttpURLConnection
which is part of the JRE? The advantage is that this class at least
picks up any proxy configurations which are set in the Java control
panel. The other advantage is that this would be one less external
library (actually 2) that we would be dependent on.

Regards,
Dominic.

On Jan 6, 5:51 pm, Sasha Ovsankin <sa...@codebistro.com> wrote:
> Dominic --
> I have added the Client.getHostConfiguration() method. You can find the patch against the current jabsorb trunk here:http://issues.jabsorb.org/attachment.cgi?id=9Let me know if this can work for you.Ideally the Apache httpclient authors should provide better support for any proxy server settings which are configured in the Java control panel.I am not sure to what extent this is possible within the scope of commons-httpclient. This setting in the Java control panel may be relevant for applets running within the (IE) browser; implementing proxy autoconfiguration may be much more complicated.

Sasha Ovsankin

unread,
Jan 10, 2008, 1:22:38 AM1/10/08
to json-rp...@googlegroups.com
Dominic --

The problem is that the
System.getProperty("deployment.proxy.http.host") call is not
sufficient for finding the proxy information.
I am not surprised. Proxy parameters are browser-dependent and available only to the applet running in this browser; certainly not to the JWS application.

As much as I hate to say this, but you have to either run in the applet or set the proxy parameters manually in the application. There might be ways to squeeze these out of the system, but these ways would be system-dependent and very unreliable. Good news is that proxy servers are coming out of fashion and you need to deal with them much less than used to be.

Is it not possible to for us to use the standard HttpURLConnection
which is part of the JRE? The advantage is that this class at least
picks up any proxy configurations which are set in the Java control
panel. The other advantage is that this would be one less external
library (actually 2) that we would be dependent on.
Doesn't URLConnection instantiate HTTPConnection for http: protocol? Would you care to check this in the debugger?

Thanks,

Dominic

unread,
Jan 10, 2008, 5:37:31 AM1/10/08
to json-rpc-client
Hi Sasha,

the Apache commons HTTP client that you are using is simply an
alternative for the java.net.HttpURLConnection which is provided as
part of the standard Java libraries. The advantatge of using
java.net.HttpURLConnection is that it picks up the proxy settings from
the Java control panel (which can detect them from the browser). This
is why when I use Spring remoting (which uses the
java.net.HttpURLConnection class) I have no issues with proxy servers.
The Apache commons HTTP library doesn't call HttpURLConnection but is
rather a replacement which operates at the socket level.

Proxy servers will always be around for intranet environments. If apps
are hosted on the Internet and accessed from an intranet one will
always need to pass through a proxy.

Regards,
Dominic.

On Jan 10, 5:22 pm, Sasha Ovsankin <sa...@codebistro.com> wrote:
> Dominic --The problem is that the System.getProperty("deployment.proxy.http.host") call is not sufficient for finding the proxy information.I am not surprised. Proxy parameters are browser-dependent and available only to the applet running in this browser; certainly not to the JWS application.
> As much as I hate to say this, but you have to either run in the applet or set the proxy parameters manually in the application. There might be ways to squeeze these out of the system, but these ways would be system-dependent and very unreliable. Good news is that proxy servers are coming out of fashion and you need to deal with them much less than used to be.Is it not possible to for us to use the standard HttpURLConnection which is part of the JRE? The advantage is that this class at least picks up any proxy configurations which are set in the Java control panel. The other advantage is that this would be one less external library (actually 2) that we would be dependent on.Doesn't URLConnection instantiate HTTPConnection for http: protocol? Would you care to check this in the debugger?
> Thanks,
> -- Sasha
> Dominic wrote:Hi Sasha, I have compiled the latest code and the getHostConfiguration() effectively allows me to configure the proxy server for the Apache commons HTTP client. The problem is that the System.getProperty("deployment.proxy.http.host") call is not sufficient for finding the proxy information. In fact, after doing some research, there isn't a good technique (the best option is the undocumented com.sun.java.browser.net.ProxyService but even this doesn't work with Java WebStart). Is it not possible to for us to use the standard HttpURLConnection which is part of the JRE? The advantage is that this class at least picks up any proxy configurations which are set in the Java control panel. The other advantage is that this would be one less external library (actually 2) that we would be dependent on. Regards, Dominic. On Jan 6, 5:51 pm, Sasha Ovsankin<sa...@codebistro.com>wrote:Dominic -- I have added the Client.getHostConfiguration() method. You can find the patch against the current jabsorb trunk here:http://issues.jabsorb.org/attachment.cgi?id=9Letme know if this can work for you.Ideally the Apache httpclient authors should provide better support for any proxy server settings which are configured in the Java control panel.I am not sure to what extent this is possible within the scope of commons-httpclient. This setting in the Java control panel may be relevant for applets running within the (IE) browser; implementing proxy autoconfiguration may be much more complicated. Regards, -- Sasha Dominic wrote:Hi all, I've discovered that json-rpc doesn't work too well behind an HTTP proxy. The issue seems to be related to how that Apache commons httpclient is instantiated. The HTTPSession class simply instantiates a HttpClient object and doesn't allow you to modify any of the parameters. Ideally, we need to do the following: proxyHost = System.getProperty("deployment.proxy.http.host"); proxyPort = System.getProperty("deployment.proxy.http.port"); httpclient = new Client.HttpClient(); hostConfig = httpclient .getHostConfiguration(); hostConfig.setProxy(proxyHost, proxyPort); Obviously one should check to see whether the "proxyHost" is set before updating the host config object. This solution won't deal with more complex situations (such as if an autoconfig parameter is set for the proxy in the Java control panel). Ideally the Apache httpclient authors should provide better support for any proxy server settings which are configured in the Java control panel. Regards, Dominic.

Dominic

unread,
Jan 10, 2008, 5:37:45 AM1/10/08
to json-rpc-client
Hi Sasha,



On Jan 10, 4:33 pm, Dominic <dominic.cioccare...@gmail.com> wrote:
> Hi Sasha,
>
> I have compiled the latest code and the getHostConfiguration()
> effectively allows me to configure the proxy server for the Apache
> commons HTTP client. The problem is that the
> System.getProperty("deployment.proxy.http.host") call is not
> sufficient for finding the proxy information.
>
> In fact, after doing some research, there isn't a good technique (the
> best option is the undocumented com.sun.java.browser.net.ProxyService
> but even this doesn't work with Java WebStart).
>
> Is it not possible to for us to use the standard HttpURLConnection
> which is part of the JRE? The advantage is that this class at least
> picks up any proxy configurations which are set in the Java control
> panel. The other advantage is that this would be one less external
> library (actually 2) that we would be dependent on.
>
> Regards,
> Dominic.
>
> On Jan 6, 5:51 pm, Sasha Ovsankin <sa...@codebistro.com> wrote:
>
> > Dominic --
> > I have added the Client.getHostConfiguration() method. You can find the patch against the current jabsorb trunk here:http://issues.jabsorb.org/attachment.cgi?id=9Letme know if this can work for you.Ideally the Apache httpclient authors should provide better support for any proxy server settings which are configured in the Java control panel.I am not sure to what extent this is possible within the scope of commons-httpclient. This setting in the Java control panel may be relevant for applets running within the (IE) browser; implementing proxy autoconfiguration may be much more complicated.

Sasha Ovsankin

unread,
Jan 10, 2008, 8:57:48 PM1/10/08
to json-rp...@googlegroups.com
Dominic ---

I ran ClientTestCase in the debugger, put a breakpoint on line 78 of URLConnectionSession and got the following:
connection    HttpURLConnection  (id=905)
sun.net.www.protocol.http.HttpURLConnection:http://localhost:8083/jabsorb-trunk/JSON-RPC

From sun.net.www.protocol.http.HttpURLConnection :
public class HttpURLConnection extends java.net.HttpURLConnection {
...
So it seems, URLConnection already uses HttpURLConnection.

Am I missing something?

Thanks,
-- Sasha

P.S. org.apache.commons.httpclient.util.HttpURLConnection is also subsclass of java.net.HttpURLConnection but it doesn't implementthe usingProxy() method.

Dominic

unread,
Jan 11, 2008, 12:06:12 AM1/11/08
to json-rpc-client
Hi Sasha,

the Apache HTTP client is definately working at the socket level. If I
use JSON-RPC with your library and don't specify the proxy directly
(in the Apache HTTP object) the I will get the following error:

Caused by: org.jabsorb.client.ClientError: java.net.ConnectException:
Connection refused: connect
at org.jabsorb.client.HTTPSession.sendAndReceive(HTTPSession.java:
117)
at org.jabsorb.client.Client.invoke(Client.java:154)
at org.jabsorb.client.Client.invoke(Client.java:134)
at $Proxy0.getUsers(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:142)
... 39 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:
80)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:
122)
at
org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:
707)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:
387)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:
171)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:
397)
at org.jabsorb.client.HTTPSession.sendAndReceive(HTTPSession.java:97)
... 47 more

... it is clear that the Apache commons HTTP client is operating at
the socket level (java.net.socket). If I use a different remoting
protocol (which uses the the Sun HttpURLConnection class) then the
browser proxy settings are picked up and all is well. As an example,
if I force a connection failure with another protocol which uses the
Sun HTTP library, I get:

java.net.ConnectException: Connection refused: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown
Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown
Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown
Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at javax.swing.JEditorPane.getStream(Unknown Source)

I hope that this illustrates the issue. Typically there should be no
hard requirement to use the Apache HTTP library. The one which comes
with the JRE is quite advanced now and the advantage is that it
integrates better with the host environment.

Regards,
Dominic.


On Jan 11, 12:57 pm, Sasha Ovsankin <sa...@codebistro.com> wrote:
> Dominic ---
> I ran ClientTestCase in the debugger, put a breakpoint on line 78 of URLConnectionSession and got the following:connection HttpURLConnection (id=905)
> sun.net.www.protocol.http.HttpURLConnection:http://localhost:8083/jabsorb-trunk/JSON-RPCFrom sun.net.www.protocol.http.HttpURLConnection :public class HttpURLConnection extends java.net.HttpURLConnection {
> ...So it seems, URLConnection already uses HttpURLConnection.
> Am I missing something?
> Thanks,
> -- Sasha
> P.S. org.apache.commons.httpclient.util.HttpURLConnection is also subsclass of java.net.HttpURLConnection but it doesn't implementthe usingProxy() method.
> Dominic wrote:Hi Sasha, the Apache commons HTTP client that you are using is simply an alternative for the java.net.HttpURLConnection which is provided as part of the standard Java libraries. The advantatge of using java.net.HttpURLConnection is that it picks up the proxy settings from the Java control panel (which can detect them from the browser). This is why when I use Spring remoting (which uses the java.net.HttpURLConnection class) I have no issues with proxy servers. The Apache commons HTTP library doesn't call HttpURLConnection but is rather a replacement which operates at the socket level. Proxy servers will always be around for intranet environments. If apps are hosted on the Internet and accessed from an intranet one will always need to pass through a proxy. Regards, Dominic. On Jan 10, 5:22 pm, Sasha Ovsankin<sa...@codebistro.com>wrote:Dominic --The problem is that the System.getProperty("deployment.proxy.http.host") call is not sufficient for finding the proxy information.I am not surprised. Proxy parameters are browser-dependent and available only to the applet running in this browser; certainly not to the JWS application. As much as I hate to say this, but you have to either run in the applet or set the proxy parameters manually in the application. There might be ways to squeeze these out of the system, but these ways would be system-dependent and very unreliable. Good news is that proxy servers are coming out of fashion and you need to deal with them much less than used to be.Is it not possible to for us to use the standard HttpURLConnection which is part of the JRE? The advantage is that this class at least picks up any proxy configurations which are set in the Java control panel. The other advantage is that this would be one less external library (actually 2) that we would be dependent on.Doesn't URLConnection instantiate HTTPConnection for http: protocol? Would you care to check this in the debugger? Thanks, -- Sasha Dominic wrote:Hi Sasha, I have compiled the latest code and the getHostConfiguration() effectively allows me to configure the proxy server for the Apache commons HTTP client. The problem is that the System.getProperty("deployment.proxy.http.host") call is not sufficient for finding the proxy information. In fact, after doing some research, there isn't a good technique (the best option is the undocumented com.sun.java.browser.net.ProxyService but even this doesn't work with Java WebStart). Is it not possible to for us to use the standard HttpURLConnection which is part of the JRE? The advantage is that this class at least picks up any proxy configurations which are set in the Java control panel. The other advantage is that this would be one less external library (actually 2) that we would be dependent on. Regards, Dominic. On Jan 6, 5:51 pm, Sasha Ovsankin<sa...@codebistro.com>wrote:Dominic -- I hav e added the Client.getHostConfiguration() method. You can find the patch against the current jabsorb trunk here:http://issues.jabsorb.org/attachment.cgi?id=9Letmeknow if this can work for you.Ideally the Apache httpclient authors should provide better support for any proxy server settings which are configured in the Java control panel.I am not sure to what extent this is possible within the scope of commons-httpclient. This setting in the Java control panel may be relevant for applets running within the (IE) browser; implementing proxy autoconfiguration may be much more complicated. Regards, -- Sasha Dominic wrote:Hi all, I've discovered that json-rpc doesn't work too well behind an HTTP proxy. The issue seems to be related to how that Apache commons httpclient is instantiated. The HTTPSession class simply instantiates a HttpClient object and doesn't allow you to modify any of the parameters. Ideally, we need to do the following: proxyHost = System.getProperty("deployment.proxy.http.host"); proxyPort = System.getProperty("deployment.proxy.http.port"); httpclient = new Client.HttpClient(); hostConfig = httpclient .getHostConfiguration(); hostConfig.setProxy(proxyHost, proxyPort); Obviously one should check to see whether the "proxyHost" is set before updating the host config object. This solution won't deal with more complex situations (such as if an autoconfig parameter is set for the proxy in the Java control panel). Ideally the Apache httpclient authors should provide better support for any proxy server settings which are configured in the Java control panel. Regards, Dominic.

Sasha Ovsankin

unread,
Jan 11, 2008, 12:33:02 AM1/11/08
to json-rp...@googlegroups.com
OK, so basically the built-in client supports proxies better than commons-httpclient? This is cool.

Actually, the client does not need the dependency on the commons-httpclient unless you register it explicitly, so all is well :-)

Dominic

unread,
Jan 11, 2008, 1:00:19 AM1/11/08
to json-rpc-client
Hi Sasha,

On Jan 11, 4:33 pm, Sasha Ovsankin <sa...@codebistro.com> wrote:
> OK, so basically the built-in client supports proxies better than commons-httpclient? This is cool.

Yep, exactly.

> Actually, the client does not need the dependency on the commons-httpclient unless you register it explicitly, so all is well :-)

How does one do do this? My current code (sorry, in JavaScript) is:

Client = Packages.org.apache.commons.httpclient;
JsonRpc = Packages.org.jabsorb.client;

JsonRpc.HTTPSession.register(JsonRpc.TransportRegistry.i());
var httpSession = JsonRpc.TransportRegistry.i().createSession(hostRoot
+ "JSON-RPC");
setProxy(httpSession);

var state = new Client.HttpState();
httpSession.setState(state);
client = new JsonRpc.Client(httpSession);

var userManager = client.openProxy("userManager",
Uidl.example.UserManager);

So you are saying that I don't need to call httpSession.setState()?

I could only find examples which used the Apache HTTP client.

Regards,
Dominic.

Dominic

unread,
Jan 11, 2008, 1:14:35 AM1/11/08
to json-rpc-client
Ok, never mind, I worked it out (it is much simpler now):

var session = JsonRpc.TransportRegistry.i().createSession(hostRoot +
"JSON-RPC");
client = new JsonRpc.Client(session);
var userManager = client.openProxy("userManager",
Uidl.example.UserManager);

Getting rid of the Apache HTTP library also removed dependencies on
commons-codec and commons-logging, so this makes a huge difference to
the client which would normally need to download these JAR files.

Thanks!
Dominic,

Sasha Ovsankin

unread,
Jan 11, 2008, 1:19:19 AM1/11/08
to jabsorb-user, json-rp...@googlegroups.com
Great!

Please note though that this scheme wouldn't support cookie-based authorization (yet).


Getting rid of the Apache HTTP library also removed dependencies on
commons-codec and commons-logging, so this makes a huge difference to
the client which would normally need to download these JAR files.
Indeed, that was one of the reasons to use URLConnection in the first place.
Reply all
Reply to author
Forward
0 new messages