Scala I/O and Proxy

[Daniel Sobral]

It seems Scala I/O respects the proxy configuration, but not the proxy
authentication configuration. I have the following configured on my
SBT:

java \
-Dhttp.auth.preference=ntlm -Dhttp.auth.ntlm.domain=mydomain \
-Dhttp.proxyUser=my.user -Dhttp.proxyPassword=mypassword \
-Dhttps.proxyUser=my.user -Dhttps.proxyPassword=mypassword \
-Dhttp.proxyHost=proxy.domain -Dhttps.proxyHost=proxy.domain
-Dhttp.proxyPort=3128 -Dhttps.proxyPort=3128 \
-XX:+CMSClassUnloadingEnabled -XX:+UseParallelGC -Xss1m
-XX:MaxPermSize=512m -Xmx1536M \
-jar `dirname $0`/sbt-launch.jar "$@"

When using I/O (following the Daily Scala articles), all accesses go
to the proxy, but without authentication.

--
Daniel C. Sobral

I travel to the future all the time.

[Jesse Eichar]

That is very surprising because I am not doing anything special with the urls.  If you look at the code for Resource.fromURL I think it is simply url.openStream.  I see one case where it opens the connection to get the content length with 

val conn = url.openConnection

conn.open

conn.getContentLength

conn.close

(with some more safety of course)

I have not researched the issue but should that not automatically use the authentication?

Perhaps you could give me the code snippet you used?

Jesse

[Daniel Sobral]

Every Resource.fromURL I try here goes like this. If the proxy lines
are present on SBT conf, then it goes to the proxy but does not
authenticate (and I get a 407). If they are not present, then it goes
directly and everything is fine. Here, check the gist:
https://gist.github.com/3353398

I know that proxy authentication in the Java side of things is a bitch
-- from personal experience (see
https://github.com/dcsobral/Databinder-Dispatch/commit/826c80298067d5a4f93e76dc20d58154cf088ff7).

More comments below.

On Tue, Aug 14, 2012 at 3:13 AM, Jesse Eichar <jesse....@gmail.com> wrote:
> That is very surprising because I am not doing anything special with the
> urls. If you look at the code for Resource.fromURL I think it is simply
> url.openStream. I see one case where it opens the connection to get the
> content length with
>
> val conn = url.openConnection
> conn.open
> conn.getContentLength
> conn.close
>
> (with some more safety of course)
>
> I have not researched the issue but should that not automatically use the
> authentication?

I don't know. The way Dispatch does things requires one to create a
security realm for the authentication. I haven't looked much into it
because our _servers_ do not go through proxy, so this thing only
bites me on my developer tooling (sbt and g8r, the latter being the
reason I patched Dispatch).

[Jesse Eichar]

So this topic is a bit beyond what I had originally considered to be the basic 1.0 scala-io API.  I have done a bit of reading now and I can see that the topic is not simple to create a generic API for it.  

http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-auth.html

Is one of the main pages I was looking and as you said an Authentication strategy is needed.

Although according to the document NTLM is a bit special (although it does still need an authentication strategy).  Here is a quote:

NTLM

NTLM is a scheme defined by Microsoft. It is more secure scheme than Basic, but less secure than Digest. NTLM can be used with proxies or servers, but not with both at the same time. If a proxy is being used, then it cannot be used for server authentication. This is because the protocol actually authenticates the TCP connection rather than the individual HTTP interactions.

On Microsoft Windows platforms, NTLM authentication attempts to acquire the user credentials from the system without prompting the user's authenticator object. If these credentials are not accepted by the server then the user's authenticator will be called.

Because the Authenticator class was defined prior to NTLM being supported, it was not possible to add support in the API for the NTLM domain field. There are three options for specifying the domain:

1. Do not specify it. In some environments, the domain is not actually required and the application need not specify it.
2. The domain name can be encoded within the username by prefixing the domain name followed by a back-slash '\' before the username. With this method, existing applications that use the Authenticator class do not need to be modified, so long as users are made aware that this notation must be used.
3. If a domain name is not specified as in method 2) and the system property "http.auth.ntlm.domain" is defined, then the value of this property will be used as the domain name.

 

I was curious if you have suggestions on how I should handle this.  The easy thing for me is obviously to say that you have to define your authentication before calling Scala-IO.  Something like:

https://gist.github.com/3356853

(I haven't tested the code but I think the idea makes sense)

Obviously this is a bit of a cop-out.  But the basic Java libraries are kind of limited.  I could add a optional parameter that allows configuration of the connection but again it seems kind of lame.  I am leaning towards having 3rd party libraries do this and ideally implement the Input and Output interfaces. 

Maybe I can find time to create a fork of Dispatch with Scala-IO interfaces

Jesse

[Daniel Sobral]

I see... I think you should cop out at this point. Authenticated
proxies are not all that common (or so I assume, given how broken the
tooling for that is all around). Just document the fact that proxies
are used automatically if configured through java settings, but
authentication is up to the user.

[Jesse Eichar]

Done.  I have added some docs to this effect.  I have also changed the fromURL method to take a method for configuring the connection so that fairly common actions like settings headers etc... can be easily done.

Thanks for the feedback.

Jesse