Respect HTTP_PROXY, HTTPS_PROXY and NO_PROXY env vars

[Václav Muzikář]

Hello,

I'm investigating the possibility of implementing support for HTTP_PROXY, HTTPS_PROXY and NO_PROXY env vars. The main motivation is KEYCLOAK-19080 but since the variables are not limited only to OpenShift or cloud environment in general, I'm considering implementing it directly in the ConnectionsHttpClient SPI. Currently it is possible to configure proxy mappings only through standalone.xml. My thoughts were something like this:

if (proxy mappings not specified in standalone.xml) {

   if (HTTPS_PROXY != null) {

      route all traffic through HTTPS_PROXY

   }

   else if (HTTP_PROXY != null) {

      route all traffic through HTTP_PROXY

   }

   if (NO_PROXY != null) {

      define exceptions that won't use proxy

   }

}

else {

   route traffic as configured in standalone.xml

}

The only risk is that it is not backward compatible. Let's say that somebody doesn't use a proxy (doesn't have it configured in standalone.xml) but has HTTP_PROXY env var set on his system. Then Keycloak would unexpectedly start using a proxy. On the other hand, if empty proxy mapping would be specified in standalone.xml, Keycloak wouldn't effectively use any proxy (regardless the env vars). This behaviour would be of course documented.

WDYT?

Thanks!

--

Václav Muzikář
Senior Software Engineer
Keycloak / Red Hat Single Sign-On
Red Hat Czech s.r.o.

[Václav Muzikář]

Just for a reference, there's a nice summary of the proxy env vars: https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/

[Thomas Darimont]

Hi Václav,

The support for proxy configuration via wildfly / jboss configuration allows a bit more than just configuring a http_proxy / no_proxy. 

The proxy configuration can be set per target host, e.g. use a proxy / no_proxy for some destinations but not others or use a different proxy for certain destinations.

If you want to provide an alternative configuration via env variables, wouldn't it make sense to support this destination host based proxy selection mechanism as well?

As a fallback we could of course consider the http(s)_proxy / no_proxy env variables.

WDYT?

Cheers,

Thomas

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAMQSZyhGbrD%2BJF5O%2BXohz-yQbZz485ZgM69%3DsU_4PPK%2BwXBAHA%40mail.gmail.com.

[Václav Muzikář]

Thanks for your feedback!

That's a good point but the env vars are not meant as a replacement or true alternative for this fine grained config. And IMHO we don't need to introduce a completely new env vars, we could just leverage what already exists in OCP (and even Linux systems in general). The http_proxy / no_proxy env vars is a bit something else and are used for all hosts in a given app. At least that seems to be the case for all apps that use it. The env vars would not be specific to Keycloak. We would have to mimic the behaviour of already existing implementations (even though there's probably no official specs).

[Stian Thorgersen]

What if we support using the standard proxy environment variables within proxy mappings, and add a new environment variable for the mappings? Something like:

<spi name="connectionsHttpClient">
    <provider name="default" enabled="true">
        <properties>
            <property
            name="proxy-mappings"
            value="${env.KEYCLOAK_PROXY_MAPPINGS}"/>
        </properties>
    </provider>
</spi>

Then folks can easily define it with just:

KEYCLOAK_PROXY_MAPPINGS=[.*;${env.HTTPS_PROXY}]

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAMQSZyhxt7LiaUvunuFwgrnFbxgF3DC68R0MgzsQuJT_tpQbFQ%40mail.gmail.com.

[Václav Muzikář]

That seems a little bit overcomplicated, TBH. Creating one env var just to use some other env var. What about introducing some switch in the config instead to enable reading HTTP(S)_PROXY and NO_PROXY env vars, if we don't want to read them by default?

[Stian Thorgersen]

I would probably argue that most will want to use the proxy for only some requests, and not all. That being said I imagine that a lot of the complexity you're referring to is with regards to standalone.xml and continers/operator, rather than strictly the idea of two environment variables. Once we move fully to Quarkus this stuff will be much simper to define and configure.

[Václav Muzikář]

I think there are several point of views for this.

1) Enable configuring proxy through environmental variables to avoid cumbersome standalone.xml.

This will be implicitly resolved in Quarkus dist, so maybe we don't need to think about some explicit and temporary workarounds just for WF/EAP dist (if we implement 2)).

2) Adding support for env vars (HTTP(S)_PROXY and NO_PROXY) that are already respected by many *nix tools.

IMHO it might make sense to implement this regardless of 1).

[Václav Muzikář]

I just wanted to get back to discussing this. So, which approach do we want to take? :) I'd vote for 2 – properly support established env vars (implement the same behaviour as other tools that use it), just maybe not by default.

[Sebastian Łaskawiec]

+1 for the env vars. In OpenShift this is a standard mechanism for establishing proxy. It's being used all over the place: https://github.com/openshift/openshift-docs/search?q=HTTP_PROXY

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAMQSZyiT3rEoK1gUNFpRyf9UivOOamDTamcHimq3tZG%2BZZgEUQ%40mail.gmail.com.

--

Sebastian Łaskawiec

[Thomas Darimont]

I'm fine with the env var approach as well, but keep in mind that this only solves a part of the story. However this might be sufficient for most customers.

Customers who need a more fine grained proxy control can still use the custom provider configuration.

If that's okay, then go for it.