Mender Client HTTP Proxy Support

[tw]

I'm trying to configure Mender client to connect through an HTTP/S proxy but it seems to be ignoring the HTTP_PROXY/HTTPS_PROXY/http_proxy/https_proxy environment variables. Is there a different method for configuring a proxy server?

Thanks

[Mirza Krak]

On Tue, Jul 24, 2018 at 6:35 PM, tw <twheel...@gmail.com> wrote:

I'm trying to configure Mender client to connect through an HTTP/S proxy but it seems to be ignoring the HTTP_PROXY/HTTPS_PROXY/http_proxy/https_proxy environment variables. Is there a different method for configuring a proxy server?

Hi.

What specific errors are you getting?

I took a quick look at the code and it seems that there might be lacking support for proxy. Hopefully someone from the client team can confirm this.

Are you able to cross-compile the client? In that case I could suggest a change for you try out.

-- 

Mirza Krak | Embedded Solutions Architect | https://mender.io

 Northern.tech AS | @northerntechHQ

[Eystein Måløy Stenberg]

On 25/07/18 08:12, Mirza Krak wrote:
> On Tue, Jul 24, 2018 at 6:35 PM, tw <twheel...@gmail.com
> <mailto:twheel...@gmail.com>> wrote:
>
> I'm trying to configure Mender client to connect through an HTTP/S
> proxy but it seems to be ignoring the
> HTTP_PROXY/HTTPS_PROXY/http_proxy/https_proxy environment variables.
> Is there a different method for configuring a proxy server?
>
>
> Hi.
>
> What specific errors are you getting?
>
> I took a quick look at the code and it seems that there might be lacking
> support for proxy. Hopefully someone from the client team can confirm this.
>
> Are you able to cross-compile the client? In that case I could suggest a
> change for you try out.

Sorry to ask this, but why do you want to use a HTTP(s) proxy? Is this
for some kind of firewalling or network segmentation?

We did some research into proxies in the past and they have pretty
severe limitations when it comes to security and TLS because they expect
to see all application data.

When using a http proxy there is no transport security, so anyone on the
network could intercept and tell the devices to install some software.

htttps proxies work very differently, you would 1) either have to
terminate the TLS connection there (i.e. they will see all traffic and
need to make a new connection on behalf of every device to the Mender
server) or 2) use some kind of raw TCP redirect which I suspect defeats
the purpose you have for a proxy in the first place.

So the result of the research was that it was not worth it to support
this. Maybe the architecture can be set up differently (e.g. allow
Mender TLS connections to pass through or put a Mender server onsite).
Would be happy to hear your thoughts or if anyone else know proxies well
and how this could make sense!

>
> --
> Mirza Krak | Embedded Solutions Architect | https://mender.io
>

> Northern.tech AS <https://northern.tech> | @northerntechHQ
> <https://twitter.com/northerntechhq>
>
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Mender List mender.io" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to mender+un...@lists.mender.io
> <mailto:mender+un...@lists.mender.io>.
> To post to this group, send email to men...@lists.mender.io
> <mailto:men...@lists.mender.io>.
> Visit this group at
> https://groups.google.com/a/lists.mender.io/group/mender/.

--

Eystein

[tw]

Thanks to everyone the responses.

I'm not getting any errors, it is just failing to make use of the proxy server set in the environment (I am testing on a network where a proxy is available but not required). I do have the ability to cross-compile and would be interested to try your proposed change.

On Wednesday, July 25, 2018 at 5:12:46 AM UTC-4, Eystein Stenberg wrote:

Sorry to ask this, but why do you want to use a HTTP(s) proxy? Is this 
for some kind of firewalling or network segmentation?

We did some research into proxies in the past and they have pretty 
severe limitations when it comes to security and TLS because they expect 
to see all application data. 

When using a http proxy there is no transport security, so anyone on the 
network could intercept and tell the devices to install some software. 

htttps proxies work very differently, you would 1) either have to 
terminate the TLS connection there (i.e. they will see all traffic and 
need to make a new connection on behalf of every device to the Mender 
server) or 2) use some kind of raw TCP redirect which I suspect defeats 
the purpose you have for a proxy in the first place. 

So the result of the research was that it was not worth it to support 
this. Maybe the architecture can be set up differently (e.g. allow 
Mender TLS connections to pass through or put a Mender server onsite). 
Would be happy to hear your thoughts or if anyone else know proxies well 
and how this could make sense! 

We're trying to connect devices from inside a corporate network that we don't control. Thanks for the helpful explanation.

[tw]

I should add that we're on rocko running client 1.5.0.

[Mirza Krak]

On Wed, Jul 25, 2018 at 6:36 PM, tw <twheel...@gmail.com> wrote:

Thanks to everyone the responses.

On Wednesday, July 25, 2018 at 2:12:48 AM UTC-4, Mirza Krak wrote:

On Tue, Jul 24, 2018 at 6:35 PM, tw <twheel...@gmail.com> wrote:

I'm trying to configure Mender client to connect through an HTTP/S proxy but it seems to be ignoring the HTTP_PROXY/HTTPS_PROXY/http_proxy/https_proxy environment variables. Is there a different method for configuring a proxy server?

Hi.

What specific errors are you getting?

I took a quick look at the code and it seems that there might be lacking support for proxy. Hopefully someone from the client team can confirm this.

Are you able to cross-compile the client? In that case I could suggest a change for you try out.

-- 

Mirza Krak | Embedded Solutions Architect | https://mender.io

 Northern.tech AS | @northerntechHQ

I'm not getting any errors, it is just failing to make use of the proxy server set in the environment (I am testing on a network where a proxy is available but not required). I do have the ability to cross-compile and would be interested to try your proposed change.

What my idea was to make the following modifications:

Add "Proxy: ProxyFromEnvironment," to https://github.com/mendersoftware/mender/blob/95c74abdc628318ed646215438cf2793b9d34cba/client/client.go#L225-L227.

From golang docs [1]:

ProxyFromEnvironment returns the URL of the proxy to use for a given request, as indicated by the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions thereof). HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.

[1]. https://golang.org/pkg/net/http/#ProxyFromEnvironment

-- 

[tw]

I wanted to give an update in case anyone is interested. I tried the proposed patch, and came surprisingly close to getting everything working in my test environment. This involved a relatively locked down proxy that only allows regular traffic on ports 80 and 443, and CONNECT requests on port 443 (the goal being to mimic a restrictive corporate network), with basic authentication. Unsurprisingly, artifact download (port 9000) failed. I didn't test with a more permissive proxy configuration, but I suspect that if the proxy accommodated CONNECT requests on port 9000, it would have been a success. Perhaps this could be useful for others with a different use case in mind. All other functionality, so far as I can tell, worked fine. Thanks to everyone that commented.