Transparent e2guardian with sslmitm

[David Nikoghosyan]

Hi,

I am David from LibreRouter project. (Please check https://github.com/Librerouter/Librekernel)

In our project we are in need to check http/https traffic content by calmav and some filtering tool like e2guardian. Our platform is Debian 8.

We already have squid 3.4.13 running will ssl-bump; and am using c-icap from within squid for AV scanning (with clamav) and simple banned site url scanning with squidguardian. This part of configuration works perfectly for both http and https traffic.

Now its time for e2guardian implementation, and I faced some problems.

For http.

For http we have this scenarion

Client → iptables → e2guardian → squid → internet

e2guardian(in transparent mode) in front of squid works fine. There is no problem with http.

For https

I know that e2guardian does not support transparency for https, So at first I tried to set proxy explicity in browser, but anyway it didn't work.

I have been trying unsuccessfully to get e2guardian running with sslmitm.

Installation

I downloaded e2guardian from https://github.com/androda/ecapguardian, then builded it.

Added followinf in dgconfig.h file before build.

#define __SSLMITM 

#define __SSLCERT 

Configuration

I followed the information at e2guardian/notes/ssl_mitm to set up ssl_mitm.

Specifically:

openssl genrsa 4096 > private_root.pem

openssl req -new -x509 -days 3650 -key private_root.pem -out my_rootCA.crt

openssl x509 -in my_rootCA.crt -outform DER -out my_rootCA.der

openssl genrsa 4096 > private_cert.pem

For testing, i have made the permissions wide open on these files:

-rwxrwxrwx 1 root root 1919 Sep  8 17:28 my_rootCA.crt

-rwxrwxrwx 1 root root 1377 Sep  8 17:28 my_rootCA.der

-rwxrwxrwx 1 root root 3247 Sep  8 17:28 private_cert.pem

-rwxrwxrwx 1 root root 3243 Sep  8 17:28 private_root.pem

 

Similarly, the premssions for 'generatedcertpath' is wide open:

drwxrwxrwx 2 root root  1024 Sep  8 17:28 generatedcerts

I setup e2guardian.conf with:

cacertificatepath = '/etc/e2guardian/ssl_cert/my_rootCA.crt

caprivatekeypath = '/etc/e2guardian/ssl_cert/private_root.pem

certprivatekeypath = '/etc/e2guardian/ssl_cert/private_cert.pem

generatedcertpath = '/etc/e2guardian/generatedcerts'

And in e2guardianf1.conf with:

ssllegacylogic = off

sslmitm = on

It appears the e2guardian is just passing the https connection through to squid without decrypting/re-encrypting, despite "sslmitm = on" being set.

When I open https pages I got just 1 or 2 lines logged in my /var/log/e2guardian/access.log

2016.9.8 20:10:13 - 10.0.0.101 https://shavar.services.mozilla.com:443 CONNECT 0 0 1 200 - - -

2016.9.8 20:10:13 - 10.0.0.101 https://shavar.services.mozilla.com:443 CONNECT 0 0 1 200 - - -

I have been trying to figure this out for several days without success.  Am I missing something obvious?

Some other question.

1. If we managed to run e2guardian with sslmitm (in explicit mode), can I run e2guardian with 2 squids to get transparency ?

client → iptables → squid(ssl_bumping?) → e2guardian → squid(ssl_bumping?) - Internet

2. When will we have new release of e2guardian with enabled transparency mode for sslmitm ?

3. What kind of help/support e2guardian needs for point 2 ?

Thanks in advance.

David

[Philip Pearce]

David,

See comments below:-

Regards

Philip

---

From: "David Nikoghosyan" <nikd...@gmail.com>
To: "e2guardian" <e2gua...@googlegroups.com>
Sent: Friday, 9 September, 2016 10:35:48 PM
Subject: Transparent e2guardian with sslmitm

Hi,

I am David from LibreRouter project. (Please check https://github.com/Librerouter/Librekernel)

In our project we are in need to check http/https traffic content by calmav and some filtering tool like e2guardian. Our platform is Debian 8.

We already have squid 3.4.13 running will ssl-bump; and am using c-icap from within squid for AV scanning (with clamav) and simple banned site url scanning with squidguardian. This part of configuration works perfectly for both http and https traffic.

Now its time for e2guardian implementation, and I faced some problems.

For http.

For http we have this scenarion

Client → iptables → e2guardian → squid → internet

e2guardian(in transparent mode) in front of squid works fine. There is no problem with http.

For https

I know that e2guardian does not support transparency for https, So at first I tried to set proxy explicity in browser, but anyway it didn't work.

I have been trying unsuccessfully to get e2guardian running with sslmitm.

Installation

I downloaded e2guardian from https://github.com/androda/ecapguardian, then builded it.

.....

It appears the e2guardian is just passing the https connection through to squid without decrypting/re-encrypting, despite "sslmitm = on" being set.

When I open https pages I got just 1 or 2 lines logged in my /var/log/e2guardian/access.log

2016.9.8 20:10:13 - 10.0.0.101 https://shavar.services.mozilla.com:443 CONNECT 0 0 1 200 - - -

2016.9.8 20:10:13 - 10.0.0.101 https://shavar.services.mozilla.com:443 CONNECT 0 0 1 200 - - -

I have been trying to figure this out for several days without success.  Am I missing something obvious?

Yes,  you have downloaded ecapguardian, a very cut-down fork of e2guardian, specifically designed for use with ecap, and with many e2guardian features removed including https mitm.

Try e2guardian from the official project site https://github.com/e2guardian/e2guardian.

1. If we managed to run e2guardian with sslmitm (in explicit mode), can I run e2guardian with 2 squids to get transparency ?

client → iptables → squid(ssl_bumping?) → e2guardian → squid(ssl_bumping?) - Internet

This is really a squid question but the short answer is no.    I'm not a squid expert, but even if you can configure to use e2guardian as the upstream proxy in the 'bumped' stream this will make http requests to e2g instead of https and so the second squid will get an unencrypted http request and send the request to the target server unencrypted.

This is, sort of, what the ecapguardian author is hoping to achieve by using ecap, but I have not tested his fork and have not heard whether or not the fork works.

2. When will we have new release of e2guardian with enabled transparency mode for sslmitm ?

Sometime next year.

3. What kind of help/support e2guardian needs for point 2 ?

I have done some preliminary work on v4, and once I am able to get this in to an overall shape, I will post this work to github and invite others to contribute to the code.

Once you have e2guardian working with https interception you may like to try the e2guardian clamd module so as to avoid bumping again in squid. 

[ppc7...@gmail.com]

Hello,

I am the maintainer of ecapguardian.

The purposes of ecapguardian are primarily to stop using ecapguardian as a proxy, and effectively convert it into a filtering-only service that can be accessed via the ECAP interface.

Many good proxy servers already exist, so from my perspective it makes little sense to me to continue using e2guardian as proxy when the maintainers of the various other proxies already do well (see Squid).

You say that ecapguardian has removed HTTPS MITM, and that is true from a purely code perspective. I no longer use the paths which allow for HTTPS MITM _using e2guardian_. This is the whole point of using ECAP. Squid (or other compatible proxy) performs all the HTTPS MITM magic, and then forwards simply the request and response contents on to ecapguardian via whatever ecap adapter is specified in the configuration. Getting Squid to do SSL MITM is not hard.

Using Squid3, I have a prototype transparent SSL MITM filtering system that works. It runs pretty slow on the dev boards I have (Raspberry Pi 2B, Odroid boards, etc), but it works.

Not everyone will agree with my push to make e2guardian into a simple filter service, but that's why forks exist. :)

[Spike]

Hi there,

thank you for working on ecapguardian, personally, as an e2g user with limited experience in proxy/cf land, this seems the right path forward and the "industry standard" for that matters (well, sorta, the use ICAP instead of ecap). And they definitely do SSL MITM no problem. The closest to an open source project is diladele, https://www.diladele.com/ , which basically uses squid + their homegrown icap server as far as I understand and does SSL.

Ecap seems to be the "chosen path" by squid devs and to remove some overhead according to their tests. My only concern about that over icap is that if you were to switch away from Squid you may not necessarily find another ecap ready solution while icap seems more established.

What concerned me most from the github convo, assuming you are Androda (you didn't mention in your email), is that no info about the client is passed along which would make it impossible to create filtering groups. Is that the case? Did you find another way to manage different filtering settings for different user groups?

thank you for your work,

Spike

--
You received this message because you are subscribed to the Google Groups "e2guardian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to e2guardian+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ppc7...@gmail.com]

You're right that no client information is passed along to ecapguardian as configured by default.

You can configure Squid to send along the client's IP address in the ECAP calls, and I'd love to add MAC address pass-along (looking at the Squid code it seems to be partially implemented). Nothing in the ECAP code path is implemented to catch that IP pass-along.

Honestly, I'm not at all familiar with how e2guardian manages filtering groups or how group users are identified. Mostly I've been focusing on getting ECAP working with what little time I have to contribute.

And yes, I am the user Androda. I'm going to push a commit to ecapguardian's 'MITM-Defense' branch in a few minutes which fixes a typo and adds a header position map to the HTTPHeader.cpp/hpp file.

The MITM-Defense branch is my first stab at blocking anti-MITM tools like HPKP. Need to rewrite HPKP response headers or your MITM will fail.

[Spike]

great, ok.

ip would be a great start, but then I think a lot of ppl are using either basic auth or NTLM. If those can't be passed along it'd be quite a drawback to not say a show-stopper.

the other thing that concerns me is the rewrite in c++11 for v4, which looks a lot cleaner and nicer. It seems your code is based on the older e2g/dansguardian, which makes totally sense, but then it's not really a strong path forward from that perspective (altho I'm no C++ expert, this comment is just based on reading around, I might be completely wrong).

In any case, I think there's merit in all the different directions, from my perspective it's just a matter to figure out if and hwo we can benefit from each other's efforts and with that benefit the larger community.

take care,

Spike