SQUID + E2GUARDIAN (with KERBEROS AUTH) - Working with issues

[Fabricio Guzzy]

Hello everyone,

I am setting a FREEBSD environment with  SQUID and E2GUARDIAN. by the way, I already have WINBIND (for Kerberos auth) set and working fine with SQUID. Absolutelly no Issues.

I could make it work together putting E2GUARDIAN in front of SQUID, forwarding connections and using NTLM auth. 

It Works really fine. (in this case the browser points to to E2Guardian and E2guardian forward connections to SQUID) like this:  Browser --> E2guardian --> SQUID --> Internet

The point now is:  How to set my E2guardian to work with SQUID using my KERBEROS authentication?

Knowing E2guardian doesn't support KERBEROS at this time, I am trying the oposite, putting SQUID in front of E2guardian, making the authentication on SQUID, forwarding connections to E2Guardian.

Something Like This:  Browser --> SQUID with Kerberos --> E2GUARDIAN --> INTERNET

The configuration I have used looks like the one below:

####SENDING CONNECTIONS TO E2GUARDIAN AFTER AUTH ####

cache_peer 127.0.0.1 parent 8080 0 login=*:password

always_direct deny all

never_direct allow all

#########################################################

The configuration works, but it happens that SQUID (or the Authentication mechanism) is exhausting all E2Guardian Workers, no matter how much you set.

If you set 1000, it uses all. If you set 20000 it also uses all. In the end, the proxy stop working because there are no workers free to deal with connections.

If you use netstat command to check the amount of connections, you can see thousands of them from SQUID to E2guardian. The same doesn't happen when E2guardian is in front of Squid.

Any different configuration for SQUID (with Kerberos) to use with E2guardian? 

Thanks in Advance!!

Kind Regards,

Fabricio.

[FredB]

I guess you should take a look at ICAP mode in this case

[Philip Pearce]

I did a quick search for squid disable persistent connections.

You could try this http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Appendix+A.+Config+File+Reference/server_persistent_connections/

Regards

Philip

---

--
E2guardian:
https://groups.google.com/d/forum/e2guardian
Github:
https://github.com/e2guardian/e2guardian
Follow us on twitter:
https://twitter.com/e2guardian
---
You received this message because you are subscribed to the Google Groups "e2guardian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to e2guardian+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/e2guardian/e075f648-3a98-4555-851d-0d0875e99fd3o%40googlegroups.com.

[Fabricio Guzzy]

Hello Fred,

Thanks for the message.  I have never used ICAP before. Do we have any documentation for that?

Is the regular E2guardian (repository version) ready for it? Or, Do I have to recompile the source to enable something?

Thanks Much!

Fabricio.

[Fabricio Guzzy]

Hi Philip,

Thanks for your message.  Not sure but, I may give it a try.

Regards

Fabricio.

Philip

---

To unsubscribe from this group and stop receiving emails from it, send an email to e2gua...@googlegroups.com.

[FredB]

Yes here https://github.com/e2guardian/e2guardian/tree/v5.3/notes

With icap, kerberos will be only used through squid

--
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.

[Fabricio Guzzy]

Hello Fred,

Thanks for the Link.

I just added the config and so far it's working fine.  I will keep testing for a few hours, but I believe that did the trick.

I appreciate the help on this matter.

by the way, if the community decides to develop the Kerberos Auth (native on E2G), please count on me. 

I may not be a good programmer but I may help $$$ if required.

Thanks!

Fabricio.

[Fabricio Guzzy]

OK, after a few hours it seems it's very stable, so I guess the config is good. Thanks for the info.

Then I have another point:  I am Intercepting SSL.

I have tested the SSL interception in separate, Squid and e2guardian. 

Both are working fine all alone, the issue is when I use then together with the ICAP config.

When a https page is blocked I get the SSL error like:   ERR_TUNNEL_CONNECTION_FAILED

That doesn't happen when I use the old way (E2guardian in front of Squid)

I tried to enable the SSL interception only in E2guardian and only in Squid, also on both. No success.

I get always the same error. Not sure if the ICAP config allows me to use SSL Interception that way.

Are you guys aware of it?

Thanks!!

Regards

Fabricio.

[FredB]

E2guardian must be aware that ssl interception is used by squid

There is a couple of options in common story about.

[Fabricio Guzzy]

Hello Folks,

I finally found the issue.  It was a Squid directive (netdb-exchange) - After disabled that, it stopped to shoot thousands of connections agains the cache_peer (E2G) - I got some info from the Squid Forum. They helped to resolve the case.

Now I got another weird behavior.

See that I have Kerberos working at the Squid Side and I am using "FORWARDXFOR" option to send user´s  info.

I had to set E2G to use BASIC AUTH in order to consume the "ForwardedX" info from Squid - None of the other options worked, only the BASIC one.

Now E2G can get the username but not the IP address. It always shows 127.0.0.1

The option "useforwardedfor" is enabled at E2G side.

Is there anything in the BASIC Auth (or other Auth plugin) that needs to be set/changed?

THANKS MUCH!!

Fabricio.

[FredB]

I'm not at the office now, but as far I can remember there is a specific icap option on squid, for id, and for ip

If you are not to hurry, later I will take a look at one of my machine with a close configuration.

[Fabricio Guzzy]

Thanks Much Fred. No need to hurry. I can wait.

Thanks once again! I appreciate your help on this matter.

Regards

Fabricio.

[FredB]

Remove all your configurations about basic auth and xforwarder and try this


icap_enable on
icap_service service_req reqmod_precache bypass=on icap://127.0.0.1:1344/request
icap_service service_resp respmod_precache bypass=on icap://127.0.0.1:1344/response
adaptation_access service_req allow all
adaptation_access service_resp allow all
icap_send_client_ip on
icap_send_client_username on
adaptation_masterx_shared_names X-ICAP-E2G
icap_service_failure_limit 1 in 1 seconds
icap_connect_timeout 10 seconds
icap_io_timeout 15 seconds
icap_service_revival_delay 10

[Fabricio Guzzy]

I change the config, including some other cache_peer configs (Besides ICAP) - It´s partially working.

When I block an HTTP page I can see the real IP address of the end user side. If an HTTPS page I see 127.0.0.1 instead.

I could capture the communication using tcpdump and both have the same "XForwardedFor" information with the correct (real) IP address.

So I am wondering what is happening on the E2Guardian side to get the Localhost IP for the HTTPS sites only.

Any idea?

Thanks Much!

Fabricio.

[Fabricio Guzzy]

Additional info below:

I changed the config, including some other cache_peer configs (Besides ICAP) - It´s partially working.

When I block an HTTP page I can see the real IP address of the end user side. If an HTTPS page I see 127.0.0.1 instead. (it shows the Squid IP - I tested using a non-loopback IP as well)

[FredB]

icap_send_client_ip on
icap_send_client_username on

Is not related with xforwardfor (it's an another header X-Client-IP)

Do you have the right IP in squid's access.log ?

grep 127.0.0.1 /valog/squid/access.log ?

[Fabricio Guzzy]

Hello Fred,

Using ICAP, I can't see the Block-Page for HTTPS pages, so that way I can't see if it´s showing the right IP or not. All other non-blocked pages are fine, with or without MITM enabled.

For the Block-Page, It breaks the SSL tunnel and I got:  "ERR_TUNNEL_CONNECTION_FAILED" Message. 

It's important to say that I have MITM (SSL Interception) enabled at the E2G side Only.  (Not using MITM on SQUID - Just the Authentication Helpers)

For that reason I decided to use CACHE_PEER directives instead, then I can see the Block-Page normally, but when it blocks an HTTPS site, it shows the Squid IP (loopback in this case) instead of the CLIENT IP.

For HTTP pages, it works Fine.

See grep command:

 grep 127.0.0.1 /var/squid/logs/access.log

1596208595.417    733 192.168.0.20 TCP_TUNNEL/200 2983 CONNECT cdn.onenote.net:443 us...@DOMAIN.CORP FIRSTUP_PARENT/127.0.0.1 -

1596208595.427    646 192.168.0.20 TCP_TUNNEL/200 2983 CONNECT cdn.onenote.net:443 us...@DOMAIN.CORP FIRSTUP_PARENT/127.0.0.1 -

1596208595.646    331 192.168.0.20 TCP_TUNNEL/200 1764 CONNECT v20.events.data.microsoft.com:443 us...@DOMAIN.CORP FIRSTUP_PARENT/127.0.0.1 -

192.168.0.20 is the CLIENT IP address - So it´s correct.

Also, if we check the tcpdump info, we can see the FORWARDX info is showing the CLIENT IP for both cases, HTTP and HTTPS, so there is nothing wrong with the connection between the Client and Squid.

I am trying to understand why E2G is getting such IP for HTTPS only. 

I also have noticed that, when using Cache_Peer , E2G doesn't get USER info from the FORWARDX Header, but from the AUTH Plugin.

If you disable the auth plugin, the User info disappear from the Block Page.

See images below and you can see what I am talking about:

   

[Philip Pearce]

Hi Fabrico,

I've checked and there is a bug with the useforwardfor which means that the x-forwardedfor is not carried into the mitm session (which in the case of squid then e2g with MITM on e2g,  squid cannot see the header and so cannot add x-forwarded-for).   I will fix.

Philip

---

From: "Fabricio Guzzy" <fabric...@gmail.com>
To: "e2guardian" <e2gua...@googlegroups.com>

--
E2guardian:
https://groups.google.com/d/forum/e2guardian
Github:
https://github.com/e2guardian/e2guardian
Follow us on twitter:
https://twitter.com/e2guardian
---
You received this message because you are subscribed to the Google Groups "e2guardian" group.

To unsubscribe from this group and stop receiving emails from it, send an email to e2guardian+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/e2guardian/e25e9066-fb0e-4b22-a44d-113abd6260f8o%40googlegroups.com.

[Fabricio Guzzy]

Hi Philip,

Thanks for the heads up. That was driving me crazy cause I thought it was something wrong with my setup. 

I really appreciate the feedback... and please count on me for testing or anything else.

Will you set the fix over the "5.3 DEV" branch on Github? Here --> https://github.com/e2guardian/e2guardian/tree/v5.3.dev  ?

Let me know and I will recompile the code for FreeBSD.

Kind Regards.

Fabricio.

[FredB]

For better understanding:
https://encrypted-tbn0.gstatic.com/images?q=tbn%3AANd9GcTkT0AkKioccRojiOj305xY07RC_XTnrJYGGg&usqp=CAU

The http flow is only used through squid, icap is only used to control
http object (yes/no)

In this case e2 is no more a "proxy" so configurations for kerberos,
sslmitm, etc are for squid only, E2 is totally unaware about

I'm using this method with success with a high load

Like this, no cache_peer, no forward, just the icap config I posted before

[Philip Pearce]

See https://github.com/e2guardian/e2guardian/issues/619.

Would you check and let me know if OK?

Thanks

Philip

---

To view this discussion on the web, visit https://groups.google.com/d/msgid/e2guardian/a99a3a28-c6bb-45e8-883c-b63d84f52506n%40googlegroups.com.

[Fabricio Guzzy]

Hi Philip,

I have just recompiled the code with the changes over the file:  src/ConnectionHandler.cpp  on branch "v5.3.dev"

but still showing the same. No changes at all.

If you want me to try something else, just let me know.

Thanks  a Lot!

Fabricio.

  

[Fabricio Guzzy]

It´s important to remember that we have to enable BASIC Authentication in order to capture the USER from parent SQUID.

Without any AUTH enabled (eg. none) , E2G is not capturing the USER as well.

[Philip Pearce]

Hi Fabrico,

I have made a few more changes and pushed to v5.3.dev. Would you check if this fixes the issue?

Thanks

Philip

---

To view this discussion on the web, visit https://groups.google.com/d/msgid/e2guardian/b1ba26c4-2e30-4a19-a725-e4e5b002528an%40googlegroups.com.


[image/jpeg:https.JPG]


[image/jpeg:http.JPG]

[Philip Pearce]

Also,  make sure squid is not caching responses from e2g.

---

[Philip Pearce]

Hi Fabricio,

This is an interesting approach (putting squid first to auth).  If you get it working could you share your squid config?

The basic auth plugin is designed to used in front of squid, but I can see how it can work in your reverse approach.

If you get it working, I will add a new auth plugin, specifically tuned for this purpose to v5.4.

Regards

Philip

---

To view this discussion on the web, visit https://groups.google.com/d/msgid/e2guardian/8507b61b-a2bf-4783-bdd5-53896e5bdc6an%40googlegroups.com.

[Fabricio Guzzy]

Hi Philip

Good afternoon and Thanks for the help. I really appreciate it.

I will recompile the source this afternoon/evening and let you know for sure.

See that I am using SQUID in front of E2G just because of the KERBEROS authentication.

Otherwise, I can put E2G in front of SQUID, (but using NTLM Auth only)

Mean while, this is the sort of SQUID Auth configuration I am using today:

##################SQUID AUTH CONFIG########################

#### NEGOTIATE KERBEROS AUTHENTICATION ####

auth_param negotiate program /usr/local/libexec/squid/negotiate_kerberos_auth -i -s HTTP/proxy45.d...@DOMAIN.CORP

auth_param negotiate children 100 startup=40 idle=10

auth_param negotiate keep_alive off

#### NTLM AUTHENTICATION ONLY - IN CASE KERBEROS FAILS ####

auth_param ntlm program /usr/local/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp

auth_param ntlm children 100 startup=40 idle=10

auth_param ntlm keep_alive off

#### STARTING CUSTOM ACLs ####

acl auth proxy_auth REQUIRED

#### STARTING CUSTOM AUTHORIZATION ACL ###

http_access allow auth

####SENDING CONNECTIONS TO E2GUARDIAN AFTER AUTH ####

cache_peer 127.0.0.1 parent 8080 0 login=*:password no-digest no-netdb-exchange 

always_direct deny all

never_direct allow 

#################END OF SQUID AUTH CONFIG#########################

Let me recompile the code and I will send you the results.

Thanks Once again!

Kind Regards

Fabricio.

[Fabricio Guzzy]

Hello Philip;

I just recompiled the code and... IT WORKED!! (see evidences below).

The error page is now showing the correct IP from the Client and not the Loopback anymore in both cases (HTTP and HTTPS).

I have tested using both authentications, Kerberos and NTLM. Both are fine.

 

I will perform some more tests but I really believe it fixed the issue.

Also, let me know if you want me to perform any specific test using this setup (SQUID in Front of E2Guardian)

Thanks Much! I really appreciate your help on this. ... and count on me!

Kind regards

Fabricio.

[Fabricio Guzzy]

Hi Philip.

Good afternoon

You may consider this issue as "Closed-Resolved"  - I have exhausted all options and everything works really fine.

Thanks

Fabricio.

[Jesús Miguel Iriarte Oñoz]

Greetings brothers, I was testing the auth and partly managed to pass through icap to the E2G the original IP, but not quite right; It happens to me that it only shows me in some connections the IP_REAL of the client, other times it only comes out 127.0.0.1 or IP-LOCAL. Likewise, I do not get anything on the blocking website, it continues to leave without a user and with the LOCAL-IP (not the real one of the client)

see attached photo...

[Fabricio Guzzy]

Hello jesus

Did you enable "XForwardedFor" ?  That is required!

Fabricio.

[Jesús Miguel Iriarte Oñoz]

Thank you very much Fabricio and everyone. Yes it is a fact; I had to enable it for it to work for me, both in the e2g and in the squid (which had it removed). Now I wonder this http headers (xforward) is not a somewhat complex issue from a security point of view, how could I mitigate / control this a bit? I tell you then in the e2g doc they indicate that the headers can poison you ... Greetings.