bypassing squid transparent proxy

[jp munroe]

I have a CentOS 7 self built VM with diladele 4.8 installed, running as a dedicated transparent proxy. I also have an Ubuntu server with 4.8 running in 'normal proxy' mode, this is for clients such as Windows machines which support entering proxy servers with authentication for most applications.

For the 'normal proxy' I have to bypass squid authentication and diladele completely for some trusted destinations and I have done this by creating/editing 'whitelist.conf' and ensuring this is called in 'squid.conf'. This has been necessary after I have excluded in 'web safety' and also in HTTPS inspection and these methods have not resolved my problems.

I have now encountered for the first time an issue which I believe is the same or similar for the transparent proxy. On Android clients, the game 'super mario run' does not connect to the login servers (I assume hosted by Nintendo, a packet capture on my firewall seems to confirm this) When the clients are changed to use 3G or a route which bypasses the proxy entirely on WIFI, it works, so it is proxy related. 

Despite seeing no traffic blocks in the monitoring log when the issue occurs on the Android devices (I see 'pass'), I have added .nintendo.com to web safety and HTTPS inspections exclude lists, but it doesn't fix the problem. 

(Basically mirroring what I did on the Ubuntu proxy). On the CentOS proxy I changed the template file:

/opt/qlproxy/var/console/squid/templates/squid/access_controls.conf

To include:

acl GoodSites dstdomain "/etc/squid/whitelist.conf"

http_access allow GoodSites

(and in whitelist.conf)

.nintendo.com

This is below the lines:

http_access allow localhost manager

http_access deny manager

I am not sure this is actually achieving anything though, because in transparent mode there is no auth enabled. All my issues with the other proxy were because the apps calling the destination didn't support proxy auth or it didn't interact properly.

So it still doesn't work and I don't know what to check next, the GUI logs don't seem detailed enough to indicate where the problem may be.

[S Irlapati]

I could be wrong, but I don't think domain names exclusions work with transparent proxy. You may try adding the IP address or subnet for nintendo.com. I am sure Rafael will have a better answer for you.

I was wondering though is there is a text file that could be edited to add ip subnet exclusions instead of using the GUI. I have about 200 ip subnets to add and using the gui will be difficult.

[rafael....@diladele.com]

Hello Jp, Samuel,


The "transparent intercept" proxy as you have (centos) works like this:

- browser needs to connect to httpS://nintendo.com
- browser makes DNS request to resolve nintendo.com into IP address; DNS server responds with something like 54.187.103.157
- browser makes a HTTPS request to 54.187.103.157 (not knowing there is a proxy!)
- centos network stack catches this request and directs it into Squid
- Squid creates a notification to the Web Safety ICAP server with the message - someone tries to connect to 54.187.103.157, allow?
- Web Safety tells "allow"
- IF YOU HAVE HTTPS FILTERING ENABLED, Squid creates a new connection to 54.187.103.157, de-crypts it and waits for the remote server to respond
- remote server responds, Squid forwards it to web safety and after "allow" verdict, repacks the response with its own certificate and sends data to browser

Implications:

- the .nintendo.com exclusion that you configured will NOT work as this name is NOT present in the transparent proxy stream *at all* (only IP address is present)
- as Samuel recommended - you need to exclude the whole subnet of nintendo.com from filtering
- now where to exclude:
    * in Squid / ICAP / Exclude / IP Subnet - in order NOT to forward traffic to that subnet from Squid to Web Safety; because it does not need to be fliltered
    * in Squid / HTTPS / Exclude / IP Subnet - in order NOT to perform decryption of HTTPS; because most probably a game running on device knows what certificate to expect from nintendo.com
    * ideally on CentOS iptables level - in order to NOT get the traffic to that subnet into Squid at all.

Hope I was able to explain :(

Raf

P.S. In explicit proxy scenario browser does not resolve the .nintendo.com but tells its name to proxy - that is why exclusion works nicely.

[rafael....@diladele.com]

@Samuel - in Squid / Tools / Import - it is possible to add exlusions from file into Web UI...

[jp munroe]

Rafael,

Thanks for the suggestions. On further investigation the app seems to connect to Amazon AWS, Nintendo must host their services on there. I don't want to whitelist/bypass all of Amazon AWS IP's addresses as I will be allowing countless other traffic through unfiltered. I am not sure how I can go about this just for a single app on an Android device...