Block Page for HTTPS

[Brenden Smerbeck]

Hey,


So far I've been loving NxFilter and have had a lot of success. My only gripe at the moment is with regards to the block page (or lack thereof) when navigating to a website using HTTPS.

Now, I'm away of the reason why the error is occuring; being that the redirection causes an interrupt in the certificate chain and the browser reports it as an error. Rather than seeing the block screen, you see a warning screen.

After looking around the forum, I've seen several people mention hosting the block pages on a local web server and then performing some action to get them to show. Does anyone have documentation on how to achieve this? I'd love for users who navigate to a blocked site with HTTPS to see an identical block screen as browsing with HTTP. I'd happily work with anyone on getting this set up.


My OS is FreeBSD 11.0 and NxFilter runs alongside pfSense; an open source firewall solution. Feel free to message me through whatever means. 

Thanks for the help!

[Jinhee]

Did you try NxForward? It is a Chrome extension and it shows block page on HTTPS for Chrome. It only works on Chrome but better than nothing. And as far as I know, to install local certificate is not just about server side. You need to do something on client side as well. If you need to work on your client PCs then it's not that expensive to install NxForwad on every PC. Still limited only to Chrome though.

[Jinhee]

I will keep this post on the top for a while. I hope you get some response from people. But our user base is not that big yet, even though you love NxFilter enough to invest your own time to improve it. Maybe someday we can solve this one.

[Brenden Smerbeck]

Thanks for the pin!


In addition to our own efforts, I want to get you into contact with the developers of the operating system I'm using. The community forums are fairly active (no less than 300 active users at any time of the day) and, from what I'm been able to create with our hardware, NxFilter is an absolute perfect match. The developers often make "packages" that integrate third party applications into the system (i.e. Squid3, Snort, Zabbix/Nagios). They're great and, like with any open source project, the more (skilled) hands, the better.

Here's the forum and some of the administrators (often developers) on the board include jwt, stephenw10, stephenw10, and jdillard. 


The biggest hurdle with this "problem" is that it's not a problem at all. It's only by nature that there's an error when HTTPS traffic is redirected. So, any solution will have to work with the protocol.

Even OpenDNS has taken the logical way out and instructed users to install the Cisco Root CA on their browser: https://support.opendns.com/hc/en-us/articles/227987007.


Since NxFilter can run on a local server/firewall like I have it set up, I think we might have some freedom and leeway that OpenDNS doesn't. I'll keep working on this of course and post any updates in this thread. If anyone wants to work on the same system as me / needs help getting NxFilter installed on a pfSense unit just message me and I can assist you one-on-one 

[Brenden Smerbeck]

So, I think I might be onto something. I was looking at this: http://www.moserware.com/2009/06/first-few-milliseconds-of-https.html to see what packets are sent during a normal web request.  DNS resolution happens before any request is made. So, the filtering rule’s action is determined before the TCP/IP handshake (including TLS). My idea was:

Let's assume we're trying to go to  https://www.facebook.com 

1. DNS Request Made for facebook.com
2. Resolution Occurs, Filtering Determined (block socialnet, block facebook.com)  <--- WE KNOW WHAT WE WANT TO DO (block/allow)

3. TCP/IP begins.

From here, we have 2 options. The first was my first idea and is super complicated, the second is better. So feel free to skip to "OR"

4. If HTTP request detected, allow though normally

4a. If HTTPS request detected (port 443), force a HTTP request first (80).

4b. If redirect to block page detected, do not make HTTPS request

4c. If redirect to block page not detected, make normal HTTPS request and allow traffic

OR

4. Block was detected during resolution, so route traffic through port 80 and perform HTTP redirect to block page
5. Block wasn’t detected during resolution, allow HTTPS traffic through 440 as usual

The idea is: Because NxFilter I've installed on a UTM device that contains a firewall, router, and more...it can route the traffic how we want based on certain conditions. If the block condition is met, we can (most likely) just transmit the traffic through whatever port we want. If we can do that, traffic to a blocked domain originally travelling through 440 will instead be routed through 80 and the block screen will happen without a hitch. All because of the fact that before TLS handshaking begins we know how we want to filter the traffic.


Let me know what you guys think, but I'm confident this might be the solution.

[Jinhee]

Thanks for all the suggestions.

Firstly, about OpenDNS solution. I don't know how they remove warning with their own certificate. Do they do something on CA root side? Something like to trust everything from OpenDNS? I don't know if it's even possible but if they need to install their certificate on every PC, I don't like it. If you use AD it might be OK though.

About your own solutions. one problem is how to redirect user HTTPS request to HTTP. You said it's possible as you are on a UTM. Then someone needs to build an UTM including NxFilter. I know there are many people using NxFilter on pfSense and expecting someone to work on some kind of UTM integrating NxFilter to pfSense but so far no one.

Even if there's someone wanting to work on integrating NxFilter into pfSense, now the problem is how do we instruct pfSense to redirect the request from the user which is on HTTPS and to the exactly the website blocked to be on HTTP? If we block or redirect it based on target IP then what if there are other sites on the same IP by virtual hosting? Though it will be a rare case.

Actually, we do that kind of thing with NxForward. We redirect HTTPS request to be on HTTP when it gets blocked. Thing is that it's only for Chrome at the moment.

pfSense is something interesting to me. But at the moment I am busy with many other things. We need to go through a major change for future. After that I might work on several things for helping people wanting to integrate NxFilter into pfSense or install NxFilter on pfSense.

[Brenden Smerbeck]

Just an update that I'm working with some developers to see if there's any possibility.

HSTS poses a large problem, and this may be impossible given the current nature of the clients and their interaction. This enforces a HTTPS connection and cannot be altered (without client interaction). This means that any solution to the problem will have to involve working with HTTPS and the TLS protocol. Unfortunately, this is where we have the most issues to solve.


The problem is largely TLS and the nature of redirecting traffic. To even transmit data in a HTTPS connection, we have to first perform the TLS handshake. The TLS handshake involves verifying the identity of the server and then exchanging keys and certificate information. The only way certificate validation will succeed without an error is if it is signed by a Trusted Root CA. Getting a certificate like this is easy and free (startssl, for instance). The problem therein lies that the common name of the certificate, in most cases, must match the hostname of the website (a public domain verified by a hosting provider like GoDaddy, etc). 

Then, and let me note I don't know much about this yet, there's client verification of the server to which it's trying to connect. This could involve the URL in the address bar, the SNI extension of the TLS packet (which would be a hostname, used in SSL peek and splice exploits), and anything available from the server including the common name of the certificate. This leaves us several scenarios:

1. Verification using URL hostname in browser address bar and certificate Common Name:
Redirection is impossible without client interaction. There is no way to dynamically change the common name of a certificate.

2. Redirection using a hostname derived from the DNS resolution process:
Possible if some form of spoofing could be used, point now at the block page we've hosted.

3. Redirection using SNI
Possible if there was a MITM exploit used with the TLS connection. The TLS SNI extension could be changed and made to match the public block page (ie. block.mydomain.com).


Regardless, with a HTTPS request, we have to have a valid TLS authentication before any data is transmitted; including HTTP_REDIRECT, etc. The only way to do this is to have a valid cert signed by a trusted CA. Then, we'd have to see if there was any weak point in the HTTPS transmission protocol that doesn't interfere with HSTS.


Will keep working and keep you updated

[jeroen...@gmail.com]

Perhaps Jinhee is interested in providing a valid ssl certificate on NXfilter.com for paying customers that can set the redirect to that ip?

Needs to be a fast line on fast hosted site to be able to serve that blocked page.

The blocked page can be simple and can probably not tell why something is blocked, just blocked.

I see no other way eiher with serving https blocked pages without having a valid ssl cert.

Or you can setup  a windows server 2012 with IIS and IIS configured on https with "require server name indication"

You will not get a cert warning anymore, just a page not found..

Seeing your posts above it seems you know your way around in apache.

Perhaps you know how to setup "require server name indication" to bogus.com in apache? I can not find it.

It does not solve your problem, but it does no longer give the annoying ssl warning or timeout.

[feeli...@yahoo.fr]

Hi Jeroen, 

i'm interessing about your solution, can you tell me more, if it's possible submit a step by step solution , how you fixed it with an IIS windows server

[Jinhee]

Another way is to go with NxForward. Only for Chrome but we see 1200 users for that. Means it works well.

  https://chrome.google.com/webstore/detail/nxforward/ohhmhnionmgplhblinhpijfbaelmaojd

And Jeroen explained his solution on the other thread,

  https://groups.google.com/forum/#!searchin/nxfilter200/server$20name$20indication%7Csort:relevance/nxfilter200/S_WCEtNgVDY/jeIAZDwTBAAJ

You can find some documents about 'server name indication'.

[jeroen...@gmail.com]

Hi,

Basicly it's pretty simple.

Take a windows 2012 server and install the basics for running a small webserver (install iis)

First setup a http server:

Disable the default build in page and create a new site.

Copy the website data from NXfilter to your server and start the website by using this directory on port 80

Connect to http://yourwindowsserverip and you should see:

This page is blocked: reason unknown.

Now just setup a https server on 443 in windows:

Create a self signed cerificate. (see google for easy instructions to setup https on windows IIS) and in the connector settings enable server name indication on "bogus.com" or whatever you want.

restart the website.

connect to: https://yourwindowsserverip

You should see an errorpage: acces denied immediately.

Have this working.?

In NX filter Gui change the block page setting to your windows server ip.

Try a few pages that should be blocked according to your policy (clear the cache on the client first)

On any blocked http page:

"this page is blocked, reason unkwown"

(with deverting the blocked pages to another ip, nxfilter cannot send the reason anymore, do not know how to fix this, but personaly I do not care)

On any blocked https page:

error, access denied in a millisecond.

[jeroen...@gmail.com]

Just noticed the initial request:

Update: On a https blocked page using server name indication.

: You do not see a real webpage "blocked"

: You do not see acces denied, but page not found. (instead of the slower certificate error)

To explain: without a real valid ssl certificate including root cert. your clients will always give ssl error on https blocked pages in current build of NXfilter and some websites have a weird timeout (youtube)

All I discribed here regarding server name indication on a windows server is to create a workaround for those timeouts and give the clients a much faster resonse by NOT serving the clients the requested https blocked  page.

It is not a solution to show a https "this page is blocked for this reason" to the end user.

[jeroen...@gmail.com]

See: https://groups.google.com/d/msg/nxfilter200/S_WCEtNgVDY/OuxQyDRSBgAJ

So my workaround has just become obsolete :)

[rbl...@wdf.school]

Hi,
I’m trying to help a small school with their Nxfilter setup. They are reporting lots of ssl errors but I think it’s on blocked sites and has just confused them. I saw your NxForward chrome extension and have installed this but it does not seem to be working properly. I can see the icon on chrome but it is grey (not sure if that means anything) and the ssl messages still appear on the block page. Is there something else I should have configured for this extension?
Thanks

[Jahastech]

What's your NxFilter version? It should be newer than v4.0.5. If it it doesn't work even if you use the newest version what do you see on Chrome extension 'background page'? You can see its logging on,

  Chrome > More tools > Extensions > NxForward > backgroud page

You need to enable 'Developer mode' on Chrome Extensions menu.

[rbl...@wdf.school]

Hi,
Thanks for replying so quickly!
We are on the latest version (updated today).
Please see screenshot of the console screen on Chrome attached.

[Jahastech]

It seems like that it forwarded 'www.gambling.com' to your NxFilter block page as it's on SSL. Did it not work for you? Can you get me several example domains? Probably it will be working on my side but I will test them anyway to see if I can find anything.

[rbl...@wdf.school]

So in the school we have got the paid block list and we have the gambling category set to block for everyone. I typed things like gambling and poker and bingo and such into google and any of the results that are https I click on. With further testing a couple seem to work ok and the block page appears as it should but most seem to hang for a few seconds and the get the ssl error page.

[Jahastech]

Maybe it's a problem of NxForward itself. Just can't find it in our testing environment. Anyway I will try your gambling search result. And if you have a problem with that 'hang' how about this kind of approach?

  http://nxfilter.org/tutorial.html#hide-ssl

[Jahastech]

I tried several sites,

  http://192.168.0.103/block,chrome.jsp?domain=www.fulltilt.com

  http://192.168.0.103/block,chrome.jsp?domain=www.replaypoker.com

  http://192.168.0.103/block,chrome.jsp?domain=www.pokerstars.com

As you see it above, when it gets forwarded you will see your NxFilter IP on the address bar. And you don't get these. Do you get any redirection related logging on the background page when you get that SSL error? Or any communication between NxFilter and NxForward about those domains?

[Jahastech]

Do you use clustering? Maybe your slave node can't find the blocked log. In that case, I will patch it.

[Jahastech]

On my clustering it works fine. But there's a slight possibility of not getting a correct answer from your slave node for NxForward. I will patch it but look into SSL warning hide if you have many users in your network.

[Jahastech]

I think I know what caused the problem. In your case, you opened a new tab from Google search result. Probably your Google search page redirected to NxFilter block page leaving your new tab showing SSL error. NxForward is a new kind of solution by us but we don't have many users for it and nobody reported anything on this so far. If you don't tell me we can't solve your problem. Anyway we will try to solve the problem.

[Jahastech]

NxForward will be updated to v1.5 today. In my testing, mostly working. But I only can say 'mostly'. Thing is some Chrome API doesn't work the way we expected some time. However, in the first place it's against the normal behavior of a browser. We can accept some exceptions if it's small. Test it and see if it's working for you.

[rbl...@wdf.school]

Hi, 
So sorry for the late reply, I think we are in different timezones, it was about 12pm when i was messaging yesterday!

We don't use clustering, just a single server.

I will do some more testing tonight and get some more logging information.  We love the software and so are keen to help with any testing and logs you need :)

[Jahastech]

Nothing urgent. You can tell me on Monday.

Yes. We are in a different timezone. We are in Korea at the moment.

[rbl...@wdf.school]

Hi, 

We are now updated to 1.5 but still having the same problem. Sometimes when you go on a site the first time it does not work and you get the https error page and the second time it works, but not always. See log below - please let me know if there is anything i can help with for testing.

INFO [2017/11/29 20:45:08] Init..

background.js:1 INFO [2017/11/29 20:45:09] get_block_ip, g_block_ip = 10.41.16.12

background.js:1 INFO [2017/11/29 20:45:12] chrome.tabs.onUpdated, https://chrome.google.com/webstore/detail/nxforward/ohhmhnionmgplhblinhpijfbaelmaojd

background.js:1 DEBUG [2017/11/29 20:45:12] is_blocked_recently, Checking for chrome.google.com

background.js:1 DEBUG [2017/11/29 20:45:12] nx_lookup for chrome.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:45:12] nx_lookup, Adding cache-allow for chrome.google.com

background.js:1 INFO [2017/11/29 20:45:26] chrome.webRequest.onBeforeRequest????????????, https://www.google.co.uk/search?q=porn&oq=porn&aqs=chrome..69i57j0l5.925j0j4&sourceid=chrome&ie=UTF-8

background.js:1 INFO [2017/11/29 20:45:26] chrome.webRequest.onBeforeRequest, https://www.google.co.uk/search?q=porn&oq=porn&aqs=chrome..69i57j0l5.925j0j4&sourceid=chrome&ie=UTF-8

background.js:1 DEBUG [2017/11/29 20:45:26] is_blocked_recently, Checking for www.google.co.uk

background.js:1 DEBUG [2017/11/29 20:45:26] nx_lookup for www.google.co.uk, /ALLOW

background.js:1 DEBUG [2017/11/29 20:45:26] nx_lookup, Adding cache-allow for www.google.co.uk

background.js:1 INFO [2017/11/29 20:45:26] chrome.tabs.onUpdated, https://www.google.co.uk/search?q=porn&oq=porn&aqs=chrome..69i57j0l5.925j0j4&sourceid=chrome&ie=UTF-8

background.js:1 DEBUG [2017/11/29 20:45:26] is_blocked_recently, Checking for www.google.co.uk

background.js:1 DEBUG [2017/11/29 20:45:26] nx_lookup for www.google.co.uk, /ALLOW

background.js:1 DEBUG [2017/11/29 20:45:26] nx_lookup, Adding cache-allow for www.google.co.uk

background.js:1 INFO [2017/11/29 20:45:40] chrome.tabs.onUpdated, http://porn.com/

background.js:1 DEBUG [2017/11/29 20:45:40] is_blocked_recently, Checking for porn.com

background.js:1 DEBUG [2017/11/29 20:45:40] nx_lookup for porn.com, /BLOCK

background.js:1 DEBUG [2017/11/29 20:45:40] nx_lookup, Adding cache-block for porn.com

background.js:1 INFO [2017/11/29 20:45:54] chrome.webRequest.onBeforeRequest????????????, https://www.porn.com/

background.js:1 INFO [2017/11/29 20:45:54] chrome.webRequest.onBeforeRequest, https://www.porn.com/

background.js:1 DEBUG [2017/11/29 20:45:54] is_blocked_recently, Checking for www.porn.com

background.js:1 DEBUG [2017/11/29 20:45:54] nx_lookup for www.porn.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:45:54] nx_lookup, Adding cache-allow for www.porn.com

background.js:1 INFO [2017/11/29 20:45:58] chrome.webRequest.onBeforeRequest????????????, https://www.google.co.uk/search?q=porn&oq=porn&aqs=chrome..69i57j0l5.925j0j4&sourceid=chrome&ie=UTF-8

background.js:1 INFO [2017/11/29 20:45:58] chrome.webRequest.onBeforeRequest, https://www.google.co.uk/search?q=porn&oq=porn&aqs=chrome..69i57j0l5.925j0j4&sourceid=chrome&ie=UTF-8

background.js:1 DEBUG [2017/11/29 20:45:58] is_blocked_recently, Checking for www.google.co.uk

background.js:1 DEBUG [2017/11/29 20:45:58] nx_lookup for www.google.co.uk, /ALLOW

background.js:1 DEBUG [2017/11/29 20:45:58] nx_lookup, Adding cache-allow for www.google.co.uk

background.js:1 INFO [2017/11/29 20:45:58] chrome.tabs.onUpdated, https://www.google.co.uk/search?q=porn&oq=porn&aqs=chrome..69i57j0l5.925j0j4&sourceid=chrome&ie=UTF-8

background.js:1 DEBUG [2017/11/29 20:45:58] is_blocked_recently, Checking for www.google.co.uk

background.js:1 DEBUG [2017/11/29 20:45:58] nx_lookup for www.google.co.uk, /ALLOW

background.js:1 DEBUG [2017/11/29 20:45:58] nx_lookup, Adding cache-allow for www.google.co.uk

background.js:1 INFO [2017/11/29 20:46:02] chrome.webRequest.onBeforeRequest????????????, https://www.google.co.uk/search?safe=strict&ei=RRwfWr3KLsmFgAagtY-4BA&q=bingo&oq=bingo&gs_l=psy-ab.3..0i131i67k1j0i67k1l4j0i131k1j0i67k1j0j0i131k1j0.3041.3484.0.3641.5.5.0.0.0.0.121.405.4j1.5.0....0...1c.1.64.psy-ab..0.5.390...0i46i67k1j46i67k1.0.3BtVSId9deU

background.js:1 INFO [2017/11/29 20:46:02] chrome.webRequest.onBeforeRequest, https://www.google.co.uk/search?safe=strict&ei=RRwfWr3KLsmFgAagtY-4BA&q=bingo&oq=bingo&gs_l=psy-ab.3..0i131i67k1j0i67k1l4j0i131k1j0i67k1j0j0i131k1j0.3041.3484.0.3641.5.5.0.0.0.0.121.405.4j1.5.0....0...1c.1.64.psy-ab..0.5.390...0i46i67k1j46i67k1.0.3BtVSId9deU

background.js:1 DEBUG [2017/11/29 20:46:02] is_blocked_recently, Checking for www.google.co.uk

background.js:1 DEBUG [2017/11/29 20:46:02] nx_lookup for www.google.co.uk, /ALLOW

background.js:1 DEBUG [2017/11/29 20:46:02] nx_lookup, Adding cache-allow for www.google.co.uk

background.js:1 INFO [2017/11/29 20:46:02] chrome.tabs.onUpdated, https://www.google.co.uk/search?safe=strict&ei=RRwfWr3KLsmFgAagtY-4BA&q=bingo&oq=bingo&gs_l=psy-ab.3..0i131i67k1j0i67k1l4j0i131k1j0i67k1j0j0i131k1j0.3041.3484.0.3641.5.5.0.0.0.0.121.405.4j1.5.0....0...1c.1.64.psy-ab..0.5.390...0i46i67k1j46i67k1.0.3BtVSId9deU

background.js:1 DEBUG [2017/11/29 20:46:02] is_blocked_recently, Checking for www.google.co.uk

background.js:1 DEBUG [2017/11/29 20:46:02] nx_lookup for www.google.co.uk, /ALLOW

background.js:1 DEBUG [2017/11/29 20:46:02] nx_lookup, Adding cache-allow for www.google.co.uk

background.js:1 INFO [2017/11/29 20:46:07] chrome.webRequest.onBeforeRequest????????????, https://www.crownbingo.com/

background.js:1 INFO [2017/11/29 20:46:07] chrome.webRequest.onBeforeRequest, https://www.crownbingo.com/

background.js:1 DEBUG [2017/11/29 20:46:07] is_blocked_recently, Checking for www.crownbingo.com

background.js:1 DEBUG [2017/11/29 20:46:07] nx_lookup for www.crownbingo.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:46:07] nx_lookup, Adding cache-allow for www.crownbingo.com

background.js:1 INFO [2017/11/29 20:46:08] get_block_ip, g_block_ip = 10.41.16.12

background.js:1 INFO [2017/11/29 20:47:08] get_block_ip, g_block_ip = 10.41.16.12

background.js:1 INFO [2017/11/29 20:48:08] get_block_ip, g_block_ip = 10.41.16.12

background.js:1 INFO [2017/11/29 20:48:37] chrome.webRequest.onBeforeRequest????????????, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/nxfilter200/_256PXA6o0o/ntajQPV7AAAJ

background.js:1 INFO [2017/11/29 20:48:37] chrome.webRequest.onBeforeRequest, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/nxfilter200/_256PXA6o0o/ntajQPV7AAAJ

background.js:1 DEBUG [2017/11/29 20:48:37] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:48:37] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:48:37] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:48:38] chrome.tabs.onUpdated, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/nxfilter200/_256PXA6o0o/ntajQPV7AAAJ

background.js:1 DEBUG [2017/11/29 20:48:38] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:48:38] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:48:38] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:48:53] chrome.tabs.onUpdated, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/nxfilter200

background.js:1 DEBUG [2017/11/29 20:48:53] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:48:53] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:48:53] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:49:08] get_block_ip, g_block_ip = 10.41.16.12

background.js:1 INFO [2017/11/29 20:49:09] chrome.tabs.onUpdated, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!aboutgroup/nxfilter200

background.js:1 DEBUG [2017/11/29 20:49:09] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:49:09] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:49:09] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:49:35] chrome.tabs.onUpdated, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/nxfilter200

background.js:1 DEBUG [2017/11/29 20:49:35] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:49:35] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:49:35] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:49:45] chrome.tabs.onUpdated, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/nxfilter200/_256PXA6o0o

background.js:1 DEBUG [2017/11/29 20:49:45] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:49:45] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:49:45] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:50:07] chrome.tabs.onUpdated, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/nxfilter200

background.js:1 DEBUG [2017/11/29 20:50:07] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:50:07] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:50:07] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:50:08] get_block_ip, g_block_ip = 10.41.16.12

background.js:1 INFO [2017/11/29 20:50:09] chrome.tabs.onUpdated, https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/nxfilter200/_256PXA6o0o

background.js:1 DEBUG [2017/11/29 20:50:09] is_blocked_recently, Checking for groups.google.com

background.js:1 DEBUG [2017/11/29 20:50:09] nx_lookup for groups.google.com, /ALLOW

background.js:1 DEBUG [2017/11/29 20:50:09] nx_lookup, Adding cache-allow for groups.google.com

background.js:1 INFO [2017/11/29 20:51:08] get_block_ip, g_block_ip = 10.41.16.12

background.js:1 INFO [2017/11/29 20:52:08] get_block_ip, g_block_ip = 10.41.16.12

[Jahastech]

On your log, 'porn.com' is blocked but 'www.porn.com' is not blocked. How do you block 'porn.com' on NxFilter side? Did you use whitelist or custom categories? In that case you have to use '*.'.

[Jahastech]

How about other domains? Do you have the same problem with your Ad domain search result?

[rbl...@wdf.school]

Actually both URLS said they were blocked, though the www. was typed in as https://www.porn.com .  We pay for the category block lists and its blocked by that, we have not specifically set the site to be blocked,

[Jahastech]

On NxFilter side, you get blocked log for both of them but not on NxForward side? Can you recreate it constantly? I will test the same thing with my setup anyway. One possibility is a latency between your NxFilter and NxForward. You use them locally though?

[rbl...@wdf.school]

Hi, 

I just tested with the alcohol/tobacco and so typed alcohol into google, this site came up in the list but it has the same problem:  https://www.drinkaware.co.uk/alcohol-facts/alcoholic-drinks-units/what-is-an-alcohol-unit/

[rbl...@wdf.school]

Both pages were blocked and showed the block page, but one (the https://www.) it showed the HTTPS error page and when I click proceed it shows the block page.

[rbl...@wdf.school]

[Jahastech]

OK. I will test them and see if it's related to 'www'.

[rbl...@wdf.school]

I just tried this website, which appears in the google results, and unfortunately the same problem and this one didn't have www.   - https://dontbottleitup.org.uk/alcohol-test

[rbl...@wdf.school]

Not sure if its helpful or if you have already seen this, but at my main school we use a different filtering system, and their way to get round https errors is that the system allows you to export an inception SSL certificate that we deploy to all pcs on the network via Group Policy. - detailed here https://help.smoothwall.net/Hearst/Content/modules/guardian3/cgi-bin/guardian/httpssettings.htm

[Jahastech]

We don't know that we can do the same thing as Smoothwall as it's a webproxy based one. I guess it has Squid inside it. We will review it again in near future though.

About your domains, dontbottleitup.org.uk is not classified in 'Alcohol'. And drinkaware.co.uk is classified into 'Health/Medical' on my system. Did did you add them into 'Alcohol'?

[Jahastech]

And can you show me all the logs about those domains?

[Jahastech]

I recreated it. I will try to fix it. See you tomorrow. :)

[Jahastech]

But I recreated it only once. Though I guess it might be from caching. We have a cache on NxForward to reduce the amount of queries to NxFilter. We changed the way of caching a bit. Now it's v1.6. See if it's working.

[rbl...@wdf.school]

We used the paid version of Cloudlist, perhaps the classifications are different on there?

[Jahastech]

I was using Jahaslist. I will switch it to Cloudlist then. Was it not working yet?

[rbl...@wdf.school]

Not tested just yet, will try shortly

[rbl...@wdf.school]

Ok, so a bit of an update.

If type bingo into google, and then click on one of the https results that come up, the first time I go to the site I get the https error message, but if I tell it to proceed to the site I then get the nxfilter block page.  Now if I close Chrome and repeat the process again this time it then shows the block page right away.  

So it seems the first time you go to a site it fails, the next time it seems ok.

[Jahastech]

I think there's a real problem with this one. There might have been something changed on Chrome side since we developed NxForward. And the reason for why I get a different result is that I use 9443 for https port of NxFilter. Chrome acts differently. And on https error we don't get event anymore on Chrome API. I will see if there's another method to solve this one.

[Jahastech]

Just updated it to v1.7. It will be mostly working. At least it was on my system. I tested with my system using 443 port for HTTPS this time. However, one problem of NxForward is that we tried to reduce the request number on NxFilter. So the protocol is kinda asynchronous. NxFilter needs to populate its block log before NxFoward making queries against it. But it seems like Chrome sometimes fires up its event before making DNS request. So even if I say "mostly" it might be different for you. The best solution is to change the protocol between NxFilter and NxForward. We will review on how to proceed with this and probably work on it on v4.1.9 of NxFilter.

[Suporte SAC - Kernel TI]

Are you using the resource of proxy by Google? Or some tools of proxy on Chrome?