Anyone have any idea in how to block Youtube Ads

[Ashley Primo]

Hello,

I was just wondering if anyone would happen to have a list of IP's Youtube Video ads use, I am trying to achieve a block on all youtube ads including video ones.

If anyone could help, it would be much apricated.

Kind Regards, 

Ashley Primo

[jeroen...@gmail.com]

Hi Ashley,

First make sure you you use shallalist. This is nicely documented on the nxfilter man pages. Change policy to block ads and trackers.

Then Take a look at privoxy.

It (still) blocks all the ads in youtube  (and a lot more!) if you combine it with nxfilter.

You can install privoxy on the same machine as NX filter.

Default install of privoxy should do fine, only need to make sure privoxy listens for outside clients: privoxy=>config.txt=> listen addres=>change it to "your local ipv4 address:8118" instead of localhost or 127.0.0.1.

Set the proxy options for your browser (yourserver:8118) on the client (in the internet browser or system wide, depends on the OS) (and test)

See the privoxy documentation how to test if privoxy is used (http://config.privoxy.org on your browser should give a message that privoxy is used)

Optional:

If you run a descent DHCP server you could send out the proxy settings automatically with a wpad option and/ or create an alias in NXfilter called WPAD pointing to your server that servers the wpad files.

If you use dhcp/dns for wpad deploy you need to have a webserver running @port 80. NX filter already does that job. Just put the wpad files in the root of the web folder in nxfilter.

Search google for wpad deployment to see how to create the 2 needed files and deply ith through DHCP and/or DNS

Note: most routers serving dhcp have no option to create wpad options. You need a "real" dhcp server to do that.

NX filter is fine to serve wpad. Make sure nxfilter webserver serves the wpad files before you enable wpad broadcast.

(http://yournxfilterserver/wpad.dat and http://yournxfilterserver/wpad.pac) should present you your wpad contents or ask you what to do with the file (safe or open). Then you know wpad deploy works.

The files wpad.dat and wpad.pac should have the same contents. Just create wpad.pac and copy the contents to wpad.pac.

See: http://findproxyforurl.com/deploying-wpad/

Why is wpad broadcast the best option:

Sometimes a page does not want to play with privoxy.

As there is no easy way to whitelist a site in privoxy, use wpad.pac to exclude that difficult nasty site:

if (shExpMatch(host, "*.example.com"))
{
return "DIRECT";
}

With this example.com will not go through privoxy

Final step:

Make sure to block the Ads and trackers in NX filter. (if you use shallalist => default policy=> block)

That will stop all the ads in youtube on all your clients including ipads etc.

For ipads/phonestablets/android you need to enable proxy in the wifi settings. Either manual yournxfilterserver:8118 or automatically (set to auto on device) by using WPAD mentioned above.

Another nice feature for ipads: the proxy setting is system wide: no more/way less in-app ads.

Optional 2

If you get all above running:

Combining privoxy with adblock pro rules will give you an even almost total ad-free browsing experience.

There is plenty scripts available for importing adblock-pro rules in privoxy.

But for that you need some knowledge editing cron jobs or windows batch programming.

Optional 3

Replace your isp dns forwarder with opendns servers in the nxfilter config.

With opendns you can block even more. Opendns is free and "cloud based" and stops a lot of bad sites.

See google how to get opendns working

Best regards,

Jeroen

[jeroen...@gmail.com]

mmm...

As youtube is from google it was probably not such a good idea to post this solution in google groups:)

Won't surprise me if within a few weeks I get all the ads back in youtube.

[jeroen...@gmail.com]

Wow, almost 40 views and no feedback.

Nice to see writing a solution is appreciated in this forum.

[Andy Morris]

Ashley,

If you just want to block the ads, then I would suggest installing adblock plus and adblock as a browser addon and not have the webfilter do the work.  Put it on the end users control

https://adblockplus.org/

https://getadblock.com/

Andy

[jeroen...@gmail.com]

Yes,

That is an option, but I think the main goal for using NXFilter is to centralize the filter. And this can be done like I described above.

Not seen any ads in any device on youtube for years as I import the adbock rules in privoxy

Probably a reason why Ashley has posted this in this forum, but looks like she does not read her own posts anyway...

[Jinhee]

How do you use Privoxy with NxFilter when you have authentication with NxFilter? Privoxy is a webfilter on HTTP level

so if you put Privoxy before NxFilter you lose authentication as NxFilter can't differentiate users.

How many users do you have with Privoxy? I even saw people having trouble with Squid for just several hundreds users.

Some might say that they can handle more users with Squid but what if you just can use VM with 4G of memory instead

of a real machine? And your users making a massive traffic on HTTP/HTTPS? The reason I am talking about Squid is that

at least it does caching. And Privoxy does not do that. But actually with a small number of users a non-caching proxy might be

better. We see the network speed is faster than disk access speed these days.

Lastly, Ashley might want to build a cloud service over Internet. In that case sending/receiving HTTP traffic over Internet

is a very bad idea. But I see some companies do that and taking money for their service and end up with having an endless

comlaining from their users.

If it's that much effective we can consider of having it on NxClient which is our local HTTP proxy without any performance loss

or without losing authentication. But we don't think it's neccessary at the moment.

[jeroen...@gmail.com]

Hi Jinhee,

-My privoxy setup is not used as transparent (in traffic chain) but can be set as optional in either the users IE or wifi settings on their devices.

Using wpad deployment to make it more easier (just set proxy to auto and users have all settings)

DNS lookup through NXFilter can be seen completely separate from privoxy (for now)

-Squid, tried that, but as already indicated by yourself: not needed with my current internet speed (500Mbit)

In fact on my windows server using squid actualy slows things down...

Windows and squid work, but in that setup it is not easy to configure squid as a filter and squid becomes obsolete as NXFilter is much easier.

Squid is build for unix

-Only a few users (10) on my home network as the NXFilter and privoxy is in testing phase.

Need to do a lot of configuration to find the right balance: stop most ads, but keep speed and all sited loading.

I am still missing a global whitelist in privoxy, so using wpad files to exclude sites not to go through privoxy's filters.

-NXFilter as a proxy: I could try to chain privoxy with NXFilter (or NXFilter=Privoxy , but do not know where to put the forwarding settings in NXFilter proxy config.) to do a test for you (in non transparent mode only) and see what happens.

Have not tried using NXFilter as a proxy. Can do a test, but need to know the port number

What is the port number for the proxy service in NXFilter?

-No clue what you mean with the cloud base solution for Ashley.

-Also using NX filter without agents.

[Jinhee]

Are you saying that you use Privoxy to block only Google ads using WPAD file. I think it's possible. You let all the traffic bypssing your Privoxy except the traffic to Google ads and you block it. In that case it's possible you have authentication with NxFilter. But if you send all the traffic to your Privoxy or whatever the web proxy, your web proxy will do the DNS query on behalf of your browser. NxFilter does authentication based on IP address or IP session. So everybody becomes one user now. That's what I was talking about.

NxFilter is not a proxy. It's a DNS server with filtering ability. But NxClient is a proxy. It's working on client PC so there's no performance degrading and authentication is still possible. It's like everybody having one local proxy for them. So we can see the user IP addresses. And that's why we can do everything a web proxy does if we want to. At least theoritically.

Ashley wanted to build a cloud filtering service based on NxCloud. You know OpenDNS had a huge success. It's becasue they worked on DNS level not HTTP proxy level. If you send all these HTTP/HTTPS traffic to somwhere on the Internet that'd be very slow. So DNS filter got a chance here.

[jeroen...@gmail.com]

No, I am blocking all ads (as much as possible) with privoxy

To have the clients get the proxy settings automatically  I am serving wpad files. Proxy settings are in the wpad.pac file including exceptions that bypass privoxy.

Wpad files can also be served to portable devices, making that a system wide proxy setting for those devices.

For windows, the wpad files are only used in browser settings. Without special software you cannot set a system wide proxy for windows.

You are making this more complicated than it actually is.

Just running NXFilter as DNS server (with shallalist. to filter ads and trackers etc.)

Have privoxy running with adblock pro filters, so all clients have adblock pro filters.. Privoxy does not do DNS lookups itself, the clients do DNS lookup, and then depending if a proxy setting is defined or enabled, traffic runs it through privoxy or not.

I used open DNS, but find NXFilter way better and faster. Also OpenDNS has some issues with windows updates en cdn portals (content delivery from google/youtube) and opendns is missing the most important: ad and tracker blocking. For cdn's I always got the US ones with opendns , but I live in Europe. WIth NXFilter I now get the correct cdn's and all is much faster.

So Base level (read: DNS) filtering is done on NXFilter, Privoxy perform the more intelligent filtering. This kind of filtering cannot yet be done on DNS level as far as I know.

DNS lookup by proxy: squid does that, perhaps that is why you assume "Everyone becomes one user" Privoxy does not do that as long as it is not in transparent mode.

[Jinhee]

I tried Privoxy on my Ubuntu server. I use Firefox network setup for redirecting traffic from my Firefox. It does DNS queries and I don't get filtered by NxFilter. If your clients do DNS queries by themselves then there are 2 DNS queries. One by your client PC using NxFilter and then by your Privoxy for resolving the webserver domains in HTTP requests. Don't know if it can work that way.

If a web proxy does DNS queries on behalf of its clients using a local proxy would be the only option to use a web proxy and DNS filter at the same time. And in that way we don't lose any performance and authentication. But it's not the most important purpose of using NxClient. NxClient is for remote user filtering. It does single sign-on as well. And does application control. Many possibilities actually.

It's good to know that OpenDNS being no good in EU. I guess it would have the same problem in Asia and Africa.

[jeroen...@gmail.com]

Clear,

You are probably right about the 2 DNS queries.

1 by the client and then by the proxy. I can verify 100% this is working flawlessly. All filters are applied on the clients and also privoxy filtering works.

Here is why I never noticed: my server running privoxy (and NXFilter) is not using NXFilter as DNS

(If I use NXFilter as DNS server on the NXFilter server itself , the shallalist updates fail as I need to stop NXFilter first to import, as there is no DNS I cannot update...see my point:))

Off course, adding a few lines in the host file would solve this also, but I prefer to keep all dynamic.

Anyway, I think there is still authentication on the clients even though the browser is using a proxy. The client still needs to query DNS.

About openDNS: Still a good product, with a few small issues as explained. To say it's No good is too easy. But NXFilter is much better, I agree

For people that want an easy solution without running a server themselves, Opendns is still a good product.

Many thanks for you effort clearing this up. I think this thread is interesting reading for admins.

Just play around a little with privoxy, I am sure if you spend a few minutes you will love it. Hope they ever create a real whitelist option.

Best regards,

Jeroen

[Daniele Brugnara]

Thank you for sharing this solution. Maybe a little later from when you wrote this but this is very useful for me right now, so don't expect quick answers but long term results. Best regards.