any possible way to block all ads in headless mode?

[headless-man]

since extension not supported in headless mode, any possible way to block all ads? like ublock extension which block all ads

[Isaac Dawson]

Maybe look at the source of ublock https://github.com/gorhill/uBlock and use the devtools commands for blocking requests?

On Sun, Dec 17, 2017 at 12:31 PM headless-man <bakht...@gmail.com> wrote:

since extension not supported in headless mode, any possible way to block all ads? like ublock extension which block all ads

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.
To post to this group, send email to headle...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/ab6836f4-819e-4da4-850c-05a9f6bc8daa%40chromium.org.

[markus...@gmail.com]

I had the same problem when using puppeteer with headless. I ended up using the host file available from:

http://winhelp2002.mvps.org/hosts.txt

For puppeteer I read in this host file:

//now we read the host file

var hostFile = fs.readFileSync('hosts.txt', 'utf8').split('\n');

var hosts = {};

for (var i = 0; i < hostFile.length; i++) {

    var frags = hostFile[i].split(' ');

    if (frags.length > 1 && frags[0] === '0.0.0.0') {

        hosts[frags[1].trim()] = true;

    }

}

When loading a page I then filter out requests for these domains (and optionally images):

    page.on('request', request => {

        var domain = null;

        if (task.input.blockads) {

            var frags = request.url().split('/');

            if (frags.length > 2) {

                domain = frags[2];

            }

        }

        if ((task.input.blockads && hosts[domain] === true) || (!task.input.includephotos && request.resourceType() === 'image')) {

            request.abort();

        }

        else {

            request.continue();

        }

    });

This solution hugely improved the speed of our scraper.

[nkic...@gmail.com]

That's quite a brilliant way to block ads in puppeteer.

BTW, what is task.input.blockads ??

Where are they from? task, input, blockads

Thank you in advance.

[oluwafe...@gmail.com]

On Sunday, 17 December 2017 04:31:53 UTC+1, headless-man wrote:

since extension not supported in headless mode, any possible way to block all ads? like ublock extension which block all ads

Co-ask 

[remi....@gmail.com]

Hey there,

If anyone is still looking for a well-maintained and robust ad-blocking solution for puppeteer-based projects. I am maintaining one at https://www.npmjs.com/package/@cliqz/adblocker-puppeteer (source: https://github.com/cliqz-oss/adblocker/tree/master/packages/adblocker-puppeteer#usage ). It only takes a few lines to setup, it’s written in pure JavaScript (so no compilation step required), and it’s really efficient.

Best,

Rémi

[jellybelly.j...@gmail.com]

Hi Rémi, the adblocker you mentioned is very helpful. Does it block ads from youtube?

[remi]

Hi there,

Yes the adblocker is able to block Youtube ads. It might be the only one for Puppeteer able to do that (except if you load an adblocker extension but then I think you cannot do that in headless mode).

[Piotr Nowak]

Little late response but I just stumbled upon this thread.

I beleive task.input.blockads/task.input.includephotos are objects where there set to true and false respectivly.

Shortened it just a bit but below works for me.

    let blockads = true;

    let includephotos = false;

    page.on('request', request => {

        var domain = null;

        if (task.input.blockads) {

            var frags = request.url().split('/');

            if (frags.length > 2) {

                domain = frags[2];

            }

        }

        if ((blockads && hosts[domain] === true) || (!includephotos && request.resourceType() === 'image')) {

            request.abort();

        }

        else {

            request.continue();

        }

    });

[mertcan mıgırdag]

?

2 Ekim 2020 Cuma tarihinde saat 22:40:08 UTC+3 itibarıyla Piotr Nowak şunları yazdı: