Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

I have a fantastic HOSTS file (where can I post it for others to benefit)?

438 views
Skip to first unread message

Ned Turnbull

unread,
Aug 29, 2014, 7:46:37 PM8/29/14
to
Where can I post a 25,000 line text file for others to benefit?

I have a fantastic hosts file, improved over years, always adding
the MVP hosts file to it, and adding about a thousand obnoxious
domains that I've run into such that I almost never see a valid
popup browser (they pop up, but they are all unfound). I never
see in-page ads either.

Yes, I know that many of you use pop up blockers, and hence you have
no need for a great hosts file. Yes I know many of you use noscript,
and ghostery and all sorts of spyware blasters.

This isn't about that.
This is simply about the hosts text file. Period.
And how to upload it so that others can benefit from my effort.

All this is asking is *where* I can post my excellent HOSTS file
so that others may benefit from using it?

It's looooooong (it's almost 25K lines long!).

I just want to post it, as a text file, so others can use it and
improve it. What location do you suggest?

Caver1

unread,
Aug 29, 2014, 8:02:21 PM8/29/14
to
Do you have a Dropbox account or the like that you can post a share address?

--
Caver1

Shadow

unread,
Aug 29, 2014, 8:03:18 PM8/29/14
to
Zip it, make a torrent and upload it to a tracker. Post the
magnet URL here.
But a hosts file that big would probably have thousands of
obsolete entries, and maybe block hosts that others might not consider
eViL. For example, MVP used to block my Brazilian email provider.
My currents hosts file has under 200 entries. It just blocks
sites with major privacy issues.
[]'s
--
Don't be evil - Google 2004
We have a new policy - Google 2012

Paul

unread,
Aug 29, 2014, 8:09:33 PM8/29/14
to

Ned Turnbull

unread,
Aug 29, 2014, 8:10:03 PM8/29/14
to
On Fri, 29 Aug 2014 20:02:21 -0400, Caver1 wrote:

> Do you have a Dropbox account or the like that you can post a share address?

No. I have zero shared accounts.
I use tinypic.com when I need to upload a file.
If there is a good anonymous site for (large) TEXT files, I can use that.

I just ran a word count on the MVP HOSTS file compared to mine:
$ wc -l HOSTS
==> 15311 HOSTS

$ wc -l /etc/hosts
==> 23784 /etc/hosts

So, my hosts file is 50% larger than the MVP hosts file.
I personally collected all the obnoxious URLs of popup after popup, over
the span of years, and *every* addition was vetted by me.

Of course, you have no idea how thorough I am, but, since it's a text file,
you can easily run a sort and then a diff to easily see the eight thousand
domains I've personally added, over the fifteen thousand that are in the MVP
hosts file.

I just merged mine with the MVP hosts file today, so, my file has *every*
domain that is in today's MVP hosts file, plus the eight thousand that I
have added.

Anyway, let me know *where* I can upload the text file so that others can
stand on my shoulders if they want.

NOTE: Linux stores line feeds and the end of file character differently
than Windows does; so, this file is in Linux ASCII text format (which
Windows can easily handle). The MVP file is in Windows ASCII text format.

Ned Turnbull

unread,
Aug 29, 2014, 8:18:53 PM8/29/14
to
On Fri, 29 Aug 2014 21:03:18 -0300, Shadow wrote:

> Zip it, make a torrent and upload it to a tracker. Post the
> magnet URL here.

That's a very interesting idea!
I have *never* uploaded (seeded?) a torrent in my life!
[I have downloaded torrents, e.g., my Linux distribution.]

So, I will see if I can figure out *how* to "seed" a torrent.
If I do seed, can I keep my IP address anonymous (since it's fixed) when
I create a torrent?
[I can do it from Starbucks, but, it would be more convenient from home.]

> But a hosts file that big would probably have thousands of
> obsolete entries, and maybe block hosts that others might not consider
> eViL.

Hmmmm.... I wonder.
I know what you're saying.

And, it can certainly be the case.
But, I can tell you that the MVP hosts file does not block enough popups,
or I wouldn't have gotten them in the first place.

once a popup happens, I simply put it in the hosts file, and that
popup never happens again. Of course, as you imply, I don't go *back*,
years later, and revisit them.

So, like any hosts file, you'll have to decide for yourself if you
want the ones I've added. If it's worthwhile, I can figure out a command
that will strip out of my current hosts file all the hosts that are in
the MVP hosts file, so, the result would be *just* the eight thousand
that I have added myself.

You can then *look* at those eight thousand, and judge for yourself if
you want to add them or not. If that would be more useful, I can upload
just my additions, instead.

Note: But, of course, then the user has to *merge* the two, which is
easy for many of us on Linux but Windows users may have a problem
because of the different way Linux and Windows stores ASCII text file
line feeds and the end of file character.

Ned Turnbull

unread,
Aug 29, 2014, 8:35:56 PM8/29/14
to
On Fri, 29 Aug 2014 20:09:33 -0400, Paul wrote:

> http://pastebin.com/

I had never heard of pastebin.com.
So, as a guest, I pasted the twenty-four thousand line hosts
text file and named it:
23734_line_hosts_file_mvp_plus_mine.txt

The error was:
You have exceeded the maximum file size of 500 kilobytes
per paste. PRO users don't have this limit!

Looking at the file size, it's 719KB, so, I broke off the
first 10,000 lines, and called that first half:
20140829_ned_hosts_file_part1of2.txt
And then I put the rest of the lines into:
20140829_ned_hosts_file_part2of2.txt

This complained:
Your paste has triggered our automatic SPAM detection filter.
This happens when links or certain keywords are detected in a
paste. It can also happen if you are creating a lot of items
in a short period of time. To confirm you are not a bot,
please fill out the captcha below.

Luckily, the captcha was humanly possible (I've seen much worse),
and I was able to paste the two halves of the hosts text file.

I don't know what the location is, but I *think* the two URLs are:
http://pastebin.com/50qSsj43 for 20140829_ned_hosts_file_part1of2.txt
http://pastebin.com/kNwVWKxb for 20140829_ned_hosts_file_part2of2.txt

At least those are what the top URL bar seems to show for each file.
Let me know how that works for you.


Message has been deleted

Ned Turnbull

unread,
Aug 29, 2014, 10:09:48 PM8/29/14
to
On Sat, 30 Aug 2014 00:57:19 +0000, Dustin wrote:

> Why not .zip it and paste that instead? the zip would be much much
> smaller. It's text. It'll compress well.

I have never zipped anything (I just unzip them).
So, I ran zip on hosts, but it created nothing.
$ zip hosts
zip error: Nothing to do! (hosts.zip)

OK. So I ran gzip on hosts and it created hosts.gz
$ gzip hosts
Creates hosts.gz

A 'head' of that hosts.gz file shows that it's "stuff",
but, now it comes time to cut and paste it.

So, I run geedit to then copy and paste:
$ gedit hosts.gz
And, huh? Well, I'll be? What the ...
I see the TEXT file!

Somehow, gedit is unzipping the hosts.gz.

So, how am I supposed to copy and paste the contents into the
paste.bin form?

Normally, I run gedit, and then I can use the file command to
copy and then to paste.

But, if gedit automagically unzips the gz file, then I can't
copy and paste.

So, I gave up on that idea of zipping it, unless someone knows
how to copy and paste the scrambled egg contents of a zip file.

Mayayana

unread,
Aug 29, 2014, 10:16:45 PM8/29/14
to
| Where can I post a 25,000 line text file for others to benefit?
|

I certainly wouldn't want a HOSTS file that big. The vast majority
of the entires are bound to be sites I'll never visit, either first-
party or third-party. I know there are many sites like that already
in the MVP HOSTS file. My own HOSTS file has about 250 entries.
Even that is more than is needed. The vast majority of sleazy third-
party domains linked from webpages are going to a small number
of sites, like doubleclick, google-analytics.com, googletagmanager.com,
googletagservices.com, 2o7.net, valueclick, facebook.com, etc.
I use a Desktop script occasionally to parse webpages for URLs
and rarely find one that I don't already have in my HOSTS file.

But if you want to make it available to people then why not get
your own website? You can have a fullscale website for less than
$10/month, and have as many email addresses as you can ever
use with your own domain.

That's the whole idea of the Internet, after all. It's a public
commons where all can take part. There's no reason to regard
it as merely a set of ad-supported services.


Wildman

unread,
Aug 29, 2014, 10:21:02 PM8/29/14
to
On Sat, 30 Aug 2014 02:09:48 +0000 (UTC)
Ned Turnbull <NedTu...@example.com> wrote:

> On Sat, 30 Aug 2014 00:57:19 +0000, Dustin wrote:
>
> > Why not .zip it and paste that instead? the zip would be much much
> > smaller. It's text. It'll compress well.
>
> I have never zipped anything (I just unzip them).
> So, I ran zip on hosts, but it created nothing.
> $ zip hosts
> zip error: Nothing to do! (hosts.zip)

Should have been: zip hosts hosts.zip

> OK. So I ran gzip on hosts and it created hosts.gz
> $ gzip hosts
> Creates hosts.gz
>
> A 'head' of that hosts.gz file shows that it's "stuff",
> but, now it comes time to cut and paste it.
>
> So, I run geedit to then copy and paste:
> $ gedit hosts.gz
> And, huh? Well, I'll be? What the ...
> I see the TEXT file!
>
> Somehow, gedit is unzipping the hosts.gz.
>
> So, how am I supposed to copy and paste the contents into the
> paste.bin form?
>
> Normally, I run gedit, and then I can use the file command to
> copy and then to paste.
>
> But, if gedit automagically unzips the gz file, then I can't
> copy and paste.
>
> So, I gave up on that idea of zipping it, unless someone knows
> how to copy and paste the scrambled egg contents of a zip file.
>

You unzip it first. The idea to zip the file was to reduce
its size for posting.

--
<Wildman> GNU/Linux user #557453
The cow died so I don't need your bull!

Ned Turnbull

unread,
Aug 29, 2014, 10:56:04 PM8/29/14
to
On Fri, 29 Aug 2014 21:21:02 -0500, Wildman wrote:

> You unzip it first. The idea to zip the file was to reduce
> its size for posting.

I understand the concept of a zip file.
But, I have to PASTE the *contents* of the file into pastebin.com
in order for that web site to work. Right?

So, it's not a file upload kind of thing.

If it were a file upload, then the zip concept would work better.
(much better).
But, pastebin is a content *paste* concept (it seems).

And, well, pasting the contents of a zip file is messy.
Very messy.

So, I found a better way just now...

Here are just the thousands of sites I have personally added
that are *not* in the MVP Hosts file today:
http://pastebin.com/WJy5WH2c

I created that file by comparing my current hosts file with
the current MVP hosts file at http://winhelp2002.mvps.org/hosts.htm

To extract just my changes, I used the techniques explained here:
http://stackoverflow.com/questions/4544709/compare-two-files-line-by-line-and-generate-the-difference-in-another-file

Specifically, I called my hosts file, "file1", and then I called today's
MVP HOSTS file "file2", and then I ran this command to get the difference:
$ comm -2 -3 <(sort file1) <(sort file2) > file3

The results, i.e., "file3", were pasted into pastebin.com.

These are the thousands of web sites I have found to be obnoxious
that are *not* in the MVP Hosts file today:
http://pastebin.com/WJy5WH2c

If you find a problem, please call me on it.
If you know how to tell the MVP people about these additions, please do.

Ned Turnbull

unread,
Aug 29, 2014, 11:03:05 PM8/29/14
to
On Fri, 29 Aug 2014 22:16:45 -0400, Mayayana wrote:

> I certainly wouldn't want a HOSTS file that big. The vast majority
> of the entires are bound to be sites I'll never visit, either first-
> party or third-party.

Here are just the ten thousand sites that I've visited that I found
obnoxious. http://pastebin.com/WJy5WH2c

I created this listing by copying my current hosts file to "file1",
and then copying today's MVP Hosts file to "file2" (removing Windows
style line feeds and end of file characters) and then running
the command shown at the web site below:
http://stackoverflow.com/questions/4544709/compare-two-files-line-by-line-and-generate-the-difference-in-another-file

Namely:
$ comm -2 -3 <(sort file1) <(sort file2) > file3

That result was pasted into pastebin.com, which resulted in this URL:
http://pastebin.com/WJy5WH2c

With those ten thousand additions, I almost never see a popup or an
advertisement, plus a ton of trackers are disabled.

Sure, there are *other* ways to do that, but if you have Mac, Linux, and
Windows machines, this one file will work for all three and it's trivial
to implement.

Anyway, I give it to the world, and the world can decide, for themselves,
whether they want to avail themselves of the power, or not.

NOTE: There is no performance hit, that I know of, except for Windows there
is one service you want to turn off (anyway), which is explained in the
MVP HOSTS web site. On Linux & Mac, it works flawlessly without any performance
hit whatsoever, even with tens of thousands of entries in the hosts file.

Ned Turnbull

unread,
Aug 29, 2014, 11:05:46 PM8/29/14
to
On Fri, 29 Aug 2014 21:21:02 -0500, Wildman wrote:

> Should have been: zip hosts hosts.zip

Thank you for explaining that.

I didn't spend a lot of time on the zipping, but I do
appreciate the suggestion, so I'll save that syntax for
the future.

The reason I didn't spend a lot of time was twofold:

1. A zip file would be gibberish to cut and paste and most
people would be leery of copying and pasting it.

2. When I tried, the gedit command was "too smart" as it
automagically unzipped it, when I didn't want it to.

So, I gave up on the zip file idea pretty quickly, but,
please do note that the MVP Hosts file "is" a zipped file,
so, it's a perfectly valid concept; it just doesn't work
with the suggested pastebin.com web site.

Ned Turnbull

unread,
Aug 29, 2014, 11:08:48 PM8/29/14
to
On Fri, 29 Aug 2014 22:16:45 -0400, Mayayana wrote:

> But if you want to make it available to people then why not get
> your own website?

I write apnotes all day, every day. I *should* create my own web site.
I really should. I agree.

But, for now, here are "my" ten thousand additions to the MVP Hosts
file, which you can have, free of charge, to do what you want with:
http://pastebin.com/WJy5WH2c

Wildman

unread,
Aug 29, 2014, 11:09:46 PM8/29/14
to
I'm afraid I don't know anything about pastebin.com. The way
it works makes a zip a bad idea. Someone else suggested to
use Dropbox and that would be my recommendation as well. It
will accept any type of file without having to use copy/paste.
You just drag your file(s) into the "Dropbox" directory on the
hard drive. All you need is a valid email address to sign up
for their free account. I can't remember how much space you
get but I think it is around 3-4gig.

Ned Turnbull

unread,
Aug 29, 2014, 11:20:36 PM8/29/14
to
On Fri, 29 Aug 2014 22:09:46 -0500, Wildman wrote:

> I'm afraid I don't know anything about pastebin.com. The way
> it works makes a zip a bad idea. Someone else suggested to
> use Dropbox

Hi Wildman,
I think we're fine now.

While a zip file is generally a great idea, it won't work here
because we want the file to remain as a text file.

So the pastebin.com idea by Paul was perfect, except that my
combined file was too large for a single paste.

I've since stripped out the ten thousand additions I've manually
added to the MVP Hosts file, so, now, *anyone* can combine the
two on their own with a simple word processor.

Here are the ~15,000 domains that MVP folks think you should avoid:
http://winhelp2002.mvps.org/hosts.htm

Here are ~10,000 additional domains I've found that "I" would avoid:
http://pastebin.com/WJy5WH2c

Combining those two files gives you ~25,000 domains which will never
bother you again. This works on all platforms, and is trivial to
implement by anyone who knows how a hosts file works.

~BD~

unread,
Aug 30, 2014, 6:00:05 AM8/30/14
to
Ned Turnbull wrote:
[....]

> All this is asking is *where* I can post my excellent HOSTS file
> so that others may benefit from using it?
>
> It's looooooong (it's almost 25K lines long!).
>
> I just want to post it, as a text file, so others can use it and
> improve it. What location do you suggest?

You've had lots of advice already ...... but I'd like you to carry out a
test!

Try posting your loooooooooong Hosts file into this test group:

Newsgroups: 0.test

It's REAL! See Message-ID: <ltlsil$6u9$1...@enother.net>

An experiment really! And it doesn't COST anything! ;-)

--
First they ignore you, then they laugh at you, then they fight you, then
you win.

dadiOH

unread,
Aug 30, 2014, 6:05:47 AM8/30/14
to
"Ned Turnbull" <NedTu...@example.com> wrote in message
news:ltret8$od4$1...@news.mixmin.net
> On Fri, 29 Aug 2014 22:16:45 -0400, Mayayana wrote:
>
> > I certainly wouldn't want a HOSTS file that big. The vast majority
> > of the entires are bound to be sites I'll never visit, either first-
> > party or third-party.
>
> With those ten thousand additions, I almost never see a popup or an
> advertisement, plus a ton of trackers are disabled.
>
> Sure, there are *other* ways to do that, but if you have Mac, Linux, and
> Windows machines, this one file will work for all three and it's trivial
> to implement.
>
> Anyway, I give it to the world, and the world can decide, for
> themselves,
> whether they want to avail themselves of the power, or not.

Where should the statue be erected?

--

dadiOH
____________________________

Winters getting colder? Tired of the rat race?
Taxes out of hand? Maybe just ready for a change?
Check it out... http://www.floridaloghouse.net

p-0''0-h the cat (ES)

unread,
Aug 30, 2014, 6:19:20 AM8/30/14
to
On Sat, 30 Aug 2014 06:05:47 -0400, "dadiOH" <dad...@invalid.com> wrote:

>"Ned Turnbull" <NedTu...@example.com> wrote in message
>news:ltret8$od4$1...@news.mixmin.net
>> On Fri, 29 Aug 2014 22:16:45 -0400, Mayayana wrote:
>>
>> > I certainly wouldn't want a HOSTS file that big. The vast majority
>> > of the entires are bound to be sites I'll never visit, either first-
>> > party or third-party.
>>
>> With those ten thousand additions, I almost never see a popup or an
>> advertisement, plus a ton of trackers are disabled.
>>
>> Sure, there are *other* ways to do that, but if you have Mac, Linux, and
>> Windows machines, this one file will work for all three and it's trivial
>> to implement.
>>
>> Anyway, I give it to the world, and the world can decide, for
>> themselves,
>> whether they want to avail themselves of the power, or not.
>
>Where should the statue be erected?

Who is this geezer. Has anyone checked his bona fides?

Sent from my iFurryUnderbelly.

--
p-0.0-h the cat

Internet Terrorist, Mass sock puppeteer, Agent provocateur, Gutter rat,
Devil incarnate, Linux user#666, BaStarD hacker, Resident evil, Monkey Boy,
Certifiable criminal, Spineless cowardly scum, textbook Psychopath,
the SCOURGE, l33t p00h d3 tr0ll, p00h == lam3r, p00h == tr0ll, troll inf�me,
the OVERCAT [The BEARPAIR are dead, and we are its murderers], lowlife troll,
shyster [pending approval by STATE_TERROR], cripple, sociopath, kook,
smug prick, smartarse, arsehole, moron, idiot, imbecile, snittish scumbag,
liar, total ******* retard, shill, pooh-seur, and scouringerer.

NewsGroups Numbrer One Terrorist

Honorary SHYSTER and FRAUD awarded for services to Haberdashery.
By Appointment to God Frank-Lin.

Signature integrity check
md5 Checksum: be0b2a8c486d83ce7db9a459b26c4896

Ned Turnbull

unread,
Aug 30, 2014, 7:33:26 AM8/30/14
to
On Sat, 30 Aug 2014 06:05:47 -0400, dadiOH wrote:

> Where should the statue be erected?

In the host country, of course!

Ned Turnbull

unread,
Aug 30, 2014, 7:34:29 AM8/30/14
to
On Sat, 30 Aug 2014 11:19:20 +0100, p-0''0-h the cat (ES) wrote:

> Who is this geezer. Has anyone checked his bona fides?

Is this the "real" pooh?
If so, I'm glad to re-make your acquaintance!

Ned Turnbull

unread,
Aug 30, 2014, 7:35:04 AM8/30/14
to
On Sat, 30 Aug 2014 11:00:05 +0100, ~BD~ wrote:

> An experiment really! And it doesn't COST anything! ;-)

:)

p-0''0-h the cat (ES)

unread,
Aug 30, 2014, 7:50:51 AM8/30/14
to
Eeyore!!!!!!!!!!!!!!!!!!!!

bad sector

unread,
Aug 30, 2014, 8:09:18 AM8/30/14
to
'atta boy!!!!!!!!!!!!

thanks


Shadow

unread,
Aug 30, 2014, 9:05:55 AM8/30/14
to
On Sat, 30 Aug 2014 00:18:53 +0000 (UTC), Ned Turnbull
<NedTu...@example.com> wrote:

>On Fri, 29 Aug 2014 21:03:18 -0300, Shadow wrote:
>
>> Zip it, make a torrent and upload it to a tracker. Post the
>> magnet URL here.
>
>That's a very interesting idea!
>I have *never* uploaded (seeded?) a torrent in my life!
>[I have downloaded torrents, e.g., my Linux distribution.]
>
>So, I will see if I can figure out *how* to "seed" a torrent.
>If I do seed, can I keep my IP address anonymous (since it's fixed) when
>I create a torrent?

Your torrent client will create a torrent for you. No IP
address is involved. When you upload it to a tracker, they will get
your IP. But a hosts file would not draw any more attraction than a
private email letter. OK, OK. Someone will be watching.
The problem is when you seed it you will have to be online, so
that unique seeder will be your IP. One you get people downloading,
their IPs will appear too, so it will be hard to see which one you
are. Which is why when people seed new stuff it's usually from
multiple hosts.
Even so, you will be far more "private" than if you upload to
Dropbox.

>
>once a popup happens, I simply put it in the hosts file, and that
>popup never happens again. Of course, as you imply, I don't go *back*,
>years later, and revisit them.
>
>So, like any hosts file, you'll have to decide for yourself if you
>want the ones I've added. If it's worthwhile, I can figure out a command
>that will strip out of my current hosts file all the hosts that are in
>the MVP hosts file, so, the result would be *just* the eight thousand
>that I have added myself.

There is windows freeware that can do that, or just convert
both to Unix text and do a diff.
>
>You can then *look* at those eight thousand, and judge for yourself if
>you want to add them or not. If that would be more useful, I can upload
>just my additions, instead.
>
>Note: But, of course, then the user has to *merge* the two, which is
>easy for many of us on Linux but Windows users may have a problem
>because of the different way Linux and Windows stores ASCII text file
>line feeds and the end of file character.

I really don't mind ads. As long as they are not too annoying,
like that girl doing froggy jumps or whatever on the Piratebay pages.
Oh ... that's geolocation and analytics. Pronto ... blocked.

127.0.0.1 main.exoclick.com
127.0.0.1 static-ssl.exoclick.com
127.0.0.1 ads.exoclick.com
127.0.0.1 syndication.exoclick.com

I just block the geolocation, analytics and (best I can)
canvassing stuff. And of course, any social media links.

Shadow

unread,
Aug 30, 2014, 9:09:58 AM8/30/14
to
On Fri, 29 Aug 2014 22:16:45 -0400, "Mayayana"
<maya...@invalid.nospam> wrote:

>| Where can I post a 25,000 line text file for others to benefit?
>|
>
> I certainly wouldn't want a HOSTS file that big. The vast majority
>of the entires are bound to be sites I'll never visit, either first-
>party or third-party. I know there are many sites like that already
>in the MVP HOSTS file. My own HOSTS file has about 250 entries.
>Even that is more than is needed. The vast majority of sleazy third-
>party domains linked from webpages are going to a small number
>of sites, like doubleclick, google-analytics.com, googletagmanager.com,
>googletagservices.com, 2o7.net, valueclick, facebook.com, etc.

Hey !!! That's MY hosts file. Thief !!
;)

Big Al

unread,
Aug 30, 2014, 9:23:16 AM8/30/14
to
Heck, you don't even need $10 a month. I found a free one, limited
space but I'm sure 2 gigs maybe, but I only host a web page. I like
playing with html code, so this gives me an outlet.

Ned Turnbull

unread,
Aug 30, 2014, 9:31:07 AM8/30/14
to
On Sat, 30 Aug 2014 10:05:55 -0300, Shadow wrote:

> The problem is when you seed it you will have to be online, so
> that unique seeder will be your IP.

Can I seed from Tor?

Ned Turnbull

unread,
Aug 30, 2014, 9:33:17 AM8/30/14
to
On Sat, 30 Aug 2014 10:09:58 -0300, Shadow wrote:

> Hey !!! That's MY hosts file. Thief !!

Actually, it *could* have elements of everyone's hosts file!
I culled the entries from a huge variety of sources.

Big Al

unread,
Aug 30, 2014, 9:46:58 AM8/30/14
to
Ned Turnbull wrote on 8/29/2014 7:46 PM:
> Where can I post a 25,000 line text file for others to benefit?
>
> I have a fantastic hosts file, improved over years, always adding
> the MVP hosts file to it, and adding about a thousand obnoxious
> domains that I've run into such that I almost never see a valid
> popup browser (they pop up, but they are all unfound). I never
> see in-page ads either.
<snip>

I'm going to ask, but let's not dwell on this, but in the vein of this
subject: How does one go about finding out all these bogus host
addresses so you can build your own HOST file. Or even, other than
the one link I did see, are there starter lists available?

This all sounds like a great idea, and everyone SHOULD be doing it.
You've all made a great case between all the posts. I just can't
figure out how to get the links. I do clear cookies from my browser,
and I guess those would be a good start. Things like click.net,
adtracket.com etc. You can be sure when the names are that clear.




J.O. Aho

unread,
Aug 30, 2014, 9:56:19 AM8/30/14
to
On 30/08/14 01:46, Ned Turnbull wrote:
> Where can I post a 25,000 line text file for others to benefit?

the hosts file was never intended for this amount of hosts and this
makes things slower...

> I have a fantastic hosts file, improved over years, always adding
> the MVP hosts file to it, and adding about a thousand obnoxious
> domains that I've run into such that I almost never see a valid
> popup browser (they pop up, but they are all unfound). I never
> see in-page ads either.
>
> Yes, I know that many of you use pop up blockers, and hence you have
> no need for a great hosts file. Yes I know many of you use noscript,
> and ghostery and all sorts of spyware blasters.

Hosts file has the negative side, you need to copy it to each and every
machine you have, including portable devices, it's slow too and don't
prevent popups from popping up. It's never been the recommended way to
block sites, use proper methods for that like privoxy or addblokcer
plugins. Privoxy don't need much maintenance as it uses regular
expressions to find the advertisement, so it can block such which you
haven't seen before.


> This isn't about that.
> This is simply about the hosts text file. Period.
> And how to upload it

There are loads of free file hosting, just use one of those.


> All this is asking is *where* I can post my excellent HOSTS file
> so that others may benefit from using it?
>
> It's looooooong (it's almost 25K lines long!).

It's too long, shouldn't be much more than 79 bytes, if it's that short,
then it's excellent, if it's longer it's a bad one.


> I just want to post it, as a text file, so others can use it and
> improve it. What location do you suggest?

You can get my improved one here at once:

127.0.0.1 localhost localhost.localdomain
::1 localhost localhost.localdomain


--

//Aho

Shadow

unread,
Aug 30, 2014, 10:11:33 AM8/30/14
to
On Sat, 30 Aug 2014 09:46:58 -0400, Big Al <Big...@invalid.com> wrote:

>I'm going to ask, but let's not dwell on this, but in the vein of this
>subject: How does one go about finding out all these bogus host
>addresses so you can build your own HOST file. Or even, other than
>the one link I did see, are there starter lists available?

Most browsers have a key shortcut to see the source of the
page you are viewing.
In Firefox, it's CTRL-U
Look at the links, and put the ones that are obviously not
supposed to be there in your hosts file.
You will have to close the browser and flush your DNS cache
for the changes to take effect.
Read up on how to make a hosts file and where it should be
placed. Some AV software will not allow you to make one (Avira for
example) unless you go into advanced configuration settings.

J.O. Aho

unread,
Aug 30, 2014, 11:04:44 AM8/30/14
to
On 30/08/14 15:46, Big Al wrote:
> Ned Turnbull wrote on 8/29/2014 7:46 PM:
>> Where can I post a 25,000 line text file for others to benefit?

Just a reminder, no one benefits from a such list and they will never do.

>> I have a fantastic hosts file, improved over years, always adding
>> the MVP hosts file to it, and adding about a thousand obnoxious
>> domains that I've run into such that I almost never see a valid
>> popup browser (they pop up, but they are all unfound). I never
>> see in-page ads either.
> <snip>
>
> I'm going to ask, but let's not dwell on this, but in the vein of this
> subject: How does one go about finding out all these bogus host
> addresses so you can build your own HOST file. Or even, other than
> the one link I did see, are there starter lists available?
>
> This all sounds like a great idea, and everyone SHOULD be doing it.

No, it's not a great idea, just to show how stupid it can be, back in
2012 Australia decided to block a bite more than 1000 sites as they
contained material which ASIC did consider harmful, by blocking those
1000 IP-numbers, they blocked 250000 sites amongst others Melbourne Free
University.

By blocking an IP, you will exclude many other sites than the one you do
not want to get advertisements from.

You also make your internet slower, not just for browsing, but for all
other activity too.

If you want to block sites, then use proper tools like privoxy or those
add blocker plug-ins, as this way you can get rid of the advertisement
without blocking legitimate content and you aren't slowing down for
other protocols.

--

//Aho

M.L.

unread,
Aug 30, 2014, 11:16:04 AM8/30/14
to
http://winhelp2002.mvps.org/hosts.htm
<quote>
Important Note: The HOSTS file now contains a change in the prefix in
the HOSTS entries to "0.0.0.0" instead of the usual "127.0.0.1".
This was done to resolve a slowdown issue that occurs with the change
Microsoft made in the "TCP loopback interface" in Win8.1.

This change in the prefix should not affect users. I've had some
feedback and COMODO antivirus, Homer Webserver and System Mechanic
seems to have issues with the "0.0.0.0" prefix ... to resolve this
issue:
You can use the "Replace" function in Notepad to convert the entries,
or HostsMan (see below) has an option for converting the entries to
"0.0.0.0".
</quote>

I assume that " resolve this issue" refers to changing from 127.0.0.1
to 0.0.0.0

Ned Turnbull

unread,
Aug 30, 2014, 1:17:38 PM8/30/14
to
On Sat, 30 Aug 2014 10:16:04 -0500, M.L. wrote:

> I assume that " resolve this issue" refers to changing from 127.0.0.1
> to 0.0.0.0

There's also a dhcp service that needs to be turned off,
which, on very long hosts files, would otherwise cause a problem.

This is a bug in Windows that is about 20 years old, so, they
"may" have finally fixed it by how (I don't know as I no longer
boot to Windows ever since they killed XP I went to Linux).

Ned Turnbull

unread,
Aug 30, 2014, 1:26:14 PM8/30/14
to
On Sat, 30 Aug 2014 09:46:58 -0400, Big Al wrote:

> How does one go about finding out all these bogus host
> addresses so you can build your own HOST file. Or even, other than
> the one link I did see, are there starter lists available?

There are many ways.

I started by searching the net, years ago, for all the hosts file
that were out there.

To eliminate duplicates, I removed all extraneous characters and sorted
the files, because nobody wants to go through this stuff by hand.

Here, for example, is just the one alias I use to strip extraneous
characters out of the MVP Hosts file:
alias cleanhost='cat HOSTS | grep -v 127.0.0.1 | grep -v "::" | sed -e "s/^M//g" -e "s/[ ]/ /g" -e "s/ / /g" -e "s/#.*$//" | sed -e "s/ $//" | sed -e "s/0.0.0.0/127.0.0.1/" | sort -u > hosts.txt'

Anyway, then, EVERY time I have an obnoxious popup window, I put
the URL of that obnoxious window into the hosts file.

I use an ~/.exrc setting to make that easy to edit in the vi editor:
:map v :s;http://\([^/][^/]*\)\(/.*\)*;127.0.0.1 \1

I got that setting from this newsgroup actually, when I had searched
how to do it (I love this newsgroup because people think like I do).

Whenever an advertisement shows in the side, or those silly chat
windows, or a video that pops up on the side, etc., I just inserted
the domain of each of those obnoxious entities into the hosts file.

It used to be I was adding a dozen or score a day, and now it's
down to only one or three a week, since I have most of the bad
ones caught by now. The process will never end, but, we can pool
our efforts, which was why I posted the file in the first place.

Ned Turnbull

unread,
Aug 30, 2014, 1:28:09 PM8/30/14
to
On Sat, 30 Aug 2014 11:11:33 -0300, Shadow wrote:

> You will have to close the browser and flush your DNS cache
> for the changes to take effect.

This is an interesting point, as this has bugged me for years.

I found that Firefox just needs to be closed in order to flush
its cache of hosts. Or, you have to wait a rather long time,
even though you can set the cache flushing in about:config
(I've tried everything and have given up long ago).

You'd think the OS would flush DNS more often, but Firefox is
a weird beast when it comes to remembering the DNS cache.

Ned Turnbull

unread,
Aug 30, 2014, 1:29:01 PM8/30/14
to
On Sat, 30 Aug 2014 17:04:44 +0200, J.O. Aho wrote:

> You also make your internet slower, not just for browsing, but for all
> other activity too.

How does NOT visiting a web site, especially since these are all
obnoxious web sites, make the browsing slower?

Ned Turnbull

unread,
Aug 30, 2014, 1:29:44 PM8/30/14
to
On Sat, 30 Aug 2014 17:04:44 +0200, J.O. Aho wrote:

> If you want to block sites, then use proper tools like privoxy or those
> add blocker plug-ins, as this way you can get rid of the advertisement
> without blocking legitimate content and you aren't slowing down for
> other protocols.

I knew this would come up, and you're the first, I think, to bring it
up, but this thread isn't about the many (very many) other methods of
blocking unwanted domains. :)

Ned Turnbull

unread,
Aug 30, 2014, 1:31:13 PM8/30/14
to
On Sat, 30 Aug 2014 15:56:19 +0200, J.O. Aho wrote:

> the hosts file was never intended for this amount of hosts and this
> makes things slower...

On Linux, it does not seem to have any deleterious effect on speed.

In fact, one could argue the machine is *faster* since it doesn't
waste time visiting any number of a hosts of common tracking sites
and advertisement sites.

But, I haven't tested the speed (simply because it hasn't been an
issue, and I have as large a hosts file as anyone here).

Ned Turnbull

unread,
Aug 30, 2014, 1:33:19 PM8/30/14
to
On Sat, 30 Aug 2014 15:56:19 +0200, J.O. Aho wrote:

> Hosts file has the negative side, you need to copy it to each and every
> machine you have, including portable devices

Portable devices require root, unfortunately.

But, how can copying a text file to Windows, Linux, and Mac be harder
than trying to get a huge number of programs to perform the same task
to work on all these platforms?

Besides, the hosts file works for all protocols, whereas most of these
other methods work only for some protocols.

There's no way there is any easier yet more complete & yet totally
portable method than the hosts file method. Yes, there *are* other
methods, but let's keep this thread to just the hosts file method.

bad sector

unread,
Aug 30, 2014, 1:34:46 PM8/30/14
to
not for speed because it wouldn't really be any smaller but i wonder if
there'd be any point in translating them to the IP #'s?

Ned Turnbull

unread,
Aug 30, 2014, 1:38:24 PM8/30/14
to
On Sat, 30 Aug 2014 17:28:09 +0000, Ned Turnbull wrote:

> I found that Firefox just needs to be closed in order to flush
> its cache of hosts. Or, you have to wait a rather long time,

BTW, any web site that brought up those obnoxious "do you really
want to close" windows, was also put in that hosts file.

Once Firefox was closed and opened, you never see that message
again. If you were impatient, you could go to about:config
and turn off javascript, and then close the window.

This is one thing about Firefox caching that has been a pain,
but the two solutions are easy enough:
a) Turn off javascript temporarily (just to close the window)
b) Wait for the cache to flush (and then you can close the window)

Mayayana

unread,
Aug 30, 2014, 2:09:49 PM8/30/14
to
| > I certainly wouldn't want a HOSTS file that big. The vast majority
| > of the entires are bound to be sites I'll never visit, either first-
| > party or third-party.
|
| Here are just the ten thousand sites that I've visited that I found
| obnoxious. http://pastebin.com/WJy5WH2c
|

Another option, which you may already know about,
is to block 3rd-party images. In Firefox/Pale Moon
that used to be an option in the settings. With the
corrupting influence from Google it was changed and
hidden. (Mozilla get nearly all of their excessive
income from Google.) But the setting is still there in
about:config
set (create if necessary): permissions.default.image 3

That also seems to block other external files. It can
make some websites ugly, in cases where a CSS file and/or
images are loaded from a different domain. But for the
most part it works OK. The vast majority of ads are coming
from 3rd-party spyware servers like Google/Doubleclick, so
with 3rd-party images blocked you allow honest ads -- the
ones that are actually on the website you chose to visit --
and you block spyware ads.

There are also the issues of script and iframes. If you
only care about blocking ads then that won't matter, and
you might not want to put up with the hassle. But if you
care about being tracked, take a look at this code in the
pastebin page you linked:

<iframe src="//www.facebook.com/plugins/like.php?

Code like that is in most commercial webpages. It's loading
a webpage from Facebook into a iframe, which is essentially
a second browser window. What you see is just a small Facebook
button. If you block 3rd-party images you might not even
see that. But technically you've visited Facebook. The parameters
in the page request tell information about where you're coming
from, and by requesting the page you send your IP address,
userAgent, etc to Facebook. That means that Facebook is
following you all around the Internet, even if you've never visited
their website. If you have a Facebook account then their iframes
allow them to know what you're doing when you're not logged in.
Even if you just allow first-party cookies they can probably know
who you are easily. The iframe they use makes their hidden webpage
a page that you "chose" to visit, so any cookies loaded from it
are first-party. But that's just icing on the cake for their datamining
operation. Your IP is probably enough for them to figure out who you
are, and to track your movements online.

Your massive HOSTS file is blocking at least 5 domains from
Tomshardware.com, which you may never visit. It's blocking
image servers from various countries. It even blocks f**k.org
and zerofreepopcorn.com, whatever they are. :) But it doesn't
block Facebook. It doesn't even block google-analytics.com,
which is tracking you from the vast majority of webpages you
visit, including your pastebin page. That's the trouble with a
giant HOSTS file. It's likely to be 98% irrelevant, and while
you're blocking the oddball ad server from some obscure page
you'll probably never visit, you're not necessarily blocking the
sites that matter most.

Also worth a try is Acrylic, which is a free DNS server program.
It acts as a proxy and has it's own HOSTS file that allow wildcards.
So you can block things like *.doubleclick.net and *.doubleclick.com
to block all Google/Doubleclick ads. The normal HOSTS file requires
adding each possible subdomain. If you look at your HOSTS file
you'll see a great deal of redundancy due to that problem. Ad
servers can just keep changing the subdomain to thwart your
HOSTS file. You might have entries for 200 Doubleclick subdomains,
but you don't have an entry for the one they might create next week.


Mayayana

unread,
Aug 30, 2014, 9:18:31 AM8/30/14
to

HASM

unread,
Aug 30, 2014, 11:10:33 AM8/30/14
to
Ned Turnbull <NedTu...@example.com> writes:

> Here are ~10,000 additional domains I've found that "I" would avoid:
> http://pastebin.com/WJy5WH2c

I picked this up, sampled a few, and fed it to host/nslookup.

Lots of them don't resolve at all, so why would anyone want to enter
addresses on one's hosts file remapping to localhost hosts that don't seem
to exist, or expired?

I think you should clean your additions of those, maybe keep them in a
separate file and check periodically if they came back.

-- HASM

J�@localhost.invalid

unread,
Aug 30, 2014, 3:23:42 PM8/30/14
to
On 8/29/2014 6:46 PM, Ned Turnbull wrote:
> Where can I post a 25,000 line text file for others to benefit?
>
> I have a fantastic hosts file, improved over years, always adding
> the MVP hosts file to it, and adding about a thousand obnoxious
> domains that I've run into such that I almost never see a valid
> popup browser (they pop up, but they are all unfound). I never
> see in-page ads either.
>
> Yes, I know that many of you use pop up blockers, and hence you have
> no need for a great hosts file. Yes I know many of you use noscript,
> and ghostery and all sorts of spyware blasters.
>
> This isn't about that.
> This is simply about the hosts text file. Period.
> And how to upload it so that others can benefit from my effort.
>
> All this is asking is *where* I can post my excellent HOSTS file
> so that others may benefit from using it?
>
> It's looooooong (it's almost 25K lines long!).
>
> I just want to post it, as a text file, so others can use it and
> improve it. What location do you suggest?
>

You need to verify that the additions you have put
on your file over the years are still relevant. There
is a program available for download that will check
each and every one, and it takes hours, if not days,
to check as many as you have. I'm sorry but I can't
recall the name of this utility.

Ned Turnbull

unread,
Aug 30, 2014, 4:36:50 PM8/30/14
to
On Sat, 30 Aug 2014 08:10:33 -0700, HASM wrote:

> Lots of them don't resolve at all

I have never tested them after I put them into that file,
so, that's a good idea.

Is there a decent script, existing, which will simply test
if they resolve, and then spit out those that do versus
those that don't?

We could clean it up simply by running *that* script.

Ned Turnbull

unread,
Aug 30, 2014, 4:39:57 PM8/30/14
to
On Sat, 30 Aug 2014 14:09:49 -0400, Mayayana wrote:

> Your massive HOSTS file is blocking at least 5 domains from
> Tomshardware.com,

We should note that those domains are being blocked by
the *original* MVP Hosts file, downloaded yesterday:
http://winhelp2002.mvps.org/hosts.htm

$ grep -i tomshardware HOSTS
0.0.0.0 tracking.tomshardware.com
0.0.0.0 tomshardware.fr.intellitxt.com
0.0.0.0 tomshardware.se.intellitxt.com
0.0.0.0 tomshardware.us.intellitxt.com

So, we'd have to ask *them* why they are blocking that domain.

Ned Turnbull

unread,
Aug 30, 2014, 4:56:35 PM8/30/14
to
On Sat, 30 Aug 2014 14:09:49 -0400, Mayayana wrote:

> It even blocks f**k.org and zerofreepopcorn.com,
> whatever they are...

Hmmm.... I may be confused ... (or you?)...

Maybe you're talking about a *different* hosts file?
I just grep'd my /etc/hosts for that, and it isn't there.

$ grep zerofreepopcorn /etc/hosts
$ grep -i popcorn /etc/hosts

It's not even in the MVP HOSTS file that I had downloaded
yesterday:

$ grep zerofreepopcorn /tmp/HOSTS
$ grep -i popcorn /tmp/HOSTS

Am I confused, or are you confused?

Ned Turnbull

unread,
Aug 30, 2014, 4:58:36 PM8/30/14
to
On Sat, 30 Aug 2014 14:09:49 -0400, Mayayana wrote:

> But it doesn't
> block Facebook. It doesn't even block google-analytics.com,
> which is tracking you from the vast majority of webpages you
> visit, including your pastebin page.

My /etc/hosts does block *some* domains with facebook in
the domain name, e.g.,
$ grep -i facebook /etc/hosts
127.0.0.1 ads.ak.facebook.com
127.0.0.1 creative.ak.facebook.com
127.0.0.1 facebookinc.122.2o7.net
127.0.0.1 facebook-repto1040s2.ahlamountada.com

Of those, the first three are in the MVP HOSTS file, so,
for whatever reason, I must have run into the fourth and
blocked it myself.

The nice thing about a HOSTS file though, is that if YOU
want to block www.facebook.com, you can.

John Hasler

unread,
Aug 30, 2014, 4:51:35 PM8/30/14
to
I find that Privoxy plus NoScript blocks 99.9% of all advertising with
no need for any configuration.
--
John Hasler
jha...@newsguy.com
Dancing Horse Hill
Elmwood, WI USA

Ned Turnbull

unread,
Aug 30, 2014, 5:00:21 PM8/30/14
to
On Sat, 30 Aug 2014 14:09:49 -0400, Mayayana wrote:

> Also worth a try is Acrylic, which is a free DNS server program.
> It acts as a proxy and has it's own HOSTS file that allow wildcards.
> So you can block things like *.doubleclick.net and *.doubleclick.com
> to block all Google/Doubleclick ads.

I do agree that wildcards would be very nice, in a HOSTS file.

For the record, my /etc/hosts file blocks the following, which
is likely a similar set that the wildcard line blocks.

$ grep -i doubleclick /etc/hosts
127.0.0.1 3ad.doubleclick.net
127.0.0.1 ad2.doubleclick.net
127.0.0.1 ad.3au.doubleclick.net
127.0.0.1 ad-ace.doubleclick.net
127.0.0.1 ad.ae.doubleclick.net
127.0.0.1 ad-apac.doubleclick.net
127.0.0.1 ad.ar.doubleclick.net
127.0.0.1 ad.at.doubleclick.net
127.0.0.1 ad.au.doubleclick.net
127.0.0.1 ad.be.doubleclick.net
127.0.0.1 ad.bg.doubleclick.net
127.0.0.1 ad.br.doubleclick.net
127.0.0.1 ad.ca.doubleclick.net
127.0.0.1 ad.ch.doubleclick.net
127.0.0.1 ad.cl.doubleclick.net
127.0.0.1 adclick.g.doubleclick.net
127.0.0.1 ad.cn.doubleclick.net
127.0.0.1 ad.de.doubleclick.net
127.0.0.1 ad.dk.doubleclick.net
127.0.0.1 ad.doubleclick.net
127.0.0.1 ad-emea.doubleclick.net
127.0.0.1 ad.es.doubleclick.net
127.0.0.1 ad.fi.doubleclick.net
127.0.0.1 ad.fr.doubleclick.net
127.0.0.1 ad-g.doubleclick.net
127.0.0.1 ad.gr.doubleclick.net
127.0.0.1 ad.hk.doubleclick.net
127.0.0.1 ad.hr.doubleclick.net
127.0.0.1 ad.hu.doubleclick.net
127.0.0.1 ad.ie.doubleclick.net
127.0.0.1 ad.in.doubleclick.net
127.0.0.1 ad.it.doubleclick.net
127.0.0.1 ad.jp.doubleclick.net
127.0.0.1 ad.kr.doubleclick.net
127.0.0.1 ad.mo.doubleclick.net
127.0.0.1 ad.my.doubleclick.net
127.0.0.1 ad.n2434.doubleclick.net
127.0.0.1 ad.nl.doubleclick.net
127.0.0.1 ad.no.doubleclick.net
127.0.0.1 ad.nz.doubleclick.net
127.0.0.1 ad.pl.doubleclick.net
127.0.0.1 ad.pt.doubleclick.net
127.0.0.1 ad.ro.doubleclick.net
127.0.0.1 ad.rs.doubleclick.net
127.0.0.1 ad.ru.doubleclick.net
127.0.0.1 ad.se.doubleclick.net
127.0.0.1 ad.sg.doubleclick.net
127.0.0.1 ad.si.doubleclick.net
127.0.0.1 ad.terra.doubleclick.net
127.0.0.1 ad.th.doubleclick.net
127.0.0.1 ad.tw.doubleclick.net
127.0.0.1 ad.uk.doubleclick.net
127.0.0.1 ad.us.doubleclick.net
127.0.0.1 ad.ve.doubleclick.net
127.0.0.1 ad-yt-bfp.doubleclick.net
127.0.0.1 ad.za.doubleclick.net
127.0.0.1 amn.doubleclick.net
127.0.0.1 anon.doubleclick.speedera.net
127.0.0.1 bid.g.doubleclick.net
127.0.0.1 cm.g.doubleclick.net
127.0.0.1 creatives.doubleclick.net
127.0.0.1 dfp.doubleclick.net
127.0.0.1 doubleclick.com
127.0.0.1 doubleclick.de
127.0.0.1 doubleclick.ne.jp
127.0.0.1 doubleclick.net
127.0.0.1 dp.g.doubleclick.net
127.0.0.1 ebaycn.doubleclick.net
127.0.0.1 ebaytw.doubleclick.net
127.0.0.1 exnjadgda1.doubleclick.net
127.0.0.1 exnjadgda2.doubleclick.net
127.0.0.1 exnjadgds1.doubleclick.net
127.0.0.1 exnjmdgda1.doubleclick.net
127.0.0.1 exnjmdgds1.doubleclick.net
127.0.0.1 feedads.g.doubleclick.net
127.0.0.1 fls.au.doubleclick.net
127.0.0.1 fls.doubleclick.net
127.0.0.1 fls.uk.doubleclick.net
127.0.0.1 gan.doubleclick.net
127.0.0.1 gd10.doubleclick.net
127.0.0.1 gd11.doubleclick.net
127.0.0.1 gd12.doubleclick.net
127.0.0.1 gd13.doubleclick.net
127.0.0.1 gd14.doubleclick.net
127.0.0.1 gd15.doubleclick.net
127.0.0.1 gd16.doubleclick.net
127.0.0.1 gd17.doubleclick.net
127.0.0.1 gd18.doubleclick.net
127.0.0.1 gd19.doubleclick.net
127.0.0.1 gd1.doubleclick.net
127.0.0.1 gd20.doubleclick.net
127.0.0.1 gd21.doubleclick.net
127.0.0.1 gd22.doubleclick.net
127.0.0.1 gd23.doubleclick.net
127.0.0.1 gd24.doubleclick.net
127.0.0.1 gd25.doubleclick.net
127.0.0.1 gd26.doubleclick.net
127.0.0.1 gd27.doubleclick.net
127.0.0.1 gd28.doubleclick.net
127.0.0.1 gd29.doubleclick.net
127.0.0.1 gd2.doubleclick.net
127.0.0.1 gd30.doubleclick.net
127.0.0.1 gd31.doubleclick.net
127.0.0.1 gd3.doubleclick.net
127.0.0.1 gd4.doubleclick.net
127.0.0.1 gd5.doubleclick.net
127.0.0.1 gd7.doubleclick.net
127.0.0.1 gd8.doubleclick.net
127.0.0.1 gd9.doubleclick.net
127.0.0.1 googleads2.g.doubleclick.net
127.0.0.1 googleads.g.doubleclick.net
127.0.0.1 ir.doubleclick.net
127.0.0.1 iv.doubleclick.net
127.0.0.1 ln.doubleclick.net
127.0.0.1 m1.doubleclick.net
127.0.0.1 m2.cn.doubleclick.net
127.0.0.1 m2.doubleclick.net
127.0.0.1 m3.doubleclick.net
127.0.0.1 m4.doubleclick.net
127.0.0.1 m5.doubleclick.net
127.0.0.1 m6.doubleclick.net
127.0.0.1 m7.doubleclick.net
127.0.0.1 m8.doubleclick.net
127.0.0.1 m9.doubleclick.net
127.0.0.1 marketing.doubleclickindustries.com
127.0.0.1 m.doubleclick.net
127.0.0.1 motifcdn2.doubleclick.net
127.0.0.1 motifcdn.doubleclick.net
127.0.0.1 n3302ad.doubleclick.net
127.0.0.1 n3349ad.doubleclick.net
127.0.0.1 n4052ad.doubleclick.net
127.0.0.1 n4061ad.doubleclick.net
127.0.0.1 n4061ad.hk.doubleclick.net
127.0.0.1 n4403ad.doubleclick.net
127.0.0.1 n479ad.doubleclick.net
127.0.0.1 optimize.doubleclick.net
127.0.0.1 paypalssl.doubleclick.net
127.0.0.1 pubads.g.doubleclick.net
127.0.0.1 rd.intl.doubleclick.net
127.0.0.1 s2.video.doubleclick.net
127.0.0.1 securepubads.g.doubleclick.net
127.0.0.1 static.doubleclick.net
127.0.0.1 stats.g.doubleclick.net
127.0.0.1 survey.g.doubleclick.net
127.0.0.1 twx.doubleclick.net
127.0.0.1 uunyadgda1.doubleclick.net
127.0.0.1 uunyadgds1.doubleclick.net
127.0.0.1 www2.doubleclick.com
127.0.0.1 www3.doubleclick.com
127.0.0.1 www3.doubleclick.net
127.0.0.1 www.doubleclick.com
127.0.0.1 www.doubleclick.net

Ned Turnbull

unread,
Aug 30, 2014, 5:01:19 PM8/30/14
to
On Sat, 30 Aug 2014 14:09:49 -0400, Mayayana wrote:

> servers can just keep changing the subdomain to thwart your
> HOSTS file. You might have entries for 200 Doubleclick subdomains,
> but you don't have an entry for the one they might create next week.

Very true.

I, for one, wish the HOSTS file format allowed for regular expressions.

Peter Köhlmann

unread,
Aug 30, 2014, 5:03:06 PM8/30/14
to
Ned Turnbull wrote:

> On Sat, 30 Aug 2014 15:56:19 +0200, J.O. Aho wrote:
>
>> Hosts file has the negative side, you need to copy it to each and every
>> machine you have, including portable devices
>
> Portable devices require root, unfortunately.

Which makes this idiotic method impractical there

> But, how can copying a text file to Windows, Linux, and Mac be harder
> than trying to get a huge number of programs to perform the same task
> to work on all these platforms?

That task is "slowing down each and every network connection"?
Brilliant, you found it

> Besides, the hosts file works for all protocols, whereas most of these
> other methods work only for some protocols.

Yes, it will slow down each connection, regardless of protocal

> There's no way there is any easier yet more complete & yet totally
> portable method than the hosts file method. Yes, there *are* other
> methods, but let's keep this thread to just the hosts file method.

Yes, a very "portable method" to build a completely unmaintainable file

Ned Turnbull

unread,
Aug 30, 2014, 5:06:15 PM8/30/14
to
On Sat, 30 Aug 2014 17:04:44 +0200, J.O. Aho wrote:

> Just a reminder, no one benefits from a such list and they will never do.

I don't understand.

Are you intimating no one benefits from the MVP HOSTS file?
http://winhelp2002.mvps.org/hosts.htm

Many sites must be recommending the HOSTS file for some reason
e.g., Lifehacker, TechRepublic, SomoneWhoCares, etc.
http://lifehacker.com/5817447/how-to-block-unwanted-ads-in-all-applications-and-speed-up-web-browsing-with-the-hosts-file
http://www.techrepublic.com/blog/windows-and-office/quick-tip-block-unwanted-sites-using-the-windows-hosts-file/
http://someonewhocares.org/hosts/

BTW, that last URL is yet another HOSTS file which I will
include in mine, but since it's already published, I'll let
you integrate it into yours.

NOTE: The effort is in culling out duplicates, which I do
simply by removing all extraneous lines, and then removing
extraneous characters in the good lines.

Ned Turnbull

unread,
Aug 30, 2014, 5:07:38 PM8/30/14
to
On Sat, 30 Aug 2014 09:46:58 -0400, Big Al wrote:

> How does one go about finding out all these bogus host
> addresses so you can build your own HOST file.

Hi Big Al,
Here's another one I just found by accident:
http://someonewhocares.org/hosts/

I will run my scripts on that one to see if it adds
any value to my existing hosts file (which is 2/3
MVP HOSTS and 1/3 other hosts).

Ned Turnbull

unread,
Aug 30, 2014, 5:21:29 PM8/30/14
to
On Sat, 30 Aug 2014 14:23:42 -0500, J² wrote:

> You need to verify that the additions you have put
> on your file over the years are still relevant. There
> is a program available for download that will check
> each and every one, and it takes hours, if not days,
> to check as many as you have. I'm sorry but I can't
> recall the name of this utility.

I suspect I can run a ping, and, to capture the
results in a script, to test them.

But, really, I have never typed in a URL that I *wanted*
to go to, that came back *accidentally* blocked by my
hosts file, so, it's not really a concern of mine.

Of course, *you* might have it as a concern, as I would
think you should, since you have no idea *how* these domains
got into "my" hosts file.

BTW, you can *skip* "my" hosts file, and just use the trusted
ones out there, of which MVP HOSTS is certainly one, but, I
just found this one just now, which I'm going to incorporate:
http://someonewhocares.org/hosts/

It has 10,250 entries in it, once I cleaned up all the comments.
Here is the cleaned up file before I tested it with mine:
http://pastebin.com/vhNMYa4x

Caver1

unread,
Aug 30, 2014, 5:25:02 PM8/30/14
to
On 08/30/2014 09:31 AM, Ned Turnbull wrote:
> On Sat, 30 Aug 2014 10:05:55 -0300, Shadow wrote:
>
>> The problem is when you seed it you will have to be online, so
>> that unique seeder will be your IP.
>
> Can I seed from Tor?
>

I don't think that's a good idea. It's probably just like installing a
program that you downloaded while Tor is still open. It destroys your
anonymity. Tor warns not to do that for that reason.

--
Caver1

Ned Turnbull

unread,
Aug 30, 2014, 5:27:17 PM8/30/14
to
On Sat, 30 Aug 2014 15:51:35 -0500, John Hasler wrote:

> I find that Privoxy plus NoScript blocks 99.9% of all advertising with
> no need for any configuration.

Do others concur that Privoxy plus NoScript is the panacea that solves
all but 0.01% of the obnoxious web sites?

If so, that's a winner, at least for one browser and one port.

Ned Turnbull

unread,
Aug 30, 2014, 5:30:39 PM8/30/14
to
On Sat, 30 Aug 2014 23:03:06 +0200, Peter Köhlmann wrote:

> That task is "slowing down each and every network connection"?
> Brilliant, you found it

I have noticed absolutely no slowdown, when I run a speedtest.net
with and without a hosts file.

I notice absolutely no slowdown when I access web pages, with
and without the hosts file.

In fact, things *speed* up, because far fewer background
sites are being connected to.

If the 127.0.0.1 syntax is "slowing" you down, the 0.0.0.0
synax on Win8 machines might help, but, this site says the
speed difference is minuscule.
http://someonewhocares.org/hosts/

Since I have a decade of experience with huge HOSTS files,
I wonder where you get your *facts* from, since you've
never even tried it, it seems.

Are you just making that up?

Ned Turnbull

unread,
Aug 30, 2014, 5:36:12 PM8/30/14
to
On Sat, 30 Aug 2014 21:06:15 +0000, Ned Turnbull wrote:

> NOTE: The effort is in culling out duplicates, which I do
> simply by removing all extraneous lines, and then removing
> extraneous characters in the good lines.

Since the only work in merging hosts file is in culling out
the duplicates, I've done that work for you, for this site:
http://someonewhocares.org/hosts/

Here is the 10,250 line file, cleaned up, from that site:
http://pastebin.com/vhNMYa4x

When I compare that to yesterday's MVP HOSTS file, I see:
13626 domains in HOSTS
10250 domains in someonewhocares
11761 domains in HOSTS that are not in someonewhocares

Based on a quick output from this simple command:
$ comm -2 -3 <(sort file1) <(sort file2) > file3
$ wc -l file*

So, I'll make a *new* combined hosts file for myself.
If anyone wants it, let me know.

Ned Turnbull

unread,
Aug 30, 2014, 5:42:34 PM8/30/14
to
On Sat, 30 Aug 2014 21:07:38 +0000, Ned Turnbull wrote:

> Here's another one I just found by accident:
> http://someonewhocares.org/hosts/

I combined that recently found hosts file with mine,
and, after removing duplicates, I found that my hosts
file grew by 232 hosts (from 23,735 hosts to 23,967 hosts).

So, with about 30 seconds' effort, we are able to add over
two hundred additional hosts that we will never see on
any machine in our networks.

Ned Turnbull

unread,
Aug 30, 2014, 5:47:33 PM8/30/14
to
On Sat, 30 Aug 2014 17:25:02 -0400, Caver1 wrote:

> I don't think that's a good idea. It's probably just like installing a
> program that you downloaded while Tor is still open. It destroys your
> anonymity. Tor warns not to do that for that reason.

That's what I was worried about. Thanks.

Since a hosts file is a TEXT file, the pastebin.net method
seems to work, as long as I keep it below the 500 KB limit.

~BD~

unread,
Aug 30, 2014, 5:48:07 PM8/30/14
to
Ned

You haven't mentioned the Hosts file incorporated in the Malwarebytes
anti-malware software. Have you actually reviewed it?

You can find same here: http://hosts-file.net/?s=Download

HTH

Dave

--
First they ignore you, then they laugh at you, then they fight you, then
you win.

Ned Turnbull

unread,
Aug 30, 2014, 5:48:53 PM8/30/14
to
On Sat, 30 Aug 2014 21:27:17 +0000, Ned Turnbull wrote:

> Do others concur that Privoxy plus NoScript is the panacea that solves
> all but 0.01% of the obnoxious web sites?

OoOOps. I was off by one decimal place, but the question remains
whether Privoxy & NoScript solve almost all the problems that the
hosts file attempts to solve on the three major PC platforms for
all the browsers one would use, and for all the ports.

~BD~

unread,
Aug 30, 2014, 5:55:08 PM8/30/14
to
I should mention the the guy preparing this Hosts file also hosts the
web site of Dustin Cook's BugHunter software. Verify here:-

http://mysteryfcm.co.uk/?mode=Contact

Ned Turnbull

unread,
Aug 30, 2014, 6:14:49 PM8/30/14
to
On Sat, 30 Aug 2014 22:48:07 +0100, ~BD~ wrote:

> You haven't mentioned the Hosts file incorporated in the Malwarebytes
> anti-malware software. Have you actually reviewed it?
>
> You can find same here: http://hosts-file.net/?s=Download

Hi Dave,

I was not aware of that hosts-file.net hosts file!
I was only using these hosts files:
a) Mine
b) Plus MVP HOSTS (http://winhelp2002.mvps.org/hosts.htm)
c) Plus someonewhocares (http://someonewhocares.org/hosts/)

Thanks for helping out!

I had just added 232 unique lines to my current hosts file based on
what I accidentally found at http://someonewhocares.org/hosts/ so,
my current hosts file contained 23,967 unique hosts.

That file from http://hosts-file.net/?s=Download, once cleaned up
and extraneous ASCII text removed, contains a whopping 901,983
unique hosts (after I removed extraneous spaces, tabs, & comments).

Renaming my hosts lines "file1" and this malwarebytes hosts "file2"
and running the sugested commmand gives me a surprising result:
$ comm -2 -3 <(sort file1) <(sort file2) > file3
$ wc -l *
23967 file1 (my hosts file unique lines)
901983 file2 (the malwarebytes hosts file unique lines)
18451 file3
944401 total

So, there are 18,451 hosts in my /etc/hosts file that are *not* one
of the malwarebytes 901,983 hosts.

Likewise, renaming the malwarebytes hosts file as "file1" and my hosts
file as "file2", we find almost ninety thousand hosts listed in the
malwarebytes hosts file which are not (yet) in "my" hosts file.
901983 file1 (the malwarebytes hosts file unique lines)
23967 file2 (my hosts file unique lines)
896467 file3
1822417 total

Whew! That's a huge difference!

To test out a million-line hosts file, now "my" hosts file has all
917,434 unique hosts. With almost a million hosts, we'll see what
happens (if anything) to my Linux network performance.

If you can suggest a procedural test, to run with and without the
hosts file in place, that would be helpful to see what the performance
impact (if any) is.

Ned Turnbull

unread,
Aug 30, 2014, 6:17:37 PM8/30/14
to
On Sat, 30 Aug 2014 22:14:49 +0000, Ned Turnbull wrote:

> ninety thousand

OOOOps. nine hundred thousand!

Ned Turnbull

unread,
Aug 30, 2014, 6:22:13 PM8/30/14
to
On Sat, 30 Aug 2014 22:55:08 +0100, ~BD~ wrote:

> I should mention the the guy preparing this Hosts file also hosts the
> web site of Dustin Cook's BugHunter software. Verify here:-

Thanks.

And, I should probably mention that I grep'd out anything that
didn't start with 127.0.0.1, to ensure there were no shenanigans
going on with the redirects, so, at least the redirect is kosher
on all million lines of that astoundingly humongous hosts file!

I was surprised that "my" hosts file had about eighteen thousand
hosts that were *not* in that million-line hosts file, but, now
I'm using that million-line hosts file, and we'll see how it works
out.

I suspect I'll not be seeing an ad, or annoying popup, or locked-up
web site soon! :)

~BD~

unread,
Aug 30, 2014, 6:26:15 PM8/30/14
to
Glad to be of help, Ned.

Good luck! :-)

Mayayana

unread,
Aug 30, 2014, 6:32:52 PM8/30/14
to
| Am I confused, or are you confused?
|

I copied those from your link. And you're saying that
the MVPS people are resonsible for the tomshardware
listings, but you're the one who put those into your
HOSTS file!

What I'm trying to get at is that 10,000 entries in
a HOSTS file is no better than 5 if you don't really
know what they are.


Thad Floryan

unread,
Aug 30, 2014, 6:32:13 PM8/30/14
to
On 8/30/2014 3:14 PM, Ned Turnbull wrote:
> [...]
> To test out a million-line hosts file, now "my" hosts file has all
> 917,434 unique hosts. With almost a million hosts, we'll see what
> happens (if anything) to my Linux network performance.
>
> If you can suggest a procedural test, to run with and without the
> hosts file in place, that would be helpful to see what the performance
> impact (if any) is.

It seems you are unaware that there can be [at least] 8 entries per line
meaning the resultant file will be smaller and have fewer lines.

Using the first 16 entries from the 'file1' you previously posted:

127.0.0.1 005.free-counter.co.uk 006.free-counter.co.uk 006.fre...
127.0.0.1 011707160008.c.mystat-in.net 0427d7.se 05tz2e9.com 061...

A script can easily do that for you.

Thad

J.O. Aho

unread,
Aug 30, 2014, 6:36:06 PM8/30/14
to
On 30/08/14 19:29, Ned Turnbull wrote:
> On Sat, 30 Aug 2014 17:04:44 +0200, J.O. Aho wrote:
>
>> You also make your internet slower, not just for browsing, but for all
>> other activity too.
>
> How does NOT visiting a web site, especially since these are all
> obnoxious web sites, make the browsing slower?

As it has to make the look up, regardless where it tries to connect and
it applies to all protocols, not just http/https but also
smtp/pop/imap/ftp/ssh and so on.


Please keep the reference to the post you reply (not to your original
one), makes it a lot easier for people to see what post you are replying to.

--

//Aho

Mayayana

unread,
Aug 30, 2014, 6:43:38 PM8/30/14
to
| Heck, you don't even need $10 a month. I found a free one, limited
| space but I'm sure 2 gigs maybe, but I only host a web page. I like
| playing with html code, so this gives me an outlet.

2 Gigs is a lot, but if it's free it's probably got ads.
And 2 GB doesn't mean much if they don't let you host
binaries. (Many freebie sites don't. They're running on
a tight budget and depend on getting lots of customers
who put up a vanity webpage and then forget about it.
They all offer vast hosting and vast traffic, but it's
not always above-board.)

For a few dollars you can have a real site that you
control. I spend $8.95/month with a small, honest
and dependable company called futurequest. I can
get my web logs; they have real email; I control my
domain. Many hosts don't provide real email. They'll
often subcontract it to gmail or some other sleazeball
operation. In some cases there may not be any email
at all. There can be lots of little differences between
very cheap sites and sites for $10+-.

Another host that I discovered when doing a site for
someone else is iPower. They seem to be OK. I think
they're also about $10/month. But it's not easy to tell,
without some research, which are the honest companies
and which are depending on you not knowing what you're
doing.


J.O. Aho

unread,
Aug 30, 2014, 6:38:23 PM8/30/14
to
On 30/08/14 19:29, Ned Turnbull wrote:
> On Sat, 30 Aug 2014 17:04:44 +0200, J.O. Aho wrote:
>
>> If you want to block sites, then use proper tools like privoxy or those
>> add blocker plug-ins, as this way you can get rid of the advertisement
>> without blocking legitimate content and you aren't slowing down for
>> other protocols.
>
> I knew this would come up, and you're the first, I think, to bring it
> up, but this thread isn't about the many (very many) other methods of
> blocking unwanted domains. :)
>

Just for it's not about how to block advertisement, it shouldn't be
about a bad way to block sites, I know the hosts file trick is really
common in the microsoft sphere, but please do not try to get people to
use the wrong way to block things.

--

//Aho

J.O. Aho

unread,
Aug 30, 2014, 6:41:05 PM8/30/14
to
On 30/08/14 19:31, Ned Turnbull wrote:
> On Sat, 30 Aug 2014 15:56:19 +0200, J.O. Aho wrote:
>
>> the hosts file was never intended for this amount of hosts and this
>> makes things slower...
>
> On Linux, it does not seem to have any deleterious effect on speed.
>
> In fact, one could argue the machine is *faster* since it doesn't
> waste time visiting any number of a hosts of common tracking sites
> and advertisement sites.
>
> But, I haven't tested the speed (simply because it hasn't been an
> issue, and I have as large a hosts file as anyone here).
>

Using proper tools, you don't even make the connection to the localhost,
which you need to do with your hosts file trick, so you actually get the
parsing through the hosts file, plus the connection attempt to your
localhost, which can for some people (specifically web developers) mean
a lot extra traffic to their test environment or someones at home
owncloud setup.

--

//Aho

J.O. Aho

unread,
Aug 30, 2014, 6:45:30 PM8/30/14
to
On 30/08/14 23:30, Ned Turnbull wrote:
> On Sat, 30 Aug 2014 23:03:06 +0200, Peter Köhlmann wrote:
>
>> That task is "slowing down each and every network connection"?
>> Brilliant, you found it
>
> I have noticed absolutely no slowdown, when I run a speedtest.net
> with and without a hosts file.

It's not about the speed you get to a already connected site, it's about
the extra time it take to connect to the site, the time waisted on
looking through the hosts file.


> Since I have a decade of experience with huge HOSTS files,
> I wonder where you get your *facts* from, since you've
> never even tried it, it seems.

I have work related experiences where some customers got into issues
where they even timed out on some connections.

Sure you have two options, do it the proper way or do it the microsoft
way...

--

//Aho

J�@localhost.invalid

unread,
Aug 30, 2014, 6:56:11 PM8/30/14
to
On 8/30/2014 5:22 PM, Ned Turnbull wrote:
[...]
>
> I was surprised that "my" hosts file had about eighteen thousand
> hosts that were *not* in that million-line hosts file, but, now
> I'm using that million-line hosts file, and we'll see how it works
> out.
>

Ned, I appreciate you sharing so many hosts that you
have personally recorded over years to use in a hosts file.
That's a lot of work taking down the addresses of so many.
But-
There is one problem that I don't think you've fully recognized
the great importance of. I don't think you realize just how fast
an ad server, or spammy link, etc., vanish, and are no longer
a threat or problem. An ad server up and running one week
can be gone like the wind the very next week. What this means
is that thousands of those 18,000 unique hosts you mentioned
are not even needed on a hosts file.
You must come up with a way to periodically check each and
every host in your file to see if they are even active anymore.




Thad Floryan

unread,
Aug 30, 2014, 7:05:47 PM8/30/14
to
On 8/30/2014 3:32 PM, Thad Floryan wrote:
> [...]
> It seems you are unaware that there can be [at least] 8 entries per line
> meaning the resultant file will be smaller and have fewer lines.
>
> Using the first 16 entries from the 'file1' you previously posted:
>
> 127.0.0.1 005.free-counter.co.uk 006.free-counter.co.uk 006.fre...
> 127.0.0.1 011707160008.c.mystat-in.net 0427d7.se 05tz2e9.com 061...
>
> A script can easily do that for you.

Here's something interesting -- note the improvement using 0.0.0.0
instead of 127.0.0.1. FWIW, thadlabs.com is about 13 miles away
in Fremont CA; my home office is in Los Altos CA (Silicon Valley).

procyon bash 3056/3064> ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.156 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.112 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.113 ms
64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.111 ms
64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.114 ms
^C
--- 127.0.0.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4521ms
rtt min/avg/max/mdev = 0.111/0.121/0.156/0.018 ms
procyon bash 3056/3064> ping 0.0.0.0
PING 0.0.0.0 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.106 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.082 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.091 ms
64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.093 ms
64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.089 ms
^C
--- 0.0.0.0 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4536ms
rtt min/avg/max/mdev = 0.082/0.092/0.106/0.009 ms
procyon bash 3056/3064> ping thadlabs.com
PING thadlabs.com (192.220.75.50) 56(84) bytes of data.
64 bytes from thadlabs.com (192.220.75.50): icmp_seq=1 ttl=53 time=14.8 ms
64 bytes from thadlabs.com (192.220.75.50): icmp_seq=2 ttl=53 time=14.4 ms
64 bytes from thadlabs.com (192.220.75.50): icmp_seq=3 ttl=53 time=14.4 ms
64 bytes from thadlabs.com (192.220.75.50): icmp_seq=4 ttl=53 time=15.0 ms
^C
--- thadlabs.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3295ms
rtt min/avg/max/mdev = 14.418/14.716/15.079/0.274 ms
procyon bash 3056/3064> emacs
procyon bash 3056/3064> ll /etc/hosts
-rw-r--r-- 1 root root 668518 Jul 20 21:31 /etc/hosts
procyon bash 3056/3064> wc -l < /etc/hosts
22039


Thad

Paul

unread,
Aug 30, 2014, 7:22:53 PM8/30/14
to
Check the References: line in the post you are replying
to, to understand how threading is broken. I don't know
what is doing it, but this has been happening for a couple
weeks now.

The References: line should be one long line of text,
and some server is wrapping the header lines, adding tabs
and other useless things. I see that mixmin is involved
again, but my skill at reading headers means I can't be sure
that is the server that's doing it.

The Subject: line in the posts is broken too, with a
hard return to wrap the line at some point. Clients
seem to be ignoring this, but it is visible on
the Howardknight server. Pop in a few MIDs, to see
the damage.

http://al.howardknight.net/

Paul

Ned Turnbull

unread,
Aug 30, 2014, 11:27:17 PM8/30/14
to
On Sat, 30 Aug 2014 17:56:11 -0500, J² wrote:

> You must come up with a way to periodically check each and
> every host in your file to see if they are even active anymore.

Agreed.
A script "should" be able to test this automagically though.

Ned Turnbull

unread,
Aug 30, 2014, 11:40:43 PM8/30/14
to
On Sat, 30 Aug 2014 10:09:58 -0300, Shadow wrote:

> Hey !!! That's MY hosts file. Thief !! :)

Here is a list of updated hosts files ...
https://github.com/StevenBlack/hosts

MVPs.org Hosts file
http://winhelp2002.mvps.org/hosts.htm, updated monthly, or thereabouts.
Dan Pollock
http://someonewhocares.org/hosts/ updated regularly.
Malware Domain List
http://www.malwaredomainlist.com/, updated regularly.
Peter Lowe
http://pgl.yoyo.org/adservers/, updated regularly.
hpHosts
http://hosts-file.net/, updated regularly

Ned Turnbull

unread,
Aug 31, 2014, 12:15:19 AM8/31/14
to
On Fri, 29 Aug 2014 23:46:37 +0000, Ned Turnbull wrote:

> I have a fantastic hosts file, improved over years

As an aside, a friend *thought* he was contacting Toshiba
Technical Support based on this official-looking web page:
http:// toshibatechssupport (dot) com
But, it was the classic Indian Technical Support Scam, so,
I am adding that web page to my hosts file pronto!

Jasen Betts

unread,
Aug 31, 2014, 12:19:34 AM8/31/14
to
On 2014-08-30, Big Al <Big...@invalid.com> wrote:
> Ned Turnbull wrote on 8/29/2014 7:46 PM:
>> Where can I post a 25,000 line text file for others to benefit?
>>
>> I have a fantastic hosts file, improved over years, always adding
>> the MVP hosts file to it, and adding about a thousand obnoxious
>> domains that I've run into such that I almost never see a valid
>> popup browser (they pop up, but they are all unfound). I never
>> see in-page ads either.
><snip>
>
> I'm going to ask, but let's not dwell on this, but in the vein of this
> subject: How does one go about finding out all these bogus host
> addresses so you can build your own HOST file. Or even, other than
> the one link I did see, are there starter lists available?

running a tcpdump, or wireshark looking at traffic on port 53 will
tell you the names of all the hosts your computer is contacting
you then need to decide which ones you want to ban

> This all sounds like a great idea, and everyone SHOULD be doing it.

it's a great way to break the internet, you have to be careful not to
break off any important bits. fortunately any breakage you cause is
localised to your computer.

> You've all made a great case between all the posts. I just can't
> figure out how to get the links. I do clear cookies from my browser,
> and I guess those would be a good start. Things like click.net,
> adtracket.com etc. You can be sure when the names are that clear.

yeah, I don't think the internet needs _them_.

--
umop apisdn


--- news://freenews.netfront.net/ - complaints: ne...@netfront.net ---

Jasen Betts

unread,
Aug 31, 2014, 1:22:25 AM8/31/14
to
On 2014-08-30, Ned Turnbull <NedTu...@example.com> wrote:
> On Sat, 30 Aug 2014 15:56:19 +0200, J.O. Aho wrote:
>
>> the hosts file was never intended for this amount of hosts and this
>> makes things slower...
>
> On Linux, it does not seem to have any deleterious effect on speed.

yeah, I added 250000 entries to my hosts file and it has added about
52 milliseconds to host lookup times.

my test was :

time ping -c1 google.com

I took the best 3 of 7

I used this to add the entries,

d=google.com ; for ((x=0 ; x<10000 ; ++x )) ; do echo 127.0.0.1 \
a$x.$d b$x.$d c$x.$d d$x.$d e$x.$d f$x.$d g$x.$d h$x.$d i$x.$d j$x.$d \
k$x.$d l$x.$d m$x.$d n$x.$d o$x.$d p$x.$d q$x.$d r$x.$d s$x.$d t$x.$d \
u$x.$d v$x.$d w$x.$d x$x.$d y$x.$d ; done >> /etc/hosts

M.L.

unread,
Aug 31, 2014, 3:46:50 AM8/31/14
to


>> I assume that " resolve this issue" refers to changing from 127.0.0.1
>> to 0.0.0.0
>
>There's also a dhcp service that needs to be turned off,
>which, on very long hosts files, would otherwise cause a problem.
>
>This is a bug in Windows that is about 20 years old, so, they
>"may" have finally fixed it by how (I don't know as I no longer
>boot to Windows ever since they killed XP I went to Linux).

If you're referring to the Windows DNS service, there is debate as to
whether that service needs to be disabled with a large HOST file. I
allow it to run and notice no difference in Win8.

AFAIK, DHCP needs to be enabled to provide network IPs. I certainly
got nowhere when it was disabled.

M.L.

unread,
Aug 31, 2014, 4:17:26 AM8/31/14
to

On Sat, 30 Aug 2014 21:30:39 +0000 (UTC), Ned Turnbull
<NedTu...@example.com> wrote:

>On Sat, 30 Aug 2014 23:03:06 +0200, Peter K�hlmann wrote:
>
>> That task is "slowing down each and every network connection"?
>> Brilliant, you found it
>
>I have noticed absolutely no slowdown, when I run a speedtest.net
>with and without a hosts file.
>
>I notice absolutely no slowdown when I access web pages, with
>and without the hosts file.
>
>In fact, things *speed* up, because far fewer background
>sites are being connected to.
>
>If the 127.0.0.1 syntax is "slowing" you down, the 0.0.0.0
>synax on Win8 machines might help, but, this site says the
>speed difference is minuscule.
>http://someonewhocares.org/hosts/

http://www.dslreports.com/forum/r24621780-hosts-127001-vs-0000
<quote>
Using 0.0.0.0 instead of 127.0.0.1 is indeed faster because you don't
have to wait for a timeout. It also does not interfere if you are
running a web server on the local PC.
</quote>

http://www.dslreports.com/forum/remark,16790135
<quote>
The browser doesn't know you don't have a server waiting for
connections on localhost - for instance proxomitron, or even a server
daemon web interface. So it has to try and make a connection, and
depending on your setup, wait for a time out (especially if you use a
SPF that "stealths" ports.

0 or 0.0.0.0 is not a valid IP address - it just errors the connection
immediately unless the program is really badly written without even
basic error checking. As others have said, it likely doesn't even
spend time creating the TCP connection, much less a packet to send...
</quote>

~BD~

unread,
Aug 31, 2014, 5:31:48 AM8/31/14
to
Did you actually call the freephone number, Ned?

1-866-283-6613

Did your friend?

p-0''0-h the cat (ES)

unread,
Aug 31, 2014, 6:34:41 AM8/31/14
to
0.0.0.0/0 is a network address in CIDR notation. It's used in routing to
say send packets whose destination address either by network or host isn't
described in the routing table to the default gateway. You cannot assign
any network address to a host. That's just gibberish. The IP stack would
just drop it if it even got that far.

--
p-0.0-h the cat

Internet Terrorist, Mass sock puppeteer, Agent provocateur, Gutter rat,
Devil incarnate, Linux user#666, BaStarD hacker, Resident evil, Monkey Boy,
Certifiable criminal, Spineless cowardly scum, textbook Psychopath,
the SCOURGE, l33t p00h d3 tr0ll, p00h == lam3r, p00h == tr0ll, troll
inf�me, the OVERCAT [The BEARPAIR are dead, and we are its
murderers], lowlife troll, shyster [pending approval by STATE_TERROR],
cripple, sociopath, kook, smug prick, smartarse, arsehole, moron, idiot,
imbecile, snittish scumbag, liar, total ******* retard, and shill.

Honorary SHYSTER and FRAUD awarded for services to Haberdashery.
By Appointment to God Frank-Lin.

Signature integrity check
md5 Checksum: be0b2a8c486d83ce7db9a459b26c4896

Mayayana

unread,
Aug 31, 2014, 8:40:12 AM8/31/14
to
| >There's also a dhcp service that needs to be turned off,
| >which, on very long hosts files, would otherwise cause a problem.
| >
| >This is a bug in Windows that is about 20 years old, so, they
| >"may" have finally fixed it by how (I don't know as I no longer
| >boot to Windows ever since they killed XP I went to Linux).
|
| If you're referring to the Windows DNS service, there is debate as to
| whether that service needs to be disabled with a large HOST file. I
| allow it to run and notice no difference in Win8.
|
| AFAIK, DHCP needs to be enabled to provide network IPs. I certainly
| got nowhere when it was disabled.

I don't know about problems with DHCP and HOSTS file,
but for most people neither DHCP nor DNS service is
necessary. I'm not sure exactly what DNS service is. It
seems to be for use only within an intranet. DHCP is needed
if your machine is being assigned a dynamic IP address.

I have both services disabled and use a fixed IP address
for the router. There's no problem with leaving DHCP enabled,
as far as I know, but I found that it was one of the few things
running under svchost that actually needs to go outbound.
Svchost, by wrapping numerous services, is a veiled security
and privacy risk. By using a fixed IP I can disable DHCP and
that allows me to block svchost through the firewall.


Ned Turnbull

unread,
Aug 31, 2014, 8:37:39 AM8/31/14
to
On Sun, 31 Aug 2014 02:46:50 -0500, M.L. wrote:

> If you're referring to the Windows DNS service,

Ah, yes. My bad. That was it.

Ned Turnbull

unread,
Aug 31, 2014, 8:38:52 AM8/31/14
to
On Sun, 31 Aug 2014 03:17:26 -0500, M.L. wrote:

> 0 or 0.0.0.0 is not a valid IP address - it just errors the connection

Does the 0.0.0.0 work on Linux also?
Or just Windows?

Ned Turnbull

unread,
Aug 31, 2014, 8:47:14 AM8/31/14
to
On Sun, 31 Aug 2014 10:31:48 +0100, ~BD~ wrote:

> Did you actually call the freephone number, Ned?
> 1-866-283-6613
> Did your friend?

Yes. My friend called the support number.

He was fooled, for a while, and said that the oddities added up
only slowly. He only thought of me, belatedly, but he thought I
would be interested in what happened.

For example, "Sean" answered the phone right away, both times,
and he got the same guy the second time. Also, "Sean" moved
icons around the desktop, organizing them for some odd reason.

In addition, "Sean" had him look at the event viewer, but, since
it was a brand new laptop, there weren't many events that he could
warn the owner about. Yet, the kill was that "Sean" ran a "dir /s"
and then showed an "error 404" and that the modem was infected with
a virus.

At that point, my friend pulled the Ethernet cable out of the
laptop (he was debugging wireless connectivity issues which is
why he called Toshiba Technical Support).

Anyway, he's running scans (Windows 8) and nothing has been
found yet. He asked me if anyone could install something without
him seeing it, since he watched the entire session. I don't know
if they can remotely log in, without you knowing it. Do you?

Ned Turnbull

unread,
Aug 31, 2014, 8:47:43 AM8/31/14
to
On Sun, 31 Aug 2014 10:31:48 +0100, ~BD~ wrote:

> Did you actually call the freephone number, Ned?

Nope. I didn't bother.

Message has been deleted
Message has been deleted

Ned Turnbull

unread,
Aug 31, 2014, 10:25:54 AM8/31/14
to
On Sun, 31 Aug 2014 09:19:29 -0500, G. Morgan wrote:

> I see no improvement for a general failure:

I'm on Linux, so I just tried both (with a large hosts file)
and I saw essentially the same round trip times for both.

Average round-trip time for 127.0.0.1 was 0.037 ms
Average round-trip time for 0.0.0.0 was 0.038 ms

$ ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.036 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.036 ms
64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.043 ms
64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.041 ms
^C
--- 127.0.0.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3997ms
rtt min/avg/max/mdev = 0.031/0.037/0.043/0.006 ms

$ ping 0.0.0.0
PING 0.0.0.0 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.035 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.041 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.039 ms
64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.040 ms
64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.038 ms
64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.040 ms
^C
--- 0.0.0.0 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 4999ms
rtt min/avg/max/mdev = 0.035/0.038/0.041/0.008 ms

Jasen Betts

unread,
Aug 31, 2014, 1:23:59 AM8/31/14
to
On 2014-08-30, Ned Turnbull <NedTu...@example.com> wrote:
> On Sat, 30 Aug 2014 15:56:19 +0200, J.O. Aho wrote:
>
>> Hosts file has the negative side, you need to copy it to each and every
>> machine you have, including portable devices
>
> Portable devices require root, unfortunately.
>
> But, how can copying a text file to Windows, Linux, and Mac be harder
> than trying to get a huge number of programs to perform the same task
> to work on all these platforms?

opendns
It is loading more messages.
0 new messages