Alan Poulter
al...@poulter.demon.co.uk
No problems.
Mailed as well, although I shouldn't...
--
Peter
What links do you get when you go to
http://web.archive.org/web/*/http://www.demon.net ?
I get links to, for example,
http://iwfwebfilter.thus.net/web/20080113145506/http://www.demon.net/index.html
rather than the proper link to
http://web.archive.org/web/20080113145506/http://www.demon.net/index.html
which itself is full of broken iwfwebfilter.thus.net URLs.
--
James Coupe
PGP Key: 0x5D623D5D YOU ARE IN ERROR.
EBD690ECD7A1FB457CA2 NO-ONE IS SCREAMING.
13D7E668C3695D623D5D THANK YOU FOR YOUR COOPERATION.
> What links do you get when you go to
> http://web.archive.org/web/*/http://www.demon.net ?
I don't really know. I get Waybackmachine, with 864 results; urls.
But I've never tried this before and don't know what they're for.
Last link is Jan 13 2008. 'The requested URL
/web/20080113145506/http://www.demon.net/index.html was not found on
this server.'
--
Peter
They're links to automatically archived copies of websites, recording
changes over time.
>Last link is Jan 13 2008. 'The requested URL
>/web/20080113145506/http://www.demon.net/index.html was not found on
>this server.'
Look at the URL in your address bar:
http://iwfwebfilter.thus.net/web/20080113145506/http://www.demon.net/index.html
^^^^^^^^^^^^^^^^^^^^^
i.e. THUS is intercepting access to archive.org for some reason,
presumably because one website ever has had an IWF warning at some point
in history, or something.
Without THUS's fiddling, you would go to:
http://web.archive.org/web/20080113145506/http://www.demon.net/index.html
Which works.
I'm seeing the same thing.
It looks like Demon are messing with our HTTP connections, replacing the
text "web.archive.org" with "iwfwebfilter.thus.net"
Googling for iwfwebfilter.thus.net is interesting.
>i.e. THUS is intercepting access to archive.org for some reason,
Oh joy ... yet more "unintended collateral damage" from the IWF.
Didn't they actually learn ANYTHING from their Wikipedia disaster just
before Christmas?
--
Paul Terry
>It looks like Demon are messing with our HTTP connections, replacing the
>text "web.archive.org" with "iwfwebfilter.thus.net"
http://www.demon.net/helpdesk/technicallibrary/faq/iwf_web_filter/
--
Paul Terry
That's clever - how did they do that?
--
Andy Taylor [Editor, Austrian Philatelic Society].
Visit <URL:http://www.austrianphilately.com>
The irony is that www.iwf.org.uk and www.thus.net, those
known havens of child porn, are both blocked via Archive.org ;-)
I have emailed TheRegister the story but no interest so far :-(
Alan
I get the irony, what I am not sure about is why archive.org is
important. It could be I misunderstand the importance of your message,
I remain puzzled as to why archive.org should be the measure.
The IWF have done a number of embarrassing things that they probably
don't want everyone to know about. thus, on the other hand is a company
that should expect itself to be examined by investors, etc.
Are you joining thus and the IWF together in a significant way?
My understanding is that they are separate.
--
Wm...
Reply-To: address valid for at least 7 days
> Tue, 13 Jan 2009 01:59:58
> <Xns9B925146386Bal...@216.196.109.144> demon.service Alan
> Poulter <al...@poulter.demon.co.uk>
>
>>
>>Paul Terry <ne...@nospam.demon.co.uk> wrote in
>>news:wbLdGhBe...@musonix.demon.co.uk:
>>
>>> In message <4D9q4yKH...@gratiano.zephyr.org.uk>, James Coupe
>>> <ja...@zephyr.org.uk> writes
>>>
>>>>i.e. THUS is intercepting access to archive.org for some reason,
>>>
>>> Oh joy ... yet more "unintended collateral damage" from the IWF.
>>>
>>> Didn't they actually learn ANYTHING from their Wikipedia disaster just
>>> before Christmas?
>>
>>The irony is that www.iwf.org.uk and www.thus.net, those
>>known havens of child porn, are both blocked via Archive.org ;-)
>>I have emailed TheRegister the story but no interest so far :-(
>
> I get the irony, what I am not sure about is why archive.org is
> important. It could be I misunderstand the importance of your message,
> I remain puzzled as to why archive.org should be the measure.
Archive.org I find to be a very useful site and I am not alone. According
to alexa.com it is ranked 408th in the world for traffic. It spiders sites
according to set policies in order to preserve them and it will remove
sites if notified. Therefore it is extremely unlikely to be a child porn
haven or have anything that the IWF should worry about.
> The IWF have done a number of embarrassing things that they probably
> don't want everyone to know about. thus, on the other hand is a company
> that should expect itself to be examined by investors, etc.
>
> Are you joining thus and the IWF together in a significant way?
>
> My understanding is that they are separate.
That is my understanding as well. Since I know other UK ISPs have
not blocked Archive.org (I can access it via my mobile ISP and from work)
then I can only assume someone at Thus has been stupid enough to use Thus's
IWF filter to block it. It makes you wonder how much coordination there is
between the IWF and UK ISPs. Are other ISPs operating rogue blocks under
IWF auspices? Are some ISPs not blocking sites they should?
Alan
>That is my understanding as well. Since I know other UK ISPs have
>not blocked Archive.org (I can access it via my mobile ISP and from work)
>then I can only assume someone at Thus has been stupid enough to use Thus's
>IWF filter to block it.
That is a leap of imagination I cannot make. demon have, in the past,
been over zealous about IWF matters but I don't think they really don't
want people to see what they had to say. Could it be archive.org is in
error?
> It makes you wonder how much coordination there is
>between the IWF and UK ISPs. Are other ISPs operating rogue blocks under
>IWF auspices? Are some ISPs not blocking sites they should?
Hmmmn. Have you considered paranoia? I am not suggesting you are
paranoid so much as suggesting archive.org may have a mental block.
I can access the archive.org homepage. What's being blocked?
==== j a c k at c a m p i n . m e . u k === <http://www.campin.me.uk> ====
Jack Campin, 11 Third St, Newtongrange EH22 4PU, Scotland == mob 07800 739 557
CD-ROMs and free stuff: Scottish music, food intolerance, and Mac logic fonts
>I can access the archive.org homepage. What's being blocked?
In the slot at the top of the page (labelled Wayback Machine) enter:
http://www.demon.net and click "take me back"
You will see a list of the archived front pages of Demon's website.
Click on any of the links, and you will see that they have been rendered
useless because Demon's iwfwebfilter has wrecked the URL by adding the
following:
http://iwfwebfilter.thus.net/web/ ..........
Repeat for any URL you like (e.g. bbc.co.uk)
The site has been rendered unusable to Demon customers for weeks, but
nobody seems to be bothered to fix it.
Personally, I don't think it's Web Archive's fault that Demon haven't
got their IWF filters properly sorted.
--
Paul Terry
>Could it be archive.org is in error?
I'm not sure how. Demon's IWF web filter is rewriting every link on
their site - I don't think Demon intend to block every WWW site that has
ever been archived by the company, but that's what they have been doing
for quite some weeks.
Try going to http://www.archive.org/index.php and then entering
www.bbc.co.uk in the "Wayback Machine". Click on any of the archived
links, and instead of seeing the BBC's frontpage you will see "Not
found" because Demon's web filter has added:
http://iwfwebfilter.thus.net/web/ ....
to the front of the URL.
We complained to the IWF today, and they say they are not responsible:
"I can confirm that there is no entry for webarchive.org ... on the IWF
URL list or reported to the IWF and therefore I am unable to take your
complaint further".
I guess I could complain to archive.org, but they are almost certain to
say that it's not their fault that Demon is rewriting the URL (without
the authority of the IWF it would now seem).
--
Paul Terry
>In message <$8gZmyGUyHbJFwzr@[127.0.0.1]>, Wm...
><tcn...@blackhole.do-not-spam.me.uk> writes
>
>>Could it be archive.org is in error?
>
>I'm not sure how. Demon's IWF web filter is rewriting every link on
>their site - I don't think Demon intend to block every WWW site that has
>ever been archived by the company, but that's what they have been doing
>for quite some weeks.
>
>Try going to http://www.archive.org/index.php and then entering
>www.bbc.co.uk in the "Wayback Machine". Click on any of the archived
>links, and instead of seeing the BBC's frontpage you will see "Not
>found" because Demon's web filter has added:
>
>http://iwfwebfilter.thus.net/web/ ....
Doesn't do that for me, all looks fine, who's DNS are you using ?
>Doesn't do that for me, all looks fine,
So what exactly do you see?
> who's DNS are you using ?
Demon's.
Same results using three different browsers.
--
Paul Terry
> I guess I could complain to archive.org, but they are almost certain to
> say that it's not their fault that Demon is rewriting the URL (without
> the authority of the IWF it would now seem).
I tried (using Demon's DNS as always) news.bbc.co.uk and it works fine-
all the web pages I tried are accessible. But an attempt to get plain
bbc.co.uk fails, with the iwf redirection.
So it's not so simple. Demon is rewriting selectively (which they should
do, of course) but it looks like the grounds for rewriting are broken,
by any standards.
As I understand from the IWF page on Demon's site, the onus to get a
site unblocked is on the site owner. So it's up to Demon to contact the
IWF and ask what the hell is going on?
--
Peter
>In message <$8gZmyGUyHbJFwzr@[127.0.0.1]>, Wm...
><tcn...@blackhole.do-not-spam.me.uk> writes
>
>>Could it be archive.org is in error?
>
>I'm not sure how. Demon's IWF web filter is rewriting every link on
>their site - I don't think Demon intend to block every WWW site that
>has ever been archived by the company, but that's what they have been
>doing for quite some weeks.
I feel ill. I mean physically sick. I didn't want to believe this.
>Try going to http://www.archive.org/index.php and then entering
>www.bbc.co.uk in the "Wayback Machine". Click on any of the archived
>links, and instead of seeing the BBC's frontpage you will see "Not
>found" because Demon's web filter has added:
>
>http://iwfwebfilter.thus.net/web/ ....
>
>to the front of the URL.
I see what you say, Paul. I am horrified.
>We complained to the IWF today, and they say they are not responsible:
>"I can confirm that there is no entry for webarchive.org ... on the IWF
>URL list or reported to the IWF and therefore I am unable to take your
>complaint further".
>
>I guess I could complain to archive.org, but they are almost certain to
>say that it's not their fault that Demon is rewriting the URL (without
>the authority of the IWF it would now seem).
What is a person meant to say at this point?
If the bbc is bad what recourse do we have? Yes I can still access the
bbc website but I am very unhappy demon is abusing the IWF (I am not
currently a fan of theirs) in order to (presumably) prevent me seeing
something the bbc has produced.
I think this goes beyond Daily Mail puritanism.
Fix this very soon, demon
>Try going to http://www.archive.org/index.php and then entering
>www.bbc.co.uk in the "Wayback Machine". Click on any of the archived
>links, and instead of seeing the BBC's frontpage you will see "Not
>found" because Demon's web filter has added:
>http://iwfwebfilter.thus.net/web/ ....
>to the front of the URL.
Indeed ... :-(
--
Rex M F Smith
I'm not greatly impressed either.
There's a further oddity - the URL-as-displayed doesn't work if I remove
the http://iwfwebfilter.thus.net/web/ bit, as it then has a numerical
string that looks like the date you're trying to go back to *followed
by* the BBC front page URL.
Ie, it looks as if someone has not only added a prefix, but also
inverted the original URL.
Here, www.bbc.co.uk works and gets me "all the news that's fit to read",
or some of it anyway.
If I try bbc.co.uk then my browser (IE7) turns it into www.bbc.co.uk.
news.bbc.co.uk gets me a different BBC News page; possibly it's the UK
news while the www variant is world news.
Yes, that's what I see too. Is it a cock-up, or has someone at Demon
decided that, as one or more of the archived pages at the site contain
paedophilic images (assuming that to be the case), the site itself is
therefore verboten and /all/ the links on the site must be made
inaccessible?
--
John Hall
"It is a very sad thing that nowadays there is so little useless
information."
Oscar Wilde (1854-1900)
Then, choose an archived link, and hover your mouse over it. Your
browser may well display the text of the link; if so write down the very
long number (eg 19991023132451).
At the top of the browser you should be seeing the URL that's on screen.
It will resemble
web.archive.org/web/*/http://your.desired.website.
Select the * and replace it with the long number. Click 'Go'.
Works here :)
No problems for me either viewing the BBC's archive although I'm
getting not found on most pages dated 2001 and earlier, I'm using
Demon's standard DNS.
--
Les
> Here, www.bbc.co.uk works and gets me "all the news that's fit to read",
> or some of it anyway.
Ah; that works here too.
> news.bbc.co.uk gets me a different BBC News page; possibly it's the UK
> news while the www variant is world news.
No; www.bbc.co.uk is the BBC's general home page, news.bbc.co.uk is the
home page for BBC News.
--
Peter
Seems to be a bit hit and miss here, when I first tried the BBC archives
they were ok after 2001 as I posted first. After trying a few other URLs
and getting the IWF filter notification I tried the BBC again and now
occasionally get the IWF message but mostly get the BBC pages ok.
A friends web site which I know has no dodgy content brings up the IWF
filter every time.
--
Les
>In article <r1pFgUB9...@gehena.demon.co.uk>,
> Rex M F Smith <use...@gehena.demon.co.uk> writes:
>>In message <FpRwq$IwtNb...@musonix.demon.co.uk>, Paul Terry
>><ne...@nospam.demon.co.uk> writes
>>
>>>Try going to http://www.archive.org/index.php and then entering
>>>www.bbc.co.uk in the "Wayback Machine". Click on any of the archived
>>>links, and instead of seeing the BBC's frontpage you will see "Not
>>>found" because Demon's web filter has added:
>>
>>>http://iwfwebfilter.thus.net/web/ ....
>>>to the front of the URL.
>> Indeed ... :-(
>
>Yes, that's what I see too. Is it a cock-up, or has someone at Demon
>decided that, as one or more of the archived pages at the site contain
>paedophilic images (assuming that to be the case), the site itself is
>therefore verboten and /all/ the links on the site must be made
>inaccessible?
PaulT said the IWF weren't interfering. I am inclined to believe him on
the basis of "you have to trust someone once or else the whole thing
falls apart"
The likelihood of the bbc hosting images of child abuse is remote.
So, we are left with an idiot at demon.
Or, in plain words, a cock up.
>In article <r1pFgUB9...@gehena.demon.co.uk>,
> Rex M F Smith <use...@gehena.demon.co.uk> writes:
>>In message <FpRwq$IwtNb...@musonix.demon.co.uk>, Paul Terry
>><ne...@nospam.demon.co.uk> writes
>>
>>>Try going to http://www.archive.org/index.php and then entering
>>>www.bbc.co.uk in the "Wayback Machine". Click on any of the archived
>>>links, and instead of seeing the BBC's frontpage you will see "Not
>>>found" because Demon's web filter has added:
>>
>>>http://iwfwebfilter.thus.net/web/ ....
>>>to the front of the URL.
>> Indeed ... :-(
>
>Yes, that's what I see too. Is it a cock-up, or has someone at Demon
>decided that, as one or more of the archived pages at the site contain
>paedophilic images (assuming that to be the case), the site itself is
>therefore verboten and /all/ the links on the site must be made
>inaccessible?
I favour cockup. If you go to http://iwfwebfilter.thus.net there is an
example of the error message we should see if we try to access
something that IWF has censored. We are not seeing that.
When web.archive.org is looked up on Demon's nameserver, the IP returned
is 193.195.3.33.
From the OpenDNS servers it gives 207.241.232.5
Going backwards,
5.232.241.207.in-addr.arpa domain name pointer ia410305.us.archive.org.
33.3.195.193.in-addr.arpa domain name pointer iwfwebfilter.thus.net.
--
-- Michael "Soruk" McConnell Eridani Star System
MailStripper - http://www.MailStripper.eu/ - SMTP spam filter
Second Number - http://secondnumber.matrixnetwork.co.uk/
http://www.theregister.co.uk/2009/01/14/demon_muzzles_wayback_machine/
--
Les
I find that the archive of my own business web pages is blocked. I
can't tolerate an ISP that won't deliver functional web access.
--
Paul
That could be tested by someone who uses another UK ISP which
"subscribes" to the IWF seeing if they can access the pages in question.
>
>The likelihood of the bbc hosting images of child abuse is remote.
When I said "site" I meant www.archive,org not www.bbc.co.uk
>
>So, we are left with an idiot at demon.
>
>Or, in plain words, a cock up.
>
A cock-up is generally a more likely explanation than malice, in my
experience. It's a pity that Clive Feather, who also wore an IWF hat, is
no longer with Demon, as he might have been the best person to sort it
out.
>> PaulT said the IWF weren't interfering. I am inclined to believe him
>> on the basis of "you have to trust someone once or else the whole
>> thing falls apart"
>
> That could be tested by someone who uses another UK ISP which
> "subscribes" to the IWF seeing if they can access the pages in
> question.
I can see www.bbc.co.uk pages on http://www.archive.org/index.php just fine
on my VirginMedia connection (which obviously subscribes to the IWF because
the Wikipedia fiasco affected VM customers too). So from here it looks like
a Demon issue.
--
Bruce
That nice Dr Clayton said you have to use the proper channels, didn't
he?
The fully trained helpdesk staff will take your issue on board,
faithfully log it without demur, and pass it up through the
properly-laid-down escalation procedure so it can be speedily dealt with
at Demon HQ, who will ensure that it appears on status@gate from the
moment the issue is confirmed by them until the moment Demon are sure it
has been resolved.
Won't they now?
I look forward to the verbatim record of your conversation with
Bangalore :-)
--
Roy Brown 'Have nothing in your houses that you do not know to be
Kelmscott Ltd useful, or believe to be beautiful' William Morris
>That could be tested by someone who uses another UK ISP which
>"subscribes" to the IWF seeing if they can access the pages in question.
According to the very long thread that has emanated from this morning's
article in The Register, a number of others (including someone in
Romania!) are finding their HTTP requests being re-routed through
http://iwfwebfilter.thus.net - so I'm starting to wonder if it is some
odd sort of DNS problem.
--
Paul Terry
>That nice Dr Clayton said you have to use the proper channels, didn't
>he?
>
>The fully trained helpdesk staff will take your issue on board,
>faithfully log it without demur, and pass it up through the
>properly-laid-down escalation procedure so it can be speedily dealt with
>at Demon HQ
Much of the problem appears to be the way in which the Internet Archive
have configured their own web proxies... (which as Paul Terry observes,
has been bad news for some Romanians!)
... that aside, I believe that you will find the problem now fixed.
--
richard writing to inform and not as company policy
"Assembly of Japanese bicycle require great peace of mind" quoted in ZAMM
And will we be given an explanation of how it all came about?
--
Les
Criticising the government is not illegal, but often on investigation turns out
to be linked to other offences.
>Much of the problem appears to be the way in which the Internet Archive
>have configured their own web proxies... (which as Paul Terry observes,
>has been bad news for some Romanians!)
>
>... that aside, I believe that you will find the problem now fixed.
It certainly looks fixed from here: thanks!
--
Paul Terry
Poor Neil......
I've had many times
I can tell you
Times when innocence I'd trade for company
And children saw me crying
I thought I'd had my share of that
But these miss you nights
Are the longest
Midnight diamonds
Stud my heaven
Southward burning
Lie the jewels that line her place
And the warm winds
That embrace me
Just as surely kissed your face
Yeah these miss you nights
Are the longest
How I miss you
I'm not likely to tell
I'm a man and cold daylight
Buys the pride I'd rather sell
All my secrets
Are a wasted affair
You know them well
Thinking of my going
How to cut the thread
And leave it all behind
Looking windward for my compass
I take each day as it arrives
But these miss you nights
Are the longest
Lay down all thought of your surrender
Its only me who's killing time
Lay down all dreams and themes once remembered
It's just the same
This miss you game
Yeah these miss you nights
Are the longest
>Richard Clayton <ric...@highwayman.com> posted
>>
>>... that aside, I believe that you will find the problem now fixed.
>
>And will we be given an explanation of how it all came about?
That is a question better addressed to the Internet Watch Foundation and
the Internet Archive.
I might well ask them another time, but just now I'm asking you. You
seem to know that is has been fixed, so please tell us how it happened.
So it/wasn't/a filter implemented by Demon?
Strange, given that the blocked sites could be accessed through a Demon
connection using OpenDNS but not using the Demon DNS...........
--
John's Son
>In article <69hEWbIP...@satan.hell>, Big Les Wade <L...@nowhere.com>
>writes
>
>>Richard Clayton <ric...@highwayman.com> posted
>>>
>>>... that aside, I believe that you will find the problem now fixed.
>>
>>And will we be given an explanation of how it all came about?
>
>That is a question better addressed to the Internet Watch Foundation and
>the Internet Archive.
OK, so it appears the IWF listed archive.org and archive.org had some
addressing issues. Am I mistaken in thinking the DNS stuff was done
closer to home? Shouldn't demon/thus explain what they did to the DNS
of not only their own customers but also customers of other ISPs in
apparently far away places?
If I got redirected to a site in Romania I'd be confused. How do you
think someone from that country feels on being redirected to a thus
site?
>In article <69hEWbIP...@satan.hell>, Big Les Wade <L...@nowhere.com>
>writes
>>And will we be given an explanation of how it all came about?
>That is a question better addressed to the Internet Watch Foundation and
>the Internet Archive.
I have it in writing from IWF's Director of Technology and Content that
"there is no entry for webarchive.org ... on the IWF URL list or
reported to the IWF".
--
Paul Terry
>Wed, 14 Jan 2009 18:42:29 <MVFgjiDV...@highwayman.com>
>demon.service Richard Clayton <ric...@highwayman.com>
>
>>In article <69hEWbIP...@satan.hell>, Big Les Wade <L...@nowhere.com>
>>writes
>>
>>>Richard Clayton <ric...@highwayman.com> posted
>>>>
>>>>... that aside, I believe that you will find the problem now fixed.
>>>
>>>And will we be given an explanation of how it all came about?
>>
>>That is a question better addressed to the Internet Watch Foundation and
>>the Internet Archive.
>
>OK, so it appears the IWF listed archive.org
yes
>and archive.org had some
>addressing issues.
not to my knowledge
>Am I mistaken in thinking the DNS stuff was done
>closer to home?
no
>Shouldn't demon/thus explain what they did to the DNS
If you want to know what Demon/THUS is prepared to explain then please
review:
http://www.demon.net/helpdesk/technicallibrary/faq/iwf_web_filter/
>of not only their own customers but also customers of other ISPs in
>apparently far away places?
Demon has not done anything to alter the DNS results for customers at
other ISPs, far away or not. Demon has not made their DNS caches
available for non-customers for several years.
>If I got redirected to a site in Romania I'd be confused.
So would I
Has that actually happened to you ?
> How do you
>think someone from that country feels on being redirected to a thus
>site?
That's a matter for their own tech support people. I expect that they
will refer them on to the Internet Archive for answers.
Wm:
>>Fix this very soon, demon
>
>That nice Dr Clayton said you have to use the proper channels, didn't
>he?
Yes, but I suspect even he found them unnecessary in this case.
>The fully trained helpdesk staff will take your issue on board,
>faithfully log it without demur, and pass it up through the
>properly-laid-down escalation procedure so it can be speedily dealt
>with at Demon HQ, who will ensure that it appears on status@gate from
>the moment the issue is confirmed by them until the moment Demon are
>sure it has been resolved.
>
>Won't they now?
I've not tested them recently.
>I look forward to the verbatim record of your conversation with
>Bangalore :-)
I'd love to have given you a transcription but I didn't feel the need to
call as archive.org isn't a site I visit often.
Mind you I am one of the people that thinks Bangalore is OK so I'd
probably have produced a positive report. I find racism uncomfortable
and all that. (I guess others do too it is just the case that someone
needs to say it once in a while to balance things because this group
does seem to have, historically, people that prefer Sarfend).
>In article <fokRjXFHrjbJFwWH@[127.0.0.1]>, Wm... <tcn...@blackhole.do-
>not-spam.me.uk> writes
>
>>Wed, 14 Jan 2009 18:42:29 <MVFgjiDV...@highwayman.com>
>>demon.service Richard Clayton <ric...@highwayman.com>
>>
>>>In article <69hEWbIP...@satan.hell>, Big Les Wade <L...@nowhere.com>
>>>writes
>>>
>>>>Richard Clayton <ric...@highwayman.com> posted
>>>>>
>>>>>... that aside, I believe that you will find the problem now fixed.
>>>>
>>>>And will we be given an explanation of how it all came about?
>>>
>>>That is a question better addressed to the Internet Watch Foundation and
>>>the Internet Archive.
>>
>>OK, so it appears the IWF listed archive.org
>
>yes
OK
>>and archive.org had some
>>addressing issues.
>
>not to my knowledge
Hmmm, some other people have reported otherwise.
>>Am I mistaken in thinking the DNS stuff was done
>>closer to home?
>
>no
It would be easy to take that as a plain and simple "no" except you are
Richard Clayton. So it means "yes", in effect.
Students of RC should notice that I included an unintentional subtle
negative in the question and RC picked up on that.
>>Shouldn't demon/thus explain what they did to the DNS
>
>If you want to know what Demon/THUS is prepared to explain then please
>review:
>
> http://www.demon.net/helpdesk/technicallibrary/faq/iwf_web_filter/
That is policy rather than an explanation of what happened.
>>of not only their own customers but also customers of other ISPs in
>>apparently far away places?
>
>Demon has not done anything to alter the DNS results for customers at
>other ISPs, far away or not. Demon has not made their DNS caches
>available for non-customers for several years.
But people far away were affected nonetheless.
>>If I got redirected to a site in Romania I'd be confused.
>
>So would I
>
>Has that actually happened to you ?
I have never been to Romania. I have been to a number of countries
around the world, not Romania so far. I am sure most of the people
there are nice.
>> How do you
>>think someone from that country feels on being redirected to a thus
>>site?
>
>That's a matter for their own tech support people. I expect that they
>will refer them on to the Internet Archive for answers.
Hmmmn, so we are back to "not us, mate" ?
Well I can't even access the wayback machine now!
Was pointed at this thread by some folks on forums.thinkbroadband.com
after I posted a query about this.
I can confirm that theURL mangling seems to be confined to Thus and
Demon users, as everybody else who replied on the forum reported that
they had no trouble accessing the URL's I'd posted on the wayback machine.
E-mailed demon and iwf; IWF deny any responsibility and demon still
haven't replied after 3 days.
It still wasn't working for me 5 hours ago (About 3 o'clock (Today being
14/1/2009)), but now I can't access the wayback machine at all.
All I get is:
------
Failed Connection.
We're sorry. Your request failed to connect to our servers. This may be
due to temporary problems in our data center, or difficulty serving a
higher-than-usual volume of traffic.
You may want to:
Try again later
Search other dates for your page, aozh25.dsl.pipex.com/yaris/index.htm
Search for all pages on the site aozh25.dsl.pipex.com/
Try a different page address, at top
See the FAQs for more info and help, or contact us.
------
It has been doing this for at least the past 2 hours.
Ordinarily I'd put this down to congestion or overloading, but in light
of recent events I'm a bit more paranoid.
Still, let's see if it's still doing it tomorrow...
I'm getting tired of this sort of thing happening; I think I might call
it a day and risk moving to an ADSL2 LLU provider like Be. :(
It wasn't that long ago after that Phorm fiasco when my connection was
getting blocked at 213.121.102.137 for about 7 hours, and now this!
What the heck is happening at Demon anyway? They used to have a good
crew back in the day, but they're backsliding almost as fast as
Freeserve once it got bought out by Wanado and then Orango!
>Wed, 14 Jan 2009 19:53:12 <7D6NW1Eo...@highwayman.com>
>demon.service Richard Clayton <ric...@highwayman.com>
>
>>In article <fokRjXFHrjbJFwWH@[127.0.0.1]>, Wm... <tcn...@blackhole.do-
>>not-spam.me.uk> writes
>>
>>>and archive.org had some
>>>addressing issues.
>>
>>not to my knowledge
>
>Hmmm, some other people have reported otherwise.
Understanding what is happening can be quite a challenge!
But I don't believe that archive.org had addressing issues
>>>Shouldn't demon/thus explain what they did to the DNS
>>>of not only their own customers but also customers of other ISPs in
>>>apparently far away places?
>>
>>Demon has not done anything to alter the DNS results for customers at
>>other ISPs, far away or not. Demon has not made their DNS caches
>>available for non-customers for several years.
>
>But people far away were affected nonetheless.
Understanding what is happening can be quite a challenge!
But I don't believe that people far away had problems with their DNS
>All I get is:
>------
>Failed Connection.
>
>
>We're sorry. Your request failed to connect to our servers. This may be
>due to temporary problems in our data center, or difficulty serving a
>higher-than-usual volume of traffic.
You _are_ accessing the Internet Archive, but it is busy
>It has been doing this for at least the past 2 hours.
It is often very busy, as regular users will tell you
>Ordinarily I'd put this down to congestion or overloading, but in light
>of recent events I'm a bit more paranoid.
I expect it's receiving more hits than usual today :( many of which are
from people who aren't especially interested in what it returns :(
>I'm getting tired of this sort of thing happening; I think I might call
>it a day and risk moving to an ADSL2 LLU provider like Be. :(
That won't make the Internet Archive any less busy :(
I tried http://www.aozh25.dsl.pipex.com/yaris/index.htm, and that just
gives the Failed Connection error above.
A thought occurs 'tho; If everybody is still trying out URLs and are
effectively DDoS'ing the poor wayback machine, that would explain the
connection problems...!
BTW, do demon even update their finger/.plan service anymore?
Could it be significant that webarchive.org and archive.org are
different, or was that just a careless mistake by the Director of
Technology and Content in drafting his or her letter?
>I expect it's receiving more hits than usual today :( many of which are
>from people who aren't especially interested in what it returns :(
<Fx: raises hand>
>In article <BcDWeOJYzkbJFwnR@[127.0.0.1]>, Wm... <tcn...@blackhole.do-
>not-spam.me.uk> writes
>
>>Wed, 14 Jan 2009 19:53:12 <7D6NW1Eo...@highwayman.com>
>>demon.service Richard Clayton <ric...@highwayman.com>
>>
>>>In article <fokRjXFHrjbJFwWH@[127.0.0.1]>, Wm... <tcn...@blackhole.do-
>>>not-spam.me.uk> writes
>>>
>>>>and archive.org had some
>>>>addressing issues.
>>>
>>>not to my knowledge
>>
>>Hmmm, some other people have reported otherwise.
>
>Understanding what is happening can be quite a challenge!
>
>But I don't believe that archive.org had addressing issues
Richard, you are an important person to us.
Report back when you know enough.
>Understanding what is happening can be quite a challenge!
>
>But I don't believe that people far away had problems with their DNS
I look forward to your report.
P.S. I am a little disappointed you didn't follow up on the mistaken yes
/ no. Cest la vive ?
Elsewhere in this thread, it's already been mentioned that Demon's DNS
server was until earlier today returning 193.195.3.33 when queried for
www.archive.org
193.195.3.33 is the IP address of iwfwebfilter.thus.net
www.archive.org is actually 207.241.229.39 and Demon's DNS is now returning
this.
Presumably, 193.195.3.33, which is a Demon-operated server, has a
proxy/filtering function. It looks like the DNS was changed to route
www.archive.org traffic through this proxy. That would enable the proxy to
inspect all requests sent to the archive and selectively block them.
Instead of this, what appears to have happened is that, although customers'
requests were forwarded to www.archive.org by the proxy, the hostname was
changed from www.archive.org to "iwfwebfilter.thus.net" in HTML anchor tags
in every returned response. This must have been done by the Demon-operated
server. It just looks like a coding bug on Demon's part.
Subtlety alert - It was web.archive.org (not www.archive.org) that was
returning 193.195.3.33. I see it's now returning 207.241.232.5.
--
-- Michael "Soruk" McConnell Eridani Star System
MailStripper - http://www.MailStripper.eu/ - SMTP spam filter
Second Number - http://secondnumber.matrixnetwork.co.uk/
Both then.
I tried pinging www.archive.org whilst the redirection was in force and I
got 193.195.3.33
>That would enable the proxy to
>inspect all requests sent to the archive and selectively block them.
>Instead of this, what appears to have happened is that, although customers'
>requests were forwarded to www.archive.org by the proxy, the hostname was
>changed from www.archive.org to "iwfwebfilter.thus.net" in HTML anchor tags
>in every returned response.
yes it was changed (though you have the wrong hostname)
> This must have been done by the Demon-operated
>server.
no that's not the case (if you think about it, that wouldn't cause
"people in Romania" to see the altered version)
> It just looks like a coding bug on Demon's part.
no!
... and the recently arrived word from across the Atlantic is that
strange pages will not longer be created or served to anyone
Hi.
The problems have now been fixed. The explanation is not short, so bear
with me:
Firstly, yes, something on web.archive.org was blocked by the IWF. Don't
ask what, I don't know.
The filter we use uses a proxy to inspect suspect URLs. Where a URL is not
on the IWF list (ie, the server hosts some child abuse content, but only a
single URL is blocked), we have to proxy the connection on to the original
server the request was intended for.
Here's where it gets interesting. The proxy sends various bits of
information with the request. One of these is the name of the proxy itself.
Not unsurprisingly, this is 'iwfwebfilter.thus.net'.
It seems that archive.org use caches at their end to speed up access to
pages. When a page is requested, if it's not in the cache, it is built from
the archive and made available to the requestor. As part of this build
process, the server takes a hostname from the cache, along with the date
portion of the URL, etc, to create the 'base URL' of the page.
To explain: say you want to archive www.demon.net. In order to make the
page available on
http://web.archive.org/web/20070107191318/http://www.demon.net/
you need to strip out all the references to http://www.demon.net/ in the
page (in links, images, CSS, javascript, etc) and replace them with the URL
above. Since a page may not change much, it's better to do it at request
time, so that a single copy of a page can span multiple archived instances.
Unfortunately, the archive.org software would take the server name we
supplied and use it in place of 'web.archive.org', which is why you'd get a
URL like
http://iwfwebfilter.thus.net/web/20070107191318/http://www.demon.net/
That server doesn't have any content there, so you'd get a 404.
However, this only happened on a cache miss. That is, if the page was
already in the cache, and it had the correct URLs, it would work just fine.
So some people would see that everything appeared to be as it should.
Equally unfortunately, a page with the iwfwebfilter.thus.net URLs could be
cached and then served up to non-Demon customers, which explains our
friends in Romania, and other reports of people who'd not been anywhere
near the Demon caches seeing 'iwfwebfilter.thus.net' where they'd been
expecting 'web.archive.org'
Shortly before 10pm this evening (albeit a more civilised time where
they're based), the Internet Archive fixed the bug and cleared the caches,
so the problem won't return. Nor can the same technique of mis-supplying a
hostname be used for mischief.
To summarise:
* There was a bug in the Wayback Machine software, which we tickled
* Demon didn't perform any content manipulation
* Demon didn't unilaterally filter or block web.archive.org
* The Internet Archive have now fixed the bug
As Richard is fond of saying, I'm writing to inform.
Brian
--
☺
>... and the recently arrived word from across the Atlantic is that
>strange pages will not longer be created or served to anyone
I didn't want to see the pedophilic stuff anyway. So if it is gone good
riddance.
So the HTML anchor tags containing "iwfwebfilter.thus.net" were generated by
the Internet Archive at source, presumably by prior arrangement with Demon.
That suggests they have reasons for not simply removing the offending
content; but will actively facilitate local filtering schemes.
But Demon's DNS did redirect requests through iwfwebfilter.thus.net until
earlier today and presumably the Internet Archive gave them special
treatment because they came from that IP address.
>So the HTML anchor tags containing "iwfwebfilter.thus.net" were generated by
>the Internet Archive at source, presumably by prior arrangement with Demon.
>That suggests they have reasons for not simply removing the offending
>content; but will actively facilitate local filtering schemes.
nonsense -- have a look at what Brian Widdas has written
OK. I take most of this guesswork back, apart from the proxying. I've just
seen Brian Widdas' explanation.
>On 2009-01-10, Alan Poulter <l...@poulter.demon.co.uk> wrote:
>>
>> Is anyone else having problems accessing archive.org? If you are please
>> can you email me rather than reply here.
>
>Hi.
>
>The problems have now been fixed. The explanation is not short, so bear
>with me:
I like your explanation.
>Firstly, yes, something on web.archive.org was blocked by the IWF. Don't
>ask what, I don't know.
fx: wrinkles nose: I think, in truth, most of us probably don't want to
know what it was. I'm guessing an adult doing something to a child. Can
we leave it at that without going into detail?
>The filter we use uses a proxy to inspect suspect URLs. Where a URL is not
>on the IWF list (ie, the server hosts some child abuse content, but only a
>single URL is blocked), we have to proxy the connection on to the original
>server the request was intended for.
Query: how did the bbc get involved?
>Here's where it gets interesting. The proxy sends various bits of
>information with the request. One of these is the name of the proxy itself.
>Not unsurprisingly, this is 'iwfwebfilter.thus.net'.
Are you saying the filter was effectively consuming itself?
>It seems that archive.org use caches at their end to speed up access to
>pages. When a page is requested, if it's not in the cache, it is built from
>the archive and made available to the requestor. As part of this build
>process, the server takes a hostname from the cache, along with the date
>portion of the URL, etc, to create the 'base URL' of the page.
>
>To explain: say you want to archive www.demon.net. In order to make the
>page available on
>
> http://web.archive.org/web/20070107191318/http://www.demon.net/
>
>you need to strip out all the references to http://www.demon.net/ in the
>page (in links, images, CSS, javascript, etc) and replace them with the URL
>above. Since a page may not change much, it's better to do it at request
>time, so that a single copy of a page can span multiple archived instances.
But if unknown man is checking for child porn we can't tell that?
>Unfortunately, the archive.org software would take the server name we
>supplied and use it in place of 'web.archive.org', which is why you'd get a
>URL like
>
> http://iwfwebfilter.thus.net/web/20070107191318/http://www.demon.net/
>
>That server doesn't have any content there, so you'd get a 404.
I think http://iwfwebfilter.thus.net is meant to show little. It is
just puzzling if you are looking for honest content.
>However, this only happened on a cache miss. That is, if the page was
>already in the cache, and it had the correct URLs, it would work just fine.
>So some people would see that everything appeared to be as it should.
Yikes, does that mean some people got to be distressed and saw the
children being harmed?
>Equally unfortunately, a page with the iwfwebfilter.thus.net URLs could be
>cached and then served up to non-Demon customers, which explains our
>friends in Romania, and other reports of people who'd not been anywhere
>near the Demon caches seeing 'iwfwebfilter.thus.net' where they'd been
>expecting 'web.archive.org'
Why? Who was at fault?
>Shortly before 10pm this evening (albeit a more civilised time where
>they're based), the Internet Archive fixed the bug and cleared the caches,
>so the problem won't return. Nor can the same technique of mis-supplying a
>hostname be used for mischief.
I think the IWF is at heart a good organisation, I have no reason to
believe thus is per se bad. What went wrong?
>To summarise:
> * There was a bug in the Wayback Machine software, which we tickled
> * Demon didn't perform any content manipulation
> * Demon didn't unilaterally filter or block web.archive.org
> * The Internet Archive have now fixed the bug
>
>
>As Richard is fond of saying, I'm writing to inform.
I am amused and welcome your posting.
>As Richard is fond of saying, I'm writing to inform. Brian
Good men both of you. ;)
--
Jim Crowther
Thanks very much!
> Oh joy ... yet more "unintended collateral damage" from the IWF.
>
> Didn't they actually learn ANYTHING from their Wikipedia disaster just
> before Christmas?
Ah well... Turns out that it's neither Demon, nor the IWF at fault.
But that didn't stop us in d.s from giving them, a rousing rasberry.
Funny how nothing changes.
--
Peter
Wm, this sounds like you are accusing me of racism?
--
Roy Brown 'Have nothing in your houses that you do not know to be
Kelmscott Ltd useful, or believe to be beautiful' William Morris
Wm:
>>I'd love to have given you a transcription but I didn't feel the need
>>to call as archive.org isn't a site I visit often.
>>
>>Mind you I am one of the people that thinks Bangalore is OK so I'd
>>probably have produced a positive report. I find racism uncomfortable
>>and all that. (I guess others do too it is just the case that someone
>>needs to say it once in a while to balance things because this group
>>does seem to have, historically, people that prefer Sarfend).
>
>Wm, this sounds like you are accusing me of racism?
That was not my intention, Roy; if I thought you were racist I'd have
said so. I was referring to people who regularly bemoan offshore help
desks in general.
Though the comments here were positively restrained compared to some of
the wildly OTT ones made in response to The Register's article.
>Ah well... Turns out that it's neither Demon, nor the IWF at fault.
That's somewhat debatable, considering there was no problem at all
before Demon introduced Clean Feed filtering in the last few weeks. I
imagine that archive.org had not considered the possibility of an ISP
such as Demon filtering their URLs, but it is good to see that they have
responded to this new situation.
>But that didn't stop us in d.s from giving them, a rousing rasberry.
>Funny how nothing changes.
But, Peter, the thing that hasn't changed is the way in which Demon do
nothing until pressure is brought upon them. THAT is the reason for the
raspberry.
The problem with the "Wayback Machine" has existed for weeks (and
probably started at the time of the IWF Wikipedia incident in early
December). Everyone I contacted passed the buck until yesterday. Whether
the final prompt into action was the article in The Register or RC's
intervention, or something else, I don't know.
--
Paul Terry
Quite. Millions of people have been accessing archive.org through proxy
servers for many years. Demon said the bug was in the US; but there was
also something odd about Demon's proxy server, probably something
non-standard in one of the HTTP headers.
One thing I am still curious about is the use of DNS to route archive.org
traffic through the proxy server, because this would only work for customers
who use Demon's DNS. Those wishing to access blocked content could get
around it by using another DNS. So is DNS a red-herring and do Demon have
another way of routing traffic through the proxy, or is it as lame as it
appears?
>That's somewhat debatable, considering there was no problem at all
>before Demon introduced Clean Feed filtering in the last few weeks.
This isn't entirely accurate:
(a) "CleanFeed" is an internal BT project name for their system (which
works differently than Demon's) -- and in particular it's a name of yet
another firm's scheme. Still perhaps they should defend their brand.
(b) Demon introduced their scheme in December 2007, not "in the last few
weeks". It's caused a few problems, but not all that many.
> I
>imagine that archive.org had not considered the possibility of an ISP
>such as Demon filtering their URLs, but it is good to see that they have
>responded to this new situation.
It seems they had not experienced the use of a intermediate cache
recently :( The fact it was filtering was entirely irrelevant :(
>The problem with the "Wayback Machine" has existed for weeks (and
>probably started at the time of the IWF Wikipedia incident in early
>December). Everyone I contacted passed the buck until yesterday.
The problem first occurred at the end of September, but it went away too
fast for any diagnostics to reveal the problem :( Now that the problem
is understood to be at the Internet Archive end of the connection, it's
obvious why the tests run then didn't show anything :(
>Whether
>the final prompt into action was the article in The Register or RC's
>intervention, or something else, I don't know.
It's never just one thing, or one person
I didn't even know about this filter until I ran into this problem!
> OTOH, if they weren't stealth-filtering us in the first place this
> problem would never have arisen.
Stealth?
This has been discussed to death in the past, and it's in the Demon help
pages.
About as stealthy as the Lord Mayor's show.
--
Peter