Incomplete Binaries on Demon's Usenet Servers
FMURL
> Checked out a few binary newsgroups the other day when looking for some
> stuff. Was amazed to see that virtually every posting of more than 10000
> lines was incomplete.
>
> Is this normal these days on Demon's news servers? If so, it's a
> dreadful state of affairs. What's the point of claiming to offer 10s of
> thousands of newsgroups, when the contents of half of them are useless?
You'll find this is the case on many news servers, even the dedicated
commercial offerings. Anything of that size is not likely to propogate well,
better a message of around half that size (5000-7500 lines).
If you're serious about Usenet, you should get a dedicated account. Simple
as that. I doubt very much if any generic ISP has the money or the
inclination to invest in a full-on news service of the standard of, say,
Easynews or GigaNews (and many others besides). Usenet traffic is growing
exponentially, and normal ISPs can do very little to keep up with the
demand.
The number of newsgroups available by any SP means very little. It's
retention and propogation that count.
On a personal note: I'd gladly pay more oney per month to Demon for an
improved news service because all the best servers are in the States. :-(
--
FM
Stuart Millington
<ke...@xentrik.demon.co.uk> wrote:
>Is this normal these days on Demon's news servers? If so, it's a
Demon's news peering has been fucked for a long time now. See the
thread, in this group, entitled "Usenet".
>dreadful state of affairs. What's the point of claiming to offer 10s of
>thousands of newsgroups, when the contents of half of them are useless?
Ask someone from Demon. You will not get an answer.
--
------------------------------------------------------------------
- Stuart Millington -
- mailto:ph...@dsv1.co.uk http://www.z-add.co.uk/ -
Stuart Millington
>"Keith Johnson" <ke...@xentrik.demon.co.uk> wrote
>> Checked out a few binary newsgroups the other day when looking for some
>> stuff. Was amazed to see that virtually every posting of more than 10000
>> lines was incomplete.
>You'll find this is the case on many news servers, even the dedicated
>commercial offerings. Anything of that size is not likely to propogate well,
>better a message of around half that size (5000-7500 lines).
True.
>If you're serious about Usenet, you should get a dedicated account. Simple
>as that. I doubt very much if any generic ISP has the money or the
>inclination to invest in a full-on news service of the standard of, say,
>Easynews or GigaNews...
Bollox. As has been covered here many times before. Freeserve
(until their recent SNAFU's) and Clara both manage to have excellent
completion rates compared to Demon. Retention times are lower than the
US providers, but not by much.
FS's news charges are probably irrelevant given the losses they
make. However, Clara charge 20 GBP p/a for remote news access - which
many Demon customers are forced to use since Demon's peering is
fucked.
I can only **guess** that Clara[1] are using this money to
supplement their regular dial-up subscriber revenue. Aside from the
problems of the last couple of days, their service and *customer
service* has been excellent. Unlike Demon's.
>The number of newsgroups available by any SP means very little. It's
>retention and propogation that count.
Demon have had neither retention, peering (binary ng's) or a
willingness to comment on this subject for a *very* long time now :(
>On a personal note: I'd gladly pay more oney per month to Demon for an
>improved news service because all the best servers are in the States. :-(
As would I - within limits.
[1] My dislike of Clara goes back to when they, and Enterprise, were
handing out .net addresses to customers. But their news only service
cannot be beaten(sp?) on value for money terms.
FMURL
> >If you're serious about Usenet, you should get a dedicated account.
Simple
> >as that. I doubt very much if any generic ISP has the money or the
> >inclination to invest in a full-on news service of the standard of, say,
> >Easynews or GigaNews...
>
> Bollox. As has been covered here many times before. Freeserve
> (until their recent SNAFU's) and Clara both manage to have excellent
> completion rates compared to Demon. Retention times are lower than the
> US providers, but not by much.
Can't agree about Freeserve (apart from the problems of course) because I
tried them. Great internet connections (when they work) but the news service
is just not good enough. At least not for me it wasn't.
>
> FS's news charges are probably irrelevant given the losses they
> make. However, Clara charge 20 GBP p/a for remote news access - which
> many Demon customers are forced to use since Demon's peering is
> fucked.
>
> I can only **guess** that Clara[1] are using this money to
> supplement their regular dial-up subscriber revenue. Aside from the
> problems of the last couple of days, their service and *customer
> service* has been excellent. Unlike Demon's.
I heard that Clara was struggling with news of late. I'm currently testing
seven news providers at the moment, and nevr got as far as Clara after I
heard about the problems. Seems some people were taking advantage of the
non-restrictive nature of the accounts, not to mention the low cost. I
rather doubt they could afford to offer a full-on news service for 20 quid
for long once the DSL gang find their way on there... SuperNews UK charge
that much per month for 3 days binary retention <!>, although their service
is good and if you need the best text service they are the people.
> >The number of newsgroups available by any SP means very little. It's
> >retention and propogation that count.
>
> Demon have had neither retention, peering (binary ng's) or a
> willingness to comment on this subject for a *very* long time now :(
Well, that is sad of course, but isn't that just the way that most ISPs deal
with their problems? I remember CIX a few years back. Appalling service, but
they flatly refused to admit there was a problem, much less enter into a
dialogue about it (I nearly got thrown off for my rather outspoken
comments). BT Internet are doing the same thing now.
> >On a personal note: I'd gladly pay more oney per month to Demon for an
> >improved news service because all the best servers are in the States. :-(
>
> As would I - within limits.
Guess what one is prepared to pay is based on how important certain aspects
of Usenet are. Propogation and retention are important for me cos I don't
have a fast connection, so I'm happy to pay almost as much for my news
service as I would for a decent ISP - same goes for web hosting.
--
FM
Stuart Millington
>"Stuart Millington" <ne...@dsv1.co.uk> wrote:
>> Bollox. As has been covered here many times before. Freeserve
>> (until their recent SNAFU's) and Clara both manage to have excellent
>> completion rates compared to Demon. Retention times are lower than the
>> US providers, but not by much.
>
>Can't agree about Freeserve (apart from the problems of course) because I
>tried them. Great internet connections (when they work) but the news service
>is just not good enough. At least not for me it wasn't.
a.b.m.btvs was perfect until the news server started disappearing
for a couple of days at a time. At which point, I paid for Clara and
started using my (already paid for) Demon PCP upgrade.
>I heard that Clara was struggling with news of late. I'm currently testing
They're buying a couple of extra servers in, apparently, a couple
of weeks. I hope this will fix things there.
>seven news providers at the moment, and nevr got as far as Clara after I
>heard about the problems. Seems some people were taking advantage of the
>non-restrictive nature of the accounts, not to mention the low cost. I
>rather doubt they could afford to offer a full-on news service for 20 quid
>for long once the DSL gang find their way on there...
They appear to manage OK.
> SuperNews UK charge
>that much per month for 3 days binary retention <!>, although their service
>is good and if you need the best text service they are the people.
Newscene **used** to be good. Dunno, what they are like now?
>> Demon have had neither retention, peering (binary ng's) or a
>> willingness to comment on this subject for a *very* long time now :(
>
>Well, that is sad of course, but isn't that just the way that most ISPs deal
>with their problems? I remember CIX a few years back. Appalling service, but
>they flatly refused to admit there was a problem, much less enter into a
>dialogue about it (I nearly got thrown off for my rather outspoken
>comments). BT Internet are doing the same thing now.
I bet they still have not fixed the newnews problem, which they
admitted to in January 1999 - *2 years* after it was reported. Are
they still running a broken and mis-configured December 1996 version
of INN?
>> >On a personal note: I'd gladly pay more oney per month to Demon for an
>> >improved news service because all the best servers are in the States. :-(
>>
>> As would I - within limits.
>
>Guess what one is prepared to pay is based on how important certain aspects
>of Usenet are.
For me, it's more to do with what others are charging for the same
service.
> Propogation and retention are important for me cos I don't
>have a fast connection,
Likewise, just ISDN - single channel due to DEMON's artificial PCP
limitations :(
> so I'm happy to pay almost as much for my news
>service as I would for a decent ISP - same goes for web hosting.
I have my own (virtual) server for that ;)
nospam
<ke...@xentrik.demon.co.uk> wrote:
>Hi All -
>
>Checked out a few binary newsgroups the other day when looking for some
>stuff. Was amazed to see that virtually every posting of more than 10000
>lines was incomplete.
>
>Is this normal these days on Demon's news servers? If so, it's a
>dreadful state of affairs. What's the point of claiming to offer 10s of
>thousands of newsgroups, when the contents of half of them are useless?
>
>Keith
You could do a lot worse than try Freeserve.
Most binaries if posted complete get there complete.
Makes Demons news offerings look like the pile of crap that it has
been for that last 15 months. Look at this link and the reason will be
obvious that they running at maximum capacity all the time so posts
get dropped.
http://www.stats.news.uk.psi.net/Diablo.peers.html
I joined demon for the very good news feed it once had. Now freeserve
is as good as demon was 2 years ago for binaries and demon news
service is a pile of crap that has had as much proportional investment
in news as Railtrack has had in rails.
Mike Pellatt
<fm...@yahoo.com> wrote:
>I heard that Clara was struggling with news of late. I'm currently testing
>seven news providers at the moment, and nevr got as far as Clara after I
>heard about the problems. Seems some people were taking advantage of the
>non-restrictive nature of the accounts, not to mention the low cost. I
>rather doubt they could afford to offer a full-on news service for 20 quid
>for long once the DSL gang find their way on there...
<fx> waves.
--
Mike Pellatt
Chris Lawrence
<fm...@yahoo.com> writes
>I heard that Clara was struggling with news of late. I'm currently testing
>seven news providers at the moment, and nevr got as far as Clara after I
>heard about the problems. Seems some people were taking advantage of the
Normally excellent service but over the last couple of days I've been
seeing several dozen of:
News server unavailable : 400 System Capacity Exceeded (Typhoon v1.2.3)
--
Regards, Chris
John Underwood
(Reference: <4sj75tchg82lpi6rl...@4ax.com>)
> FS's news charges are probably irrelevant given the losses they make.
>However, Clara charge 20 GBP p/a for remote news access - which many
>Demon customers are forced to use since Demon's peering is fucked.
With respect no-one is forced to use any news server. Demon provide one
as part of the package. If you decide that it doesn't meet your needs,
then you can choose to use another one. You still retain the choice, it
is up to you whether you have news at all.
If, as a result of your pressure, Demon were to provide a service
comparable with the Clara one, then I expect they would have to charge
for that. Were that charge to be part of the SDU fee I would object very
strongly. As it is, you are not comparing like with like.
--
John Underwood
Use the Reply To: address for the next 30 days
After that write to jo...@the-underwoods.org.uk
Do not send anything to the From: address
John Underwood
(Reference: <nru85t0tk3ud7svlq...@4ax.com>)
>There is a technically and practically sound reason why it would be
>much better for Demon to offer a proper news service - bandwidth. I
>have considered using one of the US-based services, but I would be
>connecting to this via my Demon account. This means more use of ongoing
>bandwidth when I (and others like me) want to access news by such routes.
Superficially, you are right. Demon don't place any restrictions on your
activities and the demands on external bandwidth, so why do you have
qualms about using what comes as part of the service you pay for? True,
Demon could, perhaps, reduce this usage by a better news feed - after
all, that was why they provided an unsupported news feed in the first
place. However, I don't think you can take this in isolation and the
decision on how Demon utilise its resources is a matter for them. They
may decide to accept the consequences of what appears to some as a poor
service in one area (including some increased external bandwidth and
loss of some customers) but there may be corresponding benefits
elsewhere. I cannot imagine what those benefits could possibly be, but I
cannot see any reason why Demon should explain the basis of their
business decisions either.
Andy
<ne...@the-underwoods.org.uk> wrote
[]
>Superficially, you are right. Demon don't place any restrictions on your
>activities and the demands on external bandwidth, so why do you have
>qualms about using what comes as part of the service you pay for? True,
>Demon could, perhaps, reduce this usage by a better news feed - after
>all, that was why they provided an unsupported news feed in the first
>place. However, I don't think you can take this in isolation and the
>decision on how Demon utilise its resources is a matter for them. They
>may decide to accept the consequences of what appears to some as a poor
>service in one area (including some increased external bandwidth and
>loss of some customers) but there may be corresponding benefits
>elsewhere. I cannot imagine what those benefits could possibly be, but I
>cannot see any reason why Demon should explain the basis of their
>business decisions either.
Um, yes; but if they more clearly stated *what they were* it would save
a considerable amount of time-wasting speculation!
--
Andy
For Austrian philately <URL: http://www.kitzbuhel.demon.co.uk/austamps/>
For Lupus <URL: http://www.kitzbuhel.demon.co.uk/lupus/>
For my other interests <URL: http://www.kitzbuhel.demon.co.uk/>
John Underwood
(Reference: <IPACKTBI...@kitzbuhel.demon.co.uk>)
>Um, yes; but if they more clearly stated *what they were* it would save
>a considerable amount of time-wasting speculation!
If you believe that of demon.service you are either very naive or have
an incredibly generous nature.
We have recently spent some interesting weeks discussing a business
decision which, according to some of us, could hardly have been spelt
out more clearly but was totally misread by someone.
Denis Mcmahon
<ne...@dsv1.co.uk> wrote:
> Likewise, just ISDN - single channel due to DEMON's artificial PCP
>limitations :(
So why not sign up with another ISP that offers 128K bonded ISDN G5 24
* 7 with no useage limits or timeouts, allows you to run your own
servers (static ip is useful for this), allows direct smtp and http
(i.e no forced proxy) for 19.99 pcm instead.
(And when you find them let us all know who they are.)
Rgds
Denis
--
Denis McMahon
Mobile: +44 7802 468949
Email: de...@pickaxe.demon.co.uk
I always trim ng when posting!
Paul Harris
<ne...@amigo.co.uk> writes
>In article <4sj75tchg82lpi6rl...@4ax.com>, Stuart
>Millington <ne...@dsv1.co.uk> writes
>
>> Demon have had neither retention, peering (binary ng's) or a
>>willingness to comment on this subject for a *very* long time now :(
>
>
>Are there any Thus shareholders in here that could ask the board for
>more details of the problem, and how they intend to fix it, at the next
>AGM?
>
Almost worth buying a single share for the right to attend and ask some
such burning question which seems otherwise to be studiously avoided and
ignored. Shall we have a whip round, no best not answer that we could
go way OT!
--
Paul Harris
FMURL
> >Can't agree about Freeserve (apart from the problems of course) because I
> >tried them. Great internet connections (when they work) but the news
service
> >is just not good enough. At least not for me it wasn't.
>
> a.b.m.btvs was perfect until the news server started disappearing
> for a couple of days at a time. At which point, I paid for Clara and
> started using my (already paid for) Demon PCP upgrade.
> >I heard that Clara was struggling with news of late. I'm currently
testing
>
> They're buying a couple of extra servers in, apparently, a couple
> of weeks. I hope this will fix things there.
Signed up with Clara last night and so far the service has been very good.
Around 3 day retention on binaries, and the fact that it's only a couple of
hops from Demon helps a great deal.
> >rather doubt they could afford to offer a full-on news service for 20
quid
> >for long once the DSL gang find their way on there...
>
> They appear to manage OK.
At the moment, yes. But time will tell.
> Newscene **used** to be good. Dunno, what they are like now?
Very good. Just a couple of hops from Demon means connections are very, very
good. Retention on binaries is >7 days and propogation seems good thus far.
> >dialogue about it (I nearly got thrown off for my rather outspoken
> >comments). BT Internet are doing the same thing now.
>
> I bet they still have not fixed the newnews problem, which they
> admitted to in January 1999 - *2 years* after it was reported. Are
> they still running a broken and mis-configured December 1996 version
> of INN?
Pass. Gave up trying to talk to the people at BT Internet. I'm still owed
various explanations about account problems from four months ago. I've just
given up on the whole sorry affair now. Just shite.
> >Guess what one is prepared to pay is based on how important certain
aspects
> >of Usenet are.
>
> For me, it's more to do with what others are charging for the same
> service.
Quite so, which is the main reason for me testing so many at the moment.
Many offer good retention, but support and customer service are often a
different matter...
> > Propogation and retention are important for me cos I don't
> >have a fast connection,
>
> Likewise, just ISDN - single channel due to DEMON's artificial PCP
> limitations :(
Ditto. More change of William Hague being voted PM next year than me getting
ADSL, too. :-( Still, the main reason for staying with Demon is the fact
that I can get a compressed 128 connection if I need it (W2K) and I don't
get dumped off every two hours.
> > so I'm happy to pay almost as much for my news
> >service as I would for a decent ISP - same goes for web hosting.
>
> I have my own (virtual) server for that ;)
Ditto. It's my job. :-)
--
FM
FMURL
"Mike Pellatt" <news...@pellatt.co.uk> wrote
> >rather doubt they could afford to offer a full-on news service for 20
quid
> >for long once the DSL gang find their way on there...
>
> <fx> waves.
:-)
FMURL
> >If you're serious about Usenet, you should get a dedicated account.
Simple
> >as that. I doubt very much if any generic ISP has the money or the
> >inclination to invest in a full-on news service of the standard of, say,
> >Easynews or GigaNews (and many others besides). Usenet traffic is growing
> >exponentially, and normal ISPs can do very little to keep up with the
> >demand.
>
> There is a technically and practically sound reason why it would be much
> better for Demon to offer a proper news service - bandwidth. I have
> considered using one of the US-based services, but I would be connecting
> to this via my Demon account. This means more use of ongoing bandwidth
> when I (and others like me) want to access news by such routes.
I agree with John [Underwood] here.
> words, the relatively small investment required to provide a proper news
> service could considerably lighten the load caused by people downloading
> their binaries etc. from elsewhere.
Why does everyone make the assumption that it is a small investment? It's
not. Go check out the type of servers NewsFeeds www.newsfeeds.com are
running, for example.
> >The number of newsgroups available by any SP means very little. It's
> >retention and propogation that count.
>
> Agreed. What I mean is that if they are not going to offer decent
> retention they should not make claims about supporting vast number of
> newsgroups.
The point is, the number of groups they carry is pretty much irrelevant.
There are around 45-50K active groups. Claims over that are pointless.
> Personally, I think there might be a hidden agenda at Demon when it
> comes to news. There was that libel court case a while back which Demon
> defended on the principle of common carrier, and unfortunately lost. I
> think they are now deliberately avoiding carrying complete binaries, as
> often such material is of a dubious nature and they do not want to
> expose themselves to further lawsuits concerning either copyright or
> supposed public decency issues.
Doesn't seem to bother other providers at all, so why Demon?
--
FM
FMURL
> servers (static ip is useful for this), allows direct smtp and http
> (i.e no forced proxy) for 19.99 pcm instead.
>
> (And when you find them let us all know who they are.)
<fx: waves>
<aol>
ME TOO! ;)
</aol>
FMURL
> >I heard that Clara was struggling with news of late. I'm currently
testing
> >seven news providers at the moment, and nevr got as far as Clara after I
> >heard about the problems. Seems some people were taking advantage of the
>
> Normally excellent service but over the last couple of days I've been
> seeing several dozen of:
>
> News server unavailable : 400 System Capacity Exceeded (Typhoon v1.2.3)
Flying here at the moment. Flat out connections both ways.
--
FM
Alan Ford
>commercial offerings. Anything of that size is not likely to propogate well,
>better a message of around half that size (5000-7500 lines).
As we all know, however, Demon have a serious problem with this -- it has
been going on for EIGHTEEN MONTHS NOW! Yet they haven't done anything about
it. True, nothing can be guaranteed, but many servers do handle it alright.
>If you're serious about Usenet, you should get a dedicated account. Simple
>as that. I doubt very much if any generic ISP has the money or the
>inclination to invest in a full-on news service of the standard of, say,
>Easynews or GigaNews (and many others besides). Usenet traffic is growing
>exponentially, and normal ISPs can do very little to keep up with the
>demand.
Most other ISPs are doing very nicely thank you keeping up with news, it
seems... however most of those just take less binaries.
Demon used to keep up fine, and used to be one of the best newsfeeds
around... now there are even reports of missing text articles, which is
really worrying. The Borg was supposed to be scalable. Ooops.
The news providers offerings are looking very nice these days, and unlike
Demon they appear to be responsive to their customers, and are keeping
their hardware up to speed and scalable. Which is nice.
>The number of newsgroups available by any SP means very little. It's
>retention and propogation that count.
Indeed, especially as things like Altopia carry irrelevant groups --
anything mentioned in an x-post is created.
>On a personal note: I'd gladly pay more oney per month to Demon for an
>improved news service because all the best servers are in the States. :-(
That should act as an incentive to Demon to improve, however --
transatlantic bandwidth, while cheaper than it was, is still a fair chunk
of costs. The idea behind the web cache is to reduce load on this, and
ISP's newsservers act in a similar capacity -- act as a local cache of
the articles on Usenet. This saves Demon having to use external bandwidth
to supply their customers with news. This should be especially relevent
now with the unlimited access.
--
Alan Ford * al...@whirlnet.co.uk * http://www.whirlnet.co.uk/
PGP Key: 0x8F807D7D - email p...@whirlnet.co.uk or see keyservers
Demon Newsgroups Info + FAQs: http://www.whirlnet.co.uk/demon/
Alan Ford
<ne...@the-underwoods.org.uk> wrote:
>If, as a result of your pressure, Demon were to provide a service
>comparable with the Clara one, then I expect they would have to charge
>for that. Were that charge to be part of the SDU fee I would object very
>strongly. As it is, you are not comparing like with like.
Indeed. They could keep binaries separate, and people would pay an extra
fiver a month or something to get excellent access to those groups. For
the rest of us, who just use Usenet for what it was designed for --
text -- the basic text-only offering should remain part of the standard
service. But I don't see it as being economically efficient to provide
the binaries server as an extra, TBH. There is a lot of hardware costs
and I don't see that many people (no more than a few thousand) wanting
it... probably wouldn't break even.
Alan Ford
>Are there any Thus shareholders in here that could ask the board for
>more details of the problem, and how they intend to fix it, at the next
>AGM?
OTOH I can't help wondering if everbody who was concerned with this
made an effort to write letters of complaint to somebody high up,
somebody would notice the amount of complaints and may prod somebody
lower down at Demon to do something about it.
If, say, about a hundred letters could be organised, that could have
quite an effect, I would have hoped.
John Underwood
(Reference: <978649565.4161.3...@news.demon.co.uk>)
>I agree with John [Underwood] here.
With what that I said, you haven't quoted me at all in your article. I'm
flattered, but I'd like to know what your are holding me responsible for
- your final comment isn't quite in line with anything I said.
Stuart Millington
<den...@pickaxe.demon.co.uk> wrote:
>On Thu, 04 Jan 2001 03:05:15 +0000, Stuart Millington
><ne...@dsv1.co.uk> wrote:
>
>> Likewise, just ISDN - single channel due to DEMON's artificial PCP
>>limitations :(
>So why not sign up with another ISP that offers 128K bonded ISDN G5 24
ML-PPP not bonding ;)
>* 7 with no useage limits or timeouts, allows you to run your own
>servers (static ip is useful for this), allows direct smtp and http
>(i.e no forced proxy) for 19.99 pcm instead.
Who said I was asking for it at 19.99 GBP pcm? For work 39.99 GBP
pcm would be acceptable given that we could saturate that when it's
up. For home use, I would probably be willing to pay 29.99 GBP pcm
since I would not be saturating both channels that much of the time,
but it would be very useful for larger downloads.[1]
Given that there would have to be a separate G5 number pointed at
Blue for this to work, AFAIK, it should not be *technically* difficult
to implement this.
Of course, it would be useful if BoD on Blue was consistently
reliable - it was not last year.
[1] No, I'm not suggesting multiple price points. Just that different
users, or types of users, would have differing view on acceptable
prices for this.
Stuart Millington
<ne...@the-underwoods.org.uk> wrote:
>With respect no-one is forced to use any news server. Demon provide one
>as part of the package. If you decide that it doesn't meet your needs,
>then you can choose to use another one. You still retain the choice, it
>is up to you whether you have news at all.
Demon claim to offer a "full news feed". They should provide it.
>If, as a result of your pressure, Demon were to provide a service
>comparable with the Clara one, then I expect they would have to charge
>for that. Were that charge to be part of the SDU fee I would object very
>strongly. As it is, you are not comparing like with like.
Rubbish. The Clara news server is provided as part of their
standard dial-up service, as is the borg.
Clara simply have an additional type of account which does not
allow dial-up access or web space, just remote news access. The fact
that I use that, rather than their dial-up account is irrelevant. The
news server is the same for both account types.
Stuart Millington
>OTOH I can't help wondering if everbody who was concerned with this
>made an effort to write letters of complaint to somebody high up,
>somebody would notice the amount of complaints and may prod somebody
>lower down at Demon to do something about it.
Oink, oink, flap, flap.
Or am I being too cynical about Demon's management?!?
Stuart Millington
>That should act as an incentive to Demon to improve, however --
>transatlantic bandwidth, while cheaper than it was, is still a fair chunk
>of costs. The idea behind the web cache is to reduce load on this, and
>ISP's newsservers act in a similar capacity -- act as a local cache of
>the articles on Usenet. This saves Demon having to use external bandwidth
>to supply their customers with news. This should be especially relevent
>now with the unlimited access.
Indeed, with Smurftime/xDSL and the growing use/size of the binary
ng's there has to be a point at which the broken borg/peering is
costing Demon more (in bandwidth charges) than it is gaining (by lack
of investment).
At the end of the day, Demon has to make profits for it's (or it's
parent company's) shareholders. The financial argument is going to be
more effective than any claims of lack of service. After all, broken
services have been ignored for years by Demon in the past.
FMURL
> >I agree with John [Underwood] here.
>
> With what that I said, you haven't quoted me at all in your article. I'm
> flattered, but I'd like to know what your are holding me responsible for
> - your final comment isn't quite in line with anything I said.
Apologies, it was a rushed comment. It was your paragraph beginning,
"Superficially...".
--
FM
FMURL
> >If you're serious about Usenet, you should get a dedicated account.
Simple
> >as that. I doubt very much if any generic ISP has the money or the
> >inclination to invest in a full-on news service of the standard of, say,
> >Easynews or GigaNews (and many others besides). Usenet traffic is growing
> >exponentially, and normal ISPs can do very little to keep up with the
> >demand.
>
> Most other ISPs are doing very nicely thank you keeping up with news, it
> seems... however most of those just take less binaries.
But that's simply not true, is it? Supply and demand. If there was no demand
for dedicated news providers, they wouldn't be there. Likewise web hosting
providers. Would anyone serious about web development use their ISP for
their web developent projects? Unlikely in all but a few cases I feel.
Most people might *think* their ISP is keeping up with news - until they try
a dedicated news provider...
> Demon used to keep up fine, and used to be one of the best newsfeeds
> around... now there are even reports of missing text articles, which is
> really worrying. The Borg was supposed to be scalable. Ooops.
Indeed they did. They had an enviable reputation for newsfeeds in the UK,
but they never bothered to keep it up. Personally, I think the explosion of
users cable/dsl connections has a lot to do with it. It's one thing to go
from 28/33 to 56K, but entirely another thing to go from 56K to 512/256. One
of the biggest problems with news at the moment is users with newly aquired
cable/dsl connections flooding Usenet simply because they can (much
discussed by users and providers over in
alt.binaries.news-server-comparison).
> The news providers offerings are looking very nice these days, and unlike
> Demon they appear to be responsive to their customers, and are keeping
> their hardware up to speed and scalable. Which is nice.
There are some very professional outfits providing single user news at the
moment, but even that are being caught out by the level of traffic. I think
it was one of the SuperNews guys posted here earlier - 200GB per day.
> >The number of newsgroups available by any SP means very little. It's
> >retention and propogation that count.
>
> Indeed, especially as things like Altopia carry irrelevant groups --
> anything mentioned in an x-post is created.
>
> >On a personal note: I'd gladly pay more oney per month to Demon for an
> >improved news service because all the best servers are in the States. :-(
>
> That should act as an incentive to Demon to improve, however --
> transatlantic bandwidth, while cheaper than it was, is still a fair chunk
> of costs. The idea behind the web cache is to reduce load on this, and
> ISP's newsservers act in a similar capacity -- act as a local cache of
> the articles on Usenet. This saves Demon having to use external bandwidth
> to supply their customers with news. This should be especially relevent
> now with the unlimited access.
I'd like to see Demon improve both the news service *and* transatlantic
bandwidth, especially, as you say, in light of unmetered access. Indeed, I
feel they have a duty to their customers to do so, and *that's* how we
should be lobbying them. Not just bitching about stuff.
--
FM
James Coupe
Millington <ne...@dsv1.co.uk> writes
> Given that there would have to be a separate G5 number pointed at
>Blue for this to work, AFAIK, it should not be *technically* difficult
>to implement this.
IIRC, Blue was originally part of the Red ROMP. Red is on Energis
lines, not Thus lines. (The phone number both being 0845 079 tends to
confirm my recollections.)
AIUI, Demon's current system works by taking Surftime calls off onto the
Thus network from BT, rather than going to BT not-modems etc. I would
have thought that taking BT calls off onto Energis would require
significantly more work from Demon, since they would find it much harder
to move the calls there and would require more investment (in time and
money) in infrastructure.
Whilst it would not be "technically" difficult, necessarily (since they
know how to do it), it would seem to require rather a lot of work.
--
James Coupe | PGP Key 0x5D623D5D
"What's wrong with wanting more?
If you can fly -- then soar!
With all there is, why settle for just a piece of sky?"
Stuart Millington
wrote:
>> Given that there would have to be a separate G5 number pointed at
>>Blue for this to work, AFAIK, it should not be *technically* difficult
>>to implement this.
> IIRC, Blue was originally part of the Red ROMP. Red is on Energis
> lines, not Thus lines. (The phone number both being 0845 079 tends to
> confirm my recollections.)
The physical ML-PPP kit was part of Red after the initial beta
testing. As you say, the phone numbers imply the same teleco for the
current set-up.
> AIUI, Demon's current system works by taking Surftime calls off onto the
> Thus network from BT, rather than going to BT not-modems etc. I would
> have thought that taking BT calls off onto Energis would require
> significantly more work from Demon, since they would find it much harder
> to move the calls there and would require more investment (in time and
> money) in infrastructure.
Since there are a - relatively - small number of lines into Blue[1]
the whole thing could be moved to the Thrush network.
> Whilst it would not be "technically" difficult, necessarily (since they
> know how to do it), it would seem to require rather a lot of work.
Hence the word "technically" in my original comment ;)
[1] Unless, Demon have overcome the persistent scaleing(sp?) problems
and significantly increased it's size without telling anyone.
Mike Pellatt
<ne...@dsv1.co.uk> wrote:
>On Thu, 4 Jan 2001 13:54:44 +0000, John Underwood
><ne...@the-underwoods.org.uk> wrote:
>
>>With respect no-one is forced to use any news server. Demon provide one
>>as part of the package. If you decide that it doesn't meet your needs,
>>then you can choose to use another one. You still retain the choice, it
>>is up to you whether you have news at all.
>
> Demon claim to offer a "full news feed". They should provide it.
<panto>
Oh, no they don't.
</panto>
What they offer is "access to Usenet". This was made quite clear
at the time of one of the GNSNAFUs.
This, of course, means that Demon don't have any commitment to
provide any news server themselves at all.
--
Mike Pellatt
Andy
<ne...@the-underwoods.org.uk> wrote
>On Thu, 4 Jan 2001 at 19:12:08, Andy wrote in demon.service
>(Reference: <IPACKTBI...@kitzbuhel.demon.co.uk>)
>
>>Um, yes; but if they more clearly stated *what they were* it would save
>>a considerable amount of time-wasting speculation!
>
>If you believe that of demon.service you are either very naive or have
>an incredibly generous nature.
>
Mark Chater
wrote:
>What they offer is "access to Usenet". This was made quite clear
>at the time of one of the GNSNAFUs.
Excuse my ignorance, but what is a GNSNAFU? Specifically the "GN"
part of it?
-
Mark @ work
http://www.onomatopoeia.plus.com
- motorsport in Hillman Imps and derivatives, plus other errant nonsense
Andy
<fm...@yahoo.com> wrote
>"Keith Johnson" <ke...@xentrik.demon.co.uk> wrote
[]
>> Personally, I think there might be a hidden agenda at Demon when it
>> comes to news. There was that libel court case a while back which Demon
>> defended on the principle of common carrier, and unfortunately lost. I
>> think they are now deliberately avoiding carrying complete binaries, as
>> often such material is of a dubious nature and they do not want to
>> expose themselves to further lawsuits concerning either copyright or
>> supposed public decency issues.
>
>Doesn't seem to bother other providers at all, so why Demon?
>
rodg...@my-deja.com
John Underwood <news...@the-underwoods.org.uk> wrote:
> On Thu, 4 Jan 2001 at 01:43:49, Stuart Millington wrote in
> demon.service
> (Reference: <4sj75tchg82lpi6rl...@4ax.com>)
>
> >However, Clara charge 20 GBP p/a for remote news access - which many
> >Demon customers are forced to use since Demon's peering is fucked.
> With respect no-one is forced to use any news server. Demon provide
> one as part of the package. If you decide that it doesn't meet your
> needs, then you can choose to use another one. You still retain the
> choice, it is up to you whether you have news at all.
But what if the newsfeed formed part of the reason for signing up with
Demon in the first place. When I first signed up with Demon a few years
ago it was because their advertising made such a big deal over their
complete uncensored newsfeed. That newsfeed was fine till early last
year when it now appears to have become an unsupported "extra" on your
SDU. Plus if it is part of the SDU package as you say then surely if
it's performance is dropping then one has the right to complain about
it. One would complain if one only recieved 70% of emails sent to their
Demon address.
> If, as a result of your pressure, Demon were to provide a service
> comparable with the Clara one, then I expect they would have to charge
> for that. Were that charge to be part of the SDU fee I would object
> very strongly.
I always though the newsfeed was part of the SDU anyway. You could
always "object very strongly" for paying for something all this time
that *you* don't use or want. ;)
Would you object to Demon offering packages that for an extra sum, a
decent newsfeed would be provided to those that want one. As it stands
I am paying $9 per month on top of my Demon TAM for a very excellent
newsfeed from Giganews. Could/would Demon be able to offer a comparable
newsfeed for that sort of figure on top of a TAM for those that want it?
Sent via Deja.com
http://www.deja.com/
Stuart Millington
><ne...@dsv1.co.uk> wrote:
>> Demon claim to offer a "full news feed". They should provide it.
>
><panto>
>Oh, no they don't.
></panto>
<panto>
Oh, yes they do.
</panto>
http://www.demon.net/helpdesk/products/news/
Quote: "Demon Internet provides free access to a full feed of all the
different newsgroups (currently over 25,000), covering almost anything
you can think of."
They do claim to offer a full feed.
Denis Mcmahon
Chater) wrote:
>On 5 Jan 2001 06:28:58 GMT, news...@pellatt.co.uk (Mike Pellatt)
>wrote:
>
>>What they offer is "access to Usenet". This was made quite clear
>>at the time of one of the GNSNAFUs.
>
>Excuse my ignorance, but what is a GNSNAFU? Specifically the "GN"
>part of it?
Great News SNAFU.
Mike Pellatt
<ne...@dsv1.co.uk> wrote:
>On 5 Jan 2001 06:28:58 GMT, news...@pellatt.co.uk (Mike Pellatt)
>wrote:
>>On Fri, 05 Jan 2001 00:42:13 +0000, Stuart Millington
>><ne...@dsv1.co.uk> wrote:
>
>>> Demon claim to offer a "full news feed". They should provide it.
>>
>><panto>
>>Oh, no they don't.
>></panto>
>
><panto>
>Oh, yes they do.
></panto>
>
>http://www.demon.net/helpdesk/products/news/
>
>Quote: "Demon Internet provides free access to a full feed of all the
>different newsgroups (currently over 25,000), covering almost anything
>you can think of."
So, show me in that statement where it says that this full feed will
be on one of their servers ?? I (were I still a Demon customer) could
get free access to the Claranet news feed. Sure, I'd have to pay for
that feed, but my access to it would be free. Or, I could get access
to that German one, except that doesn't meet the spec since it
doesn't have binaries.
Yes, I know this unbelievably pedantic and pathetic. But this is the
state the argument got into in the days of Refund Richard, and the
position outlined above is the one Demon took. I have no reason (least
of all an announcement that news is now a Supported Service) to
believe that this position has changed.
One of the reasons that I, my company, and a charity left Demon ??
You guess.
> They do claim to offer a full feed.
I won't do the panto bit again....
--
Mike Pellatt
Mike Pellatt
<rodg...@my-deja.com> wrote:
>
>But what if the newsfeed formed part of the reason for signing up with
>Demon in the first place. When I first signed up with Demon a few years
>ago it was because their advertising made such a big deal over their
>complete uncensored newsfeed. That newsfeed was fine till early last
>year when it now appears to have become an unsupported "extra" on your
>SDU.
You are Refund Richard AICMFP.
--
Mike Pellatt
David Lawson
>On Thu, 4 Jan 2001 at 01:43:49, Stuart Millington wrote in demon.service
>(Reference: <4sj75tchg82lpi6rl...@4ax.com>)
>
>>Demon customers are forced to use since Demon's peering is fucked.
>With respect no-one is forced to use any news server. Demon provide one
>as part of the package. If you decide that it doesn't meet your needs,
>then you can choose to use another one. You still retain the choice, it
>is up to you whether you have news at all.
>
>comparable with the Clara one, then I expect they would have to charge
>for that. Were that charge to be part of the SDU fee I would object
>
I have a saved message in this group from 1996 in which Giles Todd and
Malcolm Muir tried to explain the whole business of news being a bonus
rather than a right.
Quoth Malcolm:
"The reason for the statement the news is unsupported is more related
to the nature of news on the internet (no one 'owns' news, no one can
guarantee that there will be news etc.) than the provision of a news
server."
--
David Lawson - London
www.davidlawson.co.uk
DaveN
FMURL <fm...@yahoo.com> wrote:
>"Alan Ford" <al...@whirlnet.co.uk> wrote:
[...]
>> The news providers offerings are looking very nice these days, and unlike
>> Demon they appear to be responsive to their customers, and are keeping
>> their hardware up to speed and scalable. Which is nice.
>
>There are some very professional outfits providing single user news at the
>moment, but even that are being caught out by the level of traffic. I think
>it was one of the SuperNews guys posted here earlier - 200GB per day.
This raises an interesting point, to my mind. It sets me to wondering
where the global "internet" is heading in the future, especially in
relation to the provision of news. Is the global community going to have
to take steps to limit the resources expended on transferring news? Is
the present news-peering system the most efficient method in terms of
the use of bandwidth, on a global scale?
I know that predicting the imminent death of the internet is a well-worn
path but *if* the exponential growth in newsgroup traffic is outpacing
the growth in server capacity and speed, then a number of options become
apparent (to me). The imperative for ISPs, I suspect, will be to limit
their use of external bandwidth as much as possible.
Firstly, the simple option is to give up the battle entirely and not
provide a news server.
Secondly, providers could move towards the bulletin-board systems of old
where the postings were accepted only from, and available only to,
subscribers to their own server. (Or even FIDONET, anyone?)
Third (ly), providers could offer just a limited number of text only
newsgroups which were clearly predefined and published. This might be
very attractive to some in view of the growing vulnerability of ISPs to
legal sanctions being imposed for objectionable material in newsgroups.
Are there any other possibilities I've overlooked? News being available
only online via http to a limited number of sites via proxy, perhaps?
--
DaveN
Stuart Millington
wrote:
>>http://www.demon.net/helpdesk/products/news/
>>
>>Quote: "Demon Internet provides free access to a full feed of all the
>>different newsgroups (currently over 25,000), covering almost anything
>>you can think of."
>
>So, show me in that statement where it says that this full feed will
>be on one of their servers ?? I (were I still a Demon customer) could
In the "further information" section of the page, the whole of
which would be considered "the statement" and the paragraph I quoted
was just one bit of it. The "further information" detail pages
implicitly(?) refer to the borg, not a 3rd party server.
Ultimately, Demon will re-interpret their own statements however
they wish, to come up with the interpretation they want.
This will never be fixed because no-one (in a position of power) at
Demon gives a shit, or cares about providing the quality of service
which is expected/promised. Now where was that SLA? :(
--
------------------------------------------------------------------
John Underwood
(Reference: <9348lf$77s$1...@nnrp1.deja.com>)
>But what if the newsfeed formed part of the reason for signing up with
>Demon in the first place.
In that case, they are failing to provide what you saw as the service
for which you signed up. If that formed part of the contract, then you
might be able to sue for compensation for non-delivery, but I very much
doubt whether you would get very far with that. If it is an essential
part of the service you need then you could go somewhere else, one of
the options available to you could be Clara, but that is not a matter on
which you are forced.
As to a question you asked later, I would not object to any service
Demon provided, whether or not the made a specific charge for it. If it
impinged sufficiently on the service they give me I would consider
whether to stay and make representation to them as part of the
preparation for such a decision.
John Underwood
(Reference: <vajb5t45f23vi3bqc...@4ax.com>)
>That is true in my case too. For me the determining factors were the
>fixed IP address and the full newsfeed.
Suppose, hypothetically, that Demon were to announce the cessation of
all but a very restricted news feed. What would you do? Go elsewhere and
find a dynamic IP or stay for the fixed IP and pay for a full news feed?
Wm ...
o.uk> demon.service Col Morrison <ne...@air.demon.co.uk>
>To announce that
>they no longer intended to carry some or all binary newsgroups would
>be a more proper way to proceed.
I'm starting to think this would be preferable to text only ngs not
getting full feeds. I don't like the idea but ....
>Having said that, cock-up seems more likely than conspiracy, as ever.
Probably. But what was the cock-up? The "full feed"?
--
Wm...
address valid for at least 31 days from date of posting
Richard Clayton
<ke...@xentrik.demon.co.uk> writes
>By contrast, if Demon provided a better news service in house, everyone
>could use it with zero additional demand on external bandwidth. In other
>words, the relatively small investment required to provide a proper news
>service could considerably lighten the load caused by people downloading
>their binaries etc. from elsewhere.
just as a matter of interest, how much do you think a "proper news
service" might cost ?
also, in passing, could you provide your estimate of the cost of
"external bandwidth"
I'm interested to see how these sums work
--
richard writing to inform and not as company policy
"Assembly of Japanese bicycle require great peace of mind" quoted in ZAMM
Wm ...
>I don't quite see how you can extrapolate fears over something as
>vague as 'public decency issues' from something as specific as a libel
>case, but if that were so they'd simply remove the supposedly
>sensitive groups entirely.
I think you should think about that a bit more.
Try http://www.iwf.org.uk/ and related links; take a look at what they
think about it.
C. Newport
>
> In article <nru85t0tk3ud7svlq...@4ax.com>, Keith Johnson
> <ke...@xentrik.demon.co.uk> writes
>
> >By contrast, if Demon provided a better news service in house, everyone
> >could use it with zero additional demand on external bandwidth. In other
> >words, the relatively small investment required to provide a proper news
> >service could considerably lighten the load caused by people downloading
> >their binaries etc. from elsewhere.
>
> just as a matter of interest, how much do you think a "proper news
> service" might cost ?
>
Let's see - We'll need a proper SLA with 99.99 percent availability.
A clustered set of Starfire servers fully configured
with a couple of racks of A5100 arrays should just about cope.
You should probably budget to upgrade to Serengeti within 2-3
years.
Then you will need about 8 or 10 Mbps dedicated bandwidth to
each of at least 3 news peers on a fully redundant network, plus
an upgrade to the UPS and the aircon. Floor loading may be
an issue as well .....
Subject to a detailed investigation I reckon I could make a good
business case if sales could find a way to double the number of
TAM users on the basis of the project. <B-(.
Oook !
Call me for a quote ?.
Maybe it would be a better idea to just drop the binary newsgroups
and let the Borg live for another few years.
--
Two things are infinite: the universe and human stupidity; and I'm
not sure about the universe. [Albert Einstein].
James Coupe
<ne...@the-underwoods.org.uk> writes
>Go elsewhere and find a dynamic IP or stay for the fixed IP and pay for
>a full news feed?
You could, potentially, go elsewhere and get both static IP and a full
news feed.
Stuart Millington
<ric...@highwayman.com> wrote:
>just as a matter of interest, how much do you think a "proper news
>service" might cost ?
Included (at no extra charge) with a Clara (or FS) dial-up account.
20-29 GBP p/a from Clara without a dial-up account.
>I'm interested to see how these sums work
I'm interested as to whether Demon will produce an official
statement why the borg has been fucked for the last year and what
plans, if any, they (the TPTB, not you personally) have to restore the
service to a working condition.
Indeed, a statement (official or otherwise) on why no Demon
representative has yet commented on the fucked borg/peering would be
welcome. Even if the statement is "We know it is fucked and don't
intend to fix it." would be better than the permanent silence here for
the last 6-12 months.
Although, I doubt that you are allowed to make such a statement
without permission from the clueless PTB - catch 22?
PS: highwayman.com - from the TV program of the 80's?
Ben Newsam
>What I mean is that if Demon only offers decent retention and completion
>on say 10,000 non-binary newsgroups they should say this, rather than
>claiming to offer a "full news feed" on XX,000 groups.
One good question is "Does *anyone* provide a full news feed of all
available newsgroups with no articles missing?" And if they do, who are
they? I certainly wouldn't want to try it myself, and even if I did, I
would make no guarantees of being able to provide such a service.
--
Ben
Ben Newsam
>Maybe it would be a better idea to just drop the binary newsgroups
>and let the Borg live for another few years.
Maybe it would, at that. The howls of anguish will be loud and long,
though. *All* binary newsgroups? What about alt.binaries.sounds.midi.cla
ssical? Or the PGP groups? Or quite a few other "legitimate" groups
carrying small binaries from time to time?
--
Ben
Andrew Gierth
Ben> One good question is "Does *anyone* provide a full news feed of
Ben> all available newsgroups with no articles missing?" And if they
Ben> do, who are they?
That is roughly what Altopia try and do, as I understand it.
Of course, from what their users are currently saying, they aren't
succeeding (as far as the 'no articles missing' bit goes, anyway).
--
Andrew.
Richard Clayton
Millington <ne...@dsv1.co.uk> writes
>On Sat, 6 Jan 2001 01:22:59 +0000, Richard Clayton
><ric...@highwayman.com> wrote:
>
>>just as a matter of interest, how much do you think a "proper news
>>service" might cost ?
>
> Included (at no extra charge) with a Clara (or FS) dial-up account.
>20-29 GBP p/a from Clara without a dial-up account.
You misunderstood the question -- the claim was that siting kit at Demon
was an obvious thing to do because of the saving that would be made in
external bandwidth
I was not asking what price marketing people had put on a service. I
learnt several decades ago that price and cost seldom have anything to
do with each -- and in the ISP industry the tenuous link that used to
apply has been completely broken.
There are people who read Demon service who have some idea what sort of
price needs to be paid for (a) external bandwidth and (b) the sort of
kit needed to handle the newsfeed and distribute it to tens of thousands
of users ... I thought it would be interesting to see these figures
>PS: highwayman.com - from the TV program of the 80's?
no
Andy
<b...@microser.demon.co.uk> wrote
>
>One good question is "Does *anyone* provide a full news feed of all
>available newsgroups with no articles missing?" And if they do, who are
>they? I certainly wouldn't want to try it myself, and even if I did, I
>would make no guarantees of being able to provide such a service.
Ask GCHQ where they get their from!
James Murray
>The idea behind the web cache is to reduce load on this, and
>ISP's newsservers act in a similar capacity -- act as a local cache of
>the articles on Usenet. This saves Demon having to use external bandwidth
>to supply their customers with news.
I don't agree. The web cache is just that and stores recently used data and
will without doubt reduce external bandwidth. News, however is far closer to
a mirror (of multiple sources) and naturally requires huge storage space.
Without any technical data to back it up, I can't believe that Demon storing
a (nearly) full feed of news will use less bandwidth than for example keeping
just text groups and users retrieving binary articles themselves.
Although I can't see Demon ever doing it, maybe they could stop storing any
binary groups but build an nntp cache and have an arrangement to pull these
groups from another source. Maybe as someone mentioned, the whole news
propagation idea needs to be revisited. The binary groups are obviously high
bandwidth. _If_ they are also low user, then a move towards fewer servers
storing these groups and other servers accessing them via nntp caches
would reduce the overall storage and bandwidth required by each ISP and
could improve the service delivered to users. This would require collaboration
between potential business rivals which may well be the largest stumbling
block. Maybe Demon could give up on binaries and use Giganews for that feed?
jsm
C. Newport
OK, let's look at the problem in more detail :-
1) The customer-facing news servers seem to be coping OK, but
the net-facing side seems to be backlogged.
2) Most upstream sites put very short expires on binary groups
( many just drop them ) so a badly backlogged server
it likely to miss articles. This would be most obvious
with multipart messages.
3) A quick sampling of headers in a selection of groups indicates
that Demon is accepting peer connections from too many
sources. This seriously eats resources to little benefit.
4) Usenet was never designed as a binary distribution medium, and
is the least efficient and most expensive method of
binary distribution.
5) Abuse of binary newsgroups to distribute illegal material is
widespread, and it is clear that expensive legal actions
will be brought against random ISPs by offended parties.
I do NOT have any inside knowledge of Demon's systems, but the
following conclusions have a high probability of being correct :-
1) Improvements to the net-facing servers only would probably
help, at less cost than a major upgrade.
2) Cutting the number of peers to 3 or 4 well-connected sites
would probably resolve the immediate problem at little
or no cost.
3) The general concensus among ISPs is that binary newsgroups are
simply unsustainable, so upstream unreliability is
increasingly becoming a fact of life.
4) Peering with a large number of sites to avoid 3) is extremely
expensive and doomed to failure as more ISPs drop support
for binary groups.
5) It MIGHT be possible to make a business case for a wholesale
news peering hub service if enough ISPs could be
persuaded to subscribe. A "hub and spoke" architecture
would be much more efficient than the current mess and
could offer cost savings to subsribing ISPs.
It seems clear to me that binary newsgroups are simply more
trouble than they are worth. Would it be possible to do some
statistics on the client-facing servers to find out what
proportion of customers actually use them ?. Are we talking
about offending a large number of customers or just a tiny
but noisy minority ?.
Is it possible to justify a business case for a solution which
includes binary newsgroups ?. If not how can we justify
degrading the mainstream usenet service for everyone in a futile
attempt to please all of the people all of the time.
I have been a Demon customer since very close to the beginning, and
lived through the good times and the bad times. A potential spend
of £10M ( My guess ) on a project that is probably doomed from the
start by upstream problems beyond Demon's control seems like the
kind of folly which could kill a good company in the current
marketplace.
Ten years ago a 'full and uncensored' service for a tenner a month
was an excellent objective. Unfortunately usenet has grown faster
than any reasonably scaleable implementation and we are faced with
a choice. Either pay more or get a little less for the same money.
When the 'little less' is something that I do not need then my
choice is clear. Some will disagree.
John Underwood
(Reference: <nTXTkJAU...@microser.demon.co.uk>)
>What about alt.binaries.sounds.midi.cla ssical?
What? Listen to serious music on my computer speakers with the sound of
the fans in the background, I'd rather go to a concert [1]. Indeed,
before SurfTime, it would have been cheaper to go to a concert.
(And that is before I start asking my friends what they think of it - I
mean those who try to earn a living by performing music).
[1] Since I usually prefer going to a concert, indeed prefer to perform
in one before merely attend, than listen to any sort of recording, such
a group doesn't interest me.
John Underwood
(Reference: <kPWkqEAnkuV6Ewc$@clara.net>)
>
>Why don't you do a deal with another supplier it must be easier than
>trying to do it yourself (you only have to look at the incomplete
>binaries)
I have been watching this discussion from the side-lines, not being
particularly interested in binary groups, but do have an interest in
groups which seem to be missing a few postings [1]. From what has been
said here, there doesn't seem to be any news provider which isn't having
similar problems. Who would you suggest should be the other supplier
and, if it turned out to be more expensive than would be the cost of
continuing as at present, then how would that be financed?
What proportion of Demon customers use usenet at all? How many of these
subscribe to non Demon groups? How many subscribe to binary groups?
One possible conclusion from the answers to those questions might be [2]
that the cost of provision of Usenet for all but Demon groups is
disproportionate to the requirement of the users.
[1] And wonder if those would have arrived if the binaries weren't
taking up the bandwidth.
[2] Note I say "might be", not "is", we don't have the facts either way.
John Underwood
(Reference: <2001Jan6.1...@jsm-net.demon.co.uk>)
>This would require collaboration between potential business rivals
>which may well be the largest stumbling block.
If it were restricted to binary groups, it might be very attractive. If,
instead of each ISP holding its own copies of material, much of which,
potentially, could cause serious legal problems (copyright, pornography
...) then I could well imagine many ISPs feeling that dumping it on to a
resource with shared funding would share the responsibility for meeting
the cost of such a risk.
On the other hand, if a court were to uphold a claim for damages against
such an organisation, its managers might be constrained to avoid that
sort of risk in the future and, since a monopoly had, in effect, been
created, one court case decided against the "common carrier" principle
would lead to inevitable censorship.
Richard Tobin
Perhaps the security services could offset their cost by selling full
news feeds?
(Or for the paranoid, read http://www.clara.net/business/rip.html
and then consider who else might consider it worthwhile offering such
a "secure" offshore service...)
-- Richard
--
Insert witty comment here
FMURL
> >The point is, the number of groups they carry is pretty much irrelevant.
> >There are around 45-50K active groups. Claims over that are pointless.
>
> What I mean is that if Demon only offers decent retention and completion
> on say 10,000 non-binary newsgroups they should say this, rather than
> claiming to offer a "full news feed" on XX,000 groups.
I rather doubt that Demon will make *any* claims about retention,
propogation or completion, sadly...
--
FM
FMURL
> >>On a personal note: I'd gladly pay more oney per month to Demon for an
> >>improved news service because all the best servers are in the States.
:-(
>
> News-service.com are very good, and based in Holland I believe.
Really? I love to see anyone else's comments on this - their server spcs in
the FAQ section don't inspire a great deal of confidence...
--
FM
Andy
<ric...@highwayman.com> wrote
[]
>There are people who read Demon service who have some idea what sort of
>price needs to be paid for (a) external bandwidth and (b) the sort of
>kit needed to handle the newsfeed and distribute it to tens of thousands
>of users ... I thought it would be interesting to see these figures
>
But aren't there people employed at Demon who should know?
John Underwood
(Reference: <978805468.29432.0...@news.demon.co.uk>)
>Perhaps the security services could offset their cost by selling full
>news feeds?
That could be useful, especially if they can keep a record of who
accesses what.
Denis Mcmahon
wrote:
>In article <978661743.11424.1...@news.demon.co.uk>,
>FMURL <fm...@yahoo.com> wrote:
>>"Alan Ford" <al...@whirlnet.co.uk> wrote:
>[...]
>>> The news providers offerings are looking very nice these days, and unlike
>>> Demon they appear to be responsive to their customers, and are keeping
>>> their hardware up to speed and scalable. Which is nice.
>>
>>There are some very professional outfits providing single user news at the
>>moment, but even that are being caught out by the level of traffic. I think
>>it was one of the SuperNews guys posted here earlier - 200GB per day.
>
>This raises an interesting point, to my mind. It sets me to wondering
>where the global "internet" is heading in the future, especially in
>relation to the provision of news. Is the global community going to have
>to take steps to limit the resources expended on transferring news? Is
>the present news-peering system the most efficient method in terms of
>the use of bandwidth, on a global scale?
If you define the following types of news traffic currently in the
system:
Real valid news articles
Spam
Spam cancels
Binary posts
I would imagine that Spam, Spam cancels and binaries account for a
significant portion of that 200 Gb a day.
If the spam detection could be carried out locally to each node, it is
possible that the spam-cancel traffic could be reduced, as could the
the spam traffic as the closer to source it is detected and killed the
lower the overall network load.
In addition, I get the impression that there is a lot of reposting of
binary articles due to missing parts. Some of this may be due to third
party cancels issued as a denial of service attack either maliciously
or to protect copyright (which is a valid cancel afair).
News, being a 7 bit method, is certainly not an effective method of
transmitting binaries, that's what ftp is for, and an ftp network with
mirrors would be a much more efficient method.
The problem is that the operators of the ftp sites and mirrors would
want assurances that they were not transmitting copyright, illegal or
malicious (ie containing virii) material, and the effort of validating
material at the insertion site to such a network would place a load on
the insertion site.
However, if you could reduce some other cost to the ISP, there may be
a worthwhile trade off.
Imagine, a single ISDN line can handle 8 Kbyte / sec, that's 480 Kbyte
/ minute, 28.8 Mbyte / hour, 691.2 Mbyte / day.
200 Gbyte / day is > 2.3 Mbyte / sec. An E1 can handle 2 Mbit / sec or
240 Kbyte / sec.
This means that a full news feed now occupies in the region of 18 Mbit
/ sec or 9 E1s.
Here goes with some suck it and see assumptions:
(1) The news feed required can probably be reduced to 1 or 2 E1s if
the binaries, spam and cancels can be significantly reduced.
(2) An ftp mirroring network might require an additional 2 E1s of
transmission.
That means that we may be able to save 5 E1s of bandwidth.
I would guess that in Demon's case this might be 2 E1s to the USA, 1
to the NL and 2 to the LInX.
I don't know what the cost of an E1 is across the pond, but I guess it
may be enough to justify the staff and / or resources to implement
such a scheme at demons end.
However, such a scheme is only going to work if a majority of ISPs
adopt it ......
...... and that's why, although it might save money, it's never going
to get off the ground.
>Firstly, the simple option is to give up the battle entirely and not
>provide a news server.
News is a valuable discussion medium which can not easily be replaced,
indeed replacing new with eg mailing lists would actually increase the
bandwidth used.
>Secondly, providers could move towards the bulletin-board systems of old
>where the postings were accepted only from, and available only to,
>subscribers to their own server. (Or even FIDONET, anyone?)
This is the way I believe most responsible ISP's and public news
server admins work, unfortunately it seems that there are ISPs and
news server operators out there who take what might be deemed a less
responsible attitude to ensuring users have a limited ability to screw
up, as well as those whose attitude towards spamming from their
servers is less aggressive than it could be.
There is also the issue of allowing anonymous access for people living
in restrictive regimes who might wish to make statements critical of
their governments or governmental systems but who might face some form
of reprisal for daring to think disloyal thoughts if their identities
were not hidden from those in power in those regimes - remember we are
discussing a global issue and therefore must consider all angles, and
not every government is enlightened enough to allow it's people to
openly criticise it as in most Western democracies.
>Third (ly), providers could offer just a limited number of text only
>newsgroups which were clearly predefined and published. This might be
>very attractive to some in view of the growing vulnerability of ISPs to
>legal sanctions being imposed for objectionable material in newsgroups.
The biggest problems exist in the unregulated hierarchies, where there
is nothing to stop anyone creating binaries groups. In addition, there
is the freedom of speech argument that some posters will maintain
entitles them to post anything they want anywhere they want. The troll
from uk.legal that some idiots in here insist on feeding everytime he
sticks his head round the door is a prime example.
Filtering and dropping binaries in all groups at the ISP news-servers
is probably the only realistic way of removing the volume of this
traffic, unfortunately I suspect that doing so will merely drive the
"underground" element of software pirates, paedophiles etc into
finding other methods of propogating their postings, and the load on
the net will not be significantly reduced.
Any ISP that employed such a technique would for example be
disqualified from taking the "freeuk" hierarchy which has a "no
moderation, anyone can create any group they like, no-one can issue
rmgroups" style of charter, although who would try and enforce the
issue I don't know, as ultimately as long as they didn't issue cancels
externally they would be within their rights.
>Are there any other possibilities I've overlooked? News being available
>only online via http to a limited number of sites via proxy, perhaps?
I don't know, the problem of binaries choking the feed is one aspect
that I believe should be dealt with by taking binaries out of the feed
- I firmly subscribe to the camp that newsgroups are an inappropriate
means of distributing binary files.
The spam issue will probably never go away, but I do think that if all
ISPs and news server operators actively lokked for spam at customer
connections it would help, if a spammer has to connect to multiple
news servers to send his spam spew it makes it harder.
I don't know if the IETF network security area are looking at
considering improved methods of handling spamspew as a denial of
service attack or not, but it may be an appropriate area to start
discussion.
As far as removing binaries from the news spool is concerned, we need
an alternative binary distribution method first, it may be that
binaries could be ditributed using multicast protocols on the mbone, I
don't know if that would mean you'd have to be logged on at the right
time to receive the files though.
It does seem to me that protocols designed for "broadcast" might be
better suited to distributing binaries than protocols designed for
many to many discussion.
Rgds
Denis
--
Denis McMahon
Mobile: +44 7802 468949
Email: de...@pickaxe.demon.co.uk
I always trim ng when posting!
Simon Slavin
Keith Johnson <ke...@xentrik.demon.co.uk> wrote:
> Personally, I think there might be a hidden agenda at Demon when it
> comes to news. There was that libel court case a while back which Demon
> defended on the principle of common carrier, and unfortunately lost. I
> think they are now deliberately avoiding carrying complete binaries, as
> often such material is of a dubious nature and they do not want to
> expose themselves to further lawsuits concerning either copyright or
> supposed public decency issues.
Nope, it works the other way around. If Demon say "We don't carry
a full news feed." questions can be asked about how the items they
carry are selected. If Demon say "We just put everything back out
that comes into us." then they are simply a 'channel' and can't be
accused of any type of selection. This brings their status closer
to that of a 'medium' or 'common carrier'.
By failing to carry every article available from peers, Demon are
exposing themselves to various attacks in court. If, for example,
they say "We are doing this so we don't carry a full feed of child
pornography." they can be attacked for every article of child porn
that they /do/ carry, on the grounds that they have no claim to be
a common carrier.
This means that Demon /can't/ announce any policy that states that
they carry news selectively. This, I think, may have something to
do with the fact that there has been no comment from Demon on the
fact that, in contradiction to at least one advert I've seen, they
don't carry a full news feed.
I have just been through the equivalent discussion with a library-
official: they /cannot/ have any policy by which books are stocked
besides that of satisfying customer-demand, since doing so would
open them to claims of censorship, being in the pay of publishers,
and having a politically-motivated policy on what their readership
has access to. Consequently, they will satisfy customer-demand by
subscribing to _The Sunday Sport_ instead of spending the same sum
of money on _Nature_.
Simon.
--
http://www.hearsay.demon.co.uk | Come to think of it, just what are we going
No junk email please. | to say to an alien race if we make contact?
| "Do you have Napster?"
| "Stop making crop circles!" -- Scott Barber
Andrew Gierth
Denis> If you define the following types of news traffic currently in
Denis> the system:
Denis> Real valid news articles
Denis> Spam
Denis> Spam cancels
Denis> Binary posts
Denis> I would imagine that Spam, Spam cancels and binaries account
Denis> for a significant portion of that 200 Gb a day.
spam cancels account for only a miniscule fraction of the volume, and
spam accounts only for a percent or so at most. Binaries (including
spammed binaries) accounts for more than 99% of the volume now.
Denis> In addition, I get the impression that there is a lot of
Denis> reposting of binary articles due to missing parts. Some of
Denis> this may be due to third party cancels issued as a denial of
Denis> service attack either maliciously
while this claim is frequently made, it has __NEVER__ been backed up
with facts, in the form of copies of the alleged cancel messages.
Denis> or to protect copyright (which is a valid cancel afair).
cancelling on grounds of copyright has been widely considered to be
off-limits for a couple of years now, thanks in large part to the Co$
Denis> News, being a 7 bit method, is certainly not an effective
Denis> method of transmitting binaries, that's what ftp is for, and
Denis> an ftp network with mirrors would be a much more efficient
Denis> method.
the figures for this kind of argument have been raised before and
generally found to be bogus. for example:
Denis> Here goes with some suck it and see assumptions:
[...]
Denis> (2) An ftp mirroring network might require an additional 2 E1s of
Denis> transmission.
is this number merely pulled from a hat?
Denis> Filtering and dropping binaries in all groups at the ISP
Denis> news-servers is probably the only realistic way of removing
Denis> the volume of this traffic, unfortunately I suspect that doing
Denis> so will merely drive the "underground" element of software
Denis> pirates, paedophiles etc into finding other methods of
Denis> propogating their postings, and the load on the net will not
Denis> be significantly reduced.
as long as the net exists there will be methods for people to
communicate things that other people might find undesirable.
Denis> Any ISP that employed such a technique would for example be
Denis> disqualified from taking the "freeuk" hierarchy
nonsense - there is nothing to prevent an ISP carrying free.* while
nevertheless filtering it for spam and binaries. Indeed there is no
shortage of sites that already do this.
--
Andrew.
Andrew Gierth
Simon> By failing to carry every article available from peers, Demon
Simon> are exposing themselves to various attacks in court.
you may have noticed that this strategy appeared to significantly
backfire on them in the recent past.
--
Andrew.
Jim Crowther
<den...@pickaxe.demon.co.uk> writes
>News, being a 7 bit method, is certainly not an effective method of
>transmitting binaries, that's what ftp is for, and an ftp network with
>mirrors would be a much more efficient method.
>
binaries are ok, but no article may be over 20kbyte in total. This
allows for small binaries for illustration etc, and concentrates the
mind on keeping article size small. Even some top-posting full-quoters
have mended their ways...
Any other binary has to be pointed to via a URL.
I have no idea if this sort of idea would be of value for Usenet at
large, but for the 50 or so ngs I lurk in, it would be seem useful.
--
Jim (Cruncher) Crowther "It's MY computer"
* It's our turn to help pchelp *
* <http://www.cozmikshirts.co.uk/rooms/pchelpers.shtml> *
* http://pchelpers.org http://www.cafepress.com/PCHelp *
James Coupe
<fm...@yahoo.com> writes
>I rather doubt that Demon will make *any* claims about retention,
>propogation or completion, sadly...
When cancels exist, as they do, it would be difficult to make any such
claims anyway.
A cancelled, or superseded, article could be retained for a matter of
seconds or, indeed, never offered to Demon at all.
Ben Newsam
>Elsewhere, on a private server (grc.com) an approach has been taken that
>binaries are ok, but no article may be over 20kbyte in total. This
>allows for small binaries for illustration etc, and concentrates the
>mind on keeping article size small. Even some top-posting full-quoters
>have mended their ways...
>
>Any other binary has to be pointed to via a URL.
>
>I have no idea if this sort of idea would be of value for Usenet at
>large, but for the 50 or so ngs I lurk in, it would be seem useful.
It wouldn't work. It is quite possible to split large binaries up into
separate posts of less than 20k.
--
Ben
Denis Mcmahon
<c...@NOSPAM.netunix.com> wrote:
>3) A quick sampling of headers in a selection of groups indicates
> that Demon is accepting peer connections from too many
> sources. This seriously eats resources to little benefit.
There may be space here for a review by demon of their connections,
however it may be that the result of that review is that to reduce the
number of peers may not solve any problems.
>5) Abuse of binary newsgroups to distribute illegal material is
> widespread,
Alleged abuse, as I don't take the binary groups I have no peronal
knowledge of such abuse, also in the case of multiparts I suspect that
there may be single part cancels issued to protect copyright (I
believe these are legitimate third party cancels).
> and it is clear that expensive legal actions
> will be brought against random ISPs by offended parties.
The lack of any such to date suggests that there may be less will to
do so on the part of those offended parties than you suggest.
>1) Improvements to the net-facing servers only would probably
> help, at less cost than a major upgrade.
Please detail your "improvements", this is like the box in a project
plan that equates to "a miracle happens"
>2) Cutting the number of peers to 3 or 4 well-connected sites
> would probably resolve the immediate problem at little
> or no cost.
Those peering arrangements may involve contractual agreements relating
to traffic in both directions, and removing them may require
negotiation of new arrangements for transit of news or other traffic
to the networks whose servers are no longer peered with.
Would you accept that it became impossible to send email between demon
and some other isp because demon had unilaterally cancelled a news
peering agreement.
>5) It MIGHT be possible to make a business case for a wholesale
> news peering hub service if enough ISPs could be
> persuaded to subscribe. A "hub and spoke" architecture
> would be much more efficient than the current mess and
> could offer cost savings to subsribing ISPs.
Are you familiar with the Fidonet model of "echomail" distribution,
which loosely corresponds to newsgroups.
You have a backbone across the continents, with a tree below each
backbone system which carries the bulk of the distribution, and then
create additional links where a "non backbone" group is wanted by
limited numbers of systems.
I believe that additional features of such a network could be spam
detection at each hub, to prevent downstream inserted spam propogating
beyond the hub into the rest of the network, and binary detection and
dropping at each hub (either strip out the attachment, or drop the
whole message), which would IMO significantly reduce the total traffic
load.
Fidonet also had a facility called file-echoes, which was designed for
the automated transfer of files in a similar way, the files ending up
in bbs download areas. I guess this is analogous to ftp mirroring,
which is the appropriate way for tramnsmitting binaries.
However, any "regulated" method of transmitting binaries is invariably
limited to legitimate binaries, and those who wish to distribute and
receive "illegitimate" binaries (by which I include all forms of
copyrighted material, material illegal by nature of it's content,
malware etc) will probably find some other method to do so which still
places a strain on network resources.
Denis Mcmahon
Categorically NO, as there are many news-servers that do not peer some
groups outside a limited "zone", whether that be an intranet, or an
isp and it's customers, or whatever.
As far as "all public newsgroups" is concerned, any news server can
only get messages in groups that one of it's peers receive, there are
also groups that have been rmgrouped but still appear on servers that
have not received or processed the rmgroup message, and the same
applies with the newgroup messages.
Denis Mcmahon
<ric...@highwayman.com> wrote:
>just as a matter of interest, how much do you think a "proper news
>service" might cost ?
>
>also, in passing, could you provide your estimate of the cost of
>"external bandwidth"
>
>I'm interested to see how these sums work
Perhaps you'd like to review my comments on the subject elsewhere in
this thread.
Denis Mcmahon
<and...@erlenstar.demon.co.uk> wrote:
> Denis> I would imagine that Spam, Spam cancels and binaries account
> Denis> for a significant portion of that 200 Gb a day.
>
>spam cancels account for only a miniscule fraction of the volume, and
>spam accounts only for a percent or so at most. Binaries (including
>spammed binaries) accounts for more than 99% of the volume now.
Okay, that gives a positive pointer to addressing an overloaded
network.
Given the earlier comment of 200Gb / day of news, I assume that the
valid non spam text traffic (what might be considered real ng
messages) is about 2Gb / day.
> Denis> In addition, I get the impression that there is a lot of
> Denis> reposting of binary articles due to missing parts. Some of
> Denis> this may be due to third party cancels issued as a denial of
> Denis> service attack either maliciously
>
>while this claim is frequently made, it has __NEVER__ been backed up
>with facts, in the form of copies of the alleged cancel messages.
Accepted.
> Denis> or to protect copyright (which is a valid cancel afair).
>
>cancelling on grounds of copyright has been widely considered to be
>off-limits for a couple of years now, thanks in large part to the Co$
Co$ ?????
> Denis> News, being a 7 bit method, is certainly not an effective
> Denis> method of transmitting binaries, that's what ftp is for, and
> Denis> an ftp network with mirrors would be a much more efficient
> Denis> method.
>
>the figures for this kind of argument have been raised before and
>generally found to be bogus. for example:
Are you suggesting that distribution by ftp with mirror sites would be
less efficient than newsgroup distribution for the legitimate
binaries?
> Denis> Here goes with some suck it and see assumptions:
> [...]
> Denis> (2) An ftp mirroring network might require an additional 2 E1s of
> Denis> transmission.
>
>is this number merely pulled from a hat?
200Gb / Day came out at about 7 E1s. My guesstimate is that the
legitimate binaries (as opposed to the spammed copies, copyright and
other material that might be considered undesirable due to content, eg
that carrying malware, or identified by the likes of IWF) is less than
25% of the total, 2 E1s would be sufficient, especially if a more
efficient protocol were used.
Is it your opinion that the legitimate binary traffic exceeds 50Gb a
day?
> Denis> Filtering and dropping binaries in all groups at the ISP
> Denis> news-servers is probably the only realistic way of removing
> Denis> the volume of this traffic, unfortunately I suspect that doing
> Denis> so will merely drive the "underground" element of software
> Denis> pirates, paedophiles etc into finding other methods of
> Denis> propogating their postings, and the load on the net will not
> Denis> be significantly reduced.
>
>as long as the net exists there will be methods for people to
>communicate things that other people might find undesirable.
I think I recognised that, one mans reasonable limit is another ones
censorship, and the net interprets censorship as damage and finds a
way to reroute the traffic etc .......
> Denis> Any ISP that employed such a technique would for example be
> Denis> disqualified from taking the "freeuk" hierarchy
>
>nonsense - there is nothing to prevent an ISP carrying free.* while
>nevertheless filtering it for spam and binaries. Indeed there is no
>shortage of sites that already do this.
I think I meant to add a "technically" in there.
DaveN
Andrew Gierth <and...@erlenstar.demon.co.uk> wrote:
>>>>>> "Denis" == Denis Mcmahon <den...@pickaxe.demon.co.uk> writes:
> Denis> Filtering and dropping binaries in all groups at the ISP
> Denis> news-servers is probably the only realistic way of removing
> Denis> the volume of this traffic, unfortunately I suspect that doing
> Denis> so will merely drive the "underground" element of software
> Denis> pirates, paedophiles etc into finding other methods of
> Denis> propogating their postings, and the load on the net will not
> Denis> be significantly reduced.
>
>as long as the net exists there will be methods for people to
>communicate things that other people might find undesirable.
I am out of my depth in "technical" matters but, nevertheless, I would
be very interested in your thoughts on the following:
1. Do you believe that the current degree of binary content in newsgroup
articles (90% by your figures), is leading to a significant or even
threatening overload of ISP bandwidth resources generally?
2. If so, how do you think the global community can mitigate the
effects?
--
DaveN
Phil Payne
> Hi All -
>
> Checked out a few binary newsgroups the other day when looking for some
> stuff. Was amazed to see that virtually every posting of more than 10000
> lines was incomplete.
It's not just the binaries - it's just more obvious there. When I
finally got pissed off and dumped Demon, I compared a few newsgroups
with their parallels at Freeserve and Tesco. Well, no comparison,
really - the others had about 40% more posts in the same retention.
--
Phil Payne
Phone +44 7785 302803 Fax +44 7785 309674
Richard Clayton
<an...@kitzbuhel.demon.co.uk> writes
>In article <Opog8pEa...@turnpike.com>, Richard Clayton
><ric...@highwayman.com> wrote
>[]
>>There are people who read Demon service who have some idea what sort of
>>price needs to be paid for (a) external bandwidth and (b) the sort of
>>kit needed to handle the newsfeed and distribute it to tens of thousands
>>of users ... I thought it would be interesting to see these figures
>>
>Yes.
>
>But aren't there people employed at Demon who should know?
yes, but the numbers are commercially confidential
however, if you work them out yourself (with assistance from others)
then you can see whether assertions such as "building local news servers
saves external bandwidth" are true when one tries to say that saving the
external bandwidth is actually cheaper. It appears to be "obvious" to
some people that this would be so -- I am interested in how much they
think bandwidth costs and how much they think servers cost.
However, it may not be interesting to anyone else - so you don't need to
do the sums if you don't wish to.
peter
>On Sat, 06 Jan 2001 13:28:34 +0000, "C. Newport"
><c...@NOSPAM.netunix.com> wrote:
>
<snip>
>>2) Cutting the number of peers to 3 or 4 well-connected sites
>> would probably resolve the immediate problem at little
>> or no cost.
>
>Those peering arrangements may involve contractual agreements relating
>to traffic in both directions, and removing them may require
>negotiation of new arrangements for transit of news or other traffic
>to the networks whose servers are no longer peered with.
>
I would not want to be downstream on a news feed from Demon.
God would I get some flak! Any downstream news severs (rather than
peer) would get a better feed if Demon improved theirs. Also what if
the peers have changed their retention and group lists since the
peering agreement was made? If one no longer gives an adequate feed
for a group which formed part of the basis of the peering contract,
are they not in breach of that contract? Also how do those ISP's with
peering arrangements with Demon feel - what was a good feed is now
crap. No doubt they would welcome the opportunity to drop Demon from
their peering arrangement.
>Would you accept that it became impossible to send email between demon
>and some other isp because demon had unilaterally cancelled a news
>peering agreement.
>
I would expect most would be happy to continue to exchange E-mail in
exchange for dropping a broken and worthless bandwidth sapping
newsfeed. They might be able to use the money saved by reducing the
bandwidth between them to increase the bandwidth to a better news
feed.
<snip>
Given that some UK ISP's do get a good completion on the news groups
they collect, including biniaries. Demon must be doing it very very
wrong. What else are they doing wrong?
--
Peter Hill
Can of worms - what every fisherman wants.
Can of worms - what every PC owner has.
Andrew Gierth
Denis> Given the earlier comment of 200Gb / day of news, I assume
Denis> that the valid non spam text traffic (what might be considered
Denis> real ng messages) is about 2Gb / day.
A feed with no binaries at all, filtered for spam, is significantly
less than 2GB per day, but I don't have accurate figures right now.
>> cancelling on grounds of copyright has been widely considered to be
>> off-limits for a couple of years now, thanks in large part to the Co$
Denis> Co$ ?????
Church of Scientology
--
Andrew.
Andrew Gierth
DaveN> I am out of my depth in "technical" matters but, nevertheless,
DaveN> I would be very interested in your thoughts on the following:
DaveN> 1. Do you believe that the current degree of binary content in
DaveN> newsgroup articles (90% by your figures), is leading to a
DaveN> significant or even threatening overload of ISP bandwidth
DaveN> resources generally?
It is the ISP's choice whether or not to devote resources to binary
content.
The decision of whether or not to run a news server is a tradeoff
between various factors: on the one hand you have the cost of the
server, the bandwidth to take in a feed, and the manpower to keep it
running; on the other hand if you outsource you potentially have
increased bandwidth costs, or if you simply fail to provide news at
all you may lose customers, or your customers who want news may buy
individual accounts (again, possibly costing more external bandwidth).
This tradeoff can be made at several levels, depending on the ISP's
estimate of what its customers require. For example it could run a
text-only feed (which consumes negligible bandwidth and requires a
much smaller commitment of server hardware); or it could carry a
binaries feed restricted to only single-part binaries (i.e. exclude
the videos, sounds, warez and cd-images groups completely, and filter
the rest of the feed to exclude multiparts). Either way some
proportion of the customers will decide this is insufficient for them
and will either buy individual subscriptions or move to another ISP.
--
Andrew.
Richard Clayton
<pe...@skyshack.demon.co.uk> writes
>Also what if
>the peers have changed their retention and group lists since the
>peering agreement was made? If one no longer gives an adequate feed
>for a group which formed part of the basis of the peering contract,
>are they not in breach of that contract?
why do you imagine that the swapping of news articles is done on the
basis of formal contracts ?
Denis Mcmahon
<ric...@highwayman.com> wrote:
>>Also what if
>>the peers have changed their retention and group lists since the
>>peering agreement was made? If one no longer gives an adequate feed
>>for a group which formed part of the basis of the peering contract,
>>are they not in breach of that contract?
>
>why do you imagine that the swapping of news articles is done on the
>basis of formal contracts ?
Okay, maybe not contracts, but I assume that there is some agreement
that might tie other issues in with the news peering, such as
transiting etc.
Bearing in mind that my experience is more telecomms than ISP, and
there is usually a formal agreement concerning which traffic may be
sent across an interconnect, both terminating in the connected
network, and transiting through it to another network.
Andrew Gierth
>> 2) Cutting the number of peers to 3 or 4 well-connected sites
>> would probably resolve the immediate problem at little or no cost.
Denis> Those peering arrangements may involve contractual agreements
news peering is very rarely done according to explicit contracts - the
general procedure is more like 'you send me yours and I'll send you
mine' with an understanding that either site is free to reconsider the
arrangement at any time. Explicit contract terms are only likely to be
used in the case of paid-for newsfeeds rather than peering.
but in fact reducing the number of peers does very little to improve
matters and if anything is more likely to make things worse
--
Andrew.
Denis Mcmahon
<and...@erlenstar.demon.co.uk> wrote:
>>>>>> "Denis" == Denis Mcmahon <den...@pickaxe.demon.co.uk> writes:
>
> Denis> Given the earlier comment of 200Gb / day of news, I assume
> Denis> that the valid non spam text traffic (what might be considered
> Denis> real ng messages) is about 2Gb / day.
>
>A feed with no binaries at all, filtered for spam, is significantly
>less than 2GB per day, but I don't have accurate figures right now.
I was combining your "1% of the feed" with an earlier posters quote
from another news server operator of "200 Gb / day" :-)
> >> cancelling on grounds of copyright has been widely considered to be
> >> off-limits for a couple of years now, thanks in large part to the Co$
>
> Denis> Co$ ?????
>
>Church of Scientology
Ah. That bunch of [insert an appropriate term here].
DaveN
Andrew Gierth <and...@erlenstar.demon.co.uk> wrote:
>The decision of whether or not to run a news server is a tradeoff
>between various factors: on the one hand you have the cost of the
>server, the bandwidth to take in a feed, and the manpower to keep it
>running; on the other hand if you outsource you potentially have
>increased bandwidth costs, or if you simply fail to provide news at
>all you may lose customers, or your customers who want news may buy
>individual accounts (again, possibly costing more external bandwidth).
So, in the simplest of terms which I hope doesn't oversimplify, the
commercial value of providing a "full" news feed (i.e., on the ISP's own
server) becomes a trade-off between the apportioned costs of doing so,
set against the number of potential subscribers and the amount they are
willing to pay.
Do you think that the overall cost of providing a "full" news feed on an
in-house serer is increasing significantly?
Would it be true to assume that there are self-limiting market forces at
work? If fewer people are demanding a "full" news feed and/or people are
less willing to pay, presumably the availability of such a service will
reduce to zero as costs/prices increase.
--
DaveN
C. Newport
> but in fact reducing the number of peers does very little to improve
> matters and if anything is more likely to make things worse
Could you explain this comment please.
Please note, however, that I have no internal knowlege of Demon's
specific problems. General principles almost always apply.
In me experience ( and conventional wisdom ) Assuming well-connected
peers:
1 will provide over 95% of articles with no resilience.
2 will provide about 98% of articles with resilience.
3 will fill in a few gaps but 98% of it's offerings will result
in fruitless history searches.
4 adds no real benefit for the cost of 100% extra history searches.
The only benefit of having more than 3 feeds in in propagation time,
some articles may arrive earlier via the extra route, but time
is not of the essence. The extra history searches ( one for
every article offered ) are expensive in CPU time and even more
expensive in disk I/O unless the entire history data is kept in
memory. This is a primary cause of backlogged servers.
If an article does not arrive from 3 well-connected feeds it is
unlikely ever to arrive.
--
Two things are infinite: the universe and human stupidity; and I'm
not sure about the universe. [Albert Einstein].
michael lefevre
news:PG3PSLAk...@charlecote.org.uk...
[snip]
> So, in the simplest of terms which I hope doesn't oversimplify, the
> commercial value of providing a "full" news feed (i.e., on the ISP's own
> server) becomes a trade-off between the apportioned costs of doing so,
> set against the number of potential subscribers and the amount they are
> willing to pay.
that sounds about right to me...
> Do you think that the overall cost of providing a "full" news feed on an
> in-house serer is increasing significantly?
well it has, i think, recently, and so i imagine it will continue to do
so... there was a discussion elsewhere which i followed (possibly the
globalnet support group, bicbw) around this time last year, at which point a
figures of 1Gb without binarieis and 50Gb with binaries were mentioned for
the size of a full news feed... that would seem to suggest that the volume
has increased by 300% over the last year, and that the increase is pretty
much entirely due to binaries... (i'm no expert, just going on what i've
read...)
[snip]
michael
Julie Brandon
>Maybe it would be a better idea to just drop the binary newsgroups
>and let the Borg live for another few years.
Seconded. Personally, I'd prefer it if the binary groups were dropped (&
also a filter on binaries in general) if that meant that handling
news would become cheaper for Demon.
Ta-ra,
--
Julie Brandon, Derby, UK
<URL:http://www.computergeeks.co.uk/>
+++ See homepage for details of my present E-Bay auctions +++
Andrew Gierth
> Andrew Gierth wrote:
>> but in fact reducing the number of peers does very little to
>> improve matters and if anything is more likely to make things
>> worse
C> Could you explain this comment please.
C> Please note, however, that I have no internal knowlege of Demon's
C> specific problems. General principles almost always apply.
always a dangerous assumption - and your general principles are
themselves rather suspect given the reality of current propagation
characteristics
C> In me experience ( and conventional wisdom ) Assuming
C> well-connected peers:
C> 1 will provide over 95% of articles with no resilience.
no chance, if you're talking about full binary feeds
C> 2 will provide about 98% of articles with resilience.
C> 3 will fill in a few gaps but 98% of it's offerings will result
C> in fruitless history searches.
C> 4 adds no real benefit for the cost of 100% extra history searches.
I would not expect decent completion for binaries with less than about
8 good-quality peers.
C> The only benefit of having more than 3 feeds in in propagation
C> time, some articles may arrive earlier via the extra route, but
C> time is not of the essence. The extra history searches ( one for
C> every article offered ) are expensive in CPU time and even more
C> expensive in disk I/O unless the entire history data is kept in
C> memory. This is a primary cause of backlogged servers.
this is nonsense - there is a reason why history caches exist, and the
CPU time consumed is negligible compared to other factors (the only
serious CPU bottleneck on a transit feeder is usually the spam-filter,
assuming one is present). A few megabytes of history cache eliminates
virtually all history lookups except for the essential ones.
disk I/O throughput and network issues are by far the more common
causes of inability to keep up with a feed.
--
Andrew.
Andy
<c...@NOSPAM.netunix.com> wrote
[]
>The only benefit of having more than 3 feeds in in propagation time,
>some articles may arrive earlier via the extra route, but time
>is not of the essence.
It is, if the article expires before it arrives.
--
Andy
For Austrian philately <URL: http://www.kitzbuhel.demon.co.uk/austamps/>
For Lupus <URL: http://www.kitzbuhel.demon.co.uk/lupus/>
For my other interests <URL: http://www.kitzbuhel.demon.co.uk/>
Denis Mcmahon
<ric...@highwayman.com> wrote:
>>>There are people who read Demon service who have some idea what sort of
>>>price needs to be paid for (a) external bandwidth and (b) the sort of
>>>kit needed to handle the newsfeed and distribute it to tens of thousands
>>>of users ... I thought it would be interesting to see these figures
>>>
>>Yes.
>>
>>But aren't there people employed at Demon who should know?
>
>yes, but the numbers are commercially confidential
>
>however, if you work them out yourself (with assistance from others)
>then you can see whether assertions such as "building local news servers
>saves external bandwidth" are true when one tries to say that saving the
>external bandwidth is actually cheaper. It appears to be "obvious" to
>some people that this would be so -- I am interested in how much they
>think bandwidth costs and how much they think servers cost.
I'm using a GSM roaming in Germany at the moment, so have no wish to
run up a masive bill surfing, however:
I seem to recall that a 64K leased line is Circa GBP 3K / Year plus a
distance based element between the two sites.
I imagine that the cost of provisioning an private E1 circuit between
two points is in the BT price list.
I expect that the cost of an E1 would be at least 10 times that of a
64K circuit, ie > 30K / year.
At a transatlantic level, that probably increases significantly, but
in Demons case possibly offset by use of higher order circuits.
However, I'd guess that GBP 100K / year would be a minimum for an E1
transatlantic.
If you can reduce the Transatlantic bandwidth requirement by 2 E1
circuits through cutting the news load from 200Gb / day to 2 Gb / day
(assuming that the rest of the reduction will be in UK peering
bandwidth) by cutting out the binaries, that would probably pay for
additional ftp equipment and staff sufficient to provide a mirror for
the "legitimate" binaries, which I get the impression probably amount
to less than a couple of Gb per day, and could be held with a months
retention on a reasonably sized system.
Richard Tibbetts
>On Sat, 06 Jan 2001 02:15:10 +0000, C. Newport (c...@NOSPAM.netunix.com) said:
>>Maybe it would be a better idea to just drop the binary newsgroups
>>and let the Borg live for another few years.
>
>Seconded. Personally, I'd prefer it if the binary groups were dropped (&
>also a filter on binaries in general) if that meant that handling
>news would become cheaper for Demon.
I suspect that if binaries are dropped, a filter would be necessary.
Is automatic filtering of binary attachments feasible, though?
--
Richard Tibbetts
http://www.primepeace.ltd.uk/
Anthony
<nos...@charlecote.org.uk> wrote:
>
>So, in the simplest of terms which I hope doesn't oversimplify, the
>commercial value of providing a "full" news feed (i.e., on the ISP's own
>server) becomes a trade-off between the apportioned costs of doing so,
>set against the number of potential subscribers and the amount they are
>willing to pay.
>
It's probably also worth bearing mind that, at a rough estimate, less
than 5% of the user base of a typical mass market UK ISP even know
that Usenet exists, let alone want to use it. The ratio is probably a
little higher in Demon's case, given the nature of the ISP, however
it's easy to fall into the trap of believing that, because *we* like
to, everyone else knows/likes/wants to post to and read Usenet also.
That's far from being the case. For the vast majority of users, the
Internet = email & Web.
--
Anthony
ant...@catfish.demon.co.uk
C. Newport
Rejecting *.binary.* is a simple matter, this catches most of
the problem very cheaply.
For filtering a good strategy would be to look for MIME in the
header, or Lines: greater than (say 200) and shuffle any matches
off to a dedicated filtering machine rather that clog up the
spool with filtering.
Tony Evans
>Is automatic filtering of binary attachments feasible, though?
It's an interesting question. You can avoid the majority of on-topic
binaries, by not carrying the hierarchies in which they are present,
ensuring that your peers do not offer them to you. That would cut out
the significant chunk.
Then you need to filter out the binaries you do get offered, and if my
understanding is correct, at least one of your machines needs to
accept the article, before it can decide to reject it and remove it
from the spool (or never add it). So, you're still using bandwidth in
advance. However, the other factors, like disk space, become far
cheaper over all.
Tony Evans
--
Want to buy a house in Stockton on Tees? £32,950? Mail me.
For every action, there is an equal and opposite criticism.
LRP : http://www.lorgaire.co.uk
DaveN
Anthony <ant...@catfish.demon.co.uk> wrote:
>It's probably also worth bearing mind that, at a rough estimate, less
>than 5% of the user base of a typical mass market UK ISP even know
>that Usenet exists, let alone want to use it. The ratio is probably a
>little higher in Demon's case, given the nature of the ISP, however
>it's easy to fall into the trap of believing that, because *we* like
>to, everyone else knows/likes/wants to post to and read Usenet also.
>
>That's far from being the case. For the vast majority of users, the
>Internet = email & Web.
So, if that is the case, the potential "income" (i.e., the total number
of those using news multiplied by willingness to pay) is limited and
inflexible; a small market comparatively. It would seem, then, that the
economic worth of providing a full news feed is highly susceptible to
small changes in cost.
--
DaveN