[Individual] Improvement of Cancel Processing
NetNews Team Individual.NET
overwriting articles) existed since the ancient times of early
electronic communication and relied - without special validation and
protection - on the sense of responsibility and good faith of all
Usenet users. Unfortunately, with growing numbers of Usenet
participants and years passing by, this cannot be assumed anymore.
During the last months, we noticed a strong increase of rogue Cancels
and rogue Supersedes in the German language hierarchy de.* as well as
- with significant higher numbers - the Big-8 groups, including very
sophisticated methods of obfuscation and appearance which make it
almost impossible for an avarage Usenet user to even notice or
understand what happened. As a consequence of this incidents, we made
the decision to stop the usual unchecked execution of Cancels and
Supersedes on our server, and implemented methods that honor only
Cancels and Supersedes from trusted and/or verifiable sources.
We created a whitelist of senders that contains well-known Despammers
("professional" Spam Cancellers) so that we can honor their desirable
Spam Cancels and NoCeM Notices as well as senders of periodic Usenet
postings like FAQs so that they can overwrite older versions of their
texts by Supersedes.
Furthermore, we activated the "Cancel Lock" (aka "canlock")
verification system on our server. Our server does now honor the
article headers "Cancel-Lock:" and "Cancel-Key:" that create a
verifiable connection between an article and a corresponding
Cancel/Supersedes. Users that add Cancel-Lock to their articles and
Cancel-Key to their Cancels/Supersedes can still remove and overwrite
articles on our server. For our own users, our server automatically
creates Cancel-Lock at posting time; a Cancel-Lock already present at
posting time (added by the user himself) will be extended according to
the technical specifications of the system. Analogously, Cancels and
Supersedes through our server will automatically get the corresponding
Cancel-Key after verification and will be executed.
As a result of the new Cancel/Supersedes processing rules, our users
may start seeing several instances of the same article on our server,
or may see articles that do not exist elsewhere (anymore) - our server
will not execute Cancel/Supersedes that are not covered by our
whitelist or secured by canlock because we cannot verify the
legitimacy of the Cancel/Supersedes otherwise.
Our recommendations for all members of Usenet:
* Regular Users
o Do not send Cancels, send Supersedes instead. On servers
that do not execute Cancels/Supersedes (like ours),
Supersedes will cause a second article to appear near your
first article so that other readers will see that there is
a second, newer version of your article.
o If your newsreading software supports the Cancel Lock
feature, please activate it. Cancel Lock creates a
verifiable connection between an article and a corresponding
Cancel or Supersedes, resulting in a trusted Cancel/Supersedes
that will be executed. As an added bonus, this system does
not allow profiling like e.g. PGP signatures do.
* Senders of periodic Usenet postings (e.g. FAQs) with "Supersedes:"
header
o Supply your article with an "Expires:" header.
o Supply your article with "Cancel-Lock:" and "Cancel-Key:"
headers.
o Sign your article's header with PGP ("X-PGP-Sig:").
* Despammer
o Sign your Cancels with a suitable cryptographic signature
o and/or send cryptographic signed NoCeM Notices. The server
honors NoCeMs (most despammers are already in our whitelist;
if you think you are missing, please drop us a note at
ne...@individual.net, we will add you after verification).
We apologize for any inconvenience our changes might cause, but we are
confident that diversifying the processing of Cancels and Supersedes
will be a strong change for the better. And we would be glad if other
servers decide to change their processing as well - so sustained
improvement for Usenet at large can be achieved.
Best regards,
NetNews Team Individual.NET
Whiskers
> The system of Cancels and Supersedes in Usenet (removing and
> overwriting articles) existed since the ancient times of early
> electronic communication and relied - without special validation and
> protection - on the sense of responsibility and good faith of all
> Usenet users. Unfortunately, with growing numbers of Usenet
> participants and years passing by, this cannot be assumed anymore.
[...]
> We apologize for any inconvenience our changes might cause, but we are
> confident that diversifying the processing of Cancels and Supersedes
> will be a strong change for the better. And we would be glad if other
> servers decide to change their processing as well - so sustained
> improvement for Usenet at large can be achieved.
>
> Best regards,
> NetNews Team Individual.NET
Interesting. A little detective-work found
<http://tools.ietf.org/wg/usefor/draft-ietf-usefor-cancel-lock/> but if it
is a workable system perhaps Individual.net's espousal of it will
precipitate a change of the draft's status from 'Expired'. Certainly,
rogue cancels have been a problem for years.
I also found <http://www.templetons.com/usenet-format/howcancel.html>
which looks worth a read, at least.
At least this explains the appearance of a Cancel-Lock header in posts
made to Individual.net recently.
--
-- ^^^^^^^^^^
-- Whiskers
-- ~~~~~~~~~~
Bettina Fink
> Interesting. A little detective-work found
> <http://tools.ietf.org/wg/usefor/draft-ietf-usefor-cancel-lock/> but if it
> is a workable system perhaps Individual.net's espousal of it will
> precipitate a change of the draft's status from 'Expired'. Certainly,
> rogue cancels have been a problem for years.
Yes, the Cancel-Lock idea is old and neglected, but it's there. The other
options would have been giving up all kind of Cancels and Supersedes, or
inventing something completely new. So let's see if Cancel-Lock can be
reanimated. ;-)
Whiskers
Certainly worth a try :))
Whiskers
> Whiskers <catwh...@operamail.com> wrote...
>
>> Interesting. A little detective-work found
>> <http://tools.ietf.org/wg/usefor/draft-ietf-usefor-cancel-lock/> but if it
>> is a workable system perhaps Individual.net's espousal of it will
>> precipitate a change of the draft's status from 'Expired'.
>
Given that the concept seems to have been well thought out so long ago,
and the problem it addresses must have been around before then, I'm a
little surprised that it fell by the wayside.
>> I also found <http://www.templetons.com/usenet-format/howcancel.html>
>> which looks worth a read, at least.
>
> I read that document - which is also a quiet old one, it's back from
> 1999 -
The whole canlock idea seems to have been moribund for years. There's a
thread over in news.software.readers where slrn users are trying to get
slrn to set Cancel-Lock headers, and I think Gnus and Tin are supposed to
be able to do the trick too, but I don't know how many other usenet
clients can.
> during preparation and do not agree with the author's opinion.
> Basically:
> - The overhead for a canlock header is neglectable in times of massive
> top postings, X-Face headers and other things.
I agree. 'Bandwidth' is less of a constraint now for most people, than it
was a decade ago.
> - All the pgp thingies are complicated to understand for a user and
> difficult to set up, and careless usage might easily compromise
> his secret. Using a CA means introducing a third party, using
> simple keys puts a lot of work onto each news admin.
> canlock ist extremely simple and in a case like Individual the user
> doesn't need to do anything. The news server does the job and
> you're done. It works using any newsreader.
I agree. That is probably even more of a consideration now, with most
usenetters being home users without the support of knowledgeable system
admins and without the interest that characterised the early 'computer
hobbyist' types who must have predominated in usenet in the days before
mass internet access ('before this everlasting September').
> - Yes, pgp is needed for e.g. verifiable third party cancel. That's
> nocem.
People engaged in that sort of activity will probably be quite
comfortable with 'digital signatures' etc.
>> At least this explains the appearance of a Cancel-Lock header in posts
>> made to Individual.net recently.
>
> :-)
>
> Christoph
I was new to usenet in 2000 so I've never encountered Cancel-Lock before,
and I'm still at the bottom of the learning curve. So I thought I'd post
explanatory links that I've found so far, in case anyone else out there is
equally ignorant.
Curt Welch
> On 2007-06-28, NetNews Team Individual.NET <ne...@individual.net> wrote:
> > The system of Cancels and Supersedes in Usenet (removing and
> > overwriting articles) existed since the ancient times of early
> > electronic communication and relied - without special validation and
> > protection - on the sense of responsibility and good faith of all
> > Usenet users. Unfortunately, with growing numbers of Usenet
> > participants and years passing by, this cannot be assumed anymore.
>
> [...]
>
> > We apologize for any inconvenience our changes might cause, but we are
> > confident that diversifying the processing of Cancels and Supersedes
> > will be a strong change for the better. And we would be glad if other
> > servers decide to change their processing as well - so sustained
> > improvement for Usenet at large can be achieved.
> >
> > Best regards,
> > NetNews Team Individual.NET
>
> Interesting. A little detective-work found
> <http://tools.ietf.org/wg/usefor/draft-ietf-usefor-cancel-lock/> but if
> it is a workable system perhaps Individual.net's espousal of it will
> precipitate a change of the draft's status from 'Expired'. Certainly,
> rogue cancels have been a problem for years.
It's very workable. It was workable back in 1992 when I made up the names
for the headers and half implemented the first version of it:
http://groups.google.com/group/news.admin/msg/5456ba58a23988d9
The improvements people have thought up over the years as outlined in the
ietf draft makes it even better - such as allowing multiple locks and
allowing the article to be canceled if you have a key for any locks so that
multiple people such as the author, the admin, and a moderator which are
all responsible for the article can cancel the article. And the idea of
using a single password to generate the keys and locks instead of having to
store the key for each article posted is a an obvious improvement I didn't
think of that makes it all far more workable.
The idea however has been met with resistance since the beginning. Back on
those earlier days, most systems still supported cancels and the basic
issue was that people didn't want to give up the power to cancel any
article they wanted to. The counter argument however was that without
authentication, abuse was going to force the admins to disable cancels and
then no one would be able to cancel anything. But people didn't buy that.
Now (15 years later), most sites have disabled cancels, and we just can't
cancel or supersede anything. The good thing however is that most cancel
abuse has been stopped.
At the same time, for binary Usenet, most posters like the fact that once
something is posted, it can't be removed. This is an advantage - though
it's one that few will talk openly about. It's an advantage for the admins
as well because if cancel locks became the norm, we would be expected to
use them and send out cancels with valid keys for any valid DMCA complaint
we would receive. It's much easier for us if we don't have to do that (not
to mention since our users would rather there be no way to cancel the
articles we have negative incentive to do the work required to make cancel
locks work). Authenticated cancels and supersedes are a feature that has
more value on the text groups. But it's not the text groups that are
paying to keep Usenet running these days.
Implementing cancel-locks on Usenet is a chicken and egg problem as well.
They won't do much good until a large percentage of sites support them.
This means that there is not much motivation for people to implement the
code in the first place - it does almost no good to run it on a single
system.
And there are problems that show up when some sites support cancel locks
and others don't. For example, a moderator can't cancel articles which
were forged. And though a good cancel-lock system might support
unauthenticated cancels for articles without a Cancel-Lock, someone who
wants to cause problems can simply add a cancel-lock with a random lock on
it - so that no one can cancel it.
Even though I thought Cancel-locks were a good idea long ago when Usenet
was still mostly a text medium, I never bothered to finish the
implementation because I felt there was too much resistance to the idea -
and if I did it, and no one else did, it would be pointless.
Today however, where the bulk of Usenet is for trading binaries and most
text discussions happen off of Usenet on email lists, or web sites, I don't
see a lot of value trying to get Usenet converted to support Cancel-Locks.
Simply disabling cancels solves the cancel abuse problem, and the other
abuse issues are solved the same way they always have been, by filtering,
and by putting pressure on the originating site to block the abuse.
I'm happy to see Individual.NET take the time to implement them. And it
would be cool to see more sites do it. But after seeing the idea go
nowhere for 15 years, and since there is less reason to do it today, than
there was in the past, I don't really expect it to go anywhere now.
--
Curt Welch http://CurtWelch.Com/
cu...@kcwc.com http://NewsReader.Com/
Ian Gregory
> As a consequence of this incidents, we made
> the decision to stop the usual unchecked execution of Cancels and
> Supersedes on our server, and implemented methods that honor only
> Cancels and Supersedes from trusted and/or verifiable sources.
Sounds like a good idea - it means the FAQ on the NIN website
needs updating though. Answers that might need updating are 1.5,
2.13, 2.14, 2.19 - also could dew with a new question about canlock.
Ian
--
Ian Gregory
http://www.zenatode.org.uk/ian/
Whiskers
> Whiskers <catwh...@operamail.com> wrote:
>> On 2007-06-28, NetNews Team Individual.NET <ne...@individual.net> wrote:
[...] [snip interesting comments]
> Implementing cancel-locks on Usenet is a chicken and egg problem as well.
> They won't do much good until a large percentage of sites support them.
> This means that there is not much motivation for people to implement the
> code in the first place - it does almost no good to run it on a single
> system.
But someone has to make the first step. Individual.net are highly
regarded, I think, which should give them some influence.
> And there are problems that show up when some sites support cancel locks
> and others don't. For example, a moderator can't cancel articles which
> were forged. And though a good cancel-lock system might support
> unauthenticated cancels for articles without a Cancel-Lock, someone who
> wants to cause problems can simply add a cancel-lock with a random lock on
> it - so that no one can cancel it.
Indeed. But preventing rogue cancels makes that nuisance a price worth
paying, I think - certainly if one's own posts are the victims of rogue
cancels. I think a properly configured news-server would not allow any
post to appear that was not correctly 'authorised' by the proper moderator.
Perhaps there is room for improvement in that area, as well as in the
operation of the more usual sort of un-moderated newsgroup.
> Even though I thought Cancel-locks were a good idea long ago when Usenet
> was still mostly a text medium, I never bothered to finish the
> implementation because I felt there was too much resistance to the idea -
> and if I did it, and no one else did, it would be pointless.
>
> Today however, where the bulk of Usenet is for trading binaries and most
> text discussions happen off of Usenet on email lists, or web sites, I don't
> see a lot of value trying to get Usenet converted to support Cancel-Locks.
> Simply disabling cancels solves the cancel abuse problem, and the other
> abuse issues are solved the same way they always have been, by filtering,
> and by putting pressure on the originating site to block the abuse.
Disabling cancels only seems to work 'properly' if all NSPs implement it;
which they do not. I think there will always be dissidents in usenet,
even among news admins :))
> I'm happy to see Individual.NET take the time to implement them. And it
> would be cool to see more sites do it. But after seeing the idea go
> nowhere for 15 years, and since there is less reason to do it today, than
> there was in the past, I don't really expect it to go anywhere now.
Despite the gross distortions generated by the binary groups (an
abomination, in my opinion, and the root of much of the difficulty
experienced by ISPs trying to provide a usenet service and thus indirectly
responsible for the decline in support of usenet), text newsgroups are
still an important and active resource for many. Anything that can
improve the experience of text newsgroup users, is worth trying.
Bettina Fink
>> As a consequence of this incidents, we made
>> the decision to stop the usual unchecked execution of Cancels and
>> Supersedes on our server, and implemented methods that honor only
>> Cancels and Supersedes from trusted and/or verifiable sources.
>
> Sounds like a good idea - it means the FAQ on the NIN website
> needs updating though. Answers that might need updating are 1.5,
> 2.13, 2.14, 2.19 - also could dew with a new question about canlock.
Yes, the FAQ is already WIP, but we modernized our FAQ generating
system(*) as well (we've been quite active, see? ;-), but it is still
a bit stubborn (output looks broken in some places) so we had to delay
the update of the FAQ until this is fixed. Should be done soon ...
(*) transforms text files into web pages
Whiskers
Gopher-space hasn't entirely vanished ... another good idea worth a
revival? ;))