Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Issues with Non-Standard and Misconfigured News Servers
12 views
Skip to first unread message
Paul W. Schleck
Nov 9, 2013, 8:59:23 PM11/9/13
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
As a moderator or co-moderator of several newsgroups, I have noticed
some concerning trends in misconfigured or non-standard news servers and
how they interact with moderated newsgroups, their moderators, and
users. These behaviors appear contrary to established customary
behavior of reference news server implementations like INN, and in the
relevant RFC's, including the most current, standards-track, RFC 5537:
http://tools.ietf.org/html/rfc5537
They also cause specific annoyances to the moderators and users of
moderated newsgroups. When the administrators of these non-standard
sites are (politely) queried about these behaviors, they often reply
that they have some local justification for running their servers in
the way that they do, or they disagree with the interpretation of RFC's,
or they have no control over the behavior of their news server software,
which is custom or proprietary.
Example 1:
Some news servers, when presented with a locally-originated article
crossposted to more than one moderated newsgroup, will send separate
e-mail submissions to each moderated newsgroup's submission address.
Sometimes these duplicate submissions will contain the same Message-ID,
sometimes different. This is obviously hazardous, as it may cause
inconsistent approval decisions, duplicate articles, or submissions to
be rejected by the news server due to a duplicate Message-ID. Some
teams of closely-related newsgroups where this happens often, will try
to trap for these through forwarding and duplicate checking, but it can
still cause annoying and confusing behavior, including article
rejections back to the submitter as a duplicate.
This behavior is clearly contrary to standards, both from a plain-text
reading of the above RFC (particularly sections 3.5.1 and 3.9), and from
consultation with the RFC's authors, Russ Allbery and Charles Lindsey.
One news server administrator was finally convinced to change the
behavior of his server after the weight of all of this evidence, but
still considers it a low-priority change to make sometime in the
medium-term future. There may be others out there.
Questions to moderators:
- Have you observed this behavior? What techniques do you have in
place to deal with it?
- Have you attempted to write to the administrators of non-standard
news servers that exhibit this behavior? What was their reply?
- Is this behavior exhibited in specific versions of
commonly-available news server software, or only in customized/
proprietary versions used locally?
- Do any moderators do re-injected, sequential approval like in
Section 3.9 of RFC 5537, or do you do other techniques like
duplicate checking and auto-forwarding to one designated/delegated
team that makes a single approval decision for all of the newsgroups
in the Newsgroups: header?
Example 2:
Some news server sites do not have up-to-date or accurate lists of which
newsgroups are moderated, and which are not. Some sites will create
newsgroups, but set all of them to unmoderated. At least one major news
server site appears to reserve the right to use the deliberate removal
of moderated status of a known moderated newsgroup on their news server
as a threat/punishment to moderation teams that complain about abuse and
SPAM originated by the local users of their server who attempt to make
such submissions to moderated newsgroups. This last one is based on
personal experience.
If an article is posted to a moderated newsgroup, but originated by a
user posting to a news server that has the newsgroup configured to
unmoderated, the article will be injected into the news stream and not
e-mailed to the moderator for approval. This is probably an unavoidable
error from misconfiguration, and not behavior in violation of standards.
However, what is non-standard behavior is when this article that
bypassed moderation reaches some news servers. The article is supposed
to be dropped, not displayed, and certainly not resubmitted to the
moderators (this last behavior was a very famous bug in old news server
software that caused moderators to be flooded by multiple submissions
from sites that received these articles, thankfully long since fixed).
Some servers will not drop, and not resubmit, but will just display it
in the newsgroup as if that newsgroup was unmoderated, often whether the
newsgroup is configured on that news server to be unmoderated or not.
This has an obvious problem of bypassing moderation. Also, if the
article is obviously non-approvable due to content or crossposting to
too many newsgroups, readers may incorrectly assume that it was approved
by a moderator, and complain to the moderators about non-existent or
inconsistent moderation decisions. For those moderated newsgroups that
implement PGPMoose checking, this can also cause false-positives, and
even confusion if the article appears on a server on which the PGPMoose
checking is performed, but does not appear on a server where other
moderators, particularly those who may originate cancel or NoCeM
messages, may use.
This second example does not appear to be directly addressed by RFC's.
Rather, the dropping of such incorrectly injected articles is the
customary, de-facto behavior of reference news servers like INN.
Experiences writing to news server administrators about this is often
that they consider this display of unmoderated articles on moderated
newsgroups to be a feature, even a diagnostic aid, rather than a bug,
and decline to fix it. The lack of specific RFC language on this
appears to leave little recourse for appeal.
Questions to moderators:
- Have you observed this behavior? What techniques do you have in
place to deal with it?
- Have you attempted to write to the administrators of non-standard
news servers that exhibit this behavior? What was their reply?
- Is this behavior exhibited in specific versions of
commonly-available news server software, or only in proprietary/
customized versions used locally?
Thanks in advance for your insights, and suggested solutions, to these
problems.
- --
Paul W. Schleck
psch...@novia.net
http://www.novia.net/~pschleck/
Finger psch...@novia.net for PGP Public Key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (FreeBSD)
iEYEARECAAYFAlJ+O0QACgkQ6Pj0az779o7XrQCggrNwQYJ3VIsYKn/rl6sEwKE/
T30AmgPwDx4YbQDZbOJTv7NNDeIGBZlv
=F1FN
-----END PGP SIGNATURE-----
Hash: SHA1
As a moderator or co-moderator of several newsgroups, I have noticed
some concerning trends in misconfigured or non-standard news servers and
how they interact with moderated newsgroups, their moderators, and
users. These behaviors appear contrary to established customary
behavior of reference news server implementations like INN, and in the
relevant RFC's, including the most current, standards-track, RFC 5537:
http://tools.ietf.org/html/rfc5537
They also cause specific annoyances to the moderators and users of
moderated newsgroups. When the administrators of these non-standard
sites are (politely) queried about these behaviors, they often reply
that they have some local justification for running their servers in
the way that they do, or they disagree with the interpretation of RFC's,
or they have no control over the behavior of their news server software,
which is custom or proprietary.
Example 1:
Some news servers, when presented with a locally-originated article
crossposted to more than one moderated newsgroup, will send separate
e-mail submissions to each moderated newsgroup's submission address.
Sometimes these duplicate submissions will contain the same Message-ID,
sometimes different. This is obviously hazardous, as it may cause
inconsistent approval decisions, duplicate articles, or submissions to
be rejected by the news server due to a duplicate Message-ID. Some
teams of closely-related newsgroups where this happens often, will try
to trap for these through forwarding and duplicate checking, but it can
still cause annoying and confusing behavior, including article
rejections back to the submitter as a duplicate.
This behavior is clearly contrary to standards, both from a plain-text
reading of the above RFC (particularly sections 3.5.1 and 3.9), and from
consultation with the RFC's authors, Russ Allbery and Charles Lindsey.
One news server administrator was finally convinced to change the
behavior of his server after the weight of all of this evidence, but
still considers it a low-priority change to make sometime in the
medium-term future. There may be others out there.
Questions to moderators:
- Have you observed this behavior? What techniques do you have in
place to deal with it?
- Have you attempted to write to the administrators of non-standard
news servers that exhibit this behavior? What was their reply?
- Is this behavior exhibited in specific versions of
commonly-available news server software, or only in customized/
proprietary versions used locally?
- Do any moderators do re-injected, sequential approval like in
Section 3.9 of RFC 5537, or do you do other techniques like
duplicate checking and auto-forwarding to one designated/delegated
team that makes a single approval decision for all of the newsgroups
in the Newsgroups: header?
Example 2:
Some news server sites do not have up-to-date or accurate lists of which
newsgroups are moderated, and which are not. Some sites will create
newsgroups, but set all of them to unmoderated. At least one major news
server site appears to reserve the right to use the deliberate removal
of moderated status of a known moderated newsgroup on their news server
as a threat/punishment to moderation teams that complain about abuse and
SPAM originated by the local users of their server who attempt to make
such submissions to moderated newsgroups. This last one is based on
personal experience.
If an article is posted to a moderated newsgroup, but originated by a
user posting to a news server that has the newsgroup configured to
unmoderated, the article will be injected into the news stream and not
e-mailed to the moderator for approval. This is probably an unavoidable
error from misconfiguration, and not behavior in violation of standards.
However, what is non-standard behavior is when this article that
bypassed moderation reaches some news servers. The article is supposed
to be dropped, not displayed, and certainly not resubmitted to the
moderators (this last behavior was a very famous bug in old news server
software that caused moderators to be flooded by multiple submissions
from sites that received these articles, thankfully long since fixed).
Some servers will not drop, and not resubmit, but will just display it
in the newsgroup as if that newsgroup was unmoderated, often whether the
newsgroup is configured on that news server to be unmoderated or not.
This has an obvious problem of bypassing moderation. Also, if the
article is obviously non-approvable due to content or crossposting to
too many newsgroups, readers may incorrectly assume that it was approved
by a moderator, and complain to the moderators about non-existent or
inconsistent moderation decisions. For those moderated newsgroups that
implement PGPMoose checking, this can also cause false-positives, and
even confusion if the article appears on a server on which the PGPMoose
checking is performed, but does not appear on a server where other
moderators, particularly those who may originate cancel or NoCeM
messages, may use.
This second example does not appear to be directly addressed by RFC's.
Rather, the dropping of such incorrectly injected articles is the
customary, de-facto behavior of reference news servers like INN.
Experiences writing to news server administrators about this is often
that they consider this display of unmoderated articles on moderated
newsgroups to be a feature, even a diagnostic aid, rather than a bug,
and decline to fix it. The lack of specific RFC language on this
appears to leave little recourse for appeal.
Questions to moderators:
- Have you observed this behavior? What techniques do you have in
place to deal with it?
- Have you attempted to write to the administrators of non-standard
news servers that exhibit this behavior? What was their reply?
- Is this behavior exhibited in specific versions of
commonly-available news server software, or only in proprietary/
customized versions used locally?
Thanks in advance for your insights, and suggested solutions, to these
problems.
- --
Paul W. Schleck
psch...@novia.net
http://www.novia.net/~pschleck/
Finger psch...@novia.net for PGP Public Key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (FreeBSD)
iEYEARECAAYFAlJ+O0QACgkQ6Pj0az779o7XrQCggrNwQYJ3VIsYKn/rl6sEwKE/
T30AmgPwDx4YbQDZbOJTv7NNDeIGBZlv
=F1FN
-----END PGP SIGNATURE-----
Phillip Helbig---undress to reply
Nov 10, 2013, 1:18:39 PM11/10/13
to
In article <527e3b5e$0$2886$740e...@novia.net>, Paul W. Schleck
> Some news server sites do not have up-to-date or accurate lists of which
> newsgroups are moderated, and which are not.
I have seen this. Occasionally, my own news server did not correctly
handle moderated groups. I managed to get it fixed, but on the
timescale of a week or so. Otherwise, news.online.de is OK, but I
suspect that only a small fraction of people who are allowed to use it
(authentication is based on IP as it is run by a DSL provider) do use
it. Frankly, I am happy that it gets little attention, as long as it
works. (It will not accept moderated articles, so I have to use another
server for injecting them.)
There is also at least one moderated group (sci.physics.foundations)
which does not accept any cross-posted articles (basically because they
don't know how to handle them technically). This forces people who want
to cross-post to post at least two separate articles with identical
content.
My biggest problem at the moment seems to be people posting via Google
Groups: There is an extra blank line between successive lines of quoted
text, the quoted text is one-line-per-paragraph, posts are unnecessarily
encoded. It seems extremely unlikely that Google Groups will fix this,
and it might even be intentional (if posts look good only in Google
Groups, then everyone will use Google Groups---like Microsoft
intentionally breaking HTML standards leading to the "best viewed in
Internet Explorer" stuff).
<psch...@oasis.novia.net>" writes:
> Some news servers, when presented with a locally-originated article
> crossposted to more than one moderated newsgroup, will send separate
> e-mail submissions to each moderated newsgroup's submission address.
I haven't seen this.
> Some news servers, when presented with a locally-originated article
> crossposted to more than one moderated newsgroup, will send separate
> e-mail submissions to each moderated newsgroup's submission address.
> Some news server sites do not have up-to-date or accurate lists of which
> newsgroups are moderated, and which are not.
handle moderated groups. I managed to get it fixed, but on the
timescale of a week or so. Otherwise, news.online.de is OK, but I
suspect that only a small fraction of people who are allowed to use it
(authentication is based on IP as it is run by a DSL provider) do use
it. Frankly, I am happy that it gets little attention, as long as it
works. (It will not accept moderated articles, so I have to use another
server for injecting them.)
There is also at least one moderated group (sci.physics.foundations)
which does not accept any cross-posted articles (basically because they
don't know how to handle them technically). This forces people who want
to cross-post to post at least two separate articles with identical
content.
My biggest problem at the moment seems to be people posting via Google
Groups: There is an extra blank line between successive lines of quoted
text, the quoted text is one-line-per-paragraph, posts are unnecessarily
encoded. It seems extremely unlikely that Google Groups will fix this,
and it might even be intentional (if posts look good only in Google
Groups, then everyone will use Google Groups---like Microsoft
intentionally breaking HTML standards leading to the "best viewed in
Internet Explorer" stuff).
Julien ÉLIE
Mar 17, 2014, 9:22:11 AM3/17/14
to
Hi Paul W. Schleck,
> Example 2:
>
[...]
3.7. Duties of a Serving Agent
[...]
A serving agent MUST have available a list (possibly empty) of
moderated groups for which it accepts articles so that it can reject
unapproved articles posted to moderated groups. Frequently, a
serving agent is deployed in combination with an injecting agent and
can use the same list as the injecting agent.
[...]
5. It MUST reject any article without an Approved header field
posted to any newsgroup listed as moderated.
Dropping such incorrectly injected articles is required, and said in the
above sentence of RFC 5537.
Unless I am misunderstanding something in your example?
--
Julien ÉLIE
« Nous agissons comme si le confort et le luxe étaient essentiels à
notre existence, alors qu'il suffit pour être réellement heureux de
trouver quelque chose qui nous intéresse passionnément. » (Charles
Kingsley)
> Example 2:
>
[...]
> If an article is posted to a moderated newsgroup, but originated by a
> user posting to a news server that has the newsgroup configured to
> unmoderated, the article will be injected into the news stream and not
> e-mailed to the moderator for approval. This is probably an unavoidable
> error from misconfiguration, and not behavior in violation of standards.
> However, what is non-standard behavior is when this article that
> bypassed moderation reaches some news servers. The article is supposed
> to be dropped, not displayed, and certainly not resubmitted to the
> moderators (this last behavior was a very famous bug in old news server
> software that caused moderators to be flooded by multiple submissions
> from sites that received these articles, thankfully long since fixed).
> Some servers will not drop, and not resubmit, but will just display it
> in the newsgroup as if that newsgroup was unmoderated, often whether the
> newsgroup is configured on that news server to be unmoderated or not.
>
[...]
> user posting to a news server that has the newsgroup configured to
> unmoderated, the article will be injected into the news stream and not
> e-mailed to the moderator for approval. This is probably an unavoidable
> error from misconfiguration, and not behavior in violation of standards.
> However, what is non-standard behavior is when this article that
> bypassed moderation reaches some news servers. The article is supposed
> to be dropped, not displayed, and certainly not resubmitted to the
> moderators (this last behavior was a very famous bug in old news server
> software that caused moderators to be flooded by multiple submissions
> from sites that received these articles, thankfully long since fixed).
> Some servers will not drop, and not resubmit, but will just display it
> in the newsgroup as if that newsgroup was unmoderated, often whether the
> newsgroup is configured on that news server to be unmoderated or not.
>
>
> This second example does not appear to be directly addressed by RFC's.
> Rather, the dropping of such incorrectly injected articles is the
> customary, de-facto behavior of reference news servers like INN.
>
> Experiences writing to news server administrators about this is
> often that they consider this display of unmoderated articles on
> moderated newsgroups to be a feature, even a diagnostic aid, rather
> than a bug, and decline to fix it. The lack of specific RFC language
> on this appears to leave little recourse for appeal.
Isn't it what RFC 5537 addresses?
> This second example does not appear to be directly addressed by RFC's.
> Rather, the dropping of such incorrectly injected articles is the
> customary, de-facto behavior of reference news servers like INN.
>
> Experiences writing to news server administrators about this is
> often that they consider this display of unmoderated articles on
> moderated newsgroups to be a feature, even a diagnostic aid, rather
> than a bug, and decline to fix it. The lack of specific RFC language
> on this appears to leave little recourse for appeal.
3.7. Duties of a Serving Agent
[...]
A serving agent MUST have available a list (possibly empty) of
moderated groups for which it accepts articles so that it can reject
unapproved articles posted to moderated groups. Frequently, a
serving agent is deployed in combination with an injecting agent and
can use the same list as the injecting agent.
[...]
5. It MUST reject any article without an Approved header field
posted to any newsgroup listed as moderated.
Dropping such incorrectly injected articles is required, and said in the
above sentence of RFC 5537.
Unless I am misunderstanding something in your example?
--
Julien ÉLIE
« Nous agissons comme si le confort et le luxe étaient essentiels à
notre existence, alors qu'il suffit pour être réellement heureux de
trouver quelque chose qui nous intéresse passionnément. » (Charles
Kingsley)
Jon Ribbens
Mar 17, 2014, 12:07:11 PM3/17/14
to
On 2013-11-10, Paul W Schleck <psch...@oasis.novia.net> wrote:
> This second example does not appear to be directly addressed by RFC's.
> Rather, the dropping of such incorrectly injected articles is the
> customary, de-facto behavior of reference news servers like INN.
No, it is in RFC 5537. Section 3.7 says news servers that serve
> This second example does not appear to be directly addressed by RFC's.
> Rather, the dropping of such incorrectly injected articles is the
> customary, de-facto behavior of reference news servers like INN.
readers MUST reject unapproved articles in moderated groups.
Section 3.6 basically says the same thing about servers that only
forward articles to news peers but does so using some unfortunately
convoluted and vague wording that means some people argue that it does
not say what it says.
> - Have you observed this behavior? What techniques do you have in
> place to deal with it?
not seem interested in trying to persuade the major news servers
to fix their broken software that is damaging Usenet.
> - Have you attempted to write to the administrators of non-standard
> news servers that exhibit this behavior? What was their reply?
> - Is this behavior exhibited in specific versions of
> commonly-available news server software, or only in proprietary/
> customized versions used locally?
0 new messages