Re: B8 checkgroups exceeds 128KB
Russ Allbery
> The fact that it grew to exceed 128kB (131,072 bytes) would explain
> it. I guess it used to be smaller. I didn't think that the size was
> exceeding my limit. Oops.
The Big Eight checkgroups has been larger than 128KB for as long as I've
been issuing control messages. CVS says the initial import in
2002-10-05 was 150,571 characters.
> I run one of many text-only servers and alot of us have limits like
> the above (some as low as 32kB) as a "stupidity filter" against
> misplaced binaries/images and such.
> I bet that checkgroups messages would compress well. Maybe it's time
> to add message compression to INN's protocols?
> Although some other hierarchies (e.g. fido7.* and microsoft.*) also
> exceed 128kB, for the B8, as there's 8 different hierarchies in there,
> perhaps some systems might want the message to be split into two or
> more (e.g. separate comp.* from the rest to start). Comments?
I think 128KB is an unreasonable cutoff and you should fix your server.
Text postings are often larger than that (large FAQs and works of
fiction in the creative groups, for instance).
--
Russ Allbery (r...@stanford.edu) <http://www.eyrie.org/~eagle/>
Please post questions rather than mailing me directly.
<http://www.eyrie.org/~eagle/faqs/questions.html> explains why.
D. Stussy
news:87ljpqe...@windlord.stanford.edu...
> ...
> I think 128KB is an unreasonable cutoff and you should fix your server.
> Text postings are often larger than that (large FAQs and works of
> fiction in the creative groups, for instance).
I'm thinking that FAQs that large should probably have a static web page
and have a summary with a pointer for more information as the periodic
posting.
I haven't seen that many posts that do exceed the limit - but I also don't
read the creative groups. Most of what I see averages about 10kB. I
lowered the limit to 128k (from a higher value) because of some of the
trash posts I was receiving that exceeded that [lowered] size. I may
re-raise my setting in the near future.
What would you suggest as a limit for a text-groups only server?
Julien ÉLIE
> Comments? (Note the follow-up groups)
Follow-up to news.admin.technical,news.admin.hierarchies.
Initial post to news.software.nntp,news.admin.peering.
Hmm... I guess it will be difficult for people to follow the discussion
with such changes...
*sigh* and I even repost that article because I do not have news.admin.technical
on my own news server so it did not know it was a moderated newsgroup...
> I bet that checkgroups messages would compress well. Maybe it's time to
> add message compression to INN's protocols?
It would be an extension to NNTP protocol. See the previous discussion
between Curt and Russ about binary streams.
> Although some other hierarchies (e.g. fido7.* and microsoft.*) also exceed
> 128kB
Moreover, a newsgroup in microsoft.* is not created on ftp.isc.org
because of the length of itsname:
microsoft.public.windows.vista.administration_accounts_passwords
There is an additional criteria that is also taken from the current
USEFOR draft, but which is still a matter of some debate -- the limit on
the length of a component. The de facto limit was 14 octets for over a
decade and a half. It is currently proposed that this be lifted to a
"soft" limit of 30 octets (in other words, this limit is recommended but
not required). This limit is currently enforced. Entire newsgroup
names are limited to 80 characters, rather than the soft limit of 72
characters proposed by the standard, because two existing non-joke
groups have names longer than 72 characters.
There is also this one existing newsgroup:
microsoft.public.vb.6.webdevelopment
with an all-numeric component... I understand that ftp.isc.org does not
accept it. I still send it in checkgroups, though.
Incidentally, INN controls the names of the newsgroups in newgroup control
messages but not in checkgroups. I wonder if INN should also check the
syntax of newsgroups names in checkgroups (and refuse to honour bad ones).
Any comments about that?
> for the B8, as there's 8 different hierarchies in there, perhaps
> some systems might want the message to be split into two or more (e.g.
> separate comp.* from the rest to start).
It is done because of the limitation of INN's parsing control.ctl...
Only one match is done for checkgroups, with the name of the sender.
It also means that current control.ctl on ftp.isc.org will *not* work
with for instance:
## AUS (Australia)
newgroup:*:aus.*:drop
rmgroup:*:aus.*:drop
checkgroups:ausa...@aus.news-admin.org:aus.*:verify-...@aus.news-admin.org
newgroup:ausa...@aus.news-admin.org:aus.*:verify-...@aus.news-admin.org
rmgroup:ausa...@aus.news-admin.org:aus.*:verify-...@aus.news-admin.org
## MELB (Melbourne, Australia)
newgroup:*:melb.*:drop
rmgroup:*:melb.*:drop
checkgroups:ausa...@aus.news-admin.org:melb.*:verify-...@aus.news-admin.org
newgroup:ausa...@aus.news-admin.org:melb.*:verify-...@aus.news-admin.org
rmgroup:ausa...@aus.news-admin.org:melb.*:verify-...@aus.news-admin.org
No aus.* checkgroups will be honoured by INN 2.4.6 (melb.*, or a subsequent one,
will match because of the mail address).
Note that this issue is now fixed in INN 2.5.0.
--
Julien ÉLIE
« Le travail, c'est le refuge des gens qui n'ont rien
de mieux à faire. » (Oscar Wilde)
Russ Allbery
> I'm thinking that FAQs that large should probably have a static web
> page and have a summary with a pointer for more information as the
> periodic posting.
Probably these days, but that doesn't mean it will ever happen. :)
> I haven't seen that many posts that do exceed the limit - but I also
> don't read the creative groups. Most of what I see averages about
> 10kB. I lowered the limit to 128k (from a higher value) because of
> some of the trash posts I was receiving that exceeded that [lowered]
> size. I may re-raise my setting in the near future.
>
> What would you suggest as a limit for a text-groups only server?
I bet you'd be fine with 256KB. When I last looked at this, I saw a few
posts over 128KB on a regular basis, but almost nothing over 256KB.
You're going to lose some stuff with any setting below 1MB, but you lose
an ever-decreasing amount of stuff that you care about. Hell, if you
wanted to keep 128KB, there are other ways of getting checkgroups or
syncing groups.
D. Stussy
> sp...@bde-arc.ampr.org (D. Stussy) writes:
>...
> > I haven't seen that many posts that do exceed the limit - but I also
> > don't read the creative groups. Most of what I see averages about
> > 10kB. I lowered the limit to 128k (from a higher value) because of
> > some of the trash posts I was receiving that exceeded that [lowered]
> > size. I may re-raise my setting in the near future.
> >
> > What would you suggest as a limit for a text-groups only server?
>
> I bet you'd be fine with 256KB. When I last looked at this, I saw a few
> posts over 128KB on a regular basis, but almost nothing over 256KB.
>
> You're going to lose some stuff with any setting below 1MB, but you lose
> an ever-decreasing amount of stuff that you care about. Hell, if you
> wanted to keep 128KB, there are other ways of getting checkgroups or
> syncing groups.
Noted. Thank you.