nnrpdcheckart speedup possible?
Lars Magne Ingebrigtsen
User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3.50 (gnu/linux)
(Setup: inn 2.4.1 with tradspool/tradindexed.)
I've been running news.gmane.org with nnrpdcheckart set to false for
a couple of years. Since Gmane doesn't expire much (and doesn't
accept cancels), this works quite well, but it's not exactly ideal.
Sometimes we want messages to really go away, and having them still
appear in XOVER output can be awkward.
Now, I don't know whether there's a configuration setting that'll
just fix up the .DAT NOV file when a cancel arrives. I've peeked
around in the documentation, but haven't seen anything likely...
Running tdx-util over the groups that needs fixing once in a while
would almost be acceptable -- but some of these groups are so large
that it doesn't really make much sense. tdx-util over a group with
300K messages takes a while.
Perhaps something that reads the directory, and then reads the .DAT
file and weeds out the articles that aren't there any more? That
should be pretty fast. One directory read, one (big) file read, and
then one (big) file write. And some locking, of course. And the
.IDX file would need touching up, I guess.
If something like this doesn't exist, and there's no configuration
option to make this happen automatically, I could take a whack at
writing something like this. Would looking at the tdx-util source
code be the right place to start?
Another approach to take would be to have tradindex mark the NOV line
as "removed". Looking at the code for outputting the NOV lines, it
looks like if the Xref data is blanked out, the XOVER command won't
output the lines. The comments for tradindexed_cancel() say that
it's not implemented yet because you can't go from token to group
name, which I guess is true, but tradspool does just that by looking
at the Xref header and cancelling all the instances there. Can't
tradindexed_cancel() use the same approach? (And the other backends
as well, but that doesn't really interest me as much. :-)
--
(domestic pets only, the antidote for overdose, milk.)
la...@gnus.org * Lars Magne Ingebrigtsen
Jeffrey M.Vinocur
> Now, I don't know whether there's a configuration setting that'll
> just fix up the .DAT NOV file when a cancel arrives.
There's not, sadly. I believe Russ has thought about how to add it,
perhaps he can make suggestions if you're interested in coding.
> Perhaps something that reads the directory, and then reads the .DAT
> file and weeds out the articles that aren't there any more?
Well, that's expireover, in essence.
I'm assuming your question is how to make cancelled articles vanish
more quickly than waiting for the nightly news.daily run -- if your
articles are remaining indefinitely, then something is up with your
expireover. (Also, note that you can run expireover more often than
daily, if that helps you any; it still won't provide instant response,
though.)
--
Jeffrey M. Vinocur
je...@litech.org
Lars Magne Ingebrigtsen
> I'm assuming your question is how to make cancelled articles vanish
> more quickly than waiting for the nightly news.daily run -- if your
> articles are remaining indefinitely, then something is up with your
> expireover.
I don't run expireover -- news.gmane.org doesn't expire any messages.
But I guess I could run it anyway. The problem being that it doesn't
really scale very well if you don't actually expire anything -- the
expireover runs will take longer and longer.
> (Also, note that you can run expireover more often than daily, if
> that helps you any; it still won't provide instant response,
> though.)
Instant response would be really, really nice.