Accumulating innfeed cruft
Jeffrey M. Vinocur
to notice some buildup of Stuff. In particular, a couple dozen
spool/innfeed/innfeed-dropped.* files. The manpage seems to
imply that these are from misconfiguring (like adding a peer to
newsfeeds but forgetting about innfeed.conf), and thus I should
be able to delete the old ones without doing any harm.
Neh?
--
Jeffrey M. Vinocur
je...@litech.org
Paul Tomblin
>I've been running a small INN setup for a while, and I happenned
>to notice some buildup of Stuff. In particular, a couple dozen
>spool/innfeed/innfeed-dropped.* files. The manpage seems to
>imply that these are from misconfiguring (like adding a peer to
>newsfeeds but forgetting about innfeed.conf), and thus I should
>be able to delete the old ones without doing any harm.
I seem to get those when a site goes off the air for a while, even after
it comes back on. I wish I knew what they are, but I just blow them away.
--
Paul Tomblin <ptom...@xcski.com>, not speaking for anybody
Real programmers don't comment their code. It was hard to write, it
should be hard to understand.
Russ Allbery
> In a previous article, je...@litech.org (Jeffrey M. Vinocur) said:
>> I've been running a small INN setup for a while, and I happenned to
>> notice some buildup of Stuff. In particular, a couple dozen
>> spool/innfeed/innfeed-dropped.* files. The manpage seems to imply that
>> these are from misconfiguring (like adding a peer to newsfeeds but
>> forgetting about innfeed.conf), and thus I should be able to delete the
>> old ones without doing any harm.
Nope, they happen all the time, usually whenever a lot of control messages
are acted on in a short period or you issue a whole bunch of ctlinnd
commands. You want to pick them up and process them.
> I seem to get those when a site goes off the air for a while, even after
> it comes back on. I wish I knew what they are, but I just blow them
> away.
Here's the internal document I wrote here at Stanford saying how to deal
with them:
Author: Russ Allbery <ea...@windlord.stanford.edu>
Subject: Processing dropped articles from innfeed
Revision: $Id: innfeed-dropped,v 1.1 1999/12/15 23:58:57 eagle Exp $
innfeed is the process on newsfeed.stanford.edu that sends outgoing
articles to our peers (including to nntp.stanford.edu). It will sometimes
drop articles; this happens when innfeed can't initialize a remote site
for whatever reason, or when INN feeds it articles for a site that isn't
in innfeed.conf (a misconfiguration). The most common cause of this is
the problems that innfeed has with being stopped and restarted a bunch of
times in active succession. INN stops and restarts innfeed whenever it
acts on a control message, so this can happen when a flurry of control
messages come in; when it does, innfeed will sometimes be unable to lock a
site for a while, and articles for that site will get dropped.
Dropped articles are noted in files named innfeed-dropped.<id> where <id>
is a unique ID for a given innfeed process. These files should normally
always be zero length. If they're non-zero length, innfeed has dropped
some articles, and those dropped article files have to be manually
processed or those articles will never be correctly sent to peers. A cron
job checks every hour for such non-zero dropped article files and will
notify news@news if they show up.
To process one of these dropped article files, first make sure that the
dropped article file doesn't correspond to a currently running innfeed.
The easiest way of checking this is to notice that every running innfeed
process has a dropped article file, and there are always three copies of
innfeed running on newsfeed.stanford.edu. If there are only three dropped
article files, they all correspond to running innfeed processes. You can
also note that the digits at the end of the file name are the PID of
innfeed and see if that process is running.
If the file is owned by a running innfeed, stop and restart that instance
of innfeed. The easy way to do that is the command ctlinnd reload
newsfeeds 'restart innfeed' -- the bit between single quotes is just the
log message and is arbitrary. The better way, which only restarts the
innfeed you need to restart, is to use ps to determine whether the
corresponding innfeed is the innfeed-sm instance, the innfeed-lg instance,
or the innfeed instance (the one that doesn't take a configuration file on
the command line), and then flush the corresponding pseudosite. For
example, if ps reveals that that innfeed is using the innfeed-sm.conf
file, type ctlinnd flush innfeed-sm\!. (All pseudosite names end in !,
which has to be protected from the shell.)
After the dropped article file is safely unowned by any running process,
run procbatch -vm <dropped-file>. That will generate innfeed backlog
files for each site mentioned in the dropped article file and put them
somewhere where innfeed will pick them up a while later. procbatch doesn't
delete the dropped article files itself, so after running procbatch you
should rm the files.
If the dropped articles were due to a misconfiguration, the generated
backlog file won't correspond to any configured innfeed peer. The best
way of detecting this is to see if there's a .lock file for whatever
backlog file procbatch generated (it will tell you what files it's
generating). If there's no .lock file, innfeed doesn't have that site
configured as a peer, and some more tweaking and fixes to innfeed.conf
will probably be needed to correct whatever's going wrong.
--
Russ Allbery (r...@stanford.edu) <http://www.eyrie.org/~eagle/>
Please post questions rather than mailing me directly.
<http://www.eyrie.org/~eagle/faqs/questions.html> explains why.
Barry Bouwsma Is A Certified Engineer
> I've been running a small INN setup for a while, and I happenned
> to notice some buildup of Stuff. In particular, a couple dozen
> spool/innfeed/innfeed-dropped.* files. The manpage seems to
> imply that these are from misconfiguring (like adding a peer to
> newsfeeds but forgetting about innfeed.conf), and thus I should
> be able to delete the old ones without doing any harm.
I have the same problem with my bofh.* feeds.
Paul Tomblin
>Nope, they happen all the time, usually whenever a lot of control messages
>are acted on in a short period or you issue a whole bunch of ctlinnd
>commands. You want to pick them up and process them.
Mine are almost always zero length.
--
Paul Tomblin <ptom...@xcski.com>, not speaking for anybody
`And when you've been *plonk*ed by Simon C., you've been *plonked*
by someone who knows when, and why, and how.' - Mike Andrews, asr
Russ Allbery
> In a previous article, Russ Allbery <r...@stanford.edu> said:
>> Nope, they happen all the time, usually whenever a lot of control
>> messages are acted on in a short period or you issue a whole bunch of
>> ctlinnd commands. You want to pick them up and process them.
> Mine are almost always zero length.
innfeed always creates a zero-length file when it starts. There aren't
any dropped articles unless the file is of non-zero length. (It should be
removing those zero-length files when it exits.)
Kai Henningsen
> Here's the internal document I wrote here at Stanford saying how to deal
> with them:
There really should be a standard distributed way of coping with them.
Kai
--
http://www.westfalen.de/private/khms/
"... by God I *KNOW* what this network is for, and you can't have it."
- Russ Allbery (r...@stanford.edu)
Sven Paulus
> There really should be a standard distributed way of coping with them.
I run the following script every hour. It relies on the presence of fuser on
the system. Of course this doesn't help if your sending articles to a site
(in newsfeeds) which isn't configured in innfeed.conf.
#! /bin/sh
# innfeed-undrop - requeues dropped articles
#
# 1-Apr-'99 sven
#
INNFEEDDIR=/news/spool/innfeed
cd $INNFEEDDIR || exit 1
# check, if innfeed-dropped files exist
ls innfeed-dropped.* > /dev/null 2> /dev/null || exit 0
for a in innfeed-dropped.*; do
# still in use?
if /bin/fuser $a > /dev/null; then
continue
fi
# distribute to the feeds
/news/bin/filechan -d $INNFEEDDIR -f 2 < $a && rm -f $a
done
Barry Bouwsma is a Certified Engineer
:: to notice some buildup of Stuff. In particular, a couple dozen
:: spool/innfeed/innfeed-dropped.* files. The manpage seems to
:: imply that these are from misconfiguring (like adding a peer to
`ls -l spool/innfeed/innfeed-dropped*` ??
`ls -l log/innfeed.log` ???
The answers I give will depend whether the innfeed-dropped files
are zero-length and the dates of them, and whether the infeed.log
file is empty or if it has some error messages within.
Ihr
Kvindeservice
**** Post for FREE via your newsreader at post.newsfeeds.com ****
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
*** Newsfeeds.com - The #1 Usenet Newsgroup Service on The Planet! ***
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
http://www.newsfeeds.com | http://www.newsfeeds.com
|
* Anonymous posting server! | * Totally Uncensored!
* SUPER Servers! | * Over 80,000 Newsgroups!
* BINARIES ONLY Servers! | * 16 seperate Newsgroup Servers!
* SPAM FILTERED Server! | * Instant access!
* ADULT ONLY Server! | * Multiple OC 3's and OC 12's!
* MP3 ONLY Server! | * 99% Article Completion!
* MULTIMEDIA ONLY Server! | * Months of Retention!
* 7 UNCENSORED Newsgroup Servers | * Lightning FAST downloads!
|
http://www.newsfeeds.com | http://www.newsfeeds.com
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
**** Point your newsreader to post.newsfeeds.com ****
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=