INN 2.x FAQ

[Russ Allbery]

Last-modified: 2022-01-20
Posted-by: postfaq 1.17 (Perl 5.28.1)
Archive-name: usenet/software/inn2-faq
URL: https://www.eyrie.org/~eagle/faqs/inn.html
Posting-frequency: monthly

This FAQ is intended to answer frequently asked questions concerning the
current versions of INN (INN 2.x and later) seen on news.software.nntp.
It should be referred to in preference to the old INN FAQ, which only
documents versions up to 1.7. It mostly covers INN 2.3 and later; earlier
versions of INN may behave differently or use different configuration
files.

If you're reading this on Usenet, this FAQ is formatted as a minimal
digest, so if your news or mail reader has digest handling capabilities
you can use them to navigate between sections. In rn variants, you can
use Ctrl-G to skip to the next section; in Gnus, press Ctrl-D to break
each section into a separate article.

Please send any comments, suggestions, or updates to <ea...@eyrie.org>.
Bear in mind when sending me e-mail that I receive upwards of 800 mail
messages a day and have unanswered personal e-mail dating back six months
or more, so please don't expect an immediate response. You may receive
quicker responses by posting to news.software.nntp (even, due to the
quirky way in which I read mail and news, from me).

This FAQ is posted monthly to news.software.nntp, and is available on the
web at <https://www.eyrie.org/~eagle/faqs/inn.html>.

------------------------------

Subject: Contents

1. General Questions
1.1. What is INN?
1.2. What is the current version?
1.3. Where can I get INN?
1.4. Where can I find documentation?
1.5. What newsgroups are there for INN?
1.6. What mailing lists are there for INN?
1.7. How can I support INN development?
1.8. How can I contribute to INN?

2. Terms
2.1. What is tradspool (traditional spool)?
2.2. What is CNFS?
2.3. What are timehash and timecaf?
2.4. What is overview?
2.5. What are deferrals (NNTP code 431)?

3. Specific Problems
3.1. INN won't start after a new installation
3.3. The news server isn't keeping up with incoming news
3.4. news.notice is empty and the nightly report is missing things
3.5. INN is running out of file descriptors
3.6. Can't get debugging information out of INN
3.7. Articles aren't being sent to remote peers
3.8. sendmail isn't installed

4. Error Messages
4.1. innd: SERVER cant store article
4.2. innd: SERVER internal no control and/or junk group
4.3. Modification of read-only value attempted (Cleanfeed)
4.4. tradspool: could not open ... File exists
4.5. Binary posting to non-binary group (Cleanfeed)

5. Problems on Specific Systems
5.1. INN won't compile on SCO OpenServer / UnixWare / OpenUNIX
5.2. Using raw devices on Solaris destroys the partition table
5.3. Will INN run on Windows?
5.4. Why aren't INN's files where the documentation says they are?
5.5. Running INN on macOS

6. How Do I...
6.1. Set up a server with no external feeds, just local groups
6.2. Process a single control message
6.4. Feed all articles on a server to another server
6.5. Rename a newsgroup
6.6. Change the domain used for message IDs
6.7. Use INN without a direct news feed
6.8. Generate MRTG graphs for INN
6.9. Hide the junk and control groups from users
6.10. Modify the body of posts made through my server
6.11. Hide the Injection-Info header field
6.12. Run innd and nnrpd on separate ports
6.13. Back up and restore an INN installation
6.14. Find external feeds and set up peering

(Note that some numbers have been skipped. When questions are removed,
the remaining questions are not renumbered to avoid breaking links in
Usenet and mailing list archives.)

------------------------------

Subject: 1. General Questions

Contained in this section are general questions about INN, where to find
it, and things of that sort. It is aimed at the person who is not yet
running INN, or who has general questions about how it works.

------------------------------

Subject: 1.1. What is INN?

The README that comes with INN has this to say (in part):

INN (InterNetNews), originally written by Rich Salz, is an extremely
flexible and configurable Usenet / Netnews news server. For a
complete description of the protocols behind Usenet and Netnews, see
RFC 3977 (NNTP), RFC 4642 updated by RFC 8143 (TLS/NNTP), RFC 4643 (NNTP
authentication), RFC 4644 (streaming NNTP feeds), RFC 5536 (USEFOR), RFC
5537 (USEPRO), RFC 6048 (NNTP LIST additions), RFC 8054 (NNTP compression)
and RFC 8315 (Cancel-Lock) or their replacements.

In brief, Netnews is a set of protocols for exchanging messages between
a decentralized network of news servers. News articles are organized
into newsgroups, which are themselves organized into hierarchies.
Each individual news server stores locally all articles it has received
for a given newsgroup, making access to stored articles extremely fast.
Netnews does not require any central server; instead, each news server
passes along articles it receives to all of the news servers it peers
with, those servers pass the articles along to their peers, and so on,
resulting in "flood fill" propagation of news articles.

INN is free software, supported by Internet Systems Consortium and
volunteers around the world.

For a more complete answer, see that file. A full description of what
Usenet and Netnews are is beyond the scope of this document; for a
beginner's introduction, see the news.newusers.questions home page at
<http://www.tokak.us/nnq/>.

------------------------------

Subject: 1.2. What is the current version?

The most recently released version of INN is 2.6.4.

INN development proceeds in two branches, as with many other free software
projects. The STABLE branch is maintenance of the most recently released
stable version, and only bug fixes are added to it. The CURRENT branch is
the development version of the next release of INN.

As mentioned in the next section, when installing a new INN server, you
may wish to download the latest snapshot of the STABLE branch rather than
the current full release.

Note that the previous STABLE series for INN 2.5 terminated in the release
of INN 2.5.5 and current STABLE snapshots are based on INN 2.6. You
should therefore read the upgrade instructions in NEWS when upgrading from
a STABLE snapshot before September 12th, 2015 to one dated after that.

------------------------------

Subject: 1.3. Where can I get INN?

The download site for INN is <https://ftp.isc.org/isc/inn/>. In that
directory are the various releases of INN, some additional documentation
(particularly of security holes), the original INN Usenix paper.

There is also a snapshots subdirectory, in which you will find two sets
of snapshots: ones at the top level, which are updated only when the
code changes, and ones in the daily subdirectory, which are generated
every day and retained for seven days. The daily snapshots with
STABLE in the name are the latest versions of the STABLE branch and
may have some additional bug fixes over the current released version.
The daily snapshots with CURRENT in the name are of the current
development version.

Please note: There is no guarantee that a snapshot will even compile, let
alone function well as a news server. In particular, the CURRENT branch
is under active development, and all sorts of things may be broken at any
given point in time. Use snapshots with caution, and don't use snapshots
from the CURRENT branch on any production system unless you're prepared to
debug the inevitable problems in code that's actively changing and not yet
thoroughly tested. (The STABLE snapshots should be fairly reliable,
however.)

------------------------------

Subject: 1.4. Where can I find documentation?

INN comes with extensive documentation. See the files INSTALL and README
at the top level of the source tree, for starters. In addition, nearly
every program and configuration file has its own Unix man page. The best
place to start is by reading the entire INSTALL file and then from there
discovering which configuration files and programs do what you want to do
and reading their individual man pages.

There are HTML conversions of the documentation that comes with recent
versions of INN available at:

<https://www.eyrie.org/~eagle/software/inn/>

For additional documentation beyond what is distributed with INN, follow
the links suggested in the above page.

The documentation that comes with INN is fairly technical in nature and
lacking in some more general details on configuring news servers. Some of
the links off of the INN home page have additional overview documentation
or documentation on how to set up servers for specific roles.

Another good resource is the newsgroup news.software.nntp (and the Google
archives thereof) and the archive of the inn-workers mailing list. A link
to the latter is off the INN page referenced above.

Finally, the following additional links may be useful:

<http://aplawrence.com/Unixart/newsserver.html>
A tutorial on setting up INN aimed at beginners using SCO Unix. While
it's mostly focused on SCO, it may be useful for any beginner to INN
and news servers.

<http://www.kozubik.com/published/inn_tutorial.txt>
A tutorial on setting up INN on FreeBSD. Contains a lot of
information focused on FreeBSD and its preferred file layout, so may
be easier to follow than the generic instructions on that platform.

------------------------------

Subject: 1.5. What newsgroups are there for INN?

news.software.nntp discusses all NNTP-based news servers, including INN.
It's the best newsgroup for technical questions and discussion. The
newsgroup news.software.b is also chartered for such discussion, but it's
essentially dead now. General news administration questions are also
on-topic in news.admin.technical (moderated) and news.admin.misc
(unmoderated).

news.admin.hierarchies covers questions of general hierarchy configuration
and is where announcements of new news hierarchies are generally posted.
news.admin.net-abuse.* covers the topic of network abuse and prevention
(including spam), but is not for the faint of heart; it is extremely noisy
to the point of being essentially unreadable without a lot of time and
patience (and a good killfile).

------------------------------

Subject: 1.6. What mailing lists are there for INN?

There are several INN-related mailing lists:

inn-an...@lists.isc.org
Where announcements about INN are set (only maintainers may post).

inn-w...@lists.isc.org
Discussion of INN development. If you prefer not to use GitHub to
create an issue or a pull request, it is also where to send bug reports
and patches for consideration for inclusion into INN (postings by
members only). If you're an INN expert and have the time to help out
other users, we encourage you to join this mailing list to answer
questions. (You may also want to read the newsgroup
news.software.nntp, which gets a lot of INN-related questions.)

inn-com...@lists.isc.org
Git commit messages for INN are sent to this list (only the
automated messages are sent here, no regular posting).

inn-...@lists.isc.org
GitHub issues for INN are sent to this list (only the automated
messages are sent here, no regular posting). Bug reports should be
sent to the inn-workers mailing list, or a GitHub issue created.

To join these lists, send a subscription request to the `-request'
address. The addresses for the above lists are:

inn-announ...@lists.isc.org
inn-worke...@lists.isc.org
inn-committ...@lists.isc.org
inn-bugs...@lists.isc.org

You can alternatively join them from the subscription form in their public
web pages:

<https://lists.isc.org/mailman/listinfo/inn-announce>
<https://lists.isc.org/mailman/listinfo/inn-workers>
<https://lists.isc.org/mailman/listinfo/inn-committers>
<https://lists.isc.org/mailman/listinfo/inn-bugs>

inn-workers tends to be moderate volume (3-5 messages a day, but varying a
lot depending on what's being discussed). inn-bugs and inn-committers are
occasionally higher volume but entirely automatically generated GitHub bug
tracker and push notifications. inn-announce is a low-volume moderated list
containing only major announcements.

------------------------------

Subject: 1.7. How can I support INN development?

There are four major ways. First, like with any other free software
project, a great way to support INN development is to join in yourself.
If you know how to program and have an interest in working on a widely
deployed and fairly intricate news server, we'd love to have your help.
See the next question for more details.

Second, even if you don't have the time or expertise to write much code,
any contributions of documentation are *greatly* appreciated. There's
always documentation work to be done, from maintenance of INN's technical
documentation to tutorials and overviews for the new user or the user who
wants to do something specific. Listen on news.software.nntp for what
people are looking for, or ask on inn-workers. Similarly, beta testers
are always welcome; if you have a test news server and some knowledge of
how to diagnose server problems and want to try out the current
development code and report any bugs you run into, that helps the
developers immensely.

Third, there are always more questions from new INN users to answer.
news.software.nntp gets a regular stream of them, and it's a great way to
help out intermittantly when you have a few moments to read news. If you
can identify general solutions to frequent problems and pass them along to
the INN maintainers in the form of documentation or suggestions, even
better.

Fourth, from the README:

Note that INN is supported by Internet Systems Consortium, and
although it is free for use and redistribution and incorporation into
vendor products and export and anything else you can think of, it
costs money to produce. That money comes from ISP's, hardware and
software vendors, companies who make extensive use of the software,
and generally kind hearted folk such as yourself.

Internet Systems Consortium has also commissioned a DHCP server
implementation and handles the official support/release of BIND. You
can learn more about the ISC's goals and accomplishments from the web
page at <https://www.isc.org/>.

The ISC provides ftp and web space and mailing lists and archives.
Donations to help support all of that are greatly appreciated.

------------------------------

Subject: 1.8. How can I contribute to INN?

First, join inn-workers, since that's where all the development discussion
takes place. The traffic isn't that high.

Next, download a snapshot of the INN CURRENT branch as described above so
that you have a relatively current source base to work from. You may want
to check out or clone the current source from GitHub; just point a Git
client at:

https://github.com/InterNetNews/inn.git

Read the HACKING file at the top of the INN source tree for some general
information and tips for working on INN.

Then pick something that looks interesting to you, mention what you're
doing on inn-workers if it's likely to affect other parts of the
development, and have at it! The GitHub bug tracker and the TODO file in
the CURRENT tree have a pretty comprehensive list of things that could be
done. Best to start with something small (getting INN to work correctly
on a platform where it doesn't currently and which you have available is
often a great start, or working on one of the supporting programs or
scripts that's a bit easier to wrap one's mind around than the core INN
daemons). Patches to INN should either be submitted as a pull request on
GitHub or sent to inn-w...@lists.isc.org, possibly put on an ftp or web
site somewhere and the URL sent to inn-workers if they're extremely large.

------------------------------

Subject: 2. Terms

Here are definitions of some commonly used terms related to INN. (More
definitions are welcome; this section is extremely incomplete at the
moment and the FAQ maintainer tends not to recognize terms that need a
definition for people unfamiliar with INN.)

------------------------------

Subject: 2.1. What is tradspool (traditional spool)?

Traditional spool is called that because it's the way that all news
servers used to store articles. A traditional news spool is a tree of
directories matching the hierarchical structure of newsgroups. For
example, the newsgroup news.software.nntp would be stored in a directory
news/software/nntp under the root of the news spool, and next to the
"nntp" directory in news/software would be a "readers" directory for the
group news.software.readers.

As of INN 2.3, traditional spool is completely integrated into the storage
API as the tradspool storage method and use the same overview mechanisms
as the rest of INN.

Storing articles in the traditional spool format is slow relative to other
storage mechanisms. It's probably nearly impossible to keep up with a
full Usenet feed using pure traditional spool. It is, however, the
recommended storage method for low-traffic local newsgroups and any
newsgroups that you want to back up.

For more details, see the INSTALL file.

------------------------------

Subject: 2.2. What is CNFS?

CNFS is the Cyclic News File System, written by Scott Fritchie. It is a
high-performance method of storing news articles, designed to avoid the
high overhead involved in interacting with the file system when storing
articles in individual files. CNFS stores articles sequentially in
pre-configured buffer files. When the end of the buffer is reached, new
articles are stored from the beginning of the buffer, overwriting older
articles.

It's the fastest article storage method in terms of write performance, and
is recommended for storing binaries.

For more details, see the INSTALL file.

------------------------------

Subject: 2.3. What are timehash and timecaf?

These are two less-used storage mechanisms available under the INN storage
API (similar in that respect to CNFS). Both can usefully be thought of as
compromises between the write speed of CNFS and the fine-grained control
over article expiration. INSTALL says for timehash:

Articles are stored as individual files as in tradspool, but are
divided into directories based on the arrival time to ensure that no
single directory contains so many files as to cause a bottleneck.

and for timecaf:

Similar to timehash, articles are stored by arrival time, but instead
of writing a separate file for each article, multiple articles are put
in the same file.

timecaf was new in INN 2.3.

------------------------------

Subject: 2.4. What is overview?

Overview is summary information about articles in a newsgroup that is
returned to news reading clients as a response to the OVER command. It's
a very common extension to the NNTP protocol that allows readers to review
summary information about articles before taking the time (and bandwidth)
to download the entire article.

The canonical items of information included in an overview record are the
Subject, From, Date, References, and Message-ID header field bodies of the
article, the byte count of the article, and the line count of the article.
Nearly every server now also returns the Xref header field (a list of the
newsgroups carried by the server to which the article was posted and the
article number in each of those newsgroups) as an additional field.

Note that with the References and Message-ID header fields, the overview
record contains enough information to do article threading. It also
contains all of the fields normally keyed on for client-side filtering
(killfiles and the like).

Generating overview information for a newsgroup on the fly would be
prohibitively expensive, particularly for large groups, since the server
daemon would have to find all of those articles and scan them to build the
information. It would also be inefficient, since the overview information
for a particular group will generally be requested many times by different
clients.

Any INN server that supports readers must therefore have an overview
method configured. There are four different methods to choose from:

- buffindexed, which stores overview data and index information
into preconfigured large files like CNFS. Fast at writing, the
buffindexed overview storage method can keep up with a large feed
more easily and never consumes additional disk space beyond that
allocated to these buffers. The downside is that these buffers
are hard to recover in case of corruption and somewhat slower for
readers and the expiry process;

- ovdb, which stores overview information into a Berkeley DB database,
whose development pace has stalled these last years. This method
is fast and very robust, but may require more disk space, unless
compression is enabled. Overview data is fetched one article at a
time, which makes this method a bit slower than ovsqlite for readers;

- ovsqlite, which stores overview information into an SQLite database,
known for its long-term stability and compatibility. Robust and
faster than ovdb at reading ranges of overview data (since overview
data is transferred in 128-kilobyte chunks between ovsqlite-server
and nnrpd) but somewhat slower at writing, this method may require
more disk space, unless compression is enabled;

- tradindexed, which uses two files per newsgroup, one containing
the overview data and one containing the index. Fast for readers,
but slow to write to because it has to update two files for each
incoming article. Its main advantage is to be the best tested,
the most reliable and the method with the best recovery tools.

Here are a few elements that can be helpful in choosing the right
overview method for your needs and estimating the associated storage
size. In 2020, the volume for a full-text Usenet feed is about 18,000
articles per day, with peaks to 1,200 articles per hour. Article storage
size is about 65 MB per day.

As for overview storage size, if you have 5 million articles, you'll
need at least 3.25 GB of disk space for buffindexed, 5.5 GB for ovdb
(4.5 GB if compressed), 4.65 GB for ovsqlite (2 GB if compressed),
and 3.10 GB for tradindexed.

If you store more header fields in overview data than the standard
ones, the space needed to store overview data will be superior than
these estimates. (This is configured in the extraoverviewadvertised
and extraoverviewhidden inn.conf parameters.)

------------------------------

Subject: 2.5. What are deferrals (NNTP code 431)?

Consider the following situation. You have two incoming peers, both of
which are getting ready to offer you an article in streaming mode. The
first sends you a CHECK <message-id> message, to which you respond
affirmatively (i.e., you don't already have the article). Then, before
that peer sends you the article with TAKETHIS, you receive a CHECK
<message-id> from the second peer for the same message. What response
does INN send to the second peer?

If deferrals are enabled (noresendid == false in incoming.conf for that
peer, the default), INN will send a 431 deferral telling that peer that
you may or may not want the article; try again later. Chances are that
when it retries, you will have received the article from the first peer
and you'll just refuse it. But if the first peer dies before it ever
sends you the article, this way you can still get it from the second peer.

If deferrals are disabled, INN will refuse the article from the second
peer, which means there's a possibility you'll lose news if the first peer
dies before sending you the article.

As a side note, some older versions of Diablo, upon receiving a deferral,
turn around and immediately send the article via TAKETHIS, which is
basically exactly what you don't want. (Chances are extremely high in
practice that the first peer will come through with the article.)

------------------------------

Subject: 3. Specific Problems

This section contains specific problems that are frequently reported when
using INN, and includes fixes or suggestions for fixes. Candidates for
inclusion in this section are any problems reported frequently on
news.software.nntp or inn-w...@lists.isc.org. Contributions, including
fixes, are very welcome.

------------------------------

Subject: 3.1. INN won't start after a new installation

The most common cause of this problem is that inndstart isn't setuid root
(please note that it only affects versions prior to INN 2.5.0 because
inndstart was removed in INN 2.5.0). inndstart must be installed owned
by root and group news, mode 4550. The ls -l output for inndstart should
look something like:

-r-sr-x--- 1 root news 53768 Jan 8 00:47 inndstart*

inndstart will automatically be installed with the right permissions if
you run make install as root. If inndstart isn't setuid root, it will log
errors to syslog when it tries to start and cannot. If you aren't seeing
those error messages in syslog either, you probably haven't set up syslog
properly (see 3.4).

The other most frequent cause of this problem is not correctly following
the instructions in INSTALL on how to set up the initial history database.
If you run makedbz without the -o flag, the initial history database files
will have names starting with history.n. These files must be renamed to
remove the ".n" before innd will start.

------------------------------

Subject: 3.3. The news server isn't keeping up with incoming news

Start by looking for the profile information in your nightly report. That
will tell you where the news server is spending most of its time and may
identify the exact nature of the problem.

This problem is quite frequently due to using the traditional spool
storage format for news articles. This storage method is now too slow to
be able to handle a full Usenet news feed (although with a more limited
selection of groups it can still do just fine). If your server is
spending a lot of time writing articles and you're using traditional
spool, this is probably the problem.

One possible solution would be to switch to CNFS as a storage mechanism.
You can do this simply by configuring CNFS (see INSTALL for details),
changing storage.conf to direct some or all of the incoming traffic to
CNFS buffers, and then restarting INN. Older articles will continue to be
stored in tradspool until they expire, but new articles will go into CNFS.

------------------------------

Subject: 3.4. news.notice is empty and the nightly report is missing things

You have syslog set up incorrectly.

INN logs nearly everything except article trace information via syslog.
It expects syslog to write its log messages into particular files under
~news/log, unless you gave it a different path at configure time (see
the pathlog parameter in inn.conf). You'll need to set up logging of
INN-related log messages in your system /etc/syslog.conf. See the
"Configuring syslog" section in INSTALL.

Note that you don't have to worry about rotating these log files;
news.daily (which should be run nightly from cron) will take care of that
and innreport generates a daily summary report from them.

------------------------------

Subject: 3.5. INN is running out of file descriptors

You may need to increase your system file descriptor limits. See the
"File Descriptor Limits" section of INSTALL for more details. This is
particularly a concern on Solaris systems, since Solaris by default has an
exceptionally low file descriptor limit.

------------------------------

Subject: 3.6. Can't get debugging information out of INN

The INN startup process is quite complicated, involving the rc.news shell
script (and the setuid inndstart wrapper for versions of INN prior to 2.5.0).
This can make it rather difficult to get enough debugging information out
of it to determine what's going wrong if it's crashing immediately after
startup or otherwise having serious difficulties.

One approach is to run innd by hand directly, giving it the -d option.
This requires setting up a configuration where innd doesn't need to bind
to privileged ports, however.

Another, sometimes better option, is move the innd binary to another name,
like innd-real, and put a shell script in its place. Here's an example,
from Kai Henningsen:

#! /bin/bash
# allow core dumps
ulimit -c unlimited
# save any output
exec > /tmp/innd.log 2>&1
# who are we running as, anyway?
id
# show exported environment
export
# start innd (don't forget the arguments, or it will complain)
exec strace -o /tmp/innd.strace -f -F /path/to/innd-orig "$@"

This starts innd under strace, suitable for debugging startup core dumps
and the like. You can use this as a general model for a variety of
debugging; for example, you could replace the strace invocation with an
invocation of gdb and then start innd from inside gdb with the -d option.

------------------------------

Subject: 3.7. Articles aren't being sent to remote peers

(This entry is based on a post by Jeffrey M. Vinocur.)

Here's how to trace through INN's logs to figure out what's happened to a
particular article. This should help you discover where the process of
feeding an article to a peer broke down.

1. First you look in the $pathlog/news file. There should be one line per
article (search for the Message-ID, or they're in order by time of
arrival if you know that).

If you don't see a line for the article in question, your innd has
never seen it. For articles being fed remotely, this means your peer
didn't feed it to you. For articles being posted to your server, this
generally indicates some sort of problem in nnrpd.

(The only other time you wouldn't see a line for the article in
question is if your innd has seen it in the past, and is considering
this attempt a "duplicate".)

2. Next, look at the second field of the line you've found in
$pathlog/news.

If it's "-", then your innd rejected the article. The reason should be
at the end of that line.

3. At this point, you should be looking at a line with "+" in the second
field. The article should be on your server at this point.

If it's not, either it's been cancelled, or has already expired.

4. You're now interested in whether the article was sent to your peers.
At the end of the same line in $pathlog/news, innd puts all of the
peers it thinks should receive this article.

If you don't see a peer you expect there, it indicates that
$pathetc/newsfeeds is not configured in the way you think it is.

5. If a peer is listed at the end of the line, the article should have
been fed to that peer.

If a peer doesn't have that article, it's possible that the article is
spooled on your system somewhere. Check $pathoutgoing, or the
innfeed spool if the peer is configured to use innfeed. (It's probably
easier to look for error messages in $pathlog/news.notice than to
actually wade around in $pathspool/innfeed.)

6. If you're sure the article isn't spooled, and it doesn't show up on the
peer, you have to consider the possibility that the peer has rejected
the article. Alternatively, it's possible that the peer has some
misconfiguration like the ones described above.

In either case, if you're sure that the article was offered to the peer
and not spooled, you will need the assistance of the peer's admin to
investigate further. INN does not generally log enough information
about outgoing articles to be able to tell more from your server alone.

It may be possible to get a slight bit of information from the remote
server by connecting with telnet (usually to port 119) and issuing
"IHAVE <message-id>". The peer may respond with something like "435
Duplicate" which means that the problem is not likely to be with your
server (it may be still a problem with the article itself). If the
peer responds with something like "335", your server probably did not
offer the article after all.

If you really are at a dead end and need to get more information about
what's going on with an outgoing feed, you can switch it from innfeed
to nntpsend (see INSTALL for instructions). You can then run it
manually with innxmit -dv, which will show the full conversation with
your remote peer.

------------------------------

Subject: 3.8. sendmail isn't installed

Yes, INN really does require sendmail. It uses sendmail to send out the
daily reports and to mail messages to moderators, and it assumes that you
have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
it can use to do this. It does not speak SMTP, nor is it likely to ever
speak SMTP; it's hard enough maintaining a package to speak NNTP.

If you need a very simple local sendmail implementation that just sends
mail to a smarthost, there are several available (nullmailer, for
example).

------------------------------

Subject: 4. Error Messages

Explanations of specific error messages, including solutions where
applicable.

INN logs nearly all messages to syslog, so in general these error messages
will be found in syslog. If you aren't seeing anything from INN in syslog
at all, make sure that you have it set up correctly (see 3.3).

------------------------------

Subject: 4.1. innd: SERVER cant store article

You probably have a misconfigured storage.conf. In current versions of
INN, "no matching entry in storage.conf" is added to the end of this
message unless it really is a disk I/O problem, making the cause
considerably clearer.

storage.conf(5) has this to say:

If an article doesn't match any entry, either by being posted to a
newsgroup that doesn't match any of the <wildmat> patterns or by being
outside the size and expires ranges of all entries whose newsgroups
pattern it does match, the article is not stored and is rejected by
innd(8). When this happens, the error message

cant store article: no matching entry in storage.conf

is logged to syslog. If you want to silently drop articles matching
certain newsgroup patterns or size or expires ranges, assign them to the
"trash" storage method rather than having them not match any storage
method entry.

One of the more frequent causes of this problem is misuse of the expires
key in storage.conf entries. Read the man page for storage.conf very
carefully if you're using the expires key, since it may not do what you
think it does. In particular, if you have a storage class that specifies
expires with a min-time greater than 0, it won't match any article without
an Expires header field (the vast majority of Usenet articles).

------------------------------

Subject: 4.2. innd: SERVER internal no control and/or junk group

Your active file isn't complete. Either it's been mangled by something or
it's missing some required entries. Even if you're running a small
stand-alone server for internal use that only carries a handful of groups,
there are some pseudogroups used internally by INN that you have to have.

Since INN isn't running (it won't start when this error occurs), you can
edit the active file by hand without worrying about stepping on INN's
toes. Make sure the following lines are present in the active file (if
the numbers are different, that's fine):

control 0000000000 0000000000 n
control.cancel 0000000000 0000000000 n
control.checkgroups 0000000000 0000000000 n
control.newgroup 0000000000 0000000000 n
control.rmgroup 0000000000 0000000000 n
junk 0000000000 0000000000 n

and then start INN again. The control* groups are for control messages
(messages with a named group will be filed into it, and all other control
messages will go into the top-level catch-all group). The n flag is so
that users won't post messages directly to the control* groups; control
messages should be posted to the groups that they affect instead and INN
will refile them automatically based on the Control header field.

If you have mergetogroups: set in inn.conf, you will also need to create
a newsgroup named "to". Otherwise, you will get the following error:

innd: SERVER internal no to group

------------------------------

Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

INN 2.3 and later have an internal optimization to the interface to
embedded filters that makes filtering about 15-20% faster, but which
disallows a trick that many versions of Cleanfeed use to count the number
of lines in the article. (This problem is fixed in current versions of
Cleanfeed.)

To correct this problem, find the line in Cleanfeed that looks like:

$lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

and change it to:

$lines = $hdr{'__LINES__'};

The __LINES__ hash value is set internally by all recent versions of INN
and is guaranteed to be correct.

------------------------------

Subject: 4.4. tradspool: could not open ... File exists

This error generally happens after a crash or unclean shutdown of innd
using the tradspool storage method, and is caused by overview information
being out of sync with what articles are in the spool. When innd was
restarted, it renumbered its active file (which determines the range of
existing articles in each group and therefore what article number is
assigned to new articles) based on the overview information. If there are
newer articles already on disk that aren't mentioned in the overview
(because the overview information for those articles hasn't been flushed
to disk yet), new incoming articles will get assigned the same number as
the existing article and then innd will fail to store the article and
throttle with this error.

In INN 2.4 and later when using the tradindexed overview method, you can
solve this problem by rebuilding the overview for any affected group.
Throttle the server (if it isn't already) and then run:

tdx-util -R <path-to-articles> -n <newsgroup>

where <newsgroup> is the newsgroup that INN is complaining about and
<path-to-particles> is the full path to the directory where the articles
for that group are stored (it's generally in the error message).
Immediately afterwards, run ctlinnd renumber for that newsgroup, and then
unthrottle the server.

The general solution to this problem, which works with any version of INN
and any overview method, is to shut down the server, delete all of your
overview database, and then rebuild it from your news spool with:

makehistory -O -x -F

This takes a long time and is to some degree overkill. For versions of
INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately
after restarting INN.

A third and better solution in some cases is to just remove all articles
in the spool that have higher numbers than the numbers in the active file.
Here's a Perl script that will do that. Just save this to a file, make it
executable, and run it, giving it the path to the active file as the first
argument and the path to the top of your tradspool news spool as the
second argument:

#!/usr/bin/perl
die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
while (<ACTIVE>) {
my ($group, $hi, $lo, $flag) = split;
my $directory = $group;
next if ($hi == 0 and $lo <= 1);
$directory =~ tr%.%/%;
$directory = $ARGV[1] . '/' . $directory;
if (-d $directory) {
opendir (DIR, $directory) or die "Can't open $directory: $!\n";
while (defined ($_ = readdir DIR)) {
unlink "$directory/$_" if ($_ > $hi);
}
closedir DIR;
}
}

If you're not already running INN 2.4, upgrade if you can. Not only can
you recover directly from this problem if you're using tradindexed
overview, but INN 2.4 does a better job of flushing data to disk and is
less likely to have this problem in the first place.

------------------------------

Subject: 4.5. Binary posting to non-binary group

This message does not actually come from INN. It's generated by
Cleanfeed, and if you're seeing it, that means that you have Cleanfeed
installed. At least at one point, the default Red Hat installation of INN
included Cleanfeed without documenting this particularly well.

In order to allow binaries in your local hierarchies, you should modify
the Cleanfeed configuration file to set bin_allowed to a regular
expression matching the groups that should allow binaries. Please don't
allow binary postings to regular Usenet newsgroups that you don't know
should have binaries, as they consume large amounts of bandwidth and
possibly disk space for other sites.

For more information on Cleanfeed configuration options, see the Cleanfeed
documentation and the comments in the default configuration file.

------------------------------

Subject: 5. Problems on Specific Systems

Problems specific to particular operating systems or platforms. Look here
if INN doens't behave as expected on your particular system, or if you're
having trouble compiling INN in the first place.

------------------------------

Subject: 5.1. INN won't compile on SCO OpenServer

On SCO OpenServer, it's worth noting that with a shared Perl library,
Perl on this platform doesn't apparently generate the right link magic
to include the path to the dynamic Perl libraries. You need to either
set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
binaries so that they can find the Perl libraries. (The former is
preferred, since then the path is encoded into the binaries and you
don't have to remember to set LD_LIBRARY_PATH later.)

------------------------------

Subject: 5.2. Using raw devices on Solaris destroys the partition table

If you use slice 2, or some other disk slice that includes the entire
disk, under Solaris as a raw partition for CNFS, you may run into this
problem. The symptoms are that INN manages to initialize the cycbuffs
just fine, but then gets invalid device errors when it tries to open them
again, and the disks show up in format as needing to be repartitioned.

The solution is to not use raw devices that include the first cylinder of
the disk. Solaris doesn't protect the superblock from being overwritten
by an application writing to raw devices and includes it in the first
cylinder of the disk, so unless you use a slice that starts with cylinder
1 instead of 0, INN will invalidate the partition table when it tries to
initialize the cycbuff and all further accesses will fail until you
repartition.

Generally all that has to be done is to repartition the disk with slice 0
starting from cylinder 1 and extending to the end of the disk and then
point INN at slice 0 instead of slice 2. You lose some small amount of
space, but generally not enough to care about.

------------------------------

Subject: 5.3. Will INN work on Windows?

It won't out of the box. The standard INN distribution doesn't build on
Windows. It has, however, been built for Cygwin (a Unix-like environment
for Windows) in the past and some of the necessary patches (although
perhaps not all of them) have been incorporated into current INN releases.

Search for http://homepage.mac.com/imeowbot/inn/ at
<http://web.archive.org/> for the previous work. Don't forget to peruse
INSTALL if you download and want to try this.

------------------------------

Subject: 5.4. Why aren't INN's files where the documentation says they are?

INN's default installation locations are intended to be convenient for
sysadmins adding INN to their system without disturbing other software.
They don't match any of the standards used by various Linux distributions
or other Unix packaging systems. Because of that, distributors who supply
INN packages often rearrange the files and directories.

Unfortunately, this is very confusing for system administrators, because
the documentation is not updated to reflect the modified locations of
files.

You can always get the details of how your system is configured by looking
in inn.conf at "pathnews" and similar parameters. But for convenience,
here are comparisons of INN's default locations with some of the most
common packages.

(Data courtesy of John F. Morse.)

DEFAULT DEBIAN
pathnews: /usr/local/news /usr/lib/news
pathbin: /usr/local/news/bin /usr/lib/news/bin
pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control
pathdb: /usr/local/news/db /var/lib/news
pathetc: /usr/local/news/etc /etc/news
pathfilter: /usr/local/news/bin/filter /etc/news/filter
pathhttp: /usr/local/news/http /var/www/inn
pathlog: /usr/local/news/log /var/log/news
pathrun: /usr/local/news/run /run/news
pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp
pathspool: /usr/local/news/spool /var/spool/news
patharchive: /usr/local/news/spool/archive /var/spool/news/archive
patharticles: /usr/local/news/spool/articles /var/spool/news/articles
pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming
pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing
pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

DEFAULT FEDORA
pathnews: /usr/local/news /usr/libexec/news
pathbin: /usr/local/news/bin /usr/libexec/news
pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control
pathdb: /usr/local/news/db /var/lib/news
pathetc: /usr/local/news/etc /etc/news
pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter
pathhttp: /usr/local/news/http /var/lib/news/http
pathlog: /usr/local/news/log /var/log/news
pathrun: /usr/local/news/run /run/news
pathtmp: /usr/local/news/tmp /var/lib/news/tmp
pathspool: /usr/local/news/spool /var/spool/news
patharchive: /usr/local/news/spool/archive /var/spool/news/archive
patharticles: /usr/local/news/spool/articles /var/spool/news/articles
pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming
pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing
pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

In addition, the FreeBSD port uses the standard INN paths except that it
puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

Most packages install INN's man pages into a system man directory
(/usr/share/man or /usr/local/man) rather than into a separate man
directory under news's home directory.

------------------------------

Subject: 5.5. Running INN on macOS

Richard Tobin provided the following advice in news.software.nntp on
2013-06-29 based on experience with running INN on Snow Leopard:

Mac OS X, at least through the GUI, won't let you create a group with
the same name as a user. So you can't use "news" for both.

The Perl module GD isn't installed by default. GPG is not installed
by default.

You probably want to turn off Spotlight for the news spool directory.

Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
"-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
files for the other architectures don't seem to be installed. I
edited Makefile.global by hand to remove them.

I needed to tell the application firewall to allow innd to accept
incoming connections. (A window pops up to ask you, but this doesn't
help when you're connected by ssh!)

When I ran rc.news form a terminal window, it stopped working when I
logged out. This is because of MacOS's convoluted and undocumented
way of doing DNS lookups. Using "nohup" fixed it -- not because of
anything to do with SIGHUP, but because nohup calls an undocumented
function related to "vprocmgr". Running from launchd shouldn't have
this problem, and it appears to be fixed in Mountain Lion.

The Perl flags come from the Perl configuration; this problem is fixed
with current builds of macOS.

------------------------------

Subject: 6. How Do I...

This section documents various common or uncommon tasks or configurations
that people want to do with INN. It is mostly taken from frequently asked
questions in news.software.nntp.

------------------------------

Subject: 6.1. Set up a server with no external feeds, just local groups

The basic steps are to set up a newsfeeds file empty except for internal
feeds like controlchan or overchan (if you're using either), have only
localhost in incoming.conf, and start INN with the default minimal active
file. Then, create the groups you want to carry with ctlinnd newgroup.
Set up reading permissions using readers.conf as appropriate for your
organization.

In other words, it's very much like setting up any other instance of INN,
but you don't bother with innfeed, nntpsend, or any of their configuration
files. INN may also complain that you have no feeds in newsfeeds; this is
harmless and can be ignored.

------------------------------

Subject: 6.2. Process a single control message

To process a single control message, you can use controlchan from the
command line. Just type either:

echo /path/to/article-file | controlchan

or:

echo @token@ | controlchan

if you have the storage API token of the article. (This assumes
controlchan is in a directory in your path.) This is useful mostly for
testing; if you just want to create, remove, or change a group, it's
easier to use ctlinnd (newgroup, rmgroup, or changegroup).

------------------------------

Subject: 6.4. Feed all articles on a server to another server

To feed all articles on an existing server to another one, regardless of
how they're stored on the server, first tell the new server to accept
articles regardless of how old they are (otherwise, INN will reject
articles older than artcutoff in inn.conf) and disable your filtering:

ctlinnd param c 0
ctlinnd perl n
ctlinnd python n

Note that rejected articles are remembered during the number of days
specified by the /remember/ line in expire.ctl; so, in case you forgot
to change the above parameters, you'll have to wait that number of
days before being able to inject them again. You can of course set
/remember/ to 0, run the expire process (for instance via news.daily
called with the same parameters as in crontab, plus "notdaily"),
undo the change in expire.ctl and then start the feed again.

You may also want to set xrefslave to true in inn.conf and then restart
INN on the new server if you want to keep the same article numbers as you
had on the old server. (It is notably helpful for news clients because
they otherwise get confused by an article renumbering in newsgroups they
are subscribed to.)

Next, make sure that the old server is listed in incoming.conf of the new
server, and reload incoming.conf with ctlinnd to pick up that change.
Also make sure that the new server carries exactly the same set of
newsgroups as the old server.

You may also want the new server not to propagate the articles it will
receive during this feeding operation, by checking that the newsfeeds
file of the new server is not configured to propagate articles to other
peers or controlchan (otherwise old control articles may be reprocessed).

Then try these commands (a variation on commands posted by Katsuhiro
Kondou to inn-workers) on the old server:

cd <pathdb in inn.conf>
perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
print "$_\n" if $_' history \
| tr . / > <pathoutgoing in inn.conf>/list
innxmit server list

where <pathdb> is the path to the directory containing the history file
(usually ~news/db), <pathoutgoing> is the path to the outgoing spool
directory (usually ~news/spool/outgoing), and server is the name of the
new news server to which you're feeding the articles.

In case you wish to only feed articles arrived on the old server
between two dates, you can adapt the previous commands. For instance,
the following commands will feed articles arrived between two given
timestamps (that can be computed with the convdate utility shipped
with INN).

convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

returns the two corresponding timestamps 1397586540 and 1397644620 that
can then be used to retrieve a subset of articles to feed:

cd <pathdb in inn.conf>
perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
my ($arrived, $expires, $posted) = split("~", $timestamps); \
print "$_\n" if $_ and $arrived >= 1397586540 \
and $arrived <= 1397644620' history \
| tr . / > <pathoutgoing in inn.conf>/list
innxmit server list

If innxmit stops transferring articles (with for instance an error like
"rewriting batch file and exiting"), just re-execute it.

When done, set xrefslave to false in inn.conf again if you changed it and
then either restart INN on the new server (necessary if you changed
xrefslave) or use another ctlinnd param command to set the cutoff value
back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
python to reactivate your filters.

Please note that when using xrefslave, this method requires that all of
the articles in your spool have Xref header fields. Current versions of INN
will always add an Xref header field, but very old versions (earlier 1.x
versions) will only add an Xref header field to crossposted articles. If
you're trying to import such a spool, you'll need to modify all of those
articles to add an Xref header field.

------------------------------

Subject: 6.5. Rename a newsgroup

INN has no native support for renaming a newsgroup, and doing so is
difficult, so the best advice is to not do this. If there's a way that
you can just create the new newsgroup, encourage people to start using it,
and then remove the old newsgroup, I recommend that. It's much easier.

Although it is not a renaming, it is also possible to create an alias.
Articles cannot be posted to that newsgroup, but they can be received
from other sites and treated as if they were actually posted to the
group named after the equal sign. However, their Newsgroups header field
body is not modified.

ctlinnd newgroup group.to.file.under y
ctlinnd changegroup old.group =group.to.file.under

Creating an alias newsgroup is useful in case you want residual articles
received under the old newsgroup name to be filed into the new group.

As for a renaming, if it really must be done, it's best if you're using
the tradspool storage method. The newsgroup of an article is stored in
the Newsgroups header field and in the Xref header field of the article
as stored on disk (and possibly in Followup-To), as well as determining
where the overview information is stored, and in the case of tradspool
is also encoded in the article's storage token. To rename a newsgroup
in tradspool, stop the server, move the directory containing all of
the articles to its appropriate new location in the news spool, edit
every article to change the old name to the new name in Newsgroups,
Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
and then rebuild history and overview with makehistory.

The following bit of Perl may help with the renaming (from Jeffrey
Vinocur):

#!/usr/bin/perl -wi
my ($src, $dst) = (shift, shift);
die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
unless(defined $dst);
while(<>) {
s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
print;
} continue {
close ARGV if eof;
}

Note that this may cause some problems if the newsgroup you're renaming is
contained in the name of another newsgroup to which messages in that group
are crossposted. If that's a problem, you may have to use a more
sophisticated script.

If any articles were crossposted to other newsgroups, you'll also have to
find and recreate the links in those newsgroups to the new location of the
articles (if the links were hard links and the process of changing the
Xref, Followup-To, Newsgroups header fields didn't break those links, you
may be lucky and be able to skip this).

If you're using another storage method, this is harder, although with
timehash you may be able to just change the Newsgroups, Xref, Followup-To
header fields of the articles in that newsgroup and then rebuild history
and overview as above.

One other approach that can be used regardless of storage method is to
refeed the articles to the server into a new newsgroup. This approach
works best if you're also changing news servers at the same time;
otherwise, the message IDs of the articles will already be in history, and
you'll have to change the message IDs of all of the messages or remove
them from the history database (such as by moving the articles away,
changing /remember/ to 0 so that old history entries won't be retained,
and then running expire to purge them out of history). To do this, get
all of the messages into a directory (by pulling them down via NNTP or
some other method), change the Newsgroups, Xref, and Followup-To header
fields to rename the newsgroup, and then create a file containing paths to
all of the articles, one per line. You can then use that file as input to
innxmit, pointing it at the server to which to feed the articles, and if
the articles aren't listed in history on that server and it carries the
new group, they will be accepted into the new newsgroup.

Note that if you use this method and something goes wrong the first time,
the message IDs will probably have all been added to history on the new
server and the articles now will never be accepted until those entries are
removed from history again (or all the message IDs changed).

------------------------------

Subject: 6.6. Change the domain used for message IDs

By default, any message IDs generated by INN will use the domain of the
local system for the right-hand-side of the message ID. In some cases,
this isn't desirable for various reasons (the server may have an internal
name that doesn't make sense on Usenet at large, or one may not want to
expose the name of the server).

In INN 2.3.3 and later, you can set virtualhost: to true in an access
stanza of readers.conf and then set domain: in the same stanza, and all
posts coming from connections to which that access stanza applies will use
that domain to generate message IDs. So if you need to change the domain
used to generate message IDs for every local post from your server, just
add virtualhost: and domain: keys to every access stanza in readers.conf.

This is really overkill for this option, and eventually the domain:
parameter in inn.conf will probably be changed to allow this to be
modified for the whole server. (Right now, domain: in inn.conf means
something completely different.)

------------------------------

Subject: 6.7. Use INN without a direct news feed

INN is designed to be used as a regular news server, receiving direct news
feeds from other news servers and sending news directly to other news
servers using the peer-to-peer portions of the NNTP protocol. However,
with some additional software, it is also possible to use INN as, in
essence, a local cache for a news server that you can use to read and post
but which doesn't treat your server like a peer.

This configuration is generally called a "suck" feed, because rather than
having news fed directly to your server, you pull it down or "suck" it
from another news server, and because possibly the first and one of the
most widely used packages for doing this is named suck.

The software to pull down articles from another server and to feed
articles to another server using post rather than peer-to-peer commands
does not come with INN (INN has a few utilities to do this on a small
scale, but not really anything designed to handle a lot of groups or a lot
of articles). You will need an external package to do this. The two most
popular are suck and newsx; however, both sites appear to be unavailable
as of thos writing. You may be able to find a package in your local
distribution or package repository.

Note that current versions of INN refer to articles internally using a
storage API token, not a path name, which is not always what suck or newsx
expects. Read the documentation carefully; you'll need to use a script or
configuration that retrieves articles using the sm program that comes with
INN rather than trying to open files directly.

It's also worth noting that INN is a fairly complex package, and while
many people are running it successfully using this sort of configuration
and like having a full-fledged news server available to them, other people
have found INN rather complicated and difficult to configure for a small,
simple personal news cache. If your needs and goals are simple and the
number of groups you're interested in is small, you may be better off with
a smaller, lighter package such as LeafNode or NNTPcache.

------------------------------

Subject: 6.8. Generate MRTG graphs for INN

INN's CNFS storage system has direct support for producing information
suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running
cnfsstat -p will generate sample MRTG configuration fragments for each
cycbuff.

To generate MRTG graphs of the usage of the buffindexed overview system,
try the following configuration fragment:

Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
MaxBytes[overview-BUFF]: 100
Title[overview-BUFF]: BUFF1 Usage
Options[overview-BUFF]: growright gauge
YLegend[overview-BUFF]: Overview Buffers
ShortLegend[overview-BUFF]: %
PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
<BR><TT>overview</TT>

where the overview.sh script is:

#!/bin/sh
echo "100"
<pathbin in inn.conf>/inndf -o | awk '{print $1}'
echo "0"
echo "overview"

This sample configuration is from Basil Kruglov. Note that you can
instead use -n (for total count of articles); in that case, you'll want to
remove the MaxBytes setting above or change it to be some sensible limit
on the total number of articles you receive. You'll also want to change a
few of the other labels in the MRTG configuration.

I'm not aware of any packaged solutions for generating MRTG data from
other things, such as incoming or outcoming news flows. If anyone has any
pointers, let me know.

------------------------------

Subject: 6.9. Hide the junk and control groups from users

The junk, control, and control.cancel groups must exist in the active file
for the proper operation of INN, so you can't remove the groups entirely.
You can, however, hide them completely from users.

To do this, edit readers.conf, and for each user access group where you
want to hide the junk and control groups, add "!junk,!control,!control.*"
to the newsgroups pattern. In other words, if you have a line like:

newsgroups: *

just change that to:

newsgroups: *,!junk,!control,!control.*

If you use read and post patterns instead, do the same for each of them
individually. The groups will then no longer show up on the server for
users to which that access group applies; it will be as if they do not
exist.

------------------------------

Subject: 6.10. Modify the body of posts made through my server

You can't without either making code changes to INN or putting your own
software in the path of incoming posts. This is intentional.

Some sites like to try to append a standard signature to all posts through
their service, generally as advertising. This creates the appearance of
users saying things that they didn't, runs the risk of corrupting messages
by appending text without regard to what's in the message, and can
possibly modify messages that arrive via a suck/rpost connection. It also
adds advertising in an obnoxious location, rather than in the Organization
header field which is more widely used for that purpose. Accordingly, INN
does not support this, or any other modification of the body of a message
from inside the news server.

If you only want to do this for a private hierarchy, the easiest way to do
this (as well as any other modifications and internal filtering that you
want to perform) is to mark all of the groups as moderated and route all
submissions through a script that makes whatever modifications you want
and then posts the messages with an Approved header field.

If you want to do this in order to advertise your service, please
reconsider. You can add your advertisements to the headers, like many
other news service providers.

------------------------------

Subject: 6.11. Hide the Injection-Info header field

There is no built-in support for suppressing generation of the
Injection-Info header field. You can, however, remove it from inside
a Perl posting filter. Try using a posting filter like this:

sub filter_post {
$modify_headers = 1;
delete $hdr{'Injection-Info'};
return '';
}

Note that you have to set $modify_headers to make changes to the article
header field effective in the actual posted article. Instead of removing
the header field, you can also alter it if you modify
$hdr{'Injection-Info'}. If you only want to alter the host name used in
Injection-Info, see the virtualhost: and domain: parameters in
readers.conf.

------------------------------

Subject: 6.12. Run innd and nnrpd on separate ports

Originally, innd was designed to handle all incoming connections and hand
them off to nnrpd as appropriate. It is, however, becoming increasingly
common to run innd and nnrpd on separate ports for a variety of reasons,
such as wanting to handle connections to nnrpd with a smart network
connection handling daemon like xinetd that can do things like rate
limiting of connections. INN does support this configuration, but be
warned that since you need to run nnrpd on port 119 for most reader
clients to be able to find it, you'll need to tell all of your news peers
to use a different port to feed you news.

The recommended alternate port for innd transit-only connections is port
433, which has been reserved for that purpose. If you want to use some
low-numbered port (less than 1024) other than 119 or 433 for innd, you
will need to build INN with the --with-innd-port option specifying that
port.

Now, set port in inn.conf to the port you want to run innd on and add
noreader: true (so that innd will never attempt to hand connections off to
nnrpd). Then, restart INN. It will now be listening on the new port.
You should now set up nnrpd to run via xinetd, inetd, tcpserver, or
some other similar network connection handling daemon on port 119. Make
sure that nnrpd is run as the news user, not as root. You don't have to
pass any arguments to nnrpd (unless you want to).

------------------------------

Subject: 6.13. Back up and restore an INN installation

(This entry is based on a post by Jeffrey M. Vinocur.)

For a full backup, you need, at a minimum, to save all of the articles in
$patharticles, the configuration files in $pathetc, and the active and
newsgroup files in $pathdb. If you have any custom filters you've
installed, or a cleanfeed.local file, you'll want to keep that, as well as
any custom authentication programs or files you're using (like a password
file for news accounts). You may also want to save HTML versions of the
news.daily reports, if you've been generating them, and you may want to
look at the first few lines of config.status in your original source tree
so that you can be sure to use the same options to configure.

Note that most people only back up those portions of the news spool that
they retain for a long time (like local hierarchies) and don't bother with
all the regular Usenet articles.

It's considerably easier to back up and restore articles from tradspool
than any other storage mechanism, and it's quite hard to back up and
restore timecaf or CNFS. Remember that you can use different storage
methods for different articles. I highly recommend saving the hierarchies
you want to back up in tradspool and use the higher-performance storage
mechanisms for news you don't care as much about.

To restore a single newsgroup using tradspool and the tradindexed overview
method, you can just restore the articles into the news spool and then
rebuild overview for just that group with tdx-util -R.

Otherwise, for more general restorations, compile INN on the new system
with the same ./configure command if you've lost the installation, run
make install, then put all the pieces back where they belong. Now, you
have to run:

makehistory -O

to rebuild the history and overview databases. When that finishes, cd to
the $pathdb directory and run:

makedbz -s `wc -l < history` -o

You should now be able to start the server and read and post news to it.

------------------------------

Subject: 6.14. Find external feeds and set up peering

You can ask for an external feed in the news.admin.peering newsgroup.
Several news administrators will certainly respond and gracefully provide
you with a news feed. And why not keep subscribed to this newsgroup to
help others find a news feed once you get yours?

You then have to parameter the NNTP feed in incoming.conf, newsfeeds and
innfeed.conf (assuming you'll use innfeed, the most common way to feed
articles). Follow the examples in the man pages of these configuration
files.

[Julien ÉLIE]

Hi all,

Following recent discussions here, I've added a reference to the
news.admin.peering newsgroup in the FAQ:


> 6. How Do I...

> 6.14. Find external feeds and set up peering >
> You can ask for an external feed in the news.admin.peering newsgroup.
> Several news administrators will certainly respond and gracefully provide
> you with a news feed. And why not keep subscribed to this newsgroup to
> help others find a news feed once you get yours?
>
> You then have to parameter the NNTP feed in incoming.conf, newsfeeds and
> innfeed.conf (assuming you'll use innfeed, the most common way to feed
> articles). Follow the examples in the man pages of these configuration
> files.

Do not hesitate to make suggestions of new questions (with the
appropriate answer if you already know it) or additions to existing answers.

--
Julien ÉLIE

« Moi, je n'ai pas goûté le sel de cette plaisanterie. » (Astérix)

[Miner]

Russ Allbery wrote:

> Subject: 6.14. Find external feeds and set up peering

After the discussion in group news.admin.peering, it became clear
that chapter 6.14 of the INN 2.x FAQ need to be updated with the
following conditions:

* administrator must have an account on "reliable email service" [1];
* administrator should assist the third party in achieving
commercial interests [2].

It probably makes sense to write a definition of "reliable email
service" and list them.

References:
1. Message-ID: <tamep7$2r8ph$1...@news.mixmin.net>
2. Message-ID: <tamtot$12n$1...@txtcon.i2p>
--
Miner

[Russ Allbery]

Miner <inv...@invalid.invalid> writes:

> After the discussion in group news.admin.peering, it became clear that
> chapter 6.14 of the INN 2.x FAQ need to be updated with the following
> conditions:

> * administrator must have an account on "reliable email service" [1];
> * administrator should assist the third party in achieving
> commercial interests [2].

> It probably makes sense to write a definition of "reliable email
> service" and list them.

Some people may have those requirements, some people may not. I don't
think the INN FAQ is the place to get into specifics about what peering
policies other people may have.

I certainly completely disagree with the second point and that is not part
of any peering policy that I have (if anything, the exact opposite; I
generally don't peer with commercial sites).

--
Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

[G.K.]

On 7/14/22 05:38, Miner wrote:

<snip>

It appears you may be interested in Bitmessage. It is similar to having
usenet and mixmaster rolled into one.

https://bitmessage.org

Bitmessage is a anonymous, peer-to-peer messaging protocol that allows
for public and private chans and direct private messages. Chans are
similar to newsgroups. It is completely decentralized and totally
uncensorable. It has support for Tor and Tor hidden services built-in.

Additionally, Bitmessage supports broadcasts and decentralized mailing
lists.

Messages are encrypted and signed with the algorithms used by Bitcoin.
It also employs dandelion mix routing for privacy. A QT interface is
available with the standard reference implementation.

--

G.K.

[Grant Taylor]

On 7/14/22 8:45 AM, Russ Allbery wrote:
> Some people may have those requirements, some people may not.

I'll copy & paste a comment I made about this in the news.admin.peering
newsgroup.

I consider the desire ~> requirement for valid email address as one of
network operations. As in server administrators need a reliable way to
get a hold of each other to communicate about repairing the network when
the network is down. Or out of bands if you will.

> I don't think the INN FAQ is the place to get into specifics about
> what peering policies other people may have.

I think that it's probably worth while for one of the INN documents, if
not the FAQ, to mention what is likely needed to establish peering
connections.

Lest we end up with documentation on how to stand up a server that's an
island with no information on how to build bridges to / from that island.

I think /not/ having /some/ information on this is a disservice to new
news masters.

> I certainly completely disagree with the second point and that is
> not part of any peering policy that I have (if anything, the exact
> opposite; I generally don't peer with commercial sites).

I wonder if that's old verbiage meant to discourage de-peering with a
peer in the event that a previously non-commercial peer tries to turn
commercial. I feel like the spirit of RFC "SHOULD" / "SHOULDN'T" vs
"MUST" / "CAN'T" comes to mind.



--
Grant. . . .
unix || die

[Russ Allbery]

Grant Taylor <gta...@tnetconsulting.net> writes:

> I think that it's probably worth while for one of the INN documents, if
> not the FAQ, to mention what is likely needed to establish peering
> connections.

I think this would fall into a similar category as the separate hierarchy
administration FAQ that I maintain (badly, or at least slowly). It's a
bit outside the scope of INN as a piece of software, which is perfectly
happy to be used to exchange news among a private set of peers who
communicate only via carrier pigeon.

The nice thing about a general FAQ on being a good public Usenet news
admin is that anyone can write it and maintain it and it doesn't need to
be tied to a specific piece of news software. :)

> I think /not/ having /some/ information on this is a disservice to new
> news masters.

I would be happy to review and offer suggestions on any document you chose
to start!

I get the impression that the original message was mostly a continuation
of some argument in news.admin.peering by other means, though, which to me
is another reason for the INN FAQ to stay out of it.

[Miner]

G.K. wrote:

> On 7/14/22 05:38, Miner wrote:
>
> <snip>
>
> It appears you may be interested in Bitmessage. It is similar
> to having usenet and mixmaster rolled into one.

Bitmessage causes an little justified load on CPU. Gretta Tunberg
probably denounces this. :-)

--
Miner

[Miner]

Get new additions to the FAQ.

Before asking for a peering:
* administrator must register the domain and probably should pay
for it [3];
* administrator should set up and run the MTA [4];

References:
3. Message-ID: <taq3dh$34539$1...@news.mixmin.net>
4. Message-ID: <tapl1i$3369a$1...@news.mixmin.net>

--
Miner

[Ray Banana]

Thus spake Russ Allbery <ea...@eyrie.org>

> I get the impression that the original message was mostly a continuation
> of some argument in news.admin.peering by other means, though, which to me
> is another reason for the INN FAQ to stay out of it.

+1

--
Too many ingredients in the soup, no room for a spoon
http://www.eternal-september.org

[Matija Nalis]

On Fri, 15 Jul 2022 13:06:00 +0200, Ray Banana <ray...@raybanana.net> wrote:
> Thus spake Russ Allbery <ea...@eyrie.org>
>
>> I get the impression that the original message was mostly a continuation
>> of some argument in news.admin.peering by other means, though, which to me
>> is another reason for the INN FAQ to stay out of it.
>
> +1

I agree. Also, many things are implied by common logic and it would be waste of everybody's time to include. e.g.

- to contact other people, one should have some common agreed medium for contact.
These days e-mail is usually implied, in times past it was postal adress, in a century it might be facebook account
or some other horrible distopian stuff.
- in order to peer over network, one needs functional network connectivity of some kind
- to edit config files, one should know how to use some available editor, etc.

There is absolutely no need to specify those in *INN* FAQ. Perhaps is some "computing 101" FAQ...

--
Opinions above are GNU-copylefted.

[Miner]

Matija Nalis wrote:

> On Fri, 15 Jul 2022 13:06:00 +0200, Ray Banana <ray...@raybanana.net> wrote:
> > Thus spake Russ Allbery <ea...@eyrie.org>
> >
> >> I get the impression that the original message was mostly a
> >> continuation of some argument in news.admin.peering by other
> >> means, though, which to me is another reason for the INN FAQ
> >> to stay out of it.
> >
> > +1
>
> I agree. Also, many things are implied by common logic and it
> would be waste of everybody's time to include. e.g.

Administrator must have comprehensive information before setting
up the service. Lack of information turns into wasted time.

--
Miner

[Grant Taylor]

On 7/15/22 5:23 AM, Matija Nalis wrote:
> I agree. Also, many things are implied by common logic and it would
> be waste of everybody's time to include.

I disagree.

1st "common logic" is both regionally and chronologically dependent. It
used to be common logic that smoking on a plane was okay.

2nd given the above, I find it's better to clarify things, even if
believed to be common logic, so that people that are in a different
region or different time (years later) have a frame of reference from
the document that no longer exists for them.

Perhaps some of these things don't need to be /in/ the INN FAQ itself.
But I do think it would be good to have a pointer in the INN FAQ
referencing some of these things. E.g.

Q: What do I do now that my INN server is up and running?
A: See the Peering FAQ document for more details.

[Grant Taylor]

On 7/15/22 12:02 AM, Miner wrote:
> Before asking for a peering:
> * administrator must register the domain and probably should pay for it
> [3];

I can't support "/must/ register a domain name". I see zero problems
with using a hostname in an existing domain name.

I can support "/must/ use a /registered/ domain name".

The difference to me has to do with if a new news master needs to
register a new domain for their own use or not.

> * administrator should set up and run the MTA [4];

I question if the news master really /should/ set up and run an MTA. I
completely agree that an email address MUST be serviceable. But why
does there need to be an MTA on news.example.net when example.net
already has a fully functional MTA stack? What's more is why can't
news.example.net use news-example-net@<$FREE_EMAIL_PROVIDER>?

I agree that using an email address in the same (parent) domain is
preferred. But I don't think that preference even qualifies as a SHOULD.

[Russ Allbery]

Grant Taylor <gta...@tnetconsulting.net> writes:

> I can't support "/must/ register a domain name". I see zero problems with
> using a hostname in an existing domain name.

> I can support "/must/ use a /registered/ domain name".

> The difference to me has to do with if a new news master needs to register
> a new domain for their own use or not.

I feel like this thread is assuming the very specific and narrow context
of wanting to peer with other random people one doesn't already know, on
the public Internet or on some privacy-preserving overlay of it, carrying
generally-distributed Usenet groups. In other words, that's a
news.admin.peering FAQ addressing the specific social context of that
group, but not really taking into account the broader news.software.nntp
world.

Nothing wrong with that! But this is why I don't think the INN FAQ is the
place for it apart from a pointer (which I'd be happy to add once such a
document existed).

Just to say this for the record: none of INN nor news.software.nntp nor
netnews are limited to this configuration. It works with IP addresses
with no DNS. It works on private networks. INN can provide stand-alone
service for entirely private groups. Netnews as a protocol is a much
bigger world than public peering of Usenet groups.

[Grant Taylor]

On 7/15/22 12:25 PM, Russ Allbery wrote:
> ...

ACKs all around and well said Russ.

[Miner]

Grant Taylor wrote:

> On 7/15/22 12:02 AM, Miner wrote:
> > Before asking for a peering:
> > * administrator must register the domain and probably should pay for it
> > [3];
>

> The difference to me
No difference for me. I have no domain.

> > * administrator should set up and run the MTA [4];

> But why
Discuss details with Ishmel.

--
Miner

[Grant Taylor]

On 7/15/22 2:12 PM, Miner wrote:
> Discuss details with Ishmel.

Who and where?

I don't see Ishmel listed as a sender in this thread.

[Ishmel Rowe Dent]

On 7/15/22 13:04, Grant Taylor wrote:
> It used to be common logic that smoking on a plane was okay.

Prude.

--

Ishmel Rowe Dent

[Miner]

Russ Allbery wrote:

> In other words, that's a news.admin.peering FAQ addressing the
> specific social context of that group, but not really taking
> into account the broader news.software.nntp world.

Just the facts.
1. News.admin.peering FAQ does not exist or hasn't been posted in
the past few months.
2. INN 2.x FAQ does not cover an important aspect of usage. The
lack of complete information caused a waste of time.

Therefore, the inaction of the author of the INN 2.x FAQ from now
on can be regarded as disinformation. Probably with the aim of
making attractive a little popular software product or, in other
words, fossil of ancient times.

> INN can provide stand-alone service for entirely private
> groups.

I have never seen usage in this way. Companies and organizations
prefer other solutions.
Norton Commander also provide something. However, no one is using
it.

--
Miner

[Miner]

Grant Taylor wrote:

> On 7/15/22 2:12 PM, Miner wrote:
> > Discuss details with Ishmel.
>
> Who and where?
> I don't see Ishmel listed as a sender in this thread.

I noticed you're abusing [skip] often. There was references on
Message-ID.

--
Miner

[Ishmel Rowe Dent]

On 7/16/22 06:40, Miner wrote:
> Russ Allbery wrote:

<snip>

>> INN can provide stand-alone service for entirely private
>> groups.
>
> I have never seen usage in this way. Companies and organizations
> prefer other solutions.

YGBSM. You can't seem to set up a peering without a mountain of friction
and now you're pretending to be an expert on Usenet software proliferation.

I am subscribed to several private NNTP servers that don't peer with
Usenet. There are thousands of private NNTP servers in active use
world-wide. Nobody acts as toxic as you on any of the servers I
subscribe to, and many of them are using opsec to hide their identities.
If they want to peer, they know that email and domain name is standard.

There are many such servers. There are also special-purpose NNTP feed
forwarding servers such as gmane.io and gwene.io that don't peer with
the larger Usenet hierarchies. You're just a crank troll and you've
proven it repeatedly. I smell rodent.

STFU and peer with someone by giving them a email address and domain
name, or just go away and run a private server and be happy in your
anonymous pedo-pirate-carder-tweaker-opsec bubble. I think you are a
crank and your wank disgusts me.

Really dude, it's time to stop this nonsense and grow up. Your trolling
smells of Kerman the German wank.

--

Ishmel Rowe Dent

[Miner]

Ishmel Rowe Dent wrote:

> On 7/16/22 06:40, Miner wrote:
> > Russ Allbery wrote:
>
> <snip>
>
> > > INN can provide stand-alone service for entirely private
> > > groups.
> >
> > I have never seen usage in this way. Companies and
> > organizations prefer other solutions.
>
> YGBSM. You can't seem to set up a peering without a mountain of
> friction and now you're pretending to be an expert on Usenet
> software proliferation.

Instead of commenting the facts, you generate unclaimed thoughts
in abundance.

As it became known, the peering requires many conditions. The
conditions remained unknown at the stage of setting up the
service. This was the result of misinformation (or is it better
to say disinformation?) in the FAQ. Now let me improve FAQ. No
thanks needed.

> I am subscribed to several private NNTP servers that don't peer
> with Usenet. There are thousands of private NNTP servers in
> active use world-wide. Nobody acts as toxic as you on any of
> the servers I subscribe to, and many of them

I announced the InterNetNews to potential users. They called the
InterNetNews "petrified shit of mammoth". For the common good
ongoing testing of its toxicity. :-)

--
Miner

[Russ Allbery]

Miner <inv...@invalid.invalid> writes:

> Just the facts.
> 1. News.admin.peering FAQ does not exist or hasn't been posted in
> the past few months.
> 2. INN 2.x FAQ does not cover an important aspect of usage. The
> lack of complete information caused a waste of time.

> Therefore, the inaction of the author of the INN 2.x FAQ from now
> on can be regarded as disinformation. Probably with the aim of
> making attractive a little popular software product or, in other
> words, fossil of ancient times.

Oh, you're adorable. It's been about twenty years since this sort of
thing has worked on me, but nice try! A very honest attempt to capture
the spirit of 1990s Usenet.

This fossil of ancient times has learned one of the lessons that fossils
learn, which is that if you're doing something as a hobby, you should do
it however makes you happy and not care too much about what anyone else
thinks about what you're doing. Life is too short to argue about your
hobbies. If someone doesn't like what you're doing, they can go do
something else.

>> INN can provide stand-alone service for entirely private groups.

> I have never seen usage in this way.

Yes, I'm not surprised. Nonetheless, if you pay attention, people talk
about it from time to time in this very group.

Like all uses of INN and Usenet in general, it's gone down over time as
people have mostly switched to other protocols and other software stacks.

> Companies and organizations prefer other solutions. Norton Commander
> also provide something. However, no one is using it.

I hate to break this to you, but "no one is using it" is true about Usenet
and netnews *in general*. We are a very small backwater with a very
obscure hobby.

And not only is that fine, it's often a nice place to be. We're part of
the world's diversity. It would be boring and oppressive if everyone was
using the popular thing.

[Miner]

Russ Allbery wrote:

> Miner <inv...@invalid.invalid> writes:
>
> > Just the facts.
> > 1. News.admin.peering FAQ does not exist or hasn't been
> > posted in the past few months.
> > 2. INN 2.x FAQ does not cover an important aspect of usage.
> > The lack of complete information caused a waste of time.
>
> > Therefore, the inaction of the author of the INN 2.x FAQ from
> > now on can be regarded as disinformation. Probably with the
> > aim of making attractive a little popular software product
> > or, in other words, fossil of ancient times.
>

> This fossil of ancient times has learned one of the lessons
> that fossils learn, which is that if you're doing something as
> a hobby, you should do it however makes you happy and not care
> too much about what anyone else thinks about what you're doing.
> Life is too short to argue about your hobbies. If someone
> doesn't like what you're doing, they can go do something else.

I can't be happy when the result of my efforts is very different
from the expected. My expectations were based on the content of
the FAQ.

> >> INN can provide stand-alone service for entirely private groups.
>
> > I have never seen usage in this way.
>
> Yes, I'm not surprised. Nonetheless, if you pay attention,
> people talk about it from time to time in this very group.
>
> Like all uses of INN and Usenet in general, it's gone down over
> time as people have mostly switched to other protocols and
> other software stacks.

There is reason to think about the cause of this phenomenon. The
conceptual advantages of the Usenet are undeniable.

> > Companies and organizations prefer other solutions. Norton
> > Commander also provide something. However, no one is using
> > it.
>
> I hate to break this to you, but "no one is using it" is true
> about Usenet and netnews *in general*. We are a very small
> backwater with a very obscure hobby.

Degradation is often the result of error. It remains to find the
error and fix before it's too late.

--
Miner

[Russ Allbery]

Miner <inv...@invalid.invalid> writes:

> I can't be happy when the result of my efforts is very different from
> the expected. My expectations were based on the content of the FAQ.

You've got a point that the answer to that question expressed rather more
certainty about getting a feed than may be warranted. I've reworded it a
bit:

One way to find peers is to ask for an external feed in the
news.admin.peering newsgroup. Some news administrators read that
group and may be willing to peer. It's common for news administrators
to have different criteria for peering (specific hierarchies,
geographic or network proximity, spam filtering, no binaries,
binaries, specific network protocols; the variation is endless), so
finding someone with matching goals may require some patience and
possibly some configuration changes. And why not keep subscribed to

this newsgroup to help others find a news feed once you get yours?

> There is reason to think about the cause of this phenomenon. The
> conceptual advantages of the Usenet are undeniable.

Usenet has an incredibly clunky, awkward, and insecure mechanism for
moderation, which is a fatal flaw at any level of popularity much higher
than its current level and which (combined with the constant problem of
finding enough people to do the moderation) is a lot of the reason why
Usenet only has its current level of popularity. General-distribution
newsgroups only survive the lack of moderation tools because this is a
niche hobby with a tiny community, which means it's relatively rare that
someone will actively try to break it for everyone else.

Back when it was far more popular, other people routinely tried to break
it for everyone else and, due to the inherent insecurities in the
protocol, mostly succeeded.

Even with that small community, there is absolutely no way that I would
legally risk running a Usenet server that accepted binaries given the
utter lack of tools or support for keeping child sexual abuse material off
of Usenet. Other people obviously make different decisions and, well,
good luck to them, I guess.

> Degradation is often the result of error. It remains to find the error
> and fix before it's too late.

Have fun! I'm just going to be over here doing my thing. :)

[Julien ÉLIE]

Hi Russ,

> You've got a point that the answer to that question expressed rather more
> certainty about getting a feed than may be warranted.

Yup, agreed, my mistake. Re-reading the previous wording, I indeed was
too much optimistic; of course nothing is ever guaranteed.

> I've reworded it a bit

You did well, thanks.
You proved not to be a fossil :-)

Both you and Grant have always proven to be reliable, pro-active and
helpful to others!

--
Julien ÉLIE

« C'est comme chercher une aiguille dans du foin en bottes ! »
(Jolitorax)

[Grant Taylor]

On 7/16/22 6:02 AM, Miner wrote:
> I noticed you're abusing [skip] often. There was references on
> Message-ID.

I think the messages that you're referring to didn't make it to my news
server.

I now see only two messages from Ishmel on this thread.

[Sn!pe]

Grant Taylor <gta...@tnetconsulting.net> wrote:

> On 7/16/22 6:02 AM, Miner wrote:
> > I noticed you're abusing [skip] often. There was references on
> > Message-ID.
>
> I think the messages that you're referring to didn't make it to my news
> server.
>
> I now see only two messages from Ishmel on this thread.
>

PMFJI. It's the same on E-S.

[relurk]

--
^Ï^ My pet rock Gordon just is.

~ Slava Ukraini ~

[Miner]

Russ Allbery wrote:

> You've got a point that the answer to that question expressed
> rather more certainty about getting a feed than may be
> warranted. I've reworded it a bit:
>

> for news administrators to have different criteria for
> peering (specific hierarchies, geographic or network
> proximity, spam filtering, no binaries, binaries, specific
> network protocols; the variation is endless), so finding

Several criteria are missing: reveal a information (what
exactly?), promoting the commercial interests of a third party,
having an external IP address, buying a domain, setting up
additional software.

String "the variation is endless" need to be updated with
"including the insane".

--
Miner

[Richard Kettlewell]

Miner <inv...@invalid.invalid> writes:
> Russ Allbery wrote:

>> You've got a point that the answer to that question expressed
>> rather more certainty about getting a feed than may be
>> warranted. I've reworded it a bit:
>>
>> for news administrators to have different criteria for
>> peering (specific hierarchies, geographic or network
>> proximity, spam filtering, no binaries, binaries, specific
>> network protocols; the variation is endless), so finding
>
> Several criteria are missing: reveal a information (what
> exactly?), promoting the commercial interests of a third party,
> having an external IP address, buying a domain, setting up
> additional software.

I would add “don’t be high-maintenance” as a general rule.

--
https://www.greenend.org.uk/rjk/

[Thomas Hochstein]

Miner schrieb:

> Just the facts.
> 1. News.admin.peering FAQ does not exist or hasn't been posted in
> the past few months.

No problem. Create it, post it. Otherwise you'll cause a waste of time,
and your inaction could be regarded as disinformation. We wouldn't want
that, would we?

-thh

[Matija Nalis]

On Fri, 15 Jul 2022 12:04:20 -0600, Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 7/15/22 5:23 AM, Matija Nalis wrote:
>> I agree. Also, many things are implied by common logic and it would
>> be waste of everybody's time to include.
>

> 1st "common logic" is both regionally and chronologically dependent. It
> used to be common logic that smoking on a plane was okay.

And it was okay back then, wasn't it? That is exactly my point.
Things change, and things are often different in different regions.

For example, back when I was still admining public news servers, e-mail was not
required for peering in this region. Instead, XMPP and IRC were often
alternatives, even preferred by some. At least in my region...

> 2nd given the above, I find it's better to clarify things, even if
> believed to be common logic, so that people that are in a different
> region or different time (years later) have a frame of reference from
> the document that no longer exists for them.

Sure. It would probably need to cover lots of regions (and maybe historical
information, as it would need to be kept up-to-date), or would need to be very
open-ended/vague, lest it possibly lead to more confusion than it solves.
It is likely much harder than it looks on first sight.

Even things like language and peering places used differ wildly.
(e.g. do high percentage of Chinese newsadmins choose to find peers by posting
in English on news.admin.peering? Neither did most newsadmins in .hr;
yet peer they did - and some still do. Not all news servers even carry Big8!)

While some peering requests on news.admin.peering and elsewhere might gain
much more traction because they have made popular choices, this does not
_per se_ invalidate the other (more quirky) peering requests.

There were several issues raised, none of which IMHO have *definite* answer,
only some *recommendations* iff one wants to get more exposure (which is not to
be taken as implied, either [1]):

- "you need to buy a domain / have a working domain name to peer" (not true at all)

- "you need a working e-mail / MTA to peer" (not true, see above about my experiences
with other communication methods, as well as recent threads in news.admin.peering,
I still prefer XMPP to this day for things I care about - it is faster, and things
tend to actually reach intended recipient much more reliably than e-mail, which is
IME getting borderline useless these days due to spam and antispam issues).
It might impact your access to moderated groups, though (in addition to
getting less potential peers)

- "you need public IP address to peer" (not true; e.g. I started my peerings via
UUCP on private SLIP dial-up links back in the day. In fact, *lack* of always-on
internet connectivity and public IP addresses was main factor of interest in
setting up my own news server initially. It would still work today, provided you
find interested peers). There was also interesting discussion of privacy TCP/IP
layers lately in news.admin.peering, for example.

- "you must not be commercial news server" (not true either; although it would
*also reduce* the size of group of peers that are interested in your request - just
like the other *suggestions* mentioned above)

- "your peering request must have all of the following information (XXXXX, email, ...)
and be posted at news.admin.peering" (not true either, see above)

It would be less problematic if all of "need" / "must" / "should" etc. in such
potential FAQ were replaced with "In order to enhance your chances of finding
many peers, you might want to consider" (or similar) phrase.

> Perhaps some of these things don't need to be /in/ the INN FAQ itself.
> But I do think it would be good to have a pointer in the INN FAQ
> referencing some of these things. E.g.
>
> Q: What do I do now that my INN server is up and running?
> A: See the Peering FAQ document for more details.

I agree it should be _kept out_ of *INN* FAQ, and have said as much.

Note that "Peering FAQ" would likely only offer help on News peering; while the
OP seemed to me to have much wider range of general computing issues.
That's why I originally suggested:

>> There is absolutely no need to specify those in *INN* FAQ. Perhaps is some "computing 101" FAQ...

[1] e.g. only peering proposal I considered lately was quite esoteric (which is
why it piqued my interest, in fact).

[Matija Nalis]

On Fri, 15 Jul 2022 12:15:35 -0600, Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 7/15/22 12:02 AM, Miner wrote:
>> Before asking for a peering:
>> * administrator must register the domain and probably should pay for it
>

> I can't support "/must/ register a domain name". I see zero problems
> with using a hostname in an existing domain name.

... that could probably be simplified as "FQDN", but...

> I can support "/must/ use a /registered/ domain name".

... why do you see that as a "MUST", though?
Which part of news peering *cannot possibly work* without valid public domain name?

It looks to me to be (in RFC2119/BCP14 sense) *at most* "SHOULD" (or even "MAY", depending).

> I agree that using an email address in the same (parent) domain is
> preferred. But I don't think that preference even qualifies as a SHOULD.

Why even that? It would be at best "MAY", if it even warrants a mention (it
does not, IMHO - it is pointless policy minutia, like requiring that email
addresses must always be in form "firstname...@company.tld" or some such)

[Grant Taylor]

On 7/18/22 4:38 PM, Matija Nalis wrote:
> And it was okay back then, wasn't it? That is exactly my point.
> Things change, and things are often different in different regions.

I feel like we're talking past each other.

I'm saying that things should be documented, even if they are common
logic to some people. Because they may not be common logic to everyone
now or at some point in the future. Hence the value in documenting what
is common logic so that there is no doubt in another location or another
time.

Said another way, simply writing the common logic removes any ambiguity.

> For example, back when I was still admining public news servers, e-mail
> was not required for peering in this region. Instead, XMPP and IRC were
> often alternatives, even preferred by some. At least in my region...

That sounds like an example of a good thing to document that was
probably common knowledge to you / those around you but not as common to
others.

> Sure. It would probably need to cover lots of regions (and maybe
> historical information, as it would need to be kept up-to-date),
> or would need to be very open-ended/vague, lest it possibly lead
> to more confusion than it solves. It is likely much harder than it
> looks on first sight.

I think we're talking about two different things. I'm talking about a
subset of what I think you are talking about. I'm specifically talking
about people documenting what they are doing / using / expecting in a
way that will remove ambiguity for others later.

A corollary: I've heard / read multiple times / places that scientists
are unable to build rocket engines using plans from the '70s & '80s,
despite having full blue prints, /because/ the knowledge of /how/ to do
some things was lost because it wasn't documented. Either for
intellectual property / trade secret reasons or simply because everybody
thought it was common knowledge, thus they didn't need to document /how/
they did things. Instead they only documented /what/ they did.

I hope that makes sense.

> Even things like language and peering places used differ wildly.

Sure. Hence why providing some background on what some consider to be
common knowledge will help those that are translating the languages and
having to deduce things.

> (e.g. do high percentage of Chinese newsadmins choose to find peers by
> posting in English on news.admin.peering? Neither did most newsadmins
> in .hr; yet peer they did - and some still do. Not all news servers
> even carry Big8!)
>
> While some peering requests on news.admin.peering and elsewhere might
> gain much more traction because they have made popular choices, this
> does not _per se_ invalidate the other (more quirky) peering requests.

Agreed.

> It would be less problematic if all of "need" / "must" / "should"
> etc. in such potential FAQ were replaced with "In order to enhance
> your chances of finding many peers, you might want to consider"
> (or similar) phrase.

I believe that most of the SHOULD / MUST / et al. are borrowing from RFC
standards and meanings therein.

[Grant Taylor]

On 7/18/22 4:57 PM, Matija Nalis wrote:
> ... that could probably be simplified as "FQDN", but...

My concern is that FQDNs can be locally / privately resolved.

I take the spirit of the requirement to be that whatever name is used
can be externally / publicly resolved.

> ... why do you see that as a "MUST", though?

I was restating the previous comment with a modification. E.g.

s#register a#use a /registered/#

> Which part of news peering *cannot possibly work* without valid public
> domain name?

I agree that it can. I was seeking what I could support, even if I
thought it was a bit of an over reach.

> It looks to me to be (in RFC2119/BCP14 sense) *at most* "SHOULD"
> (or even "MAY", depending).

I like should better than must. But I was focusing on a different part
of the statement.

> Why even that? It would be at best "MAY", if it even warrants
> a mention

How about refactoring / simplifying from email to an out of band
communications method that both peers are happy with?

[Matija Nalis]

On Mon, 18 Jul 2022 17:38:15 -0600, Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 7/18/22 4:38 PM, Matija Nalis wrote:
>> Even things like language and peering places used differ wildly.
>
> Sure. Hence why providing some background on what some consider to be
> common knowledge will help those that are translating the languages and
> having to deduce things.

Yes, there I agree with you.
Documenting current (and even historical) state is good.

I was more targeting the angle that *extra care* should be taken so that
resulting document does not end up being US/EU-English-region-centric
(or even more narrow: Big8-centric).

Which it quite probable result, if it would be created by collecting input from
Big-8 news.* groups only; even if some of cross-polination of newsadmins from few
different countries who roam here (mostly western EU and US, it looks to me).
Thus my comment about it being hard to document things correctly without
introducing such strong bias.

>> It would be less problematic if all of "need" / "must" / "should"
>> etc. in such potential FAQ were replaced with "In order to enhance
>> your chances of finding many peers, you might want to consider"
>> (or similar) phrase.

Ah, I was refering to common conversentional style as was used my own bullet
points that were above that paragraph and might end up in such document
(e.g. "you need public IP address to peer" etc.)

> I believe that most of the SHOULD / MUST / et al. are borrowing from RFC
> standards and meanings therein.

Agreed - and they should be in all uppercase then. I doubt there would be much
(or any) of MUST, mostly "MAY" and perhaps a few "SHOULD". "NEED" is not BCP14,
but is AFAICT common in conversational style used in such documents.

>> Which part of news peering *cannot possibly work* without valid public
>> domain name?

>> It looks to me to be (in RFC2119/BCP14 sense) *at most* "SHOULD"
>> (or even "MAY", depending).
>
>I like should better than must. But I was focusing on a different part
>of the statement.

Oh, OK then. I was confused by MUST part, as it clearly is not a "MUST".

>> Why even that? It would be at best "MAY", if it even warrants a mention
>
>How about refactoring / simplifying from email to an out of band
>communications method that both peers are happy with?

Yes, that sound like a good idea to me. Few popular examples of such OOB might
then be mentioned in parenthesis ("e.g. email or various instant messaging") to
give some context to the user, without sounding too prescriptive.

[Miner]

Thomas Hochstein wrote:

> Miner schrieb:
>
> > Just the facts.
> > 1. News.admin.peering FAQ does not exist or hasn't been
> > posted in the past few months.
>
> No problem. Create it, post it.

rejecting[*] <defa.2022071...@scatha.ancalagon.de> 439 [*] Insolent or mentally retarded.
rejecting[*] <dcomm.202207...@scatha.ancalagon.de> 439 [*] Insolent or mentally retarded.
rejecting[*] <desd.2022071...@scatha.ancalagon.de> 439 [*] Insolent or mentally retarded.
rejecting[*] <dcomm.202207...@scatha.ancalagon.de> 439 [*] Insolent or mentally retarded.
rejecting[*] <dcomm.202207...@scatha.ancalagon.de> 439 [*] Insolent or mentally retarded.
--
Miner

[Newsmaster XeNET]

KI seems to made great progress the last years. :)

With best regards
MM


--
Diese E-Mail wurde von AVG auf Viren geprüft.
http://www.avg.com

[Thomas Hochstein]

Miner schrieb:

> Thomas Hochstein wrote:
>> Miner schrieb:

>>> 1. News.admin.peering FAQ does not exist or hasn't been
>>> posted in the past few months.
>>
>> No problem. Create it, post it.
>
> rejecting[*] <defa.2022071...@scatha.ancalagon.de> 439 [*] Insolent or mentally retarded.

Yep, that's what I thought.

[Russ Allbery]

Last-modified: 2022-07-17
Posted-by: postfaq 1.17 (Perl 5.28.1)
Archive-name: usenet/software/inn2-faq
URL: https://www.eyrie.org/~eagle/faqs/inn.html
Posting-frequency: monthly

This FAQ is intended to answer frequently asked questions concerning the
current versions of INN (INN 2.x and later) seen on news.software.nntp.
It should be referred to in preference to the old INN FAQ, which only
documents versions up to 1.7. It mostly covers INN 2.3 and later; earlier
versions of INN may behave differently or use different configuration
files.

If you're reading this on Usenet, this FAQ is formatted as a minimal
digest, so if your news or mail reader has digest handling capabilities
you can use them to navigate between sections. In rn variants, you can
use Ctrl-G to skip to the next section; in Gnus, press Ctrl-D to break
each section into a separate article.

Please send any comments, suggestions, or updates to <ea...@eyrie.org>.
Bear in mind when sending me e-mail that I receive upwards of 800 mail
messages a day and have unanswered personal e-mail dating back six months
or more, so please don't expect an immediate response. You may receive
quicker responses by posting to news.software.nntp (even, due to the
quirky way in which I read mail and news, from me).

This FAQ is posted monthly to news.software.nntp, and is available on the
web at <https://www.eyrie.org/~eagle/faqs/inn.html>.

------------------------------

Subject: Contents

1. General Questions
1.1. What is INN?
1.2. What is the current version?
1.3. Where can I get INN?
1.4. Where can I find documentation?
1.5. What newsgroups are there for INN?
1.6. What mailing lists are there for INN?
1.7. How can I support INN development?
1.8. How can I contribute to INN?

2. Terms
2.1. What is tradspool (traditional spool)?
2.2. What is CNFS?
2.3. What are timehash and timecaf?
2.4. What is overview?
2.5. What are deferrals (NNTP code 431)?

3. Specific Problems
3.1. INN won't start after a new installation
3.3. The news server isn't keeping up with incoming news
3.4. news.notice is empty and the nightly report is missing things
3.5. INN is running out of file descriptors
3.6. Can't get debugging information out of INN
3.7. Articles aren't being sent to remote peers
3.8. sendmail isn't installed

4. Error Messages
4.1. innd: SERVER cant store article
4.2. innd: SERVER internal no control and/or junk group
4.3. Modification of read-only value attempted (Cleanfeed)
4.4. tradspool: could not open ... File exists
4.5. Binary posting to non-binary group (Cleanfeed)

5. Problems on Specific Systems
5.1. INN won't compile on SCO OpenServer / UnixWare / OpenUNIX
5.2. Using raw devices on Solaris destroys the partition table
5.3. Will INN run on Windows?
5.4. Why aren't INN's files where the documentation says they are?
5.5. Running INN on macOS

6. How Do I...
6.1. Set up a server with no external feeds, just local groups
6.2. Process a single control message
6.4. Feed all articles on a server to another server
6.5. Rename a newsgroup
6.6. Change the domain used for message IDs
6.7. Use INN without a direct news feed
6.8. Generate MRTG graphs for INN
6.9. Hide the junk and control groups from users
6.10. Modify the body of posts made through my server
6.11. Hide the Injection-Info header field
6.12. Run innd and nnrpd on separate ports
6.13. Back up and restore an INN installation
6.14. Find external feeds and set up peering

(Note that some numbers have been skipped. When questions are removed,
the remaining questions are not renumbered to avoid breaking links in
Usenet and mailing list archives.)

------------------------------

Subject: 1. General Questions

Contained in this section are general questions about INN, where to find
it, and things of that sort. It is aimed at the person who is not yet
running INN, or who has general questions about how it works.

------------------------------

Subject: 1.1. What is INN?

The README that comes with INN has this to say (in part):

INN (InterNetNews), originally written by Rich Salz, is an extremely
flexible and configurable Usenet / Netnews news server. For a
complete description of the protocols behind Usenet and Netnews, see
RFC 3977 (NNTP), RFC 4642 updated by RFC 8143 (TLS/NNTP), RFC 4643 (NNTP
authentication), RFC 4644 (streaming NNTP feeds), RFC 5536 (USEFOR), RFC
5537 (USEPRO), RFC 6048 (NNTP LIST additions), RFC 8054 (NNTP compression)
and RFC 8315 (Cancel-Lock) or their replacements.

In brief, Netnews is a set of protocols for exchanging messages between
a decentralized network of news servers. News articles are organized
into newsgroups, which are themselves organized into hierarchies.
Each individual news server stores locally all articles it has received
for a given newsgroup, making access to stored articles extremely fast.
Netnews does not require any central server; instead, each news server
passes along articles it receives to all of the news servers it peers
with, those servers pass the articles along to their peers, and so on,
resulting in "flood fill" propagation of news articles.

INN is free software, supported by Internet Systems Consortium and
volunteers around the world.

For a more complete answer, see that file. A full description of what
Usenet and Netnews are is beyond the scope of this document; for a
beginner's introduction, see the news.newusers.questions home page at
<http://www.tokak.us/nnq/>.

------------------------------

Subject: 1.2. What is the current version?

The most recently released version of INN is 2.7.0.

INN development proceeds in two branches, as with many other free software
projects. The STABLE branch is maintenance of the most recently released
stable version, and only bug fixes are added to it. The CURRENT branch is
the development version of the next release of INN.

As mentioned in the next section, when installing a new INN server, you
may wish to download the latest snapshot of the STABLE branch rather than
the current full release.

Note that the previous STABLE series for INN 2.6 terminated in the release
of INN 2.6.5 and current STABLE snapshots are based on INN 2.7. You
should therefore read the upgrade instructions in NEWS when upgrading from
a STABLE snapshot before July 11th, 2022 to one dated after that.

------------------------------

Subject: 1.3. Where can I get INN?

The download site for INN is <https://ftp.isc.org/isc/inn/>. In that
directory are the various releases of INN, some additional documentation
(particularly of security holes), the original INN Usenix paper.

There is also a snapshots subdirectory, in which you will find two sets
of snapshots: ones at the top level, which are updated only when the
code changes, and ones in the daily subdirectory, which are generated
every day and retained for seven days. The daily snapshots with
STABLE in the name are the latest versions of the STABLE branch and
may have some additional bug fixes over the current released version.
The daily snapshots with CURRENT in the name are of the current
development version.

Please note: There is no guarantee that a snapshot will even compile, let
alone function well as a news server. In particular, the CURRENT branch
is under active development, and all sorts of things may be broken at any
given point in time. Use snapshots with caution, and don't use snapshots
from the CURRENT branch on any production system unless you're prepared to
debug the inevitable problems in code that's actively changing and not yet
thoroughly tested. (The STABLE snapshots should be fairly reliable,
however.)

------------------------------

Subject: 1.4. Where can I find documentation?

INN comes with extensive documentation. See the files INSTALL and README
at the top level of the source tree, for starters. In addition, nearly
every program and configuration file has its own Unix man page. The best
place to start is by reading the entire INSTALL file and then from there
discovering which configuration files and programs do what you want to do
and reading their individual man pages.

There are HTML conversions of the documentation that comes with recent
versions of INN available at:

<https://www.eyrie.org/~eagle/software/inn/>

For additional documentation beyond what is distributed with INN, follow
the links suggested in the above page.

The documentation that comes with INN is fairly technical in nature and
lacking in some more general details on configuring news servers. Some of
the links off of the INN home page have additional overview documentation
or documentation on how to set up servers for specific roles.

Another good resource is the newsgroup news.software.nntp (and the Google
archives thereof) and the archive of the inn-workers mailing list. A link
to the latter is off the INN page referenced above.

Finally, the following additional links may be useful:

<http://aplawrence.com/Unixart/newsserver.html>
A tutorial on setting up INN aimed at beginners using SCO Unix. While
it's mostly focused on SCO, it may be useful for any beginner to INN
and news servers.

<http://www.kozubik.com/published/inn_tutorial.txt>
A tutorial on setting up INN on FreeBSD. Contains a lot of
information focused on FreeBSD and its preferred file layout, so may
be easier to follow than the generic instructions on that platform.

------------------------------

Subject: 1.5. What newsgroups are there for INN?

news.software.nntp discusses all NNTP-based news servers, including INN.
It's the best newsgroup for technical questions and discussion. The
newsgroup news.software.b is also chartered for such discussion, but it's
essentially dead now. General news administration questions are also
on-topic in news.admin.technical (moderated) and news.admin.misc
(unmoderated).

news.admin.hierarchies covers questions of general hierarchy configuration
and is where announcements of new news hierarchies are generally posted.
news.admin.net-abuse.* covers the topic of network abuse and prevention
(including spam), but is not for the faint of heart; it is extremely noisy
to the point of being essentially unreadable without a lot of time and
patience (and a good killfile).

------------------------------

Subject: 1.6. What mailing lists are there for INN?

There are several INN-related mailing lists:

inn-an...@lists.isc.org
Where announcements about INN are sent (only maintainers may post).

inn-w...@lists.isc.org
Discussion of INN development. If you prefer not to use GitHub to
create an issue or a pull request, it is also where to send bug reports
and patches for consideration for inclusion into INN (postings by
members only). If you're an INN expert and have the time to help out
other users, we encourage you to join this mailing list to answer
questions. (You may also want to read the newsgroup
news.software.nntp, which gets a lot of INN-related questions.)

inn-com...@lists.isc.org
Git commit messages for INN are sent to this list (only the
automated messages are sent here, no regular posting).

inn-...@lists.isc.org
This list used to receive Trac issues for INN, before the migration to
GitHub (only the automated messages were sent here, no regular
posting). Bug reports should be sent to the inn-workers mailing list,
or a GitHub issue created.

To join these lists, send a subscription request to the `-request'
address. The addresses for the above lists are:

inn-announ...@lists.isc.org
inn-worke...@lists.isc.org
inn-committ...@lists.isc.org
inn-bugs...@lists.isc.org

You can alternatively join them from the subscription form in their public
web pages:

<https://lists.isc.org/mailman/listinfo/inn-announce>
<https://lists.isc.org/mailman/listinfo/inn-workers>
<https://lists.isc.org/mailman/listinfo/inn-committers>
<https://lists.isc.org/mailman/listinfo/inn-bugs>

inn-workers tends to be moderate volume (3-5 messages a day, but varying a
lot depending on what's being discussed). inn-committers is occasionally
higher volume but entirely automatically generated GitHub push
notifications. inn-announce is a low-volume moderated list containing
only major announcements. inn-bugs no longer has any activity.

------------------------------

Subject: 1.7. How can I support INN development?

There are four major ways. First, like with any other free software
project, a great way to support INN development is to join in yourself.
If you know how to program and have an interest in working on a widely
deployed and fairly intricate news server, we'd love to have your help.
See the next question for more details.

Second, even if you don't have the time or expertise to write much code,
any contributions of documentation are *greatly* appreciated. There's
always documentation work to be done, from maintenance of INN's technical
documentation to tutorials and overviews for the new user or the user who
wants to do something specific. Listen on news.software.nntp for what
people are looking for, or ask on inn-workers. Similarly, beta testers
are always welcome; if you have a test news server and some knowledge of
how to diagnose server problems and want to try out the current
development code and report any bugs you run into, that helps the
developers immensely.

Third, there are always more questions from new INN users to answer.
news.software.nntp gets a regular stream of them, and it's a great way to
help out intermittantly when you have a few moments to read news. If you
can identify general solutions to frequent problems and pass them along to
the INN maintainers in the form of documentation or suggestions, even
better.

Fourth, from the README:

Note that INN is supported by Internet Systems Consortium, and
although it is free for use and redistribution and incorporation into
vendor products and export and anything else you can think of, it
costs money to produce. That money comes from ISP's, hardware and
software vendors, companies who make extensive use of the software,
and generally kind hearted folk such as yourself.

Internet Systems Consortium has also commissioned a DHCP server
implementation and handles the official support/release of BIND. You
can learn more about the ISC's goals and accomplishments from the web
page at <https://www.isc.org/>.

The ISC provides ftp and web space and mailing lists and archives.
Donations to help support all of that are greatly appreciated.

------------------------------

Subject: 1.8. How can I contribute to INN?

First, join inn-workers, since that's where all the development discussion
takes place. The traffic isn't that high.

Next, download a snapshot of the INN CURRENT branch as described above so
that you have a relatively current source base to work from. You may want
to check out or clone the current source from GitHub; just point a Git
client at:

https://github.com/InterNetNews/inn.git

Read the HACKING file at the top of the INN source tree for some general
information and tips for working on INN.

Then pick something that looks interesting to you, mention what you're
doing on inn-workers if it's likely to affect other parts of the
development, and have at it! The GitHub bug tracker and the TODO file in
the CURRENT tree have a pretty comprehensive list of things that could be
done. Best to start with something small (getting INN to work correctly
on a platform where it doesn't currently and which you have available is
often a great start, or working on one of the supporting programs or
scripts that's a bit easier to wrap one's mind around than the core INN
daemons). Patches to INN should either be submitted as a pull request on
GitHub or sent to inn-w...@lists.isc.org, possibly put on an ftp or web
site somewhere and the URL sent to inn-workers if they're extremely large.

------------------------------

Subject: 2. Terms

Here are definitions of some commonly used terms related to INN. (More
definitions are welcome; this section is extremely incomplete at the
moment and the FAQ maintainer tends not to recognize terms that need a
definition for people unfamiliar with INN.)

------------------------------

Subject: 2.1. What is tradspool (traditional spool)?

Traditional spool is called that because it's the way that all news
servers used to store articles. A traditional news spool is a tree of
directories matching the hierarchical structure of newsgroups. For
example, the newsgroup news.software.nntp would be stored in a directory
news/software/nntp under the root of the news spool, and next to the
"nntp" directory in news/software would be a "readers" directory for the
group news.software.readers.

As of INN 2.3, traditional spool is completely integrated into the storage
API as the tradspool storage method and use the same overview mechanisms
as the rest of INN.

Storing articles in the traditional spool format is slow relative to other
storage mechanisms. It's probably nearly impossible to keep up with a
full Usenet feed using pure traditional spool. It is, however, the
recommended storage method for low-traffic local newsgroups and any
newsgroups that you want to back up.

For more details, see the INSTALL file.

------------------------------

Subject: 2.2. What is CNFS?

CNFS is the Cyclic News File System, written by Scott Fritchie. It is a
high-performance method of storing news articles, designed to avoid the
high overhead involved in interacting with the file system when storing
articles in individual files. CNFS stores articles sequentially in
pre-configured buffer files. When the end of the buffer is reached, new
articles are stored from the beginning of the buffer, overwriting older
articles.

It's the fastest article storage method in terms of write performance, and
is recommended for storing binaries.

For more details, see the INSTALL file.

------------------------------

Subject: 2.3. What are timehash and timecaf?

These are two less-used storage mechanisms available under the INN storage
API (similar in that respect to CNFS). Both can usefully be thought of as
compromises between the write speed of CNFS and the fine-grained control
over article expiration. INSTALL says for timehash:

Articles are stored as individual files as in tradspool, but are
divided into directories based on the arrival time to ensure that no
single directory contains so many files as to cause a bottleneck.

and for timecaf:

Similar to timehash, articles are stored by arrival time, but instead
of writing a separate file for each article, multiple articles are put
in the same file.

timecaf was new in INN 2.3.

------------------------------

Subject: 2.4. What is overview?

Overview is summary information about articles in a newsgroup that is
returned to news reading clients as a response to the OVER command. It's
a very common extension to the NNTP protocol that allows readers to review
summary information about articles before taking the time (and bandwidth)
to download the entire article.

The canonical items of information included in an overview record are the
Subject, From, Date, References, and Message-ID header field bodies of the
article, the byte count of the article, and the line count of the article.
Nearly every server now also returns the Xref header field (a list of the
newsgroups carried by the server to which the article was posted and the
article number in each of those newsgroups) as an additional field.

Note that with the References and Message-ID header fields, the overview
record contains enough information to do article threading. It also
contains all of the fields normally keyed on for client-side filtering
(killfiles and the like).

Generating overview information for a newsgroup on the fly would be
prohibitively expensive, particularly for large groups, since the server
daemon would have to find all of those articles and scan them to build the
information. It would also be inefficient, since the overview information
for a particular group will generally be requested many times by different
clients.

Any INN server that supports readers must therefore have an overview
method configured. There are four different methods to choose from:

- buffindexed, which stores overview data and index information
into preconfigured large files like CNFS. Fast at writing, the
buffindexed overview storage method can keep up with a large feed
more easily and never consumes additional disk space beyond that
allocated to these buffers. The downside is that these buffers
are hard to recover in case of corruption and somewhat slower for
readers and the expiry process;

- ovdb, which stores overview information into a Berkeley DB database,
whose development pace has stalled these last years. This method
is fast and very robust, but may require more disk space, unless
compression is enabled. Overview data is fetched one article at a
time, which makes this method a bit slower than ovsqlite for readers;

- ovsqlite, which stores overview information into an SQLite database,
known for its long-term stability and compatibility. Robust and
faster than ovdb at reading ranges of overview data (since overview
data is transferred in 128-kilobyte chunks between ovsqlite-server
and nnrpd) but somewhat slower at writing, this method may require
more disk space, unless compression is enabled;

- tradindexed, which uses two files per newsgroup, one containing
the overview data and one containing the index. Fast for readers,
but slow to write to because it has to update two files for each
incoming article. Its main advantage is to be the best tested,
the most reliable and the method with the best recovery tools.

Here are a few elements that can be helpful in choosing the right
overview method for your needs and estimating the associated storage
size. In 2020, the volume for a full-text Usenet feed is about 18,000
articles per day, with peaks to 1,200 articles per hour. Article storage
size is about 65 MB per day.

As for overview storage size, if you have 5 million articles, you'll
need at least 3.25 GB of disk space for buffindexed, 5.5 GB for ovdb
(4.5 GB if compressed), 4.65 GB for ovsqlite (2 GB if compressed),
and 3.10 GB for tradindexed.

If you store more header fields in overview data than the standard
ones, the space needed to store overview data will be superior than
these estimates. (This is configured in the extraoverviewadvertised
and extraoverviewhidden inn.conf parameters.)

------------------------------

Subject: 2.5. What are deferrals (NNTP code 431)?

Consider the following situation. You have two incoming peers, both of
which are getting ready to offer you an article in streaming mode. The
first sends you a CHECK <message-id> message, to which you respond
affirmatively (i.e., you don't already have the article). Then, before
that peer sends you the article with TAKETHIS, you receive a CHECK
<message-id> from the second peer for the same message. What response
does INN send to the second peer?

If deferrals are enabled (resendid == true in incoming.conf for that peer,
the default), INN will send a 431 deferral telling that peer that you may
or may not want the article; try again later. Chances are that when it
retries, you will have received the article from the first peer and you'll
just refuse it. But if the first peer dies before it ever sends you the
article, this way you can still get it from the second peer.

If deferrals are disabled, INN will refuse the article from the second
peer, which means there's a possibility you'll lose news if the first peer
dies before sending you the article.

As a side note, some older versions of Diablo, upon receiving a deferral,
turn around and immediately send the article via TAKETHIS, which is
basically exactly what you don't want. (Chances are extremely high in
practice that the first peer will come through with the article.)

------------------------------

Subject: 3. Specific Problems

This section contains specific problems that are frequently reported when
using INN, and includes fixes or suggestions for fixes. Candidates for
inclusion in this section are any problems reported frequently on
news.software.nntp or inn-w...@lists.isc.org. Contributions, including
fixes, are very welcome.

------------------------------

Subject: 3.1. INN won't start after a new installation

The most common cause of this problem is that inndstart isn't setuid root
(please note that it only affects versions prior to INN 2.5.0 because
inndstart was removed in INN 2.5.0). inndstart must be installed owned
by root and group news, mode 4550. The ls -l output for inndstart should
look something like:

-r-sr-x--- 1 root news 53768 Jan 8 00:47 inndstart*

inndstart will automatically be installed with the right permissions if
you run make install as root. If inndstart isn't setuid root, it will log
errors to syslog when it tries to start and cannot. If you aren't seeing
those error messages in syslog either, you probably haven't set up syslog
properly (see 3.4).

The other most frequent cause of this problem is not correctly following
the instructions in INSTALL on how to set up the initial history database.
If you run makedbz without the -o flag, the initial history database files
will have names starting with history.n. These files must be renamed to
remove the ".n" before innd will start.

------------------------------

Subject: 3.3. The news server isn't keeping up with incoming news

Start by looking for the profile information in your nightly report. That
will tell you where the news server is spending most of its time and may
identify the exact nature of the problem.

This problem is quite frequently due to using the traditional spool
storage format for news articles. This storage method is now too slow to
be able to handle a full Usenet news feed (although with a more limited
selection of groups it can still do just fine). If your server is
spending a lot of time writing articles and you're using traditional
spool, this is probably the problem.

One possible solution would be to switch to CNFS as a storage mechanism.
You can do this simply by configuring CNFS (see INSTALL for details),
changing storage.conf to direct some or all of the incoming traffic to
CNFS buffers, and then restarting INN. Older articles will continue to be
stored in tradspool until they expire, but new articles will go into CNFS.

------------------------------

Subject: 3.4. news.notice is empty and the nightly report is missing things

You have syslog set up incorrectly.

INN logs nearly everything except article trace information via syslog.
It expects syslog to write its log messages into particular files under
~news/log, unless you gave it a different path at configure time (see
the pathlog parameter in inn.conf). You'll need to set up logging of
INN-related log messages in your system /etc/syslog.conf. See the
"Configuring syslog" section in INSTALL.

Note that you don't have to worry about rotating these log files;
news.daily (which should be run nightly from cron) will take care of that
and innreport generates a daily summary report from them.

------------------------------

Subject: 3.5. INN is running out of file descriptors

You may need to increase your system file descriptor limits. See the
"File Descriptor Limits" section of INSTALL for more details. This is
particularly a concern on Solaris systems, since Solaris by default has an
exceptionally low file descriptor limit.

------------------------------

Subject: 3.6. Can't get debugging information out of INN

The INN startup process is quite complicated, involving the rc.news shell
script (and the setuid inndstart wrapper for versions of INN prior to 2.5.0).
This can make it rather difficult to get enough debugging information out
of it to determine what's going wrong if it's crashing immediately after
startup or otherwise having serious difficulties.

One approach is to run innd by hand directly, giving it the -d option.
This requires setting up a configuration where innd doesn't need to bind
to privileged ports, however.

Another, sometimes better option, is move the innd binary to another name,
like innd-real, and put a shell script in its place. Here's an example,
from Kai Henningsen:

#! /bin/bash
# allow core dumps
ulimit -c unlimited
# save any output
exec > /tmp/innd.log 2>&1
# who are we running as, anyway?
id
# show exported environment
export
# start innd (don't forget the arguments, or it will complain)
exec strace -o /tmp/innd.strace -f -F /path/to/innd-orig "$@"

This starts innd under strace, suitable for debugging startup core dumps
and the like. You can use this as a general model for a variety of
debugging; for example, you could replace the strace invocation with an
invocation of gdb and then start innd from inside gdb with the -d option.

------------------------------

Subject: 3.7. Articles aren't being sent to remote peers

(This entry is based on a post by Jeffrey M. Vinocur.)

Here's how to trace through INN's logs to figure out what's happened to a
particular article. This should help you discover where the process of
feeding an article to a peer broke down.

1. First you look in the $pathlog/news file. There should be one line per
article (search for the Message-ID, or they're in order by time of
arrival if you know that).

If you don't see a line for the article in question, your innd has
never seen it. For articles being fed remotely, this means your peer
didn't feed it to you. For articles being posted to your server, this
generally indicates some sort of problem in nnrpd.

(The only other time you wouldn't see a line for the article in
question is if your innd has seen it in the past, and is considering
this attempt a "duplicate".)

2. Next, look at the second field of the line you've found in
$pathlog/news.

If it's "-", then your innd rejected the article. The reason should be
at the end of that line.

3. At this point, you should be looking at a line with "+" in the second
field. The article should be on your server at this point.

If it's not, either it's been cancelled, or has already expired.

4. You're now interested in whether the article was sent to your peers.
At the end of the same line in $pathlog/news, innd puts all of the
peers it thinks should receive this article.

If you don't see a peer you expect there, it indicates that
$pathetc/newsfeeds is not configured in the way you think it is.

5. If a peer is listed at the end of the line, the article should have
been fed to that peer.

If a peer doesn't have that article, it's possible that the article is
spooled on your system somewhere. Check $pathoutgoing, or the
innfeed spool if the peer is configured to use innfeed. (It's probably
easier to look for error messages in $pathlog/news.notice than to
actually wade around in $pathspool/innfeed.)

6. If you're sure the article isn't spooled, and it doesn't show up on the
peer, you have to consider the possibility that the peer has rejected
the article. Alternatively, it's possible that the peer has some
misconfiguration like the ones described above.

In either case, if you're sure that the article was offered to the peer
and not spooled, you will need the assistance of the peer's admin to
investigate further. INN does not generally log enough information
about outgoing articles to be able to tell more from your server alone.

It may be possible to get a slight bit of information from the remote
server by connecting with telnet (usually to port 119) and issuing
"IHAVE <message-id>". The peer may respond with something like "435
Duplicate" which means that the problem is not likely to be with your
server (it may be still a problem with the article itself). If the
peer responds with something like "335", your server probably did not
offer the article after all.

If you really are at a dead end and need to get more information about
what's going on with an outgoing feed, you can switch it from innfeed
to nntpsend (see INSTALL for instructions). You can then run it
manually with innxmit -dv, which will show the full conversation with
your remote peer.

------------------------------

Subject: 3.8. sendmail isn't installed

Yes, INN really does require sendmail. It uses sendmail to send out the
daily reports and to mail messages to moderators, and it assumes that you
have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
it can use to do this. It does not speak SMTP, nor is it likely to ever
speak SMTP; it's hard enough maintaining a package to speak NNTP.

If you need a very simple local sendmail implementation that just sends
mail to a smarthost, there are several available (nullmailer, for
example).

------------------------------

Subject: 4. Error Messages

Explanations of specific error messages, including solutions where
applicable.

INN logs nearly all messages to syslog, so in general these error messages
will be found in syslog. If you aren't seeing anything from INN in syslog
at all, make sure that you have it set up correctly (see 3.3).

------------------------------

Subject: 4.1. innd: SERVER cant store article

You probably have a misconfigured storage.conf. In current versions of
INN, "no matching entry in storage.conf" is added to the end of this
message unless it really is a disk I/O problem, making the cause
considerably clearer.

storage.conf(5) has this to say:

If an article doesn't match any entry, either by being posted to a
newsgroup that doesn't match any of the <wildmat> patterns or by being
outside the size and expires ranges of all entries whose newsgroups
pattern it does match, the article is not stored and is rejected by
innd(8). When this happens, the error message

cant store article: no matching entry in storage.conf

is logged to syslog. If you want to silently drop articles matching
certain newsgroup patterns or size or expires ranges, assign them to the
"trash" storage method rather than having them not match any storage
method entry.

One of the more frequent causes of this problem is misuse of the expires
key in storage.conf entries. Read the man page for storage.conf very
carefully if you're using the expires key, since it may not do what you
think it does. In particular, if you have a storage class that specifies
expires with a min-time greater than 0, it won't match any article without
an Expires header field (the vast majority of Usenet articles).

------------------------------

Subject: 4.2. innd: SERVER internal no control and/or junk group

Your active file isn't complete. Either it's been mangled by something or
it's missing some required entries. Even if you're running a small
stand-alone server for internal use that only carries a handful of groups,
there are some pseudogroups used internally by INN that you have to have.

Since INN isn't running (it won't start when this error occurs), you can
edit the active file by hand without worrying about stepping on INN's
toes. Make sure the following lines are present in the active file (if
the numbers are different, that's fine):

control 0000000000 0000000000 n
control.cancel 0000000000 0000000000 n
control.checkgroups 0000000000 0000000000 n
control.newgroup 0000000000 0000000000 n
control.rmgroup 0000000000 0000000000 n
junk 0000000000 0000000000 n

and then start INN again. The control* groups are for control messages
(messages with a named group will be filed into it, and all other control
messages will go into the top-level catch-all group). The n flag is so
that users won't post messages directly to the control* groups; control
messages should be posted to the groups that they affect instead and INN
will refile them automatically based on the Control header field.

If you have mergetogroups: set in inn.conf, you will also need to create
a newsgroup named "to". Otherwise, you will get the following error:

innd: SERVER internal no to group

------------------------------

Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

INN 2.3 and later have an internal optimization to the interface to
embedded filters that makes filtering about 15-20% faster, but which
disallows a trick that many versions of Cleanfeed use to count the number
of lines in the article. (This problem is fixed in current versions of
Cleanfeed.)

To correct this problem, find the line in Cleanfeed that looks like:

$lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

and change it to:

$lines = $hdr{'__LINES__'};

The __LINES__ hash value is set internally by all recent versions of INN
and is guaranteed to be correct.

------------------------------

Subject: 4.4. tradspool: could not open ... File exists

This error generally happens after a crash or unclean shutdown of innd
using the tradspool storage method, and is caused by overview information
being out of sync with what articles are in the spool. When innd was
restarted, it renumbered its active file (which determines the range of
existing articles in each group and therefore what article number is
assigned to new articles) based on the overview information. If there are
newer articles already on disk that aren't mentioned in the overview
(because the overview information for those articles hasn't been flushed
to disk yet), new incoming articles will get assigned the same number as
the existing article and then innd will fail to store the article and
throttle with this error.

In INN 2.4 and later when using the tradindexed overview method, you can
solve this problem by rebuilding the overview for any affected group.
Throttle the server (if it isn't already) and then run:

tdx-util -R <path-to-articles> -n <newsgroup>

where <newsgroup> is the newsgroup that INN is complaining about and
<path-to-particles> is the full path to the directory where the articles
for that group are stored (it's generally in the error message).
Immediately afterwards, run ctlinnd renumber for that newsgroup, and then
unthrottle the server.

The general solution to this problem, which works with any version of INN
and any overview method, is to shut down the server, delete all of your
overview database, and then rebuild it from your news spool with:

makehistory -O -x -F

This takes a long time and is to some degree overkill. For versions of
INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately
after restarting INN.

A third and better solution in some cases is to just remove all articles
in the spool that have higher numbers than the numbers in the active file.
Here's a Perl script that will do that. Just save this to a file, make it
executable, and run it, giving it the path to the active file as the first
argument and the path to the top of your tradspool news spool as the
second argument:

#!/usr/bin/perl
die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
while (<ACTIVE>) {
my ($group, $hi, $lo, $flag) = split;
my $directory = $group;
next if ($hi == 0 and $lo <= 1);
$directory =~ tr%.%/%;
$directory = $ARGV[1] . '/' . $directory;
if (-d $directory) {
opendir (DIR, $directory) or die "Can't open $directory: $!\n";
while (defined ($_ = readdir DIR)) {
unlink "$directory/$_" if ($_ > $hi);
}
closedir DIR;
}
}

If you're not already running INN 2.4, upgrade if you can. Not only can
you recover directly from this problem if you're using tradindexed
overview, but INN 2.4 does a better job of flushing data to disk and is
less likely to have this problem in the first place.

------------------------------

Subject: 4.5. Binary posting to non-binary group

This message does not actually come from INN. It's generated by
Cleanfeed, and if you're seeing it, that means that you have Cleanfeed
installed. At least at one point, the default Red Hat installation of INN
included Cleanfeed without documenting this particularly well.

In order to allow binaries in your local hierarchies, you should modify
the Cleanfeed configuration file to set bin_allowed to a regular
expression matching the groups that should allow binaries. Please don't
allow binary postings to regular Usenet newsgroups that you don't know
should have binaries, as they consume large amounts of bandwidth and
possibly disk space for other sites.

For more information on Cleanfeed configuration options, see the Cleanfeed
documentation and the comments in the default configuration file.

------------------------------

Subject: 5. Problems on Specific Systems

Problems specific to particular operating systems or platforms. Look here
if INN doens't behave as expected on your particular system, or if you're
having trouble compiling INN in the first place.

------------------------------

Subject: 5.1. INN won't compile on SCO OpenServer

On SCO OpenServer, it's worth noting that with a shared Perl library,
Perl on this platform doesn't apparently generate the right link magic
to include the path to the dynamic Perl libraries. You need to either
set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
binaries so that they can find the Perl libraries. (The former is
preferred, since then the path is encoded into the binaries and you
don't have to remember to set LD_LIBRARY_PATH later.)

------------------------------

Subject: 5.2. Using raw devices on Solaris destroys the partition table

If you use slice 2, or some other disk slice that includes the entire
disk, under Solaris as a raw partition for CNFS, you may run into this
problem. The symptoms are that INN manages to initialize the cycbuffs
just fine, but then gets invalid device errors when it tries to open them
again, and the disks show up in format as needing to be repartitioned.

The solution is to not use raw devices that include the first cylinder of
the disk. Solaris doesn't protect the superblock from being overwritten
by an application writing to raw devices and includes it in the first
cylinder of the disk, so unless you use a slice that starts with cylinder
1 instead of 0, INN will invalidate the partition table when it tries to
initialize the cycbuff and all further accesses will fail until you
repartition.

Generally all that has to be done is to repartition the disk with slice 0
starting from cylinder 1 and extending to the end of the disk and then
point INN at slice 0 instead of slice 2. You lose some small amount of
space, but generally not enough to care about.

------------------------------

Subject: 5.3. Will INN work on Windows?

It won't out of the box. The standard INN distribution doesn't build on
Windows. It has, however, been built for Cygwin (a Unix-like environment
for Windows) in the past and some of the necessary patches (although
perhaps not all of them) have been incorporated into current INN releases.

Search for http://homepage.mac.com/imeowbot/inn/ at
<http://web.archive.org/> for the previous work. Don't forget to peruse
INSTALL if you download and want to try this.

------------------------------

Subject: 5.4. Why aren't INN's files where the documentation says they are?

INN's default installation locations are intended to be convenient for
sysadmins adding INN to their system without disturbing other software.
They don't match any of the standards used by various Linux distributions
or other Unix packaging systems. Because of that, distributors who supply
INN packages often rearrange the files and directories.

Unfortunately, this is very confusing for system administrators, because
the documentation is not updated to reflect the modified locations of
files.

You can always get the details of how your system is configured by looking
in inn.conf at "pathnews" and similar parameters. But for convenience,
here are comparisons of INN's default locations with some of the most
common packages.

(Data courtesy of John F. Morse.)

DEFAULT DEBIAN
pathnews: /usr/local/news /usr/lib/news
pathbin: /usr/local/news/bin /usr/lib/news/bin
pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control
pathdb: /usr/local/news/db /var/lib/news
pathetc: /usr/local/news/etc /etc/news
pathfilter: /usr/local/news/bin/filter /etc/news/filter
pathhttp: /usr/local/news/http /var/www/inn
pathlog: /usr/local/news/log /var/log/news
pathrun: /usr/local/news/run /run/news
pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp
pathspool: /usr/local/news/spool /var/spool/news
patharchive: /usr/local/news/spool/archive /var/spool/news/archive
patharticles: /usr/local/news/spool/articles /var/spool/news/articles
pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming
pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing
pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

DEFAULT FEDORA
pathnews: /usr/local/news /usr/libexec/news
pathbin: /usr/local/news/bin /usr/libexec/news
pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control
pathdb: /usr/local/news/db /var/lib/news
pathetc: /usr/local/news/etc /etc/news
pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter
pathhttp: /usr/local/news/http /var/lib/news/http
pathlog: /usr/local/news/log /var/log/news
pathrun: /usr/local/news/run /run/news
pathtmp: /usr/local/news/tmp /var/lib/news/tmp
pathspool: /usr/local/news/spool /var/spool/news
patharchive: /usr/local/news/spool/archive /var/spool/news/archive
patharticles: /usr/local/news/spool/articles /var/spool/news/articles
pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming
pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing
pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

In addition, the FreeBSD port uses the standard INN paths except that it
puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

Most packages install INN's man pages into a system man directory
(/usr/share/man or /usr/local/man) rather than into a separate man
directory under news's home directory.

------------------------------

Subject: 5.5. Running INN on macOS

Richard Tobin provided the following advice in news.software.nntp on
2013-06-29 based on experience with running INN on Snow Leopard:

Mac OS X, at least through the GUI, won't let you create a group with
the same name as a user. So you can't use "news" for both.

The Perl module GD isn't installed by default. GPG is not installed
by default.

You probably want to turn off Spotlight for the news spool directory.

Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
"-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
files for the other architectures don't seem to be installed. I
edited Makefile.global by hand to remove them.

I needed to tell the application firewall to allow innd to accept
incoming connections. (A window pops up to ask you, but this doesn't
help when you're connected by ssh!)

When I ran rc.news form a terminal window, it stopped working when I
logged out. This is because of MacOS's convoluted and undocumented
way of doing DNS lookups. Using "nohup" fixed it -- not because of
anything to do with SIGHUP, but because nohup calls an undocumented
function related to "vprocmgr". Running from launchd shouldn't have
this problem, and it appears to be fixed in Mountain Lion.

The Perl flags come from the Perl configuration; this problem is fixed
with current builds of macOS.

------------------------------

Subject: 6. How Do I...

This section documents various common or uncommon tasks or configurations
that people want to do with INN. It is mostly taken from frequently asked
questions in news.software.nntp.

------------------------------

Subject: 6.1. Set up a server with no external feeds, just local groups

The basic steps are to set up a newsfeeds file empty except for internal
feeds like controlchan or overchan (if you're using either), have only
localhost in incoming.conf, and start INN with the default minimal active
file. Then, create the groups you want to carry with ctlinnd newgroup.
Set up reading permissions using readers.conf as appropriate for your
organization.

In other words, it's very much like setting up any other instance of INN,
but you don't bother with innfeed, nntpsend, or any of their configuration
files. INN may also complain that you have no feeds in newsfeeds; this is
harmless and can be ignored.

------------------------------

Subject: 6.2. Process a single control message

To process a single control message, you can use controlchan from the
command line. Just type either:

echo /path/to/article-file | controlchan

or:

echo @token@ | controlchan

if you have the storage API token of the article. (This assumes
controlchan is in a directory in your path.) This is useful mostly for
testing; if you just want to create, remove, or change a group, it's
easier to use ctlinnd (newgroup, rmgroup, or changegroup).

------------------------------

Subject: 6.4. Feed all articles on a server to another server

To feed all articles on an existing server to another one, regardless of
how they're stored on the server, first tell the new server to accept
articles regardless of how old they are (otherwise, INN will reject
articles older than artcutoff in inn.conf) and disable your filtering:

ctlinnd param c 0
ctlinnd perl n
ctlinnd python n

Note that rejected articles are remembered during the number of days
specified by the /remember/ line in expire.ctl; so, in case you forgot
to change the above parameters, you'll have to wait that number of
days before being able to inject them again. Another possibility is to
set /remember/ to 0, run the expire process (for instance via news.daily
called with the same parameters as in crontab, plus "notdaily"),
undo the change in expire.ctl and then start the feed again.

You may also want to set xrefslave to true in inn.conf and then restart
INN on the new server if you want to keep the same article numbers as you
had on the old server. (It is notably helpful for news clients because
they otherwise get confused by an article renumbering in newsgroups they
are subscribed to.)

Next, make sure that the old server is listed in incoming.conf of the new
server, and reload incoming.conf with ctlinnd to pick up that change.
Also make sure that the new server carries exactly the same set of
newsgroups as the old server.

You may also want the new server not to propagate the articles it will
receive during this feeding operation, by checking that the newsfeeds
file of the new server is not configured to propagate articles to other
peers or controlchan (otherwise old control articles may be reprocessed).

Then try these commands (a variation on commands posted by Katsuhiro
Kondou to inn-workers) on the old server:

cd <pathdb in inn.conf>
perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
print "$_\n" if $_' history \
| tr . / > <pathoutgoing in inn.conf>/list
innxmit server list

where <pathdb> is the path to the directory containing the history file
(usually ~news/db), <pathoutgoing> is the path to the outgoing spool
directory (usually ~news/spool/outgoing), and server is the name of the
new news server to which you're feeding the articles.

In case you wish to only feed articles arrived on the old server
between two dates, you can adapt the previous commands. For instance,
the following commands will feed articles arrived between two given
timestamps (that can be computed with the convdate utility shipped
with INN).

convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

returns the two corresponding timestamps 1397586540 and 1397644620 that
can then be used to retrieve a subset of articles to feed:

cd <pathdb in inn.conf>
perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
my ($arrived, $expires, $posted) = split("~", $timestamps); \
print "$_\n" if $_ and $arrived >= 1397586540 \
and $arrived <= 1397644620' history \
| tr . / > <pathoutgoing in inn.conf>/list
innxmit server list

If innxmit stops transferring articles (with for instance an error like
"rewriting batch file and exiting"), just re-execute it.

When done, set xrefslave to false in inn.conf again if you changed it and
then either restart INN on the new server (necessary if you changed
xrefslave) or use another ctlinnd param command to set the cutoff value
back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
python to reactivate your filters.

Please note that when using xrefslave, this method requires that all of
the articles in your spool have Xref header fields. Current versions of INN
will always add an Xref header field, but very old versions (earlier 1.x
versions) will only add an Xref header field to crossposted articles. If
you're trying to import such a spool, you'll need to modify all of those
articles to add an Xref header field.

------------------------------

Subject: 6.5. Rename a newsgroup

INN has no native support for renaming a newsgroup, and doing so is
difficult, so the best advice is to not do this. If there's a way that
you can just create the new newsgroup, encourage people to start using it,
and then remove the old newsgroup, I recommend that. It's much easier.

Although it is not a renaming, it is also possible to create an alias.
Articles cannot be posted to that newsgroup, but they can be received
from other sites and treated as if they were actually posted to the
group named after the equal sign. However, their Newsgroups header field
body is not modified.

ctlinnd newgroup group.to.file.under y
ctlinnd changegroup old.group =group.to.file.under

Creating an alias newsgroup is useful in case you want residual articles
received under the old newsgroup name to be filed into the new group.

As for a renaming, if it really must be done, it's best if you're using
the tradspool storage method. The newsgroup of an article is stored in
the Newsgroups header field and in the Xref header field of the article
as stored on disk (and possibly in Followup-To), as well as determining
where the overview information is stored, and in the case of tradspool
is also encoded in the article's storage token. To rename a newsgroup
in tradspool, stop the server, move the directory containing all of
the articles to its appropriate new location in the news spool, edit
every article to change the old name to the new name in Newsgroups,
Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
and then rebuild history and overview with makehistory.

The following bit of Perl may help with the renaming (from Jeffrey
Vinocur):

#!/usr/bin/perl -wi
my ($src, $dst) = (shift, shift);
die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
unless(defined $dst);
while(<>) {
s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
print;
} continue {
close ARGV if eof;
}

Note that this may cause some problems if the newsgroup you're renaming is
contained in the name of another newsgroup to which messages in that group
are crossposted. If that's a problem, you may have to use a more
sophisticated script.

If any articles were crossposted to other newsgroups, you'll also have to
find and recreate the links in those newsgroups to the new location of the
articles (if the links were hard links and the process of changing the
Xref, Followup-To, Newsgroups header fields didn't break those links, you
may be lucky and be able to skip this).

If you're using another storage method, this is harder, although with
timehash you may be able to just change the Newsgroups, Xref, Followup-To
header fields of the articles in that newsgroup and then rebuild history
and overview as above.

One other approach that can be used regardless of storage method is to
refeed the articles to the server into a new newsgroup. This approach
works best if you're also changing news servers at the same time;
otherwise, the message IDs of the articles will already be in history, and
you'll have to change the message IDs of all of the messages or remove
them from the history database (such as by moving the articles away,
changing /remember/ to 0 so that old history entries won't be retained,
and then running expire to purge them out of history). To do this, get
all of the messages into a directory (by pulling them down via NNTP or
some other method), change the Newsgroups, Xref, and Followup-To header
fields to rename the newsgroup, and then create a file containing paths to
all of the articles, one per line. You can then use that file as input to
innxmit, pointing it at the server to which to feed the articles, and if
the articles aren't listed in history on that server and it carries the
new group, they will be accepted into the new newsgroup.

Note that if you use this method and something goes wrong the first time,
the message IDs will probably have all been added to history on the new
server and the articles now will never be accepted until those entries are
removed from history again (or all the message IDs changed).

------------------------------

Subject: 6.6. Change the domain used for message IDs

By default, any message IDs generated by INN will use the domain of the
local system for the right-hand-side of the message ID. In some cases,
this isn't desirable for various reasons (the server may have an internal
name that doesn't make sense on Usenet at large, or one may not want to
expose the name of the server).

In INN 2.3.3 and later, you can set virtualhost: to true in an access
stanza of readers.conf and then set domain: in the same stanza, and all
posts coming from connections to which that access stanza applies will use
that domain to generate message IDs. So if you need to change the domain
used to generate message IDs for every local post from your server, just
add virtualhost: and domain: keys to every access stanza in readers.conf.

This is really overkill for this option, and eventually the domain:
parameter in inn.conf will probably be changed to allow this to be
modified for the whole server. (Right now, domain: in inn.conf means
something completely different.)

------------------------------

Subject: 6.7. Use INN without a direct news feed

INN is designed to be used as a regular news server, receiving direct news
feeds from other news servers and sending news directly to other news
servers using the peer-to-peer portions of the NNTP protocol. However,
with some additional software, it is also possible to use INN as, in
essence, a local cache for a news server that you can use to read and post
but which doesn't treat your server like a peer.

This configuration is generally called a "suck" feed, because rather than
having news fed directly to your server, you pull it down or "suck" it
from another news server, and because possibly the first and one of the
most widely used packages for doing this is named suck.

The software to pull down articles from another server and to feed
articles to another server using post rather than peer-to-peer commands
does not come with INN (INN has a few utilities to do this on a small
scale, but not really anything designed to handle a lot of groups or a lot
of articles). You will need an external package to do this. The two most
popular are suck and newsx; however, both sites appear to be unavailable
as of thos writing. You may be able to find a package in your local
distribution or package repository.

Note that current versions of INN refer to articles internally using a
storage API token, not a path name, which is not always what suck or newsx
expects. Read the documentation carefully; you'll need to use a script or
configuration that retrieves articles using the sm program that comes with
INN rather than trying to open files directly.

It's also worth noting that INN is a fairly complex package, and while
many people are running it successfully using this sort of configuration
and like having a full-fledged news server available to them, other people
have found INN rather complicated and difficult to configure for a small,
simple personal news cache. If your needs and goals are simple and the
number of groups you're interested in is small, you may be better off with
a smaller, lighter package such as LeafNode or NNTPcache.

------------------------------

Subject: 6.8. Generate MRTG graphs for INN

INN's CNFS storage system has direct support for producing information
suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running
cnfsstat -p will generate sample MRTG configuration fragments for each
cycbuff.

To generate MRTG graphs of the usage of the buffindexed overview system,
try the following configuration fragment:

Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
MaxBytes[overview-BUFF]: 100
Title[overview-BUFF]: BUFF1 Usage
Options[overview-BUFF]: growright gauge
YLegend[overview-BUFF]: Overview Buffers
ShortLegend[overview-BUFF]: %
PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
<BR><TT>overview</TT>

where the overview.sh script is:

#!/bin/sh
echo "100"
<pathbin in inn.conf>/inndf -o | awk '{print $1}'
echo "0"
echo "overview"

This sample configuration is from Basil Kruglov. Note that you can
instead use -n (for total count of articles); in that case, you'll want to
remove the MaxBytes setting above or change it to be some sensible limit
on the total number of articles you receive. You'll also want to change a
few of the other labels in the MRTG configuration.

I'm not aware of any packaged solutions for generating MRTG data from
other things, such as incoming or outcoming news flows. If anyone has any
pointers, let me know.

------------------------------

Subject: 6.9. Hide the junk and control groups from users

The junk, control, and control.cancel groups must exist in the active file
for the proper operation of INN, so you can't remove the groups entirely.
You can, however, hide them completely from users.

To do this, edit readers.conf, and for each user access group where you
want to hide the junk and control groups, add "!junk,!control,!control.*"
to the newsgroups pattern. In other words, if you have a line like:

newsgroups: *

just change that to:

newsgroups: *,!junk,!control,!control.*

If you use read and post patterns instead, do the same for each of them
individually. The groups will then no longer show up on the server for
users to which that access group applies; it will be as if they do not
exist.

------------------------------

Subject: 6.10. Modify the body of posts made through my server

You can't without either making code changes to INN or putting your own
software in the path of incoming posts. This is intentional.

Some sites like to try to append a standard signature to all posts through
their service, generally as advertising. This creates the appearance of
users saying things that they didn't, runs the risk of corrupting messages
by appending text without regard to what's in the message, and can
possibly modify messages that arrive via a suck/rpost connection. It also
adds advertising in an obnoxious location, rather than in the Organization
header field which is more widely used for that purpose. Accordingly, INN
does not support this, or any other modification of the body of a message
from inside the news server.

If you only want to do this for a private hierarchy, the easiest way to do
this (as well as any other modifications and internal filtering that you
want to perform) is to mark all of the groups as moderated and route all
submissions through a script that makes whatever modifications you want
and then posts the messages with an Approved header field.

If you want to do this in order to advertise your service, please
reconsider. You can add your advertisements to the headers, like many
other news service providers.

------------------------------

Subject: 6.11. Hide the Injection-Info header field

There is no built-in support for suppressing generation of the
Injection-Info header field. You can, however, remove it from inside
a Perl posting filter. Try using a posting filter like this:

sub filter_post {
$modify_headers = 1;
delete $hdr{'Injection-Info'};
return '';
}

Note that you have to set $modify_headers to make changes to the article
header field effective in the actual posted article. Instead of removing
the header field, you can also alter it if you modify
$hdr{'Injection-Info'}. If you only want to alter the host name used in
Injection-Info, see the virtualhost: and domain: parameters in
readers.conf.

------------------------------

Subject: 6.12. Run innd and nnrpd on separate ports

Originally, innd was designed to handle all incoming connections and hand
them off to nnrpd as appropriate. It is, however, becoming increasingly
common to run innd and nnrpd on separate ports for a variety of reasons,
such as wanting to handle connections to nnrpd with a smart network
connection handling daemon like xinetd that can do things like rate
limiting of connections. INN does support this configuration, but be
warned that since you need to run nnrpd on port 119 for most reader
clients to be able to find it, you'll need to tell all of your news peers
to use a different port to feed you news.

The recommended alternate port for innd transit-only connections is port
433, which has been reserved for that purpose. If you want to use some
low-numbered port (less than 1024) other than 119 or 433 for innd, you
will need to build INN with the --with-innd-port option specifying that
port.

Now, set port in inn.conf to the port you want to run innd on and add
noreader: true (so that innd will never attempt to hand connections off to
nnrpd). Then, restart INN. It will now be listening on the new port.
You should now set up nnrpd to run via xinetd, inetd, tcpserver, or
some other similar network connection handling daemon on port 119. Make
sure that nnrpd is run as the news user, not as root. You don't have to
pass any arguments to nnrpd (unless you want to).

------------------------------

Subject: 6.13. Back up and restore an INN installation

(This entry is based on a post by Jeffrey M. Vinocur.)

For a full backup, you need, at a minimum, to save all of the articles in
$patharticles, the configuration files in $pathetc, and the active and
newsgroup files in $pathdb. If you have any custom filters you've
installed, or a cleanfeed.local file, you'll want to keep that, as well as
any custom authentication programs or files you're using (like a password
file for news accounts). You may also want to save HTML versions of the
news.daily reports, if you've been generating them, and you may want to
look at the first few lines of config.status in your original source tree
so that you can be sure to use the same options to configure.

Note that most people only back up those portions of the news spool that
they retain for a long time (like local hierarchies) and don't bother with
all the regular Usenet articles.

It's considerably easier to back up and restore articles from tradspool
than any other storage mechanism, and it's quite hard to back up and
restore timecaf or CNFS. Remember that you can use different storage
methods for different articles. I highly recommend saving the hierarchies
you want to back up in tradspool and use the higher-performance storage
mechanisms for news you don't care as much about.

To restore a single newsgroup using tradspool and the tradindexed overview
method, you can just restore the articles into the news spool and then
rebuild overview for just that group with tdx-util -R.

Otherwise, for more general restorations, compile INN on the new system
with the same ./configure command if you've lost the installation, run
make install, then put all the pieces back where they belong. Now, you
have to run:

makehistory -O

to rebuild the history and overview databases. When that finishes, cd to
the $pathdb directory and run:

makedbz -s `wc -l < history` -o

You should now be able to start the server and read and post news to it.

------------------------------

Subject: 6.14. Find external feeds and set up peering

One way to find peers is to ask for an external feed in the
news.admin.peering newsgroup. Some news administrators read that group

and may be willing to peer. It's common for news administrators to have

different criteria for peering (specific hierarchies, geographic or
network proximity, spam filtering, no binaries, binaries, specific network

protocols; the variation is endless), so finding someone with matching
goals may require some patience and possibly some configuration changes.
And why not keep subscribed to this newsgroup to help others find a news
feed once you get yours?

You will then have to configure your new feed in incoming.conf, newsfeeds
and innfeed.conf (assuming you'll use innfeed, the most common way to feed
articles). Follow the examples in the man pages of these configuration
files.

[Miner]

The author of the INN 2.x FAQ Russ Allbery continues to spread
disinformation. I express my contempt for all organizers of
disinformation.

[Russ Allbery]

Miner <inv...@invalid.invalid> writes:

> The author of the INN 2.x FAQ Russ Allbery continues to spread
> disinformation. I express my contempt for all organizers of
> disinformation.

In case anyone was wondering, this followed apparently the same person
trying to harass me in direct email about not wording the FAQ answer in
the way that they'd prefer. I'm not really sure what they expected the
outcome to be.

This is not the weirdest hill that I've seen someone try to die on, but it
certainly has flashback early 2000s Usenet kook energy.

Anyway, go write your own FAQ if you don't like this one. I'm not going
to change it further for your weird obsession.

[Russ Allbery]

Last-modified: 2022-08-13

and may be willing to peer. It's common for news administrators to have

different criteria for peering (specific hierarchies, geographic or
network proximity, spam filtering, no binaries, binaries, specific network

[Thomas Hochstein]

Russ Allbery wrote:

> In case anyone was wondering, this followed apparently the same person
> trying to harass me in direct email about not wording the FAQ answer in
> the way that they'd prefer.

Who would have thought that?

> Anyway, go write your own FAQ if you don't like this one.

I fear that makes you "Insolent or mentally retarded." ...

-thh

[se...@home.sethhurst.com]

I thought the faq was good so yeah.
go pound sand.
Don't let those people get you down.