Re: Replace Google Groups with ?

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>My NNTPSERVER is set to my cable ISP's NSP, which does not require any
>authentication.

The way NN does authentication is as follows:

1: Connect to the server and see whether you can post
2: If you can post, your fine, start reading
3: If you cannot post, you might want to post, so try an authentication
4: If you need authentication to read, then try an authentication

There are notes in the source code suggesting that it might be nice to
have a variable to control whether or not authentication is performed,
but my guess is that it is not done so right now.

>The /etc/hostname is correct (I do not have /etc/HOSTNAME). My hostname
>shouldn't matter anyway.

It shouldn't, but depending on the configuration of NN, it might prevent
logging in. Specifically, in Slackware, I had to make it use the hostname
file because NN expects an FQDN when getting the hostname, which
the hostname(3) was not giving it. It does not sound like this is a problem
for you, but it's something to note just in case.

>When I call nn, I see another local INN server answering, then the
>connection immediately disconnects. Don't know where this other server's
>hostname came from, nor how to change it.

In all the installations that I have seen of NN, if you specify an NNTP
server, it will use that over the default one that was specified, or
the nnmaster(1) database.

>I have /etc/news/server but it is for SLRN, and points to localhost for
>Stunnel support, and for a different local INN reader server which
>requires TLS on port 563.

This may or may not be causing NN to barf, but I would be suprised if it
is assuming that you are properly specifying the NNTP server. I'd have to
check the code.

>If I use "nn -g" I am asked "Enter Group or Folder (+./~)" and the only
>group I can access is one that unauthenticated readers are allowed access.

If you get a different set of groups when you are authenticated or
unauthenticated, but you can post in both cases, then NN might not be
asking for authentication, because it thinks that you already have all
the privileges you need.

>There is no prompt for authentication. I've found no information for
>placing authentication (username and password) into a file that NN can read.

At the moment, I believe that you always authenticate interactively, and
you cannot specify an username or a password in a file. If it is not prompting
you for authentication, my guess is that NN thinks it unnecessary.

>I have the file ~/.nn/init with the recommended configuration, which is
>similar to these munged values:

>set nntp-server {my_NNTP_FQDN}

It is useless to put this in an init file, because the server is determined
before the init file is read. The nn(1) man page has more information, but
you want to specify this option either on the command line explicitly, or
through an environment variable. The environment variable overrides the
command line option, which in turn overrides the default.

>set nntp-user {my_username}
>set nntp-password {my_password}

These are not valid settings, methinks.

>set news-header Organization: {My_Organization_Name}
>set mail-header Organization: example.invalid

Good, you know about these settings. :-) They are very useful.

>Changing the first three set lines by using : or = causes an "unknown
>variable" error message for all three lines.

That's because they don't exist in those environments.

>I can read -- and post -- but my real e-mail address is shown, which is
>definitely not wanted.

I actually change my From header to display my real email address,
which I know a lot of people do not like, but the process is the same
for using invalid From headers as well. From the nn(1) man page, the
news-header and mail-header settings can be used to specify more than a
single header. Each header line can be separated by a semicolon. See
the man page's documentation of the mail-header option for an example,
which sets the Reply-To and Organization headers.

>So, the two stumbling blocks are authentication, and changing (munging)
>my e-mail address and substituting "example.invalid" in its place.

I wonder if authentication will just be a problem in the current version,
because NN simply will not ask for authentication if you have posting
access without needing to authenticate. I think I should release a patch
of NN that fixes this. :-)

>I'd also prefer to use the command line to switch to various NNTP
>servers instead of needing to edit the ~/.nn/init file.

Actually, this is the only way that you *can* switch servers. Keep in
mind that switching servers really also requires that you switch your
newsrc and .nn directories, too; the data stored in each is not portable
across servers. I use GigaNews and Gmane. Normally I just set GigaNews
up as the defualt, and then I have an alias in my shell when I want to
run NN for gmane, like so:

alias nngmane='NNTPSERVER=news.gmane.org nn nntp-directory=~/.nn-gmane \
newsrc=~/.newsrc-gmane'

[Line truncated for transmission.]

>I must admit that every time I've actually had time to fool around with
>NN, it's been past my bedtime, and I was on a quest to try and whip it.
>Perhaps I was too tired to think?

Not necessarily. NN *is* different compared to the other news clients
out there, and it does a lot of things differently. It's not always
obvious how to achieve something, and I've acquired a lot of this
knowledge through reading through old FAQs, testing, and reading the
sources (which are actually remarkably clear and coherent). However,
most of the questions you are dealing with can be answered with a patient
read through the man page. Of course, patience and reading through man pages
does not always go together at all!

>I have NN on a couple other computers, and have spent time on it in the
>past, but never have successfully figured it out, nor configured it to
>run as seamlessly as TIN, SLRN, Gnus, ....

>If you have suggestions, I'd be grateful for any assistance.

>BTW, I'm running Debian on most of my boxen, and I do have various
>versions of Slackware, but the distro shouldn't matter.

Actually, in NN's case, the distro *does* matter. Slackware had a broken
version of NN for a while until some patches were sent in and those were
fixed. NN does a lot of configuration of settings and code paths at
configuration time, and it's sometimes easy to pick a combination that
works in most cases, but breaks in certain other cases. I do not know
how Debian's packaging is, but I know that the one on Slackware works well,
partly because I contributed to it a bit.

The distro shouldn't matter, and Debian should have a good set of defaults,
but those may be different than what are configured for the Slackware
version.

At any rate, kudos to you for setting up Gnus! Gnus is quite powerful, but
in terms of sheer editing prowess necessary to get it up and running, I
think NN wins, though Gnus has a ton of how-tos and walkthroughs, which NN
lacks.

I have a number of things that I want to do to enhance NN a bit,
foremost among them being Unicode support. However, the real reason I like
such an old piece of software is that reading through news is *so* much
faster when I read with NN. It's so easy to get through newsgroups with its
interface and workflow. I don't know how you'll like it, but I imagine it
might fit somewhere in your toolbox, once you get the thing to work! :-)

--
--
Aaron W. Hsu | arc...@sacrideo.us | http://www.sacrideo.us
Programming is just another word for the lost art of thinking.

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>Aaron W. Hsu wrote:

>> 1: Connect to the server and see whether you can post

>The problem right here is without proper authentication, the news server
>restricts unauthenticated connects to only one group (on my servers), or
>however many groups an administrator may assign for connections where
>authentication fails.

[...]

>> 2: If you can post, your fine, start reading
>> 3: If you cannot post, you might want to post, so try an authentication
>> 4: If you need authentication to read, then try an authentication

>Here, NN is just not prompting for authentication, or is not providing
>it after receiving a 200 reply.

If you read the above steps that NN takes when doing an initial server
negotiation you would see why it is not authenticating. After you detailed
the above and below, it is now clear to me, I think, why you are seeing
what you are. In particular, let's look at these logs...

>Without being authenticated, any access command is going to receive a
>480 reply.

>Here is a sample telnet session (with munged areas):

>john@hardy:~$ telnet news5 119
>Trying 192.168.33.5...
>Connected to news5.my.net.
>Escape character is '^]'.
>200 news5.my.net InterNetNews NNRP server INN 2.5.2 ready (no posting)
>group local.test
>480 Read access denied
>authinfo user {my_username}
>381 Enter password
>authinfo pass {my_password}
>281 Authentication succeeded

>Here I tried to access the local.test group, but it requires
>authentication, so the server gave me the 480 error.

If you had tried to do the same thing with NN, it would have also asked
you for authentication.

>Now if I connect without authentication, the only group I can access is
>the local.info group:

>john@hardy:~$ telnet news5 119
>Trying 192.168.33.5...
>Connected to news5.my.net.
>Escape character is '^]'.
>200 news5.my.net InterNetNews NNRP server INN 2.5.2 ready (no posting)
>group local.info
>211 1 3 3 local.info
>list
>215 Newsgroups in form "group high low status"
>local.info 0000000003 0000000003 m

>I can use the article command with 3 and read that article.

Seeing the above makes it obvious why NN is not asking for authentication.
Firstly, when it first connects, the message that it is receiving is a
200 code, which indicates that posting is allowed. Thus, NN, does not
even try to do any initial authentication, because it does not need to
do any right then and their. Instead, if you did not specify a specific
group, it will just run the LIST command, which I am guessing does not
require authentication either on your server. In this case, NN happily
continues, because the server has not requested authentication.

If you had instead told NN to do something that required authentication,
then it would have asked for it. If the server had responded with a 201,
indicating that posting was not allowed, then NN would have tried to
authenticate, to see if authenticating would give it more privileges.
Since the server did not respond so, and everything that NN tried was
okay without authentication, NN did not ask you for any authentication
details, nor use those that you provided.

>> There are notes in the source code suggesting that it might be nice to
>> have a variable to control whether or not authentication is performed,
>> but my guess is that it is not done so right now.

>It should be provided on a server basis. Some servers require
>authentication, and some don't.

You misread or misunderstand me: NN does authentication on a per command
basis, not per server; you seem to think that NN does authentication at
some coarser granularity than either of these, NN is just being more
precise about authentication than what you expect. It is commands that
require authentication. NN will not give authentication unless it is
required to do so by the server, or if it is restricted in a way that it
thinks it can overcome by authenticating; namely, it will try to authenticate
when it receives a 480 reply, or when the initial message gives it a 201
code. Your use of NN is not generating either of these messages, so NN is
not authenticating.

The note I have above is saying that the coders of NN *know* that there
is no configurability (say, in the init file) of whether to blindly
authenticate at log in, even if the server gives a 200 instead of 201
as its opening message, and they acknowledge that having a variable to
set this might be nice. However, for the vast majority of installations,
this is not a problem, even today. It is a problem for you because you
actually change information that you send based on authentication
information, but you don't do anything to indicate this. Your server
does not give a 201 code, and I am guessing that the LIST command works
without authentication, so NN has no way of knowing that it could have
"gotten more groups" if only it had authenticated.

>> It shouldn't, but depending on the configuration of NN, it might prevent
>> logging in. Specifically, in Slackware, I had to make it use the hostname
>> file because NN expects an FQDN when getting the hostname, which
>> the hostname(3) was not giving it. It does not sound like this is a problem
>> for you, but it's something to note just in case.

>Well, it could be if it is associated with the problem of using my real
>e-mail address. That would bring on the spammers for sure.

That problem is a problem with you not using a custom From field, and it
has nothing to do with your hostname. NN is using a default address based
on your hostname because you have not overridden the From field in your
configuration, but it would do this regardless of the hostname information
it receives. Thus, I am certain that this is not your problem in any of
the gripes you have with NN right now.

>IIRC, I could only get NN to access the server that is listed in the
>~/.nn/init file.

>Perhaps there is another configuration value that would put one file
>above another, but I think I've tried them all.

If there is an NNTPSERVER variable, NN should use that; if not, it should
use the value on the command-line; and if not, then it should use the
init file value.

>>> I have /etc/news/server but it is for SLRN, and points to localhost for
>>> Stunnel support, and for a different local INN reader server which
>>> requires TLS on port 563.
>>>
>>
>> This may or may not be causing NN to barf, but I would be suprised if it
>> is assuming that you are properly specifying the NNTP server. I'd have to
>> check the code.

>This is not used for NN. It is for SLRN, and maybe Tin, if I don't
>override it with a launch commend parameter.

You seemed to think that it was grabbing a local spool rather than using
the NNTP server that you specified. That's how I read this part of your
message. If that is not the case, then we can dispense with this part
entirely.

>Please review the authentication info above. Basically, without
>authentication, there is no access except to one group (local.info), and
>posting is prohibited anywhere.

But your initial log in server response gives a 200 reply instead of a 201,
which in the protocol indicates that posting *is* allowed. As I mentioned
above, NN will not try to authenticate initially if it does not receive
a 201 on login. It *will* try to authenticate later on if a command it
tries gets a 480 reply.

>NN cannot know whether a news server requires authentication unless the
>server asks (none do AFAIK), or NN has a configuration file setting
>(which we are looking for and not finding).

All servers that I have ever used, including GigaNews, Gmane, and my ISP,
among others, either work without passwords, or request a password
explicitly. For example, with GigaNews, here's what I get from a Telnet
session:

Trying 216.196.97.131...
Connected to news.giganews.com.
Escape character is '^]'.
200 News.GigaNews.Com
MODE READER
200 reading enabled
LIST
480 authentication required

The above simulates what NN does on logging into the server. NN sees the
480 reply and then asks me to authenticate. The problem you are
experiencing is that you give different results for a command such as
LIST depending on authentication rather than just returning 480 on
the command, as most servers that require authentication would.

>Some cases allow authentication by putting username:password on the
>command line as an attribute. I do not find it works with NN, and don't
>recollect reading it in the various documentation.

If you have a version of NN with nntp-user and nntp-password variables,
then you can set them explicitly on the command line:

nn nntp-user={username} nntp-password={password} ...

>In fact, the word "authentication" is not even in the docs.

That's because in most versions of NN, this was always done interactively,
it is only in relatively recent NN versions that nntp-user and nntp-password
were supported, and the vanilla source of 6.7.3 only has that in the README,
and does not document it in the man page.

>The nn(1) man page is where I read this, and it works.

It is obvious that you are using a patched version of NN, since the
6.7.3 version that I have does not have this in the man page, but instead
explicitly recommends against putting nntp-server in the init file, and
recommends always putting it in the command-line. However,
while the man page says this, I think that some patches were introduced
that caused the init file to be loaded earlier, so that you can get away
with using nntp-server in the init file, which is what you have been doing,
apparently.

>>> set nntp-user {my_username}
>>> set nntp-password {my_password}
>>
>> These are not valid settings, methinks.

>They are also mentioned in the man page, but they obviously do not
>permit authentication.

>If the syntax is incorrect, then NN will fail to launch as mentioned below.

Actually, these are not documented in the vanilla version of the man page,
but they are in the code (I just checked) nonetheless, and the README
talks about their introduction. Thus, someone seems to have patched your
man page to include them and make adjustments for recent code changes.

You incorrectly state that the "obviously do not permit authentication,"
because they *will* be used if NN ever needs to authenticate to the server.
The problem here is not that NN cannot authenticate, but that it does
not authenticate with your server because your server does not suggest
anything to NN that makes NN think it should authenticate.

>>> set news-header Organization: {My_Organization_Name}
>>> set mail-header Organization: example.invalid
>>
>> Good, you know about these settings. :-) They are very useful.

>Yes, but neither is for the e-mail address (From or Sender).

Actually, they both are, and I explained how to use them further
down, and the man page explains how to use them. This is what I wrote
that explains how to use these settings.

>> I actually change my From header to display my real email address,
>> which I know a lot of people do not like, but the process is the same
>> for using invalid From headers as well. From the nn(1) man page, the
>> news-header and mail-header settings can be used to specify more than a
>> single header. Each header line can be separated by a semicolon. See
>> the man page's documentation of the mail-header option for an example,
>> which sets the Reply-To and Organization headers.

And since the above obviously did not help you, here is the relevant text
from the nn(1) man page:

mail-header headers (string, default not set)
The headers string specifies one or more extra header lines
(separated by semi-colons `;') which are added to the header of
mail sent from nn using the reply and mail commands. For exam?
ple:
set mail-header Reply-To: st...@texas.dk;Organization: TI - DK
To include a semicolon `;' in a header, precede it by a back?
slash (which must be doubled because of the conventions for
entering strings).

news-header headers (string, default not set)
The headers string specifies one or more extra header lines
(separated by semi-colons `;') which are added to the header of
articles posted from nn using the follow and post commands. See
mail-header for an example.

As you can see, you use these settings if you want to change the From
address or the Sender.

>So you are manually editing the From in the editor? That would be a real
>task if you post many articles!

That would be a task, but as you can see, I am not.

>How do people use NN with authentication? The INN news server is the
>most common on Usenet, and many Usenet servers require authentication.

I am using it every day with authentication and without, with by using
GigaNews, and without by using gmane.

>Perhaps NN is just not very common. I rarely see anyone posting with it.

It is not very common nowadays, that's true, but not because it doesn't
know how to authenticate, because it does.

>I read headers most of the time, and have the Display Mail User Agent
>extension in Thunderbird. If a big red NN shows up, I will notice it. I
>rarely see it.

Do you see it on my messages? :-)

>I know newsrc is different for each server. However several newsreaders
>can often share the same newsrc for a particular server.

Yes, you can use the same newsrc for the same server with different
newsreaders, and NN should play nicely with other clients that use ~/.newsrc,
but if you want to use NN on *different* servers, unless you know
that they can play nicely, you will need to use the newsrc variable
to change the newsrc file appropriately for each server. You don't have
to, but then things could go wrong (of course, you know what you are
doing, so feel free to pick and choose).

>I access many servers, so some kind of separation is definitely required.

Yes, so then you would want to call nn with nntp-server, newsrc, and
nn-directory all set explicitely on the command-line for each server
that you want to connect to.

>The ~/.nn/init file does work for choosing a server. Sometimes it is the
>only way to choose a server, and it overrides whatever server is given
>on the command line.

You should set it on the command line if you have a lot of different
servers to connect to. As you mentioned, setting it in the init file
and editing it each time you want to connect to a new server is a pain.

>I have read everything there is to read concerning NN. I have then
>searched (grepped) the documentation, and came up with nothing.

Debian seems to have a slightly customized version of NN. Slackware's is
much closer to vanilla. However, the above information on authentication,
in all its gory detail is not available in documentation. The variables
and the other settings are, but the exact workings of the authentication
I derived by reading the source code. Still, most of your other inquiries
*are* answered in the documentation. For example, your query about
the nntp-server, I get this from the man page:

newsrc file (string, default "~/.newsrc") Specifies the
file used by nn to register which groups and articles have been
read. The default setting corresponds to the .newsrc file used
by other news readers. Notice that nn release 6.4 onwards does
allow individual articles to be marked unread, and some articles
marked unread, and thus no longer messes up .newsrc for other
news readers! Also see nntp-server.

nn-directory directory (string, default "~/.nn")
It only makes sense to set this variable on the command line,
e.g. "nn-directory=$HOME/.nn2" since it is looked at before the
init file is read. It must be set to a full pathname. Usually
set when using multiple servers; see newsrc above and nntp-
server below.

nntp-server hostname or filename (string)
It only makes sense to set this variable on the command line,
e.g. "nntp-server=news.some.domain", since it is looked at
before the init file, If you use multiple servers, you probably
want to set the nn-directory and newsrc variables on the command
line to alternate names as well, since some of the data files
are server dependent.

That explains pretty much all you need to know about multiple servers
and nntp servers. The only thing not discussed there is how to actually
set those options on the command line, and here is the relevant man page
text on that:

Some variables only make sense when set on the command line,
since they are examined early in startup, before the init
files are read. The syntax for setting variables on the command
line is:
variable=value
The value may need to be quoted if it contains white space or
special characters. They can be intermixed with other options,
and are exam? ined prior to other argument parsing.

And there you go.

>That is a Slackware problem, not NN nor Debian. ;-)

Actually, Debian patches its software a lot, it seems, in this case, though
none of the patches seem nefarious, unless your man page lacks the above
text that I quoted, in which case you are missing out a lot.

>Patches should have been provided for NN, and not specifically for just
>Slackware's NN package.

The patches were related to the configuration of Slackware's NN, not in
NN itself. A patch that Slackware used to provide a nicer error message
had a missing brace that caused universal failure, and it had a bad
configuration that did not allow for use on systems that did not have an
FQDN. Neither of these were upstream problems. They were also fixed
some years back.

>The authentication problem should be at the head of the list. Without
>it, I'm afraid NN is something of a relic from the very distant past.

I have added this as an issue in my tracker at GitHub. The code is
public, and this is an easy fix, so I just might knock this one out. :-)
However, please understand that the issue is *not* that NN cannot do
authentication or that it is somehow non-compliant. It is doing things
just fine, but it lacks an explicit option to always send authentication,
regardless of the server information. This is useful, but it is not
a failing, IMO.

>Another problem I just found is the port is not assignable. That is
>something available in modern newsreaders.

Yes, to my knowledge NN does not allows a custom port unless you
edit the service database to change the NNTP/TCP service.

>NN does not show the selected article by using Z or X. It is broken I'm
>afraid.

Z and X are not meant to show a single article. They are meant to take
you into reading mode where you can read the articles that you have
selected. Thus, if you have a single screenful of articles and you
hit 'a' and 'c', then hit 'Z', NN will show you article 'a', which you
can page through using the space character. When you page through 'a',
the article that you marked with 'c' will then be displayed and you can
page through it the same way. The difference in Z and X is what happens
when you have gone through all the articles that you marked when you
were in selection mode. 'Z' takes you back to your list of articles
with all the articles that you did not mark still marked as being unread,
whereas 'X' will mark all the articles that you did not select to read
as having been seen or read, and when you finish reading the articles that
you did select, it will not take you back to the group you were in,
but will instead take you to the next group.

These are also documented clearly in the man page.

>And it is much slower than any other newsreader I use.

This used to be the case for me on slow connections as NN downloads
the list of new articles at startup before it displays the first
group, but with fast connections these days, that is a very small time,
and this time is actually shorter than the aggregate amount of time
for some other readers I have used to update their message lists.
As for actually reading news, I have not found NN to be any slower, and
actually faster in workflow than the others by a long shot. Only
initial startup may be slightly slower than others, do to its batch
nature.

>I particularly do not like the following idea, found at
>http://en.wikipedia.org/wiki/Nn_%28newsreader%29 :

> NN was known for its mode separation between choosing and reading
> articles. Threads were presented for reading or skipping, and once
> all choices in a newsgroup were made, the user continued on to
> reading the selected articles.


>How can you choose something if you don't read it? Topic/Subject drift
>long ago killed that glorious idea!

Ah, but you see, that *is* the primary reason why I like NN so much!
This separation of selecting what articles I want to read first and
then reading through them is *so* much nicer for me than having to
laboriously click or navigate through the header list while simultaneously
reading the news.

Here's how it goes for me; I obviously don't want to read some articles
and I can tell this from the subject line. With NN I simply don't select
those articles. That usually eliminates my need to read most of the articles
in a group, or at least, the obviously useless articles. Then comes
the ones that I might just be interested in. These I select for reading.
Then I go into reading mode and start going at it. Many of the articles
are what I wanted, and I read through them entirely. Some are useless,
and I don't care about them, so I skip reading them in their entirety
and use the 'n' key to go to the next article in my selected reading list.
Sometimes, it is obvious that a thread has devolved and that it no longer
contains useful information. At that point, rather than plog through
the rest of the articles of that subject that I may have selected, I just
use the 'k' key to temporarily kill that subject just for this session,
and I can continue reading the next thread. If I decide I want to
permanently kill a subject, then I can, of course, do that as well.
Now, the other way around sometimes happens. I read the first post of
an article and think that I would like to follow more, well, rather
than going back to selection mode, I can just hit '*' and all the articles
that are followups to the article I am reading will be selected and added
to my reading list for me.

This is so much more efficient for me than what I have to do in other
readers, which I have to scroll through the list in one way, and then
jump around as I proceed to read though the lists. Certainly it is no
slower or more wasteful than having to open each article, since I can
already eliminate articles that I do not want to read. So, in fact,
I can deal with topic drift and useless articles much faster this way than
with the normal approaches.

Obviously, you may not like this approach, and if you don't then you
probably will not like NN. However, I think it's worth giving it a shot,
since it is surprisingly nice to use once you do.

>21 years have passed, and Michael's site is broken badly. <sigh>

Yes, it is, which is why I have my own GitHub where I keep my own work
on NN.

>If NN is doing what you want it to do, what you need, then who am I to
>suggest you try Tin or SLRN? They both have what I need for my
>text-based netnews desires, and are easy to navigate, especially Tin.

Oh, they are both fine readers, and I have used them both, but in both
cases I found them slower and harder to use efficiently than NN. They
have a lot of nice features, but NN is more usable for my workflow.

[Aaron W. Hsu]

Well, for "potential" users like John F. Morse,

I have just pushed patches to my GitHub version of NN to support the
forcing of NN to authenticate itself whether or not the server requires it
or not. Normally, NN tries to authenticate only if it receives a
201 response code (it want to see if it can get a 200 posting allowed code)
or if it receives a 480 response during normal use. The latest
master branch of my NN development trunk enables an option nntp-auth,
which is boolean. When set, NN will always try to authenticate, and
when not set, it will use the traditional authenticate on demand approach.

The code is located here: https://github.com/arcfide/Unicode-NN

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>If you don't mind, and you wish to reply, could you trim this too-long
>article and get rid of anything that isn't really necessary? It's grown
>way out of hand, and a lot is due to my replying to each paragraph, and
>leaving the quoted text intact.

Yep, I am definitely doing that here, since I think we got most of the
issues solved or at least identified.

>If you think it would be better, since this doesn't really pertain to
>Slackware, move any reply to n.s.nn where I am also subscribed, and I'll
>reply there if need be.

Yep, I agree, and I am moving it over there. I'll be responding shortly,
and I just wanted to put this one here, for the archives.

[Aaron W. Hsu]

Hey John:

Thanks for this discussion. I hope we can stick with it as it is generating
a lot of useful information and it's improving NN as well. Firstly, I
want to address the protocol discussion that we are having, which I have
included below in snipped fashion. I think we are disagreeing about what
the correct protocol responses for servers should be. I am using the
RFC 3977 document from here as a reference:

https://tools.ietf.org/html/rfc3977

You have a server configured to allow read-only, unauthenticated access.
You have a limited set of groups that you want to allow read access to,
above and beyond the basic restriction that the users who are unauthenticated
cannot post. You are doing two things that affect NN: first, you send
different listings for unauthenticated and authenticated users; second,
you always send the 200 code as the initial greeting.

Let's consider the first (different listings). If, instead of giving
a different groups listing to unauthenticated users, you gave the same
group listing in both cases, you could still limit read and write
access to specific groups, because an attempt to read the group would
result in an access denied message. NN user's would see the groups,
but would authenticate when trying to get to one that was restricted.
(Incidently, I am addressing the groups access in absence of a complete
listing in a separate message.) In the RFC, I get this information from
section 3.2.1:

The server MUST respond to any command with the appropriate one of
the following generic responses if it represents the situation.

[...]

If the client is not authorized to use the specified facility when
the server is in its current state, then the appropriate one of the
following response codes MUST be used.

[...]

480: The client must authenticate itself to the server (that is, it
must provide information as to the identity of the client)
before the facility can be used on this connection.

This code will prompt NN to ask for authentication, which will happen when
it tries to access your restricted groups.

Now, the second issue is the initial greeting. Here is section 5.1.1 from
the RFC:

Responses [1]
200 Service available, posting allowed
201 Service available, posting prohibited
400 Service temporarily unavailable [2]
502 Service permanently unavailable [2]

[1] These are the only valid response codes for the initial
greeting; the server MUST not return any other generic
response code.

Notably, 201 is the code to send when you are not allowing posting on
the server, but you sill want to give access. The way I am reading the
RFC, your server should send 201 as its initial greeting, as the client
is initially unauthenticated and thus has not posting priveleges.
Now, at that point, NN will see the 201, and try an authentication to
see if it can get a 200 code instead by authenticating.

"John F. Morse" <jo...@example.invalid> writes:

>200 server ready - posting allowed. Sent by the server upon initiation
>of the session, if the client is allowed to post messages.

>In the case of a read-only newsgroup, the client is not allowed to post,
>but that is beyond the initial connect stage. Attempts to post will be
>blocked when the article post command is given. Perhaps one of these?:

>440 posting not allowed. POST command issued when posting is not allowed.

>502 access restriction or permission denied. Permission denied; sent if
>the client has not properly authenticated but the server requires it.

[...]

>I think your problem is you believe "posting" comes before "reading" --
>or in this case "accessing."

>Nobody can post if they can't get there in the first place. If they can
>get there, but posting in that group is prohibited (a "read-only"
>group), then there is no 201 issued.

>201 server ready - no posting allowed. Sent by the server upon
>initiation of the session, if the client is not allowed to post messages.

>Getting in the "posting door" requires authentication, as well as access
>into a group that allows posting or is moderated. If you can't get in
>the door, then the server has no idea who you are, nor if you are
>allowed to post.

>It does allow you to read any group that is assigned anonymous access
>privileges though. Since you are connected as an anonymous reader, you
>remain that until the connection is terminated.

>The initial 200 code is all NN knows, so it will not authenticate. It
>must be forced with a command option so the user can use it when needed
>for those servers that do require authentication.

>Again, you and I see the problem is NN won't ask for authentication if a
>server has issued the 200 response code.

>The reason the 200 is sent by INN (probably other NNTP/NNRP servers
>also) is there are one or more newsgroups available for unauthenticated
>connections. This is how I provide anyone access to a limited number of
>groups (actually only one) so they can obtain some information on how to
>apply for an account.

[...]

>I know what you are saying. My server gives the 200 because it allows
>non-authenticated readers -- to one group only -- and that group is
>read-only, so they cannot post. The initial 200 connect response is not
>repeated after connection.

[Aaron W. Hsu]

Hey John:

I want to talk about this little chunk here...

"John F. Morse" <jo...@example.invalid> writes:

>Aaron W. Hsu wrote:

>> If you had tried to do the same thing with NN, it would have also asked
>> you for authentication.

>I cannot do the same thing with NN, such as send the "authinfo" command,
>which is easy with telnet. Therefore I'm only presented the group that
>doesn't require authentication.

Firstly, NN should never *present* any groups to you, as it should always
just jump into the first subscribed group in selection mode, ready for
reading, unless you use the -a option for catchup. At any rate, NN will
still not "know" about any of the other groups because you only list some
of them, *but* this should not prevent you from accessing those other
groups. You should be able to specify the group explicitly and NN should
try to get to it, get a 480 authentication error from your server, whereupon
it will send the authinfo. You can do this on the command line, but please
also try with the 'G' key. It should try to access that group. I would
appreciate knowing what response it gives.

>The 480 error is actually designed for a transit server:

>480 transfer permission denied. Response to CHECK if transfer is not
>allowed.

See my NNTP discussion message for my thoughts on 480.

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>I discovered this problem a few years ago when using TIN. Urs Jan�en,
>the TIN developer, who is active in the news.software.readers group
>(where this discussion really should be taking place), added the -A
>command option to "force authentication on initial connect." That fixed TIN.

So this is not just an NN issue, and there is precedent historically for
newsreaders not to authenticate explicitely at the beginning. Anyways,
you will be pleased to know that I have already commited a fix for this
in my 6.8.0 release of NN (not yet released, but available via GitHub [1]).

It adds an nntp-auth boolean flag to indicate whether or not to force
authentication on connect.

So, this will fix NN with servers that are configured the way that yours
are, though I still maintain that you should change the response codes
that your server is sending as its initial greeting, and that you should
consider listing all the groups, not just one, to unauthenticated users;
you don't have to give them read access, just let them know they are there.

>That is correct, and I do understand you. NN is broken for use with a
>news server that operates with authentication and has groups which
>require no authentication.

More precisely 6.7.3 does not work as may be expected with a server
that does not present the full groups list *and* does not send a 201
code as its initial greeting. It *will* work as expected for a server
that simply requires authentication for some groups but not others.
Or that does one or the other of the above two troublesome things.

>This can be accomplished easily with a command line option such as TIN
>uses. Or it can be stored in a configuration file that NN reads. Or both.

It can now be done with both in NN as well. :-)

>So it is a stalemate, unless NN is patched to force
>authentication or at least have it as an option when connecting, like
>TIN does with the -A command option.

Done.

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>I saw these somewhere, but I don't find them in the nn(1) man page.

I have just updated my branch of NN with an updated man page documenting
the use of nntp-user and nntp-password. Thanks for bringing this to my
attention.

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>Alright, I got this working now from the command line. I had to remove
>the "set nntp-server" from the init file because it has final authority.

I suspect that this is not actually supposed to be the case, but I think
that this must have happened when changes to the loading process of the
init file changed.

At any rate, I have filed an issue on this on my GitHub just in case I have
time to get to it.

[John F. Morse]

The problem I see here is INN (and most other NNTP/NNRP servers) are
written to serve a complex audience.

A corporate news server is configured to allow access -- including
seeing the names of newsgroups (the active file) -- to specific groups
of users.

You wouldn't want the cafeteria department to be looking over the
corporate leadership's listing of newsgroups.

News servers in universities are configured to disallow students access
to the staff's newsgroups. This is quite common.

There are Usenet groups, local groups, and private groups. Not to
mention the various control.* groups.

In a NSP setting, where the NSP resells NNTP (NNRP) access, it becomes
more clear why it is important to only show information which is
appropriate to specific users. It's called revenue, and the almighty
dollar is the key!

Very important when an NSP, like Highwinds, provides backend group
access for resale to smaller NSPs, and/or to the reading public. I'm not
speaking of transit "peering" but reader access determined by incoming
IP netblocks.

My cable ISP is an example. Unlike many big ISPs, mine still provides
Usenet for its subscribers, but has never operated any NNTP/NNRP servers.

Instead, the news FQDN is forwarded (routed, whatever) to an NSP that is
a reseller. In my case, that is usenetserver.com (a few years ago it was
a different NSP reseller, newsfeeds.com).

Highwinds bought usenetserver.com, and has bought many other smaller
NSPs in that last few years.

http://www.newsgroupservers.net/usenetserver_newshosting_highwinds_review/

Why? Most of these big ISPs used Highwind's software (Typhoon, Cyclone,
Tornado, Hurricane, etc.).

The rich ISPs killed their news server farms when Andrew Cuomo, then the
NY Attorney General, scared them in his quest of political excellence,
and the governorship of NY. These NSPs no longer paid Highwinds for
software nor for support.

Most believe the ISPs saw this as a good "politically correct" reason to
stop selling NNTP, which cost them a lot of money, and didn't provide
any income.

I'd argue that Usenet is a selling point for an ISP account, a value,
but you know most computer users today only know how to operate a
browser, and Facebook serves all of their desires. <sigh>

So the ISPs killed Usenet. Others followed, like Duke University, where
it all started in the beginning.

Highwinds Media started feeling the pain of lost revenue. Plus many
server farms were using INN and Diablo, both free of cost and Libre
free, and likely more configurable and more powerful (I've never used
Highwinds' software). These guys are both commercial and private.

A look at the ranking in the number of articles they have seen by
reported Path data, can be found at the Top1000 listing:

http://top1000.anthologeek.net/topsum.current.txt

INN is designed with this screening and assigning reader privilege in
mind. I wouldn't be surprised if other news server software includes a
similar ability.

http://en.wikipedia.org/wiki/InterNetNews

From your viewpoint as a newsreader developer, and that of the vast
majority of regular news server "users," most don't even realize such
selective newsgroup screening is even possible.

Your task is to make NN adhere to both the RFCs and to work with as many
news servers as possible.

NN is "old school" (that is NOT a negative evaluation) which was
designed in a simpler time. Usenet has evolved over the years, and so
have the news server programs.

Is it like a cat chasing its tail? ;-)


--
John

When a person has -- whether they knew it or not -- already
rejected the Truth, by what means do they discern a lie?

[John F. Morse]

Aaron W. Hsu wrote:
> Hey John:
>
> I want to talk about this little chunk here...
>
> "John F. Morse" <jo...@example.invalid> writes:
>> Aaron W. Hsu wrote:
>>
>>> If you had tried to do the same thing with NN, it would have also asked
>>> you for authentication.
>>>
>> I cannot do the same thing with NN, such as send the "authinfo" command,
>> which is easy with telnet. Therefore I'm only presented the group that
>> doesn't require authentication.
>>
>
> Firstly, NN should never *present* any groups to you, as it should always
> just jump into the first subscribed group in selection mode, ready for
> reading, unless you use the -a option for catchup. At any rate, NN will
> still not "know" about any of the other groups because you only list some
> of them, *but* this should not prevent you from accessing those other
> groups. You should be able to specify the group explicitly and NN should
> try to get to it, get a 480 authentication error from your server, whereupon
> it will send the authinfo. You can do this on the command line, but please
> also try with the 'G' key. It should try to access that group. I would
> appreciate knowing what response it gives.
>

If I run nn with no options, the process ends immediately.

john@hardy:$ time nn

real 0m0.138s
user 0m0.012s
sys 0m0.012s


I must use the -g option to keep NN running.

john@hardy:~$ nn -g

Enter Group or Folder (+./~)

At this prompt I must enter a group name.

"No group matching `local test'" is what I receive here.

Or after successfully entering the one open group local.info -- and then
after using G and supplying the local.test newsgroup name -- I still
receive the "No group matching `local test'" as before.

The group exists but NN cannot see it because the server isn't providing
groups that are protected by authentication.

>> The 480 error is actually designed for a transit server:
>>
>> 480 transfer permission denied. Response to CHECK if transfer is not
>> allowed.
>>
>
> See my NNTP discussion message for my thoughts on 480.
>

Yes, you wrote:

480: The client must authenticate itself to the server (that is, it
must provide information as to the identity of the client)
before the facility can be used on this connection.

That is exactly how it is working.

Note the wording "used on this connection" above.

I'm already into a "connection" at this time. Some newsreaders allow
more "connections" usually one to four.

Here are the two "connection" (#1547 and #1548) attempts from a tail on
the server's /var/log/news/news.notice file:


Aug 6 22:03:29 news5 nnrpd[1547]: hardy.my.net (192.168.30.10) connect
- port 119
Aug 6 22:03:36 news5 nnrpd[1547]: hardy.my.net times user 0.992 system
0.188 idle 0.000 elapsed 7.312
Aug 6 22:03:36 news5 nnrpd[1547]: hardy.my.net time 7440 idle 5972(4)
nntpwrite 3(12)
Aug 6 22:03:40 news5 nnrpd[1548]: hardy.my.net (192.168.30.10) connect
- port 119
Aug 6 22:03:49 news5 nnrpd[1548]: hardy.my.net group local.info 0


Connection 1547 was the initial connection attempt to local.test, which
I do not have access privileges because I didn't authenticate, because
NN wasn't pushing authentication.

Connection 1548 is where I used G to enter local.info which is the only
group I could access while unauthenticated.

I went back and ran the tests again, accessing local.info and actually
reading one article:


Aug 6 22:16:39 news5 nnrpd[1548]: hardy.my.net group local.info 1
Aug 6 22:16:39 news5 nnrpd[1548]: hardy.my.net exit articles 1 groups 2
Aug 6 22:16:39 news5 nnrpd[1548]: hardy.my.net times user 1.516 system
0.172 idle 0.000 elapsed 780.329
Aug 6 22:16:39 news5 nnrpd[1548]: hardy.my.net artstats get 1 time 0
size 2103
Aug 6 22:16:40 news5 nnrpd[1548]: hardy.my.net time 780465 idle
778013(8) readart 0(1) nntpwrite 1(24)


Notice the second line is the exit ("Q" in NN), and shows the total of
two group commands, both for local.info. The article command was
received only once (I didn't use "Z" on the second test).

Now compare the above NN session with this news server's
/var/log/news/news.notice log file when I use TIN, and it asks for
authentication, which I supply:


Aug 6 22:44:08 news5 nnrpd[2211]: hardy.my.net (192.168.30.10) connect
- port 119
Aug 6 22:44:16 news5 nnrpd[2211]: hardy.my.net user john
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group alt.nn.test 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group
alt.os.linux.slackware 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group local.announce 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group local.binaries 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group local.computers 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group local.info 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group local.test 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group news.software.nn 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group de.comp.os.os2 0
Aug 6 22:44:20 news5 nnrpd[2211]: hardy.my.net group
nl.comp.os.linux.techniek 0
Aug 6 22:44:47 news5 nnrpd[2211]: hardy.my.net exit articles 1 groups 10
Aug 6 22:44:47 news5 nnrpd[2211]: hardy.my.net times user 2.508 system
2.148 idle 0.029 elapsed 40.036
Aug 6 22:44:47 news5 nnrpd[2211]: hardy.my.net artstats get 1 time 0
size 1539
Aug 6 22:44:48 news5 nnrpd[2211]: hardy.my.net time 40186 idle
33955(21) readart 18(1) nntpwrite 2786(78418)


Here the exit totals, shown in the first 22:44:47 line, were 10 group
commands and one article command.

The terminal command I used to launch TIN was:

tin -rAg news5

From tin(1), those command options are:

-r Read news remotely from the default NNTP server specified
in the environment variable $NNTPSERVER or contained in
the file /etc/news/server.
-A Force authentication on initial connect.
-g server Use this server, overriding $NNTPSERVER.


TIN can read news locally from a local spool (e.g. /var/spool/news), or
remotely with "rtin" or "tin -r" option).

The environment variable $NNTPSERVER is empty.

My /etc/news/server has "localhost" because I usually use TIN to access
another local server that requires TLS, which is provided by Stunnel on
this localhost computer.

john@hardy:~/.tin$ man 1 tin | wc -l
2966

john@hardy:~/.tin$ man 5 tin | wc -l
2178

john@hardy:~/.tin$ man nn | wc -l
4618

HTH.

[John F. Morse]

Aaron W. Hsu wrote:
> "John F. Morse" <jo...@example.invalid> writes:
>
>> I saw these somewhere, but I don't find them in the nn(1) man page.
>>
>
> I have just updated my branch of NN with an updated man page documenting
> the use of nntp-user and nntp-password. Thanks for bringing this to my
> attention.
>

You are welcome.

Sometimes, like I mentioned before, when I'm tired I read things and
remember them, or parts of them, but not where I saw them. ;-)

Add to that I've done a ton of Googling and read a lot that others have
posted. Where? I have no idea. I've slept since then. ;-)

[John F. Morse]

Aaron W. Hsu wrote:
> "John F. Morse" <jo...@example.invalid> writes:
>
>> I discovered this problem a few years ago when using TIN. Urs Jan�en,
>> the TIN developer, who is active in the news.software.readers group
>> (where this discussion really should be taking place), added the -A
>> command option to "force authentication on initial connect." That fixed TIN.
>>
>
> So this is not just an NN issue, and there is precedent historically for
> newsreaders not to authenticate explicitely at the beginning. Anyways,
> you will be pleased to know that I have already commited a fix for this
> in my 6.8.0 release of NN (not yet released, but available via GitHub [1]).
>

Historically there was little or no need to authenticate. Spammers were
not a problem yet, and pay subscriptions from NSPs were not widespread.
An ISP could control access to their news servers by checking user
accounts when accessing the ISP (e.g. CHAP, MAC, or via IP netblocks).

Newsreaders of yesterday were rather simple. The first improvement they
provided was probably threading.

Later access problems required authentication. That was added as well as
all the other bells and whistles, like scoring and killfiles, HTML,
attachments, displaying images, decoding, joining multiparts, UTF
support, various storage formats (mbox, maildir, MH), colors, flagging,
multiple server access, ....

That is why I made the statement that NN was rather "old" (or however I
worded it). You are using NN in the old school philosophy, on servers
protected in different ways (known IP netblock access from a cable ISP
for instance). And from NSPs that have only one set of authentication
access path privilege.

NN certainly works for your style, and likely for many others, but time
has changed a lot of things. Many newsreaders have kept up with current
needs, and some still lack some things, like TLS, which is important if
you don't want the "man-in-the-middle" to sniff your password.

Corporations, universities, governments, and Usenet resellers need more
complicated authentication methods.

Consider how many people now broadcast their account info, including
password, when they use WiFi without TLS/SSL.

The "man-in-the-middle" works nights somewhere, in an Internet hub, at
your ISP, for an NSP, for Google, Yahoo, Hotmail, AT&T, Verizon,
TimeWarner, ..., and steals data to sell it. Money is the root of all
evil, and they aren't satisfied with the low pay these companies give them.

Who could have guessed five years ago that NNTP was on its deathbed from
disuse, and from troll flames, and most computer users would buy a
laptop from BestBuy or OfficeDepot, and spend their entire online life
on Facebook? :-(

Things change, and even though there is a familiarity factor and love
for a certain historic software program, it is often time to let it take
it's last breath and move on, maybe focusing your precious time to help
on a similar software product that already is way ahead.

That is a choice that you have to make, of course. I somehow hope you
will keep plodding and fixing NN, if for no other reason than to satisfy
your own desire to use an old friend. I'm sure others will chime in with
similar views.

I'll do what I can to provide what I can, which is probably limited to
testing with INN.

> It adds an nntp-auth boolean flag to indicate whether or not to force
> authentication on connect.
>
> So, this will fix NN with servers that are configured the way that yours
> are, though I still maintain that you should change the response codes
> that your server is sending as its initial greeting, and that you should
> consider listing all the groups, not just one, to unauthenticated users;
> you don't have to give them read access, just let them know they are there.
>

INN is one huge set of programs, and I wouldn't know where to start.

I'm not the developer, but only an occasional submitter of bugs and
ideas, submitting a rare patch here and there.

Considering your patch, I don't know GitHub at all, and barely can make
my way around the trac system that INN developers use.

I can apply patches using the patch command, but that requires a diff
file, and is for source or Perl, Python, BASH, and other scripting
languages.

If what you are doing for a patch is writing a new source, I would need
to compile it. I hesitate to do this because my NN is a Debian package,
and as you suspect, is customized for Debian-based systems.

I suppose I could compile it and "make install" into /usr/local/bin
which would protect it from the Debian APT updates and upgrades, and
also protect the Debian version of NN.

But I'd need to worry about the data and config files created in /etc
and /home getting clobbered.

I could use a different computer (I have 65 or so here, wired to the LAN
and ready to go), but they are mostly all in the basement. I could and
usually do use SSH when working on them, but I think I'd just as soon
wait until the patched (new) NN is available for Debian (hint: submit it).

And I'm out of time. I have two big projects setting idle while I do
what I like to do, which is this NNTP stuff.

I'm disabled, so physical activity is rather difficult, and I'm really
slowing down (age is 67). That is why I use SSH to maintain headless
basement servers remotely from three floors above. No elevator.

I have plans and supplies sitting to insulate my two-car garage, but it
is too hot right now. That is what I said last summer!

My peripheral neuropathy (numb feet and legs) makes getting around
rather difficult. It's like living on stilts, and trying to go to sleep
with spiders crawling around on my legs.

I also have designed large shelving units for the basement for storing
so much collected junk, and need to keep pressing on with that project
before I croak, and my wife is stuck with it. This "junk" includes
everything my mother owned when she died in 2002, and that includes
everything her parents owned when they died in 1955-58. Both of them
were hoarders.

When people ask me, "how are you doing?" I always answer with, "I reckon
I'm doing about the bast I can."

That keeps me from lying. ;-)

Learned that from what Hoke Colburn (Morgan Freeman) told Boolie Werthan
(Dan Aykroyd) while they were working on the freight elevator at the
start of "Driving Miss Daisy."

Hope this newsgroup doesn't mind a little occasional OT gab. ;-)

>> That is correct, and I do understand you. NN is broken for use with a
>> news server that operates with authentication and has groups which
>> require no authentication.
>>
>
> More precisely 6.7.3 does not work as may be expected with a server
> that does not present the full groups list *and* does not send a 201
> code as its initial greeting. It *will* work as expected for a server
> that simply requires authentication for some groups but not others.
> Or that does one or the other of the above two troublesome things.
>

INN authenticates on initial connection, so later unauthorized access
"groups" are unknown at that time.

The initial connection authentication does authorize the connection to
certain groups, but you cannot go back and re-authenticate and get into
another authorized set of groups.

My readers.conf is pretty simple. You get * (wildmat regex for
everything) if you successfully authenticate.

If not you fall through to only the local.info access.


auth "global" {
hosts: "*"
auth: "/usr/lib/news/bin/auth/passwd/ckpasswd -f
/var/lib/news/passlist"
default: "<anonymous>"
}

auth "localhost" {
hosts: "stdin,localhost"
auth: "/usr/lib/news/bin/auth/passwd/ckpasswd -f
/var/lib/news/passlist"
default: "<localhost>"
}

access "authenticated" {
users: "*"
newsgroups: "*,!junk,!control,!control.*"
}

access "special" {
users: "<localhost>,newsmaster"
newsgroups: "*"
access: "RPNAL"
}

access "public" {
users: "<anonymous>"
read: "local.info"
}


For info on the above, please see:

http://www.eyrie.org/~eagle/software/inn/docs-2.5/readers.conf.html

The file is parsed down, then back up to the last match, if I remember
correctly.

>> This can be accomplished easily with a command line option such as TIN
>> uses. Or it can be stored in a configuration file that NN reads. Or both.
>>
>
> It can now be done with both in NN as well. :-)
>
>
>> So it is a stalemate, unless NN is patched to force
>> authentication or at least have it as an option when connecting, like
>> TIN does with the -A command option.
>>
>
> Done.
>

Thank you for your efforts, Aaron. I'll try and get this version running
on something soon.

Please submit to Debian where it will be available for Sid and maybe Wheezy.

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>If I run nn with no options, the process ends immediately.

This is because NN will check for new messages in subscribed groups,
and if there are none, it exits immediately.

>"No group matching `local test'" is what I receive here.

So NN is not running the command on the server, but just checking its
local list, thanks.

>The group exists but NN cannot see it because the server isn't providing
>groups that are protected by authentication.

I do not think this is the reason. If I understand your server configuration,
if NN had asked to read the group on the server, it would have sent a
480 error requesting authentication, and then NN would have been able to
access things. As is stands, though, it seems that NN does not actually
run the command until after it assures itself that the group exists based
on its local listing, which, in your server configuration, will not contain
the restricted groups.

--

[Aaron W. Hsu]

"John F. Morse" <jo...@example.invalid> writes:

>Historically there was little or no need to authenticate.

The important thing out of this discussion relating to authentication is
that there are many different points at which NNTP permits authentication
to happen, and some combinations of server settings can result in missed
information unless the client side preempts any explicit authentication
request (480's) on the server side.

Unlike a protocol like HTTP or SSH, authentication is not an "on/off"
switch in NNTP, and authentication does not occur as any part of the
initial handshake between client and server in NNTP. Rather, a client
can, on its own, and at any time during the course of an NNTP session
connection, send authentication information if it feels like it. The server,
on its side, has no means of asking for authentication information *before*
a command is completed, but instead, can only respond that actions
are not permitted unless authentication information not yet provided
is provided by the client. This requires more flexibility in the clients,
because some servers (your server for instance) may do different things
with authenticated and non-authenticated users, without ever indicating
this to the client; indeed, NNTP provides for no means of indicating such
things explicitly, but only allows for the server to reject a command
based on authentication reasons. In order to map over all combinations
of authentication patterns, therefore, it is requisite on the client to
provide a means for explicit authentication at any time during a connection,
including user specified points such as after the initial greeting, as
well as on demand after receiving a 480 response code.

Up until my recent patches, NN provided for on demand authentication,
as well as a special case at the beginning at the only time when the
server can indicate privileges in an explicit code (200 vs 201), but
did not provide for forced user authentication after the initial
handshake; it does now.

(The above is just a recap in case anyone searches the archives.)

>That is why I made the statement that NN was rather "old" (or however I
>worded it). You are using NN in the old school philosophy, on servers
>protected in different ways (known IP netblock access from a cable ISP
>for instance). And from NSPs that have only one set of authentication
>access path privilege.

I wouldn't call my use old school, just less dynamic than your configuration,
which, in its own is not really "new" since servers could have done this
sort of things for a long time now.

>That is a choice that you have to make, of course. I somehow hope you
>will keep plodding and fixing NN, if for no other reason than to satisfy
>your own desire to use an old friend. I'm sure others will chime in with
>similar views.

The reason that I continue to maintain and enhance NN, despite the existance
of hipper and newer alternatives (trn, tin, and slrn are all quite old
alternatives, actually, and tin is a contemporary of NN), is that the
reading workflow of NN is the workflow that allows me to read through my
messages with the most ease and the least active mental motion. Very
little clicking or keystrokes per message read and especially per message
*not* read are needed in NN, and that's really the main reason I stick
with it. It's a testament to how useful I find this approach that I am
willing to invest time into enhancing and maintaining the code, which is
something that I would not normally do in most cases.

>Considering your patch, I don't know GitHub at all, and barely can make
>my way around the trac system that INN developers use.

Sorry about that. :-) Using git(1) is probably more than you want to learn
to get the patch, but you can get a snapshop tarball of the code here:

https://github.com/arcfide/Unicode-NN/tarball/master

With that you can build a new NN from source.

>I can apply patches using the patch command, but that requires a diff
>file, and is for source or Perl, Python, BASH, and other scripting
>languages.

You can use patch on the NN source, and the git-format-patch(1) command
is what you would use to generate such a patch from the git source, but
that is all probably more than you want to do. It's easier to just grab
the new tarball.

>If what you are doing for a patch is writing a new source, I would need
>to compile it. I hesitate to do this because my NN is a Debian package,
>and as you suspect, is customized for Debian-based systems.

Indeed. You could probably get away with using their build scripts, but,
as you say, it's more work than you probably want to invest in something
like this.

>I think I'd just as soon wait until the patched (new) NN is available
>for Debian (hint: submit it).

I won't be submitting a new NN for a while yet, there are a couple more
things that I consider must-haves before I release a 6.8 release. These
are primarily:

* Unicode support for quoted-printable
* A unified and comprehensive collection of the documentation surrounding
NN in a convenient form

Neither of these takes little time, and I think it might be a while till
I can release these things. However, the tarball is always around, as above,
so people can get to it.

>INN authenticates on initial connection, so later unauthorized access
>"groups" are unknown at that time.

INN as a news server actually doesn't authenticate anything, IIUC. Rather,
your clients send authentication information on their own to INN before
doing anything else. I also fully expect your INN server to send 480
codes later in the connection, and a client that successfully authenticates
later in the connection should be able to reget the active file, for
instance, with the new listing.

>The initial connection authentication does authorize the connection to
>certain groups, but you cannot go back and re-authenticate and get into
>another authorized set of groups.

You should be able to according to the NNTP protocol. That is, if I run a
LIST as an unauthenticated user, and then I authenticate, and run LIST
again, I should get a different result.

>Thank you for your efforts, Aaron. I'll try and get this version running
>on something soon.

>Please submit to Debian where it will be available for Sid and maybe Wheezy.

There is still more work to be done for NN before sending it out to these
other distros, unfortunately. 6.7.3 is likely to stick around for a while,
yet.

[John F. Morse]

Aaron W. Hsu wrote:
> "John F. Morse" <jo...@example.invalid> writes:
>> If I run nn with no options, the process ends immediately.
>>
>
> This is because NN will check for new messages in subscribed groups,
> and if there are none, it exits immediately.
>

It may check, but it can't see beyond the single group that doesn't
require authentication.

The single local.info article would have been seen before, so that may
be why it bails out.

If I use the nn command with the name of my NSP, then it does enter a group.

If I put "set nntp-server news.my.isp.net" in ~/.nn.init then it also
enter the group.

So I can assume this behavior is expected.

>> "No group matching `local test'" is what I receive here.
>>
>
> So NN is not running the command on the server, but just checking its
> local list, thanks.
>

It should be looking at the server.

But when it does, it's only going to see the single local.info group
because that is the only group which it can access.

>> The group exists but NN cannot see it because the server isn't providing
>> groups that are protected by authentication.
>>
>
> I do not think this is the reason. If I understand your server configuration,
> if NN had asked to read the group on the server, it would have sent a
> 480 error requesting authentication, and then NN would have been able to
> access things. As is stands, though, it seems that NN does not actually
> run the command until after it assures itself that the group exists based
> on its local listing, which, in your server configuration, will not contain
> the restricted groups.
>

It isn't going to send 480 because it hasn't been afforded the privilege
of looking beyond the single group (local.info) that the connection allows.

The INN nnrpd server, using the readers.config file, gets the authorized
groups when a user connects to the nnrpd server. Not later.

You cannot backtrack on that connection, and further connection attempts
always go through the same nnrpd validation path.

The server doesn't ask for authentication. The newsreader must supply
("force") the authinfo when it first connects.

I cannot change that.

If you can understand the readers.conf man page, and it is quite
difficult, then you would see how it works. You might consider posting
in news.software.nntp and one of the two or three expert developers
might respond.

It was discussed a long time ago, and that is why TIN was patched. I
have no other newsreader that cannot "push" authentication. They all
work with INN. Only NN is unable (but you have now fixed it).

[John F. Morse]

Aaron W. Hsu wrote:
> "John F. Morse" <jo...@example.invalid> writes:
>>
>
> The reason that I continue to maintain and enhance NN, despite the existance
> of hipper and newer alternatives (trn, tin, and slrn are all quite old
> alternatives, actually, and tin is a contemporary of NN), is that the
> reading workflow of NN is the workflow that allows me to read through my
> messages with the most ease and the least active mental motion. Very
> little clicking or keystrokes per message read and especially per message
> *not* read are needed in NN, and that's really the main reason I stick
> with it. It's a testament to how useful I find this approach that I am
> willing to invest time into enhancing and maintaining the code, which is
> something that I would not normally do in most cases.
>

Understood. It's one of those personal preferences and familiarization
issues.

> I won't be submitting a new NN for a while yet, there are a couple more
> things that I consider must-haves before I release a 6.8 release. These
> are primarily:
>
> * Unicode support for quoted-printable
> * A unified and comprehensive collection of the documentation surrounding
> NN in a convenient form
>
> Neither of these takes little time, and I think it might be a while till
> I can release these things. However, the tarball is always around, as above,
> so people can get to it.
>

Understood. It will take me some time to even compile the patched NN you
supplied.

Also note the ChangeLog in that tarball, lines 855-857:

From: Tim <t...@sleepy.wojomedia.com>
nntp-user and nntp-password should be settable in the init file.
Fixed.


This now works after I modified the readers.conf on a test server, and
could then be asked for the authinfo data.

I was also able to change the default editor from vi to nano with the
":set editor nano" command. I made it permanent by putting the entry in
the ~/.nn/init file.

> INN as a news server actually doesn't authenticate anything, IIUC. Rather,
> your clients send authentication information on their own to INN before
> doing anything else. I also fully expect your INN server to send 480
> codes later in the connection, and a client that successfully authenticates
> later in the connection should be able to reget the active file, for
> instance, with the new listing.
>

Clients do send authentication information, or requests, but the server
is actually authenticating them.

You might think of this authenticating as checking for access
privileges. If found, then the server authenticates them ("makes them
authenticated").

It is very complicated as I mentioned earlier. Please read the
readers.conf(5) man page, especially the Summary and the Examples sections.

http://www.eyrie.org/~eagle/software/inn/docs/readers.conf.html

Older version: http://linux.die.net/man/5/readers.conf

>> The initial connection authentication does authorize the connection to
>> certain groups, but you cannot go back and re-authenticate and get into
>> another authorized set of groups.
>>
>
> You should be able to according to the NNTP protocol. That is, if I run a
> LIST as an unauthenticated user, and then I authenticate, and run LIST
> again, I should get a different result.
>

Are you using the latest NNTP RFCs?

http://asg.web.cmu.edu/rfc/rfc4643.html#sec-2

http://asg.web.cmu.edu/rfc/rfc4643.html#sec-2.2

What about NNRP, which is actually doing the authentication checks, and
setting the flags when a client does gain access privileges?
http://en.wikipedia.org/wiki/NNRP#Network_News_Reader_Protocol


200 news102.my.net InterNetNews NNRP server INN 2.5.2 ready (no posting)
CAPABILITIES
101 Capability list:
VERSION 2
IMPLEMENTATION INN 2.5.2
AUTHINFO USER SASL
HDR
LIST ACTIVE ACTIVE.TIMES DISTRIB.PATS HEADERS NEWSGROUPS OVERVIEW.FMT
NEWNEWS
OVER
READER
SASL CRAM-MD5 DIGEST-MD5 LOGIN PLAIN NTLM
STARTTLS
.


I've sent this message to the current INN maintainer, and we'll see if
he can clear the issue so we both can be on the same page.

His job involves foreign travel away from his home in France, so it may
be a few days, or even weeks.

When I hear from him, I'll let you know.

I might add what I discovered tonight while testing NN with the modified
test server.

I edited the news server's /etc/news/readers.conf file to allow only
authenticated users. Basically by commenting out the stanzas associated
with a fall-through into the unauthenticated access stanza providing
access to only the local.info group.

When I connected with NN, I was asked for a username and then password,
and I then had full access to all of the groups.

That proves NN will work for a server that doesn't have a combination of
open and closed groups, where the unauthenticated connections fall
through and can only access the open groups.

Here is the /var/log/news/news.notice log for the NNRP daemon using PID
26489 for the test message:


Aug 8 01:56:54 news5 nnrpd[26489]: hardy.my.net (192.168.30.10) connect
- port 119
Aug 8 01:57:03 news5 nnrpd[26489]: hardy.my.net user john
Aug 8 01:57:10 news5 nnrpd[26489]: hardy.my.net group local.test 0
Aug 8 02:03:08 news5 nnrpd[26489]: SERVER perl filtering enabled
Aug 8 02:03:09 news5 nnrpd[26489]: hardy.my.net post ok
<jvt2vc$prp$1...@news5.my.net>
Aug 8 02:04:29 news5 nnrpd[26489]: hardy.my.net group local.test 1
Aug 8 02:04:29 news5 nnrpd[26489]: hardy.my.net exit articles 1 groups 2
Aug 8 02:04:29 news5 nnrpd[26489]: hardy.my.net posts received 1 rejected 0
Aug 8 02:04:29 news5 nnrpd[26489]: hardy.my.net times user 2.484 system
1.604 idle 0.029 elapsed 455.987
Aug 8 02:04:29 news5 nnrpd[26489]: hardy.my.net artstats get 1 time 0
size 839
Aug 8 02:04:29 news5 nnrpd[26489]: hardy.my.net time 456141 idle
451217(13) readart 0(1) nntpwrite 1743(61644)


You will notice the second line is where the authentication was
accepted. Then the group command was accepted. I posted one test message
which was filtered by the Cleanfeed Perl filter, the post was OKed and a
M-ID assigned.

Then it was handed off to innd, and stats were noted.

The innd server accepted and wrote the article to the spool, then caused
innfeed to send it to all of the peers. From /var/log/news/news :


Aug 8 02:03:09.000 + localhost <jvt2vc$prp$1...@news5.my.net> 792 news11
news102 inpaths!


The above shows the date and time the article arrived at the innd (INN
daemon), the "+" means it was accepted, and "localhost" was from where.
Then the M-ID, followed by 792, the size in bytes. It then was sent out
by innfeed to news11, news102, and to a file :inpaths" for later
processing and mailing to the Top1000 site.

The news102 is the main reader server. The news11 is a filtering server
that hands off articles to news2, which is a backup transit server. Then
the article goes to the main transit server which sends the article out
to multiple peers around the world. However, in this case, since this
article was posted in a "local" group, the transit server will not
propagate the article into Usenet.

Here is what innfeed logged in /var/log/news/news.notice :


Aug 8 02:03:09 news5 innfeed[17439]: news11:0 connected
Aug 8 02:03:09 news5 innfeed[17439]: news11 remote MODE STREAM
Aug 8 02:12:43 news5 innfeed[17439]: news102 checkpoint seconds 600
offered 513 accepted 248 refused 0 rejected 265 missing 0 accsize 662424
rejsize 2725196 spooled 0 on_close 0 unspooled 0 deferred 0/0.0 requeued
0 queue 0.0/200:97,3,0,0,0,0
Aug 8 02:13:09 news5 innfeed[17439]: news11 checkpoint seconds 600
offered 1 accepted 1 refused 0 rejected 0 missing 0 accsize 795 rejsize
0 spooled 0 on_close 0 unspooled 0 deferred 0/0.0 requeued 0 queue
0.0/200:100,0,0,0,0,0
Aug 8 02:13:09 news5 innfeed[17439]: news11:0 idle tearing down connection
Aug 8 02:13:09 news5 innfeed[17439]: news11:0 checkpoint seconds 600
offered 1 accepted 1 refused 0 rejected 0 accsize 795 rejsize 0
Aug 8 02:13:09 news5 innfeed[17439]: news11:0 final seconds 600 offered
1 accepted 1 refused 0 rejected 0 accsize 795 rejsize 0
Aug 8 02:13:09 news5 innfeed[17439]: news11 checkpoint seconds 0
offered 0 accepted 0 refused 0 rejected 0 missing 0 accsize 0 rejsize 0
spooled 0 on_close 0 unspooled 0 deferred 0/0.0 requeued 0 queue
0.0/200:100,0,0,0,0,0
Aug 8 02:13:10 news5 innfeed[17439]: news11 final seconds 600 offered 1
accepted 1 refused 0 rejected 0 missing 0 accsize 795 rejsize 0 spooled
0 on_close 0 unspooled 0 deferred 0/0.0 requeued 0 queue
0.0/200:100,0,0,0,0,0


I will wait until the INN developer responds with something.

Again, thanks for all the assistance you've provided.

[smw]

In <RsidnW7iOaxirIPN...@giganews.com> Aaron W. Hsu <arc...@sacrideo.us> writes:

>Well, for "potential" users like John F. Morse,

...and for existing real users. Yes, I'm one of them, and this is where
I thank you for this and everything else you've done with nn.

>I have just pushed patches to my GitHub version of NN to support the
>forcing of NN to authenticate itself whether or not the server requires it
>or not.

I apologize for not thinking to share it at the time, but I did something
similar with my local copy of nn in 2014.

I'd just started using news.eternal-september.org as my news server,
and discovered its habit of returning 200 even if no authentication
is attempted, but in that case it grants access only to its private
newsgroups.

- Steven
--
___________________________________________________________________________
Steven Winikoff | The consultant's 3 rules of crisis management:
Montreal, QC, Canada | 1) When life hands you a lemon, make lemonade.
s...@smwonline.ca | 2) When life hands you a hemlock, don't make
http://smwonline.ca | hemlock-ade.
| 2a) Always know the difference between a lemon
| and a hemlock.
| - Rick Cook