>Aaron W. Hsu wrote:
>> 1: Connect to the server and see whether you can post
>The problem right here is without proper authentication, the news server >restricts unauthenticated connects to only one group (on my servers), or >however many groups an administrator may assign for connections where >authentication fails.
[...]
>> 2: If you can post, your fine, start reading
>> 3: If you cannot post, you might want to post, so try an authentication
>> 4: If you need authentication to read, then try an authentication
>Here, NN is just not prompting for authentication, or is not providing >it after receiving a 200 reply.
If you read the above steps that NN takes when doing an initial server negotiation you would see why it is not authenticating. After you detailed the above and below, it is now clear to me, I think, why you are seeing what you are. In particular, let's look at these logs...
>Without being authenticated, any access command is going to receive a >480 reply.
>Here is a sample telnet session (with munged areas):
>john@hardy:~$ telnet news5 119
>Trying 192.168.33.5...
>Connected to news5.my.net.
>Escape character is '^]'.
>200 news5.my.net InterNetNews NNRP server INN 2.5.2 ready (no posting)
>group local.test
>480 Read access denied
>authinfo user {my_username}
>381 Enter password
>authinfo pass {my_password}
>281 Authentication succeeded
>Here I tried to access the local.test group, but it requires >authentication, so the server gave me the 480 error.
If you had tried to do the same thing with NN, it would have also asked you for authentication.
>Now if I connect without authentication, the only group I can access is >the local.info group:
>john@hardy:~$ telnet news5 119
>Trying 192.168.33.5...
>Connected to news5.my.net.
>Escape character is '^]'.
>200 news5.my.net InterNetNews NNRP server INN 2.5.2 ready (no posting)
>group local.info
>211 1 3 3 local.info
>list
>215 Newsgroups in form "group high low status"
>local.info 0000000003 0000000003 m
>I can use the article command with 3 and read that article.
Seeing the above makes it obvious why NN is not asking for authentication. Firstly, when it first connects, the message that it is receiving is a 200 code, which indicates that posting is allowed. Thus, NN, does not even try to do any initial authentication, because it does not need to do any right then and their. Instead, if you did not specify a specific group, it will just run the LIST command, which I am guessing does not require authentication either on your server. In this case, NN happily continues, because the server has not requested authentication.
If you had instead told NN to do something that required authentication, then it would have asked for it. If the server had responded with a 201, indicating that posting was not allowed, then NN would have tried to authenticate, to see if authenticating would give it more privileges. Since the server did not respond so, and everything that NN tried was okay without authentication, NN did not ask you for any authentication details, nor use those that you provided.
>> There are notes in the source code suggesting that it might be nice to >> have a variable to control whether or not authentication is performed, >> but my guess is that it is not done so right now.
>It should be provided on a server basis. Some servers require >authentication, and some don't.
You misread or misunderstand me: NN does authentication on a per command basis, not per server; you seem to think that NN does authentication at some coarser granularity than either of these, NN is just being more precise about authentication than what you expect. It is commands that
require authentication. NN will not give authentication unless it is
required to do so by the server, or if it is restricted in a way that it
thinks it can overcome by authenticating; namely, it will try to authenticate when it receives a 480 reply, or when the initial message gives it a 201 code. Your use of NN is not generating either of these messages, so NN is not authenticating.
The note I have above is saying that the coders of NN *know* that there is no configurability (say, in the init file) of whether to blindly authenticate at log in, even if the server gives a 200 instead of 201 as its opening message, and they acknowledge that having a variable to set this might be nice. However, for the vast majority of installations, this is not a problem, even today. It is a problem for you because you actually change information that you send based on authentication information, but you don't do anything to indicate this. Your server does not give a 201 code, and I am guessing that the LIST command works without authentication, so NN has no way of knowing that it could have "gotten more groups" if only it had authenticated.
>> It shouldn't, but depending on the configuration of NN, it might prevent >> logging in. Specifically, in Slackware, I had to make it use the hostname >> file because NN expects an FQDN when getting the hostname, which >> the hostname(3) was not giving it. It does not sound like this is a problem >> for you, but it's something to note just in case.
>Well, it could be if it is associated with the problem of using my real >e-mail address. That would bring on the spammers for sure.
That problem is a problem with you not using a custom From field, and it has nothing to do with your hostname. NN is using a default address based on your hostname because you have not overridden the From field in your configuration, but it would do this regardless of the hostname information it receives. Thus, I am certain that this is not your problem in any of the gripes you have with NN right now.
>IIRC, I could only get NN to access the server that is listed in the >~/.nn/init file.
>Perhaps there is another configuration value that would put one file >above another, but I think I've tried them all.
If there is an NNTPSERVER variable, NN should use that; if not, it should use the value on the command-line; and if not, then it should use the init file value.
>>> I have /etc/news/server but it is for SLRN, and points to localhost for >>> Stunnel support, and for a different local INN reader server which >>> requires TLS on port 563.
>> This may or may not be causing NN to barf, but I would be suprised if it >> is assuming that you are properly specifying the NNTP server. I'd have to >> check the code.
>This is not used for NN. It is for SLRN, and maybe Tin, if I don't >override it with a launch commend parameter.
You seemed to think that it was grabbing a local spool rather than using the NNTP server that you specified. That's how I read this part of your message. If that is not the case, then we can dispense with this part entirely.
>Please review the authentication info above. Basically, without >authentication, there is no access except to one group (local.info), and >posting is prohibited anywhere.
But your initial log in server response gives a 200 reply instead of a 201, which in the protocol indicates that posting *is* allowed. As I mentioned above, NN will not try to authenticate initially if it does not receive a 201 on login. It *will* try to authenticate later on if a command it tries gets a 480 reply.
>NN cannot know whether a news server requires authentication unless the >server asks (none do AFAIK), or NN has a configuration file setting >(which we are looking for and not finding).
All servers that I have ever used, including GigaNews, Gmane, and my ISP, among others, either work without passwords, or request a password explicitly. For example, with GigaNews, here's what I get from a Telnet session:
Trying 216.196.97.131...
Connected to news.giganews.com.
Escape character is '^]'.
200 News.GigaNews.Com
MODE READER
200 reading enabled
LIST
480 authentication required
The above simulates what NN does on logging into the server. NN sees the 480 reply and then asks me to authenticate. The problem you are experiencing is that you give different results for a command such as LIST depending on authentication rather than just returning 480 on the command, as most servers that require authentication would.
>Some cases allow authentication by putting username:password on the >command line as an attribute. I do not find it works with NN, and don't >recollect reading it in the various documentation.
If you have a version of NN with nntp-user and nntp-password variables, then you can set them explicitly on the command line:
nn nntp-user={username} nntp-password={password} ...
>In fact, the word "authentication" is not even in the docs.
That's because in most versions of NN, this was always done interactively, it is only in relatively recent NN versions that nntp-user and nntp-password were supported, and the vanilla source of 6.7.3 only has that in the README, and does not document it in the man page.
>The nn(1) man page is where I read this, and it works.
It is obvious that you are using a patched version of NN, since the 6.7.3 version that I have does not have this in the man page, but instead explicitly recommends against putting nntp-server in the init file, and recommends always putting it in the command-line. However, while the man page says this, I think that some patches were introduced that caused the init file to be loaded earlier, so that you can get away with using nntp-server in the init file, which is what you have been doing, apparently.
I have just pushed patches to my GitHub version of NN to support the forcing of NN to authenticate itself whether or not the server requires it or not. Normally, NN tries to authenticate only if it receives a 201 response code (it want to see if it can get a 200 posting allowed code)
or if it receives a 480 response during normal use. The latest master branch of my NN development trunk enables an option nntp-auth, which is boolean. When set, NN will always try to authenticate, and when not set, it will use the traditional authenticate on demand approach.
Aaron W. Hsu wrote:
> "John F. Morse" <j...@example.invalid> writes:
>> Aaron W. Hsu wrote:
>>> 1: Connect to the server and see whether you can post
>> The problem right here is without proper authentication, the news server >> restricts unauthenticated connects to only one group (on my servers), or >> however many groups an administrator may assign for connections where >> authentication fails.
> [...]
>>> 2: If you can post, your fine, start reading
>>> 3: If you cannot post, you might want to post, so try an authentication
>>> 4: If you need authentication to read, then try an authentication
>> Here, NN is just not prompting for authentication, or is not providing >> it after receiving a 200 reply.
> If you read the above steps that NN takes when doing an initial server > negotiation you would see why it is not authenticating. After you detailed > the above and below, it is now clear to me, I think, why you are seeing > what you are. In particular, let's look at these logs...
>> Without being authenticated, any access command is going to receive a >> 480 reply.
>> Here is a sample telnet session (with munged areas):
>> john@hardy:~$ telnet news5 119
>> Trying 192.168.33.5...
>> Connected to news5.my.net.
>> Escape character is '^]'.
>> 200 news5.my.net InterNetNews NNRP server INN 2.5.2 ready (no posting)
>> group local.test
>> 480 Read access denied
>> authinfo user {my_username}
>> 381 Enter password
>> authinfo pass {my_password}
>> 281 Authentication succeeded
>> Here I tried to access the local.test group, but it requires >> authentication, so the server gave me the 480 error.
> If you had tried to do the same thing with NN, it would have also asked > you for authentication.
I cannot do the same thing with NN, such as send the "authinfo" command, which is easy with telnet. Therefore I'm only presented the group that doesn't require authentication.
The 480 error is actually designed for a transit server:
480 transfer permission denied. Response to CHECK if transfer is not allowed.
>> Now if I connect without authentication, the only group I can access is >> the local.info group:
>> john@hardy:~$ telnet news5 119
>> Trying 192.168.33.5...
>> Connected to news5.my.net.
>> Escape character is '^]'.
>> 200 news5.my.net InterNetNews NNRP server INN 2.5.2 ready (no posting)
>> group local.info
>> 211 1 3 3 local.info
>> list
>> 215 Newsgroups in form "group high low status"
>> local.info 0000000003 0000000003 m
>> I can use the article command with 3 and read that article.
> Seeing the above makes it obvious why NN is not asking for authentication. > Firstly, when it first connects, the message that it is receiving is a > 200 code, which indicates that posting is allowed. Thus, NN, does not > even try to do any initial authentication, because it does not need to > do any right then and their. Instead, if you did not specify a specific > group, it will just run the LIST command, which I am guessing does not > require authentication either on your server. In this case, NN happily > continues, because the server has not requested authentication.
200 server ready - posting allowed. Sent by the server upon initiation of the session, if the client is allowed to post messages.
In the case of a read-only newsgroup, the client is not allowed to post, but that is beyond the initial connect stage. Attempts to post will be blocked when the article post command is given. Perhaps one of these?:
440 posting not allowed. POST command issued when posting is not allowed.
502 access restriction or permission denied. Permission denied; sent if the client has not properly authenticated but the server requires it.
> If you had instead told NN to do something that required authentication, > then it would have asked for it. If the server had responded with a 201, > indicating that posting was not allowed, then NN would have tried to > authenticate, to see if authenticating would give it more privileges. > Since the server did not respond so, and everything that NN tried was > okay without authentication, NN did not ask you for any authentication > details, nor use those that you provided.
I think your problem is you believe "posting" comes before "reading" -- or in this case "accessing."
Nobody can post if they can't get there in the first place. If they can get there, but posting in that group is prohibited (a "read-only" group), then there is no 201 issued.
201 server ready - no posting allowed. Sent by the server upon initiation of the session, if the client is not allowed to post messages.
Getting in the "posting door" requires authentication, as well as access into a group that allows posting or is moderated. If you can't get in the door, then the server has no idea who you are, nor if you are allowed to post.
It does allow you to read any group that is assigned anonymous access privileges though. Since you are connected as an anonymous reader, you remain that until the connection is terminated.
The initial 200 code is all NN knows, so it will not authenticate. It must be forced with a command option so the user can use it when needed for those servers that do require authentication.
Again, you and I see the problem is NN won't ask for authentication if a server has issued the 200 response code.
The reason the 200 is sent by INN (probably other NNTP/NNRP servers also) is there are one or more newsgroups available for unauthenticated connections. This is how I provide anyone access to a limited number of groups (actually only one) so they can obtain some information on how to apply for an account.
Now when someone who has an account tries to connect, and their newsreader authenticates them, they are not asked again to authenticate because the 200 response code was sent. They can access whatever privileges correspond to their account as configured in the /etc/news/readers.conf file.
>>> There are notes in the source code suggesting that it might be nice to >>> have a variable to control whether or not authentication is performed, >>> but my guess is that it is not done so right now.
I discovered this problem a few years ago when using TIN. Urs Jan�en, the TIN developer, who is active in the news.software.readers group (where this discussion really should be taking place), added the -A command option to "force authentication on initial connect." That fixed TIN.
The -A was added to TIN in 1998, which was 12 years ago!
TIN pre-1.4 release 19981002 ("Phobia"):
ADD. -A cmd-line switch to force auth on connect
TIN pre-1.4 release 19990927 ("Nine While Nine"):
BUG. authentication was not first command after MODE READER if invoked with -A
TIN version 1.5.2 release 20000206 ("Black Planet"):
ADD. allow interactive authentication on startup if invoked with -A
TIN version 1.9.5 20091224 ("Lochruan"):
ADD. try AUTHINFO USER/PASS on -A if CAPABILITIES didn't announce any supported auth methods
I am using TIN version 1.9.2 20070201 ("Dalaruan"). The -A works fine.
I'm checking with Julien Elie, the current INN developer, about this problem when there are one or more groups which are accessible without authentication, and the server is providing the 200 result code.
However this does sound like a normal and reasonable result from a news server under the conditions mentioned.
>> It should be provided on a server basis. Some servers require >> authentication, and some don't.
> You misread or misunderstand me: NN does authentication on a per command > basis, not per server; you seem to think that NN does authentication at > some coarser granularity than either of these, NN is just being more > precise about authentication than what you expect. It is commands that
> require authentication. NN will not give authentication unless it is
> required to do so by the server, or if it is restricted in a way that it
> thinks it can overcome by authenticating; namely, it will try to authenticate > when it receives a 480 reply, or when the initial message gives it a 201 > code. Your use of NN is not generating either of these messages, so NN is > not authenticating.
That is correct, and I do understand you. NN is broken for use with a news server that operates with authentication and has groups which require no authentication.
What I meant by "per-server basis" is some servers require authentication, and some do not. Therefore the user must be able to control NN to authenticate or not.
This can be accomplished easily with a command line option such as TIN uses. Or it can be stored in a configuration file that NN reads. Or both.
> The note I have above is saying that the coders of NN *know* that there > is no configurability (say, in the init file) of whether to blindly > authenticate at log in, even if the server gives a 200 instead of 201 > as its opening message, and they acknowledge that having a variable to > set this might be nice. However, for the vast majority of installations, > this is not a problem, even today. It is a problem for you because you > actually change information that you send based on authentication > information, but you don't do anything to indicate this. Your server > does not give a 201 code, and I am guessing that the LIST command works > without authentication, so NN has no way of knowing that it could have > "gotten more groups" if only it had authenticated.
There are a lot of INN servers in use on Usenet. More than
...
I admire you guys attention to detail and using the commandline
interface for news. Thunderbird would greatly simplify your life
reading and posting news, thus making it no fun.
-- Steal a little and go to jail, steal a lot and become King.
> I admire you guys attention to detail and using the commandline
> interface for news. Thunderbird would greatly simplify your life
> reading and posting news, thus making it no fun.
How?
So I'd have to constantly take my fingers off the keyboard to use the mouse?
I've used Pine for sixteen years, it was August 1996 that I first used it, and when I finally got around to running Linux, I cast Debian aside for Slackware since Debian didn't include Pine.
I can't see the point of doing graphics from the command line, unless I had a lot of standard size pictures that needed to be cropped or whatever in a standard fashion. There, a mouse makes sense since it is good for manipulating things.
But I can't see how using a mouse would help with newsgroups. My fingers are already on the keys, so it's much easier to use cursor keys than the mouse to move around.
Even if a GUI made sense, the fact that some of us have used the same newsreader for over a decade is not to be denied, it's familiar and why bother changing?
>> I admire you guys attention to detail and using the commandline
>> interface for news. Thunderbird would greatly simplify your life
>> reading and posting news, thus making it no fun.
> How?
> So I'd have to constantly take my fingers off the keyboard to use the
> mouse?
> I've used Pine for sixteen years, it was August 1996 that I first used
> it, and when I finally got around to running Linux, I cast Debian aside
> for Slackware since Debian didn't include Pine.
> I can't see the point of doing graphics from the command line, unless I
> had a lot of standard size pictures that needed to be cropped or
> whatever in a standard fashion. There, a mouse makes sense since it is
> good for manipulating things.
> But I can't see how using a mouse would help with newsgroups. My
> fingers are already on the keys, so it's much easier to use cursor keys
> than the mouse to move around.
> Even if a GUI made sense, the fact that some of us have used the same
> newsreader for over a decade is not to be denied, it's familiar and why
> bother changing?
> Michael
indeed
-- Steal a little and go to jail, steal a lot and become King.
>If you don't mind, and you wish to reply, could you trim this too-long >article and get rid of anything that isn't really necessary? It's grown >way out of hand, and a lot is due to my replying to each paragraph, and >leaving the quoted text intact.
Yep, I am definitely doing that here, since I think we got most of the issues solved or at least identified.
>If you think it would be better, since this doesn't really pertain to >Slackware, move any reply to n.s.nn where I am also subscribed, and I'll >reply there if need be.
Yep, I agree, and I am moving it over there. I'll be responding shortly, and I just wanted to put this one here, for the archives.
-- Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us Programming is just another word for the lost art of thinking.
>I don't see it spelled out in nn(1), but I may have inferred it from the >following in nn(1):
>33- This will be done very quickly, because nn uses the NOV
>database via
>34: the NNTP XOVER command. The news server to use can be >overridden by
>35- setting the environment variable $NNTPSERVER to the name of >the system
>36: (such as news.newserver.com), or by setting the variable >nntp-server
>37- (on the command line only, since it is looked at before the >init file),
>38: as "nntp-server=news.some.domain").
>Where is the news server name that you are overriding, mentioned above >in line 34?
When NN is initially configured at compile time, the config file can be used to specify a default NNTP server based on either a static name (more favored in the old days), or a filename, or the like. In the Slackware configuration it grabs the NNTP server name from /etc/nntpserver if nothing overrides it, I think. Other installations will likely have a slightly different set up, or may have patched away the default server name entirely.
-- Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us Programming is just another word for the lost art of thinking.
I have not looked through the code in depth, but the documentation and what I can search for seems to indicate that NN does not do anything explicit with message-ids, they appear to be generated server side; certainly, there seems to be no way of customizing the message-id.
-- Aaron W. Hsu | arcf...@sacrideo.us | http://www.sacrideo.us Programming is just another word for the lost art of thinking.
> I admire you guys attention to detail and using the commandline
> interface for news. Thunderbird would greatly simplify your life
> reading and posting news, thus making it no fun.
I am a simple man, with a simple life, and am using Thunderbird.
However, like you say, it's not much fun.
-- John
When a person has -- whether they knew it or not -- already
rejected the Truth, by what means do they discern a lie?
Michael Black wrote:
> On Mon, 6 Aug 2012, telsar wrote:
>> I admire you guys attention to detail and using the commandline
>> interface for news. Thunderbird would greatly simplify your life
>> reading and posting news, thus making it no fun.
> How?
> So I'd have to constantly take my fingers off the keyboard to use the > mouse?
Very true!
> I've used Pine for sixteen years, it was August 1996 that I first used > it, and when I finally got around to running Linux, I cast Debian > aside for Slackware since Debian didn't include Pine.
Debian has Alpine. Have you ever tried it? If so, what's your opinion, your comparison, etc.
> I can't see the point of doing graphics from the command line, unless > I had a lot of standard size pictures that needed to be cropped or > whatever in a standard fashion. There, a mouse makes sense since it > is good for manipulating things.
For modifying image properties, here is a great command line program:
> But I can't see how using a mouse would help with newsgroups. My > fingers are already on the keys, so it's much easier to use cursor > keys than the mouse to move around.
> Even if a GUI made sense, the fact that some of us have used the same > newsreader for over a decade is not to be denied, it's familiar and > why bother changing?
> Michael
Wouldn't that be like always drinking *everything* from your favorite coffee cup? ;-)
I know how familiarity can enslave people. Just look at the numbers of griping, bellyaching Windows users, who won't take the plunge and move to Linux.
Maybe there are just too many choices of Linux (over 700), like too many newsreaders? If it is choices overload, I don't have the answer for "them" but I know I get bored with repetition. I like experimentation. Plus I need different newsreaders for testing news servers.
Since I have your attention, can you give me a brief HOWTO for using Pine/Alpine/Mutt as a newsreader? I've read these e-mail programs can be used for netnews, but haven't yet stumbled upon any documentation.
If you've been following the NN discussion, you may have seen where I need a newsreader that can authenticate on connect, and can be used on multiple news servers (not necessary more than one at the same time though).
Configuration files which need command line editing are fine. Command options to override a config file are a plus.
Ideas?
-- John
When a person has -- whether they knew it or not -- already
rejected the Truth, by what means do they discern a lie?
On 2012-08-06, John F. Morse <j...@example.invalid> wrote:
> If you've been following the NN discussion, you may have seen where I > need a newsreader that can authenticate on connect, and can be used on > multiple news servers (not necessary more than one at the same time though).
> Configuration files which need command line editing are fine. Command > options to override a config file are a plus.
> Ideas?
I don't recall if you've tried slrn, but it would do everything you
listed above, John.
Another option would be to use slrnpull or leafnode or any other spooler
program to download newsgroup articles into a local cache and use
whatever newsreader you prefer to read them from the cache. I know
slrnpull allows you to post replies to articles, even to news servers
that require authentication (this reply is a demonstration of that), and I suspect other caching programs would allow it as well.
-- Chick Tower
For e-mail: aols2 DOT sent DOT towerboy AT xoxy DOT net
> On 2012-08-06, John F. Morse <j...@example.invalid> wrote:
>> If you've been following the NN discussion, you may have seen where I >> need a newsreader that can authenticate on connect, and can be used on >> multiple news servers (not necessary more than one at the same time though).
>> Configuration files which need command line editing are fine. Command >> options to override a config file are a plus.
>> Ideas?
> I don't recall if you've tried slrn, but it would do everything you
> listed above, John.
> Another option would be to use slrnpull or leafnode or any other spooler
> program to download newsgroup articles into a local cache and use
> whatever newsreader you prefer to read them from the cache. I know
> slrnpull allows you to post replies to articles, even to news servers
> that require authentication (this reply is a demonstration of that), and > I suspect other caching programs would allow it as well.
Hi Chuck,
Yes, I have SLRN and use it occasionally. like right now.
SLRN is a good newsreader. I do prefer TIN though because I can navigate
anywhere and read by using only the left and right arrow keys. My brain
is getting stiff and there is less to remember. ;-)
I run my own news servers so "local" could mean this localhost, or a
server in the basement, which could be running a CLI newsreader and I
can access it via SSH, which I've done in the past using TIN (or
sometimes mailx or mutt).
Getting the real news spool from an INN news server to my local console
computer could easily be accomplished with rsync, but I don't care to
copy 150 GB to the local hard drive! The spool is in a CNFS (Cyclic
News File System) where many large 2 GB file buffers are used, and news
articles fill them in a round-robin method so they never run out of
disk space. IOW, I couldn't pick and choose individual articles.
I do have the option to reconfigure reader access to allow my IP or LAN
netblock to access groups without authentication. But that would defeat
my need for testing in a real world scenario.
Anyway, thanks for your ideas.
-- John
When a person has -- whether they knew it or not -- already
rejected the Truth, by what means do they discern a lie?
On 2012-08-07, John F. Morse <j...@example.invalid> wrote:
> Getting the real news spool from an INN news server to my local console
> computer could easily be accomplished with rsync, but I don't care to
> copy 150 GB to the local hard drive! The spool is in a CNFS (Cyclic
> News File System) where many large 2 GB file buffers are used, and news
> articles fill them in a round-robin method so they never run out of
> disk space. IOW, I couldn't pick and choose individual articles.
Are you saying you have a local news article cache of 150GB? Do you
actually read all the newsgroups in that cache? slrnpull only downloads
the newsgroups you specify, and you can specify expiration periods so
that you can occasionally purge old messages.
-- Chick Tower
For e-mail: aols2 DOT sent DOT towerboy AT xoxy DOT net
Chick Tower wrote:
> On 2012-08-07, John F. Morse <j...@example.invalid> wrote:
>> Getting the real news spool from an INN news server to my local console
>> computer could easily be accomplished with rsync, but I don't care to
>> copy 150 GB to the local hard drive! The spool is in a CNFS (Cyclic
>> News File System) where many large 2 GB file buffers are used, and news
>> articles fill them in a round-robin method so they never run out of
>> disk space. IOW, I couldn't pick and choose individual articles.
> Are you saying you have a local news article cache of 150GB? Do you
> actually read all the newsgroups in that cache? slrnpull only downloads
> the newsgroups you specify, and you can specify expiration periods so
> that you can occasionally purge old messages.
First off, let me apologize for the "Chuck" typo in your name. I saw the error after sending the message. The U-I-O keys are just too close for me, and my eyes aren't as good as they once were. Then the speel chucker passed it. ;-)
The news spool total is around 150GB. It is not a "cache" but a spool.
For just the "slow text" newsgroups, there are 30 cycbuffs of 2 GB each. That's 60 GB just for the one metacycbuff, and it is but one metacycbuff of six total.
The others would include fast text, slow bins, fast bins, very fast, and junk.
Then there is a traditional spool hierarchy holding another gigabyte or so.
Of course I do not read everything! But I do read a couple dozen groups which are fairly low traffic.
I also check several other Usenet servers for missing articles, Path data, etc.
All Usenet text groups, and a half dozen binary groups, are fed to me by my peers. I don't "pull" groups from other NSPs, which would be impossible with the numbers shown below.
My active file stores 32,665 groups, those which are available for newsreader access.
Yesterday's totals indicate there are now 13,690,191 articles available in the news spool after the daily expiration of 49,491 articles:
It only needs to retain articles for around 10 days, in case it needs to feed them to any peer which might have been off-line for some reason. Most news servers will not accept an article older than ten days. These are "streamed" articles, not pulled.
Even then, the age of the slow text groups is currently 40 days (five 2 GB cycbuffs = 10 GB spool), and fast text is 69 days (one 2 GB cycbuff). The other three single 2 GB cycbuffs are:
fast bins has 196 days of articles
very fast has 322 days of articles
slow bins has 650 days of articles
The junk group has two 2 GB cycbuffs, and the oldest article is 3 days, 6:38.
More than enough "retention" buffer space available, just in case there is a Usenet "flood" some day.
Again, expiry is a cron job that runs at midnight every night.
The transit server is the fastest, finishing first in six minutes flat last night, because it has fewer articles to expire nightly:
Expire messages:
expireover start Wed Aug 8 00:00:27 CDT 2012: ( -z/var/log/news/expire.rm -Z/var/log/news/expire.lowmark)
Article lines processed 5258
Articles dropped 0
Overview index dropped 0
expireover end Wed Aug 8 00:04:58 CDT 2012
lowmarkrenumber begin Wed Aug 8 00:04:58 CDT 2012: (/var/log/news/expire.lowmark)
lowmarkrenumber end Wed Aug 8 00:04:59 CDT 2012
expire begin Wed Aug 8 00:05:29 CDT 2012: (-v1)
Article lines processed 3089880
Articles retained 2978169
Entries expired 111711
expire end Wed Aug 8 00:06:00 CDT 2012
all done Wed Aug 8 00:06:00 CDT 2012
The reader server takes longer, 22:15, because the spool is much larger:
Expire messages:
expireover start Wed Aug 8 00:00:22 CDT 2012: ( -z/var/log/news/expire.rm -Z/var/log/news/expire.lowmark)
Article lines processed 51221
Articles dropped 0
Overview index dropped 0
expireover end Wed Aug 8 00:19:00 CDT 2012
lowmarkrenumber begin Wed Aug 8 00:19:00 CDT 2012: (/var/log/news/expire.lowmark)
lowmarkrenumber end Wed Aug 8 00:19:00 CDT 2012
expire begin Wed Aug 8 00:19:31 CDT 2012: (-v1)
Article lines processed 13739682
Articles retained 13690191
Entries expired 49491
expire end Wed Aug 8 00:22:15 CDT 2012
all done Wed Aug 8 00:22:15 CDT 2012
The backup reader server is quite old, with a 266 MHz AMD-K6 CPU and only 256 MB of RAM. Plus the spool is much smaller. using a 40 GB hard drive:
Expire messages:
expireover start Wed Aug 8 00:06:44 CDT 2012: ( -z/var/log/news/expire.rm -Z/var/log/news/expire.lowmark)
Article lines processed 51214
Articles dropped 0
Overview index dropped 0
expireover end Wed Aug 8 01:32:28 CDT 2012
lowmarkrenumber begin Wed Aug 8 01:32:29 CDT 2012: (/var/log/news/expire.lowmark)
lowmarkrenumber end Wed Aug 8 01:32:31 CDT 2012
expire begin Wed Aug 8 01:33:02 CDT 2012: (-v1)
Article lines processed 7898846
Articles retained 7780311
Entries expired 118535
expire end Wed Aug 8 01:48:11 CDT 2012
all done Wed Aug 8 01:48:11 CDT 2012
It takes 1:48:11 to complete the nightly expiration of 118,535 articles.
You can see that the big time-consuming job is expiring the overview database. The CNFS spool buffers have nothing to rm, since they operate in a fixed file block size. The traditional spool groups are configured to never expire, so they don't add into the time calculations.
Yesterday's totals of incoming articles for the transit server were:
Incoming Feeds (INN):
Server Connects Offered Taken Refused Reject %Accpt Elapsed
Looking at the 433,443 incoming article total, many are not accepted. Many (321.086) were refused because we already had the article determined by an M-ID check, and 212 rejected because an article was not wanted (duplicates sent at the same instant from two or more peers after M-ID check indicated the article hadn't been seen), articles that are too old, etc.
Sites sending bad articles:
Server Total Group Dist Duplic Unapp TooOld Site Line Other
TOTAL: 5 62026 58165 0 152 1 0 0 0 3708
Unwanted newsgroups [Top 20]:
Newsgroup Count
TOTAL: 778 58165
These figures are far above what any article pulling or sucking program, like Leafnode, slrnpull, PullNews, Suck, etc. could ever hope to accomplish.
High enough to rank my news server farm at #24 in the Usenet Top1000 list for today.
On Wed, 08 Aug 2012 17:33:58 -0500, John F. Morse wrote:
> Chick Tower wrote:
>> On 2012-08-07, John F. Morse <j...@example.invalid> wrote:
>>> Getting the real news spool from an INN news server to my local
>>> console computer could easily be accomplished with rsync, but I don't
>>> care to copy 150 GB to the local hard drive! The spool is in a CNFS
>>> (Cyclic News File System) where many large 2 GB file buffers are used,
>>> and news articles fill them in a round-robin method so they never run
>>> out of disk space. IOW, I couldn't pick and choose individual
>>> articles.
>> Are you saying you have a local news article cache of 150GB? Do you
>> actually read all the newsgroups in that cache? slrnpull only
>> downloads the newsgroups you specify, and you can specify expiration
>> periods so that you can occasionally purge old messages.
> First off, let me apologize for the "Chuck" typo in your name. I saw the
> error after sending the message. The U-I-O keys are just too close for
> me, and my eyes aren't as good as they once were. Then the speel chucker
> passed it. ;-)
> The news spool total is around 150GB. It is not a "cache" but a spool.
> For just the "slow text" newsgroups, there are 30 cycbuffs of 2 GB each.
> That's 60 GB just for the one metacycbuff, and it is but one metacycbuff
> of six total.
> The others would include fast text, slow bins, fast bins, very fast, and
> junk.
> Then there is a traditional spool hierarchy holding another gigabyte or
> so.
> Of course I do not read everything! But I do read a couple dozen groups
> which are fairly low traffic.
> I also check several other Usenet servers for missing articles, Path
> data, etc.
> All Usenet text groups, and a half dozen binary groups, are fed to me by
> my peers. I don't "pull" groups from other NSPs, which would be
> impossible with the numbers shown below.
> My active file stores 32,665 groups, those which are available for
> newsreader access.
> Yesterday's totals indicate there are now 13,690,191 articles available
> in the news spool after the daily expiration of 49,491 articles:
> It only needs to retain articles for around 10 days, in case it needs to
> feed them to any peer which might have been off-line for some reason.
> Most news servers will not accept an article older than ten days. These
> are "streamed" articles, not pulled.
> Even then, the age of the slow text groups is currently 40 days (five 2
> GB cycbuffs = 10 GB spool), and fast text is 69 days (one 2 GB cycbuff).
> The other three single 2 GB cycbuffs are:
> fast bins has 196 days of articles very fast has 322 days of articles
> slow bins has 650 days of articles
> The junk group has two 2 GB cycbuffs, and the oldest article is 3 days,
> 6:38.
> More than enough "retention" buffer space available, just in case there
> is a Usenet "flood" some day.
> Again, expiry is a cron job that runs at midnight every night.
> The transit server is the fastest, finishing first in six minutes flat
> last night, because it has fewer articles to expire nightly:
> Expire messages:
> expireover start Wed Aug 8 00:00:27 CDT 2012: (
> -z/var/log/news/expire.rm -Z/var/log/news/expire.lowmark)
> Article lines processed 5258 Articles dropped 0
> Overview index dropped 0
> expireover end Wed Aug 8 00:04:58 CDT 2012 lowmarkrenumber begin Wed
> Aug 8 00:04:58 CDT 2012: (/var/log/news/expire.lowmark)
> lowmarkrenumber end Wed Aug 8 00:04:59 CDT 2012 expire begin Wed Aug 8
> 00:05:29 CDT 2012: (-v1)
> Article lines processed 3089880 Articles retained 2978169
> Entries expired 111711
> expire end Wed Aug 8 00:06:00 CDT 2012 all done Wed Aug 8 00:06:00 CDT
> 2012
> The reader server takes longer, 22:15, because the spool is much larger:
> Expire messages:
> expireover start Wed Aug 8 00:00:22 CDT 2012: (
> -z/var/log/news/expire.rm -Z/var/log/news/expire.lowmark)
> Article lines processed 51221 Articles dropped 0
> Overview index dropped 0
> expireover end Wed Aug 8 00:19:00 CDT 2012 lowmarkrenumber begin Wed
> Aug 8 00:19:00 CDT 2012: (/var/log/news/expire.lowmark)
> lowmarkrenumber end Wed Aug 8 00:19:00 CDT 2012 expire begin Wed Aug 8
> 00:19:31 CDT 2012: (-v1)
> Article lines processed 13739682 Articles retained 13690191
> Entries expired 49491
> expire end Wed Aug 8 00:22:15 CDT 2012 all done Wed Aug 8 00:22:15 CDT
> 2012
> The backup reader server is quite old, with a 266 MHz AMD-K6 CPU and
> only 256 MB of RAM. Plus the spool is much smaller. using a 40 GB hard
> drive:
> Expire messages:
> expireover start Wed Aug 8 00:06:44 CDT 2012: (
> -z/var/log/news/expire.rm -Z/var/log/news/expire.lowmark)
> Article lines processed 51214 Articles dropped 0
> Overview index dropped 0
> expireover end Wed Aug 8 01:32:28 CDT 2012 lowmarkrenumber begin Wed
> Aug 8 01:32:29 CDT 2012: (/var/log/news/expire.lowmark)
> lowmarkrenumber end Wed Aug 8 01:32:31 CDT 2012 expire begin Wed Aug 8
> 01:33:02 CDT 2012: (-v1)
> Article lines processed 7898846 Articles retained 7780311
> Entries expired 118535
> expire end Wed Aug 8 01:48:11 CDT 2012 all done Wed Aug 8 01:48:11 CDT
> 2012
> It takes 1:48:11 to complete the nightly expiration of 118,535 articles.
> You can see that the big time-consuming job is expiring the overview
> database. The CNFS spool buffers have nothing to rm, since they operate
> in a fixed file block size. The traditional spool groups are configured
> to never expire, so they don't add into the time calculations.
> Yesterday's totals of incoming articles for the transit server were:
> Incoming Feeds (INN):
> Server Connects Offered Taken Refused Reject %Accpt > Elapsed
> Looking at the 433,443 incoming article total, many are not accepted.
> Many (321.086) were refused because we already had the article
> determined by an M-ID check, and 212 rejected because an article was not
> wanted (duplicates sent at the same instant from two or more peers after
> M-ID check indicated the article hadn't been seen), articles that are
> too old, etc.
> Sites sending bad articles:
> Server Total Group Dist Duplic Unapp TooOld Site
> Line Other
> These figures are far above what any article pulling or sucking program,
> like Leafnode, slrnpull, PullNews, Suck, etc. could ever hope to
> accomplish.
> High enough to rank my news server farm at #24 in the Usenet Top1000
> list for today.
> "j...@wexfordpress.com" <j...@wexfordpress.com> wrote:
>> I prefer something that is GUI based and not too hair
>> shirtish. Is the Newsgroup feature of Seamonkey a winner or a bust?
> I haven't tried seamonkey for news, but for many years I have used pan
> which is included in Slackware and find it really useful and easy to use.
> Of course you will also need a nntp server to connect to, but this applies
> to most usenet news clients.
I've used Thunderbird ever since it was Netscape .9. Before that I used tin. I can't imagine why anybody would choose a web-based newsreader rather than a real nntp reader.
I've been personalizing it (along with Firefox) since it was born and it's nearly perfect (TB14) now :-)
I tried installing Seamonkey a few months ago, but something burped and I gave up. Not much reason to try to fix something broken unless you really want or need it!
-- Cheers, Bev
=============================================================
"On the other hand, I live in California so I'd be willing to
squeeze schoolchildren to death if I thought some oil would
come out." -- Scott Adams
The Real Bev wrote:
> On 08/01/2012 12:20 PM, Henrik Carlqvist wrote:
>> "j...@wexfordpress.com" <j...@wexfordpress.com> wrote:
>>> I prefer something that is GUI based and not too hair
>>> shirtish. Is the Newsgroup feature of Seamonkey a winner or a bust?
>> I haven't tried seamonkey for news, but for many years I have used pan
>> which is included in Slackware and find it really useful and easy to use.
>> Of course you will also need a nntp server to connect to, but this
>> applies
>> to most usenet news clients.
> I've used Thunderbird ever since it was Netscape .9. Before that I used
> tin. I can't imagine why anybody would choose a web-based newsreader
> rather than a real nntp reader.
> I've been personalizing it (along with Firefox) since it was born and
> it's nearly perfect (TB14) now :-)
> I tried installing Seamonkey a few months ago, but something burped and
> I gave up. Not much reason to try to fix something broken unless you
> really want or need it!
I never got over the demise of Netscape Communicator ;-) SeaMonkey is
the only thing close. Using the default theme and text only button display.
I've tried Pan, tin, slrn and lynx. Not even close.
-- -Rinaldi-
It is the business of little minds to shrink.
-- Carl Sandburg
On 2012-08-19, The Real Bev <bashley...@gmail.com> wrote:
> I tried installing Seamonkey a few months ago, but something burped and > I gave up. Not much reason to try to fix something broken unless you > really want or need it!
The official Slackware package failed to install correctly on your
system, Bev?
-- Chick Tower
For e-mail: aols2 DOT sent DOT towerboy AT xoxy DOT net
> The Real Bev wrote:
>> On 08/01/2012 12:20 PM, Henrik Carlqvist wrote:
>>> "j...@wexfordpress.com" <j...@wexfordpress.com> wrote:
>>>> I prefer something that is GUI based and not too hair
>>>> shirtish. Is the Newsgroup feature of Seamonkey a winner or a bust?
>>> I haven't tried seamonkey for news, but for many years I have used pan
>>> which is included in Slackware and find it really useful and easy to use.
>>> Of course you will also need a nntp server to connect to, but this
>>> applies
>>> to most usenet news clients.
>> I've used Thunderbird ever since it was Netscape .9. Before that I used
>> tin. I can't imagine why anybody would choose a web-based newsreader
>> rather than a real nntp reader.
>> I've been personalizing it (along with Firefox) since it was born and
>> it's nearly perfect (TB14) now :-)
>> I tried installing Seamonkey a few months ago, but something burped and
>> I gave up. Not much reason to try to fix something broken unless you
>> really want or need it!
> I never got over the demise of Netscape Communicator ;-) SeaMonkey is
> the only thing close. Using the default theme and text only button display.
Ha. Mine is prettier than yours!
> I've tried Pan, tin, slrn and lynx. Not even close.
We all move on...
-- Cheers, Bev
I have six locks on my door all in a row. When I go out, I lock
every other one. I figure no matter how long somebody stands there
picking the locks, they are always locking three.
> On 2012-08-19, The Real Bev <bashley...@gmail.com> wrote:
>> I tried installing Seamonkey a few months ago, but something burped and
>> I gave up. Not much reason to try to fix something broken unless you
>> really want or need it!
> The official Slackware package failed to install correctly on your
> system, Bev?
Can't remember. I probably downloaded it directly from mozilla.
-- Cheers, Bev
I have six locks on my door all in a row. When I go out, I lock
every other one. I figure no matter how long somebody stands there
picking the locks, they are always locking three.
