[slrn] Unable to score articles based on Xref: or Newsgroups: headers.

[Lafe]

Hello folks,

I'm a bit new to slrn, and I've been having this scoring issue for a bit, and
haven't been able to figure out where I'm going wrong.

Any scores that I apply to the Xref: or Newsgroups: headers don't seem to work
for me at all.

I was using version 1.0.3 on ubuntu 20.04. I saw that there were developer
preview versions available, so I updated to the latest version and built from
source. But this didn't contain a fix for my issue.

slrn --version gives me:

>slrn pre1.0.4-5
> * Note: This version is a developer preview.
>S-Lang Library Version: pre2.3.3-58
>Operating System: Linux
>
>COMPILE TIME OPTIONS:
> Backends: +nntp -slrnpull -spool
> External programs / libs: -canlock -inews +ssl -uudeview +iconv
> Features: +decoding +emphasized_text +end_of_thread +fake_refs +gen_msgid
> -grouplens -msgid_cache +piping +rnlock +spoilers -strict_from
> Using 64 bit integers for article numbers.
>
>DEFAULTS:
> Default server object: nntp
> Default posting mechanism: nntp

Below is what's currently in my score file, I added the Google and killfile
scores just to see if scoring was working at all, and it is, both of the other
two scores work and I'm able to see that articles are killed and/or scored by
them. Only the first one does nothing, and I've tried a number of different
variations... but this Xref: rule pulled directly from the examples doesn't have
any effect.

I have also tried it with and without prefer_head. I can bring up articles and
see that there are multiple colons in the Xref: header as displayed by slrn.

Any ideas? Thanks so much for any help!

score file:
---------------------

[*]
Score: -100
Xref: :.*:

Score: -50
From: Google

Score: =-9999
From: test-user@nospam\.example

[Tavis Ormandy]

On 2021-08-25, Lafe wrote:
> I have also tried it with and without prefer_head. I can bring up articles and
> see that there are multiple colons in the Xref: header as displayed by slrn.
>
> Any ideas? Thanks so much for any help!
>
> score file:
> ---------------------
>
> [*]
> Score: -100
> Xref: :.*:
>

It looks okay to me, have you looked at the view_scores dialog on an
article you think it should match on? (Default key is 'v'), maybe the
score is being cancelled out...

Tavis.

--
_o) $ lynx lock.cmpxchg8b.com
/\\ _o) _o) $ finger tav...@sdf.org
_\_V _( ) _( ) @taviso

[Lafe]

On 2021-08-26, Tavis Ormandy <tav...@gmail.com> wrote:

> On 2021-08-25, Lafe wrote:
>> score file:
>> ---------------------
>>
>> [*]
>> Score: -100
>> Xref: :.*:
>>
>
> It looks okay to me, have you looked at the view_scores dialog on an
> article you think it should match on? (Default key is 'v'), maybe the
> score is being cancelled out...

Thanks for looking! I didn't know about that view_scores, but I tried it on an
article in this newsgroup that had an Xref header that contained two colon ":"
characters in its contents, and it told me that the article didn't match on any
rules.

I'm guessing that's the root of the problem. It doesn't seem to like that
specific regex, or just isn't able to apply it to that header correctly.

You say that it does work for you though? I'm not sure what could be
different.

If I add a Newsgroups: rule, and then attempt open a newsgroup, it appears that
slrn will download articles automatically to be able to check the rule. But
only sometimes. And if it attempts this against a newsgroup with more than
about 20 unread articles it says that there's an error and disconnects me and
quits/crashes to the command line. That may be something weird with my nntp
provider though (newsdemon).

Scoring on Xref or Newsgroups headers isn't a deal breaker, it's just strange
and I wish I could figure out why it doesn't work.

[Tavis Ormandy]

On 2021-08-26, Lafe wrote:
> On 2021-08-26, Tavis Ormandy <tav...@gmail.com> wrote:
> Thanks for looking! I didn't know about that view_scores, but I tried it on an
> article in this newsgroup that had an Xref header that contained two colon ":"
> characters in its contents, and it told me that the article didn't match on any
> rules.
>
> I'm guessing that's the root of the problem. It doesn't seem to like that
> specific regex, or just isn't able to apply it to that header correctly.
>
> You say that it does work for you though? I'm not sure what could be
> different.

Yes, I use rules like this that work fine:

Score: -10 % X-Posted > 3
Xref: :.*:.*:.*:
Score: -10 % X-Posted > 4
Xref: :.*:.*:.*:.*:
Score: -10 % X-Posted > 5
Xref: :.*:.*:.*:.*:.*:

>
> If I add a Newsgroups: rule, and then attempt open a newsgroup, it appears that
> slrn will download articles automatically to be able to check the rule. But
> only sometimes. And if it attempts this against a newsgroup with more than
> about 20 unread articles it says that there's an error and disconnects me and
> quits/crashes to the command line. That may be something weird with my nntp
> provider though (newsdemon).

Hmm, maybe try `set prefer_head 2`? If that fixes it, seems like it is
your provider (Note that scoring will be much slower without overviews,
but maybe that doesn't bother you!).

>
> Scoring on Xref or Newsgroups headers isn't a deal breaker, it's just strange
> and I wish I could figure out why it doesn't work.
>

Maybe also verify there is no accidental weird line endings or
something? You could create a fresh scorefile like:

$ printf "Score: -10\nXref: :.*:\n" > scorefile

[Lafe]

On 2021-08-26, Tavis Ormandy <tav...@gmail.com> wrote:
> Hmm, maybe try `set prefer_head 2`? If that fixes it, seems like it is
> your provider (Note that scoring will be much slower without overviews,
> but maybe that doesn't bother you!).

Thanks for the suggestion! Setting it to prefer_head did fix it, so I agree that
seems to point to an issue with my nntp server. It changed to the behavior I see
with "Newsgroups:" rules, where it wants to download all of the articles to
perform the scoring.

So now if I pull more than 15 or 20 articles when I enter a group, slrn says
there's an error and "crashes" to the command line saying that it can't recover.
I think this is also likely a weirdness with how it requests the articles and my
news provider killing the connection or something.

I appreciate your help in the troubleshooting! I can live without cumulative
scoring on crossposts while I'm using this nntp provider.

[Eli the Bearded]

In news.software.readers, Lafe <la...@lafes.newlafe.net> wrote:
> On 2021-08-26, Tavis Ormandy <tav...@gmail.com> wrote:
>> Hmm, maybe try `set prefer_head 2`? If that fixes it, seems like it is
>> your provider (Note that scoring will be much slower without overviews,
>> but maybe that doesn't bother you!).
> Thanks for the suggestion! Setting it to prefer_head did fix it, so I
> agree that seems to point to an issue with my nntp server. It changed
> to the behavior I see with "Newsgroups:" rules, where it wants to
> download all of the articles to perform the scoring.

Have you tried to look at the overview records yourself?

Path: [...]
+ feed.usenet.farm!feed.usenet.farm!tr2.eu1.usenetexpress.com!feeder.
+ usenetexpress.com!tr1.iad1.usenetexpress.com!
+ 2a07:8080:119:fe:f19b:d5b2:4545:7979.MISMATCH!news.newsdemon.com!not-
+ for-mail

Newsdemon seems to be using IPv6 without IPv6 reverse names.

Anyway, general method is:

telnet news.newsdemon.com 119
AUTHINFO USER username
AUTHINFO PASS password
LIST overview.fmt
XOVER group.name:articlenum
QUIT

AUTHINFO only used if needed, there's also an "AUTHINFO GENERIC ..."
variation. The overview.fmt file gives the details of what to expect.
The local one here is:

Subject:
From:
Date:
Message-ID:
References:
Bytes:
Lines:
Xref:full
NNTP-Posting-Host:full

That gives the meaning of each line in the output, and the ones marked
"full" will have a header name in front of them, the overs just the
field value.

The XOVER command can get a single article or a range.
XOVER news.software.readers:1000
XOVER news.software.readers:1000-1050

I've heard of bugs in newsreaders with values after the standard
Subject: to Lines: entries that _don't_ have the header name. One
such bug is why this site has "NNTP-Posting-Host:full". I think
it was in Xnews.

> So now if I pull more than 15 or 20 articles when I enter a group,
> slrn says there's an error and "crashes" to the command line saying
> that it can't recover.
> I think this is also likely a weirdness with how it requests the
> articles and my news provider killing the connection or something.

As someone who never uses slrn, but has a vague interest in nntp
clients, I'm curious what's going on here. I looked into this thread
because I suspected some overview db issues.

Elijah
------
uses trn and perl NNTP module scripts for his news

[Lafe]

Eli the Bearded <*@eli.users.panix.com> wrote in
news:eli$21082...@qaz.wtf: <snip>

I'm sorry that I somehow missed your follow-up to my post before. I tried
your suggestion, and I had the following results:

~$ telnet news.newsdemon.com 119
Trying 185.151.13.247...
Connected to nd.tlsusenet.com.
Escape character is '^]'.
200 Welcome (n14.iad1)
AUTHINFO USER **************
381 Proceed with PASS
AUTHINFO PASS **************
281 Hello **************!
LIST overview.fmt
215 Information follows (multi-line)

Subject:
From:
Date:
Message-ID:
References:
Bytes:
Lines:
Xref:full

.
XOVER news.software.readers:1000
400 Unrecognized command
OVER eli$21082...@qaz.wtf
400 Unrecognized command
OVER news.software.readers:1000
400 Unrecognized command
QUIT
205 Goodbye
Connection closed by foreign host.

I'm guessing the complete lack of support for XOVER/OVER is what was
causing this issue with SLRN, and why prefer-head 2 "sort of" worked.

Thanks for the suggestion!

Lafe