Overview (?) problem
Bjoern von Prollius
my new newsserver (see thread "pullnews") is now up and running, with
all messages imported. Still it shows some strange behaviour:
(1) When I refresh the groups in my client (MozTB), total message count
is way more than there are actual messages in the group. As soon as I
click the group, the number is corrected. I ruled out client-side
caching problems. I think maybe something got cofused in the overview
database, due to my multiple pullnews tries.
I tried
# ctlinnd renumber [group]
as well as
# makedbz -i -f history
but that did not help. Any suggestions?
(2) On some groups the client shows 1 new message. As soon as the group
is opened, there is no new message (and there really is none). Maybe
that's the same problem as (1)?
Thanks again for every advice,
Bjoern
Julien ÉLIE
> (1) When I refresh the groups in my client (MozTB), total message count
> is way more than there are actual messages in the group. As soon as I
> click the group, the number is corrected.
I have ideas on that subject but better be sure.
Could you "telnet your.server.name 119" and send:
MODE READER
LIST ACTIVE group.name*
LISTGROUP group.name
where group.name is the name of one of the newsgroup causing problems.
Yes, there is a "*" at the end of its name for LIST ACTIVE (because
it will give information from the active file -- without the "*", it
will give the same information as GROUP).
What are the results of these commands?
> (2) On some groups the client shows 1 new message. As soon as the group
> is opened, there is no new message (and there really is none). Maybe
> that's the same problem as (1)?
Same thing as above.
--
Julien ï¿œLIE
ￜ -- Il est parti comme il est venu...
-- Il ne faisait que passer... ￜ (Astￜrix)
Ray Banana
> (1) When I refresh the groups in my client (MozTB), total message count
> is way more than there are actual messages in the group. As soon as I
> click the group, the number is corrected. I ruled out client-side
> caching problems. I think maybe something got cofused in the overview
> database, due to my multiple pullnews tries.
I assume the article numbers started from 0 on your new server, so the
high and low watermarks are lower than on the old server. If you use the
same account in TB as before, TB will go nuts over the discrepancies
between its newsrc file and what it gets from the server. TB is just a
PITA when it comes to resetting article numbers (you will have to edit
the newsrc file with an editor), so you'd better create a new account
for the new server and see if that solves the issue.
--
Time flies like an arrow, fruit flies like a Banana.
http://www.eternal-september.org
Bjoern von Prollius
> so you'd better create a new account
> for the new server and see if that solves the issue.
I did this before, even removed the corresponding files in my TB
profile, that's why I thought (and wrote) that I ruled out client side
problems.
Thx, Bjoern
Bjoern von Prollius
> Could you "telnet your.server.name 119" and send:
Did that. Btw, it was not easy to issue the commands, because INN liked
to close the connection spontaneously - I think that happened when
someone else connected to the server.
The results:
--------------------------------------------
[root@news ~]# telnet localhost 119
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
200 news.hdm-stuttgart.de InterNetNews server INN 2.5.1 ready
MODE READER
200 news.hdm-stuttgart.de InterNetNews NNRP server INN 2.5.1 ready
(posting ok)
LIST ACTIVE hdm.mi.allgemein*
215 Newsgroups in form "group high low flags"
hdm.mi.allgemein 0000001336 0000000001 y
.
LISTGROUP hdm.mi.allgemein
211 1782 1 1336 hdm.mi.allgemein
891
892
893
894
[... long list of numbers ...]
1335
1336
.
quit
205 Bye!
Connection closed by foreign host.
--------------------------------------------
Interestingliy, the highmark (1336) plus number of messages in the group
(446) is exactly the number of messages my newsreader tells me to be in
the group (1782) until I click the group and the display is corrected to
446.
Conclusions?
Guys, you are so helpful!
Bjoern
Julien ÉLIE
>> Could you "telnet your.server.name 119" and send:
>
> Did that. Btw, it was not easy to issue the commands, because INN liked
> to close the connection spontaneously - I think that happened when
> someone else connected to the server.
That's not normal!
INN does not close the connection spontaneously (except if you do not
send a command during the first 10 seconds after MODE READER -- is that
what happened?).
> LIST ACTIVE hdm.mi.allgemein*
> 215 Newsgroups in form "group high low flags"
> hdm.mi.allgemein 0000001336 0000000001 y
> .
> LISTGROUP hdm.mi.allgemein
> 211 1782 1 1336 hdm.mi.allgemein
> 891
> 892
> 893
> 894
> [... long list of numbers ...]
> 1335
> 1336
> .
>
> Interestingliy, the highmark (1336) plus number of messages in the group
> (446) is exactly the number of messages my newsreader tells me to be in
> the group (1782) until I click the group and the display is corrected to
> 446.
I would have counted the other way: 1336-891+1 = 446.
It is the number of articles you have (in response to LISTGROUP).
There is a bug here: the server announces 1782 articles, which is more
than the high water mark (1336)...
It is normal that Thunderbird shows 1782 (it takes the number with a similar
command named GROUP) and, after you click on the group, the right number 446
because it only receives 446 articles.
> Conclusions?
Did you delete the overview files before running "makehistory -O -b"?
According to the man page:
WARNING: If you're trying to rebuild the overview database, be sure to stop
innd(8) and delete or zero out the existing database before you start for
the best results. An overview rebuild should not be done while the server
is running. Unless the existing overview is deleted, you may end up with
problems like out-of-order overview entries, excessively large overview
buffers, and the like.
If you did it, could you try to run "tdx-util -A"? Does it return errors?
Bjoern von Prollius
> INN does not close the connection spontaneously (except if you do not
> send a command during the first 10 seconds after MODE READER -- is that
> what happened?).
That might have been what happened - I will keep an eye on that.
> There is a bug here: the server announces 1782 articles, which is more
> than the high water mark (1336)...
Actually, today it only announced 1336. Maybe news.daily cleaned
something up...
> Did you delete the overview files before running "makehistory -O -b"?
It seems that I have missed that. So now I deleted the content of
/var/spool/news/overview and after that did another "makehistory -O -b".
While restarting INN reported "INND: overview was rebuilt, and active
needs to be renumbered!", and now the message count in my Thunderbird
seems corret! GREAT!
What still remains is the problem, that in some groups Thunderbird shows
one new message, but when I open the group there is no new messsage and
TB also corrects its display. Message count stays the same (446 before
and after opening the group).
LIST ACTIVE shows the same results as yesterday, and LISTGROUP now
results in
-----------------
LISTGROUP hdm.mi.allgemein
211 446 1 1336 hdm.mi.allgemein
-----------------
so only the number of articles is now corrected.
> could you try to run "tdx-util -A"? Does it return errors?
Indeed it still does:
------------------------------------
[root@news news]# /usr/lib/news/bin/tdx-util -A
tdx-util: tradindexed: low water mark incorrect for hdm.suche_biete: 334
!= 1
tdx-util: tradindexed: low water mark incorrect for hdm.allgemein: 651 != 2
tdx-util: tradindexed: low water mark incorrect for hdm.mi.allgemein:
891 != 1
tdx-util: tradindexed: low water mark incorrect for
hdm.mi.mib-offiziell: 1536 != 1
tdx-util: tradindexed: low water mark incorrect for
hdm.mi.csm-offiziell: 259 != 1
tdx-util: tradindexed: low water mark incorrect for
hdm.mi.mmb-offiziell: 22 != 1
------------------------------------
All those groups where feeded by pullnews initially. There are more
groups on the server, but the others started empty and thus show no
problems. Only two of the groups above have the one-new-message-problem.
Thanks,
Bjoern
PS: I am sorry that it takes so long for me to get all the details and
connections. I read a lot of man pages, but what I am still missing is
something like "The INN Big Picture" that explains how everything's tied
together, what happens when, where everything is stored and so on...
Julien ÉLIE
>> There is a bug here: the server announces 1782 articles, which is more
>> than the high water mark (1336)...
>
> Actually, today it only announced 1336. Maybe news.daily cleaned
> something up...
Yes.
>> Did you delete the overview files before running "makehistory -O -b"?
>
> It seems that I have missed that. So now I deleted the content of
> /var/spool/news/overview and after that did another "makehistory -O -b".
> While restarting INN reported "INND: overview was rebuilt, and active
> needs to be renumbered!"
That automatic renumbering is a new feature in INN 2.5
> and now the message count in my Thunderbird seems corret! GREAT!
That's then better.
> What still remains is the problem, that in some groups Thunderbird shows
> one new message, but when I open the group there is no new messsage and
> TB also corrects its display. Message count stays the same (446 before
> and after opening the group).
Maybe a TB bug, as Ray said.
> LIST ACTIVE shows the same results as yesterday, and LISTGROUP now
> results in
> -----------------
> LISTGROUP hdm.mi.allgemein
> 211 446 1 1336 hdm.mi.allgemein
> -----------------
> so only the number of articles is now corrected.
>
>> could you try to run "tdx-util -A"? Does it return errors?
>
> Indeed it still does:
> ------------------------------------
> [root@news news]# /usr/lib/news/bin/tdx-util -A
> tradindexed: low water mark incorrect for hdm.mi.allgemein: 891 != 1
Run "tdx-util -F" to fix these errors.
Has news.daily been run after your makehistory? If no, it is normal
that the low water mark is still 1 after your fresh rebuild.
Maybe it should be run (once) with "lowmark"?
> All those groups where feeded by pullnews initially. There are more
> groups on the server, but the others started empty and thus show no
> problems. Only two of the groups above have the one-new-message-problem.
What is the contents of the .newsrc TB file for these groups?
> PS: I am sorry that it takes so long for me to get all the details and
> connections. I read a lot of man pages, but what I am still missing is
> something like "The INN Big Picture" that explains how everything's tied
> together, what happens when, where everything is stored and so on...
Sure!
We also need to define the boundaries of such "Big Picture" (because
"everything" and "what happens when" cannot be easily explained...).
--
Julien ï¿œLIE
ￜ Cogitationis poenam nemo patitur. ￜ
Bjoern von Prollius
> Maybe a TB bug, as Ray said.
Seems so, because today the problem is gone... Maybe also some
news.daily-cleanup? Anyway, problem solved!
>> [root@news news]# /usr/lib/news/bin/tdx-util -A
>> tradindexed: low water mark incorrect for hdm.mi.allgemein: 891 != 1
>
> Run "tdx-util -F" to fix these errors.
Seems also to have fixed itself magically... today
"/usr/lib/news/bin/tdx-util -A" reports nothing.
> Has news.daily been run after your makehistory? If no, it is normal
> that the low water mark is still 1 after your fresh rebuild.
Yeah, that's probably it. Some nights passed, news.daily ran,
everything's clean.
> Sure!
> We also need to define the boundaries of such "Big Picture" (because
> "everything" and "what happens when" cannot be easily explained...).
Yes, you're right. If I could contribute in any way, I would be glad to.
I had the qmail big picture [1] in mind when I said that. Maybe I could
ask the questions (from my rookie point of view) , and somebody else
could give the answers...
[1] http://www.nrg4u.com/qmail/the-big-qmail-picture-103-p1.gif
Regards,
Bjï¿œrn
Julien ÉLIE
> Yeah, that's probably it. Some nights passed, news.daily ran,
> everything's clean.
news.daily is automagically fixes problems :)
> I had the qmail big picture [1] in mind when I said that. Maybe I could
> ask the questions (from my rookie point of view) , and somebody else
> could give the answers...
You will find useful pictures here:
http://web.inter.nl.net/users/Elena.Samsonova/unix/INN/v2.3/architectures.html
They are still up-to-date, and pretty complete!
--
Julien ï¿œLIE
ￜ La vie est un sommeil, l'amour en est le rￜve et vous aurez vￜcu
si vous avez aimￜ. ￜ (Alfred de Musset)