Spamhalter stopped working

6 views
Skip to first unread message

brightside

unread,
Sep 24, 2019, 9:48:28 AM9/24/19
to
Spamhalter has been working since 2012 without any problems.
Suddenly two days ago all emails that used to be detected and placed
in a junk folder no longer get detected. The traffic light is green
for all messages..

I can find no way of reseting spanhalter. I have 'cleaned' the
database to no effect.

Here's what system messgaes show whem pmail v4.73 is started.

Tue, 14:36:02 Loading local legacy folders from cache file...
Tue, 14:36:02 Spamhalter beysian plugin 4.5.2.180 started.
Tue, 14:06:02 Spamhalter message size limit is: 250000 bytes
Tue, 14:36:02 Spamhalter database size is: 17758 tokens
Tue, 14:06:03 Program started

How do I get the spamhalter 'plugin' to start working again?

--
brightside S9

Euler German

unread,
Sep 24, 2019, 10:34:58 AM9/24/19
to
On article <hp6koe5fbo6res1sh...@4ax.com>, brightside
wrote (at least in part):
It is working but may be mis-configured. Go to Tools > Spam and
content controls > SpamHalter... and look if it is properly
configured.

https://i.ibb.co/M8LKXjz/2019-09-24-110924.gif

--
Kind regards,
Euler German

Please, reply preferably to the list.
Reply-To: partially ROT13, invalid=com
Due to spam I'm filtering-out GoogleGroups. Sorry. :(

brightside

unread,
Sep 24, 2019, 11:40:37 AM9/24/19
to
On Tue, 24 Sep 2019 11:34:56 -0300, Euler German
<rstrezn...@znvyahyy.invalid> wrote:

>On article <hp6koe5fbo6res1sh...@4ax.com>, brightside
>wrote (at least in part):
>
>>
>> Spamhalter has been working since 2012 without any problems.
>> Suddenly two days ago all emails that used to be detected and placed
>> in a junk folder no longer get detected. The traffic light is green
>> for all messages..
>>
>> I can find no way of reseting spanhalter. I have 'cleaned' the
>> database to no effect.
>>
>> Here's what system messgaes show whem pmail v4.73 is started.
>>
>> Tue, 14:36:02 Loading local legacy folders from cache file...
>> Tue, 14:36:02 Spamhalter beysian plugin 4.5.2.180 started.
>> Tue, 14:06:02 Spamhalter message size limit is: 250000 bytes
>> Tue, 14:36:02 Spamhalter database size is: 17758 tokens
>> Tue, 14:06:03 Program started
>>
>> How do I get the spamhalter 'plugin' to start working again?
>
>It is working but may be mis-configured. Go to Tools > Spam and
>content controls > SpamHalter... and look if it is properly
>configured.
>
>https://i.ibb.co/M8LKXjz/2019-09-24-110924.gi


Yes it looks OK to me.

spamhalter = enabled
train always selected
spam level (%) = 80
not spam boost = 2


Clean up has been done as a trial to get it to work.

'Purge database records older than' 180 ' days',
'Purge preview' gives result 0 out of 17758 records will be removed.

--
brightside s9

Euler German

unread,
Sep 24, 2019, 5:37:30 PM9/24/19
to

On article <lndkoe9f83j93eidh...@4ax.com>, brightside
wrote (at least in part):

> spamhalter = enabled
> train always selected
> spam level (%) = 80
> not spam boost = 2
>
First, check if SpamHalter is pointing to a valid (existing) folder.
Second, check if this folder is functional (right-click > Check
consistency).

>
> Clean up has been done as a trial to get it to work.
>
> 'Purge database records older than' 180 ' days',
> 'Purge preview' gives result 0 out of 17758 rec
>
That's fine, but it is only meant to reduce corpus size. As you're
using "train always" database size is not a problem.

Lastly, you can set SH to build a log. With Pmail shut down open its
settings file WI_sph.ini and change the line debug=0 to debug=1. If
it is already 1 then you already have a WI_sph.log file in your home
mailbox folder. Maybe it may give you a hint of what is going wrong.

brightside

unread,
Sep 25, 2019, 4:28:17 AM9/25/19
to
On Tue, 24 Sep 2019 18:37:28 -0300, Euler German
<rstrezn...@znvyahyy.invalid> wrote:

>
>On article <lndkoe9f83j93eidh...@4ax.com>, brightside
>wrote (at least in part):
>
>> spamhalter = enabled
>> train always selected
>> spam level (%) = 80
>> not spam boost = 2
>>
>First, check if SpamHalter is pointing to a valid (existing) folder.
>Second, check if this folder is functional (right-click > Check
>consistency).
>
>>
>> Clean up has been done as a trial to get it to work.
>>
>> 'Purge database records older than' 180 ' days',
>> 'Purge preview' gives result 0 out of 17758 rec
>>
>That's fine, but it is only meant to reduce corpus size. As you're
>using "train always" database size is not a problem.
>
>Lastly, you can set SH to build a log. With Pmail shut down open its
>settings file WI_sph.ini and change the line debug=0 to debug=1. If
>it is already 1 then you already have a WI_sph.log file in your home
>mailbox folder. Maybe it may give you a hint of what is going wrong.

Here's the WI+sph file data:
[spamhalter]
junkfolder=3JM6AHDJ:367A:FOL04BA6
bayspamprob=8
baynospamboost=2
enabled=1
trainalways=1
debug=0
ForcedWrites=0
MaxTokens=25
SizeLimit=250000
CustomHeaders=
WALmode=0

I changed debiug to '1'


OK I'll see what happens.

--
brightside S9

brightside

unread,
Sep 26, 2019, 4:17:49 AM9/26/19
to
On Tue, 24 Sep 2019 18:37:28 -0300, Euler German
<rstrezn...@znvyahyy.invalid> wrote:

>
>On article <lndkoe9f83j93eidh...@4ax.com>, brightside
>wrote (at least in part):
>
>> spamhalter = enabled
>> train always selected
>> spam level (%) = 80
>> not spam boost = 2
>>
>First, check if SpamHalter is pointing to a valid (existing) folder.
>Second, check if this folder is functional (right-click > Check
>consistency).
>
>>
>> Clean up has been done as a trial to get it to work.
>>
>> 'Purge database records older than' 180 ' days',
>> 'Purge preview' gives result 0 out of 17758 rec
>>
>That's fine, but it is only meant to reduce corpus size. As you're
>using "train always" database size is not a problem.
>
>Lastly, you can set SH to build a log. With Pmail shut down open its
>settings file WI_sph.ini and change the line debug=0 to debug=1. If
>it is already 1 then you already have a WI_sph.log file in your home
>mailbox folder. Maybe it may give you a hint of what is going wrong.

This morning, start pmail, get mail, 5 messgaes that should be
detected as spam were not.

Here is the log file:
20190926 090044.526 W_STARTUP
20190926 090044.541 C:\PMAIL\MAIL\aml\
20190926 090044.682 SB_ENABLED
20190926 090044.854 SB_ENABLED
20190926 090114.260 SB_ENABLED
20190926 090114.541 SB_ENABLED
20190926 090114.572 SB_ENABLED
20190926 090114.635 SB_ENABLED
20190926 090114.697 SB_ENABLED
20190926 090114.760 SB_ENABLED
20190926 090114.854 SB_ENABLED
20190926 090119.994 SB_ENABLED
20190926 090120.026 SB_ENABLED
20190926 090120.072 SB_ENABLED
20190926 090120.151 SB_ENABLED
20190926 090123.276 SB_ENABLED
20190926 090125.791 SB_ENABLED
20190926 090125.838 SB_ENABLED
20190926 090125.869 SB_ENABLED
20190926 090136.838 SB_ENABLED
20190926 090136.869 SB_ENABLED
20190926 090237.447 W_SHUTDOWN 1

Nothing there to indicate what is happeneing.

--
brightside S9

Euler German

unread,
Sep 26, 2019, 12:10:25 PM9/26/19
to

On article <89sooelhh4kkh4hi7...@4ax.com>, brightside
It's quite clear to me SpamHalter is working fine BUT your corpus has
no usable data (i.e. corrupt). I'd discard it (don't delete, rename
words4.db3 to words4.db3.bak), of course with Pmail shut down.

If you have a backup of it, use it in place. That will save training
time. If not, I can send you mine. It is not the ideal world but
again will save you training time. Let me know.

Lastly, change SpamHalter setting "Train always (larger database,
self-trained)" to "Train on classification errors only (smaller
database)". This will save you disk space and (more important) data
processing time. Use this setting even if you're using your own
corpus backup.

Below is how your SpamHalter log should look:

20190924 183012.449 W_STARTUP
20190924 183012.450 C:\PMAIL\MAIL\german\
20190924 183012.630 SB_ENABLED
20190924 183014.841 SB_ENABLED
20190924 183014.842 SB_ENABLED
20190924 183014.877 SB_ENABLED
20190924 183014.915 SB_ENABLED
20190924 183015.027 SB_ENABLED
20190924 190817.364 SB_ENABLED
20190924 190817.365 SB_CLASSIFY 1633148
20190924 190817.367 Size: 77615
20190924 190817.505 - 73 0 3 60 0
20190924 190817.506 Tokens: 142
20190924 190817.507 Probability: 0.0000
20190924 193303.492 W_SHUTDOWN 1
20190925 110614.349 W_STARTUP
20190925 110614.351 C:\PMAIL\MAIL\german\
20190925 110616.632 SB_ENABLED
20190925 110617.853 SB_ENABLED
20190925 110617.856 SB_ENABLED
20190925 110618.283 SB_ENABLED
20190925 110619.862 SB_ENABLED
20190925 110620.335 SB_ENABLED
20190925 111147.031 SB_ENABLED
20190925 111147.032 SB_CLASSIFY 1633148
20190925 111147.033 Size: 9360
20190925 111147.125 - 5 0 1 86 0
20190925 111147.126 Tokens: 118
20190925 111147.126 Probability: 0.0000
20190925 111147.670 SB_ENABLED
20190925 111147.671 SB_CLASSIFY 1633148
20190925 111147.672 Size: 12079
20190925 111147.764 - 3 0 2 85 1
20190925 111147.765 Tokens: 176
20190925 111147.765 Probability: 0.0000
20190925 111148.231 SB_ENABLED
20190925 111148.233 SB_CLASSIFY 1633148
20190925 111148.235 Size: 10020
20190925 111148.274 - 5 0 1 33 0
20190925 111148.275 Tokens: 87
20190925 111148.276 Probability: 0.0000
20190925 111148.664 SB_ENABLED
20190925 111148.666 SB_CLASSIFY 1633148
20190925 111148.667 Size: 10947
20190925 111148.702 - 3 1 1 28 0
20190925 111148.702 Tokens: 121
20190925 111148.702 Probability: 0.0000
20190925 111213.713 SB_ENABLED
20190925 111213.714 SB_CLASSIFY 1633148
20190925 111213.715 Size: 73676
20190925 111213.804 - 4 0 5 79 0
20190925 111213.805 Tokens: 247
20190925 111213.806 Probability: 0.0000
20190925 111214.329 SB_ENABLED
20190925 111214.330 SB_CLASSIFY 1633148
20190925 111214.331 Size: 13321
20190925 111214.589 - 2 1 5 250 0
20190925 111214.590 Tokens: 252
20190925 111214.590 Probability: 0.0000
20190925 111227.887 SB_ENABLED
20190925 111227.888 SB_CLASSIFY 1633148
20190925 111227.889 Size: 11961
20190925 111227.981 - 3 0 4 85 0
20190925 111227.982 Tokens: 185
20190925 111227.982 Probability: 0.3393
20190925 113747.280 SB_ENABLED
20190925 113747.285 SB_ENABLED
20190925 113749.028 SB_LEVELS
20190925 113819.348 SB_ENABLED
20190925 113819.350 SB_MOVE 87475312 ZQW73F3E:6FFB:FOL06E43
6E9MP40S:041F:JUNK 0
20190925 113819.411 - 2 0 6 44 0
20190925 113844.047 SB_ENABLED
20190925 113844.426 SB_LEVELS
20190925 113947.302 SB_LEVELS
20190925 114001.889 SB_LEVELS
20190925 114459.527 SB_ENABLED
20190925 114542.764 SB_ENABLED
20190925 114626.933 SB_ENABLED
20190925 121338.159 SB_ENABLED
20190925 121338.160 SB_CLASSIFY 1633148
20190925 121338.162 Size: 4184
20190925 121338.237 - 1 0 1 72 0
20190925 121338.238 Tokens: 97
20190925 121338.238 Probability: 0.0000
20190925 161822.671 SB_ENABLED
20190925 161822.722 SB_CLASSIFY 1633148
20190925 161822.762 Size: 11803
20190925 161823.160 - 95 1 3 280 0
20190925 161823.160 Tokens: 132
20190925 161823.173 Probability: 0.0000
20190925 164535.156 SB_ENABLED
20190925 164535.159 SB_CLASSIFY 1633148
20190925 164535.161 Size: 10132
20190925 164535.216 - 3 12 1 23 0
20190925 164535.216 Tokens: 123
20190925 164535.217 Probability: 0.0000
20190925 171951.020 SB_ENABLED
20190925 171955.693 SB_LEVELS
20190925 171959.132 SB_LEVELS
20190925 172041.035 SB_LEVELS
20190925 172140.183 SB_ENABLED
20190925 172141.332 SB_LEVELS
20190925 172412.683 SB_ENABLED
20190925 172412.954 SB_LEVELS
20190925 182648.868 SB_ENABLED
20190925 182648.882 SB_ENABLED
20190925 182648.883 SB_CLASSIFY 45254928
20190925 182648.899 Size: 19229
20190925 182649.156 - 19 0 6 228 0
20190925 182649.157 Tokens: 240
20190925 182649.157 Probability: 0.8308
20190925 182649.743 SB_ENABLED
20190925 182649.745 SB_ENABLED
20190925 182649.745 SB_CLASSIFY 45254688
20190925 182649.746 Size: 16007
20190925 182649.796 - 4 0 6 39 0
20190925 182649.796 Tokens: 176
20190925 182649.797 Probability: 0.0000
20190925 182650.366 SB_ENABLED
20190925 182650.368 SB_ENABLED
20190925 182650.368 SB_CLASSIFY 45254448
20190925 182650.369 Size: 10607
20190925 182650.417 - 2 1 1 44 0
20190925 182650.418 Tokens: 133
20190925 182650.418 Probability: 0.0000
20190925 183218.067 SB_ENABLED
20190925 183218.068 SB_CLASSIFY 1633148
20190925 183218.069 Size: 12667
20190925 183218.195 - 110 0 1 15 0
20190925 183218.195 Tokens: 81
20190925 183218.196 Probability: 0.0000
20190925 184736.395 SB_ENABLED
20190925 184736.641 SB_LEVELS
20190925 184736.809 SB_LEVELS
20190925 184809.816 SB_ENABLED
20190925 184809.975 SB_LEVELS
20190925 184810.090 SB_LEVELS
20190925 184845.558 SB_ENABLED
20190925 184845.827 SB_LEVELS
20190925 184846.526 SB_LEVELS
20190925 184902.099 SB_LEVELS
20190925 185109.313 SB_ENABLED
20190925 185109.325 SB_MOVE 87314848 6E9MP40S:041F:JUNK 7C665252:C:
\PMAIL\MAIL\No:NEW 0
20190925 185109.413 - 2 0 6 68 0
20190925 190007.392 SB_ENABLED
20190925 190007.393 SB_CLASSIFY 1634256
20190925 190007.394 Size: 19229
20190925 190007.470 - 2 1 14 58 0
20190925 190007.473 Tokens: 240
20190925 190007.475 Probability: 0.0000
20190925 190207.825 SB_ENABLED
20190925 190207.991 SB_LEVELS
20190925 190208.168 SB_LEVELS
20190925 190249.441 SB_MOVE 87314848 New mail 692C6AD9:3796:FOL0740C
0
20190925 190816.180 SB_ENABLED
20190925 190816.185 SB_CLASSIFY 1633148
20190925 190816.240 Size: 79541
20190925 190816.340 - 18 0 23 51 0
20190925 190816.341 Tokens: 142
20190925 190816.341 Probability: 0.0000
20190926 124749.668 W_STARTUP
20190926 124749.764 C:\PMAIL\MAIL\german\
20190926 124750.316 SB_ENABLED
20190926 124750.913 SB_ENABLED
20190926 124750.915 SB_ENABLED
20190926 124751.069 SB_ENABLED
20190926 124754.531 SB_ENABLED
20190926 124754.954 SB_ENABLED
20190926 125323.826 SB_ENABLED
20190926 125323.828 SB_CLASSIFY 1633148
20190926 125323.829 Size: 7869
20190926 125325.789 - 3 0 1 1956 0
20190926 125325.790 Tokens: 71
20190926 125325.791 Probability: 0.0004
20190926 125330.131 SB_ENABLED
20190926 125330.132 SB_CLASSIFY 1633148
20190926 125330.133 Size: 8182
20190926 125331.975 - 3 0 0 1840 0
20190926 125332.012 Tokens: 61
20190926 125332.014 Probability: 0.0000
20190926 125336.525 SB_ENABLED
20190926 125336.527 SB_CLASSIFY 1633148
20190926 125336.528 Size: 11320
20190926 125338.516 - 3 1 1 1982 0
20190926 125338.517 Tokens: 168
20190926 125338.517 Probability: 0.0000
20190926 125340.209 SB_ENABLED
20190926 125340.210 SB_CLASSIFY 1633148
20190926 125340.210 Size: 9986
20190926 125340.884 - 3 0 1 669 0
20190926 125340.885 Tokens: 107
20190926 125340.885 Probability: 0.0000
20190926 125344.454 SB_ENABLED
20190926 125344.455 SB_CLASSIFY 1633148
20190926 125344.455 Size: 4272
20190926 125344.471 - 2 0 0 13 0
20190926 125344.472 Tokens: 71
20190926 125344.472 Probability: 0.0004
20190926 125348.776 SB_ENABLED
20190926 125348.779 SB_CLASSIFY 1633148
20190926 125348.780 Size: 11245
20190926 125349.764 - 4 1 1 977 0
20190926 125349.765 Tokens: 136
20190926 125349.765 Probability: 0.0000
20190926 125415.572 SB_ENABLED
20190926 125415.573 SB_CLASSIFY 1633148
20190926 125415.574 Size: 68371
20190926 125423.656 - 5 0 5 8070 0
20190926 125423.656 Tokens: 254
20190926 125423.657 Probability: 0.0000
20190926 125427.973 SB_ENABLED
20190926 125427.974 SB_CLASSIFY 1633148
20190926 125427.975 Size: 43854
20190926 125433.066 - 3 0 4 5084 0
20190926 125433.067 Tokens: 228
20190926 125433.067 Probability: 1.0000
20190926 130301.033 SB_SETUP 0
20190926 130327.605 SB_SETUP 0

brightside

unread,
Sep 26, 2019, 2:24:24 PM9/26/19
to
On Thu, 26 Sep 2019 13:10:19 -0300, Euler German
Thanks. Pegasus mail here is a two user mail system. The other user,
my wife stopped using it over a year ago, hence I have copied her
word4.db3 file, date 25082018 ( when the spamhalter was definitely
working), to my C\PMAIL\MAIL\aml directory. Is this action
satisfactory?

Spamhalter is enabled and train on smaller database selected.

Now it is wait and see for a couple of days or so.

--
brightside S9

Euler German

unread,
Sep 27, 2019, 8:56:01 AM9/27/19
to

On article <c20qoeh80g32aq2qe...@4ax.com>, brightside
wrote (at least in part):

> Thanks. Pegasus mail here is a two user mail system. The other user,
> my wife stopped using it over a year ago, hence I have copied her
> word4.db3 file, date 25082018 ( when the spamhalter was definitely
> working), to my C\PMAIL\MAIL\aml directory. Is this action
> satisfactory?
>
Indeed better. I'm pretty much sure you and your wife have the same
taste of what is ham and spam. ;-)

> Spamhalter is enabled and train on smaller database selected.
>
That is fine. I use Pegasus Mail quick actions (Folders > Quick
open... > Define quick actions...) to deal with SpamHalter. That way,
first action (Ctrl+1) moves messages into "Suspicious or junk mail"
folder telling SpamHalter they are evil. OTOH fourth action (Ctrl+4)
moves out messages from "Suspicious or junk mail" to an outside
folder. You can use "New mail folder" for this, considering that if
message is marked "Unread" it will be taken care (auto-magically) by
your New mail filtering rules as they are arriving from a POP3
session. If message is marked "Read" it will rest on your NMF waiting
for your manual action.

Moving messages in to and out of "Suspicious or junk mail" is the
best and most effective way to train SpamHalter, as stated by its
author Lukas Gebauer. Of course marking them as this or that either
from the traffic light button or context menu will also do, but as
Lukas told me once, moving them add extra emphasis on training and so
SpamHalter will respond faster to that.

> Now it is wait and see for a couple of days or so.
>
>
I hope you get it sorted fast.

brightside

unread,
Sep 29, 2019, 6:14:26 AM9/29/19
to
Thanks for your help. It looks like a part of the problem is
addresses in the Global whitelist which shouldn't be there. Deleting
individual entries will take many hours. Is there way to delete all
entries with "one stroke of the prn" (enter key ! ).

I am prepared to start again with a balnk whitelist

Thanks,

--
brightside S9

Euler German

unread,
Sep 29, 2019, 10:20:49 AM9/29/19
to

On article <nn01pe17q5kgppa7k...@4ax.com>, brightside
YW

> It looks like a part of the problem is
> addresses in the Global whitelist which shouldn't be there. Deleting
> individual entries will take many hours. Is there way to delete all
> entries with "one stroke of the prn" (enter key ! ).
>
Long time ago when I used a Bayesian classifier for the first time
(POPFile) I realized that either whitelists and blacklists should be
abandoned for good as it would compromise the more effective filter
-- the Bayesian one. That said, blacklists, whitelists, content
control, etc are all turned OFF here in favor of SpamHalter. Sorry if
I didn't advise this at first time but that's because it is taken as
mandatory here. My bad.

I thought once to use this white/blacklist's entries as a form or
SpamHalter auto-training but again I realized that it would foul my
corpus by the same reason: addresses are easily spoofed.

Anyway, these list are plain text files. Here they are named
BLACK.pml, WHITE.pml, and WHITE.pm. Look at your home mailbox folder
after them. Those with the .PML extension have a title line like
\TITLE Junkmail Whitelist or \TITLE Junkmail Blacklist.

> I am prepared to start again with a balnk whitelist
>
Do that. I'm sure SpamHalter training won't take much longer and
after that it will spot ham and spam very accurately. The use of
Quick Actions make the task yet easier.

> Thanks,

You're very welcome.

brightside

unread,
Sep 30, 2019, 5:30:22 AM9/30/19
to
On Sun, 29 Sep 2019 11:20:47 -0300, Euler German
Latest update. I never use a blacklist, I just use filters to sort
incoming mail to wherever I want it, including 'delete'. White list
cleared out and number of records set to zero (help says that stops
the white list being used, so outgoing mail does not add an entry).

Now incoming mail triggers spamhalter. So all I have to do now is
check all incoming mail and train spamhalter on false positives and
negatives. Not a problem.

Thanks for all your help.

--
brightside S9,

Euler German

unread,
Sep 30, 2019, 8:54:51 AM9/30/19
to

On article <gdi3pe928anav205f...@4ax.com>, brightside
wrote (at least in part):

>
> Latest update. I never use a blacklist, I just use filters to sort
> incoming mail to wherever I want it, including 'delete'. White list
> cleared out and number of records set to zero (help says that stops
> the white list being used, so outgoing mail does not add an entry).
>
> Now incoming mail triggers spamhalter. So all I have to do now is
> check all incoming mail and train spamhalter on false positives and
> negatives. Not a problem.

Great news! Glad it sorted out. Anyways, maybe you should also try
either the PM-WIN mailing list or the Community Forum. I think
there's not many Pmail people here on the Usenet. I'm fond of the
mailing list but David Harris and the Beta Testers are pushing hard
for the forum. Many of them are today forum-only users.

> Thanks for all your help.

You're always welcome.
Reply all
Reply to author
Forward
0 new messages