Writing my own New Rules?

[marc perkel]

As soon as I get bayes working and begin testing I'm going to start adding custom rules.

My question:

Should I modify the stock files or should I create new files for my own rules with the idea that the stock rules might get updated?

Thanks in advance for your help.

[Vsevolod Stakhov]

I plan to write a detailed guide tonight.

--
Vsevolod Stakhov

[marc perkel]

That would be great. Thanks.

[Vsevolod Stakhov]

https://rspamd.com/doc/tutorials/writing_rules.html

English is not my native language so please do not hesitate to point on
any mistakes found.

--
Vsevolod Stakhov

[Vsevolod Stakhov]

On 17/12/2015 16:17, Marc Perkel wrote:

> Your written English is better than most Americans. :)
>
> As soon as I get bayes working and start testing I'll probably start
> creating rules.

What's your problem with bayes then?

--
Vsevolod Stakhov

[Marc Perkel]

On 12/17/15 08:30, Vsevolod Stakhov wrote:
>
> What's your problem with bayes then?
>

Not sure what I'm doing wrong. I upgraded to version 1.0.10. I deleted
everything in the /var/lib/rspamd directory and started the server.

These are my files:

drwxr-xr-x 2 _rspamd _rspamd 4096 Dec 17 08:26 .
drwxr-xr-x 16 root root 4096 Dec 17 04:02 ..
-rw-r--r-- 1 _rspamd _rspamd 9216 Dec 16 14:27 bayes.ham.sqlite
-rw-r--r-- 1 _rspamd _rspamd 9216 Dec 16 14:27 bayes.spam.sqlite
-rw------- 1 root root 54936 Dec 17 08:26 rspamd.history
srw------- 1 root root 0 Dec 17 08:26 rspamd.sock
-rw-r--r-- 1 _rspamd _rspamd 276 Dec 17 08:26 stats.ucl
-rw-r--r-- 1 root root 56004 Dec 17 08:26 symbols.cache

It created the sqlite files initially but those files haven't changed
since I activated the server. The file dates are the time when the files
were created. The permissions look right to me. It's just not learning.

[Vsevolod Stakhov]

And what do you see in the logs?

--
Vsevolod Stakhov

[Marc Perkel]

This is the only bayes/sqlite messages in logs

2015-12-17 08:50:43 #32058(normal) <ifhyz3>; cfg; rspamd_sqlite3_init:
enable per language statistics for BAYES_SPAM
2015-12-17 08:50:43 #32058(normal) <ifhyz3>; cfg; rspamd_sqlite3_init:
enable per language statistics for BAYES_HAM

[Vsevolod Stakhov]

I don't understand. And what `rspamc stat` shows?

--
Vsevolod Stakhov

[Marc Perkel]

Results for command: stat (0.000 seconds)
Messages scanned: 52216
Messages with action reject: 4811, 9.21%
Messages with action soft reject: 0, 0.00%
Messages with action rewrite subject: 0, 0.00%
Messages with action add header: 6142, 11.76%
Messages with action greylist: 6431, 12.31%
Messages with action no action: 34832, 66.70%
Messages treated as spam: 10953, 20.97%
Messages treated as ham: 41263, 79.02%
Messages learned: 0
Connections count: 4038
Control connections count: 101
Pools allocated: 4172
Pools freed: 4135
Bytes allocated: 1M
Memory chunks allocated: 65859
Shared chunks allocated: 21
Chunks freed: 65703
Oversized chunks: 1113
Fuzzy hashes stored: 0
Fuzzy hashes expired: 0
Fuzzy hashes checked: 0 0 0 0
Fuzzy hashes found: 0 0 0 0
Statfile: BAYES_SPAM type: sqlite3; length: 9k; free blocks: 0B; total
blocks: 0B; free: 0.00%; learned: 0; users: 1; languages: 1
Statfile: BAYES_HAM type: sqlite3; length: 9k; free blocks: 0B; total
blocks: 0B; free: 0.00%; learned: 0; users: 1; languages: 1
Total learns: 0

[Vsevolod Stakhov]

Ok, and what is in the logs during learning then?

--
Vsevolod Stakhov

[Marc Perkel]

Nothing in the logs that I can see. What should I look for?

[Vsevolod Stakhov]

On 17/12/2015 17:27, Marc Perkel wrote:
>
> On 12/17/15 09:15, Vsevolod Stakhov wrote:
>> On 17/12/2015 17:09, Marc Perkel wrote:
>>> On 12/17/15 09:03, Vsevolod Stakhov wrote:
>>>> On 17/12/2015 16:53, Marc Perkel wrote:
>>>>> On 12/17/15 08:43, Vsevolod Stakhov wrote:
>>>>>> On 17/12/2015 16:36, Marc Perkel wrote:
>>>>>>> On 12/17/15 08:30, Vsevolod Stakhov wrote:
>>>>>>>> What's your problem with bayes then?

>>>>> This is the only bayes/sqlite messages in logs
>>>>>
>>>>> 2015-12-17 08:50:43 #32058(normal) <ifhyz3>; cfg; rspamd_sqlite3_init:
>>>>> enable per language statistics for BAYES_SPAM
>>>>> 2015-12-17 08:50:43 #32058(normal) <ifhyz3>; cfg; rspamd_sqlite3_init:
>>>>> enable per language statistics for BAYES_HAM
>>>>>
>>>>>
>>>> I don't understand. And what `rspamc stat` shows?
>> Ok, and what is in the logs during learning then?
>>
>
>
> Nothing in the logs that I can see. What should I look for?
>

Erm, so what are you doing, what are the errors shown, what can you find
in the logs? I don't understand what's happening.

--
Vsevolod Stakhov

[Marc Perkel]

On 12/17/15 09:33, Vsevolod Stakhov wrote:
> Erm, so what are you doing, what are the errors shown, what can you find
> in the logs? I don't understand what's happening.
>

Actually - I'm stumped. It's like it's not even trying to learn. When I
run rspamc to learn I get this:

/usr/bin/rspamc -c learn_spam >> /tmp/ham.log


Results for file: stdin (0.161 seconds)
[Metric: default]
Spam: true
Score: 25.39 / 15.00
Action: reject
Symbol: ONCE_RECEIVED_STRICT (4.00)
Symbol: JP_SURBL_MULTI (5.50)[go2buy1.com.multi.surbl.org]
Symbol: MIME_HTML_ONLY (1.00)
Symbol: SEM_URIBL_FRESH15 (3.00)[go2buy1.com.fresh15.spameatingmonkey.net]
Symbol: RBL_SENDERSCORE (2.00)
Symbol: RBL_UCEPROTECT_LEVEL1 (1.00)
Symbol: R_SPF_ALLOW (-1.10)[mx]
Symbol: RBL_SORBS_RECENT (1.50)
Symbol: ONCE_RECEIVED (1.00)
Symbol: RBL_MAILSPIKE_WORST (2.00)
Symbol: WS_SURBL_MULTI (5.50)[go2buy1.com.multi.surbl.org]
Message: (SPF): spf allow
Urls: ["shop.go2buy1.com"]
Message-ID:
3130196871317501370.354ab5995b4...@shop.go2buy1.com

But it's not learning it.

[Vsevolod Stakhov]

On 17/12/2015 18:00, Marc Perkel wrote:
>
> On 12/17/15 09:33, Vsevolod Stakhov wrote:
>> Erm, so what are you doing, what are the errors shown, what can you find
>> in the logs? I don't understand what's happening.
>>
>
> Actually - I'm stumped. It's like it's not even trying to learn. When I
> run rspamc to learn I get this:
>
> /usr/bin/rspamc -c learn_spam >> /tmp/ham.log
>

Remove '-c'

--
Vsevolod Stakhov

[Marc Perkel]

YES! It's learning! Thank you!


Question - isn't it supposed to learn automatically also from messages
it determines as very ham/spam?

[Marc Perkel]

Looks like it's pretty easy to overload the learner with parallel processes.