Attempting to use local fuzzy storage

[Rob Gunther]

I am setting up local fuzzy storage following these instructions.

Trying to populate it with just a single test message like this:

rspamc -S FUZZY_DENIED -w 10 fuzzy_add spam.eml

The reply I get back is:

HTTP error: 403, Message is conditionally skipped for flag 1

The rspamd log says this about my attempt:

... <e7dca4>; csession; rspamd_controller_check_password: allow unauthorized connection from a trusted IP ::1

... <9d74c6>; task; rspamd_message_parse: loaded message; id: <CA...@mail.wdg.com>; queue-id: <undef>; size: 18590; checksum: <04f9...d13>

... <9d74c6>; task; fuzzy_process_handler: skip rule local as it has no flag 1 defined false

From that I assume I am connecting from a trusted IP v6 address, it received the actual message and was able to parse it because it got the message id.

Then it says it was skipped because it has no flag, that I have no clue.

Here is my 'local' snippet of code from /etc/rspamd/local.d/fuzzy_check.conf

rule "local" {

    # Fuzzy storage server list

    servers = "localhost:11335";

    # Default symbol for unknown flags

    symbol = "LOCAL_FUZZY_UNKNOWN";

    # Additional mime types to store/check

    mime_types = ["application/*"];

    # Hash weight threshold for all maps

    max_score = 20.0;

    # Whether we can learn this storage

    read_only = no;

    # Ignore unknown flags

    skip_unknown = yes;

    # Hash generation algorithm

    algorithm = "mumhash";

    # Map flags to symbols

    fuzzy_map = {

        LOCAL_FUZZY_DENIED {

            # Local threshold

            max_score = 20.0;

            # Flag to match

            flag = 11;

        }

        LOCAL_FUZZY_PROB {

            max_score = 10.0;

            flag = 12;

        }

        LOCAL_FUZZY_WHITE {

            max_score = 2.0;

            flag = 13;

        }

    }

# Fuzzy check plugin configuration snippet

learn_condition = <<EOD

return function(task)

  return true -- Always learn

end

EOD;

}

I assumed the learn_condition would accept all mail, regardless of what it was an add it to the fuzzy hash system.

Can anyone spot what I've done wrong?

Rob

[Alexander Moisseev]

On 7/22/2017 10:07 AM, Rob Gunther wrote:
> I am setting up local fuzzy storage following these instructions <https://rspamd.com/doc/fuzzy_storage.html>.

>
> Trying to populate it with just a single test message like this:
>
> rspamc -S FUZZY_DENIED -w 10 fuzzy_add spam.eml
>

Actually you are attempting to populate "rspamd.com" fuzzy storage.

modules.d/fuzzy_check.conf

rule "rspamd.com" {
...
fuzzy_map = {
FUZZY_DENIED {
max_score = 20.0;
flag = 1;
}

>
> /rule "local" {/
> / # Fuzzy storage server list/
> / servers = "localhost:11335";/

> / # Map flags to symbols/
> / fuzzy_map = {/
> / LOCAL_FUZZY_DENIED {/
> / # Local threshold/
> / max_score = 20.0;/
> / # Flag to match/
> / flag = 11;/
> / }/

You need to use LOCAL_FUZZY_DENIED symbol or flag 11 :

rspamc -S LOCAL_FUZZY_DENIED -w 10 fuzzy_add spam.eml
rspamc -f 11 -w 10 fuzzy_add spam.eml


You can remove following configuration snippet completely if you don't need learn condition.

> /
> /# Fuzzy check plugin configuration snippet/
> /learn_condition = <<EOD/
> /return function(task)/
> / return true -- Always learn/
> /end/
> /EOD;/
> /

[Rob Gunther]

Beautiful, it works.  I managed to push in both individual messages and multiple messages in a folder.

For the fuzzy system to work correctly, so I need to feed it both ham & spam?  In about equal parts?  Getting spam messages is easy, plenty of those but legit messages are a bit more rare.

[Alexander Moisseev]

On 7/22/2017 10:56 AM, Rob Gunther wrote:
> For the fuzzy system to work correctly, so I need to feed it both ham & spam? In about equal parts? Getting spam messages is easy, plenty of those but legit messages are a bit more rare.
>

That's true for bayes, but for fuzzy it's not necessary. You can have just one fuzzy hash learned and it will work perfectly.