cannot open databases, lock failed, file exists

[Alex]

Hi,

I have a spamassassin-v3.3.2, amavisd-2.6.6 on fedora15. I'm working
on trying to do some performance tuning. During the normal course of
operation, I have the following messages in the logs:

Oct 25 13:31:36 mail02 amavis[20312]: (20312-01-30) _WARN: bayes:
cannot open bayes databases /var/spool/amavisd/.spamassassin/bayes_*
R/W: lock failed: File exists

Is this a result of having too many amavisd processes running at once?

I have a quad-core Intel Xeon processor, with the mail server running
in a kvm that is otherwise idle, and have set up a maximum of 8
amavisd processes, as well as 8 smtpd processes.

When I stopped amavisd, it didn't delete the lock files. Is this expected?

Thanks,
Alex

[Pelletier, Robert]

Yes, it won't be deleted. This is created by spamassassin when it auto-learns mail it scanned.

I'd recommend you to move this data to a MySQL database.

This article could help you do it. http://www.starbridge.org/spip/spip.php?article15

Good luck!

-----Message d'origine-----
De : amavis-users-bounces+pelletierr=csdhr...@amavis.org [mailto:amavis-users-bounces+pelletierr=csdhr...@amavis.org] De la part de Alex
Envoyé : 25 octobre 2011 13:41
À : amavis...@amavis.org
Objet : cannot open databases, lock failed, file exists

[Mark Martinec]

Alex,

> I have a spamassassin-v3.3.2, amavisd-2.6.6 on fedora15. I'm working
> on trying to do some performance tuning. During the normal course of
> operation, I have the following messages in the logs:
>
> Oct 25 13:31:36 mail02 amavis[20312]: (20312-01-30) _WARN: bayes:
> cannot open bayes databases /var/spool/amavisd/.spamassassin/bayes_*
> R/W: lock failed: File exists
>
> Is this a result of having too many amavisd processes running at once?

More likely: too many amavisd processes trying to do an auto-expiry
run at once. A workaround is to disable bayes auto-expiration
and expire tokens periodically by: sa-learn --force-expire.

Btw, you do have a 'lock_method flock' in local.cf, don't you?

A better solution is to move bayes database to SQL, as Robert
suggested.

Mark

[Alex]

Hi,

>> I have a spamassassin-v3.3.2, amavisd-2.6.6 on fedora15. I'm working
>> on trying to do some performance tuning. During the normal course of
>> operation, I have the following messages in the logs:
>>
>> Oct 25 13:31:36 mail02 amavis[20312]: (20312-01-30) _WARN: bayes:
>> cannot open bayes databases /var/spool/amavisd/.spamassassin/bayes_*
>> R/W: lock failed: File exists
>>
>> Is this a result of having too many amavisd processes running at once?
>
> More likely: too many amavisd processes trying to do an auto-expiry
> run at once. A workaround is to disable bayes auto-expiration
> and expire tokens periodically by: sa-learn --force-expire.

Yes, auto-expiry was already disabled.

> Btw, you do have a 'lock_method flock' in local.cf, don't you?

I hadn't heard of that before, so have now added it. It only happened
a few times (at least that I noticed), and only under extreme load, so
I also suspect it was a locking issue?

> A better solution is to move bayes database to SQL, as Robert
> suggested.

Yes, have now done this, but the document referenced is cut down too
much from the original, and has several problems.

To start, the schema doesn't work with current versions of mysql. Use
this one instead:

http://svn.apache.org/repos/asf/spamassassin/trunk/sql/bayes_mysql.sql

I'm also having problems with the AWL instructions from that document.
It seems I don't have an existing AWL to convert to be imported into
mysql. How do I now create and begin to use the auto-whitelist?

Thanks again,
Alex

[Mark Martinec]

Alex,

> > A better solution is to move bayes database to SQL, as Robert
> > suggested.
>
> Yes, have now done this, but the document referenced is cut down too
> much from the original, and has several problems.
>
> To start, the schema doesn't work with current versions of mysql. Use
> this one instead:
>
> http://svn.apache.org/repos/asf/spamassassin/trunk/sql/bayes_mysql.sql

Right, the bayes_mysql.sql from trunk (on its way to become a SA 3.4.0)
has an up-to-date schema, which is still compatible with SA 3.3 series.

Also the README* documents there were updated, and still compatible
with 3.3:
http://svn.apache.org/repos/asf/spamassassin/trunk/sql/

> I'm also having problems with the AWL instructions from that document.
> It seems I don't have an existing AWL to convert to be imported into
> mysql. How do I now create and begin to use the auto-whitelist?

Just start awl from scratch. Create a database (awl_mysql.sql, README.awl),
make sure it is r/w accessible to a given username, and enable it in your
local.cf, e.g.:

user_awl_sql_username vscan
user_awl_sql_password xxxxxxsecretxxxx
auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList
user_awl_dsn DBI:mysql:sa:127.0.0.1:3306

I also use:

auto_whitelist_ipv4_mask_len 24
auto_whitelist_ipv6_mask_len 48
auto_whitelist_distinguish_signed 1

auto_whitelist_factor 0.05


Mark