Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.

Dismiss

100% CPU on amavisd childs on with amavis 2.6.0 / amavis-milter 1.5.0

20 views

Skip to first unread message

david touzeau

unread,

Apr 15, 2011, 11:36:50 AM4/15/11

Dear

I'm using amavis 2.6.0 and amavisd-milter 1.5.0 on Postfix 2.8.2 with
libmilter on i386
each time amavisd child hang to 100% CPU
By googlize have made severals parameters in order to reduce the CPU
hang but now way, child come up to 100% cpu
I have reduced Spamassassin plugins that trying to check DNS and
internet connexion.
But the problem still here

where i'm wrong ?

Here it is the Spamassassin local.cf (/etc/spamassassin/local.cf)
=============================================================================================
report_safe 1
lock_method flock
required_score 5.0
use_bayes 1
bayes_auto_learn 1
skip_rbl_checks 1
auto_whitelist_file_mode 0666
auto_whitelist_path /etc/spamassassin/auto_whitelist
header KEYWORD_RULE_1 Subject =~ /MACHROUR/i
score KEYWORD_RULE_1 7
header MAX_RPCPT_1 To =~ /(.*?(@)domain\.ma){50,}/i
header MAX_CC_1 Cc =~ /(.*?(@)domain\.ma){50,}/i
score MAX_CC_1 10
describe MAX_RPCPT_1 Sent to 50+ one.ma recipients
describe MAX_CC_1 Sent to 50+ Cc one.ma recipients
score MAX_RPCPT_1 10
rewrite_header Subject ***** SPAM *****
bayes_ignore_header X-Bogosity
bayes_ignore_header X-Spam-Flag
bayes_ignore_header X-Spam-Status
header DSPAM_SPAM X-DSPAM-Result =~ /^Spam$/
describe DSPAM_SPAM DSPAM claims it is spam
score DSPAM_SPAM 0.5
header DSPAM_HAM X-DSPAM-Result =~ /^Innocent$/
describe DSPAM_HAM DSPAM claims it is ham
score DSPAM_HAM -0.

and the loaded plugins
loadplugin Mail::SpamAssassin::Plugin::Check
loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch
loadplugin Mail::SpamAssassin::Plugin::URIDetail
loadplugin Mail::SpamAssassin::Plugin::Bayes
loadplugin Mail::SpamAssassin::Plugin::BodyEval
loadplugin Mail::SpamAssassin::Plugin::HTMLEval
loadplugin Mail::SpamAssassin::Plugin::HeaderEval
loadplugin Mail::SpamAssassin::Plugin::MIMEEval
loadplugin Mail::SpamAssassin::Plugin::RelayEval
loadplugin Mail::SpamAssassin::Plugin::URIEval
loadplugin Mail::SpamAssassin::Plugin::WLBLEval
loadplugin Mail::SpamAssassin::Plugin::ImageInfo
loadplugin Mail::SpamAssassin::Plugin::AWL
loadplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold
loadplugin Mail::SpamAssassin::Plugin::WhiteListSubject
loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody
loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch
loadplugin HitFreqsRuleTiming /etc/spamassassin/HitFreqsRuleTiming.pm

Here it is the amavisd.conf
=============================================================================================
use strict;

# Configuration file with artica builder 2011-04-15 16::30:05
# COMMONLY ADJUSTED SETTINGS:

# PERFORMANCES:
$max_servers = 15;
$child_timeout=8*60;
$max_requests=10;
$pid_file='/var/spool/postfix/var/run/amavisd-new/amavisd-new.pid';
$daemon_user = 'postfix';
$daemon_group = 'postfix';
$TEMPBASE = "$MYHOME/tmp"; # working directory, needs to exist, -T
$ENV{TMPDIR} = $TEMPBASE; # environment variable TMPDIR, used by SA,
etc.
$QUARANTINEDIR = '/var/virusmails'; # -Q
$log_level = 1;# verbosity 0..5, -d
$log_recip_templ = undef; # disable by-recipient level-0 log entries
$DO_SYSLOG = 1; # log via syslogd (preferred)
$syslog_facility = 'mail'; # Syslog facility as a string
$syslog_priority = 'debug'; # Syslog base (minimal) priority as a
string,
#$LOGFILE = "/var/log/amavis/amavis.log";
$MAXLEVELS = 14;
$MAXFILES = 1500;
$MIN_EXPANSION_QUOTA = 100*1024; # bytes (default undef, not
enforced)
$MAX_EXPANSION_QUOTA = 300*1024*1024; # bytes (default undef, not
enforced)

$enable_db = 0; # enable use of BerkeleyDB/libdb (SNMP and
nanny)
$enable_global_cache = 0; # enable use of libdb-based cache if
$enable_db=1
$nanny_details_level = 2; # nanny verbosity: 1: traditional, 2:
detailed
$enable_dkim_verification = 0;
$enable_dkim_signing = 0;
$enable_ldap = 0;

@local_domains_maps = ( ['one.ma','one.org.ma'] ); # list of all local
domains
@mynetworks = qw(192.168.3.0/24);

$unix_socketname =
"/var/spool/postfix/var/run/amavisd-new/amavisd-new.sock"; #
amavisd-release or amavis-milter
$inet_socket_port = 10024; # listen on this local TCP port(s)
$mydomain = "one.ma"; # domain name
$myhostname = "smtp.one.ma"; # must be a fully-qualified domain name!
#adding net acl 192.168.3.8 127.0.0.1 of all interfaces prevent DENIED
ACCESS from IP 172.16.X.YYY
@inet_acl = qw(192.168.3.8 127.0.0.1);
$forward_method ='pipe:flags=q argv=/usr/sbin/sendmail -Ac -i -f
${sender} -- ${recipient}';
$notify_method = $forward_method;

$policy_bank{'MYNETS'} = { # mail originating from @mynetworks
originating => 1, # is true in MYNETS by default, but let's make it
explicit
os_fingerprint_method => undef, # don't query p0f for internal clients
};

$interface_policy{'SOCK'} = 'AM.PDP-SOCK'; # only applies with
$unix_socketname

# Use with amavis-release over a socket or with Petr Rehor's
amavis-milter.c
# (with amavis-milter.c from this package or old amavis.c client use
'AM.CL'):
$policy_bank{'AM.PDP-SOCK'} = {
protocol => 'AM.PDP',
notify_method => 'pipe:flags=q argv=/usr/sbin/sendmail -Ac -i -f
${sender} -- ${recipient}', auth_required_release => 0, # do not
require secret_id for amavisd-release
};

# Use in combination with dedicated artica-filters
$policy_bank{'killAll'} = {protocol => 'AM.PDP',final_destiny_by_ccat =>
{ (CC_CATCHALL) => D_DISCARD }};
$sa_tag_level_deflt = -999; # add spam info headers if at, or above
that level
$sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that
level
$sa_tag3_level_deflt = 10.0; # add 'spam detected' headers at that
level
$sa_kill_level_deflt = 15; # triggers spam evasive actions (e.g. blocks
mail)
$sa_dsn_cutoff_level = 9; # spam level beyond which a DSN is not sent
$sa_crediblefrom_dsn_cutoff_level = 18; # likewise, but for a likely
valid From
$sa_quarantine_cutoff_level = 25; # spam level beyond which quarantine
is off
#$penpals_bonus_score = 8; # (no effect without a @storage_sql_dsn
database)
#$penpals_threshold_high = $sa_kill_level_deflt; # don't waste time on
hi spam
$bounce_killer_score = 100; # spam score points to add for joe-jobbed
bounces

$sa_configpath='/etc/spamassassin';
$sa_siteconfigpath='/etc/spamassassin/local.cf';
$sa_mail_body_size_limit = 400*1024; # don't waste time on SA if mail is
larger
$sa_local_tests_only = 0; # only tests which do not require internet
access?
#$sa_spam_subject_tag = ***SPAM*** _SCORE_ (_REQD_)
@spam_subject_tag_maps = undef;
@spam_subject_tag2_maps = ('***SPAM*** _SCORE_ (_REQD_)');
@spam_subject_tag3_maps =('***SPAM*** _SCORE_ (_REQD_)');

$timestamp_fmt_mysql = 1;

$virus_admin = "undef"; # notifications recip.
$mailfrom_notify_admin = "root\@localhost.localdomain"; #
notifications sender
$mailfrom_notify_recip = "root\@localhost.localdomain"; #
notifications sender
$mailfrom_notify_spamadmin = "root\@localhost.localdomain"; #
notifications sender
$mailfrom_to_quarantine = ''; # null return path; uses original sender
if undef

$smtpd_recipient_limit=50;

$enable_dkim_signing = 0;
# WHITELISTING, list of whitelisted senders...
@whitelist_sender_maps = ( new_RE(

qr'.*@agencedusud\.gov\.ma$'i
, qr'.*@artemis\.ma$'i
, qr'.*@attijariwafa\.com$'i
, qr'.*@aui\.ma$'i
, qr'.*@barid\.oncf\.ma$'i
, qr'.*@bmcebank\.co\.ma$'i
, qr'.*@bnpparibas\.com$'i
, qr'.*@brams\.net$'i
, qr'.*@ca-cdm\.ma$'i
, qr'.*@capitalconsulting\.com$'i
, qr'.*@capitalconsulting\.fr$'i
, qr'.*@capitalconsulting\.ma$'i
, qr'.*@cas\.apd\.ma$'i
, qr'.*@cfcim\.org$'i
, qr'.*@chadbourne\.com$'i
, qr'.*@cnesten\.org\.ma$'i
, qr'.*@colgp\.it$'i
, qr'.*@cpm\.co\.ma$'i
, qr'.*@creditagricole\.ma$'i
, qr'.*@cspservices\.de$'i
, qr'.*@dbvib\.com$'i
, qr'.*@devoteam\.com$'i
, qr'.*@ec\.europa\.eu$'i
, qr'.*@edison\.it$'i
, qr'.*@es\.abb\.com$'i
, qr'.*@finances\.gov\.ma$'i
, qr'.*@fr\.rolandbeger\.com$'i
, qr'.*@garradhassan\.com$'i
, qr'.*@gide\.com$'i
, qr'.*@h2energy-maroc\.com$'i
, qr'.*@hip\.lu$'i
, qr'.*@hp\.com$'i
, qr'.*@iam\.ma$'i
, qr'.*@intelcom\.co\.ma$'i
, qr'.*@inwi\.ma$'i
, qr'.*@ipplc\.ae$'i
, qr'.*@lazrak\.ma$'i
, qr'.*@ma\.rolandberger\.com$'i
, qr'.*@mecamidi\.com$'i
, qr'.*@meditel\.ma$'i
, qr'.*@munisys\.net\.ma$'i
, qr'.*@nareva-ona\.com$'i
, qr'.*@net2s\.ma$'i
, qr'.*@nexans\.com$'i
, qr'.*@ocpgroup\.ma$'i
, qr'.*@onep\.org\.ma$'i
, qr'.*@pop3\.amadeus\.net$'i
, qr'.*@ree\.es$'i
, qr'.*@royalairmaroc\.com$'i
, qr'.*@saaidihdid\.com$'i
, qr'.*@sap\.com$'i
, qr'.*@scania\.co\.ma$'i
, qr'.*@senelec\.sn$'i
, qr'.*@siemens\.com$'i
, qr'.*@simmons-simmons\.com$'i
, qr'.*@snep\.ma$'i
, qr'.*@socgen\.com$'i
, qr'.*@sonelgaz\.dz$'i
, qr'.*@wana\.ma$'i
, qr'.*@yahoo\.fr$'i

));

@addr_extension_virus_maps = ('virus');
@addr_extension_banned_maps = ('banned');
@addr_extension_spam_maps = ('spam');
@addr_extension_bad_header_maps = ('badh');
# $recipient_delimiter = '+'; # undef disables address extensions
altogether
# when enabling addr extensions do also Postfix/main.cf:
recipient_delimiter=+

$path =
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/sbin:/usr/bin:/bin:/usr/share/artica-postfix/bin';
$dspam = '';
$defang_virus = 1; # MIME-wrap passed infected mail
$defang_banned = 1; # MIME-wrap passed mail containing banned name
# for defanging bad headers only turn on certain minor contents
categories:
$defang_by_ccat{+CC_BADH.",3"} = 1; # NUL or CR character in header
$defang_by_ccat{+CC_BADH.",5"} = 1; # header line longer than 998
characters
$defang_by_ccat{+CC_BADH.",6"} = 1; # header field syntax error

$final_virus_destiny = D_DISCARD;
$final_banned_destiny = D_BOUNCE;
$final_spam_destiny = D_BOUNCE;
$final_bad_header_destiny = D_PASS;
$mailfrom_to_quarantine = undef;
$spam_quarantine_method = 'local:spam-%m.gz';
$defang_by_ccat{CC_SPAMMY, 1};
$quarantine_method_by_ccat{+CC_SPAMMY} = $spam_quarantine_method;
$quarantine_to_maps_by_ccat{CC_SPAMMY} = \@spam_quarantine_to_maps;
$final_destiny_by_ccat{+CC_SPAMMY} =$final_spam_destiny;
$defang_by_ccat{+CC_SPAMMY.",1"} = 1;
$quarantine_method_by_ccat{CC_SPAMMY.",1"} = $spam_quarantine_method;
$quarantine_to_maps_by_ccat{CC_SPAMMY.",1"} = \@spam_quarantine_to_maps;
$final_destiny_by_ccat{+CC_SPAMMY.",1"} =$final_spam_destiny;
$bad_header_quarantine_method = undef;

# $os_fingerprint_method = 'p0f:*:2345'; # to query p0f-analyzer.pl

#Sender Notifications
$warnbadhsender = 0;
$warnvirussender = undef;
$warnspamsender = undef;
$warnbannedsender = undef;

#Recipient Notifications
$warnvirusrecip = 0;
$warnbannedrecip = 0;
$warnbadhrecip = 0;

#Admin Notifications
$newvirus_admin = undef;
$virus_admin = undef;
$spam_admin = undef;
$bad_header_admin = undef;

#templates Notifications
$notify_sender_templ =
read_text("/usr/local/etc/amavis/template-dsn.txt");
$notify_virus_sender_templ=
read_text("/usr/local/etc/amavis/template-virus-sender.txt");
$notify_virus_admin_templ =
read_text("/usr/local/etc/amavis/template-virus-admin.txt");
$notify_virus_recips_templ=
read_text("/usr/local/etc/amavis/template-virus-recipient.txt");
$notify_spam_sender_templ =
read_text("/usr/local/etc/amavis/template-virus-sender.txt");
$notify_spam_admin_templ =
read_text("/usr/local/etc/amavis/template-spam-admin.txt");

# REMAINING IMPORTANT VARIABLES ARE LISTED HERE BECAUSE OF LONGER
ASSIGNMENTS

@keep_decoded_original_maps = (new_RE(
# qr'^MAIL$', # retain full original message for virus checking (can
be slow)
qr'^MAIL-UNDECIPHERABLE$', # recheck full mail if it contains
undecipherables (enable by EnableScanSecurity)
qr'^(ASCII(?! cpio)|text|uuencoded|xxencoded|binhex)'i,
# qr'^Zip archive data', # don't trust Archive::Zip
));

$banned_filename_re=undef;

# See http://support.microsoft.com/default.aspx?scid=kb;EN-US;q262631
# and http://www.cknow.com/vtutor/vtextensions.htm

# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING

@score_sender_maps = ({

read_hash("/usr/local/etc/sender_scores_sitewide"),
{
'nob...@cert.org' => -3.0,
'cert-a...@us-cert.gov' => -3.0,
'owner...@iss.net' => -3.0,
'slas...@slashdot.org' => -3.0,
'securityfocus.com' => -3.0,
'ntbu...@listserv.ntbugtraq.com' => -3.0,
'securit...@linuxsecurity.com' => -3.0,
'mailman-ann...@python.org' => -3.0,
'amavis-u...@lists.sourceforge.net'=> -3.0,
'amavis-us...@lists.sourceforge.net' => -3.0,
'spamassassin.apache.org' => -3.0,
'notificat...@lists.sophos.com' => -3.0,
'owner-pos...@postfix.org' => -3.0,
'owner-postf...@postfix.org' => -3.0,
'owner-sendm...@lists.sendmail.org' => -3.0,
'sendmail-ann...@lists.sendmail.org' => -3.0,
'donot...@sendmail.org' => -3.0,
'ca+en...@sendmail.org' => -3.0,
'nor...@freshmeat.net' => -3.0,
'owner-t...@postel.acm.org' => -3.0,
'ietf-12...@loki.ietf.org' => -3.0,
'cvs-commits...@gnome.org' => -3.0,
'rt-user...@lists.fsck.com' => -3.0,
'clp-r...@comp.nus.edu.sg' => -3.0,
'surveys...@lists.nua.ie' => -3.0,
'emai...@genomeweb.com' => -5.0,
'yahoo-d...@yahoo-inc.com' => -3.0,
'returns.groups.yahoo.com' => -3.0,
'clust...@linuxnetworx.com' => -3.0,
lc('lvs-use...@LinuxVirtualServer.org') => -3.0,
lc('owner-textb...@CNNIMAIL12.CNN.COM') => -5.0,
'sen...@example.net' => 3.0,
'.example.net' => 1.0,

},
],
});

@decoders = (
['mail', \&do_mime_decode],
['asc', \&do_ascii],
['uue', \&do_ascii],
['hqx', \&do_ascii],
['ync', \&do_ascii],
['F', \&do_uncompress, ['unfreeze','freeze -d','melt','fcat'] ],
['Z', \&do_uncompress, ['uncompress','gzip -d','zcat'] ],
['gz', \&do_uncompress, 'gzip -d'],
['gz', \&do_gunzip],
['bz2', \&do_uncompress, 'bzip2 -d'],
['lzo', \&do_uncompress, 'lzop -d'],
['rpm', \&do_uncompress, ['rpm2cpio.pl','rpm2cpio'] ],
['cpio', \&do_pax_cpio, ['pax','gcpio','cpio'] ],
['tar', \&do_pax_cpio, ['pax','gcpio','cpio'] ],
['deb', \&do_ar, 'ar'],
# ['a', \&do_ar, 'ar'], # unpacking .a seems an overkill
['zip', \&do_unzip],
['7z', \&do_7zip, ['7zr','7za','7z'] ],
['rar', \&do_unrar, ['rar','unrar'] ],
['arj', \&do_unarj, ['arj','unarj'] ],
['arc', \&do_arc, ['nomarch','arc'] ],
['zoo', \&do_zoo, ['zoo','unzoo'] ],
['lha', \&do_lha, 'lha'],
# ['doc', \&do_ole, 'ripole'],
['cab', \&do_cabextract, 'cabextract'],
['tnef', \&do_tnef_ext, 'tnef'],
['tnef', \&do_tnef],
# ['sit', \&do_unstuff, 'unstuff'], # broken/unsafe decoder
['exe', \&do_executable, ['rar','unrar'], 'lha', ['arj','unarj'] ],
);

@av_scanners = (
['ClamAV-clamd',\&ask_daemon, ["CONTSCAN {}",
"/var/run/clamav/clamav.sock"],qr/\bOK$/, qr/\bFOUND$/,qr/^.*?: (?!
Infected Archive)(.*) FOUND$/m ]
);
@av_scanners_backup = ();
1;

=============================================================================================
here it is the top command you will see 2 child on 100% CPU and one
using 35%
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11591 postfix 39 19 83684 68m 3668 R 100 2.2 1:47.59
amavisd
28300 postfix 39 19 86048 70m 3640 R 100 2.3 12:48.36
amavisd
11593 postfix 39 19 86736 71m 3788 S 35 2.3 0:07.41
amavisd

20919 mailflt3 20 0 38768 9544 5284 S 3 0.3 0:00.09
ap-mailfilter
30318 root 39 19 47024 13m 6520 S 2 0.5 2:04.84
php5
2255 mailflt3 20 0 217m 117m 588 S 1 3.9 711:59.75
ap-process-serv
3803 postfix 39 19 167m 117m 6264 S 1 3.9 1:13.24
clamd
1941 postfix 39 19 8360 3260 1892 S 1 0.1 0:00.30
cleanup
19495 mailflt3 20 0 59536 6908 696 S 1 0.2 0:19.95
kas-milter
6470 root 39 19 1868 576 472 S 0 0.0 3:33.81
vnstatd
20912 root 20 0 2576 1172 812 R 0 0.0 0:00.02
top
21024 root 39 19 42664 12m 6196 S 0 0.4 0:05.61
php5
27436 postfix 39 19 51136 1008 720 S 0 0.0 0:01.19
amavisd-milter
30685 postfix 39 19 12732 4284 2888 S 0 0.1 0:00.20
smtpd
1 root 20 0 2032 656 564 S 0 0.0 1:48.80
init

0 new messages