[Indimail-support] tcpserver unable to connect for imaps and pops daemons after update IndiMail

[Adrian Offerman]

Updated my IndiMail installation to the latest version today:
Aug 11 16:51:58 Updated: indimail-1.9.0-30.1.x86_64
(together with quite a number of other packages on CentOS 6)

Now the pop(s) and imap(s) services are no longer willing to start. I
have log messages like these for all four services:

@4000000055ca57ff10449aac tcpserver: fatal: unable to bind:
address already used

As far as I can see, however, there is nothing listening to these ports,
e.g. a command like:
netstat -plnt | grep ':993'
returns nothing.
And connecting to these ports returns a 'connection refused' message (as
expected if nothing is there).

tcpserver seems to be running OK, just seems to be unable to claim these
ports:

/service/clamd: down 2795 seconds
/service/fetchmail: up (pid 31147) 0 seconds
/service/freshclam: down 2795 seconds
/service/greylist.1999: up (pid 10768) 2795 seconds
/service/indisrvr.4000: up (pid 10751) 2795 seconds
/service/inlookup.infifo: up (pid 10756) 2795 seconds
/service/mysql.3306: up (pid 10783) 2795 seconds
/service/proxy-imapd.4143: up (pid 10695) 2795 seconds
/service/proxy-imapd-ssl.9143: up (pid 10782) 2795 seconds
/service/proxy-pop3d.4110: up (pid 10774) 2795 seconds
/service/proxy-pop3d-ssl.9110: up (pid 10757) 2795 seconds
/service/pwdlookup: up (pid 10753) 2795 seconds
/service/qmail-imapd.143: up (pid 31152) 0 seconds
/service/qmail-imapd-ssl.993: up (pid 31148) 0 seconds
/service/qmail-pop3d.110: up (pid 31139) 0 seconds
/service/qmail-pop3d-ssl.995: up (pid 31143) 0 seconds
/service/qmail-poppass.106: up (pid 10716) 2795 seconds
/service/qmail-qmqpd.628: up (pid 10743) 2795 seconds
/service/qmail-qmtpd.209: up (pid 10786) 2795 seconds
/service/qmail-send.25: up (pid 10698) 2795 seconds
/service/qmail-smtpd.25: up (pid 10752) 2795 seconds
/service/qmail-smtpd.366: up (pid 10794) 2795 seconds
/service/qmail-smtpd.465: up (pid 10779) 2795 seconds
/service/qmail-smtpd.587: up (pid 10762) 2795 seconds
/service/qmail-spamlog: up (pid 10749) 2795 seconds
/service/qscanq: up (pid 27766) 195 seconds

Repeating the tcpserver status request shows that the daemon process
number is changing every second, so tcpserver is trying to start a new
daemon that then somehow cannot connect to its designated port:

/var/indimail/bin/svstat /service/qmail-imapd-ssl.993/
/service/qmail-imapd-ssl.993/: up (pid 14889) 0 seconds
/service/qmail-imapd-ssl.993/: up (pid 15248) 1 seconds
/service/qmail-imapd-ssl.993/: up (pid 15266) 1 seconds
/service/qmail-imapd-ssl.993/: up (pid 15266) 1 seconds
/service/qmail-imapd-ssl.993/: up (pid 15308) 0 seconds
/service/qmail-imapd-ssl.993/: up (pid 15346) 0 seconds
/service/qmail-imapd-ssl.993/: up (pid 15364) 0 seconds
/service/qmail-imapd-ssl.993/: up (pid 15382) 0 seconds
/service/qmail-imapd-ssl.993/: up (pid 15401) 0 seconds
/service/qmail-imapd-ssl.993/: up (pid 15419) 0 seconds

At the receiving side (SMTP and the such) everything seems to be in order.

What further info and options to locate the problem?

Thanks!! %-)

------------------------------------------------------------------------------
_______________________________________________
Indimail-support mailing list
Indimail...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/indimail-support

[Manvendra Bhangui]

On 12 August 2015 at 02:16, Adrian Offerman <adr...@offerman.net> wrote:

  /service/qmail-imapd.143: up (pid 31152) 0 seconds
  /service/qmail-imapd-ssl.993: up (pid 31148) 0 seconds
  /service/qmail-pop3d.110: up (pid 31139) 0 seconds
  /service/qmail-pop3d-ssl.995: up (pid 31143) 0 seconds

I found a bug. I have removed xpg4. The upgrade does not take care of correcting existing supervise script. Do the following

(assuming the program id is in /usr/bin)

# sudo vi `grep -l xpg4 /service/*/run`

edit the files and change /usr/xpg4/bin/id to /usr/bin/id

--

Regards Manvendra - http://www.indimail.org

[Adrian Offerman]

Just fixed the run scripts, after checking that /usr/bin/id is available.

The same problem is still there, however.

Executing the run command by hand, shows the connection problem:

[root qmail-imapd-ssl.993]# ./run

tcpserver: fatal: unable to bind: address already used

Here's one of my run scripts:

#!/bin/sh
# $Id: svctool.in,v 2.217 2011-12-18 07:33:02+05:30 Cprogrammer Stab
mbhangui $
# generated on x86_64-unknown-linux-gnu on Tue Sep 25 12:46:35 CEST 2012
# /var/indimail/sbin/svctool --imap=993 --servicedir=/service
--localip=0 --maxdaemons=40 --maxperip=25 --query-cache
--default-domain=indimail.org --memory=209715200 --ssl

INDIUID=`/usr/bin/id -u indimail`
INDIGID=`/usr/bin/id -g indimail`

prefix=/var/indimail
bindir=${prefix}/bin

exec /var/indimail/bin/envdir /service/qmail-imapd-ssl.993/variables sh -c "
LIBAUTHMODULES=\"\"
for f in \`echo \$AUTHMODULES\`
do
LIBAUTHMODULES=\"\$LIBAUTHMODULES $prefix/libexec/authlib/\$f\"
done
exec /var/indimail/bin/softlimit -m \$SOFT_MEM -o 1024 \
/var/indimail/bin/tcpserver -v -c
/service/qmail-imapd-ssl.993/variables/MAXDAEMONS -C \$MAXPERIP \
-x /var/indimail/etc/tcp.imap.cdb -X \
-o -b \$MAXDAEMONS -H -l \$SSLADDRESS -R -u $INDIUID -g $INDIGID
\$SSLADDRESS \$SSLPORT \
$prefix/bin/couriertls -server -tcpd \
$prefix/sbin/imaplogin \$LIBAUTHMODULES $bindir/imapd Maildir 2>&1"


________

On 12 August 2015 at 02:16, Adrian Offerman wrote:

>
> /service/qmail-imapd.143: up (pid 31152) 0 seconds
> /service/qmail-imapd-ssl.993: up (pid 31148) 0 seconds
> /service/qmail-pop3d.110: up (pid 31139) 0 seconds
> /service/qmail-pop3d-ssl.995: up (pid 31143) 0 seconds

I found a bug. I have removed xpg4. The upgrade does not take care of
correcting existing supervise script. Do the following

(assuming the program id is in /usr/bin)

# sudo vi `grep -l xpg4 /service/*/run`

edit the files and change /usr/xpg4/bin/id to /usr/bin/id

[Manvendra Bhangui]

On 12 August 2015 at 14:11, Adrian Offerman <adr...@offerman.net> wrote:

Just fixed the run scripts, after checking that /usr/bin/id is available.

The same problem is still there, however.

Executing the run command by hand, shows the connection problem:

  [root qmail-imapd-ssl.993]# ./run
  tcpserver: fatal: unable to bind: address already used

Do the following

# service indimail stop

# ps -ef | egrep "tcpserver|imap|pop3"

# kill -9 the above

#cd /service/qmail-imapd-ss.993

# ./run

The above should work. After that come out of the above run script by CTRL-C and start indimail service

# service indimail start

[Manvendra Bhangui]

On 12 August 2015 at 14:22, Manvendra Bhangui <mbha...@gmail.com> wrote:

Do the following

# service indimail stop

# ps -ef | egrep "tcpserver|imap|pop3"

# kill -9 the above

#cd /service/qmail-imapd-ss.993

# ./run

The above should work. After that come out of the above run script by CTRL-C and start indimail service

# service indimail start

Has the problem got resolved? 

Do let me know

[Manvendra Bhangui]

On 12 August 2015 at 14:11, Adrian Offerman <adr...@offerman.net> wrote:

Just fixed the run scripts, after checking that /usr/bin/id is available.

The same problem is still there, however.

If you do the following, it will help me finding the root cause.

# /var/indimail/bin/svc -d /service/qmail-imapd-ssl.993

# strace -o /tmp/imap-ssl.log /service/qmail-imapd-ssl.993/run

When you see the error "Bind address already in use" come on the stdout, do CNTRL-C to quit strace

# gzip /tmp/imap-ssl.log

Then email me /tmp/imap-ssl.log.gz

[Manvendra Bhangui]

An update on the issue faced by Adrian. The issue was due to port 993
bound by portreserve program. However, none of the programs, netstat,
lsof or ss reveal which process has bound on the port 993. On doing
strace of portreserve program, it shows that it calls socket()
followed by bind(). But it does not call the listen() system call. Not
calling listen() causes you to have a socket state which cannot be
revealed by any of the existing programs like netstat, lsof or ss to
the best of my knowledge. This issue is worrisome, because i can write
a rogue program which calls socket() + bind() without calling listen()
in a while loop for all high number ports. And there will be no way
for the admins to find out which program has hijacked these ports.
This issue is seen only for TCP ports. For UDP ports, the address is
revealed and hence netstat, lsof and ss can be used to find it.

The issue has been posted to the linux kernel list.

https://lkml.org/lkml/2015/8/13/27

Adrian,

You can add the following line in /service/qmail-imapd-ssl.993/run to
release the ports
# /usr/sbin/portrelease dovecot