sendmail on debian testing

[Michael Grant]

Today I upgraded a test machine from wheezy to testing.

It seemed to install systemd, I'm not sure if it's using it or not.

One thing I noticed though was that sendmail no longer starts at boot. Even if I run:

/etc/init.d/sendmail start

or if I cd to /etc/mail and run:

make restart

or if I do this:

nothing except running 'sendmail -bd' will start sendmail.

In syslog I see this:

Jan 31 18:53:43 blah systemd[1]: Started LSB: powerful, efficient, and scalable Mail Transport Agent.

in mail.log I don't see anything when I try to start sendmail via /etc/init.d/sendmail.

I do not have the lsb-invalid-mta package installed.  I have tried reinstalling the sendmail package.  I have tried the testing and unstable versions of sendmail.

Any ideas where I should look next to figure out what's going on?

Michael Grant

[Michael Grant]

I'm still searching for an answer to this.

After upgrade from wheezy to testing, sendmail no longer starts.

I see that the system is using systemd.  I see that the /etc/init.d/sendmail script now runs /bin/systemctl start sendmail.service.  But sendmail isn't started.  Even running '/bin/systemctl start sendmail.service' manually, nothing happens.

I don't see any obvious way to get any debug info out of systemctl.  

# systemctl is-enabled sendmail

Failed to get unit file state for sendmail.service: No such file or directory

and

# /bin/systemctl enable sendmail.service

Synchronizing state for sendmail.service with sysvinit using update-rc.d...

Executing /usr/sbin/update-rc.d sendmail defaults

Executing /usr/sbin/update-rc.d sendmail enable

# systemctl is-enabled sendmail

Failed to get unit file state for sendmail.service: No such file or directory

[Reco]

Hi.

On Tue, Feb 03, 2015 at 02:03:25PM +0000, Michael Grant wrote:
> I'm still searching for an answer to this.
>
> After upgrade from wheezy to testing, sendmail no longer starts.
>
> I see that the system is using systemd.  I see that the /etc/init.d/sendmail script now runs /bin/systemctl start sendmail.service.  But sendmail isn't
> started.  Even running '/bin/systemctl start sendmail.service' manually, nothing happens.

A nessesary correction - /etc/init.d/sendmail *tries* to run
'/bin/systemctl start sendmail.service'.

But, since no sendmail* package provide systemd's service file -
nothing happens.

Such behaviour cannot be considered systemd's bug IMO - systemd simply
does what it's intended to do in this case.

But, at the same time, such behaviour can be considered as a sendmail
bug (given that systemd is Jessie's default init, and sendmail is not
starting with this init).

Still, there's a way to workaround this.

Try adding

export _SYSTEMCTL_SKIP_REDIRECT="true"

to /etc/init.d/sendmail


Reco


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: https://lists.debian.org/20150203160421.GA21852@x101h

[Michael Grant]

On Tue, Feb 3, 2015 at 4:04 PM, Reco <recov...@gmail.com> wrote:

 Hi.

On Tue, Feb 03, 2015 at 02:03:25PM +0000, Michael Grant wrote:
> I'm still searching for an answer to this.
>
> After upgrade from wheezy to testing, sendmail no longer starts.
>
> I see that the system is using systemd.  I see that the /etc/init.d/sendmail script now runs /bin/systemctl start sendmail.service.  But sendmail isn't
> started.  Even running '/bin/systemctl start sendmail.service' manually, nothing happens.

A nessesary correction - /etc/init.d/sendmail *tries* to run
'/bin/systemctl start sendmail.service'.

But, since no sendmail* package provide systemd's service file -
nothing happens.

Such behaviour cannot be considered systemd's bug IMO - systemd simply
does what it's intended to do in this case.

But, at the same time, such behaviour can be considered as a sendmail
bug (given that systemd is Jessie's default init, and sendmail is not
starting with this init).

Still, there's a way to workaround this.

Try adding

export _SYSTEMCTL_SKIP_REDIRECT="true"

to /etc/init.d/sendmail

Thanks, this is progress, I can now start sendmail by hand by running '/etc/init.d/sendmail start', but it's not starting automatically at boot time.

I don't know if this has anything to do with that:

# systemctl enable sendmail

Synchronizing state for sendmail.service with sysvinit using update-rc.d...
Executing /usr/sbin/update-rc.d sendmail defaults
Executing /usr/sbin/update-rc.d sendmail enable

# systemctl is-enabled sendmail
Failed to get unit file state for sendmail.service: No such file or directory

also, a better place to add this:

export _SYSTEMCTL_SKIP_REDIRECT="true"

to is /etc/default/sendmail and not modify /etc/init.d/sendmail.  Adding this to /etc/default/sendmail seems to work equally as well in that running '/etc/init.d/sendmail start' does manually start sendmail.

Incidentally, the sendmail package even in experimental is significantly out of date. The package appears orphaned. Several people seem to have tried to step up to do something about this but nothing has happened. Is sendmail dead on Debian?  

[Ansgar Burchardt]

Hi,

Michael Grant <mgr...@grant.org> writes:
> On Tue, Feb 3, 2015 at 4:04 PM, Reco <recov...@gmail.com> wrote:
>> A nessesary correction - /etc/init.d/sendmail *tries* to run
>> '/bin/systemctl start sendmail.service'.
>>
>> But, since no sendmail* package provide systemd's service file -
>> nothing happens.

Not true. Systemd is supposed to handle sysvinit scripts as well,
i.e. when there is no native .service file for systemd it will run the
scripts in /etc/init.d/*. This seems to not work here for some reason.

>> Try adding
>> export _SYSTEMCTL_SKIP_REDIRECT="true"
>> to /etc/init.d/sendmail
>
> Thanks, this is progress, I can now start sendmail by hand by running
> '/etc/init.d/sendmail start', but it's not starting automatically at boot
> time.
>
> I don't know if this has anything to do with that:
>
> # systemctl enable sendmail
> Synchronizing state for sendmail.service with sysvinit using update-rc.d...
> Executing /usr/sbin/update-rc.d sendmail defaults
> Executing /usr/sbin/update-rc.d sendmail enable
>
> # systemctl is-enabled sendmail
> Failed to get unit file state for sendmail.service: No such file or
> directory

That should be fine for services without a systemd .service file.

> also, a better place to add this:
>
> export _SYSTEMCTL_SKIP_REDIRECT="true"
>
> to is /etc/default/sendmail and not modify /etc/init.d/sendmail. Adding
> this to /etc/default/sendmail seems to work equally as well in that running
> '/etc/init.d/sendmail start' does manually start sendmail.

That is no surprise: at boot it's still systemd calling
/etc/init.d/sendmail so workarounds to bypass systemd don't work.

Could you try restarting sendmail (systemctl restart sendmail) and show
the output of `systemctl status sendmail'? It also shows the most recent
log entries, but the output of journalctl --unit sendmail --since -5min
might also be useful (if it shows more messages).

I tried installing sendmail on a minimal test installation and systemd
started at least one daemon ("sendmail: MTA: accepting connections"),
so at least something gets started (though it complained about the test
installation not having a FQDN so other parts might be broken and not
have started).

Ansgar

--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: https://lists.debian.org/87egq6k...@deep-thought.43-1.org

[Bob Proulx]

Michael Grant wrote:
> I'm still searching for an answer to this.
> After upgrade from wheezy to testing, sendmail no longer starts.
> I see that the system is using systemd.

> ...

Some comments that I think are relevant...

Since it took a while for someone to respond to your question it tells
me that it is a combination of tools that not many people are using.
You are using sendmail in combination with systemd. That is an
unusual combination. If it were widely used then many people would
have been responding already. Frankly you might be one of the few
trailblazing that combination.

Trailblazers are great! They are the ones who make things happen.
However not everyone wants to be a trailblazer. There just isn't
enough time for everyone to do everything. You sound like a busy
person without the time to debug everything. Perhaps it would be good
to change to a more mainstream combination? If it were a mainstream
combination then the problem would almost certainly already have been
seen and fixed. Something with a lot of users and a lot of support.

In this case I would suggest that Sendmail is no longer the mainstream
mail transfer agent. Instead I suggest migrating to Postfix. (Or
Exim but I personally really prefer Postfix so will recommend
Postfix.) Postfix is well tested and very well supported. There has
been discussion of making Postfix the default mta on Debian. (But
that will never happen because Exim isn't bad just not as popular.)

Since you are using Sendmail I assume you have been using Sendmail
forever. You probably have multiple editions of the O'Reilly Sendmail
book on your bookshelf. You probably hate to take the time to migrate
a working configuration tuned over decades to something different. I
have been there and glance over at my two remaining editions of the
O'Reilly Sendmail books on my bookshelf. Let me say that moving to
Postfix was very easy. I don't even have one copy of the O'Reilly
Postfix book. It has an easy to understand design and the online
documentation is excellent. I have been where you are with Sendmail
and migrating to Postfix was a good decision for me. I suggest that
it would be for you too. YMMV.

Good luck!

Bob

[Michael Grant]

On Tue, Feb 3, 2015 at 6:16 PM, Reco <recov...@gmail.com> wrote:

On Tue, Feb 03, 2015 at 05:31:26PM +0000, Michael Grant wrote:
> On Tue, Feb 3, 2015 at 4:04 PM, Reco <recov...@gmail.com> wrote:

...

>
>     Try adding
>
>     export _SYSTEMCTL_SKIP_REDIRECT="true"
>
>     to /etc/init.d/sendmail
>
> Thanks, this is progress, I can now start sendmail by hand by running '/etc/init.d/sendmail start', but it's not starting automatically at boot time.

An expected result, sadly (see below).

> I don't know if this has anything to do with that:
>
> # systemctl enable sendmail
> Synchronizing state for sendmail.service with sysvinit using update-rc.d...
> Executing /usr/sbin/update-rc.d sendmail defaults
> Executing /usr/sbin/update-rc.d sendmail enable
>
> # systemctl is-enabled sendmail
> Failed to get unit file state for sendmail.service: No such file or directory

No, it doesn't have anything with it.

Systemd uses it's own way to define a service called a 'service unit'.
Presumably, systemd has something for the compatibility with old init
(aka sysvinit), which *should* start those /etc/init.d/ scripts just as
good as if sysvinit itself would do it. Well, now we see how well it
works in the reality :)


Ok, let's try something different then - based on [1]. Try creating the
file called /etc/systemd/system/sendmail.service with the following
contents:

###cut###

[Unit]
Description=Sendmail Mail Transport Agent
After=syslog.target network.target
Conflicts=postfix.service exim.service

[Service]
Type=forking
PIDFile=/run/sendmail.pid
Environment=SENDMAIL_OPTS=-q1h
EnvironmentFile=-/etc/default/sendmail
ExecStartPre=-/etc/mail/make
ExecStartPre=-/etc/mail/make aliases
ExecStart=/usr/sbin/sendmail -bd $SENDMAIL_OPTS $SENDMAIL_OPTARG

[Install]
WantedBy=multi-user.target

###cut###


Revert the _SYSTEMCTL_SKIP_REDIRECT change, see how it goes now.
This unit file may require tweaking in $SENDMAIL_OPTS $SENDMAIL_OPTARG
part - I'm unable to check now what kind of variables are sourced by
/etc/default/sendmail.

Ok, I tried creating that file and removing the line from /etc/default/sendmail.  It still did not come up when the machine booted.

> Incidentally, the sendmail package even in experimental is significantly out of date. The package appears orphaned. Several people seem to have tried to step
> up to do something about this but nothing has happened. Is sendmail dead on Debian?  

Unknown to me. Truth to be told, personally I try to avoid using
sendmail whenever possible. Sendmail.cf's syntax is way too arcane to me.
Still, I can't stand a broken Debian package more than a certain MTA :)

I've used sendmail since the '80s.  It's difficult find a more stable  and well tested mailer.  Almost impossible to get it to drop a message to /dev/null unlike some other mailers out there.  I used to write my own cf files back in the day but you really don't have to mess with that now.  The m4 syntax is a bit ugly but usable and now it's just a configuration file.

[Michael Grant]

On Tue, Feb 3, 2015 at 6:26 PM, Ansgar Burchardt <ans...@debian.org> wrote:

...

Could you try restarting sendmail (systemctl restart sendmail) and show
the output of `systemctl status sendmail'? It also shows the most recent
log entries, but the output of journalctl --unit sendmail --since -5min
might also be useful (if it shows more messages).

So, this is interesting.   'systemctl restart sendmail' with no other changes to the system does start sendmail manually.  However, 'systemctl start sendmail' does not, at least, not without Reco's line in /etc/default/sendmail.

so after a REstart which succeeds, the status looks like this:

# systemctl status sendmail

● sendmail.service - LSB: powerful, efficient, and scalable Mail Transport Agent

   Loaded: loaded (/etc/init.d/sendmail)

   Active: active (running) since Tue 2015-02-03 18:12:38 EST; 4min 8s ago

  Process: 3733 ExecStop=/etc/init.d/sendmail stop (code=exited, status=0/SUCCESS)

  Process: 3757 ExecStart=/etc/init.d/sendmail start (code=exited, status=0/SUCCESS)

   CGroup: /system.slice/sendmail.service

           └─3785 sendmail: MTA: accepting connections

Feb 03 18:12:36 bottom.networkguild.org systemd[1]: Starting LSB: powerful, e...

Feb 03 18:12:36 bottom.networkguild.org sm-mta[3785]: starting daemon (8.14.4...

Feb 03 18:12:36 bottom.networkguild.org sm-mta[3785]: daemon could not open c...

Feb 03 18:12:36 bottom.networkguild.org sm-mta[3785]: started as: /usr/sbin/s...

Feb 03 18:12:38 bottom.networkguild.org sendmail[3757]: Starting Mail Transpo...

Feb 03 18:12:38 bottom.networkguild.org systemd[1]: Started LSB: powerful, ef...

Hint: Some lines were ellipsized, use -l to show in full.

Now here's something I can't explain.  After I do the systemctl restart, now I can do systectl stop and systemctl start and they work fine but only after doing a restart first after boot.

In case this isn't clear:

1) reboot

2) sendmail not running

3) run 'systemctl start sendmail' by hand, exits quickly, sendmail NOT started

4) run 'systemctl restart sendmail'.  It takes a few seconds, sendmail starts

5) run 'systemctl stop sendmail'.  again, takes a few seconds, sendmail stops

6) run 'systemctl start sendmail', it takes a few seconds, sendmail starts.

When I run systemctl status sendmail just after rebooting, this is what it looks like:

# systemctl status sendmail

● sendmail.service - LSB: powerful, efficient, and scalable Mail Transport Agent

   Loaded: loaded (/etc/init.d/sendmail)

   Active: active (exited) since Tue 2015-02-03 18:23:25 EST; 1min 27s ago

  Process: 2604 ExecStart=/etc/init.d/sendmail start (code=exited, status=0/SUCCESS)

Feb 03 18:23:24 bottom.networkguild.org sendmail[2604]: Starting Mail Transpo...

Feb 03 18:23:24 bottom.networkguild.org sm-mta[2822]: NOQUEUE: SYSERR(root): ...

Feb 03 18:23:25 bottom.networkguild.org sendmail[2604]: .

Feb 03 18:23:25 bottom.networkguild.org systemd[1]: Started LSB: powerful, ef...

Hint: Some lines were ellipsized, use -l to show in full.

# ps aux | grep sendmail | grep -v grep

There's no sendmail process.

I tried installing sendmail on a minimal test installation and systemd
started at least one daemon ("sendmail: MTA: accepting connections"),
so at least something gets started (though it complained about the test
installation not having a FQDN so other parts might be broken and not
have started).

So one difference is I upgraded a machine from wheezy to testing.  Yes, that's the sendmail daemon you see, that's what success looks like.  But at least you are getting it to start at boot whereas I am not. 

[Michael Grant]

On Tue, Feb 3, 2015 at 7:03 PM, Bob Proulx <b...@proulx.com> wrote:

Michael Grant wrote:
> I'm still searching for an answer to this.
> After upgrade from wheezy to testing, sendmail no longer starts.
> I see that the system is using systemd.
> ...

Some comments that I think are relevant...

Since it took a while for someone to respond to your question it tells
me that it is a combination of tools that not many people are using.
You are using sendmail in combination with systemd.  That is an
unusual combination.  If it were widely used then many people would
have been responding already.  Frankly you might be one of the few
trailblazing that combination.

I do not mind trailblazing this to get this working for the greater good.  It may be an upgrade issue or some other dependency, it would not surprise me in the least since this is a copy of an existing vm that I tried to upgrade.

It is true that fewer and fewer people are using sendmail these days, especially since it seems to have been orphaned.  Sendmail may be old but it's incredibly reliable, well tested, and stable.  

Trailblazers are great!  They are the ones who make things happen.
However not everyone wants to be a trailblazer.  There just isn't
enough time for everyone to do everything.  You sound like a busy
person without the time to debug everything.  Perhaps it would be good
to change to a more mainstream combination?  If it were a mainstream
combination then the problem would almost certainly already have been
seen and fixed.  Something with a lot of users and a lot of support.

Maybe I will one day.   

In this case I would suggest that Sendmail is no longer the mainstream
mail transfer agent.  Instead I suggest migrating to Postfix.  (Or
Exim but I personally really prefer Postfix so will recommend
Postfix.)  Postfix is well tested and very well supported.  There has
been discussion of making Postfix the default mta on Debian.  (But
that will never happen because Exim isn't bad just not as popular.)

I have tried postfix several times over the years.  I was surprised that I was able to make seemingly innocent config mistakes in postfix and it would just drop mail into /dev/null.  This is surprisingly difficult in sendmail as its failure mode is to reject or not accept the mail in this case.

Since you are using Sendmail I assume you have been using Sendmail
forever.  You probably have multiple editions of the O'Reilly Sendmail
book on your bookshelf.  You probably hate to take the time to migrate
a working configuration tuned over decades to something different.  I
have been there and glance over at my two remaining editions of the
O'Reilly Sendmail books on my bookshelf.  Let me say that moving to
Postfix was very easy.  I don't even have one copy of the O'Reilly
Postfix book.  It has an easy to understand design and the online
documentation is excellent.  I have been where you are with Sendmail
and migrating to Postfix was a good decision for me.  I suggest that
it would be for you too.  YMMV.

Well, you know, I never bought that bat book either!  But yes, you are right, I have been using Sendmail since perhaps it first appeared in BSD, I first encountered it in 1983 in BSD on the VAX 11/780, well before M4 and I used to write my own cf files back then too!

[Ansgar Burchardt]

Hi,

Michael Grant <mgr...@grant.org> writes:

> On Tue, Feb 3, 2015 at 6:26 PM, Ansgar Burchardt <ans...@debian.org> wrote:
>> Could you try restarting sendmail (systemctl restart sendmail) and show
>> the output of `systemctl status sendmail'? It also shows the most recent
>> log entries, but the output of journalctl --unit sendmail --since -5min
>> might also be useful (if it shows more messages).
>
> So, this is interesting. 'systemctl restart sendmail' with no other
> changes to the system does start sendmail manually. However, 'systemctl
> start sendmail' does not, at least, not without Reco's line in
> /etc/default/sendmail.

Okay, this is a limitation in systemd's compatability layer for init
scripts: systemd keeps track of the current state of services. Some
services (daemons) can only be "active" when the daemon process is
around, but there are also services that only run some commands and then
exit, i.e. a service setting up the keyboard layout.

For native systemd services there is an option RemainAfterExit= which
can be used to specify this (see man:systemd.service(5)). For init
scripts systemd cannot know this and defaults to RemainAfterExit=true.

So systemd considers the sendmail service running as RemainAfterExit is
set for init scripts and "/etc/init.d/sendmail start" did *not* return
an error, even though it failed to actually start the daemon (see
below).

"systemctl start" is thus not doing anything as systemd will not start
the same service twice. "systemctl restart" will mark the service
stopped first and then start it again; this is not a noop for services
already running.

> When I run systemctl status sendmail just after rebooting, this is what it
> looks like:
>
> # systemctl status sendmail
> ● sendmail.service - LSB: powerful, efficient, and scalable Mail Transport
> Agent
> Loaded: loaded (/etc/init.d/sendmail)
> Active: active (exited) since Tue 2015-02-03 18:23:25 EST; 1min 27s ago
> Process: 2604 ExecStart=/etc/init.d/sendmail start (code=exited,
> status=0/SUCCESS)

This is a bug in the init script: it should not return success if there
was an error starting the service (though it cannot always find out if
there was an error easily).

> Feb 03 18:23:24 bottom.networkguild.org sendmail[2604]: Starting Mail
> Transpo...
> Feb 03 18:23:24 bottom.networkguild.org sm-mta[2822]: NOQUEUE:
> SYSERR(root): ...

This looks like an error, but the interesting part is cut off. You might
want to run "systemctl status -l sendmail" as suggested below, or
"journalctl -b --unit sendmail" to show all log messages for the
"sendmail" service since the last reboot (or check /var/log if a syslog
daemon is running).

> Feb 03 18:23:25 bottom.networkguild.org sendmail[2604]: .
> Feb 03 18:23:25 bottom.networkguild.org systemd[1]: Started LSB: powerful,
> ef...
> Hint: Some lines were ellipsized, use -l to show in full.

I suspect that sendmail misses some dependency information in its init
script, that is it expects some other parts of the system to be already
brought up when it starts. As systemd starts more parts in parallel,
it's more likely to uncover such issues than booting with sysvinit.

Ansgar


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: https://lists.debian.org/87y4oef...@deep-thought.43-1.org

[Reco]

Hi.

Oh, but did you run 'systemctl enable sendmail' after creating the
file? Because if you did - I'm out of ideas, sorry.

Reco


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: https://lists.debian.org/20150204094244.GA10345@x101h

[Michael Grant]

I finally managed to get sendmail working using systemd.

Here is my /etc/systemd/system/sendmail.service:

[Unit]

Description=Sendmail Mail Transport Agent

Requires=clamav-daemon.service spamassassin.service

After=syslog.target network.target clamav-daemon.service spamassassin.service

Conflicts=postfix.service exim.service

[Service]

Type=forking

PIDFile=/run/sendmail/mta/sendmail.pid

Environment=SENDMAIL_OPTS=-q1h

EnvironmentFile=-/etc/default/sendmail

ExecStart=/usr/sbin/sendmail -bd $SENDMAIL_OPTS $SENDMAIL_OPTARG

[Install]

WantedBy=multi-user.target

and my /etc/tmpfiles.d/sendmail.conf file:

d /run/sendmail/ 0755 smmta smmsp

d /run/sendmail/mta/ 0755 smmta smmsp

I am using clamav-milter and spamass-milter, hence the Requires= and After= lines.  If you are not using these, probably you should remove those.

Is it wrong to include these dependencies in sendmail.system?  The thing is, these milters are not specific to sendmail.  Other mailers that support the milter interface can use them as well.  And they are not required for sendmail.  So I wonder which pakage's responsibility it would be to add these dependencies to sendmail.system or if this is even the correct place to do that.

With the init.d, clamav-milter and spamass-milter install themselves with a lower number than sendmail and always start before whatever mailer is installed.  Once you go to explicit dependencies like this, is it clam's and spamassassin's job to know all the possible mailers out there that might use it?  Or is it sendmail's job to know all the possible milters out there and state them as dependencies?

Another observation, to get this working, the only way I found to properly test this was to continually reboot.  I could get sendmail to start by hand quite early on, but it was not starting by on reboot because of the timing problem in the dependencies.  This makes systemd rather more difficult to debug things in my opinion.