Bug#804882: hwclock: /etc/init.d/hwclock.sh stop does nothing at all

Showing 1-5 of 5 messages
Bug#804882: hwclock: /etc/init.d/hwclock.sh stop does nothing at all shi...@teksavvy.com 11/12/15 8:40 AM
Package: util-linux
Version: 2.25.2-6
Severity: important

Dear Maintainer,

Daylights saving are on since last week-end.  A security linux kernel was also
issued.  I rebooted after upgrading the kernel and the time was 1h ahead of real
time.

I remembered that there was a script to adjust the hardware clock to the kernel
clock in the '/etc/init.d' folder that should run whenever the machine is
shutwown.

So I set the date 1 hour earlier, and issued an '/etc/init.d/hwclock.sh stop'
and nada, nothing happened.  No output, no message at all.

Strangely enough '/etc/init.d/hwclock.sh reload' do work as intended.  A message
is displayed as it runs to confirm it.

This is an important bug.  It occurred at 11:02 PM, which put the kernel clock
at 12:02 AM the next day.  It broke a lot of scripts relying on timespamp to
run properly.  Munin is broken for 24h, mythtv recorded wrong programs and refused
to re-record them, the log is plagged with error messages, etc.  Please fix ASAP.

It's 100% reproductible

-- System Information:
Debian Release: 8.2
  APT prefers stable-updates
  APT policy: (990, 'stable-updates'), (990, 'stable'), (100, 'oldstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.16.0-4-amd64 (SMP w/6 CPU cores)
Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Init: systemd (via /run/systemd/system)

Versions of packages util-linux depends on:
ii  initscripts    2.88dsf-59
ii  libblkid1      2.25.2-6
ii  libc6          2.19-18+deb8u1
ii  libmount1      2.25.2-6
ii  libncurses5    5.9+20140913-1+b1
ii  libpam0g       1.1.8-3.1
ii  libselinux1    2.3-2
ii  libslang2      2.3.0-2
ii  libsmartcols1  2.25.2-6
ii  libtinfo5      5.9+20140913-1+b1
ii  libuuid1       2.25.2-6
ii  lsb-base       4.1+Debian13+nmu1
ii  tzdata         2015g-0+deb8u1
ii  zlib1g         1:1.2.8.dfsg-2+b1

util-linux recommends no packages.

Versions of packages util-linux suggests:
ii  dosfstools          3.0.27-1
ii  kbd                 1.15.5-2
ii  util-linux-locales  2.25.2-6

-- debconf information:
  util-linux/noauto-with-nonzero-passnum:
Bug#804882: hwclock: /etc/init.d/hwclock.sh stop does nothing at all Andreas Henriksson 11/12/15 10:10 AM
Control: tags -1 + moreinfo

Hello shizuma.

Thanks for your bug report.

On Thu, Nov 12, 2015 at 11:17:47AM -0500, shi...@teksavvy.com wrote:
> Package: util-linux
> Version: 2.25.2-6
> Severity: important
>
> Dear Maintainer,
>
> Daylights saving are on since last week-end.  A security linux kernel was also
> issued.  I rebooted after upgrading the kernel and the time was 1h ahead of real
> time.

Works for me.

>
> I remembered that there was a script to adjust the hardware clock to the kernel
> clock in the '/etc/init.d' folder that should run whenever the machine is
> shutwown.

Thanks for using reportbug to file this bug report. At the bottom we can
see that you're using systemd. This tells me that the init script you're
trying is irrelevant!

First of all, hwclock is managed via udev hooks (unless your system
does not have udev, which is very rare nowadays). The init script
will detect this and bail out on 'start'
(Additionally the udev hook will bail out if detects you're using systemd.)

The 'stop' function of the script is only run if you're using sysvinit.
This is because systemd ships an override disabling the hwclock.sh
script.

Also, you should not directly invoke init scripts (ever!). As an admin
you should use the 'service' command (or systemctl if you only care about
systems which use systemd).

In short, you're trying to use an obsolete script which you should not
be touching.

>
> So I set the date 1 hour earlier, and issued an '/etc/init.d/hwclock.sh stop'
> and nada, nothing happened.  No output, no message at all.

See above.

>
> Strangely enough '/etc/init.d/hwclock.sh reload' do work as intended.  A message
> is displayed as it runs to confirm it.

Please always describe what you did, what happened and what you expected.
Unfortunately 'work as intended' tells me nothing of what you expect or
what happened.

>
> This is an important bug.

Given that you're the only one affected and noone else has mentioned a thing
I'm not sure I agree, but additional details from you would be useful to
determine if there's a bug at all.

> It occurred at 11:02 PM, which put the kernel clock
> at 12:02 AM the next day.  It broke a lot of scripts relying on timespamp to
> run properly.  Munin is broken for 24h, mythtv recorded wrong programs and refused
> to re-record them, the log is plagged with error messages, etc.  Please fix ASAP.
>
> It's 100% reproductible

I'm not sure what you expect me to do about this. You're doing several things
wrong. You haven't really described any details about what happens.
You're not using the proper commands to execute services.

This leaves me very much in the dark and without any information on how
to reproduce this. Please feel free to provide additional information
that might help me understand if there's a bug to be fixed.
Bug#804882: hwclock: /etc/init.d/hwclock.sh stop does nothing at all Marcel 11/12/15 6:20 PM
Hi Andreas.

Thanks for your quick reply.  I appreciate it.

I reckon that the misuse of init scripts is my responsibility
however the service integration isn't perfect yet.  A lot of
scripts/packages still use init scripts.  Monit comes immediately
to my mind.  It relies heavily on start/stop/restart.

And I was in a state of panic.  I know full well how catastrophic
the results of setting the clock too far away in time could do on this
system.  I couldn't remember the command name 'service' so I relied
on what has been working faithfully for years, initd.

I'll complete the transition to service whenever I have spare time.

That being said, I saved the console so you can see each and every
command I typed yesterday and the system reaction or absence of:

shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh
[ ok ] Usage: hwclock.sh {start|stop|reload|force-reload|show}.
[ ok ] start sets kernel (system) clock from hardware (RTC) clock.
[ ok ] stop and reload set hardware (RTC) clock from kernel (system) clock.
shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh stop
shizuma@gt5232h-a3e401d:~$ date
mercredi 11 novembre 2015, 23:26:56 (UTC-0500)
shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh show
jeu 12 nov 2015 00:28:19 EST  -0.084876 secondes
shizuma@gt5232h-a3e401d:~$ date
mercredi 11 novembre 2015, 23:27:30 (UTC-0500)
shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh reload
[info] Saving the system clock.
[info] Hardware Clock updated to mercredi 11 novembre 2015, 23:28:07 (UTC-0500).
shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh show
mer 11 nov 2015 23:28:24 EST  -0.062948 secondes
shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh stop
shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh show
mer 11 nov 2015 23:35:14 EST  -0.969351 secondes
shizuma@gt5232h-a3e401d:~$ date
mercredi 11 novembre 2015, 23:37:35 (UTC-0500)


I also tried the same commands on another system I maintain, with
identical results:  stop producing no output at all:

root@NC-PH-0657-10:~# /etc/init.d/hwclock.sh show
Thu 12 Nov 2015 06:47:32 PM MST  -0.406578 seconds
root@NC-PH-0657-10:~# date
Thu Nov 12 18:47:34 MST 2015
root@NC-PH-0657-10:~# /etc/init.d/hwclock.sh stop
root@NC-PH-0657-10:~# /etc/init.d/hwclock.sh show
Thu 12 Nov 2015 06:47:56 PM MST  -0.922150 seconds


I understand that you would not maintain initd scripts.  However an
issue is still present - and maybe this very bug description should be
changed accordingly.  It's that on shutdown for reboot, the hardware clock
wasn't properly updated.  This should be addressed.

I'd be more than happy to provide you with all the assistance you need.

Is there something particular that I should look for in dmesg|system logs?

Best regards.
Bug#804882: hwclock: /etc/init.d/hwclock.sh stop does nothing at all Andreas Henriksson 11/13/15 12:30 AM
Hello Marcel!

Quick reply below...

On Thu, Nov 12, 2015 at 09:01:49PM -0500, Marcel wrote:
> Hi Andreas.
>
> Thanks for your quick reply.  I appreciate it.
>
> I reckon that the misuse of init scripts is my responsibility
> however the service integration isn't perfect yet.  A lot of
> scripts/packages still use init scripts.  Monit comes immediately
> to my mind.  It relies heavily on start/stop/restart.

Any packaged software in Debian which directly invokes an init script
(ignoring service policy etc.) instead of doing so via invoke-rc.d
(which is the programmatical variant of the administrative 'service'
interface) is in conflict with the Debian policy and thus RC-buggy.
This is because 'policy-rc.d' should be taken into consideration which
it isn't if the script is invoked directly.

https://www.debian.org/doc/debian-policy/ch-opersys.html#s9.3.3

>
> And I was in a state of panic.  I know full well how catastrophic
> the results of setting the clock too far away in time could do on this
> system.  I couldn't remember the command name 'service' so I relied
> on what has been working faithfully for years, initd.

Yes, fortunately the Debian systemd maintainers have LSB hooks in place
to defer your direct invokation to became a systemctl command instead.
(FWIW, the service command would also translate into a systemctl command
when running under systemd.)

>
> I'll complete the transition to service whenever I have spare time.
>
> That being said, I saved the console so you can see each and every
> command I typed yesterday and the system reaction or absence of:

Awesome! Thanks.
Even more awesome would be if you can reproduce the problem while
hacking the init script to add 'set -x' at the top so we can see
each and every command being run.
Might also be useful to know your configuration for local/UTC time
in rtc, eg. /etc/adjtime
Also knowing your /etc/default/hwclock settings would be helpful.

Quick commends below (I'll try to find time to look at this
with more details in the future).

>
> shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh
> [ ok ] Usage: hwclock.sh {start|stop|reload|force-reload|show}.
> [ ok ] start sets kernel (system) clock from hardware (RTC) clock.
> [ ok ] stop and reload set hardware (RTC) clock from kernel (system) clock.

ok.

> shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh stop
> shizuma@gt5232h-a3e401d:~$ date
> mercredi 11 novembre 2015, 23:26:56 (UTC-0500)
> shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh show
> jeu 12 nov 2015 00:28:19 EST  -0.084876 secondes
> shizuma@gt5232h-a3e401d:~$ date
> mercredi 11 novembre 2015, 23:27:30 (UTC-0500)

So your system time seems it was 1 day behind your RTC time.
Some serious time drift right there (unless your system time was
updated without syncing the RTC earlier somehow).

> shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh reload
> [info] Saving the system clock.
> [info] Hardware Clock updated to mercredi 11 novembre 2015, 23:28:07 (UTC-0500).

Here you just wrote your system time to the RTC, which I guess means
you messed up your RTC time.

In the init script the 'stop' and 'reload' arguments are just aliases
for the very same thing.
The reason they still behave differently seems to simply be because
'stop', 'start', 'restart' will be translated into 'systemctl <cmd>
hwclock' by the LSB init hooks for systemd. While the 'show' and
'reload' commands will not (and just run the init script code) as
there's no direct generic systemctl equivalent.

> shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh show
> mer 11 nov 2015 23:28:24 EST  -0.062948 secondes
> shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh stop
> shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh show
> mer 11 nov 2015 23:35:14 EST  -0.969351 secondes
> shizuma@gt5232h-a3e401d:~$ date
> mercredi 11 novembre 2015, 23:37:35 (UTC-0500)
>
>
> I also tried the same commands on another system I maintain, with
> identical results:  stop producing no output at all:
>
> root@NC-PH-0657-10:~# /etc/init.d/hwclock.sh show
> Thu 12 Nov 2015 06:47:32 PM MST  -0.406578 seconds
> root@NC-PH-0657-10:~# date
> Thu Nov 12 18:47:34 MST 2015
> root@NC-PH-0657-10:~# /etc/init.d/hwclock.sh stop
> root@NC-PH-0657-10:~# /etc/init.d/hwclock.sh show
> Thu 12 Nov 2015 06:47:56 PM MST  -0.922150 seconds
>
>
> I understand that you would not maintain initd scripts.  However an
> issue is still present - and maybe this very bug description should be
> changed accordingly.  It's that on shutdown for reboot, the hardware clock
> wasn't properly updated.  This should be addressed.

So far the hwclock.sh script has no knowledge about systemd or
that it's obsolete/unused under systemd, but maybe it should
indeed grow a check that just aborts it if executed under systemd
environment to prevent issues like yours.... but really, you should
stop executing init scripts directly also. :P

>
> I'd be more than happy to provide you with all the assistance you need.

This is great. Too much of my time is wasted on people not following
up with feedback. Very much appreciated that you're interested in
seeing this through.

>
> Is there something particular that I should look for in dmesg|system logs?

I think I have an idea what's going on now and I'll have to think about
possible ways to prevent people shooting themselves in the foot.

I hope the information provided above is helpful for your as well.
I'll add that if time is important to you, you should consider running
an ntpd (via the 'ntp' package or other equivalent implementation) that
will help keep your system and hardware time in sync and up to date,
assuming you have network connection where you can reach a ntp server.
(systemd also ships a 'lightweight' SNTP variant called
systemd-timesyncd which you can enable. See:
"systemctl status systemd-timesyncd", not sure it's part of Jessie/8.2
though.)

Regards,
Andreas Henriksson
Bug#804882: hwclock: /etc/init.d/hwclock.sh stop does nothing at all Marcel 11/13/15 10:30 AM
Hi again,

Commented inline.

You'll find more details concerning this issue.

* Andreas Henriksson (and...@fatal.se) wrote:
> Hello Marcel!
>
> Quick reply below...
>
> On Thu, Nov 12, 2015 at 09:01:49PM -0500, Marcel wrote:
> > Hi Andreas.
> >
> > Thanks for your quick reply.  I appreciate it.
> >
> > I reckon that the misuse of init scripts is my responsibility
> > however the service integration isn't perfect yet.  A lot of
> > scripts/packages still use init scripts.  Monit comes immediately
> > to my mind.  It relies heavily on start/stop/restart.
>
> Any packaged software in Debian which directly invokes an init script
> (ignoring service policy etc.) instead of doing so via invoke-rc.d
> (which is the programmatical variant of the administrative 'service'
> interface) is in conflict with the Debian policy and thus RC-buggy.
> This is because 'policy-rc.d' should be taken into consideration which
> it isn't if the script is invoked directly.

I realize that, but I don't care.  I expect packages to work,  When
they don't, I reconfigure.   Only debian maintainers should care about this.
That is my opinion.

>
> https://www.debian.org/doc/debian-policy/ch-opersys.html#s9.3.3
>
> >
> > And I was in a state of panic.  I know full well how catastrophic
> > the results of setting the clock too far away in time could do on this
> > system.  I couldn't remember the command name 'service' so I relied
> > on what has been working faithfully for years, initd.
>
> Yes, fortunately the Debian systemd maintainers have LSB hooks in place
> to defer your direct invokation to became a systemctl command instead.
> (FWIW, the service command would also translate into a systemctl command
> when running under systemd.)
>
> >
> > I'll complete the transition to service whenever I have spare time.
> >
> > That being said, I saved the console so you can see each and every
> > command I typed yesterday and the system reaction or absence of:
>
> Awesome! Thanks.
> Even more awesome would be if you can reproduce the problem while
> hacking the init script to add 'set -x' at the top so we can see
> each and every command being run.

Yes of course.  See the attached file 'stop.txt'.  Strange enough,
'hwclock.sh stop' is reported doing its job with 'set -x' whereas
I know for a fact that it did not without.

> Might also be useful to know your configuration for local/UTC time
> in rtc, eg. /etc/adjtime

shizuma@gt5232h-a3e401d:~$ cat /etc/adjtime
-0.135568 1447433065 0.000000
1447433065
LOCAL

> Also knowing your /etc/default/hwclock settings would be helpful.

BADYEAR=no
HWCLOCKACCESS=yes
HWCLOCKPARS=
HCTOSYS_DEVICE=rtc0

>
> Quick commends below (I'll try to find time to look at this
> with more details in the future).

No emergency.  Daylights savings are ON for all winter.
It would be great if it were fixed for spring though.

>
> >
> > shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh
> > [ ok ] Usage: hwclock.sh {start|stop|reload|force-reload|show}.
> > [ ok ] start sets kernel (system) clock from hardware (RTC) clock.
> > [ ok ] stop and reload set hardware (RTC) clock from kernel (system) clock.
>
> ok.
>
> > shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh stop
> > shizuma@gt5232h-a3e401d:~$ date
> > mercredi 11 novembre 2015, 23:26:56 (UTC-0500)
> > shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh show
> > jeu 12 nov 2015 00:28:19 EST  -0.084876 secondes
> > shizuma@gt5232h-a3e401d:~$ date
> > mercredi 11 novembre 2015, 23:27:30 (UTC-0500)
>
> So your system time seems it was 1 day behind your RTC time.

No, it was 1 hour.  There's 1h between 11120028 and 11112328,
not 25.

> Some serious time drift right there (unless your system time was
> updated without syncing the RTC earlier somehow).
>

It's not drift.  It's daylight savings turned on.

And yes, 'system time was updated without syncing the RTC earlier
somehow':  The system time was updated sunday to compensate for
daylight savings.  It's automatic.  But on shutdown 2 days ago, the
RTC wasn't automatically updated from system time as it should have.  

Upon restart, the system time was updated from the wrong RTC time.

So the system restarted with 1 hr ahead of time, and worse, with 1 day
ahead of time too.

See excerpt of syslog:

(...)
Nov 11 23:03:26 gt5232h-a3e401d kmotion_hkd1[20621]: signal SIGHUP detected, re-reading config file
Nov 11 23:03:26 gt5232h-a3e401d freshclam[987]: Update process terminated
Nov 12 00:14:13 gt5232h-a3e401d rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="1262" x-info="http://www.rsyslog.com"] start
Nov 12 00:14:13 gt5232h-a3e401d rsyslogd-2307: warning: ~ action is deprecated, consider using the 'stop' statement instead [try http://www.rsyslog.co
m/e/2307 ]
(...)

I rebooted at 23:03
The log ends at 11:03 PM on 11/11 and restarts at 12:14 AM on 11/12
You see a 1 hr 11 minutes difference whereas it was really 11 minutes.
It took 11 minutes because a filecheck was triggered.

However I made a supplemental mistake.

I didn't realized we crossed midnight when I set the system
time with the date command.  I set the time correctly 1 hr back
but I forgot to also remove 1 day to the currently reported day,
worsening an already fragile situation.

E.g. (retrieved from history)
 5481  sudo date 11122320 (oops, we're still 11/11!)
 5482  sudo date 11112320

> > shizuma@gt5232h-a3e401d:~$ sudo /etc/init.d/hwclock.sh reload
> > [info] Saving the system clock.
> > [info] Hardware Clock updated to mercredi 11 novembre 2015, 23:28:07 (UTC-0500).
>
> Here you just wrote your system time to the RTC, which I guess means
> you messed up your RTC time.

That is precisely what I wanted.  To copy the system time, which was now correct
to the RTC which was still not.
I already do.  It's of paramount importance to me that the system time
is exact.  I do run  ntpd for years.

I repeat, the real issue is that on reboot, the system time wasn't
automatically saved to the RTC as it should have.  So the system
restarted with the time reported being 1 hour later than expected
and the day after on top of that.

The 'stop' issue is secondary and trivial as I would always realize that
something is wrong whenever I run it.

Thanks again and best regards.