Weird uptime from Weewx after power failure ...

[Bill]

I am seeing a weird uptime on my weewx page after a power failure.

Have a look at www.thomaswhitefish.com/weather and you will see the server
uptime started from this morning, but the weewx uptime seems have overflowed.

What happened is I had a power outage over 16 hours, much longer than the UPS
running the server lasted. The system has come back up and all is working
except for the weewx uptime.

I haven't restarted weewx, since I thought someone might want to have a look
first. Not sure if this is a reproducible event?

BTW, yesterday I was reporting an uptime of 120 days, nice going weewx, you
are more reliable than the power company!!!

Bill

[John Canfield]

That's weird, never had that happen before - weewx and server uptimes always matched after reboot.  I wonder what would weewx would report if you restart it.

I had a weewx uptime of 180 days before I rebooted for kernel updates to be applied and I think Tom had a box running with ~360 days of uptimes.

John
weather.janeandjohn.org

[Corbin Dallas]

I have seen that a time or two, but on a much smaller scale.  A few times after a re-boot or power outage, I have seen the Weewx up time be a few minutes higher than the server uptime.  But never more than 10 minutes or so.  I'm not sure if what I have seen is related to your issue or if it is something different.

[Rich Mulvey]

Today minus 15,737 days in Jan 1, 1970, the start of the Unix epoch.

Sounds like your hardware clock has been zeroed out.  :-)  Is this on something like a Beaglebone without a battery-backed clock?

Bill

--
You received this message because you are subscribed to the Google Groups "Weewx Weather Station Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
ri...@mulveyfamily.com

[Thomas Keffer]

Rich has it exactly right.

Weewx simply saves the unix epoch time when it starts up, then takes the difference from that for each report. The start-up time must have returned 0 ("zero"), or 1 Jan 1970. 

Later times work OK because NTP corrects the time.

Replace the battery on your board and all will likely be well the next time you have a power failure.

Oh, and restart weewx. :-)

-tk

Tom Keffer

kef...@threefools.org
+1 541-386-8891 (h)
+1 541-490-9507 (c)

Skype: tkeffer

[vds]

On Friday, February 1, 2013 12:24:24 PM UTC-8, Rich Mulvey wrote:

Today minus 15,737 days in Jan 1, 1970, the start of the Unix epoch.

Sounds like your hardware clock has been zeroed out.  :-)  Is this on something like a Beaglebone without a battery-backed clock?

My Seagate Dockstar which has no battery backed up clock at all has done this. What happens is Weewx starts up before the NTP subsystem has successfully has gotten date+time from the upstream source, even though at a quick glance the sequencing of the start links in rc2.d is correct.

It wouldn't be too hard to tweak the init.d startup files to sleep or something so that weewx doesn't start up without the date being reasonable, but the Dockstar reboots so rarely it hasn't been worth the effort to bother.  Trivial workaround is a weewx restart after your computer has correct date+time.

[Bill]

Rich,

You were bang-on for this, not sure how I missed the connection :-)

Yes, I am running on a device (a Pogoplug) without a real time clock. I
rebooted on my last visit to the lake (I am not there as often during the
winter), and now the uptime is ok.

I am trying to determine if there a a better solution to this that will make
clockless devices more robust?

/bill

[Bill]

If weewx starts with the wrong clock, does this in anyway mess up the data
collected? I realize if web pages are generated before the clock is
corrected, they will have the wrong times, but what about once the clock is
ok, are the collected data and reports ok?

I am especially thinking of the the situation where the system restarts and
the network doesn't recover. Weewx might run for hours or days with the
wrong clock until the network is reset and the clock adjusted.

Bill

[Bill]

I got reminded of this message with the recent postings about the Raspberry
PI. It occurs to me it might be nice to have a better startup solution?

I was thinking that perhaps the RC script should check the 'century' to
determine if the clock has been corrected, and if not it could sleep for a
minute. The script could keep checking and and start when it looks good, or
sleep again, perhaps for a maximum of 5 times?

It the case of my recent power outage, I realized weewx had to restart, as did
the switch and access point it was connected to, also the wireless router and
the wireless link to my ISP had to reconnect and authenticate. Altogether
this might take a little time, but less than 5 minutes.

Of course it would be easier to run with a hardware clock :-)

Bill

[Thomas Keffer]

The default setup is for weewx to attempt to sync the console's onboard clock with the computer at startup, then again every few hours. So, if the computer's clock is way off, then the console's clock will follow and be way off.

This is why the docs recommend running NTP. It not only gets the time right, but it also avoids problems when publishing to online services such as WeatherUnderground.

There is simply no good way to proceed if the computer's clock isn't right. For example, the program has no way of knowing when the archive records are due. 

If your computer does not have a clock with a battery backup, then weewx should not start until after NTP is up, running, and has set the clock. I believe the daemon startup script /etc/init.d/weewx does this --- at least, the script specifies that service $time is required --- but I must admit to never having tested it. Perhaps there is another event that signals that the clock has been synched?

As a last ditch backstop, I suppose we could have weewx check the clock on startup, and if it's some absurd time like years before weewx was even written, it would sleep and loop until the clock looks right... It wouldn't be hard to write such an optional service that could do this. An exercise for one of you "Pi Heads" out there?

Update: Just read Bill's suggestion. I like his idea better. This sort of thing should really be handled by the daemon script, not by the application. It has more information about the startup context.

-tk

You received this message because you are subscribed to the Google Groups "Weewx user's group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[vds]

On Tuesday, March 19, 2013 9:34:02 AM UTC-7, Bill wrote:

I got reminded of this message with the recent postings about the Raspberry
PI.  It occurs to me it might be nice to have a better startup solution?

My experience is nothing bad really happens other than the toplevel page having an ugly value on it.

You could certainly do all kinds of gymnastics to delay startup until the clock is recent enough (perhaps newer than a file in the /home/weewx tree or the like).   Weewx likely shouldn't get into working too much around basic sysadmin issues on the runtime platform.

[Bill]

Tom,

Weewx is syncing the console clock to the computer clock by default? What
happens if the computer clock is wrong, does it mess up the console?

This gives me another idea ... maybe there could be an option in weewx.conf
for computers that don't have hardware clocks. In this case the default
would be to sync the computer's clock to the console? This would be
justified since the console has a clock and it has battery backup.

Of course if the computer has NTPd running then after the time is synced it
might be good to sync the console clock to the computer's clock. There is a
way to tell if NTPd has a good time sync, though I don't know it offhand. (I
know it is there since for computers providing time service, they will not
give out the time until they have synced.)

Thinking out loud here, but it does look like there are some options to make
this a bit better for clockless computers.

Bill

PS: I believe the service $time just says the NTPd service is started, not
that it has managed to connect to anything or sync the clock.

[Thomas Keffer]

On Tue, Mar 19, 2013 at 11:42 AM, Bill <boob...@rogers.com> wrote:

Tom,

Weewx is syncing the console clock to the computer clock by default?  What
happens if the computer clock is wrong, does it mess up the console?

Yup. But, they'll be in synch! :-)

 

This gives me another idea ... maybe there could be an option in weewx.conf
for computers that don't have hardware clocks.  In this case the default
would be to sync the computer's clock to the console?  This would be
justified since the console has a clock and it has battery backup.

Are you suggesting that the console be used to set the computer's clock? If so, I don't like the idea of including that in weewx. It would require root privileges, and it's really outside the scope of what weewx is intended to do.

However, what you could do is run wee_config_vantage with the --info option, which, among other things, prints out the console's clock time. This could then be scraped by a shell script and fed into the system command date. 

 

PS:  I believe the service $time just says the NTPd service is started, not
that it has managed to connect to anything or sync the clock.

I think you're right. Perhaps there is some other service that does this?  In any case, I really think this is the right way to go: modify the RC file to check the time and make sure it's something recent before allowing weewx to run. 

Question to you Pi-heads: when an RPi comes up from a cold start, is the time set to zero, that is, the start of the Unix epoch time (1970), or to the shutdown time of the RPi? I've heard both.

-tk

[vds]

Question to you Pi-heads: when an RPi comes up from a cold start, is the time set to zero, that is, the start of the Unix epoch time (1970), or to the shutdown time of the RPi? I've heard both.

I think it likely depends on the distro on the Pi.  I just took a look at mine which is running a Debian distro (Raspbian maybe?), and there's a utility called 'fake-hwclock' that saves the time to a file /etc/fake-hwclock.data once per hour out of cron.

 

[Theo Bresler]

If you are using a Raspberry Pi, I would suggest adding a Real Time Clock (RTC)

There are a number of RTC modules available for the Pi on the internet.

I have designed and built a small module which includes a RTC and temperature sensor by googling a bit.  It is a really simple module and I am quite willing to share it if anybody would be interested.

[William Phelps]

I second this - Adafruit for example has an RTC board that is under $10, and she has a tutorial showing how to connect it.

[Bill]

I see this topic has come up again, and I have been thinking about it since I
started the thread (sorry it took me so long to get back to it).

I agree with Tom's reluctance to have Weewx use the console as a RTC. I did
some research and it seems actually adding the console as a RTC that NTPd can
access would be hard since Weewx wants to control the USB, and, as far as I
was able to determine there is no support for RTC connected via USB.

I believe we can solve this entirely within the init startup script.
Previously I suggested waiting until the clock got set, and then starting.
Tom pointed out there is a command for getting the console time and that the
time could be scraped from there. I don't have access to my station at this
point, so I haven't been able to implement or test this approach.

So here is what I think could be done without a lot effort, and it would solve
the problem of hardware that doesn't have it's own RTC, this is done entirely
in the init script, nothing is changed in Weewx:

1) The script starts by checking to see if the clock is set, it can do this
by testing the the year with something like 'date +%Y' and seeing if it is
>2000 (IE are we in the right century).

2) If the clock doesn't appear to be set wait 1 minute and try again. Do
this a maximum of say 5 times (a typical archive interval). If ever it
appears to be set, start Weewx normally (IE using the computer time).

3) If after the maximum number of checks the clock still isn't set, then use
the command Tom cited to grab the time from the console. Scrape the values
needed and set the computer clock (this can be done since the init script is
still running as root and hasn't switched userid yet), then start Weewx.

I believe this solution is not much more than 1-2 dozen lines of scripting and
it will solve the clock problem for devices without hardware.

Bill

On Tuesday 19 March 2013 16:56, Thomas Keffer wrote:
> On Tue, Mar 19, 2013 at 11:42 AM, Bill <boob...@rogers.com> wrote:
> > Tom,
> >
> > Weewx is syncing the console clock to the computer clock by default?
> > What happens if the computer clock is wrong, does it mess up the console?
>
> Yup. But, they'll be in synch! :-)
>
> > This gives me another idea ... maybe there could be an option in
> > weewx.conf for computers that don't have hardware clocks. In this case
> > the default would be to sync the computer's clock to the console? This
> > would be justified since the console has a clock and it has battery
> > backup.
>
> Are you suggesting that the console be used to set the computer's clock? If
> so, I don't like the idea of including that in weewx. It would require root
> privileges, and it's really outside the scope of what weewx is intended to
> do.
>
> However, what you could do is run wee_config_vantage with the

> --infooption, which, among other things, prints out the console's

[Bill]

The intention of the fake hardware clock is to keep time marching forward. So
after booting the clock will be set later than it was last time, and services
that check times, file timestamps, etc, will not be confused.

Assuming Weewx implements a scheme as I suggested in my last message, you will
probably want to disable the fake clock. It's setting the time will make the
proposed init script think the clock has been set when it hasn't been.

Bill

[Theo Bresler]

Adding an RTC which plugs directly on the GPIO header of the Pi will fix your problem easier in my opinion.

I have 9 x Pi's doing different jobs and have an RTC on each of them.  I never have issues after power failures.  I live in South Africa and have at least 2 x power failures a month.

[Bill]

I have lots of power failures, several per day, but they hardly ever last long
enough to run down my UPS, such is the power supply in the country.

I do like your suggestion of simply adding a RTC, something like this:

https://www.modmypi.com/rasclock-raspberry-pi-real-time-clock-module

In my case however I am not running on a PI but on a Pogoplug, so I'd have to
do some hardware mods or use something with a USB interface.

The suggestion I made was motivated by the fact the Vantage Vue (and I assume
other consoles), already have a clock.

Bill