How to upload missed data to Wunderground after an Internet Outage

[Roy Gilby]

Greetings,

I've been using pywws-livelog-daemon to upload weather data to Weather Underground for a few months now.  Recently, as the authorities have been upgrading the electricity supply in this area, we have been experiencing electricity outages of a few hours, multiple times each week.  These outages are expected to continue for at least the next week.

When using pywws-livelog-daemon to upload to Weather Underground, when an electricity outage occurs, the computer shuts down when the UPS battery gets too low.  When the electricity supply is restored, I re-boot the computer and the livelog daemon restarts. 

But, all weather data collected during the period of the electricity supply outage is ignored by pywws-livelog-daemon.  Data uploaded to Weather Underground commences with the first new weather data acquired by the Weather Station after restarting of pywws-livelog-daemon.

What do I need to do to ensure that all weather data collected by the weather station during an electricity outage, is uploaded to weather underground once the electricity supply is restored ?

Many thanks,

RoyG

[Jim Easterbrook]

On 03/08/15 06:51, Roy Gilby wrote:
>
> When usingpywws-livelog-daemon to upload to Weather Underground, when an

> electricity outage occurs, the computer shuts down when the UPS battery
> gets too low. When the electricity supply is restored, I re-boot the
> computer and the livelog daemon restarts.
>
> But, all weather data collected during the period of the electricity
> supply outage is ignored by pywws-livelog-daemon. Data uploaded to
> Weather Underground commences with the first new weather data acquired
> by the Weather Station after restarting of pywws-livelog-daemon.
>
> What do I need to do to ensure that all weather data collected by the
> weather station during an electricity outage, is uploaded to weather
> underground once the electricity supply is restored ?

It should do it automatically if you have included 'underground' in your
[logged] or [hourly] services:
http://pywws.readthedocs.org/en/latest/guides/integration.html#weather-underground-rapidfire-updates
--
Jim Easterbrook <http://www.jim-easterbrook.me.uk/>

[Roy Gilby]

Thanks Jim - that's what I thought was supposed to happen ... in my case, it's doesn't seem to work as advertised.

Attached is a copy of my weather.ini (with the weather underground station & password information removed).  I'm pretty sure that I have weather.ini set up as specified in your doco.  Can someone please review my weather.ini file and advise.

Thanks,

Roy

Copy of weather.ini file follows ...........

[config]
ws type = 1080
usb activity margin = 3.0
pressure offset = 10.6
logdata sync = 1
day end hour = 9
rain day threshold = 0.2
asynchronous = False
frequent writes = False
template encoding = iso-8859-1
gnuplot version = 4.2
gnuplot encoding = iso_8859_1

[paths]
work = /tmp/weather
templates = /home/royg/weather/templates/
graph_templates = /home/royg/weather/graph_templates/
local_files = /home/royg/weather/results/

[ftp]
local site = False
site = ftp.username.your_isp.co.uk
user = username
password = secret
directory = public_html/weather/data/
secure = False
port = 21

[live]
services = ['underground_rf']
plot = []
text = []

[logged]
services = ['underground']
text = []
plot = []

[hourly]
services = ['underground']
text = []
plot = []

[12 hourly]
services = []
text = []
plot = []

[daily]
services = []
text = []
plot = []

[underground]
station = ###########
password = ##########
template = default

[Jim Easterbrook]

On 03/08/15 09:16, Roy Gilby wrote:
> Thanks Jim - that's what I thought was supposed to happen ... in my
> case, it's doesn't seem to work as advertised.
>
> Attached is a copy of my weather.ini (with the weather underground
> station & password information removed). I'm pretty sure that I have
> weather.ini set up as specified in your doco. Can someone please review
> my weather.ini file and advise.

That all looks fine - knowing your station ID would help so I can see
what data has been uploaded. It's not secret info and many people like
to advertise their stations. (-:

Have a look at the 'underground' and 'underground_rf' values in the
[last update] section of your status.ini file. They should be within a
few minutes of each other in normal operation. (You have 'frequent
writes' disabled so shut down live logging to flush data to file before
doing this.)

You can try setting the 'underground' last update back a few hours and
see if catchup data gets sent when you restart live logging. Run it with
a higher verbosity to watch for this.

(All this assumes you have a reasonably up to date version of pywws
installed.)

[Roy Gilby]

Thanks Jim.   The Weather Underground Station ID is;  IQUEENSL709

The electricity supply is off today (from 5 August 2015@2200UTC until roughly 6 August 2015@0600UTC), so sometime this afternoon (i.e. after 0600 UTC) when the electricity supply has returned, you'll be able to look at what data is uploaded to Weather Underground immediately after the resumption of service.  I'll have a look at the status.ini file this afternoon, after I get power back to the server.

Regarding the version of pywws - I updated to the latest version of pywws (15.07.0) the day after you posted about the new release on this mailing list.

Thanks,

RoyG

[Roy Gilby]

Ok - Problem solved. 

I found a minor error in the weather.ini file that was causing the "underground" entry to be ignored.  I suspect that there was a control character, or some other hidden character, in the [logged] and [hourly] lines of the weather.ini file.  Once I deleted the two lines with "underground", retyped those two lines, and restarted the pywws-livelog-daemon, all of the weather observations for the previous six hours were uploaded over a period of about fifteen minutes. 

I never did find exactly what was wrong with those two lines in weather.ini, but the status.ini file showed no entries for "underground" - only entries for "underground_rf".  After the above fix, there are now entries in status.ini for both underground and underground_rf.

Thanks,

RoyG