Uploading data to wunderground

[Duncan McGugan]

I've set up my Watson W-8681 Wireless Weather Station and it has been happily collecting data using the pywws software for just over four months.

I recently set up a basic website which auto-updates every 30 here http://weather.miterraza.co.uk/ and it all seems to be working fine.

I am now trying to integrate it with wunderground. The weather station ID is IREGINDE96

I set up the following in weather.ini

[underground]

station = IREGINDE96

password = **********

template = default

[logged]

services = ['underground']

After running pywws-hourly manually, the log file reported:

23:52:36:pywws.ToService(underground):50 records sent

pywws-hourly then ran again as part of the regular cron job and reported:

00:13:23:pywws.ToService(underground):50 records sent

When I look at the my personal weather station when logged in to wunderground it has status green but shows not reporting (see attached PWS.jpg)

If I look at the weather station status it shows that it is reporting but there is no data (see attached WSStatus.jpg)

What am I doing wrong?

Thanks

[Duncan McGugan]

I think I found the answer. From the wunderground PWS Registation FAQs:

It takes about 1 day for our database to update all of our web servers and place your station on your local forecast page. Also, if we don't have current data from your station, then we will remove it automatically from the list until we get the next update. Also, make sure you have correctly entered your ID in all capitals. The password is case sensitive too.

I'll come back tomorrow with an update.

[Duncan McGugan]

All working now ☺

Now I need to decide what update interval to set 😐

--
You received this message because you are subscribed to a topic in the Google Groups "pywws" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pywws/nCJqCvJLIuA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pywws+un...@googlegroups.com.
To post to this group, send email to py...@googlegroups.com.
Visit this group at http://groups.google.com/group/pywws.
For more options, visit https://groups.google.com/d/optout.

[Jim Easterbrook]

On 04/09/15 09:56, Duncan McGugan wrote:
> Is there any way to upload all the historical data to weather
> underground? It looks like it has uploaded a week's worth, but I have
> more than 4 months' worth.

It uploads a week as that takes long enough. You can upload more as follows:
1/ Stop all pywws software and cron jobs.
2/ Edit your installed pywws/services/underground.ini file and change
the 'catchup' value to something larger, e.g. 120 (this is the number of
days of data it will upload).
3/ Remove the 'underground' entry from the [last update] section of
status.ini
4/ Restart pywws and wait.
--
Jim Easterbrook <http://www.jim-easterbrook.me.uk/>

[Duncan McGugan]

Thanks Jim, I found the answer in one of your previous responses in the group. I uploaded all the data and everything seemed fine except there were a few random updates in my data in weather underground that were copies of weather station records with different time stamps and one that was completely wrong. I tried deleting the extra records on weather underground but then the charts wouldn't display properly as the deleted records were still there but just had no data! The historical data then disappeared.

I am trying to upload everything again to see if it keeps it this time. If not, I will create a new weather station ID and follow the procedure below before I run it first time with the 7 day catchup.

[Duncan McGugan]

There is something very strange going on.

If i run pywws-hourly -vvv ~/weather/data the time stamp that is saved in the weather underground data is the time that the data is posted to weather underground in this case 21:12

21:12:07:pywws.ToService(underground):dateutc=2015-09-04+19%3A05%3A50&windgustmph=1.57&tempf=78.1&windspeedmph=0.00&baromin=29.7928&rainin=0&dailyrainin=0&winddir=315&action=updateraw&softwaretype=pywws&dewptf=63.5&humidity=61&ID=IREGINDE96&PASSWORD=

whereas, if I run 

python -m pywws.toservice -cvv ~/weather/data underground the time stamp that is saved in the weather underground data is the time in the weather station record in this case 21:05

21:19:16:pywws.ToService(underground):dateutc=2015-09-04+19%3A05%3A50&windgustmph=1.57&tempf=78.1&windspeedmph=0.00&baromin=29.7928&rainin=0&dailyrainin=0&winddir=315&action=updateraw&softwaretype=pywws&dewptf=63.5&humidity=61&ID=IREGINDE96&PASSWORD=

The log output appears to be saying it is posting exactly the same data to weather underground.

Why would there be this difference?

[Duncan McGugan]

I created a new weather station ID on weather underground and started from scratch.

I killed the cron job, edited underground.ini to 140 days catchup, edited status.ini to remove underground last update, edited weather.ini to point at the new weather station ID. Uploaded all the data. Then re-started the cron job.

I got the same issue.

All the historic data uploaded fine but I got a few spurious records added into weather underground that had no matching weather station records. Can't figure out whether weather underground is doing this or pywws.

The raw data and weather underground data are below:

Time Temperature Dew Point Humidity Wind Speed Gust Pressure Precip. Rate. Precip. Accum.

11:36 PM 24.1 °C 19.6 °C 76 % South 3.5 kph 9.7 kph 1010.4 hPa 0 mm 0 mm

11:39 PM 24.1 °C 19.6 °C 76 % South 3.5 kph 9.7 kph 1010.4 hPa 0 mm 0 mm

12:05 AM 23.8 °C 20.3 °C 81 % East 3.5 kph 6.1 kph 1010.4 hPa 0 mm 0 mm

12:18 AM 31.8 °C 12.6 °C 31 % East 7.2 kph 9.7 kph 1014.1 hPa 0 mm 0 mm

12:24 AM 24.9 °C 20.2 °C 75 % ENE 0 kph 1.1 kph 1014.8 hPa 0 mm 0 mm

12:35 AM 23.6 °C 19.7 °C 79 % SE 2.6 kph 3.5 kph 1010.7 hPa 0 mm 0 mm

12:44 AM 34.7 °C 20.6 °C 44 % NW 1.1 kph 2.6 kph 1010 hPa 0 mm 0 mm

2015-09-04 21:36:02,30,58,27.6,76,24.1,1002.7,1010.5,1.0,2.7,8.0,5.1,0

2015-09-04 22:05:56,30,59,27.6,81,23.8,1002.8,1010.6,1.0,1.7,4,5.1,0

2015-09-04 22:35:56,30,60,27.6,79,23.6,1003.0,1010.8,0.7,1.0,6,5.1,0

The records in red are the extras. The first looks like a copy. The ones in bold I have no idea where they came from!

Any ideas?

[Jim Easterbrook]

On 05/09/15 00:04, Duncan McGugan wrote:
> I created a new weather station ID on weather underground and started
> from scratch.
>
> I killed the cron job, edited underground.ini to 140 days catchup,
> edited status.ini to remove underground last update, edited weather.ini
> to point at the new weather station ID. Uploaded all the data. Then
> re-started the cron job.
>
> I got the same issue.
>
> All the historic data uploaded fine but I got a few spurious records
> added into weather underground that had no matching weather station
> records. Can't figure out whether weather underground is doing this or
> pywws.
>
> The raw data and weather underground data are below:
>
> TimeTemperatureDew PointHumidityWindSpeedGustPressurePrecip.
> Rate.Precip. Accum.
> 11:36 PM24.1 °C19.6 °C76 %South3.5 kph9.7 kph1010.4 hPa0 mm0 mm

> /11:39 PM24.1 °C19.6 °C76 %South3.5 kph9.7 kph1010.4 hPa0 mm0 mm/
> 12:05 AM23.8 °C20.3 °C81 %East3.5 kph6.1 kph1010.4 hPa0 mm0 mm
> *12:18 AM31.8 °C12.6 °C31 %East7.2 kph9.7 kph1014.1 hPa0 mm0 mm*
> 12:24 AM24.9 °C20.2 °C75 %ENE0 kph1.1 kph1014.8 hPa0 mm0 mm
> 12:35 AM23.6 °C19.7 °C79 %SE2.6 kph3.5 kph1010.7 hPa0 mm0 mm
> *12:44 AM34.7 °C20.6 °C44 %NW1.1 kph2.6 kph1010 hPa0 mm0 mm*

You could search your calib data for 34.7 - I don't suppose you've had
that temperature often.

> 2015-09-04 21:36:02,30,58,27.6,76,24.1,1002.7,1010.5,1.0,2.7,8.0,5.1,0
> 2015-09-04 22:05:56,30,59,27.6,81,23.8,1002.8,1010.6,1.0,1.7,4,5.1,0
> 2015-09-04 22:35:56,30,60,27.6,79,23.6,1003.0,1010.8,0.7,1.0,6,5.1,0
>
> The records in red are the extras. The first looks like a copy. The ones
> in bold I have no idea where they came from!
>
> Any ideas?

Not really. If you are doing 'rapid fire' uploads then Weather
Underground decides which ones to keep - they usually won't correspond
to your logged data. I've not seen them putting in spurious data though.