wee_import of Weather Underground in Weewx 4.0

[Mike Revitt]

Just trying to import some data from Weather Underground

I am getting an SSL Certificate error, even when trying to connect to my own station.

Guessing there is a certificate I need, but can't work out which one.

This is the one I want to import

Using WeeWX configuration file /Users/Mike/OneDrive/Mike/WebSites/Weewx/ModifiedScripts/4.0.0/weewx-iMac-sim.conf

Starting wee_import...

Observation history for Weather Underground station 'EGNM' will be imported.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

This is a dry run, imported data will not be saved to archive.

Starting dry run import ...

Records covering multiple periods have been identified for import.

Unable to open Weather Underground station EGNM  or  <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

**** Unable to load config file.

**** <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

**** Nothing done, exiting.

This is what happens if I use my own station ID

Using WeeWX configuration file /Users/Mike/OneDrive/Mike/WebSites/Weewx/ModifiedScripts/4.0.0/weewx-iMac-sim.conf

Starting wee_import...

Observation history for Weather Underground station 'iHUDDERS4' will be imported.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

This is a dry run, imported data will not be saved to archive.

Starting dry run import ...

Records covering multiple periods have been identified for import.

Unable to open Weather Underground station iHUDDERS4  or  <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

**** Unable to load config file.

**** <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

**** Nothing done, exiting.

Don't know what ssl.c.749 is, but that looks like the culprit

[Thomas Keffer]

Check this SO thread about certificate problems on Macs: https://stackoverflow.com/questions/41691327/ssl-sslerror-ssl-certificate-verify-failed-certificate-verify-failed-ssl-c

Or, it could be WU messing around with their certificates, again. It has been a recurring problem for the last 2 months. See https://apicommunity.wunderground.com/weatherapi/topics/ssl-certificate-error-9lwea9u1krwgd

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/118393a5-3361-48c5-a22b-90578620fc83%40googlegroups.com.

[Mike Revitt]

Thanks Thomas,

I reverted back to my version 3 installation on Python 2 and that got be past the SSL error, now onto the next one

tarting wee_import...

Observation history for Weather Underground station 'EGNM' will be imported.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

This is a dry run, imported data will not be saved to archive.

Unable to open Weather Underground station EGNM

   **** HTTP Error 403: Forbidden

**** Unable to load config file.

**** HTTP Error 403: Forbidden

**** Nothing done, exiting.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[Mike Revitt]

And when I use the api call from the help pages of Weather Underground with my site ID it works

https://api.weather.com/v2/pws/history/hourly?stationId=IHUDDERS4&format=json&units=m&date=20181001&apiKey=yourApiKey

But not consistantly

[Thomas Keffer]

Just tried it with my own key, and it works.

Mike, that's just the way the WU is. It's a very unreliable API, full of frustrations. We try our best with wee_import and wunderfixer, but we've also learned not to put too much energy into chasing WU bugs. 

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b0d27434-d223-4e1c-b02a-f34b67e0a8de%40googlegroups.com.

[Mike Revitt]

Thanks Thomas,

I was trying to download the information from the local airport which goes back 50 odd years, would have been nice to have a bit of historic trending. But I will try to get that another way

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b0d27434-d223-4e1c-b02a-f34b67e0a8de%40googlegroups.com.

[gjr80]

Just for info wee_import included with WeeWX 3.x uses the old WU WXDailyHistory.asp page to obtain PWS daily records. WXDailyHistory.asp has been retired for around a year (which is why wee_import was revised in WeeWX 4.0 to use the new API) so seeing 403 errors when using wee_import from WeeWX 3.x is to be expected.

FWIW your PWS works fine with my API key, not so for EGNM though. But then again we have been pretty immune from WU 'issues' down here. Perhaps this week it is Europe/UK's turn for special treatment :)

Gary

[Nicolas CUVILLIER]

And I was able to import more than 4 years of historical data from WUnderground

Better to use the current version 4.0.0b13. which includes a couple of fixes.

I imported only period where Wunderground has data from my station.

Nicolas

[Günther Wrana]

I am trying to complete my patchy database. This is the first dry run to import data from wunderground.

I just don't think that would work.

You used the new version of weewx and then it worked with the import of wunderground?

pi@raspberrypi:~ $ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/wu.conf --from=2019-11-01T00:00 --to=2019-11-01T23:59 --dry-run

Starting wee_import...

Observation history for Weather Underground station 'IMATZENR2' will be imported.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

Observations timestamped after 2019-11-01 00:00:00 CET (1572562800) and up to and

including 2019-11-01 23:59:00 CET (1572649140) will be imported.

This is a dry run, imported data will not be saved to archive.

Unable to open Weather Underground station IMATZENR2

   **** HTTP Error 403: Forbidden

**** Unable to load config file.

**** HTTP Error 403: Forbidden

**** Nothing done, exiting.

[gjr80]

Hi,

Are you using wee_import from WeeWX 4.0 or from an earlier WeeWX version? The only version of wee_import that will (now) work with WU is that from WeeWX 4.0. Earlier versions will not work.

Gary

[Günther Wrana]

Hi

What should I watch out for when installing version 4?

I would like to try it on another computer, whether importing from WU works with this version.

Günther

[gjr80]

Hallo Günther,

What should I watch out for when installing version 4?

Probably the biggest thing is that whilst in beta the WeeWX 4.0 install is available via setup.py only (no .rpm or .deb) so you need to manually download the package, install the pre-requisites and then install WeeWX. Also it is not advisable to install a 4.0 beta over an existing 3.x package (.deb or .rpm) due to the different locations used; a clean (WeeWX) install will be fine though. A WeeWX 4.0 setup.py install is a little more complex due to the support for python2 and python 3; choose either python 2 or 3 and follow the install instructions in the 4.0.0 docs. Not the best (links probably won't work) but you can view the 4.0.0 setup.py install instructions here. Once you have WeeWX 4.0.0 installed on your device you can browse the 4.0.0 versions of the documents on your device (WeeWX website only shows the current release docs which at this time is still 3.9.2).

 

I would like to try it on another computer, whether importing from WU works with this version.

As long as you are using 4.0.0b13 or later the WU import should be fine. wee_import shipped with WeeWX 3.x will not work as it only supported the old (now retired) WU API. wee_import in WeeWX 4.0.0 supports the new WU API. Usual caveats apply to WU; at times their data is patchy so be wary of having wee_import derive the interval field from the WU data, best to use a fixed interval value in your import config file. I would also tend to avoid importing long periods of data in one go, I would do no more than one month at a time. Also work from oldest to newest data when importing.

Gary