forecast extension version 3.2.0

[mwall]

this is a significant update to the forecasting extension.  since the last stable release (3.0.8), this adds:

- support for more forecast sources, including
  - US National Weather Service (NWS)
  - the weather underground (WU)
  - open weathermap (OWM)
  - UK Met Office (UKMO)
  - Aeris Weather
  - World Weather Online (WWO)
  - Zambretti
  - tide predictions using xtide

- built-in templates for:
  - tabular display
  - strip display
  - simplified iconic display

see the wiki for screenshots and details:

https://github.com/weewx/weewx/wiki/forecasting

m

[Thomas Keffer]

As an FYI, the NWS is coming out with a new API. 

https://forecast-v3.weather.gov/documentation

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[vince]

see the wiki for screenshots and details:

https://github.com/weewx/weewx/wiki/forecasting

How do you get the first screenshot to happen ?

(the 'simple display of one week forecast')

 

[Louis De Lange]

Thanks for a really great tool, and in general thanks for all the support you and Tom and others are providing.

I am currently using version 3.08, and it has an issue that the forecast quantities for rain or snow are in US units.  

Although I can change the display units to metric, the forecast shows up with really weird quantities that are obviously related to the US units in the base data.  For example, today's forecast (WU source, btw) shows 3.3mm preciptation, or 33.02mm of snow.

Is this issue solved in the new version?  

[mwall]

On Friday, December 9, 2016 at 11:59:36 AM UTC-5, vince wrote:

How do you get the first screenshot to happen ?

(the 'simple display of one week forecast')

the screenshots on the wiki are png images, but the forecast extension emits html.

include the file forecast_iconic.inc in one of your weewx template files.  for example:

#include "forecast_iconic.inc"

will do a 7 day forecast using WU data.

#set global $forecast_iconic_settings = dict()
#set global $forecast_iconic_settings['source'] = 'UKMO'
#set global $forecast_iconic_settings['num_days'] = 10
#include "forecast_iconic.inc"

will do a 10-day forecast using UK Met data.

you can configure in your template just before the include, in the skin.conf, or in weewx.conf - whichever best fits your customization/configuration procedures.

these assume that you have installed the forecasting extension, i.e., the search list extension is in your skin's search list extension list, and the forecast service is obtaining data from the forecast sources.

notice that the weewx forecasting extension is different from embedded forecasts in common wx templates such as meteotemplate, saratoga, or leuven.

the weewx forecasting extension pulls data into a single forecasting database whose schema works with many different forecast services.  that means you can pull data from different sources, but use a single, consistent syntax for displaying the data within weewx templates.

the weewx forecasting extension provides canned displays of forecast data (iconic, strip, and table), but it also enables you to display fine-grained (or aggregate) forecast data anywhere and anyway you want within your templates.

m

[mwall]

On Friday, December 9, 2016 at 10:33:35 AM UTC-5, Tom Keffer wrote:

As an FYI, the NWS is coming out with a new API. 

https://forecast-v3.weather.gov/documentation

luckily the forecast extension needs only a change to the nws url, as nws will continue to provide the text forecast product.

once the json api stabilizes (and the nws servers respond consistently) it will be interesting to add other nws forecast options in addition to the point forecasts.

[mwall]

sorry, i did not realize it was an issue in 3.0.8.

you'll have to be more specific about the context.  did you see this problem when including the table?  when using $forecast a certain way?

i just tried metric units on a 3.2.0 installation and it seems to work (see attached screenshots)

m

[Louis De Lange]

I marked up one of the files you attached to show where the problem is.  

In this case it shows snow of 25.4mm, which is not a sensible number to display since the forecast cannot possibly be that accurate.  

But knowing that there is 25.4mm in an inch leads me to believe that the forecast data is for 1 inch of snow, just getting displayed in mm.

[vince]

On Friday, December 9, 2016 at 10:33:56 AM UTC-8, mwall wrote:

On Friday, December 9, 2016 at 11:59:36 AM UTC-5, vince wrote:

How do you get the first screenshot to happen ?

(the 'simple display of one week forecast')

the screenshots on the wiki are png images, but the forecast extension emits html.

sure - I was asking how did you make that output happen (the bars) but you answered below....

 

include the file forecast_iconic.inc in one of your weewx template files.  for example:

#include "forecast_iconic.inc"

will do a 7 day forecast using WU data.

#set global $forecast_iconic_settings = dict()
#set global $forecast_iconic_settings['source'] = 'UKMO'
#set global $forecast_iconic_settings['num_days'] = 10
#include "forecast_iconic.inc"

yeah - it's been so many years since I installed it, I had forgotten.  Thanks.

With a vanilla installation of forecast, I'm seeing a missing hilo.inc file in my logs (none is in the .tgz ball):

Dec  9 12:31:19 debian weewx[11249]: reportengine: copied 0 files to /home/weewx/public_html/forecast

Dec  9 12:31:20 debian weewx[11249]: ****  Traceback (most recent call last):

Dec  9 12:31:20 debian weewx[11249]: ****    File "/home/weewx/bin/weewx/cheetahgenerator.py", line 315, in generate

Dec  9 12:31:20 debian weewx[11249]: ****      print >> _file, text

Dec  9 12:31:20 debian weewx[11249]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__

Dec  9 12:31:21 debian weewx[11249]: ****      rc = getattr(self, mainMethName)()

Dec  9 12:31:21 debian weewx[11249]: ****    File "_home_weewx_skins_Standard_index_html_tmpl.py", line 251, in respond

Dec  9 12:31:21 debian weewx[11249]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1597, in _handleCheetahInclude

Dec  9 12:31:21 debian weewx[11249]: ****      nestedTemplateClass = compiler.compile(source=source, file=file)

Dec  9 12:31:21 debian weewx[11249]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 706, in compile

Dec  9 12:31:21 debian weewx[11249]: ****      fileHash += str(os.path.getmtime(file))

Dec  9 12:31:21 debian weewx[11249]: ****    File "/usr/lib/python2.7/genericpath.py", line 54, in getmtime

Dec  9 12:31:21 debian weewx[11249]: ****      return os.stat(filename).st_mtime

Dec  9 12:31:21 debian weewx[11249]: ****  OSError: [Errno 2] No such file or directory: '/home/weewx/skins/forecast/hilo.inc'

[mwall]

On Friday, December 9, 2016 at 3:36:36 PM UTC-5, vince wrote:

With a vanilla installation of forecast, I'm seeing a missing hilo.inc file in my logs (none is in the .tgz ball):

vince,

i don't know where hilo is coming from - are you certain you're working with a vanilla installation?

the only use of hilo in forecast is a member of the $tide variable.

m

[mwall]

On Friday, December 9, 2016 at 2:11:18 PM UTC-5, Louis De Lange wrote:

I marked up one of the files you attached to show where the problem is.  

In this case it shows snow of 25.4mm, which is not a sensible number to display since the forecast cannot possibly be that accurate.  

But knowing that there is 25.4mm in an inch leads me to believe that the forecast data is for 1 inch of snow, just getting displayed in mm.

louis,

i understand a bit more, but i still do not understand what you want.

the forecast values come from the various forecast services, then conversions are done in weewx as necessary.

if the forecast is for 1 inch, and you specified that it should be displayed in mm, then you'll see 25.4 mm.  if the forecast is for 1.01 inches, you'll see 25.7, or 25.65, or 25.654, depending on the precision of the unit formatting.

if you prefer to see the snow depth in centimeters instead of millimeters, set group_rain = cm

or are you asking that rain should be displayed in mm but snow should be displayed in cm?

or something else?

m

[gjr80]

I think the issue is simply one of forecasting precisely 25.4mm of rain seems wrong since forecasts are never issued to tenths of mm. Not sure there is much you can do about it, it's a fact of converting inches to mm, you could change the precision of what is displayed, might look ok. 2 inches would display as 51mm which may or may not be just as a nappealing. Otherwise find a forecast source that is in your preferred units I guess.

Gary

[Dave Webb KB1PVH]

Just did a new 3.6.2 install along with the new forecast extension to mess around with and get the following errors. I haven't done anything other than install and put the correct NWS info in.

Dec  9 21:30:29 weewxpi weewx[2701]: cheetahgenerator: Generated 15 files for report StandardReport in 8.98 seconds

Dec  9 21:30:38 weewxpi weewx[2701]: genimages: Generated 21 images for StandardReport in 9.06 seconds

Dec  9 21:30:38 weewxpi weewx[2701]: reportengine: copied 0 files to /home/weewx/public_html

Dec  9 21:30:43 weewxpi weewx[2701]: cheetahgenerator: Generate failed with exception '<type 'exceptions.TypeError'>'

Dec  9 21:30:43 weewxpi weewx[2701]: cheetahgenerator: **** Ignoring template /home/weewx/skins/forecast/iconic.html.tmpl

Dec  9 21:30:43 weewxpi weewx[2701]: cheetahgenerator: **** Reason: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

Dec  9 21:30:43 weewxpi weewx[2701]: ****  Traceback (most recent call last):

Dec  9 21:30:43 weewxpi weewx[2701]: ****    File "/home/weewx/bin/weewx/cheetahgenerator.py", line 315, in generate

Dec  9 21:30:43 weewxpi weewx[2701]: ****      print >> _file, text

Dec  9 21:30:43 weewxpi weewx[2701]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__

Dec  9 21:30:43 weewxpi weewx[2701]: ****      rc = getattr(self, mainMethName)()

Dec  9 21:30:43 weewxpi weewx[2701]: ****    File "_home_weewx_skins_forecast_iconic_html_tmpl.py", line 101, in respond

Dec  9 21:30:43 weewxpi weewx[2701]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1615, in _handleCheetahInclude

Dec  9 21:30:43 weewxpi weewx[2701]: ****      self._CHEETAH__cheetahIncludes[_includeID].respond(trans)

Dec  9 21:30:43 weewxpi weewx[2701]: ****    File "_home_weewx_skins_forecast_forecast_iconic_inc.py", line 173, in respond

Dec  9 21:30:43 weewxpi weewx[2701]: ****  TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

Dec  9 21:30:49 weewxpi weewx[2701]: cheetahgenerator: Generated 8 files for report forecast in 11.21 seconds

Dec  9 21:30:49 weewxpi weewx[2701]: reportengine: copied 0 files to /home/weewx/public_html/forecast

Dave-KB1PVH

On Fri, Dec 9, 2016 at 7:40 PM, gjr80 <gjrod...@gmail.com> wrote:

I think the issue is simply one of forecasting precisely 25.4mm of rain seems wrong since forecasts are never issued to tenths of mm. Not sure there is much you can do about it, it's a fact of converting inches to mm, you could change the precision of what is displayed, might look ok. 2 inches would display as 51mm which may or may not be just as a nappealing. Otherwise find a forecast source that is in your preferred units I guess.

Gary

[mwall]

On Friday, December 9, 2016 at 9:36:55 PM UTC-5, Dave Webb wrote:

Just did a new 3.6.2 install along with the new forecast extension to mess around with and get the following errors. I haven't done anything other than install and put the correct NWS info in.

sorry about that - you'll see that if you try to generate the report when there is no forecast data yet.  it should be ok once you have some forecast data to work with.

or download 3.2.1, which skips the report if there are no forecast data.

m

[Dave Webb KB1PVH]

I have NWS data.

Dec  9 21:20:21 weewxpi weewx[2701]: forecast: NWSThread: NWS: got 40 forecast records for WORCESTER-WORCESTER MA 42.29N  71.89W ELEV. 1009 FT

Dec  9 21:20:22 weewxpi weewx[2701]: forecast: NWSThread: NWS: saved 40 forecast records

Dec  9 21:20:22 weewxpi weewx[2701]: forecast: NWSThread: NWS: deleted forecasts prior to 1480731622

Dec  9 21:20:42 weewxpi weewx[2701]: cheetahgenerator: Generated 15 files for report StandardReport in 21.12 seconds

Dec  9 21:20:45 weewxpi weewx[2701]: genimages: Generated 12 images for StandardReport in 2.78 seconds

Dec  9 21:20:45 weewxpi weewx[2701]: reportengine: copied 22 files to /home/weewx/public_html

Dec  9 21:21:03 weewxpi weewx[2701]: cheetahgenerator: Generate failed with exception '<type 'exceptions.TypeError'>'

Dec  9 21:21:03 weewxpi weewx[2701]: cheetahgenerator: **** Ignoring template /home/weewx/skins/forecast/iconic.html.tmpl

Dec  9 21:21:03 weewxpi weewx[2701]: cheetahgenerator: **** Reason: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

Dec  9 21:21:03 weewxpi weewx[2701]: ****  Traceback (most recent call last):

Dec  9 21:21:03 weewxpi weewx[2701]: ****    File "/home/weewx/bin/weewx/cheetahgenerator.py", line 315, in generate

Dec  9 21:21:03 weewxpi weewx[2701]: ****      print >> _file, text

Dec  9 21:21:03 weewxpi weewx[2701]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__

Dec  9 21:21:03 weewxpi weewx[2701]: ****      rc = getattr(self, mainMethName)()

Dec  9 21:21:03 weewxpi weewx[2701]: ****    File "_home_weewx_skins_forecast_iconic_html_tmpl.py", line 101, in respond

Dec  9 21:21:03 weewxpi weewx[2701]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1615, in _handleCheetahInclude

Dec  9 21:21:03 weewxpi weewx[2701]: ****      self._CHEETAH__cheetahIncludes[_includeID].respond(trans)

Dec  9 21:21:03 weewxpi weewx[2701]: ****    File "_home_weewx_skins_forecast_forecast_iconic_inc.py", line 173, in respond

Dec  9 21:21:03 weewxpi weewx[2701]: ****  TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

Dec  9 21:21:14 weewxpi weewx[2701]: cheetahgenerator: Generated 8 files for report forecast in 28.81 seconds

Dave-KB1PVH

--

[mwall]

On Friday, December 9, 2016 at 9:56:03 PM UTC-5, Dave Webb wrote:

I have NWS data.

forecast_iconic defaults to using WU data.  specify NWS instead in skin.conf:

[Extras]
    [[forecast_iconic_settings]]
        source = NWS

that should do it!

m

[Dave Webb KB1PVH]

I'll change it.

Thanks

Dave-KB1PVH


Sent from my Galaxy S7

--

[mwall]

i updated the wiki with more details about how to use the .inc files.

https://github.com/weewx/weewx/wiki/forecasting

hopefully that helps a bit

m

[Dave Webb KB1PVH]

I made the change you suggested and I still got the errors so I added WU key and am still getting errors.

Dec  9 23:15:22 weewxpi weewx[3478]: forecast: WUThread: WU: got 240 forecast records

Dec  9 23:15:25 weewxpi weewx[3478]: forecast: WUThread: WU: saved 240 forecast records

Dec  9 23:15:25 weewxpi weewx[3478]: forecast: WUThread: WU: deleted forecasts prior to 1480738525

Dec  9 23:15:42 weewxpi weewx[3478]: cheetahgenerator: Generated 15 files for report StandardReport in 21.84 seconds

Dec  9 23:15:45 weewxpi weewx[3478]: genimages: Generated 12 images for StandardReport in 2.89 seconds

Dec  9 23:15:45 weewxpi weewx[3478]: reportengine: copied 22 files to /home/weewx/public_html

Dec  9 23:16:26 weewxpi weewx[3478]: cheetahgenerator: Generate failed with exception '<type 'exceptions.TypeError'>'

Dec  9 23:16:26 weewxpi weewx[3478]: cheetahgenerator: **** Ignoring template /home/weewx/skins/forecast/iconic.html.tmpl

Dec  9 23:16:26 weewxpi weewx[3478]: cheetahgenerator: **** Reason: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

Dec  9 23:16:26 weewxpi weewx[3478]: ****  Traceback (most recent call last):

Dec  9 23:16:26 weewxpi weewx[3478]: ****    File "/home/weewx/bin/weewx/cheetahgenerator.py", line 315, in generate

Dec  9 23:16:26 weewxpi weewx[3478]: ****      print >> _file, text

Dec  9 23:16:26 weewxpi weewx[3478]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__

Dec  9 23:16:26 weewxpi weewx[3478]: ****      rc = getattr(self, mainMethName)()

Dec  9 23:16:26 weewxpi weewx[3478]: ****    File "_home_weewx_skins_forecast_iconic_html_tmpl.py", line 101, in respond

Dec  9 23:16:26 weewxpi weewx[3478]: ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1615, in _handleCheetahInclude

Dec  9 23:16:26 weewxpi weewx[3478]: ****      self._CHEETAH__cheetahIncludes[_includeID].respond(trans)

Dec  9 23:16:26 weewxpi weewx[3478]: ****    File "_home_weewx_skins_forecast_forecast_iconic_inc.py", line 173, in respond

Dec  9 23:16:26 weewxpi weewx[3478]: ****  TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

Dec  9 23:17:01 weewxpi /USR/SBIN/CRON[3618]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

Dec  9 23:18:01 weewxpi weewx[3478]: cheetahgenerator: Generated 8 files for report forecast in 135.68 seconds

Dec  9 23:18:01 weewxpi weewx[3478]: reportengine: copied 85 files to /home/weewx/public_html/forecast

Dec  9 23:20:20 weewxpi weewx[3478]: manager: added record 2016-12-09 23:20:00 EST (1481343600) to database 'weewx.sdb'

Dave-KB1PVH

--

[mwall]

On Friday, December 9, 2016 at 11:27:05 PM UTC-5, Dave Webb wrote:

I made the change you suggested and I still got the errors so I added WU key and am still getting errors.

dave,

is it working now?

assuming you restarted weewx after adding a WU key to weewx.conf, the first download of WU data should happen at the next archive interval, then the forecast should display properly at the next time the forecast page is generated.  by default, the forecast pages are regenerated every hour.

if it is still not working, please post the WU location so i can pull the WU forecast to see what is happening.  if you did not specify a location specifically for the forecast, then weewx will use the station lat/lon.

m

[Dave Webb KB1PVH]

m,

 I let it run for at least a couple archive periods after restarting. I'll have to check the log on it later after the dust settles after morning horse chores. 

Station lat/long

42.1603

-71.7648

Dave-KB1PVH


Sent from my Galaxy S7

--

[Dave Webb KB1PVH]

I just checked the forecast pages and all of them are being generated now. Last night it had all but the iconic page. I'll check the log later and let you know.

Dave-KB1PVH


Sent from my Galaxy S7

[mwall]

more fun!  3.2.3 includes an 'orientation' option for the iconic view.  horizontal for time increasing across the page, vertical for time increasing down the page.

[Dave Webb KB1PVH]

M,

Just a follow up...

No more errors on the forecasting now. 

Dave-KB1PVH


Sent from my Galaxy S7

On Dec 10, 2016 2:08 PM, "mwall" <mw...@users.sourceforge.net> wrote:

more fun!  3.2.3 includes an 'orientation' option for the iconic view.  horizontal for time increasing across the page, vertical for time increasing down the page.

[Louis De Lange]

It was absolutely a matter of formatting.  With the right formatting I can round to whole numbers of rain or snow.

I would prefer to display rain in mm and snow in cm, but there is only one group unit for precipitation.

[gjr80]

I suspect it is easy enough to change to have rainfall in mm and snow in cm, though it will be at the expense of creating an orphan. I am not familiar with the latest forecast templates but if Mathew is true to form he will have used ValueHelpers for his tags (I had a quick look at one template and that appears to be the case) so you can use the dot code to force a particular unit for a particular tag in the template. For example:

$period.qsf.nolabel('%.2f',' ')

would use whatever default units are defined in weex.conf/skin.conf. You can force this to cm by using:

$period.qsf.cm.nolabel('%.2f',' ')

As I said the price you pay is your template is an orphan, you can no longer (1) automatically change units from weex.conf/skin.conf (well you can but any tags you force in the template will not chnage) and (2) if you upgrade the templates when the next version is released you lose your changes.

Gary

[mwall]

On Saturday, December 10, 2016 at 9:27:02 PM UTC-5, Louis De Lange wrote:

It was absolutely a matter of formatting.  With the right formatting I can round to whole numbers of rain or snow.

I would prefer to display rain in mm and snow in cm, but there is only one group unit for precipitation.

dot code to the rescue!  as gary pointed out, you can use the dot code to do cm for snow and mm for rain.

and you can also use the dot code to make the precision for mm different than the precision for cm.

for example, one decimal place for cm, no decimal places for mm:

$period.qpf.cm.nolabel('%.1f', ' ')

$period.qsf.mm.nolabel('%.0f', ' ')

qpf is for rain, qsf is for snow (names are based on NWS conventions).

hope that helps!

m

[Louis De Lange]

Thanks guys, this is exactly the right solution.

 

[mwall]

On Friday, December 9, 2016 at 10:33:35 AM UTC-5, Tom Keffer wrote:

As an FYI, the NWS is coming out with a new API. 

https://forecast-v3.weather.gov/documentation

the weewx forecasting extension uses only one URL for NWS - the URL for point forecast matrices in plain text format:

http://forecast.weather.gov/product.php?site=NWS&product=PFM&format=txt&issuedby=XXX

where XXX is the forecast office identifier, such as BOX for Boston.  the v3 equivalent is:

https://forecast-v3.weather.gov/products/PFM/XXX/1?format=text

if you are using any version of the forecasting extension, you can try the NWS v3 servers by doing this in weewx.conf:

[Forecast]
    [[NWS]]
        ...
        url = https://forecast-v3.weather.gov/products/PFM/XXX/1?format=text

the v3 servers have been available since early november 2016.  i think they are supposed to go live in january 2017.  if you keep using the old URLs they should redirect to the new api, at least for awhile.

m

[Mario Romanello]

Hello and congratulations for a job well done with this extension. Could you help me? I installed the extension, but do not know how to implement it on my site, I could use a little guidance. Excuse me but I am a beginner, thank you so much!

[Andrew Milner]

Can you possibly be more specific?  The wiki gives lots of information as to how to include forecasts on your existing pages - and also provides the forecast skin to generate specific forecast page.  For all forecasts except zambretti you will however need to register with the specific forecast service and obtain the necessary keys/logins/etc as specified in the forecast wiki prerequisites.

[Mario Romanello]

Can i help me please?
Thanks

[Mario Romanello]

Is it correct?

weewx.conf:

# Options for extension 'forecast'

[Forecast]

data_binding = forecast_binding

[[OWM]]

api_key = INSERT_OWM_API_KEY_HERE

[[WWO]]

api_key = INSERT_WWO_API_KEY_HERE

[[WU]]

api_key = 9ed8cb535d935e48

[StdReport]

[[forecast]]

[[[Extras]]]

[[[[forecast_iconic_settings]]]]

source = WU

num_days = 7

HTML_ROOT = public_html/forecast

skin = forecast

in my html page:

<div class="right" >

<div class="card" >

<h1><i class="fa fa-umbrella m-blau" ></i> Coming soon...</h1>

#set global $forecast_iconic_settings = dict()

#set global $forecast_iconic_settings['source'] = 'WU'

#set global $forecast_iconic_settings['num_days'] = 7

#include "forecast_iconic.inc"

</div>

I'm sorry for my english......thanks.