forecasting module for weewx

[mwall]

folks,

i have implemented a forecasting module for weewx in the plotting branch of the weewx repository.  currently it consists of two forecasters - zambretti and nws.  the zambretti does a simple calculation that results in a simple forecast of 'fine weather', 'unsettled, rain later', etc.  the nws downloads and parses a point forecast matrix from the us national weather service.

both zambretti and nws are working now.  if anyone knows of european, australian, uk, or other forecasting services with data similar to the us national weather service point forecast matrices, please let me know.

i am working on icons and skinning so the forecasts can be displayed in any skin.  i am also working on graphs to compare forecasts with actuals.

each forecast is implemented as a service.

output from each are save to a single forecast database.

the intent is to keep the latest N days of forecasts.  those who are interested can turn off database pruning to do analysis of forecast quality.

the forecasting uses the following weewx files:

bin/weewx/forecast.py - contains Forecast base class as well as ZambrettiForecast and NWSForecast classes
bin/user/schemas.py  - contains schema definition for forecast database
bin/weewx/test/test_forecast.py - unit tests for the forecasting module

additions to weewx.conf look like this:

[ZambrettiForecast]                                                            
    # hemispher can be NORTH or SOUTH                                          
    hemisphere = NORTH                                                         
    # how often to calculate the forecast, in seconds                          
    forecast_interval = 300                                                    
                                                                               
[NWSForecast]                                                                  
    # how often to download the forecast, in seconds                           
    forecast_interval = 43200                                                  
    # national weather service location identifier                             
    id = MAZ014                                                                
    # national weather service forecast office identifier                      
    foid = BOX                                                                 
    # url for point forecast matrix                                            
    url = http://forecast.weather.gov/product.php?site=NWS&product=PFM&format=txt                                                                             
                                                                               
[StdArchive]                                                                   
    ...                                                                        
    forecast_database = forecast_sqlite                                        
                                                                               
[Databases]                                                                    
    ...                                                                        
    [[forecast_sqlite]]                                                        
        root = %(WEEWX_ROOT)s                                                  
        database = archive/forecast.sdb                                        
        driver = weedb.sqlite                                                  
                                                                               
    [[forecast_mysql]]                                                         
        host = localhost                                                       
        user = weewx                                                           
        password = weewx                                                       
        database = forecast                                                    
        driver = weedb.mysql                                                   
                                                                               
[Engines]                                                                      
    [[WxEngine]]                                                               
        service_list = ... , weewx.forecast.ZambrettiForecast, weewx.forecast.NWSForecast                                                                     

here is sample output from the forecast database:

Zambretti|1368629665|43200|1|Y|||||||||||||||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368608400|5|||||||||||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368619200|8|||||||||||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368630000|11||60.0|36.0|41.0|S|11.0||B1|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368640800|14||66.0|38.0|36.0|S|16.0||OV|||||C||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368651600|17||65.0|40.0|40.0|S|18.0|31.0|OV|||||C||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368662400|20|68.0|61.0|49.0|65.0|S|17.0||OV|60.0|0.06|00-00||L|S|||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368673200|23||56.0|52.0|86.0|SW|13.0||B2|||||C|S|||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368684000|2||55.0|53.0|93.0|SW|10.0||B1|||||S|||||||||PF|||
NWS|1368627480|43200|1||BOX|MAZ014|1368694800|5||55.0|50.0|83.0|W|11.0||SC|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368705600|8|53.0|60.0|48.0|64.0|W|11.0|21.0|SC|60.0|0.04|00-00||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368716400|11||69.0|48.0|47.0|W|13.0|25.0|FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368727200|14||75.0|45.0|34.0|W|20.0|34.0|FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368738000|17||78.0|43.0|29.0|W|23.0|36.0|FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368748800|20|78.0|68.0|43.0|40.0|W|13.0|32.0|FW|10.0|0.0|00-00||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368759600|23||63.0|42.0|46.0|NW|10.0|25.0|FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368770400|2||59.0|45.0|60.0|W|6.0||FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368781200|5||55.0|45.0|69.0|NW|9.0||FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368792000|8|50.0|57.0|45.0|64.0|NW|11.0||FW|5.0|0.0|||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368802800|11||67.0|42.0|40.0|NW|9.0||FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368813600|14||71.0|40.0|32.0|NW|8.0||FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368824400|17||71.0|40.0|32.0|W|1.0||FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368835200|20|73.0|64.0|41.0|43.0|S|4.0||FW|5.0|0.0|||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368856800|2||54.0|44.0|||||FW|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368878400|8|50.0|56.0|46.0||W|||FW|5.0||||||||||||||LT||
NWS|1368627480|43200|1||BOX|MAZ014|1368900000|14||68.0|42.0|||||SC|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368921600|20|69.0|62.0|44.0||NE|||SC|5.0||||||||||||||LT||
NWS|1368627480|43200|1||BOX|MAZ014|1368943200|2||53.0|47.0|||||SC|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1368964800|8|50.0|54.0|49.0||SE|||B1|10.0||||||||||||||LT||
NWS|1368627480|43200|1||BOX|MAZ014|1368986400|14||62.0|48.0|||||B2|||||||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1369008000|20|63.0|58.0|48.0||SE|||B2|10.0||||||||||||||LT||
NWS|1368627480|43200|1||BOX|MAZ014|1369029600|2||53.0|49.0|||||B2|||||S||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1369051200|8|51.0|55.0|50.0||SE|||B2|20.0||||S||||||||||LT||
NWS|1368627480|43200|1||BOX|MAZ014|1369072800|14||64.0|52.0|||||B2|||||C||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1369094400|20|65.0|60.0|52.0||S|||B2|30.0||||C||||||||||LT||
NWS|1368627480|43200|1||BOX|MAZ014|1369116000|2||55.0|54.0|||||B2|||||S||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1369137600|8|53.0|58.0|55.0||S|||B2|20.0||||S||||||||||LT||
NWS|1368627480|43200|1||BOX|MAZ014|1369159200|14||70.0|57.0|||||B2|||||S||||||||||||
NWS|1368627480|43200|1||BOX|MAZ014|1369180800|20|71.0|65.0|53.0||SW|||B1|20.0||||S||||||||||GN||
Zambretti|1368630000|43200|1|Y|||||||||||||||||||||||||||||
Zambretti|1368630600|43200|1|Y|||||||||||||||||||||||||||||
Zambretti|1368630900|43200|1|Y|||||||||||||||||||||||||||||

this is the forecast schema:

defaultForecastSchema = [('method',     'VARCHAR(10) NOT NULL'),
                         ('dateTime',   'INTEGER NOT NULL'),
                         ('period',     'INTEGER'),
                         ('usUnits',    'INTEGER NOT NULL'),

                         # Zambretti fields                                    
                         ('zcode',      'CHAR(1)'),

                         # NWS fields                                          
                         ('foid',       'CHAR(3)'),     # e.g., BOX            
                         ('id',         'CHAR(6)'),     # e.g., MAZ014         
                         ('ts',         'INTEGER'),     # seconds              
                         ('hour',       'INTEGER'),     # 00 to 23             
                         ('tempMinMax', 'REAL'),        # degree F             
                         ('temp',       'REAL'),        # degree F             
                         ('dewpoint',   'REAL'),        # degree F             
                         ('humidity',   'REAL'),        # percent              
                         ('windDir',    'VARCHAR(2)'),  # N,NE,E,SE,S,SW,W,NW  
                         ('windSpeed',  'REAL'),        # mph                  
                         ('windGust',   'REAL'),        # mph                  
                         ('clouds',     'VARCHAR(2)'),  # CL,SC,BK,OV, ...     
                         ('pop',        'REAL'),        # percent              
                         ('qpf',        'REAL'),        # inch                 
                         ('qsf',        'VARCHAR(5)'),  # inch                 
                         ('rain',       'VARCHAR(2)'),  # S,C,L,O,D            
                         ('rainshwrs',  'VARCHAR(2)'),
                         ('tstms',      'VARCHAR(2)'),
                         ('drizzle',    'VARCHAR(2)'),
                         ('snow',       'VARCHAR(2)'),
                         ('snowshwrs',  'VARCHAR(2)'),
                         ('flurries',   'VARCHAR(2)'),
                         ('sleet',      'VARCHAR(2)'),
                         ('frzngrain',  'VARCHAR(2)'),
                         ('frzngdrzl',  'VARCHAR(2)'),
                         ('obvis',      'VARCHAR(2)'),  # F,PF,F+,PF+,H,BS,K,BD
                         ('windChar',   'VARCHAR(2)'),  # GN,LT                
                         ('windChill',  'REAL'),        # degree F             
                         ('heatIndex',  'REAL'),        # degree F             
                         ]

the intent for the nws data is to have a sliding forecast window that will result in 'error bars' on forecasts.

comments and suggestions are welcome.

m

[Maarten van der Hoeven]

Thanks. Would it be an idea to change to Wunderground forecast, so that non-US users of weewx also could benefit?

[mwall]

On Wednesday, May 15, 2013 2:08:59 PM UTC-4, Maarten van der Hoeven wrote:

Thanks. Would it be an idea to change to Wunderground forecast, so that non-US users of weewx also could benefit?

hi maarten,

wunderground forecasts are a possibility. do you know of a wunderground source for forecasts?

ideally the forecast would be plain text, from a reliable source/url, with a format that does not change often.  if necessary i can parse html or xml, but i would rather not deal with all of that overhead.

NWS forecasts provide 3- and 6-hour increments for 7 days into the future that come from computational weather models.  i know there are european weather services that do similar, with various computational weather models.  but i don't know details about those yet. 

for example, this is what a point forecast matrix looks like from NWS:

MAZ014-120900-
CAMBRIDGE-MIDDLESEX MA
42.37N  71.12W ELEV. 10 FT
719 PM EDT SAT MAY 11 2013

DATE           05/11/13      SUN 05/12/13            MON 05/13/13            TUE
EDT 3HRLY     17 20 23 02 05 08 11 14 17 20 23 02 05 08 11 14 17 20 23 02 05 08
UTC 3HRLY     21 00 03 06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12

MIN/MAX                      57          71          43          60          40
TEMP             63 60 59 59 62 64 67 67 62 55 49 44 50 55 59 58 53 47 44 42 47
DEWPT            58 58 58 58 57 55 49 42 37 34 32 31 30 29 27 27 29 32 32 32 33
RH               84 93 96 96 84 73 52 40 39 45 52 60 46 37 29 30 39 56 62 67 58
WIND DIR         SW  S  S SW SW  W  W  W  W  W  W  W  W  W  W  W  W  W  W NW NW
WIND SPD         14 10  5  6 10 11 16 18 14 11 10  8 12 13 14 14  9  8  5  6  8
WIND GUST                                27 26 20    23
CLOUDS           OV OV OV OV OV B2 B2 B1 SC FW CL CL CL FW FW SC SC SC FW FW FW
POP 12HR                    100          70          10          10          10
QPF 12HR                   0.27        0.15           0           0           0
SNOW 12HR                 00-00       00-00       00-00
RAIN SHWRS        D  D  L  L  L  C  C  S
TSTMS             L  L  C
OBVIS             F  F  F  F  F


DATE           05/14  WED 05/15/13  THU 05/16/13  FRI 05/17/13  SAT 05/18/13
EDT 6HRLY     14 20   02 08 14 20   02 08 14 20   02 08 14 20   02 08 14 20
UTC 6HRLY     18 00   06 12 18 00   06 12 18 00   06 12 18 00   06 12 18 00

MAX/MIN          59      42    64      49    69      53    70      53    71
TEMP          57 52   42 51 62 61   51 60 65 63   56 63 62 66   56 60 71 63
DEWPT         30 30   33 36 35 39   45 49 52 52   52 53 51 52   50 48 46 46
PWIND DIR         W       W    SW       S     S       W    NE      NW    NW
WIND CHAR        GN      LT    GN      LT    GN      LT    LT      LT    GN
AVG CLOUDS    FW SC   FW FW FW SC   B1 B2 B2 B2   B2 OV OV B2   B1 B1 B1 B1
POP 12HR         10       5    20      20    30      40    40      30    20
RAIN SHWRS                      S    S  S  C  C    C  C  C  C    C  S     S
TSTMS                                   S  C  C          C

[Maarten van der Hoeven]

There’s a fine library of Wunderground API’s, see http://www.wunderground.com/weather/api/d/docs

 

Example given on the WU-site, forecast San Fransisco:

 

http://api.wunderground.com/api/Your_Key/forecast10day/q/CA/San_Francisco.json

 

gives

 

{ "response": { "version": "0.1", "termsofService": "http://www.wunderground.com/weather/api/d/terms.html", "features": { "forecast10day": 1 } }, "forecast": { "txt_forecast": { "date": "8:00 AM PDT", "forecastday": [{ "period": 0, "icon": "partlycloudy", "icon_url": "http://icons-ak.wxug.com/i/c/k/partlycloudy.gif", "title": "Tuesday", "fcttext": "Overcast in the morning, then clear. High of 75F. Winds from the SSW at 5 to 20 mph.", "fcttext_metric": "Overcast in the morning, then clear. High of 24C. Windy. Winds from the SSW at 5 to 30 km/h.", "pop": "0" }, { "period": 1, "icon": "partlycloudy", "icon_url": "http://icons-ak.wxug.com/i/c/k/partlycloudy.gif", "title": "Tuesday Night", "fcttext": "Mostly cloudy. Low of 55F. Winds from the SW at 5 to 15 mph.", "fcttext_metric": "Mostly cloudy. Low of 13C. Breezy. Winds from the SW at 10 to 20 km/h.", "pop": "0" }, { "period": 2, "icon": "partlycloudy", "icon_url": "http://icons-ak.wxug.com/i/c/k/partlycloudy.gif", "title": "Wednesday", "fcttext": "Mostly cloudy in the morning, then clear. High of 72F. Winds from the SW at 10 to 15 mph.", "fcttext_metric": "Mostly cloudy in the morning, then clear. High of 22C. Breezy. Winds from the SW at 15 to 25 km/h.", "pop": "0" }, { "period": 3, "icon": "partlycloudy", "icon_url": "http://icons-ak.wxug.com/i/c/k/partlycloudy.gif", "title": "Wednesday Night", "fcttext": "Mostly cloudy. Fog overnight. Low of 55F. Winds from the SW at 10 to 15 mph.", "fcttext_metric": "Mostly cloudy. Fog overnight. Low of 13C. Breezy. Winds from the SW at 15 to 20 km/h.", "pop": "0" }, { "period": 4, "icon": "partlycloudy", "icon_url": "http://icons-ak.wxug.com/i/c/k/partlycloudy.gif", "title": "Thursday", "fcttext": "Mostly cloudy. Fog early. High of 70F. Winds from the WSW at 10 to 15 mph.", "fcttext_metric": "Mostly cloudy. Fog early. High of 21C. Windy. Winds from the WSW at 15 to 25 km/h.", "pop": "0" }, { "period": 5, "icon": "clear", "icon_url": "http://icons-ak.wxug.com/i/c/k/clear.gif", "title": "Thursday Night", "fcttext": "Partly cloudy. Fog overnight. Low of 55F. Winds from the WNW at 5 to 10 mph.", "fcttext_metric": "Partly cloudy. Fog overnight. Low of 13C. Winds from the WNW at 10 to 15 km/h.", "pop": "0" }, { "period": 6, "icon": "partlycloudy", "icon_url": "http://icons-ak.wxug.com/i/c/k/partlycloudy.gif", "title": "Friday", "fcttext": "Partly cloudy in the morning, then clear. High of 72F. Winds from the West at 5 to 15 mph.", "fcttext_metric": "Partly cloudy in the morning, then clear. High of 22C. Breezy. Winds from the West at 10 to 20 km/h.",

 

… etc (did not copy all text)

 

 

Just need an API-key to apply for, but that’s for free within reasonable calls per hour. Forecast available for all over the world. Even callable with only LAT/LON positioning

[mwall]

maarten,

thank you for the wunderground info.  i have added a WeatherUndergroundForecast class that downloads a forecast from weather underground.  the json interface works very well.

i still need to do some coding/thought to get the wunderground data into the database.  it would be nice to have a single database format that can serve multiple forecasting sources...

m

[fl0atpr...@gmail.com]

Tried setting this up off the wiki last night, but still having a bit of difficulty with the full implementation.  For now, I just commented out the prune call. 

Do I need to let the database build up a bit before reactivating?

Jul 19 12:57:04 localhost weewx[28475]: wxengine: Caught unrecoverable exception in wxengine:
Jul 19 12:57:04 localhost weewx[28475]:     ****  'Archive' object has no attribute 'delRecords'
Jul 19 12:57:04 localhost weewx[28475]:     ****  Traceback (most recent call last):
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/wxengine.py", line 867, in main
Jul 19 12:57:04 localhost weewx[28475]:     ****      engine.run()
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/wxengine.py", line 142, in run
Jul 19 12:57:04 localhost weewx[28475]:     ****      self.dispatchEvent(weewx.Event(weewx.STARTUP))
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/wxengine.py", line 192, in dispatchEvent
Jul 19 12:57:04 localhost weewx[28475]:     ****      callback(event)
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/wxengine.py", line 443, in startup
Jul 19 12:57:04 localhost weewx[28475]:     ****      self._catchup()
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/wxengine.py", line 555, in _catchup
Jul 19 12:57:04 localhost weewx[28475]:     ****      self.engine.dispatchEvent(weewx.Event(weewx.NEW_ARCHIVE_RECORD, record=record, origin='hardware'))
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/wxengine.py", line 192, in dispatchEvent
Jul 19 12:57:04 localhost weewx[28475]:     ****      callback(event)
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/forecast.py", line 191, in update_forecast
Jul 19 12:57:04 localhost weewx[28475]:     ****      self.prune_forecasts(now - self.max_age)
Jul 19 12:57:04 localhost weewx[28475]:     ****    File "/usr/share/weewx/weewx/forecast.py", line 210, in prune_forecasts
Jul 19 12:57:04 localhost weewx[28475]:     ****      self.database.delRecords(ts)
Jul 19 12:57:04 localhost weewx[28475]:     ****  AttributeError: 'Archive' object has no attribute 'delRecords'
Jul 19 12:57:04 localhost weewx[28475]:     ****  Exiting

[mwall]

On Friday, July 19, 2013 2:10:56 PM UTC-4, fl0atpr...@gmail.com wrote:

Tried setting this up off the wiki last night, but still having a bit of difficulty with the full implementation.  For now, I just commented out the prune call. 

Do I need to let the database build up a bit before reactivating?

delRecords is not yet implemented in the weewx.archive.Archive class.  set forecast_max_age to None for now to disable pruning.

[Forecast]
    forecast_max_age = None
 
fwiw, i have been collecting 3 forecasts (zambretti, national weather service, weather underground) for one location for 2 months now.  i'm just starting to collect forecasts for two other locations.  hopefully from these i can work up a display of the current forecast as well as a comparison of forecasts over time against actual weather...

[fl0atpr...@gmail.com]

Alright. Thanks much.

In the mean time I might plug the Zambretti into the generator to achieve something along the lines of the "forecast trend" that is on the display. Shouldn't be too much work to remove it as a service and instead use it as a custom generator.

Alternatively, pulling weather.gov's forecast XML (http://forecast.weather.gov/MapClick.php?lat=35.09385&lon=-89.93466590000003&unit=0&lg=english&FcstType=dwml) and displaying that might be an option, if I get any time to play around with it.

Thanks for your work.

[mwall]

the forecasting module has been moved into its own 'forecast' branch in the repository.  there are three files that are different from the trunk:

bin/weewx/forecast.py
bin/weewx/test/test_forecast.py
bin/user/schemas.py

the forecast.py file contains instructions for what to put in weewx.conf to make forecasting work.

currently (15aug13) there are three forecast methods:

zambretti - calculates forecast based on recent weather conditions
national weather service - downloads the forecast from national weather service
weather underground - downloads the forecast from weather underground

[Derek Sedunary]

I have looked in the weewx repository on SourceForge but cannot find the forecast modules. Where are they? (I do have a feeling that if I'm not smart enough to find them I probably won't be smart enough to use them.

Derek...

[mwall]

hi derek,

the forecasting code now lives in a single file bin/user/forecast.py

you can find the latest in the forecast branch here:

http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/forecast.py

unit tests are here:

http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/test_forecast.py

the zambretti, nws, and wu forecasts are well-tested now.  the xtide forecasting is also solid, but you must install xtides for it to work.

there is not yet any display of the forecast data - i am trying to get that finished off now.  this means that you will be able to download/generate forecast data and save it to database, but skins do not yet have a way to display it.

my intent is to end up with a tabular display of 7 or 10 day forecast, expandable to hour/3-hour/6-hour details depending on the forecast method.  plus tides and sun/moon info in the same table.

beware that the database schema might change.  currently i put everything into a single table.  this might be split into a separate table for each forecasting service - i'll know better once i get the display stuff working.

suggestions are welcome.

m

[mwall]

this is a request for review of (1) the forecast configuration directives, (2) the forecast skin variables, and (3) the forecast database schema.  please provide suggestions about how to reduce the number of directives, improve the defaults, etc.  please provide suggestions about variables syntax.  please provide suggestions for future-proofing the schema, e.g., for use with other types of forecasting or forecasting sources.

to view the (working) code, visit

http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/forecast.py

to try the code, put forecast.py into bin/user directory, modify weewx.conf as described in forecast.py, then restart weewx.

these are the configuration directives that must be added to weewx.conf.  directives that are commented have a default value as indicated.  uncommented directives must be specified.

[Forecast]                                                                     
    # The database in which to record forecast information, defined in the     
    # 'Databases' section of the weewx configuration.                          
    database = forecast_sqlite                                                 
                                                                               
    # How often to calculate/download the forecast, in seconds                 
    #interval = 300                                                            
                                                                               
    # How long to keep old forecasts, in seconds.  use None to keep forever.   
    #max_age = 604800                                                          
                                                                               
    [[Zambretti]]                                                              
        # hemisphere can be NORTH or SOUTH                                     
        #hemisphere = NORTH                                                    
                                                                               
        # mapping between zambretti codes and descriptive labels               
        #labels = "Settled fine", "Fine weather", "Becoming fine", "Fine, becoming less settled", "Fine, possible showers", "Fairly fine, improving", "Fairly fine, possible showers early", "Fairly fine, showery later", "Showery early, improving", "Changeable, mending", "Fairly fine, showers likely", "Rather unsettled clearing later", "Unsettled, probably improving", "Showery, bright intervals", "Showery, becoming less settled", "Changeable, some rain", "Unsettled, short fine intervals", "Unsettled, rain later", "Unsettled, some rain", "Mostly very unsettled", "Occasional rain, worsening", "Rain at times, very unsettled", "Rain at frequent intervals", "Rain, very unsettled", "Stormy, may improve", "Stormy, much rain", "unknown"                                                      
                                                                               
    [[NWS]]                                                                    
        # First figure out your forecast office identifier (foid), then request
        # a point forecast using a url of this form in a web browser:          
        #   http://forecast.weather.gov/product.php?site=NWS&product=PFM&format=txt&issuedby=YOUR_THREE_LETTER_FOID                                           
        # Scan the output for a service location identifier corresponding      
        # to your location.                                                    
                                                                               
        # National Weather Service location identifier                         
        id = MAZ014                                                            
                                                                               
        # National Weather Service forecast office identifier                  
        foid = BOX                                                             
                                                                               
        # URL for point forecast matrix                                        
        #url = http://forecast.weather.gov/product.php?site=NWS&product=PFM&format=txt                                                                        
                                                                               
        # How often to download the forecast, in seconds                       
        #interval = 10800                                                      
                                                                               
    [[WU]]                                                                     
        # An API key is required to access the weather underground.            
        # obtain an API key here:                                              
        #   http://www.wunderground.com/weather/api/                           
        api_key = KEY                                                          
                                                                               
        # The location for the forecast can be one of the following:           
        #   CA/San_Francisco     - US state/city                               
        #   60290                - US zip code                                 
        #   Australia/Sydney     - Country/City                                
        #   37.8,-122.4          - latitude,longitude                          
        #   KJFK                 - airport code                                
        #   pws:KCASANFR70       - PWS id                                      
        #   autoip               - AutoIP address location                     
        #   autoip.json?geo_ip=38.102.136.138 - specific IP address location   
        # If no location is specified, station latitude and longitude are used 
        #location = 02139                                                      
                                                                               
        # How often to download the forecast, in seconds                       
        #interval = 10800                                                      
                                                                               
    [[XTide]]                                                                  
        # Location for which tides are desired                                 
        location = Boston                                                      
                                                                               
        # How often to generate the tide forecast, in seconds                  
        #interval = 604800                                                     
                                                                               
        # How often to prune old tides from database, None to keep forever     
        #max_age = 1209600                                                     

                                                                               
[Databases]                                                                    
    ...                                                                        
    [[forecast_sqlite]]                                                        
        root = %(WEEWX_ROOT)s                                                  
        database = archive/forecast.sdb                                        
        driver = weedb.sqlite                                                  
                                                                               
    [[forecast_mysql]]                                                         
        host = localhost                                                       
        user = weewx                                                           
        password = weewx                                                       
        database = forecast                                                    

        driver = weedb.mysql
 
[Engines]                                                                      
    [[WxEngine]]                                                               

        service_list = ... , user.forecast.ZambrettiForecast, user.forecast.NWSForecast, user.forecast.WUForecast, user.forecast.XTideForecast                
                                                   

these are examples of the use of forecast variables:

code:

$forecast.xtide(0).dateTime                                
$forecast.xtide(0).event_ts                                
$forecast.xtide(0).hilo                                    
$forecast.xtide(0).offset                                  

result:

20-Aug-2013 20:10                                                              
20-Aug-2013 23:04                                                              
H                                                                              
11.56                                                                          

code:

tide forecast as of $forecast.xtide(0).dateTime            
#for $tide in $forecast.xtides():                            
  $tide.hilo of $tide.offset at $tide.event_ts                                 
#end for                                                                       

result:

tide forecast as of 20-Aug-2013 20:10                                          
  H of 11.56 at 20-Aug-2013 23:04                                              
  L of -1.35 at 21-Aug-2013 05:24                                              
  H of 10.73 at 21-Aug-2013 11:38                                              
  L of -0.95 at 21-Aug-2013 17:41                                              

code:

$forecast.zambretti.dateTime                                                   
$forecast.zambretti.event_ts                                                   
$forecast.zambretti.code                                                       
$forecast.zambretti.text                                                       

result:

22-Aug-2013 12:40                                                              
22-Aug-2013 12:40                                                              
A                                                                              
Settled fine                                                                   


here is the forecast schema.  a single schema is used for all forecast methods - xtide, zambretti, NWS, WU.  i am considering a separate schema (and thus separate table) for each forecast method.  opinions and suggsions are welcome.

defaultForecastSchema = [('method',     'VARCHAR(10) NOT NULL'),

                         ('usUnits',    'INTEGER NOT NULL'),

                         ('dateTime',   'INTEGER NOT NULL'),

                         ('event_ts',   'INTEGER'),     # seconds              

                         # Zambretti fields                                    
                         ('zcode',      'CHAR(1)'),

                         # NWS fields                                          
                         ('foid',       'CHAR(3)'),     # e.g., BOX            
                         ('id',         'CHAR(6)'),     # e.g., MAZ014         

                         ('hour',       'INTEGER'),     # 00 to 23             

                         ('tempMin',    'REAL'),        # degree F             
                         ('tempMax',    'REAL'),        # degree F             

                         ('temp',       'REAL'),        # degree F             
                         ('dewpoint',   'REAL'),        # degree F             
                         ('humidity',   'REAL'),        # percent              

                         ('windDir',    'VARCHAR(3)'),  # N,NE,E,SE,S,SW,W,NW  

                         ('windSpeed',  'REAL'),        # mph                  
                         ('windGust',   'REAL'),        # mph                  

                         ('windChar',   'VARCHAR(2)'),  # GN,LT                

                         ('clouds',     'VARCHAR(2)'),  # CL,SC,BK,OV, ...     
                         ('pop',        'REAL'),        # percent              
                         ('qpf',        'REAL'),        # inch                 
                         ('qsf',        'VARCHAR(5)'),  # inch                 
                         ('rain',       'VARCHAR(2)'),  # S,C,L,O,D            
                         ('rainshwrs',  'VARCHAR(2)'),
                         ('tstms',      'VARCHAR(2)'),
                         ('drizzle',    'VARCHAR(2)'),
                         ('snow',       'VARCHAR(2)'),
                         ('snowshwrs',  'VARCHAR(2)'),
                         ('flurries',   'VARCHAR(2)'),
                         ('sleet',      'VARCHAR(2)'),
                         ('frzngrain',  'VARCHAR(2)'),
                         ('frzngdrzl',  'VARCHAR(2)'),

                         ('obvis',      'VARCHAR(3)'),  # F,PF,F+,PF+,H,BS,K,BD

                         ('windChill',  'REAL'),        # degree F             
                         ('heatIndex',  'REAL'),        # degree F             

                         # tide fields                                         
                         ('hilo',     'CHAR(1)'),       # H or L               
                         ('offset',   'REAL'),          # relative to mean low 
                         ]

[mwall]

to run the forecast unit tests, put test_forecast.py into bin/user then do this in the weewx source directory:

PYTHONPATH=bin python bin/user/test_forecast.py

the unit tests leave their detritus in /var/tmp/weewx_test

[Lucas Heijst]

Matthew,

I implemented the WUForecast module. Two remarks.

1. I had to download and install first python module python-cjson-1.0.5.

2. The syntax in the following line in weewx.conf is not correct: 

# pws:KCASANFR70       - PWS id

I had to change it into: pws = IPARAMAR4

The MYSQL forecast/archive database is filled with data now (10 rows at a time).

Cheers, Luc

PS 

My weatherstation in Surinam is online now; see: http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IPARAMAR4 and http://www.pwsweather.com/obs/SRPBMUITVL.html

[mwall]

thank you for the feedback, lucas.  here is a screenshot of the output from the forecast module (NWS) in the exfoliation skin in tabular form.  you can expand each day to get the 3-hourly and 6-hourly forecast details.

[Thomas Keffer]

Great progress!

Some comments from a quick pass through. Some of these are just style issues.

Some of the options that are to be put in weewx.conf should more properly be in a skin. Weewx is for systems-like configuration, skin.conf for localization configuration. An example is the Zambretti labels. 

It's bad style to set an attribute on a systems module (viz: setattr(json, 'dumps', json.encode)). You can just set a global variable and use that instead. Besides, I don't see anywhere this is used. Or, is this some cool Python trick I don't know about?

Rather than

        dd = config_dict['Forecast'][fid] \

            if fid in config_dict['Forecast'].keys() else {}

why not just use

  dd = config_dict['Forecast'].get(fid, {})

In prune_forecasts() you forgot to release the cursor. Use a "finally" clause in the try block. Better yet, use the helper member function getSql() in Archive and let it do it.

There should never be a null timestamp in either an archive record, nor in a loop packet. It's considered a precondition. No need to test for it.

Is the Zambretti forecast for real? :-) Man, that is one brain-dead forecast!

I'm concerned about downloading the NWS forecast in the same thread as the main console loop. This loop should be kept simple and not interrupted for very long. If the hit on the NWS website takes long, it will delay well beyond the couple seconds the loop has. Instead, the update should be done in a separate thread that has its own timer. Every XX seconds it hits the server and updates the database. In fact, this is my major problem with the code.

Same with the WU forecast.

Rather than

  if thing in mydict.keys():

the cool way is to simply say

  if thing in mydict:

Module urllib is never used.

Neither is variable nidx3.

Nor variable tstr (in CreateWUForecastMatrix()).

In function dirstr(), the local, and unchanging, variable directions should be outside of the function. Otherwise it keeps getting recreated. It doesn't look like this function does anything a simple dictionary lookup using get() wouldn't do.

Class ForecastData should inherit from object.

Line 1161 is as far as I got before getting called to dinner! :-)

Again, great progress. This is a commonly asked for feature. If we (you!) setup a proper framework for forecasts, it will be easy for users living overseas to add their own local forecasts. The one barrier to this is that you've had to hardwire in the three forecasts you've provided into the search list. Adding new forecasts means adding on to an ever expanding list. This isn't your fault: the search lists really need to be more modular. Your use case is a good example of the limitations of the present architecture.

Oh, one other thing: Class Archive may change a bit. I don't see the simple features you are using (getSql()) changing, but the getSqlVectors() stuff probably will. I can't imagine why you'd be using that, but I thought I'd warn you!

-tk

--

Tom Keffer

kef...@threefools.org
+1 541-386-8891 (h)
+1 541-490-9507 (c)

Skype: tkeffer

[Lucas Heijst]

Looking great, Matthew!

Is this exfoliation skin available for download?

Cheers, Luc

[mwall]

On Monday, August 26, 2013 2:00:06 PM UTC-4, Lucas Heijst wrote:

Looking great, Matthew!

Is this exfoliation skin available for download?

Cheers, Luc

soon.

i now have the daily aggregates working (i have to do that in the forecast module since the forecast periods do not do that automatically).  and i nearly have the precipitation and obstructions to visibility sorted out.

today's nws forecast had some unexpected characters and some bad formatting, so i am reworking the parser so that it is more robust.  i have incorporated tom's suggestions, but i still need to separate the forecasts to a separate thread.

m

[mwall]

On Sunday, August 25, 2013 9:37:34 PM UTC-4, Thomas Keffer wrote:

I'm concerned about downloading the NWS forecast in the same thread as the main console loop. This loop should be kept simple and not interrupted for very long. If the hit on the NWS website takes long, it will delay well beyond the couple seconds the loop has. Instead, the update should be done in a separate thread that has its own timer. Every XX seconds it hits the server and updates the database. In fact, this is my major problem with the code.

ok.  i will spawn a thread for wu and i will spawn a thread for nws.  xtide and zambretti will run in the main thread.

Again, great progress. This is a commonly asked for feature. If we (you!) setup a proper framework for forecasts, it will be easy for users living overseas to add their own local forecasts. The one barrier to this is that you've had to hardwire in the three forecasts you've provided into the search list. Adding new forecasts means adding on to an ever expanding list. This isn't your fault: the search lists really need to be more modular. Your use case is a good example of the limitations of the present architecture.

i have vacillated between having a single schema for all forecasts and one schema per forecast.  afaik, the nws forecast is the most detailed.  the wu forecast can be mapped into the nws structure.  in fact, the schema i am using is basically the nws structure plus a few simplifications.  it looks like the nws system was designed by committee - there are some oddly named bits and pieces here and there, and specs, such as i can infer from the documentation, do not consider all corner cases.

anyway, if we use a single schema for all forecasts, then one would only have to define the download/generation mechanism plus the mapping of downloaded/generated data to the schema.

then there is no need to define additional searchList variables.

this should work for tides as well.  currently i use xtide, but i also experimented with downloading tide (and sun/moon) info from different web sites (noaa in particular.  if you think nws pfm/afm is ugly, try noaa tides!)

i guess i am suggesting that we target a single forecast schema that can incorporate weather forecasts, tide forecasts, wave heights, and probably some almanac information (at least sunrise, sunset, moonrise, moonset, phase of moon).  the reason i suggest some almanac data is that having the almanac data in the same table as the forecast/tide/wave data makes tabular/repetitive displays much easier from a skin point of view.  for displaying future almanac events, we simply add a an almanac forecaster that uses existing almanac methods.

 

Oh, one other thing: Class Archive may change a bit. I don't see the simple features you are using (getSql()) changing, but the getSqlVectors() stuff probably will. I can't imagine why you'd be using that, but I thought I'd warn you!

i did not use getSql for the pruning because the prune only needs to execute some sql, it does not need the fetchone that getSql does after its execute.  i think that in every other case in forecast i used the archive.get/gen methods.

should i add a method 'execSql' to Archive (same as getSql but without the fetchone)?

m

[mwall]

here is a screenshot of the latest (r1308 in the weewx repository). i have also attached the html template that generated the web page shown in the screenshot.

the icon work and sorting out css nuances takes much more time than the coding :)

suggestions are welcome.

[Christian]

Thanks, this looks great. Will try out soon.

[mwall]

this forecast aggregation could be quite powerful.  here is a screenshot showing the tides inline with the weather forecast.  almanac events will be next.

it is also quite easy to put two or three different forecasts side-by-side, for example to compare NWS with WU.  there is not yet support for multiple locations from a single source, e.g., NWS forecast for two different locations.

it should be pretty easy to run an instance of weewx that *only* does forecasting - i might just do this on my laptop as a personalized weather forecasting feed.

btw, these icons are derived from adam whitcroft's climacons.  i will probably release the next exfoliation with two or three icon sets - it is trivial to switch between them using a skin.conf setting.

[Christian]

Hi Matthew,
I have an question concerning the installation of the template. I added the Field [[[forecast]]] with the value template = forecast.html.tmpl to the skin.conf, the file forecast.html becomes generated but says, that forecasting is not installed. It looks like if $varExists('forecast') is false. Do I have to define a set of variables in weewx.conf? The module user.forecast.WUForecast is loaded and writing WU-Responses into DB but they seem not to reach the filegeneration.
Thank You!

[mwall]

christian,

in weewx.conf, be sure to replace FileGenerator with ForecastFileGenerator as described in the comments in forecast.py

m

[mwall]

tides and almanac are now integrated with the forecasts.  here is a screenshot:

http://lancet.mit.edu/mwall/projects/weather/exfoliation-for-weewx-0.13/forecast.png

forecast.py is in the forecast branch of the weewx repository:

http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/forecast.py

exfoliation 0.13 is now available with full support for the forecasting module.  download it from here:

http://lancet.mit.edu/mwall/projects/weather/releases/exfoliation-for-weewx-0.13.tgz


to make it all work, do the following:

1) install the forecasting module

cp forecast.py /home/weewx/bin/user

2) modify weewx.conf as described in the comments in forecast.py

3) install/update to exfoliation 0.13

4) restart weewx

if you have been running older versions of forecast.py, be sure to delete any old forecast.sdb database, as the schema has changed.

[Derek Sedunary]

Well I have tried this a few times now and Weewx has crashed and burned each time.

Below is a a section of the syslog file after restarting weewx:

derek@debian:/home/weewx$ sudo tail -f /var/log/syslog
Aug 31 13:53:01 debian weewx[16077]:     ****    File "/home/weewx/bin/weewx/wxengine.py", line 868, in main
Aug 31 13:53:01 debian weewx[16077]:     ****      config_dict = getConfiguration(config_path)
Aug 31 13:53:01 debian weewx[16077]:     ****    File "/home/weewx/bin/weewx/wxengine.py", line 940, in getConfiguration
Aug 31 13:53:01 debian weewx[16077]:     ****      config_dict = configobj.ConfigObj(config_path, file_error=True)
Aug 31 13:53:01 debian weewx[16077]:     ****    File "/usr/lib/python2.7/dist-packages/configobj.py", line 1230, in __init__
Aug 31 13:53:01 debian weewx[16077]:     ****      self._load(infile, configspec)
Aug 31 13:53:01 debian weewx[16077]:     ****    File "/usr/lib/python2.7/dist-packages/configobj.py", line 1320, in _load
Aug 31 13:53:01 debian weewx[16077]:     ****      raise error
Aug 31 13:53:01 debian weewx[16077]:     ****  ParseError: Invalid line at line "553".
Aug 31 13:53:01 debian weewx[16077]:     ****  Exiting.
^C
To get weewx running again I would go through weewx.conf and comment out all the forecast.py settings I had entered.

None of the errors appear (to my eyes anyway) to relate to the forecast.py settings.

Below is the section I added to weewx.conf

##############################################################################

[Forecast]
    # The database in which to record forecast information, defined in the
    # 'Databases' section of the weewx configuration.

#      database = forecast_sqlite

    # How often to calculate/download the forecast, in seconds

       interval = 300

    # How long to keep old forecasts, in seconds.  use None to keep forever.

       max_age = 604800

    [[Zambretti]]
        # hemisphere can be NORTH or SOUTH
        #hemisphere = NORTH

        # mapping between zambretti codes and descriptive labels
        #labels = "Settled fine", "Fine weather", "Becoming fine", "Fine, becoming less settled", "Fine, possible showers", "Fairly fine, improving", "Fairly fine, possible showers early", "Fairly fine, showery later", "Showery early, improving", "Changeable, mending", "Fairly fine, showers likely", "Rather unsettled clearing later", "Unsettled, probably improving", "Showery, bright intervals", "Showery, becoming less settled", "Changeable, some rain", "Unsettled, short fine intervals", "Unsettled, rain later", "Unsettled, some rain", "Mostly very unsettled", "Occasional rain, worsening", "Rain at times, very unsettled", "Rain at frequent intervals", "Rain, very unsettled", "Stormy, may improve", "Stormy, much rain", "unknown"

    [[NWS]]
        # First figure out your forecast office identifier (foid), then request
        # a point forecast using a url of this form in a web browser:
        #   http://forecast.weather.gov/product.php?site=NWS&product=PFM&format=txt&issuedby=YOUR_THREE_LETTER_FOID
        # Scan the output for a service location identifier corresponding
        # to your location.

        # National Weather Service location identifier

        # id = MAZ014

        # National Weather Service forecast office identifier

        # foid = BOX

        # URL for point forecast matrix
        #url = http://forecast.weather.gov/product.php?site=NWS&product=PFM&format=txt

        # How often to download the forecast, in seconds
        #interval = 10800

    [[WU]]
        # An API key is required to access the weather underground.
        # obtain an API key here:
        #   http://www.wunderground.com/weather/api/

         api_key = "My 16 character key"

        # The location for the forecast can be one of the following:
        #   CA/San_Francisco     - US state/city
        #   60290                - US zip code
        #   Australia/Sydney     - Country/City
        #   37.8,-122.4          - latitude,longitude
        #   KJFK                 - airport code

            pws:INORTHER26       - PWS id

        #   autoip               - AutoIP address location
        #   autoip.json?geo_ip=38.102.136.138 - specific IP address location
        # If no location is specified, station latitude and longitude are used
        #location = 02139

        # How often to download the forecast, in seconds

         interval = 10800

    [[XTide]]
        # Location for which tides are desired

        # location = Boston

        # How often to generate the tide forecast, in seconds
        #interval = 604800

        # How often to prune old tides from database, None to keep forever
        #max_age = 1209600

##############################################################################
[Databases]

    #
    # This section lists possible databases.
    #

    [[archive_sqlite]]
        root = %(WEEWX_ROOT)s
        database = archive/weewx.sdb
        driver = weedb.sqlite

    [[stats_sqlite]]
        root = %(WEEWX_ROOT)s
        database = archive/stats.sdb
        driver = weedb.sqlite

    # MySQL databases require setting an appropriate 'user' and 'password'
    [[archive_mysql]]

        host = localhost
        user = weewx
        password = weewx

        database = weewx
        driver = weedb.mysql

    [[stats_mysql]]

        host = localhost
        user = weewx
        password = weewx

        database = stats
        driver = weedb.mysql

    [[forecast_sqlite]]
        root = %(WEEWX_ROOT)s
        database = archive/forecast.sdb
        driver = weedb.sqlite

    [[forecast_mysql]]
        host = localhost
        user = weewx
        password = weewx
        database = forecast
        driver = weedb.mysql

###############################################################################

[Engines]

    #
    # This section configures the internal weewx engines.
    # It is for advanced customization.
    #

    [[WxEngine]]
        # The list of services the main weewx engine should run:
        service_list = rapidfire.StdRapidFire, weewx.wxengine.StdConvert, weewx.wxengine.StdCalibrate, weewx.wxengine.StdQC, weewx.wxengine.StdArchive, weewx.wxengine.StdTimeSynch, weewx.wxengine.StdPrint, weewx.wxengine.StdRESTful, weewx.wxengine.StdReport, user.forecast.ZambrettiForecast, user.forecast.NWSForecast, user.forecast.WUForecast, user.forecast.XTideForecast, examples.lowBattery.BatteryAlarm

forecast.py is in /home/weewx/bin/user and the permissions are correct.

Can someone see what I missed?

As I have typed this I am wondering if I should have edited this root = %(WEEWX_ROOT)s to the actual path?

Derek...

[Lucas Heijst]

Derek,

The syntax for the WU location in weewx.conf is:

        location = pws:INORTHER26

Matthew is sorting out some errors in the WU forecast. This is what we discovered yesterday.

Change forecast.py to receive forecasts from WU :

Original code:

try:

    import cjson as json

except Exception, e:

    try:

        import simplejson as json

    except Exception, e:

        try:

            import json

        except Exception, e:

            json = None

New code:

try:

    import cjson as json

    # XXX: maintain compatibility w/ json module                                

    setattr(json, 'dumps', json.encode)

    setattr(json, 'loads', json.decode)

except Exception, e:

    try:

        import simplejson as json

    except Exception, e:

        try:

            import json

        except Exception, e:

            json = None

Be sure that the following lines are in your skin.conf for exfoliation:

[Extras]

    icon_dir = icons-aw

    forecast_service = WU

Cheers, Luc

[Derek Sedunary]

Luc
 

The syntax for the WU location in weewx.conf is:

        location = pws:INORTHER26

Oops, I now see the section I edited was an example of what I should have added on the location = line.

New code:

try:

    import cjson as json

    # XXX: maintain compatibility w/ json module                                

    setattr(json, 'dumps', json.encode)

    setattr(json, 'loads', json.decode)

except Exception, e:

    try:

        import simplejson as json

    except Exception, e:

        try:

            import json

        except Exception, e:

            json = None

Oddly this is what I already have in the forecast.py (I was trying this yesterday lunch (Australian) time, maybe it was already updated).

I have not downloaded the latest exfoliation, I wanted to see what would end up in the db first.

I doubt the lack of exfoliation had any thing to do with weewx not restarting, I will try again latter today.

Thanks for your help.

Derek...

[Derek Sedunary]

I have edited weewx.conf to have the correct API key from WU which has helped a bit. I have also followed the instructions for installing exfoliation, and now weewx runs and I still get my normal page for weewx and I also get an exfoliation page (have not changed default maps or units) but the forecast page is blank.

Syslog is showing some problems but I have not determined what they mean yet. A check the skin for the forecast page seems to me to suggest it wants NWS as the forecast engine.

Below is a snap shot from syslog:

Sep  1 13:00:30 debian weewx[4138]: VantagePro: Getting archive packets since 2013-09-01 12:55:00 CST (1378005900)
Sep  1 13:00:33 debian weewx[4138]: VantagePro: successfully woke up console
Sep  1 13:00:35 debian weewx[4138]: VantagePro: Retrieving 1 page(s); starting index= 0
Sep  1 13:00:36 debian weewx[4138]: Archive: added archive record 2013-09-01 13:00:00 CST (1378006200)
Sep  1 13:00:36 debian weewx[4138]: forecast: MainThread: WU: starting thread
Sep  1 13:00:36 debian weewx[4138]: VantagePro: DMPAFT complete: page timestamp 2013-08-23 15:45:00 CST (1377238500) less than final timestamp 2013-09-01 13:00:00 CST (1378006200)
Sep  1 13:00:36 debian weewx[4138]: VantagePro: Catch up complete.
Sep  1 13:00:36 debian weewx[4138]: forecast: WUThread: WU: not yet time to do the forecast
Sep  1 13:00:36 debian weewx[4138]: forecast: WUThread: WU: terminating thread
Sep  1 13:00:36 debian weewx[4138]: reportengine: Running reports for latest time in the database.
Sep  1 13:00:36 debian weewx[4138]: reportengine: Running report StandardReport
Sep  1 13:00:36 debian weewx[4138]: reportengine: Found configuration file /home/weewx/skins/Standard/skin.conf for report StandardReport
Sep  1 13:00:36 debian weewx[4138]: stats: Schema exists with 16 elements
Sep  1 13:00:36 debian weewx[4138]: filegenerator: generated 1 'SummaryByMonth' files in 0.16 seconds
Sep  1 13:00:37 debian weewx[4138]: restful: Published record 2013-09-01 13:00:00 CST (1378006200) to Wunderground station INORTHER26
Sep  1 13:00:37 debian weewx[4138]: filegenerator: generated 1 'SummaryByYear' files in 1.13 seconds
Sep  1 13:00:38 debian weewx[4138]: VantagePro: successfully woke up console
Sep  1 13:00:38 debian weewx[4138]: filegenerator: generated 12 'toDate' files in 1.12 seconds
Sep  1 13:00:39 debian weewx[4138]: VantagePro: Requesting 200 LOOP packets.
Sep  1 13:00:42 debian weewx[4138]: VantagePro: successfully woke up console
Sep  1 13:00:45 debian weewx[4138]: genimages: Generated 22 images in 6.36 seconds
Sep  1 13:00:45 debian weewx[4138]: reportengine: copied 0 files to /home/weewx/public_html
Sep  1 13:00:45 debian weewx[4138]: reportengine: Running report Exfoliation
Sep  1 13:00:45 debian weewx[4138]: reportengine: Found configuration file /home/weewx/skins/exfoliation/skin.conf for report Exfoliation
Sep  1 13:00:45 debian weewx[4138]: stats: Schema exists with 16 elements
Sep  1 13:00:45 debian weewx[4138]: filegenerator: generated 1 'SummaryByMonth' files in 0.17 seconds
Sep  1 13:00:46 debian weewx[4138]: filegenerator: generated 1 'SummaryByYear' files in 1.13 seconds
Sep  1 13:00:48 debian weewx[4138]: reportengine: Caught unrecoverable exception in generator user.forecast.ForecastFileGenerator
Sep  1 13:00:48 debian weewx[4138]:         ****  cannot concatenate 'str' and 'ValueHelper' objects
Sep  1 13:00:48 debian weewx[4138]:         ****  Traceback (most recent call last):
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 130, in run
Sep  1 13:00:48 debian weewx[4138]:         ****      obj.start()
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 293, in start
Sep  1 13:00:48 debian weewx[4138]:         ****      self.run()
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 44, in run
Sep  1 13:00:48 debian weewx[4138]:         ****      self.generateToDate(self.gen_ts)
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 236, in generateToDate
Sep  1 13:00:48 debian weewx[4138]:         ****      print >> _file, text
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__
Sep  1 13:00:48 debian weewx[4138]:         ****      rc = getattr(self, mainMethName)()
Sep  1 13:00:48 debian weewx[4138]:         ****    File "_home_weewx_skins_exfoliation_forecast_html_tmpl.py", line 707, in respond
Sep  1 13:00:48 debian weewx[4138]:         ****  TypeError: cannot concatenate 'str' and 'ValueHelper' objects
Sep  1 13:00:48 debian weewx[4138]:         ****  Generator terminated...
Sep  1 13:00:55 debian weewx[4138]: genimages: Generated 29 images in 6.79 seconds
Sep  1 13:00:55 debian weewx[4138]: reportengine: copied 0 files to /home/weewx/public_html/exfoliation
Sep  1 13:00:55 debian weewx[4138]: reportengine: Running report FTP
Sep  1 13:00:55 debian weewx[4138]: reportengine: Found configuration file /home/weewx/skins/Ftp/skin.conf for report FTP
Sep  1 13:00:55 debian weewx[4138]: reportengine: FTP upload not requested. Skipped.
Sep  1 13:00:55 debian weewx[4138]: reportengine: Running report RSYNC
Sep  1 13:00:55 debian weewx[4138]: reportengine: Found configuration file /home/weewx/skins/Rsync/skin.conf for report RSYNC
Sep  1 13:01:05 debian weewx[4138]: rsyncupload: rsync reported:#012stdin: is not a tty

Derek...

[Derek Sedunary]

in weewx.conf, be sure to replace FileGenerator with ForecastFileGenerator as described in the comments in forecast.py

I am getting an error message that suggest I need to do this but I cannot find it in weewx,conf or in the comments of forecast.py but the error messages below seem to suggest that this is the problem.

What do I do now?

Sep  1 13:00:48 debian weewx[4138]: reportengine: Caught unrecoverable exception in generator user.forecast.ForecastFileGenerator
Sep  1 13:00:48 debian weewx[4138]:         ****  cannot concatenate 'str' and 'ValueHelper' objects
Sep  1 13:00:48 debian weewx[4138]:         ****  Traceback (most recent call last):
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 130, in run
Sep  1 13:00:48 debian weewx[4138]:         ****      obj.start()
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 293, in start
Sep  1 13:00:48 debian weewx[4138]:         ****      self.run()
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 44, in run
Sep  1 13:00:48 debian weewx[4138]:         ****      self.generateToDate(self.gen_ts)
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 236, in generateToDate
Sep  1 13:00:48 debian weewx[4138]:         ****      print >> _file, text
Sep  1 13:00:48 debian weewx[4138]:         ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__
Sep  1 13:00:48 debian weewx[4138]:         ****      rc = getattr(self, mainMethName)()
Sep  1 13:00:48 debian weewx[4138]:         ****    File "_home_weewx_skins_exfoliation_forecast_html_tmpl.py", line 707, in respond
Sep  1 13:00:48 debian weewx[4138]:         ****  TypeError: cannot concatenate 'str' and 'ValueHelper' objects
Sep  1 13:00:48 debian weewx[4138]:         ****  Generator terminated...

Derek...

[mwall]

On Saturday, August 31, 2013 11:59:18 PM UTC-4, Derek Sedunary wrote:

Sep  1 13:00:48 debian weewx[4138]:         ****  TypeError: cannot concatenate 'str' and 'ValueHelper' objects
Sep  1 13:00:48 debian weewx[4138]:         ****  Generator terminated...

this is because the wu structure did not contain a 'duration' field.  i have updated forecast.py to fix this.

exfoliation 0.14 does a few things better:
- display tides column only if the information is available
- integrate forecast data directly into the 'current' weather display, as appropriate
- display today's tides on the 'current' page, if tides are available
- display today's forecast summary on the 'current' page, if forecast is available

apologies for neglecting to test on multiple environments.  i have now tested under python 2.6, 2.7 and fairly extensively with both NWS and WU.

everything *should* work now, but there will probably still be some display hiccups.

[mwall]

here are screenshots of the 'current' and 'forecast' pages in exfoliation for NWS and WU forecasts.  tides are included in each (using xtides).

[mwall]

derek and lucas,

i discovered another possible source of the str + ValueHelper error.  it turns out that the almanac sunrise/sunset are strings when you use the non-pyephem almanac, but they are ValueHelpers when you use the pyephem almanac.

exfoliation release 0.15 works around this inconsistency.

also in 0.15 is explicit enumeration of each .png file.  it would appear that the weewx CopyGenerator does not deal with directories.

exfoliation for weewx is here:

http://lancet.mit.edu/mwall/projects/weather/releases/exfoliation-for-weewx-0.15.tgz

thank you for testing!

m

[Derek Sedunary]

I have installed release 0.15 which appears to have got rid of the "Caught unrecoverable exception in generator user.forecast.ForecastFileGenerator" error.

I now have the following errors in syslog:

: filegenerator: Caught exception "<class 'NameMapper.NotFound'>"
:          ****  Message: "cannot find 'obvis' while searching for 'summary.obvis' in template /home/weewx/skins/exfoliation/index.html.tmpl"
:          ****  Ignoring template and continuing.
: filegenerator: Caught exception "<class 'NameMapper.NotFound'>"
:          ****  Message: "cannot find 'obvis' while searching for 'summary.obvis' in template /home/weewx/skins/exfoliation/forecast.html.tmpl"
:          ****  Ignoring template and continuing.

I other thing I have noticed is the Exfoliation Index Page along with Forecast Page are blank but the summary pages such as Day Summary page are displayed.

Things are getting better. Thank you for your hard work.

Derek...

[mwall]

derek, please ensure that you are using the latest revision of forecast.py:

http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/forecast.py

[Derek Sedunary]

I deleted the current forecast.py and forecast.pyc from weewx/bin/user and download the current version from svn.code.sf.net

I now see:

Sep  2 22:00:48 debian weewx[5499]: filegenerator: Caught exception "<class 'NameMapper.NotFound'>"
Sep  2 22:00:48 debian weewx[5499]:          ****  Message: "cannot find 'obvis' while searching for 'summary.obvis' in template /home/weewx/skins/exfoliation/index.html.tmpl"
Sep  2 22:00:48 debian weewx[5499]:          ****  Ignoring template and continuing.
Sep  2 22:00:48 debian weewx[5499]: filegenerator: Caught exception "<class 'NameMapper.NotFound'>"
Sep  2 22:00:48 debian weewx[5499]:          ****  Message: "cannot find 'obvis' while searching for 'summary.obvis' in template /home/weewx/skins/exfoliation/forecast.html.tmpl"
Sep  2 22:00:48 debian weewx[5499]:          ****  Ignoring template and continuing.

Derek...

[mwall]

On Monday, September 2, 2013 8:38:05 AM UTC-4, Derek Sedunary wrote:

I deleted the current forecast.py and forecast.pyc from weewx/bin/user and download the current version from svn.code.sf.net

I now see:

Sep  2 22:00:48 debian weewx[5499]: filegenerator: Caught exception "<class 'NameMapper.NotFound'>"
Sep  2 22:00:48 debian weewx[5499]:          ****  Message: "cannot find 'obvis' while searching for 'summary.obvis' in template /home/weewx/skins/exfoliation/index.html.tmpl"
Sep  2 22:00:48 debian weewx[5499]:          ****  Ignoring template and continuing.
Sep  2 22:00:48 debian weewx[5499]: filegenerator: Caught exception "<class 'NameMapper.NotFound'>"
Sep  2 22:00:48 debian weewx[5499]:          ****  Message: "cannot find 'obvis' while searching for 'summary.obvis' in template /home/weewx/skins/exfoliation/forecast.html.tmpl"
Sep  2 22:00:48 debian weewx[5499]:          ****  Ignoring template and continuing.

Derek...

sorry to be pedantic, but did you restart weewx?

other than that you could try printing the value of $summary in index.html so we can see what is happening.  in index.html.tmpl, change this:

#if $varExists('forecast')
  #set $summary = $forecast.weather_summary($Extras.forecast_service)
#end if

to this (assuming that you are using WU not NWS for the forecasts):

#if $varExists('forecast')
  #set $summary = $forecast.weather_summary('WU')
  #for $k,$v in $summary.items()
    $k $v
  #end for
#end if

then comment the lines that are causing the errors by changing this:

  #for $k in $summary.obvis
    #set $obvis_img = 'icons/' + $k + '.png'
         <img class='obvis-img' src='$obvis_img' alt='$k' title='$forecast.label('NWS', $k)' /><br/>
  #end for

to this:

##  #for $k in $summary.obvis
##    #set $obvis_img = 'icons/' + $k + '.png'
##         <img class='obvis-img' src='$obvis_img' alt='$k' title='$forecast.label('NWS', $k)' /><br/>
##  #end for

$summary.obvis should default to an empty array.

btw, i really wish cheetah/weewx would provide more details about cheetah failures.  it can be *very* difficult to debug cheetah templates in weewx since there is no line information and minimal feedback about errors.  i find the only way to develop the templates is to start with something that works, then add functionality in very small increments.

m

[Christian]

After deleting the old database and getting the latest versions of the forecast.py and the templates the forecast becomes generated. Thank you very much for your support so far, Matthew.
Temperature and dewpoint are not written in my outputfile, yet. Maybe because i switched all the other units in weewx to Celsius.
Those who are wondering why the images are not displayed: It took me half an hour to remember, that there is an default alias called "icons" on apache... :)

[mwall]

christian,

thanks for the 'icons' note - i guess i will have to rename that directory in the exfoliation skin...

temperature is not yet displayed properly for WU forecasts - there is a bug in the mapping from the WU names and structures to the template variables structure.

the WU forecast does not provide dewpoint information.

fwiw, the NWS forecast is much more detailed - it provides 3-hour and 6-hour windows for up to 7 days.  each period includes sky coverage, probability of precipitation, obstruction to visibility, and many other characteristics.

the WU forecast provides day an night periods for 10 days.  each period includes only basic characteristics such as temperature and probability of precipitation.

the UK MetOffice forecast provides UV index and air quality measures.  UKMO is not yet incorporated into the forecasting module.

btw, i have not yet hooked up the display of quantity of precipitation (snow or rain).

m

[Derek Sedunary]

sorry to be pedantic, but did you restart weewx?

Oops!!

I reloaded weewx assuming (wrongly) that would reload everything. After a restart I now have a forecast page.

Thank you for your patience and help.

Derek... 

[mwall]

the forecasting module now supports 10-day hourly forecasts from weather underground as well as 7-day 3- and 6-hourly forecasts from national weather service.  the latest forecast.py is r1327.

exfoliation 0.16 automatically displays multiple forecasts if data are available.
predicted rain and snowfall amounts are now displayed near the probability of precipitation.

you should delete any existing forecast table/database as the schema has changed slightly.

[Christian]

Thanks again for the latest update, the wu forecast with the forecast.py works perfect for me now.

[Gary Roderick]

Hi Matthew,

Had been watching this thread but was loathed to jump in too early and try forecast.py given the number of custom generators and services I have running under Weewx. In any case I decided to give it a go yesterday. As I am in Aus I only implemented the WU forecasts, no problems getting the database to populate but did run into an issue when I tried to use exfoliation to generate some reports. Symptom I found was that Weewx would 'run' my two other skins but not exfoliation. Debug=1 showed that Weewx 'ran' the first two skins fine but would get to exfoliation and display 'reportengine: Running report Exfoliation' then sit and do nothing until the next archive record came in and the procees would start again. I put a few lines of code in reportengine.py and eventually tracked the issue down to an exception 'Duplicate keyword name at line 178.' in skin.conf. Line 178 for me was

SC = Scattered

I noticed that line 164 was

SC = Scattered Clouds

which I expect was the issue. I commented out line 178 and exfoliation worked - www(dot)therodericks(dot)id(dot)au/exfoliation/

As far as I can tell I followed the install instructions correctly and the [Forecast] section in skin.conf came with exfoliation. I note the [Forecast] section in forecast.py has the same duplicate setting. Seems odd to me that this was not picked up earlier - or have I completely messed up the install/instructions. Rather ironic I thought, that I was having problems with a setting in the [[NWS]] section when I am not in US!

regards,

Gary

PS. Very impressed with exfoliation - great piece of work. Had seen it when you first posted it but had only ever seen the screenshots - they do not do the real thing justice!

[Derek Sedunary]

I have noticed than (for me) the icon directory is not being created in /weewx/public_html/exfoliation so I was not getting and icons in the forecast. I have manually copied icons-wu into /weewx/public_html/exfoliation/ and renamed it icons and now it works other than some sun and moon icons that I copied from icons-aw.

I learn my lesson from before and restarted weewx and when that did not make a difference rebooted the computer which did not help either.

I have possibly missed a setting some where but now it works so I am happy. Here is the forecast for Darwin, Australia.

Again thanks for your excellent work.

Derek...

[mwall]

On Wednesday, September 4, 2013 7:29:15 AM UTC-4, Derek Sedunary wrote:

I have noticed than (for me) the icon directory is not being created in /weewx/public_html/exfoliation so I was not getting and icons in the forecast. I have manually copied icons-wu into /weewx/public_html/exfoliation/ and renamed it icons and now it works other than some sun and moon icons that I copied from icons-aw.

hi derek,

the icons are supposed to be copied once, when weewx first runs the exfoliation skin.  you might check the log file next time you restart weewx to see if the copygenerator is complaining about something.

exfoliation contains two icon sets - icons-aw and icons-wu.  the icons-aw set is pretty much complete, and is derived from adam whitcroft's climacons.  the icons-wu set is a work in progress, based on artwork from weather underground (you might have noticed that wu has many icon sets from which you can choose).

m

[mwall]

gary,

thank you for tracking down the duplicate keyword.

the comments in forecast.py were not clear about this, but you only need to add the labels to your skin.conf if you want to override the defaults in exfoliation, for example for translations or different spelling/wording.

btw, it looks like the icons are not showing up for your forecast.  could you ensure that the 'icons' directory from the exfoliation skin source directory was copied to your public_html space?  did you see any errors in the weewx log from the copygenerator when it first ran the exfoliation skin?

(the source 'icons' directory is simply a copy of the 'icons-aw' directory - this is a hack around the copygenerator limitations in weewx 2.4)

m

[Gary Roderick]

Matthew,

Thanks, did not notice the missing icons when using the desktop but the picture holder boxes are quite apparent on my ipad just now. Will get on to them when my brain stops aching from today's effort!

Gary

[mwall]

folks,

the icons fail was my fault - i forgot to include a generic 'icons' directory in the exfoliation distribution.

in any existing installation simply copy 'icons-aw' from the source exfoliation directory to the www exfoliation directory then rename it to 'icons' (as derek pointed out previously).

exfoliation 0.17 fixes this.

it also removes the body override that resulted in smaller font in forecast tables than elsewhere in exfoliation, as well as a few other minor layout and sizing fixes.

thank you for testing - your time and effort make the code better, and your participation makes this project a pleasure to work on.

m

[Derek Sedunary]

I stopped weewx and deleted all the exfoliation files from /skins, /public_html and /bin/user and downloaded all the latest version to the appropriate directories, rebooted (just to be sure) and it all looks good (other than the wrong radar image). http://www.ninedragons.org/weewx/exfoliation/forecast.html

Thanks again.

Derek...

[mwall]

On Thursday, September 5, 2013 4:32:16 AM UTC-4, Derek Sedunary wrote:

I stopped weewx and deleted all the exfoliation files from /skins, /public_html and /bin/user and downloaded all the latest version to the appropriate directories, rebooted (just to be sure) and it all looks good (other than the wrong radar image). http://www.ninedragons.org/weewx/exfoliation/forecast.html

derek,

thank you for the update.

there are two ways to override the radar image.  either change the exfoliation skin.conf, or make changes in weewx.conf.  i make changes to skin.conf on development/testing sites, but on production sites i change weewx.conf so that i can install exfoliation updates without having to make any modifications.

for example, to change the radar url in skin.conf:

[Extras]
    ...
    radar_local_img = http://radar.weather.gov/ridge/lite/N0R/GYX_loop.gif
    ...

and to change it in weewx.conf:

[StdReport]
    ...
    [[Exfoliation]]           
        skin = exfoliation
        [[[Extras]]]
            radar_local_img = http://radar.weather.gov/ridge/lite/N0R/GYX_loop.gif
            ...

to remove the image completely, comment the line in skin.conf:

[Extras]
    ...
#    radar_local_img = http://radar.weather.gov/ridge/lite/N0R/BOX_loop.gif
    ...

this works for the urls and images in the 'links' page as well, i.e.,

radar_regional_img
radar_national_img
local_forecast_url
marine_forecast_url
tide_table_url
satellite_ir_img
satellite_vs_img
satellite_wv_img
satelite_i8_img
...

[Lucas Heijst]

Matthew,

This is the forecast of Paramaribo, Surinam.

Matthew, great job, thanks!

Perhaps it's time now for a few minor wishes :-)

a. I'd like the index page to have the same link structure as the other pages (links on top and right justified). Now I have to scroll the index page each time to reach the links.

b. The wind direction symbols on the forecast page overlap the numbers sometimes. I guess this has to do with several html-settings and isn't easy to fix in general.

Again, great job!

Cheers, Luc

[Lucas Heijst]

Forecast of Paramaribo Surinam (now as jpg image)

Cheers, Luc

[Derek Sedunary]

I may have phrased that badly - I have not found a suitable radar gif in Australia to replace the sample one. The US seems to be spoilt for choice for access to free maps, in Oz the only ones I can find (that I am allowed to use) are from a paid service.

Derek...

[Lucas Heijst]

Matthew,

The forecast.html page is cleared regularly. Don't know why. I deleted all files in the /home/weewx/public_html/exfoliation folder, restarted weewx and made successive directories. This is what I got.

ds-luc3> dir

drwxr-xr-x    4 root     root          4096 Sep  6 17:10 .

drwxr-xr-x    9 root     root          4096 Aug 26 11:26 ..

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 NOAA

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 RSS

-rw-r--r--    1 root     root         25481 Sep  6 17:16 almanac.html

-rw-r--r--    1 root     root          9842 Sep  6 17:15 day.html

-rw-r--r--    1 root     root             0 Sep  6 17:16 forecast.html

-rw-r--r--    1 root     root          6761 Sep  6 17:15 index.html

-rw-r--r--    1 root     root          5221 Sep  6 17:16 links.html

-rw-r--r--    1 root     root          9857 Sep  6 17:15 month.html

-rw-r--r--    1 root     root          9852 Sep  6 17:15 week.html

-rw-r--r--    1 root     root          9847 Sep  6 17:15 year.html

ds-luc3> dir

drwxr-xr-x    4 root     root          4096 Sep  6 17:18 .

drwxr-xr-x    9 root     root          4096 Aug 26 11:26 ..

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 NOAA

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 RSS

-rw-r--r--    1 root     root         25481 Sep  6 17:16 almanac.html

-rw-r--r--    1 root     root          9842 Sep  6 17:15 day.html

-rw-r--r--    1 root     root          3072 Sep  6 17:17 daybarometer.png

-rw-r--r--    1 root     root          2308 Sep  6 17:17 dayinhum.png

-rw-r--r--    1 root     root          3718 Sep  6 17:17 dayinouthum.png

-rw-r--r--    1 root     root          3960 Sep  6 17:17 dayinouttemp.png

-rw-r--r--    1 root     root          4724 Sep  6 17:17 dayinouttempdew.png

-rw-r--r--    1 root     root          3232 Sep  6 17:17 dayintemp.png

-rw-r--r--    1 root     root          4418 Sep  6 17:17 dayintemphilo.png

-rw-r--r--    1 root     root          2535 Sep  6 17:17 dayouthum.png

-rw-r--r--    1 root     root          2957 Sep  6 17:17 dayouttemp.png

-rw-r--r--    1 root     root          3791 Sep  6 17:17 dayouttempdew.png

-rw-r--r--    1 root     root          4011 Sep  6 17:17 dayouttemphilo.png

-rw-r--r--    1 root     root          1682 Sep  6 17:17 dayradiation.png

-rw-r--r--    1 root     root          1840 Sep  6 17:17 dayrain.png

-rw-r--r--    1 root     root          1744 Sep  6 17:17 dayrx.png

-rw-r--r--    1 root     root          3793 Sep  6 17:17 daytempchill.png

-rw-r--r--    1 root     root          5374 Sep  6 17:17 daywind.png

-rw-r--r--    1 root     root          4170 Sep  6 17:17 daywinddir.png

-rw-r--r--    1 root     root          4787 Sep  6 17:17 daywindvec.png

-rw-r--r--    1 root     root        253313 Sep  6 17:17 forecast.html

-rw-r--r--    1 root     root          6761 Sep  6 17:15 index.html

-rw-r--r--    1 root     root          5221 Sep  6 17:16 links.html

-rw-r--r--    1 root     root          9857 Sep  6 17:15 month.html

-rw-r--r--    1 root     root          4801 Sep  6 17:18 monthbarometer.png

-rw-r--r--    1 root     root          6803 Sep  6 17:18 monthinouthum.png

-rw-r--r--    1 root     root          6472 Sep  6 17:18 monthinouttemp.png

-rw-r--r--    1 root     root          7586 Sep  6 17:18 monthinouttempdew.png

-rw-r--r--    1 root     root          2230 Sep  6 17:18 monthradiation.png

-rw-r--r--    1 root     root          2680 Sep  6 17:18 monthrain.png

-rw-r--r--    1 root     root          2298 Sep  6 17:18 monthrx.png

-rw-r--r--    1 root     root          5525 Sep  6 17:18 monthtempchill.png

-rw-r--r--    1 root     root          5255 Sep  6 17:18 monthwind.png

-rw-r--r--    1 root     root          3881 Sep  6 17:18 monthwinddir.png

-rw-r--r--    1 root     root          5651 Sep  6 17:18 monthwindvec.png

-rw-r--r--    1 root     root          9852 Sep  6 17:15 week.html

-rw-r--r--    1 root     root          5765 Sep  6 17:17 weekbarometer.png

-rw-r--r--    1 root     root          7811 Sep  6 17:18 weekinouthum.png

-rw-r--r--    1 root     root          7589 Sep  6 17:18 weekinouttemp.png

-rw-r--r--    1 root     root          9512 Sep  6 17:18 weekinouttempdew.png

-rw-r--r--    1 root     root          2490 Sep  6 17:18 weekradiation.png

-rw-r--r--    1 root     root          3189 Sep  6 17:17 weekrain.png

-rw-r--r--    1 root     root          2560 Sep  6 17:18 weekrx.png

-rw-r--r--    1 root     root          6919 Sep  6 17:17 weektempchill.png

-rw-r--r--    1 root     root          7525 Sep  6 17:17 weekwind.png

-rw-r--r--    1 root     root          5577 Sep  6 17:17 weekwinddir.png

-rw-r--r--    1 root     root          8206 Sep  6 17:17 weekwindvec.png

-rw-r--r--    1 root     root          9847 Sep  6 17:15 year.html

-rw-r--r--    1 root     root          3674 Sep  6 17:18 yearbarometer.png

-rw-r--r--    1 root     root          2656 Sep  6 17:18 yearrain.png

ds-luc3> dir

drwxr-xr-x    5 root     root          4096 Sep  6 17:18 .

drwxr-xr-x    9 root     root          4096 Aug 26 11:26 ..

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 NOAA

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 RSS

-rw-r--r--    1 root     root         25481 Sep  6 17:20 almanac.html

-rw-r--r--    1 root     root          9842 Sep  6 17:20 day.html

-rw-r--r--    1 root     root          3072 Sep  6 17:17 daybarometer.png

-rw-r--r--    1 root     root          2308 Sep  6 17:17 dayinhum.png

-rw-r--r--    1 root     root          3718 Sep  6 17:17 dayinouthum.png

-rw-r--r--    1 root     root          3960 Sep  6 17:17 dayinouttemp.png

-rw-r--r--    1 root     root          4724 Sep  6 17:17 dayinouttempdew.png

-rw-r--r--    1 root     root          3232 Sep  6 17:17 dayintemp.png

-rw-r--r--    1 root     root          4418 Sep  6 17:17 dayintemphilo.png

-rw-r--r--    1 root     root          2535 Sep  6 17:17 dayouthum.png

-rw-r--r--    1 root     root          2957 Sep  6 17:17 dayouttemp.png

-rw-r--r--    1 root     root          3791 Sep  6 17:17 dayouttempdew.png

-rw-r--r--    1 root     root          4011 Sep  6 17:17 dayouttemphilo.png

-rw-r--r--    1 root     root          1682 Sep  6 17:17 dayradiation.png

-rw-r--r--    1 root     root          1840 Sep  6 17:17 dayrain.png

-rw-r--r--    1 root     root          1744 Sep  6 17:17 dayrx.png

-rw-r--r--    1 root     root          3793 Sep  6 17:17 daytempchill.png

-rw-r--r--    1 root     root          5374 Sep  6 17:17 daywind.png

-rw-r--r--    1 root     root          4170 Sep  6 17:17 daywinddir.png

-rw-r--r--    1 root     root          4787 Sep  6 17:17 daywindvec.png

-rwxrwxrwx    1 root     root          1799 Sep  6 17:18 exfoliation.css

-rwxrwxrwx    1 root     root          1981 Sep  6 17:18 exfoliation.js

-rwxrwxrwx    1 root     root          1150 Sep  6 17:18 favicon.ico

-rw-r--r--    1 root     root             0 Sep  6 17:20 forecast.html

drwxr-xr-x    2 root     root          4096 Sep  6 17:18 icons

-rw-r--r--    1 root     root          6761 Sep  6 17:20 index.html

-rw-r--r--    1 root     root          5221 Sep  6 17:20 links.html

-rw-r--r--    1 root     root          9857 Sep  6 17:20 month.html

-rw-r--r--    1 root     root          4801 Sep  6 17:18 monthbarometer.png

-rw-r--r--    1 root     root          6803 Sep  6 17:18 monthinouthum.png

-rw-r--r--    1 root     root          6472 Sep  6 17:18 monthinouttemp.png

-rw-r--r--    1 root     root          7586 Sep  6 17:18 monthinouttempdew.png

-rw-r--r--    1 root     root          2230 Sep  6 17:18 monthradiation.png

-rw-r--r--    1 root     root          2680 Sep  6 17:18 monthrain.png

-rw-r--r--    1 root     root          2298 Sep  6 17:18 monthrx.png

-rw-r--r--    1 root     root          5525 Sep  6 17:18 monthtempchill.png

-rw-r--r--    1 root     root          5255 Sep  6 17:18 monthwind.png

-rw-r--r--    1 root     root          3881 Sep  6 17:18 monthwinddir.png

-rw-r--r--    1 root     root          5651 Sep  6 17:18 monthwindvec.png

-rw-r--r--    1 root     root          9852 Sep  6 17:20 week.html

-rw-r--r--    1 root     root          5765 Sep  6 17:17 weekbarometer.png

-rw-r--r--    1 root     root          7811 Sep  6 17:18 weekinouthum.png

-rw-r--r--    1 root     root          7589 Sep  6 17:18 weekinouttemp.png

-rw-r--r--    1 root     root          9512 Sep  6 17:18 weekinouttempdew.png

-rw-r--r--    1 root     root          2490 Sep  6 17:18 weekradiation.png

-rw-r--r--    1 root     root          3189 Sep  6 17:17 weekrain.png

-rw-r--r--    1 root     root          2560 Sep  6 17:18 weekrx.png

-rw-r--r--    1 root     root          6919 Sep  6 17:17 weektempchill.png

-rw-r--r--    1 root     root          7525 Sep  6 17:17 weekwind.png

-rw-r--r--    1 root     root          5577 Sep  6 17:17 weekwinddir.png

-rw-r--r--    1 root     root          8206 Sep  6 17:17 weekwindvec.png

-rw-r--r--    1 root     root          9847 Sep  6 17:20 year.html

-rw-r--r--    1 root     root          3674 Sep  6 17:18 yearbarometer.png

-rw-r--r--    1 root     root          5036 Sep  6 17:18 yearinouthum.png

-rw-r--r--    1 root     root          4228 Sep  6 17:18 yearinouttemp.png

-rw-r--r--    1 root     root          5145 Sep  6 17:18 yearinouttempdew.png

-rw-r--r--    1 root     root          4317 Sep  6 17:18 yearouttemphilo.png

-rw-r--r--    1 root     root          2082 Sep  6 17:18 yearradiation.png

-rw-r--r--    1 root     root          2656 Sep  6 17:18 yearrain.png

-rw-r--r--    1 root     root          2148 Sep  6 17:18 yearrx.png

-rw-r--r--    1 root     root          3646 Sep  6 17:18 yeartempchill.png

-rw-r--r--    1 root     root          4695 Sep  6 17:18 yearwind.png

-rw-r--r--    1 root     root          3753 Sep  6 17:18 yearwinddir.png

-rw-r--r--    1 root     root          6612 Sep  6 17:18 yearwindvec.png

ds-luc3> dir

drwxr-xr-x    5 root     root          4096 Sep  6 17:18 .

drwxr-xr-x    9 root     root          4096 Aug 26 11:26 ..

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 NOAA

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 RSS

-rw-r--r--    1 root     root         25481 Sep  6 17:20 almanac.html

-rw-r--r--    1 root     root          9842 Sep  6 17:20 day.html

-rw-r--r--    1 root     root          3066 Sep  6 17:22 daybarometer.png

-rw-r--r--    1 root     root          2309 Sep  6 17:22 dayinhum.png

-rw-r--r--    1 root     root          3721 Sep  6 17:22 dayinouthum.png

-rw-r--r--    1 root     root          3961 Sep  6 17:22 dayinouttemp.png

-rw-r--r--    1 root     root          4726 Sep  6 17:22 dayinouttempdew.png

-rw-r--r--    1 root     root          3229 Sep  6 17:22 dayintemp.png

-rw-r--r--    1 root     root          4418 Sep  6 17:17 dayintemphilo.png

-rw-r--r--    1 root     root          2535 Sep  6 17:22 dayouthum.png

-rw-r--r--    1 root     root          2954 Sep  6 17:22 dayouttemp.png

-rw-r--r--    1 root     root          3791 Sep  6 17:22 dayouttempdew.png

-rw-r--r--    1 root     root          4011 Sep  6 17:17 dayouttemphilo.png

-rw-r--r--    1 root     root          1679 Sep  6 17:22 dayradiation.png

-rw-r--r--    1 root     root          1835 Sep  6 17:22 dayrain.png

-rw-r--r--    1 root     root          1740 Sep  6 17:22 dayrx.png

-rw-r--r--    1 root     root          3793 Sep  6 17:22 daytempchill.png

-rw-r--r--    1 root     root          5375 Sep  6 17:22 daywind.png

-rw-r--r--    1 root     root          4174 Sep  6 17:22 daywinddir.png

-rw-r--r--    1 root     root          4849 Sep  6 17:22 daywindvec.png

-rwxrwxrwx    1 root     root          1799 Sep  6 17:18 exfoliation.css

-rwxrwxrwx    1 root     root          1981 Sep  6 17:18 exfoliation.js

-rwxrwxrwx    1 root     root          1150 Sep  6 17:18 favicon.ico

-rw-r--r--    1 root     root        253313 Sep  6 17:22 forecast.html

drwxr-xr-x    2 root     root          4096 Sep  6 17:18 icons

-rw-r--r--    1 root     root          6761 Sep  6 17:20 index.html

-rw-r--r--    1 root     root          5221 Sep  6 17:20 links.html

-rw-r--r--    1 root     root          9857 Sep  6 17:20 month.html

-rw-r--r--    1 root     root          4801 Sep  6 17:18 monthbarometer.png

-rw-r--r--    1 root     root          6803 Sep  6 17:18 monthinouthum.png

-rw-r--r--    1 root     root          6472 Sep  6 17:18 monthinouttemp.png

-rw-r--r--    1 root     root          7586 Sep  6 17:18 monthinouttempdew.png

-rw-r--r--    1 root     root          2230 Sep  6 17:18 monthradiation.png

-rw-r--r--    1 root     root          2680 Sep  6 17:18 monthrain.png

-rw-r--r--    1 root     root          2298 Sep  6 17:18 monthrx.png

-rw-r--r--    1 root     root          5525 Sep  6 17:18 monthtempchill.png

-rw-r--r--    1 root     root          5255 Sep  6 17:18 monthwind.png

-rw-r--r--    1 root     root          3881 Sep  6 17:18 monthwinddir.png

-rw-r--r--    1 root     root          5651 Sep  6 17:18 monthwindvec.png

-rw-r--r--    1 root     root          9852 Sep  6 17:20 week.html

-rw-r--r--    1 root     root          5765 Sep  6 17:17 weekbarometer.png

-rw-r--r--    1 root     root          7811 Sep  6 17:18 weekinouthum.png

-rw-r--r--    1 root     root          7589 Sep  6 17:18 weekinouttemp.png

-rw-r--r--    1 root     root          9512 Sep  6 17:18 weekinouttempdew.png

-rw-r--r--    1 root     root          2490 Sep  6 17:18 weekradiation.png

-rw-r--r--    1 root     root          3189 Sep  6 17:17 weekrain.png

-rw-r--r--    1 root     root          2560 Sep  6 17:18 weekrx.png

-rw-r--r--    1 root     root          6919 Sep  6 17:17 weektempchill.png

-rw-r--r--    1 root     root          7525 Sep  6 17:17 weekwind.png

-rw-r--r--    1 root     root          5577 Sep  6 17:17 weekwinddir.png

-rw-r--r--    1 root     root          8206 Sep  6 17:17 weekwindvec.png

-rw-r--r--    1 root     root          9847 Sep  6 17:20 year.html

-rw-r--r--    1 root     root          3674 Sep  6 17:18 yearbarometer.png

-rw-r--r--    1 root     root          5036 Sep  6 17:18 yearinouthum.png

-rw-r--r--    1 root     root          4228 Sep  6 17:18 yearinouttemp.png

-rw-r--r--    1 root     root          5145 Sep  6 17:18 yearinouttempdew.png

-rw-r--r--    1 root     root          4317 Sep  6 17:18 yearouttemphilo.png

-rw-r--r--    1 root     root          2082 Sep  6 17:18 yearradiation.png

-rw-r--r--    1 root     root          2656 Sep  6 17:18 yearrain.png

-rw-r--r--    1 root     root          2148 Sep  6 17:18 yearrx.png

-rw-r--r--    1 root     root          3646 Sep  6 17:18 yeartempchill.png

-rw-r--r--    1 root     root          4695 Sep  6 17:18 yearwind.png

-rw-r--r--    1 root     root          3753 Sep  6 17:18 yearwinddir.png

-rw-r--r--    1 root     root          6612 Sep  6 17:18 yearwindvec.png

ds-luc3> dir

drwxr-xr-x    5 root     root          4096 Sep  6 17:18 .

drwxr-xr-x    9 root     root          4096 Aug 26 11:26 ..

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 NOAA

drwxr-xr-x    2 root     root          4096 Sep  6 17:10 RSS

-rw-r--r--    1 root     root         25482 Sep  6 17:25 almanac.html

-rw-r--r--    1 root     root          9843 Sep  6 17:25 day.html

-rw-r--r--    1 root     root          3066 Sep  6 17:22 daybarometer.png

-rw-r--r--    1 root     root          2309 Sep  6 17:22 dayinhum.png

-rw-r--r--    1 root     root          3721 Sep  6 17:22 dayinouthum.png

-rw-r--r--    1 root     root          3961 Sep  6 17:22 dayinouttemp.png

-rw-r--r--    1 root     root          4726 Sep  6 17:22 dayinouttempdew.png

-rw-r--r--    1 root     root          3229 Sep  6 17:22 dayintemp.png

-rw-r--r--    1 root     root          4418 Sep  6 17:17 dayintemphilo.png

-rw-r--r--    1 root     root          2535 Sep  6 17:22 dayouthum.png

-rw-r--r--    1 root     root          2954 Sep  6 17:22 dayouttemp.png

-rw-r--r--    1 root     root          3791 Sep  6 17:22 dayouttempdew.png

-rw-r--r--    1 root     root          4011 Sep  6 17:17 dayouttemphilo.png

-rw-r--r--    1 root     root          1679 Sep  6 17:22 dayradiation.png

-rw-r--r--    1 root     root          1835 Sep  6 17:22 dayrain.png

-rw-r--r--    1 root     root          1740 Sep  6 17:22 dayrx.png

-rw-r--r--    1 root     root          3793 Sep  6 17:22 daytempchill.png

-rw-r--r--    1 root     root          5375 Sep  6 17:22 daywind.png

-rw-r--r--    1 root     root          4174 Sep  6 17:22 daywinddir.png

-rw-r--r--    1 root     root          4849 Sep  6 17:22 daywindvec.png

-rwxrwxrwx    1 root     root          1799 Sep  6 17:18 exfoliation.css

-rwxrwxrwx    1 root     root          1981 Sep  6 17:18 exfoliation.js

-rwxrwxrwx    1 root     root          1150 Sep  6 17:18 favicon.ico

-rw-r--r--    1 root     root             0 Sep  6 17:25 forecast.html

drwxr-xr-x    2 root     root          4096 Sep  6 17:18 icons

-rw-r--r--    1 root     root          6762 Sep  6 17:25 index.html

-rw-r--r--    1 root     root          5222 Sep  6 17:25 links.html

-rw-r--r--    1 root     root          9858 Sep  6 17:25 month.html

-rw-r--r--    1 root     root          4801 Sep  6 17:18 monthbarometer.png

-rw-r--r--    1 root     root          6803 Sep  6 17:18 monthinouthum.png

-rw-r--r--    1 root     root          6472 Sep  6 17:18 monthinouttemp.png

-rw-r--r--    1 root     root          7586 Sep  6 17:18 monthinouttempdew.png

-rw-r--r--    1 root     root          2230 Sep  6 17:18 monthradiation.png

-rw-r--r--    1 root     root          2680 Sep  6 17:18 monthrain.png

-rw-r--r--    1 root     root          2298 Sep  6 17:18 monthrx.png

-rw-r--r--    1 root     root          5525 Sep  6 17:18 monthtempchill.png

-rw-r--r--    1 root     root          5255 Sep  6 17:18 monthwind.png

-rw-r--r--    1 root     root          3881 Sep  6 17:18 monthwinddir.png

-rw-r--r--    1 root     root          5651 Sep  6 17:18 monthwindvec.png

-rw-r--r--    1 root     root          9853 Sep  6 17:25 week.html

-rw-r--r--    1 root     root          5765 Sep  6 17:17 weekbarometer.png

-rw-r--r--    1 root     root          7811 Sep  6 17:18 weekinouthum.png

-rw-r--r--    1 root     root          7589 Sep  6 17:18 weekinouttemp.png

-rw-r--r--    1 root     root          9512 Sep  6 17:18 weekinouttempdew.png

-rw-r--r--    1 root     root          2490 Sep  6 17:18 weekradiation.png

-rw-r--r--    1 root     root          3189 Sep  6 17:17 weekrain.png

-rw-r--r--    1 root     root          2560 Sep  6 17:18 weekrx.png

-rw-r--r--    1 root     root          6919 Sep  6 17:17 weektempchill.png

-rw-r--r--    1 root     root          7525 Sep  6 17:17 weekwind.png

-rw-r--r--    1 root     root          5577 Sep  6 17:17 weekwinddir.png

-rw-r--r--    1 root     root          8206 Sep  6 17:17 weekwindvec.png

-rw-r--r--    1 root     root          9848 Sep  6 17:25 year.html

-rw-r--r--    1 root     root          3674 Sep  6 17:18 yearbarometer.png

-rw-r--r--    1 root     root          5036 Sep  6 17:18 yearinouthum.png

-rw-r--r--    1 root     root          4228 Sep  6 17:18 yearinouttemp.png

-rw-r--r--    1 root     root          5145 Sep  6 17:18 yearinouttempdew.png

-rw-r--r--    1 root     root          4317 Sep  6 17:18 yearouttemphilo.png

-rw-r--r--    1 root     root          2082 Sep  6 17:18 yearradiation.png

-rw-r--r--    1 root     root          2656 Sep  6 17:18 yearrain.png

-rw-r--r--    1 root     root          2148 Sep  6 17:18 yearrx.png

-rw-r--r--    1 root     root          3646 Sep  6 17:18 yeartempchill.png

-rw-r--r--    1 root     root          4695 Sep  6 17:18 yearwind.png

-rw-r--r--    1 root     root          3753 Sep  6 17:18 yearwinddir.png

-rw-r--r--    1 root     root          6612 Sep  6 17:18 yearwindvec.png

ds-luc3>

Any clues?

BTW. Besides the WU forecast I have also added the Zambretti forecast. This results in one icon on the index page, which have a link to the forecast page - correct?

Cheers, Luc

[mwall]

On Friday, September 6, 2013 4:42:55 PM UTC-4, Lucas Heijst wrote:

Matthew,

The forecast.html page is cleared regularly. Don't know why. I deleted all files in the /home/weewx/public_html/exfoliation folder, restarted weewx and made successive directories. This is what I got.

Any clues?

are there any message in the weewx log about failures/errors in the forecast.html.tmpl template?
 

BTW. Besides the WU forecast I have also added the Zambretti forecast. This results in one icon on the index page, which have a link to the forecast page - correct?

there should be one icon on the index page, and it should be a link to forecast.html

the icon should come from the WU forecast.

[Lucas Heijst]

Matthew,

The logging show no errors, see below.

Sep  6 16:55:18 weewx[13306]: forecast: MainThread: Zambretti: starting thread
Sep  6 16:55:18 weewx[13306]: forecast: MainThread: WU: starting thread
Sep  6 16:55:18 weewx[21553]: forecast: ZambrettiThread: Zambretti: not yet time to do the forecast
Sep  6 16:55:18 weewx[21553]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  6 16:55:18 weewx[21554]: forecast: WUThread: WU: not yet time to do the forecast
Sep  6 16:55:18 weewx[21554]: forecast: WUThread: WU: terminating thread
 

Sep  6 16:59:21 weewx[5054]: forecast: MainThread: Zambretti: using table 'archive' in database 'forecast_mysql'
Sep  6 16:59:21 weewx[5054]: forecast: MainThread: Zambretti: interval=600 max_age=604800 hemisphere=NORTH lower_pressure=950.0 upper_pressure=1050.0
Sep  6 16:59:21 weewx[5054]: forecast: MainThread: WU: using table 'archive' in database 'forecast_mysql'
Sep  6 16:59:21 weewx[5054]: forecast: MainThread: WU: interval=10800 max_age=604800 api_key=*************** location=pws:IPARAMAR5
 

Sep  6 17:00:27 weewx[5054]: forecast: MainThread: Zambretti: starting thread
Sep  6 17:00:27 weewx[5054]: forecast: MainThread: WU: starting thread
Sep  6 17:00:27 weewx[9414]: forecast: ZambrettiThread: Zambretti: generating zambretti forecast
Sep  6 17:00:27 weewx[9414]: forecast: ZambrettiThread: Zambretti: pressure=1006.5401822 month=8 wind=12 trend=None north=True
Sep  6 17:00:27 weewx[9414]: forecast: ZambrettiThread: Zambretti: code is None
Sep  6 17:00:27 weewx[9414]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  6 17:00:27 weewx[9415]: forecast: WUThread: WU: downloading forecast from 'http://api.wunderground.com/api/***************/hourly10day/q/pws:IPARAMAR5.json'
Sep  6 17:00:30 weewx[9415]: forecast: WUThread: WU: got 240 forecast records
Sep  6 17:00:30 weewx[9415]: forecast: WUThread: WU: using table 'archive' in database 'forecast_mysql'
Sep  6 17:00:31 weewx[9415]: forecast: WUThread: WU: deleted forecasts prior to 1377892827
Sep  6 17:00:31 weewx[9415]: forecast: WUThread: WU: terminating thread
 

Sep  6 17:05:27 weewx[5054]: forecast: MainThread: Zambretti: starting thread
Sep  6 17:05:27 weewx[5054]: forecast: MainThread: WU: starting thread
Sep  6 17:05:27 weewx[29081]: forecast: ZambrettiThread: Zambretti: generating zambretti forecast
Sep  6 17:05:27 weewx[29081]: forecast: ZambrettiThread: Zambretti: pressure=1006.50000984 month=8 wind=0 trend=0.0401723633837 north=True
Sep  6 17:05:27 weewx[29081]: forecast: ZambrettiThread: Zambretti: code is E
Sep  6 17:05:27 weewx[29081]: forecast: ZambrettiThread: Zambretti: generated 1 forecast record
Sep  6 17:05:27 weewx[29082]: forecast: WUThread: WU: not yet time to do the forecast
Sep  6 17:05:27 weewx[29082]: forecast: WUThread: WU: terminating thread
Sep  6 17:05:27 weewx[29081]: forecast: ZambrettiThread: Zambretti: using table 'archive' in database 'forecast_mysql'
Sep  6 17:05:27 weewx[29081]: forecast: ZambrettiThread: Zambretti: deleted forecasts prior to 1377893127
Sep  6 17:05:27 weewx[29081]: forecast: ZambrettiThread: Zambretti: terminating thread
 

Sep  6 17:10:28 weewx[5054]: forecast: MainThread: Zambretti: starting thread
Sep  6 17:10:28 weewx[5054]: forecast: MainThread: WU: starting thread
Sep  6 17:10:28 weewx[16372]: forecast: ZambrettiThread: Zambretti: not yet time to do the forecast
Sep  6 17:10:28 weewx[16372]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  6 17:10:28 weewx[16373]: forecast: WUThread: WU: not yet time to do the forecast
Sep  6 17:10:28 weewx[16373]: forecast: WUThread: WU: terminating thread
 

Sep  6 17:11:26 weewx[20120]: cpustats: started log at 50521
Sep  6 17:11:27 weewx[20120]: forecast: MainThread: Zambretti: using table 'archive' in database 'forecast_mysql'
Sep  6 17:11:27 weewx[20120]: forecast: MainThread: Zambretti: interval=600 max_age=604800 hemisphere=NORTH lower_pressure=950.0 upper_pressure=1050.0
Sep  6 17:11:27 weewx[20120]: forecast: MainThread: WU: using table 'archive' in database 'forecast_mysql'
Sep  6 17:11:27 weewx[20120]: forecast: MainThread: WU: interval=10800 max_age=604800 api_key=*************** location=pws:IPARAMAR5
 

Sep  6 17:15:32 weewx[20120]: forecast: MainThread: Zambretti: starting thread
Sep  6 17:15:32 weewx[20120]: forecast: MainThread: WU: starting thread
Sep  6 17:15:32 weewx[3474]: forecast: ZambrettiThread: Zambretti: generating zambretti forecast
Sep  6 17:15:32 weewx[3474]: forecast: ZambrettiThread: Zambretti: pressure=1006.46292562 month=8 wind=0 trend=None north=True
Sep  6 17:15:32 weewx[3474]: forecast: ZambrettiThread: Zambretti: code is None
Sep  6 17:15:32 weewx[3474]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  6 17:15:32 weewx[3475]: forecast: WUThread: WU: downloading forecast from 'http://api.wunderground.com/api/***************/hourly10day/q/pws:IPARAMAR5.json'
Sep  6 17:15:35 weewx[3475]: forecast: WUThread: WU: got 240 forecast records
Sep  6 17:15:35 weewx[3475]: forecast: WUThread: WU: using table 'archive' in database 'forecast_mysql'
Sep  6 17:15:36 weewx[3475]: forecast: WUThread: WU: deleted forecasts prior to 1377893732
Sep  6 17:15:36 weewx[3475]: forecast: WUThread: WU: terminating thread
 

Sep  6 17:20:33 weewx[20120]: forecast: MainThread: Zambretti: starting thread
Sep  6 17:20:33 weewx[20120]: forecast: MainThread: WU: starting thread
Sep  6 17:20:33 weewx[22878]: forecast: ZambrettiThread: Zambretti: generating zambretti forecast
Sep  6 17:20:33 weewx[22878]: forecast: ZambrettiThread: Zambretti: pressure=1006.44920069 month=8 wind=0 trend=0.0137249350087 north=True
Sep  6 17:20:33 weewx[22878]: forecast: ZambrettiThread: Zambretti: code is E
Sep  6 17:20:33 weewx[22878]: forecast: ZambrettiThread: Zambretti: generated 1 forecast record
Sep  6 17:20:33 weewx[22879]: forecast: WUThread: WU: not yet time to do the forecast
Sep  6 17:20:33 weewx[22879]: forecast: WUThread: WU: terminating thread
Sep  6 17:20:33 weewx[22878]: forecast: ZambrettiThread: Zambretti: using table 'archive' in database 'forecast_mysql'
Sep  6 17:20:33 weewx[22878]: forecast: ZambrettiThread: Zambretti: deleted forecasts prior to 1377894033
Sep  6 17:20:33 weewx[22878]: forecast: ZambrettiThread: Zambretti: terminating thread
 

Sep  6 17:25:18 weewx[20120]: forecast: MainThread: Zambretti: starting thread
Sep  6 17:25:18 weewx[20120]: forecast: MainThread: WU: starting thread
Sep  6 17:25:18 weewx[8868]: forecast: ZambrettiThread: Zambretti: not yet time to do the forecast
Sep  6 17:25:18 weewx[8868]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  6 17:25:18 weewx[8869]: forecast: WUThread: WU: not yet time to do the forecast
Sep  6 17:25:18 weewx[8869]: forecast: WUThread: WU: terminating thread
 

Sep  6 17:30:19 weewx[20120]: forecast: MainThread: Zambretti: starting thread
Sep  6 17:30:19 weewx[20120]: forecast: MainThread: WU: starting thread
Sep  6 17:30:19 weewx[28173]: forecast: ZambrettiThread: Zambretti: not yet time to do the forecast
Sep  6 17:30:19 weewx[28173]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  6 17:30:19 weewx[28174]: forecast: WUThread: WU: not yet time to do the forecast
Sep  6 17:30:19 weewx[28174]: forecast: WUThread: WU: terminating thread

How do I get the results of the Zambretti forecast presented?

My exfoliation skin.conf contains the line: 

    forecast_service = WU

What is the syntax to show both WU and Zambretti forecasts?

Cheers, Luc 

[Lucas Heijst]

Matthew,

Another thing...

Although I have set debug = 1 in weewx.conf I only get ws28xx and forecast logging lines in my /var/log/messages. 

No other logging from weewx such as ftp, creating html pages and so on. Any suggestions how I get these?

Cheers, Luc

[Lucas Heijst]

Matthew,

This is the pattern:

21:40:53 blanc forecast page

21:42:40 OK forecast page

21:46:08 blanc

21:47:39 OK

21:51:08 blanc

Cheers, Luc

[mwall]

On Friday, September 6, 2013 8:15:53 PM UTC-4, Lucas Heijst wrote:

No other logging from weewx such as ftp, creating html pages and so on. Any suggestions how I get these?

Although I have set debug = 1 in weewx.conf I only get ws28xx and forecast logging lines in my /var/log/messages. 

the forecast logging looks fine.  however, the part we need to see is the creating html pages part.

do you see more messages in /var/log/syslog?  have you modified your rsyslog/syslog configuration?

[mwall]

On Friday, September 6, 2013 8:04:01 PM UTC-4, Lucas Heijst wrote:

How do I get the results of the Zambretti forecast presented?

to display the zambretti forecast, you must add some code to a .tmpl file.  for example, this:
                                                  
$forecast.label('Zambretti', $forecast.zambretti.code)                         

results in something like this:

Settled fine                                                                   

here are some other zambretti variables:

$forecast.zambretti.issued_ts                                                  
$forecast.zambretti.code                                                       

i have not yet associated icons with the zambretti codes.

 

My exfoliation skin.conf contains the line: 

    forecast_service = WU

this is how exfoliation knows which source to use when displaying period/summary data.  currently the options are 'WU' or 'NWS'.

[Derek Sedunary]

Matthew

I have noticed that the precipitation column on the forecast page misses the metrication the rest of the skin receives.

I can change the label by editing the  <td class='col-pop'><img src='icons/pop.png' class='legend-img' alt='PoP' title='probability of precipitation' /><br/><span class='units'>in</span></td>  to <span class='units'>$unit.label.rain</span> which then shows "mm" as a header but I cannot figure out how to make 0.01" show as 0.25mm.

I cannot see anything the forecast template or the skin config that allows me to set the precip units to mm.

Derek...

[mwall]

On Saturday, September 7, 2013 3:16:40 AM UTC-4, Derek Sedunary wrote:

I have noticed that the precipitation column on the forecast page misses the metrication the rest of the skin receives.

hi derek,

sorry about this - the rainfall/snowfall amounts are still a work in progress.

the NWS forecast provides either a single value, a range, or 'T'.  the WU forecast provides a single value.  currently (07sep13) the forecast module just returns a string, so no units conversion happens.  this will be fixed as soon as i decide how to handle ranges.

m

[Lucas Heijst]

Matthew,

Found a file /etc/syslog.deny and uncommented the debug line. after that I restarted my Synology NAS.

# These priorities in this config file are not logged

# refer to syslog.h

#alert

#crit

###debug

#emerg

#err

info

notice

# Always keep these setting , as these are obselete

# refer to syslog.h

error

none

warn

panic

The Weewx logging looks now like this (ws28xx logging removed):

Sep  7 08:15:26 weewx[21942]: Archive: added archive record 2013-09-07 08:15:00 BRT (1378552500)
Sep  7 08:15:26 weewx[21942]: forecast: MainThread: Zambretti: starting thread
Sep  7 08:15:26 weewx[21942]: forecast: MainThread: WU: starting thread
Sep  7 08:15:26 weewx[7535]: forecast: ZambrettiThread: Zambretti: not yet time to do the forecast
Sep  7 08:15:26 weewx[7535]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  7 08:15:26 weewx[7536]: forecast: WUThread: WU: not yet time to do the forecast
Sep  7 08:15:26 weewx[7536]: forecast: WUThread: WU: terminating thread
Sep  7 08:15:26 weewx[7537]: reportengine: Running reports for latest time in the database.
Sep  7 08:15:26 weewx[7537]: reportengine: Running report StandardReport
Sep  7 08:15:26 weewx[7537]: reportengine: Found configuration file /home/weewx/skins/Standard/skin.conf for report StandardReport
Sep  7 08:15:26 weewx[7537]: stats: Schema exists with 16 elements
Sep  7 08:15:27 weewx[21946]: restful: Published record 2013-09-07 08:15:00 BRT (1378552500) to Wunderground station IPARAMAR5
Sep  7 08:15:27 weewx[7537]: filegenerator: generated 1 'SummaryByMonth' files in 0.70 seconds
Sep  7 08:15:27 weewx[21946]: restful: Published record 2013-09-07 08:15:00 BRT (1378552500) to PWSweather station SRPBMUITVL

Sep  7 08:15:30 weewx[7537]: filegenerator: generated 1 'SummaryByYear' files in 3.38 seconds
Sep  7 08:15:33 weewx[7537]: filegenerator: generated 12 'toDate' files in 2.80 seconds
Sep  7 08:15:35 weewx[7537]: genimages: Generated 11 images in 1.58 seconds
Sep  7 08:15:35 weewx[7537]: reportengine: copied 0 files to /home/weewx/public_html
Sep  7 08:15:35 weewx[7537]: reportengine: Running report Exfoliation
Sep  7 08:15:35 weewx[7537]: reportengine: Found configuration file /home/weewx/skins/exfoliation/skin.conf for report Exfoliation
Sep  7 08:15:35 weewx[7537]: stats: Schema exists with 16 elements
Sep  7 08:15:35 weewx[7537]: filegenerator: generated 1 'SummaryByMonth' files in 0.66 seconds
Sep  7 08:15:39 weewx[7537]: filegenerator: generated 1 'SummaryByYear' files in 3.34 seconds
Sep  7 08:17:35 weewx[7537]: genimages: Generated 16 images in 2.47 seconds

NOTE: around this time I see a good forecast.html file
Sep  7 08:17:35 weewx[7537]: reportengine: copied 0 files to /home/weewx/public_html/exfoliation
Sep  7 08:17:35 weewx[7537]: reportengine: Running report Amphibian
Sep  7 08:17:35 weewx[7537]: reportengine: Found configuration file /home/weewx/skins/amphibian/skin.conf for report Amphibian
Sep  7 08:17:35 weewx[7537]: stats: Schema exists with 16 elements

Sep  7 08:17:41 weewx[7537]: filegenerator: generated 6 'toDate' files in 2.70 seconds
Sep  7 08:17:43 weewx[7537]: genimages: Generated 9 images in 1.89 seconds
Sep  7 08:17:43 weewx[7537]: reportengine: copied 0 files to /home/weewx/public_html/amphibian
Sep  7 08:17:43 weewx[7537]: reportengine: Running report FTP
Sep  7 08:17:43 weewx[7537]: reportengine: Found configuration file /home/weewx/skins/Ftp/skin.conf for report FTP
Sep  7 08:17:43 weewx[7537]: reportengine: FTP upload not requested. Skipped.
Sep  7 08:17:43 weewx[7537]: reportengine: Running report RSYNC
Sep  7 08:17:43 weewx[7537]: reportengine: Found configuration file /home/weewx/skins/Rsync/skin.conf for report RSYNC
Sep  7 08:17:43 weewx[7537]: reportengine: rsync upload not requested. Skipped.
 

Sep  7 08:20:57 weewx[21942]: forecast: MainThread: Zambretti: starting thread
Sep  7 08:20:57 weewx[28725]: forecast: ZambrettiThread: Zambretti: generating zambretti forecast
Sep  7 08:20:57 weewx[21942]: forecast: MainThread: WU: starting thread
Sep  7 08:20:57 weewx[28725]: forecast: ZambrettiThread: Zambretti: pressure=957.184234728 month=8 wind=3 trend=-2.45847095876 north=True
Sep  7 08:20:57 weewx[28725]: forecast: ZambrettiThread: Zambretti: code is Z
Sep  7 08:20:57 weewx[28725]: forecast: ZambrettiThread: Zambretti: generated 1 forecast record
Sep  7 08:20:57 weewx[28727]: forecast: WUThread: WU: not yet time to do the forecast
Sep  7 08:20:57 weewx[28727]: forecast: WUThread: WU: terminating thread
Sep  7 08:20:57 weewx[28728]: reportengine: Running reports for latest time in the database.
Sep  7 08:20:57 weewx[28728]: reportengine: Running report StandardReport
Sep  7 08:20:57 weewx[28725]: forecast: ZambrettiThread: Zambretti: using table 'archive' in database 'forecast_mysql'
Sep  7 08:20:57 weewx[28725]: Archive: added archive record 2013-09-07 08:20:00 BRT (1378552800)
Sep  7 08:20:57 weewx[28725]: forecast: ZambrettiThread: Zambretti: deleted forecasts prior to 1377948057
Sep  7 08:20:57 weewx[28725]: forecast: ZambrettiThread: Zambretti: terminating thread
Sep  7 08:20:57 weewx[28728]: reportengine: Found configuration file /home/weewx/skins/Standard/skin.conf for report StandardReport
Sep  7 08:20:57 weewx[28728]: stats: Schema exists with 16 elements
Sep  7 08:20:57 weewx[21946]: restful: Published record 2013-09-07 08:20:00 BRT (1378552800) to Wunderground station IPARAMAR5
Sep  7 08:20:57 weewx[28728]: filegenerator: generated 1 'SummaryByMonth' files in 0.69 seconds
Sep  7 08:20:58 weewx[21946]: restful: Published record 2013-09-07 08:20:00 BRT (1378552800) to PWSweather station SRPBMUITVL
Sep  7 08:21:01 weewx[28728]: filegenerator: generated 1 'SummaryByYear' files in 3.43 seconds
Sep  7 08:21:04 weewx[28728]: filegenerator: generated 12 'toDate' files in 2.82 seconds

NOTE: around this time I see a forecast.html file with length null bytes

Cheers, Luc

[Lucas Heijst]

Matthew,

I showed both HTML and debug windows together and refreshed the HTML screen each second or so.

The good forecast file is generated after the message:

genimages: Generated 16 images in 2.47 seconds

When the null forecast file is created I only see the standard ws28xx logging; no reportengine, filegenerator or genimages have logging at that time.

Weird !!! Perhaps I should enable the info messages as well.

Cheers, Luc

[Derek Sedunary]

O.K.

Thanks for you great work.

Derek...

[Lucas Heijst]

Matthew,

I found something. Each time a null forecast file is created also a new links.html file is created. 

Note: Links.html is created each 5 minutes. Forecast.html is created once each 5 minutes and nulled when a new links.html is created.

Cheers, Luc

[mwall]

forecast.py r1332 includes the following:

- units for snowfall and rainfall
- aggregate values for snowfall and rainfall in summaries
- units for tide heights

http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/forecast.py


exfoliation 0.18 includes the following:

- proper use of units for snowfall and rainfall
- proper use of units for tide heights
- consistent navigation bar across all pages
- improved layout for tides

http://lancet.mit.edu/mwall/projects/weather/releases/exfoliation-for-weewx-0.18.tgz

[Derek Sedunary]

Is there any configuration changes needed.

I have stopped weewx, deleted /skins/exfoliation, /bin/user/forecast.* and for good measure /archive/forecast.sdb , downloaded and install the latest file. I also edited /exfoliation/skin.conf for metric units (this maybe my mistake) then rebooted the computer.

This is the error message from syslog:

Sep  8 10:05:47 debian weewx[2625]: reportengine: Running report Exfoliation
Sep  8 10:05:47 debian weewx[2625]: reportengine: Found configuration file /home/weewx/skins/exfoliation/skin.conf for report Exfoliation
Sep  8 10:05:47 debian weewx[2625]: stats: Schema exists with 16 elements
Sep  8 10:05:47 debian weewx[2625]: filegenerator: generated 1 'SummaryByMonth' files in 0.56 seconds
Sep  8 10:05:49 debian weewx[2625]: filegenerator: generated 1 'SummaryByYear' files in 1.64 seconds
Sep  8 10:05:50 debian weewx[2625]: reportengine: Caught unrecoverable exception in generator user.forecast.ForecastFileGenerator
Sep  8 10:05:50 debian weewx[2625]:         ****  None
Sep  8 10:05:50 debian weewx[2625]:         ****  Traceback (most recent call last):
Sep  8 10:05:50 debian weewx[2625]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 130, in run
Sep  8 10:05:50 debian weewx[2625]:         ****      obj.start()
Sep  8 10:05:50 debian weewx[2625]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 293, in start
Sep  8 10:05:50 debian weewx[2625]:         ****      self.run()
Sep  8 10:05:50 debian weewx[2625]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 44, in run
Sep  8 10:05:50 debian weewx[2625]:         ****      self.generateToDate(self.gen_ts)
Sep  8 10:05:50 debian weewx[2625]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 236, in generateToDate
Sep  8 10:05:50 debian weewx[2625]:         ****      print >> _file, text
Sep  8 10:05:50 debian weewx[2625]:         ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__
Sep  8 10:05:50 debian weewx[2625]:         ****      rc = getattr(self, mainMethName)()
Sep  8 10:05:50 debian weewx[2625]:         ****    File "_home_weewx_skins_exfoliation_index_html_tmpl.py", line 635, in respond
Sep  8 10:05:50 debian weewx[2625]:         ****    File "/home/weewx/bin/user/forecast.py", line 2314, in weather_summary
Sep  8 10:05:50 debian weewx[2625]:         ****      unit_system=rec['usUnits'])
Sep  8 10:05:50 debian weewx[2625]:         ****    File "/home/weewx/bin/user/forecast.py", line 2084, in _create_value
Sep  8 10:05:50 debian weewx[2625]:         ****      units = DEFAULT_UNITS[unit_system][group]
Sep  8 10:05:50 debian weewx[2625]:         ****  KeyError: None
Sep  8 10:05:50 debian weewx[2625]:         ****  Generator terminated...
Sep  8 10:05:52 debian weewx[2625]: genimages: Generated 16 images in 2.12 seconds
Sep  8 10:05:52 debian weewx[2625]: reportengine: copied 59 files to /home/weewx/public_html/exfoliation
Sep  8 10:05:52 debian weewx[2625]: reportengine: Running report FTP
Sep  8 10:05:52 debian weewx[2625]: reportengine: Found configuration file /home/weewx/skins/Ftp/skin.conf for report FTP
Sep  8 10:05:52 debian weewx[2625]: reportengine: FTP upload not requested. Skipped.
Sep  8 10:05:52 debian weewx[2625]: reportengine: Running report RSYNC
Sep  8 10:05:52 debian weewx[2625]: reportengine: Found configuration file /home/weewx/skins/Rsync/skin.conf for report RSYNC
Sep  8 10:06:00 debian weewx[2625]: rsyncupload: rsync reported:#012stdin: is not a tty

It should be noted that in weewx.con (aganst all advise) have set the units to METRIC:
     # DO NOT MODIFY THIS VALUE UNLESS YOU KNOW WHAT YOU ARE DOING!
    target_unit = METRIC    # Choices are 'US' or 'METRIC'

What do you think I have done wrong this time?

Derek..

[Lucas Heijst]

Matthew,

Your link:

http://lancet.mit.edu/mwall/projects/weather/releases/exfoliation-for-weewx-0.18.tgz

points to the *.15.tgz file; I had to change the link manually to download version .18.

The index.html and forecast.html files show texts $summary.qpf.format('%.2f',' ') and $summary.qpf.nolabel('%.2f','')

$summary.qsf.nolabel('%.2f',''). See attachments.

Cheers, Luc

[Derek Sedunary]

I had missed the fault in the link but I had copied the text into sudo wget http://lancet.mit.edu/mwall/projects/weather/releases/exfoliation-for-weewx-0.18.tgz so ended up with the correct file by accident.

Derek...

[mwall]

On Saturday, September 7, 2013 9:25:22 PM UTC-4, Lucas Heijst wrote:

Matthew,

The index.html and forecast.html files show texts $summary.qpf.format('%.2f',' ') and $summary.qpf.nolabel('%.2f','')

$summary.qsf.nolabel('%.2f',''). See attachments.

Cheers, Luc 

hi luc,

did you update forecast.py to the latest version then restart weewx?

qpf is now a ValueHelper with units and formatting.

m

[mwall]

On Saturday, September 7, 2013 9:05:38 PM UTC-4, Derek Sedunary wrote:

Is there any configuration changes needed.

I have stopped weewx, deleted /skins/exfoliation, /bin/user/forecast.* and for good measure /archive/forecast.sdb , downloaded and install the latest file. I also edited /exfoliation/skin.conf for metric units (this maybe my mistake) then rebooted the computer.

what unit system do you see in the forecast database?  if you are using sqlite, do this:

sqlite3 /path/to/forecast.sdb
sqlite3> select * from archive where method='WU' order by dateTime desc limit 5;

i am interested in the second column...

m
 

[Derek Sedunary]

Fairly sure I am using sqlite but the sqlite3 command did not work for me.

I can see the second column is labeled usUnits and every entry has a 1

Derek...

[mwall]

derek,

i cannot duplicate the behavior you are getting, and i cannot figure out how the exception could even happen.  we need more information.

in forecast.py, change this (around line 2304):

        ctxt = 'weather_summary'
        rec['dateTime']    = self._create_value(ctxt, rec['dateTime'],
                                                'group_time')


to this:

        ctxt = 'weather_summary'
        logerr('usUnits: %s' % rec['usUnits'])
        rec['dateTime']    = self._create_value(ctxt, rec['dateTime'],
                                                'group_time')

then restart weewx.  next time forecast.html is generated, see what the log says for usUnits.

m

[Derek Sedunary]

I have added the line logerr('usUnits: %s' % rec['usUnits']) and restarted weewx.

Below is a section of the syslog after restarting:

Sep  8 14:00:51 debian weewx[3368]: reportengine: Running report Exfoliation
Sep  8 14:00:51 debian weewx[3368]: reportengine: Found configuration file /home/weewx/skins/exfoliation/skin.conf for report Exfoliation
Sep  8 14:00:51 debian weewx[3368]: stats: Schema exists with 16 elements
Sep  8 14:00:51 debian weewx[3368]: filegenerator: generated 1 'SummaryByMonth' files in 0.58 seconds
Sep  8 14:00:53 debian weewx[3368]: filegenerator: generated 1 'SummaryByYear' files in 1.72 seconds
Sep  8 14:00:55 debian weewx[3368]: forecast: ReportThread: usUnits: None
Sep  8 14:00:55 debian weewx[3368]: reportengine: Caught unrecoverable exception in generator user.forecast.ForecastFileGenerator
Sep  8 14:00:55 debian weewx[3368]:         ****  None
Sep  8 14:00:55 debian weewx[3368]:         ****  Traceback (most recent call last):
Sep  8 14:00:55 debian weewx[3368]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 130, in run
Sep  8 14:00:55 debian weewx[3368]:         ****      obj.start()
Sep  8 14:00:55 debian weewx[3368]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 293, in start
Sep  8 14:00:55 debian weewx[3368]:         ****      self.run()
Sep  8 14:00:55 debian weewx[3368]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 44, in run
Sep  8 14:00:55 debian weewx[3368]:         ****      self.generateToDate(self.gen_ts)
Sep  8 14:00:55 debian weewx[3368]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 236, in generateToDate
Sep  8 14:00:55 debian weewx[3368]:         ****      print >> _file, text
Sep  8 14:00:55 debian weewx[3368]:         ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__
Sep  8 14:00:55 debian weewx[3368]:         ****      rc = getattr(self, mainMethName)()
Sep  8 14:00:55 debian weewx[3368]:         ****    File "_home_weewx_skins_exfoliation_index_html_tmpl.py", line 635, in respond
Sep  8 14:00:55 debian weewx[3368]:         ****    File "/home/weewx/bin/user/forecast.py", line 2315, in weather_summary
Sep  8 14:00:55 debian weewx[3368]:         ****      unit_system=rec['usUnits'])
Sep  8 14:00:55 debian weewx[3368]:         ****    File "/home/weewx/bin/user/forecast.py", line 2084, in _create_value
Sep  8 14:00:55 debian weewx[3368]:         ****      units = DEFAULT_UNITS[unit_system][group]
Sep  8 14:00:55 debian weewx[3368]:         ****  KeyError: None
Sep  8 14:00:55 debian weewx[3368]:         ****  Generator terminated...
Sep  8 14:01:03 debian weewx[3368]: genimages: Generated 29 images in 7.90 seconds
Sep  8 14:01:03 debian weewx[3368]: reportengine: copied 59 files to /home/weewx/public_html/exfoliation
Sep  8 14:01:03 debian weewx[3368]: reportengine: Running report FTP
Sep  8 14:01:03 debian weewx[3368]: reportengine: Found configuration file /home/weewx/skins/Ftp/skin.conf for report FTP
Sep  8 14:01:03 debian weewx[3368]: reportengine: FTP upload not requested. Skipped.
Sep  8 14:01:03 debian weewx[3368]: reportengine: Running report RSYNC
Sep  8 14:01:03 debian weewx[3368]: reportengine: Found configuration file /home/weewx/skins/Rsync/skin.conf for report RSYNC
Sep  8 14:01:15 debian weewx[3368]: rsyncupload: rsync reported:#012stdin: is not a tty

Derek...

[Derek Sedunary]

I have installed sqlite3

sqlite> select * from archive where method='WU' order by dateTime limit 5;
WU|1|1378600539|1378600539|1378602000|3600|pws:INORTHER26||10|||84.0|62.0|47.0|E|12.0|||FW|10.0||||||||||||||||0|||||
WU|1|1378600539|1378600539|1378605600|3600|pws:INORTHER26||11|||87.0|62.0|42.0|E|10.0|||FW|10.0||||||||||||||||0|||||
WU|1|1378600539|1378600539|1378609200|3600|pws:INORTHER26||12|||90.0|61.0|38.0|E|9.0|||CL|10.0||||||||||||||||14|||||
WU|1|1378600539|1378600539|1378612800|3600|pws:INORTHER26||13|||89.0|62.0|39.0|E|9.0|||CL|10.0||||||||||||||||14|||||
WU|1|1378600539|1378600539|1378616400|3600|pws:INORTHER26||14|||89.0|62.0|39.0|E|10.0|||CL|10.0||||||||||||||||14|||||
sqlite>

Derek...

[Lucas Heijst]

Derek,
I also have set units to METRIC in weewx.conf. I didn't set it elsewhere.
Cheers, Luc

[Derek Sedunary]

Luc

So you have version 18 working with out errors?

I have deleted all the directories and files associated with exfoliation and the forecast module.

I have downloaded exfoliation and forecast.py again and this time made no changes any of the exfoliation files or forecast.py.

I have checked the configuration for forecast.py in weewx.conf. I made no changes from the previous version of forecast.py and the configeration matches the instrucions in forecast.py (which worked previously). Weewx.conf is set to Metric.

After all that, exfoliation and forecast no longer run. I wish I had a copy of the previous version of forecast.py, I would revert back to exfoliation-V17 and the previous forecast.py.

Derek...

[mwall]

derek,

sorry about that - i forgot to test the case where there was only one forecast source.  there is no problem with your configuration.  the problem was in forecast.py and has been fixed as of r1334.

http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/forecast.py

exfoliation is still at 0.18:

http://lancet.mit.edu/mwall/projects/weather/releases/exfoliation-for-weewx-0.18.tgz

sorry for the disruption,

m

[Derek Sedunary]

Well that made a difference, Exfoliation is back along with the forecast page with no errors showing in syslog.

Thanks for that.

Now I will need to edit /exfoliation/skin.conf to change to metric and all should be good.

Derek...

[Lucas Heijst]

Derek,

No need to change exfoliation/skin.conf. You can put most exfoliation settings in weewx.conf. These settings overrule the settings in skin.conf.

Besides, these settings are saved with new updates of weewx.

Below an example for my station in Surinam (South America).

Note: I have to sort out the lines with ### (i.e. find an alternative for my region)

    [[Exfoliation]]

        skin = exfoliation

        HTML_ROOT = public_html/exfoliation

        [[[Units]]]

            [[[[Groups]]]]

                group_altitude = meter

                group_degree_day = degree_C_day

                group_pressure = hPa

                group_rain = mm

                group_rainrate = mm_per_hour

                group_speed = km_per_hour

                group_speed2 = km_per_hour2

                group_temperature = degree_C

        [[[Extras]]]

            # location of the skin includes - these must be defined

            footer = /home/weewx/skins/exfoliation/footer.inc

            header = /home/weewx/skins/exfoliation/header.inc

            hilo = /home/weewx/skins/exfoliation/hilo.inc

            # which weather forecast should be displayed on the 'current' page?

            forecast_service = WU

            

            # the following are optional.  comment to remove any one from display.

            # radar

            radar_local_img = http://www.weeronline.nl/satelliet/Suriname/255/1

            ###radar_regional_img = 

            radar_national_img = http://www.weeronline.nl/satelliet/Suriname/255/1

            # text forecast

            text_forecast_url = https://www.google.nl/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&sqi=2&ved=0CC0QFjAA&url=http%3A%2F%2Fwww.weeronline.nl%2FZuid-Amerika%2FSuriname%2F255&ei=JMjjUYa3H8zLsgaY6IHoCw&usg=AFQjCNF6j2NB1sr-ZscA_N5LtoLXLhXDuQ&sig2=ednodw7kGuwZE2X6xk8xgQ

            # table of tides

            tide_table_url = http://nl.windfinder.com/tide/paramaribo_zrog_en_hoop

            # infrared

            ###satellite_ir_img = 

            # visible spectrum

            satellite_vs_img = http://www.satelliteviews.net/cgi-bin/w.cgi/?c=ns&DG=PPLC

            # water vapor

            ###satellite_wv_img = 

            # thermal

            ###satellite_i8_img = 

            # forecast images

            forecast_48hour_img = 

            ###forecast_fronts_img = http://www.hpc.ncep.noaa.gov/basicwx/98fwbgus.gif

            ###forecast_surface_img = http://www.aviationweather.gov/adds/data/winds/ruc00hr_sfc_wind.gif

            ####forecast_fourpanel_img = http://weather.unisys.com/gfs/4panel/gfs_pres_4panel1.gif

            # forecast links

            ###nws_48hour_forecast_url = http://forecast.weather.gov/MapClick.php?w0=t&w1=td&w2=hi&w3=sfcwind&w4=sky&w5=pop&w6=rh&w7=thunder&w8=rain&AheadHour=0&Submit=Submit&FcstType=graphical&textField1=42.37580&textField2=-71.11870&site=all

            ###nws_forecast_url = http://forecast.weather.gov/afm/PointClick.php?site=box&zmx=1&zmy=1&map.x=199&map.y=117

            ###nws_marine_forecast_url = 

            weather_underground_url = http://www.wunderground.com/weather-forecast/SR/Paramaribo.html

            intellicast_forecast_url = http://www.intellicast.com/Local/Weather.aspx?location=NSXX0002

            intellicast_satellite_url = http://www.intellicast.com/Global/Satellite/Infrared.aspx?animate=true&location=USMA0046

            # intellicast images

            intellicast_satellite_img = http://images.intellicast.com/WxImages/SatelliteLoop/hiatlsat_None_anim.gif

            ###intellicast_radar_img = http://images.intellicast.com/WxImages/Radar/usa.gif

[Lucas Heijst]

hi luc,

did you update forecast.py to the latest version then restart weewx?

qpf is now a ValueHelper with units and formatting.

m

 

Well, I thought I updated forecast.py, but skipped a stage in the update process. 

BTW It would help if forecast.py has an ID like a version number or date to distinguish different versions. Now I look at the file lengths.

The forecast is looking good, see attachment.

What is the meaning of the minus sign below the rain forecast in mm?

Cheers, Luc

[Lucas Heijst]

Luc wrote:

What is the meaning of the minus sign below the rain forecast in mm?

I moved less than a month ago from Europe to a tropical country in South America and already I forgot what snow is !!!  ;-)

Believe it or not, but on September 2 we've got in Surinsam hailstones during a thunderstorm!

Cheers, Luc

[mwall]

On Sunday, September 8, 2013 10:32:50 AM UTC-4, Lucas Heijst wrote:

Well, I thought I updated forecast.py, but skipped a stage in the update process. 

BTW It would help if forecast.py has an ID like a version number or date to distinguish different versions. Now I look at the file lengths.

the Id tag is already in forecast.py - see the first line of the file.  i suppose i could make the revision number visible in the skin.  but hopefully all the changes will settle down now.

 

The forecast is looking good, see attachment.

What is the meaning of the minus sign below the rain forecast in mm?

that is the predicted snowfall amount.

[mwall]

On Sunday, September 8, 2013 10:54:33 AM UTC-4, mwall wrote:

On Sunday, September 8, 2013 10:32:50 AM UTC-4, Lucas Heijst wrote:

Well, I thought I updated forecast.py, but skipped a stage in the update process. 

BTW It would help if forecast.py has an ID like a version number or date to distinguish different versions. Now I look at the file lengths.

the Id tag is already in forecast.py - see the first line of the file.

i forgot that the Id tag is only replaced when you do a checkout.  in the repository you see simply $Id$, but in a checked out copy you see something like $Id: forecast.py 1334 2013-09-08 10:56:45Z mwall $
 

[Derek Sedunary]

Luc

Thanks for that.

I had a few problems as I did not copy your configuration but copied and edited the section out of exfoliation/skin.conf. This resulted in errors in the syslog but the exfoliation pages were produced (minus changes due to the configuration).

It turns out the extra square brackets around  "Group" are significant. In exfoliation/skin.conf there are only 2 but in your example the are 4 levels, [[[[Groups]]]]

It took a lot of #'s and restarts before I had a close look at the difference between your example and my configuration.

It's all good now.

Thanks

Derek...

[Vasya Pupkin]

Hi i have installed your module but now it not work. Can you help me to make your module workable.

in log i see this 

Sep  9 17:51:07 meteo weewx[2943]: forecast: MainThread: Zambretti: starting thread

Sep  9 17:51:07 meteo weewx[2943]: forecast: MainThread: WU: starting thread

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: generating zambretti forecast

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: pressure=1018.77968 month=8 wind=0 trend=0.0 north=True

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: forecast failure: unsupported operand type(s) for -: 'float' and 'str'

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: terminating thread

....

Sep  9 17:53:14 meteo weewx[2943]:         ****  list index out of range

Sep  9 17:53:14 meteo weewx[2943]:         ****  Traceback (most recent call last):

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 130, in run

Sep  9 17:53:14 meteo weewx[2943]:         ****      obj.start()

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 288, in start

Sep  9 17:53:14 meteo weewx[2943]:         ****      self.run()

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 44, in run

Sep  9 17:53:14 meteo weewx[2943]:         ****      self.generateToDate(self.gen_ts)

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "/home/weewx/bin/weewx/filegenerator.py", line 253, in generateToDate

Sep  9 17:53:14 meteo weewx[2943]:         ****      print >> _file, text

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "/usr/lib/python2.7/dist-packages/Cheetah/Template.py", line 1005, in __str__

Sep  9 17:53:14 meteo weewx[2943]:         ****      rc = getattr(self, mainMethName)()

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "_home_weewx_skins_exfoliation_forecast_html_tmpl.py", line 1046, in respond

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "_home_weewx_skins_exfoliation_forecast_html_tmpl.py", line 427, in __errorCatcher52

Sep  9 17:53:14 meteo weewx[2943]:         ****    File "<string>", line 1, in <module>

Sep  9 17:53:14 meteo weewx[2943]:         ****  IndexError: list index out of range

Sep  9 17:53:14 meteo weewx[2943]:         ****  Generator terminated.

[Vasya Pupkin]

 python test_forecast.py 

E..FF...F.EEE.....EEEEEEE..EE...EE..

======================================================================

ERROR: test_astronomy (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9892, in test_astronomy

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_astronomy/html/index.html'

======================================================================

ERROR: test_nws_template_periods (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9362, in test_nws_template_periods

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_nws_template_periods/html/index.html'

======================================================================

ERROR: test_nws_template_summary (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9402, in test_nws_template_summary

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_nws_template_summary/html/index.html'

======================================================================

ERROR: test_nws_template_table (__main__.ForecastTest)

exercise the period and summary template elements

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9411, in test_nws_template_table

    540)

  File "test_forecast.py", line 8943, in runTemplateLineTest

    self.compareLineCount(tdir + '/html/index.html', expected_count)

  File "test_forecast.py", line 8882, in compareLineCount

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_nws_template_table/html/index.html'

======================================================================

ERROR: test_wu_template_periods_daily (__main__.ForecastTest)

verify the period behavior

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9502, in test_wu_template_periods_daily

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_periods_daily/html/index.html'

======================================================================

ERROR: test_wu_template_periods_hourly (__main__.ForecastTest)

verify the period behavior for hourly

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9632, in test_wu_template_periods_hourly

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_periods_hourly/html/index.html'

======================================================================

ERROR: test_wu_template_summary_daily (__main__.ForecastTest)

verify the summary behavior

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9543, in test_wu_template_summary_daily

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_summary_daily/html/index.html'

======================================================================

ERROR: test_wu_template_summary_daily_metric (__main__.ForecastTest)

verify the summary behavior

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9584, in test_wu_template_summary_daily_metric

    ''', units=weewx.METRIC)

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_summary_daily_metric/html/index.html'

======================================================================

ERROR: test_wu_template_summary_hourly (__main__.ForecastTest)

verify the summary behavior for hourly

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9676, in test_wu_template_summary_hourly

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_summary_hourly/html/index.html'

======================================================================

ERROR: test_wu_template_table_daily (__main__.ForecastTest)

exercise the period and summary template elements

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9596, in test_wu_template_table_daily

    305)

  File "test_forecast.py", line 8943, in runTemplateLineTest

    self.compareLineCount(tdir + '/html/index.html', expected_count)

  File "test_forecast.py", line 8882, in compareLineCount

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_table_daily/html/index.html'

======================================================================

ERROR: test_wu_template_table_hourly (__main__.ForecastTest)

exercise the period and summary template elements for hourly

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9688, in test_wu_template_table_hourly

    515)

  File "test_forecast.py", line 8943, in runTemplateLineTest

    self.compareLineCount(tdir + '/html/index.html', expected_count)

  File "test_forecast.py", line 8882, in compareLineCount

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_table_hourly/html/index.html'

======================================================================

ERROR: test_xtide_templates (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9837, in test_xtide_templates

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_xtide_templates/html/index.html'

======================================================================

ERROR: test_xtide_templates_bad_index (__main__.ForecastTest)

verify behavior when given a bad tide index

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9857, in test_xtide_templates_bad_index

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_xtide_templates_bad_index/html/index.html'

======================================================================

ERROR: test_zambretti_template_errors (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9218, in test_zambretti_template_errors

    ''')

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_zambretti_template_errors/html/index.html'

======================================================================

ERROR: test_zambretti_templates (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9197, in test_zambretti_templates

    ''')

  File "test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_zambretti_templates/html/index.html'

======================================================================

FAIL: test_nws_date_to_ts (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9265, in test_nws_date_to_ts

    self.assertEqual(a, b)

AssertionError: '645 AM EDT WED SEP 4 2013 : 1378262700' != '645 AM EDT WED SEP 4 2013 : 1378291500'

======================================================================

FAIL: test_nws_parse_0 (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9300, in test_nws_parse_0

    self.assertEqual(matrix[label], expected[label])

AssertionError: Lists differ: [1368234000, 1368244800, 13682... != [1368262800, 1368273600, 13682...

First differing element 0:

1368234000

1368262800

Diff is 1590 characters long. Set self.maxDiff to None to see it.

======================================================================

FAIL: test_nws_parse_4 (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "test_forecast.py", line 9328, in test_nws_parse_4

    self.assertEqual(matrix[label], expected[label])

AssertionError: Lists differ: [1378170000, 1378180800, 13781... != [1378198800, 1378209600, 13782...

First differing element 0:

1378170000

1378198800

Diff is 1685 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------

Ran 36 tests in 65.815s

FAILED (failures=3, errors=15)

[mwall]

On Monday, September 9, 2013 9:59:47 AM UTC-4, Vasya Pupkin wrote:

 python test_forecast.py 

E..FF...F.EEE.....EEEEEEE..EE...EE..

run the unit tests like this:

cd /home/weewx
PYTHONPATH=bin python bin/user/test_forecast.py

[Gary Roderick]

Matthew,

Over the course of the day I have had reason to restart Weewx numerous times - but that is another story - what I did notice; however, was that each time I restarted Weewx a WU forecast was downloaded. Does not really present a problem other than filling my logs - don't think I have worn out my WU welcome - but was wondering if this was intended or whether you might be going to add a check on start as to when the last forecast was downloaded/generated.

regards,

Gary

[mwall]

gary,

the check has been added and tested as of r1336 of forecast.py

m

[mwall]

On Monday, September 9, 2013 9:55:15 AM UTC-4, Vasya Pupkin wrote:

Hi i have installed your module but now it not work. Can you help me to make your module workable.

in log i see this 

Sep  9 17:51:07 meteo weewx[2943]: forecast: MainThread: Zambretti: starting thread

Sep  9 17:51:07 meteo weewx[2943]: forecast: MainThread: WU: starting thread

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: generating zambretti forecast

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: pressure=1018.77968 month=8 wind=0 trend=0.0 north=True

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: forecast failure: unsupported operand type(s) for -: 'float' and 'str'

Sep  9 17:51:07 meteo weewx[2943]: forecast: ZambrettiThread: Zambretti: terminating thread

vasya,

please try the latest forecast.py and test_forecast.py (r1336)

i added type-checking to the zambretti forecast that should eliminate the 'unsupported operand' error.

i am not yet sure what is causing the 'IndexError: list index out of range' error when generating the forecast.html page, but it could be a result of the zambretti failure.

m

[Vasya Pupkin]

i updated scripts from  http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/forecast.py and http://svn.code.sf.net/p/weewx/code/branches/forecast/bin/user/test_forecast.py

PYTHONPATH=bin python /home/weewx/bin/user/test_forecast.py

Traceback (most recent call last):

  File "/home/weewx/bin/user/test_forecast.py", line 17, in <module>

    import weedb

ImportError: No module named weedb

errors in  weewx logs are same

понедельник, 9 сентября 2013 г., 19:28:25 UTC+4 пользователь mwall написал:

[mwall]

On Monday, September 9, 2013 12:48:26 PM UTC-4, Vasya Pupkin wrote:

PYTHONPATH=bin python /home/weewx/bin/user/test_forecast.py

Traceback (most recent call last):

  File "/home/weewx/bin/user/test_forecast.py", line 17, in <module>

    import weedb

ImportError: No module named weedb

python (like java) is very picky about paths.  to run the unit tests, do this:

[Vasya Pupkin]

 PYTHONPATH=bin python bin/user/test_forecast.py

E..FF...F.EEE.....EEEEEEE..EE...EE..

======================================================================

ERROR: test_astronomy (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9892, in test_astronomy

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_astronomy/html/index.html'

======================================================================

ERROR: test_nws_template_periods (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9362, in test_nws_template_periods

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_nws_template_periods/html/index.html'

======================================================================

ERROR: test_nws_template_summary (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9402, in test_nws_template_summary

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_nws_template_summary/html/index.html'

======================================================================

ERROR: test_nws_template_table (__main__.ForecastTest)

exercise the period and summary template elements

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9411, in test_nws_template_table

    540)

  File "bin/user/test_forecast.py", line 8943, in runTemplateLineTest

    self.compareLineCount(tdir + '/html/index.html', expected_count)

  File "bin/user/test_forecast.py", line 8882, in compareLineCount

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_nws_template_table/html/index.html'

======================================================================

ERROR: test_wu_template_periods_daily (__main__.ForecastTest)

verify the period behavior

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9502, in test_wu_template_periods_daily

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_periods_daily/html/index.html'

======================================================================

ERROR: test_wu_template_periods_hourly (__main__.ForecastTest)

verify the period behavior for hourly

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9632, in test_wu_template_periods_hourly

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_periods_hourly/html/index.html'

======================================================================

ERROR: test_wu_template_summary_daily (__main__.ForecastTest)

verify the summary behavior

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9543, in test_wu_template_summary_daily

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_summary_daily/html/index.html'

======================================================================

ERROR: test_wu_template_summary_daily_metric (__main__.ForecastTest)

verify the summary behavior

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9584, in test_wu_template_summary_daily_metric

    ''', units=weewx.METRIC)

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_summary_daily_metric/html/index.html'

======================================================================

ERROR: test_wu_template_summary_hourly (__main__.ForecastTest)

verify the summary behavior for hourly

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9676, in test_wu_template_summary_hourly

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_summary_hourly/html/index.html'

======================================================================

ERROR: test_wu_template_table_daily (__main__.ForecastTest)

exercise the period and summary template elements

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9596, in test_wu_template_table_daily

    305)

  File "bin/user/test_forecast.py", line 8943, in runTemplateLineTest

    self.compareLineCount(tdir + '/html/index.html', expected_count)

  File "bin/user/test_forecast.py", line 8882, in compareLineCount

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_table_daily/html/index.html'

======================================================================

ERROR: test_wu_template_table_hourly (__main__.ForecastTest)

exercise the period and summary template elements for hourly

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9688, in test_wu_template_table_hourly

    515)

  File "bin/user/test_forecast.py", line 8943, in runTemplateLineTest

    self.compareLineCount(tdir + '/html/index.html', expected_count)

  File "bin/user/test_forecast.py", line 8882, in compareLineCount

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_wu_template_table_hourly/html/index.html'

======================================================================

ERROR: test_xtide_templates (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9837, in test_xtide_templates

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_xtide_templates/html/index.html'

======================================================================

ERROR: test_xtide_templates_bad_index (__main__.ForecastTest)

verify behavior when given a bad tide index

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9857, in test_xtide_templates_bad_index

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_xtide_templates_bad_index/html/index.html'

======================================================================

ERROR: test_zambretti_template_errors (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9218, in test_zambretti_template_errors

    ''')

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_zambretti_template_errors/html/index.html'

======================================================================

ERROR: test_zambretti_templates (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9197, in test_zambretti_templates

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_zambretti_templates/html/index.html'

======================================================================

FAIL: test_nws_date_to_ts (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9265, in test_nws_date_to_ts

    self.assertEqual(a, b)

AssertionError: '645 AM EDT WED SEP 4 2013 : 1378262700' != '645 AM EDT WED SEP 4 2013 : 1378291500'

======================================================================

FAIL: test_nws_parse_0 (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9300, in test_nws_parse_0

    self.assertEqual(matrix[label], expected[label])

AssertionError: Lists differ: [1368234000, 1368244800, 13682... != [1368262800, 1368273600, 13682...

First differing element 0:

1368234000

1368262800

Diff is 1590 characters long. Set self.maxDiff to None to see it.

======================================================================

FAIL: test_nws_parse_4 (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9328, in test_nws_parse_4

    self.assertEqual(matrix[label], expected[label])

AssertionError: Lists differ: [1378170000, 1378180800, 13781... != [1378198800, 1378209600, 13782...

First differing element 0:

1378170000

1378198800

Diff is 1685 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------

Ran 36 tests in 66.564s

FAILED (failures=3, errors=15)

понедельник, 9 сентября 2013 г., 20:58:33 UTC+4 пользователь mwall написал:

[mwall]

On Monday, September 9, 2013 1:29:17 PM UTC-4, Vasya Pupkin wrote:

 PYTHONPATH=bin python bin/user/test_forecast.py

E..FF...F.EEE.....EEEEEEE..EE...EE..

it looks like you have template processing/generation problems and possibly date/time/locale problems.

to isolate the problem, monitor the log file while you run a single unit test.

in one shell:

tail -f /var/log/syslog

in another shell:

cd /home/weewx
PYTHONPATH=bin python bin/user/test_forecast.py --test test_zambretti_templates

[Vasya Pupkin]

Log in attachment

понедельник, 9 сентября 2013 г., 21:49:16 UTC+4 пользователь mwall написал:

[mwall]

vasya,

there are multiple things going wrong on your system.

- the unit test template files are not being generated because of a 'SummaryByMonth' key error.  that is probably because the unit test configuration file is not being written to disk.

- wu is responding with keys different than i get when testing.  this might be due to language settings.  i need *your* wu output to diagnose the problem.

- it looks like there might be date/time/locale issues, as the nws date/time tests are failing

- it looks like you did not restart weewx after installing the latest forecast.py


please do the following:

1) which version of weewx are you running

2) how much space in in /var/tmp?  does the user running the unit tests have permission to write to /var/tmp?

3) run a single unit test, and post *only* the log file messages that came from that unit test.

cd /home/weewx
PYTHONPATH=bin python bin/user/test_forecast.py --test test_zambretti_templates

4) post the json output from wu for your location.  this is how to do it:

- in test_forecast.py change 'xtest_wu_download_hourly' to 'test_wu_download_hourly'
- in test_forecast.py change 'INSERT_KEY_HERE' to your wu apikey
- run the unit test like this:

PYTHONPATH=bin python bin/user/test_forecast.py --test test_wu_download_hourly

[Vasya Pupkin]

1. i run weewx 2.3.0

2. there is 4G free in var/tmp

3. PYTHONPATH=bin python bin/user/test_forecast.py --test test_zambretti_templates

test_zambretti_templates (__main__.ForecastTest) ... ERROR

======================================================================

ERROR: test_zambretti_templates (__main__.ForecastTest)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "bin/user/test_forecast.py", line 9197, in test_zambretti_templates

    ''')

  File "bin/user/test_forecast.py", line 8937, in runTemplateTest

    self.compareContents(tdir + '/html/index.html', expected_lines)

  File "bin/user/test_forecast.py", line 8894, in compareContents

    actual = open(filename)

IOError: [Errno 2] No such file or directory: '/var/tmp/weewx_test/test_forecast/test_zambretti_templates/html/index.html'

----------------------------------------------------------------------

Ran 1 test in 3.824s

Log   tail -f /var/log/syslog |grep  forecast > test.log is in   attachment

4. PYTHONPATH=bin python bin/user/test_forecast.py --test test_wu_download_hourly >wu.log

 see wu.log

понедельник, 9 сентября 2013 г., 22:35:47 UTC+4 пользователь mwall написал:

[mwall]

vasya,

thank you for that.  the problem was weewx 2.3.0.  apparently 2.3.x requires stanzas for SummaryByMonth and SummaryByYear, but 2.4.x does not.  this was causing the unit tests to fail.  the other problem was string/float conversions of the zambretti options.

please install the latest forecast.py and test_forecast.py (r1338).  i have updated test_forecast.py so that it will work with 2.3.x, and i have added unit tests to cover the string/float conversion issue.

if you still have a problem, please post your skin.conf and weewx.conf as well as the log file.

when diagnosing problems, it is best to start with the minimal configuration, then add options incrementally, testing after each addition.  this is especially true for weewx skins/templates.

note that the forecasting module will default to usable settings.

this is the minimal weewx.conf configuration required to use WU forecasting:

[Forecast]
    database = forecast_sqlite
    [[WU]]
        api_key = YOUR_WU_API_KEY

[Engines]
    [[WxEngine]]
        service_list = ..., user.forecast.WUForecast

to do zambretti, you only need this:

[Forecast]
    database = forecast_sqlite

[Engines]
    [[WxEngine]]
        service_list = ..., user.forecast.ZambrettiForecast

to do all of the forecasting sources supported so far:

[Forecast]
    database = forecast_sqlite
    [[NWS]]
        lid = NWS_LOCATION_ID          # e.g., MEZ027
        foid = NWS_FORECAST_OFFICE_ID  # e.g., GYX
    [[WU]]
        api_key = YOUR_WU_API_KEY
    [[XTide]]
        location = Tenants Harbor

[Engines]
    [[WxEngine]]
        service_list = ..., user.forecast.NWSForecast, user.forecast.ZambrettiForecast, user.forecast.WUForecast, user.forecast.XTideForecast