Daily climatological summaries

[Powerin]

Hi All,

I have weewx up and running for testing with a Fine Offset station (soon to be updated to Davis). It seems like a good stable system and I would like to move permanently to weewx but I need a daily summary of my weather records with hourly (or more) records similar to the weewx monthly or annual summaries. For example (from my wfrog setup):

                 Temp. C         Humidity %     Pressure hPa       Rain mm         Wind km/h       UV     
                max   min   avg  max  min  avg   max   min   avg   fall   rate    avg    max  dir
2015/03/11 20  24.7  23.4  24.0  25%  23%  24%  1021  1021  1021    0.0    0.0    0.0    2.5    E    -
2015/03/11 19  29.6  25.6  27.7  22%  15%  18%  1020  1020  1020    0.0    0.0    3.7   14.8    S    -
2015/03/11 18  31.1  30.0  30.6  15%  12%  14%  1020  1019  1019    0.0    0.0   10.1   24.5  SSW    -
2015/03/11 17  31.7  31.0  31.3  13%  12%  12%  1020  1019  1019    0.0    0.0   13.0   28.1   SW    -
2015/03/11 16  32.7  31.4  32.0  13%  11%  12%  1020  1020  1020    0.0    0.0   11.2   30.6  WSW    -

Is it possible to get this info from weewx? Has anyone done this? I have read a fair amount of the docs and wiki and haven't found anything yet. Please feel free to point me in the right direction bearing in mind I don't know python (simple BASH scripts are about my limit).

Why do I need daily summaries? I am a farmer. When I spray herbicides or pesticides I am legally required to record the weather conditions at the beginning and end of the spraying run (temp, humidity, wind speed, wind max and direction). It would be great if I could get weewx to provide that info. Thanks for your help.

Peter

[mwall]

On Wednesday, March 11, 2015 at 6:14:02 AM UTC-4, Powerin wrote:

Is it possible to get this info from weewx? Has anyone done this? I have read a fair amount of the docs and wiki and haven't found anything yet. Please feel free to point me in the right direction bearing in mind I don't know python (simple BASH scripts are about my limit).

peter,

here is a working example of the 'amphibian' skin.  is this what you want?

http://sailing.mit.edu/weewx/amphibian/week-table.html

here are details about downloading and installing the skin:

https://github.com/weewx/weewx/wiki

the bits you want are in the files week-table.html.tmpl, month-table.html.tmpl, and year-table.html.tmpl

m

[Andrew Milner]

Matthew

I think what he wants would be a 'day table' with the information every hour within the day in the same way as the week table gives the info for every day within the week.

Andrew

[vince]

On Wednesday, March 11, 2015 at 3:14:02 AM UTC-7, Powerin wrote:

Why do I need daily summaries? I am a farmer. When I spray herbicides or pesticides I am legally required to record the weather conditions at the beginning and end of the spraying run (temp, humidity, wind speed, wind max and direction). 

Wow - what an interesting use of a weather station.

What you're asking for is almost certainly in your weewx data, but I sounds like you want something notionally like the NOAA reports (only hourly, not daily).  Has to be doable.

 

[Powerin]

On Thursday, 12 March 2015 06:09:41 UTC+11, vince wrote:

Wow - what an interesting use of a weather station.

What you're asking for is almost certainly in your weewx data, but I sounds like you want something notionally like the NOAA reports (only hourly, not daily).  Has to be doable.

 

Hopefully it is doable.
Both wview and wfrog do it, but I'd prefer weewx as it seems simpler and easier to customise.

Apart from the legal requirements, a farmer's next paycheck is 90% dependant on the weather so most of us have more than a passing interest in the weather ;-)
Some use a hand held weather station, such as a Kestrel to record conditions in the field. But I'm too lazy to carry around a notebook and record the weather...that's what computers are for :-)

[vince]

See if this helps you any for one (ugly) brute-force bash way to possibly get there...

This runs in just a second or two on a Pi so it's likely something you could run from cron or the like.

#!/bin/bash

# of course pass this in

day="2015-03-11"

for (( num=0; num<=23; num++ ))

do

    # secs since epoch for start/stop of hour 

    t1=`date +%s -d "$day $num:00:00"`

    t2=`date +%s -d "$day $num:59:59"`

    # uncomment to print seconds since epoch for that hour of the day

    #     echo "$day $num start=$t1 stop=$t2"

    # print min/max outTemp during that time period 

    # prettying it up is an exercise for the reader 

 

    echo "select datetime(min(dateTime),'unixepoch','localtime'),min(outTemp),max(outTemp) from archive where dateTime>=$t1 and dateTime<=$t2;" | sqlite3 weewx.sdb

done

Sample ('really' ugly) output from a Pi running weewx in simulator mode, showing hour of the day and min/max outTemp for that time period

2015-03-11 00:00:00|42.0551975560537|46.6453332148331

2015-03-11 01:00:00|47.0762391691158|51.8625806126124

2015-03-11 02:00:00|52.2965302510594|56.9528962197021

2015-03-11 03:00:00|57.360316591589|61.5693834396233

2015-03-11 04:00:00|61.9225095199|65.3974362974444

2015-03-11 05:00:00|65.6722031273085|68.1761793170612

2015-03-11 06:00:00|68.3538619911312|69.7162457537768

2015-03-11 07:00:00|69.7847354914492|69.9992434525375

2015-03-11 08:00:00|68.7521030826112|69.8673119638463

2015-03-11 09:00:00|66.3135987044488|68.5959639581928

2015-03-11 10:00:00|62.7633495340748|66.0573317400718

2015-03-11 11:00:00|58.3432991853859|62.4244188998603

2015-03-11 12:00:00|53.3546667851669|57.9448024439463

2015-03-11 13:00:00|48.1374193873876|52.9237608308843

2015-03-11 14:00:00|43.0471037802979|47.7034697489406

2015-03-11 15:00:00|38.4306165603767|42.6396834084111

2015-03-11 16:00:00|34.6025637025556|38.0774904801

2015-03-11 17:00:00|31.8238206829388|34.3277968726915

2015-03-11 18:00:00|30.2837542462232|31.6461380088689

2015-03-11 19:00:00|30.0007565474626|30.2152645085508

2015-03-11 20:00:00|30.1326880361536|31.2478969173888

2015-03-11 21:00:00|31.4040360418072|33.6864012955512

2015-03-11 22:00:00|33.9426682599282|37.2366504659252

2015-03-11 23:00:00|37.5755811001397|41.6567008146141

 

[Powerin]

Thanks for that vince. That certainly gives me something to work from.

I haven't looked at the structure of weewx yet, I wonder could you or anybody point me to the section of code that creates the monthly summary. This produces a nice table of figures for a selected month on a day by day basis. I wonder could I modifiy this to produce figures for a selected day on an hour by hour basis. If that worked then I would want to be able to select any date in the past that I had records for.

Thanks again for your time and help.

Peter

[mwall]

On Thursday, March 12, 2015 at 10:13:52 PM UTC-4, Powerin wrote:

Thanks for that vince. That certainly gives me something to work from.

I haven't looked at the structure of weewx yet, I wonder could you or anybody point me to the section of code that creates the monthly summary. This produces a nice table of figures for a selected month on a day by day basis. I wonder could I modifiy this to produce figures for a selected day on an hour by hour basis. If that worked then I would want to be able to select any date in the past that I had records for.

peter,

the weewx reporting system is designed for this.  see the 'customizing reports' section of the weewx customization guide:

http://weewx.com/docs/customizing.htm

the easiest thing would be to iterate over the hours using the $day tag.  unfortunately, the $day tag is not yet iterable :(

however, you might be able to use the hours_ago tag instead (see the 'aggregation periods' section of the guide)

for examples, see the source for the amphibian skin (e.g., week-table.html.tmpl), or the NOAA reports in the Standard skin.

m

[vince]

On Thursday, March 12, 2015 at 7:38:58 PM UTC-7, mwall wrote:

the easiest thing would be to iterate over the hours using the $day tag.  unfortunately, the $day tag is not yet iterable :(

however, you might be able to use the hours_ago tag instead (see the 'aggregation periods' section of the guide)

I did take a brief look at hours_ago to try to have cheetah do the heavy lifting, but I thought that it would be tough to retroactively calculate the hourly info for a day in the distant past, especially since even the brute-force method I did as a quickie ran so quickly.   Also looked at the tags.py code to try to extend it with something like hour_of_day(day_here,hour_here) or the like, but ran out of energy mentally.

[Powerin]

On Friday, 13 March 2015 15:26:08 UTC+11, vince wrote:

On Thursday, March 12, 2015 at 7:38:58 PM UTC-7, mwall wrote:

I did take a brief look at hours_ago to try to have cheetah do the heavy lifting, but I thought that it would be tough to retroactively calculate the hourly info for a day in the distant past, especially since even the brute-force method I did as a quickie ran so quickly.   Also looked at the tags.py code to try to extend it with something like hour_of_day(day_here,hour_here) or the like, but ran out of energy mentally.

I'm no coder, I only used write to some html years ago and used php (from other sources) with database queries in pages. I'm only just now coming to grips with the idea of templating engines (I'm an older farmer just on the downhill side of 50 ;) ). I did read through the whole customization doc before posting, and inasmuch as I understood it I couldn't see any ready way to do what I want. But I enjoy trying to puzzle it out.

So my understanding is that even though it is a reasonably simple database query to get the info I need, the problem is that there are no tags within Weewx (yet) that can get that particular query from the database (to put it very simply)? And any past selectable dates need to be converted to Unix epoch obviously? I'll see if I can blunder my way through tags.py and figure out anything. Thanks again.

[Powerin]

And....I'm not in the US and it just occurred to me - I also need to convert the database query into metric :P
...it's rapidly getting too hard for me :)

[mwall]

On Sunday, March 15, 2015 at 8:11:29 AM UTC-4, Powerin wrote:

And....I'm not in the US and it just occurred to me - I also need to convert the database query into metric :P

...it's rapidly getting too hard for me :)

don't worry about the unit conversions - there are utilities within weewx that make this pretty easy, and we can help you sort that once you have the data.

focus first on getting the data you need.  i can imaging a few options, each of which would be educational:

1) query the database independently from weewx.  use a shell script, perl, python, sql browser, or whatever language you are comfortable with.

2) try to use existing weewx tags to get at least some of the data you want.  this would be done in a weewx .tmpl file, using wee_reports and weewx itself to run your code/queries.

3) hack on tags.py to make the $day tag iterable.  follow the pattern for the other tags, but realize that they rely on queries to the daily stats tables, whereas making the $day tag iterate requires queries to the archive table (or perhaps the creation of 'hourly' stats tables).

m

[vince]

On Sunday, March 15, 2015 at 5:03:00 AM UTC-7, Powerin wrote:

So my understanding is that even though it is a reasonably simple database query to get the info I need, the problem is that there are no tags within Weewx (yet) that can get that particular query from the database (to put it very simply)? And any past selectable dates need to be converted to Unix epoch obviously? I'll see if I can blunder my way through tags.py and figure out anything. Thanks again.

Correct, weewx currently doesn't have the hooks to generate the per-hour kind of report you want.

Alsi correct that dates in the weewx.sdb have a dateTime that is seconds since the Unix epoch, but that's pretty easy to get to/from using bash. Date conversions in bash are easy, to go epoch => human-friendly, it's just date -d @1423456789 and to go the other way it's date +%s -d "2015-01-01" or date +%s -d "2015-01-01 01" or even date +%s -d "2015-01-01 01:02:03" depending on how precise you want to get to a particular time of day.

FWIW, if you're comfortable with bash, I'd suggest you start the easy way to get 'something' that is good enough out of the database, then move to the tags.py enhancements under the hood if you want to go that far.

I took 10 minutes to look up the tags to enhance the example I posted previously, and I 'think' I am getting all the data you want out so at this point it would be just a matter of formatting the output to be pretty enough.

#!/bin/bash

# of course pass this in

day="2015-03-11"

database="/home/weewx/archive/weewx.sdb"

for (( num=0; num<=23; num++ ))

do

    # secs since epoch for start/stop of hour

    t1=`date +%s -d "$day $num:00:00"`

    t2=`date +%s -d "$day $num:59:59"`

    # uncomment to print seconds since epoch for that hour of the day

    #     echo "$day $num start=$t1 stop=$t2"

    # print min/max outTemp during that time period

    # prettying it up is an exercise for the reader

    echo "select datetime(min(dateTime),'unixepoch','localtime') \

,max(outTemp),min(outTemp),avg(outTemp) \

,max(outHumidity),min(outHumidity),avg(outHumidity) \

,max(pressure),min(pressure),avg(pressure) \

,total(rain),max(rainRate) \

,avg(windSpeed), max(windGust), avg(windDir) \

    from archive where dateTime>=$t1 and dateTime<=$t2;" | sqlite3 $database

done

I was a little uncertain what you wanted for rain and rainRate, so I guessed a little bit above.  I also guessed 'pressure' rather than 'barometer' for what to grab there.

Can you run the script above and see if the units look right for what you want to do ?

[Powerin]

On Monday, 16 March 2015 05:57:31 UTC+11, vince wrote:

Can you run the script above and see if the units look right for what you want to do ?

vince,

Yes, after changing the database location to suit my setup of course, that script works fine and does what I want, thanks.

Now that I've looked more properly at the templates and what weewx produces from my own data I realise weewx just produces static html pages. I didn't realise what the NOAA mentioned earlier was either. Now I see the monthly NOAA data are simply separate formatted text files for each month (I guess in a NOAA approved format?). These have the data I want....just not on an hourly basis. A bash script will do the job nicely, but for the sake of having the info in one place and accessible out in the field or wherever, a web page will be better.

However, even if I could hack the tags, modify the templates, and get weewx to create a NOAA style daily page, it would mean ending up with 100s and 100s of text pages wouldn't it? So I'm thinking what I really need is a separate dynamic web page which can request a date from the user, run the appropriate database query and then format the results, because the database already has all the info I need. Weewx need not be involved, but it would nice to integrate it for the metric conversion and use some of the tags to label things.

PHP would do the job, but is there a better way? Can you get python/cheetah to create dynamic web pages, or is ajax a more modern solution?

[Andrew Milner]

You could feed the previous day's date in as a parameter to the bash script, and run the script as a cron task just after midnight every day.  You could have another cron script to delete daily files older than say one month.  This would at least reduce the number of files that you have got hanging around.  The somewhat primitive but useable NOAA display selection could be expanded to allow selection of the daily 'NOAA' reports.  A simple but probably good enough for your purposes method - just needing an (optional) additional deletion script - assuming of course that you get your paperwork completed within a month of the recordings ... else you need to delete files older than 2 months or even 3 ...... depending on how good you are with keeping up with the paperwork!!

[vince]

There's no wrong answer for how to approach it.  Do whatever makes you comfortable supporting the end result and gets you there before you run out of time / energy / or available blood pressure.

You could generate a text file per day via cron and use the Standard skin index.html.tmpl for example code that puts the nice buttons to select previous months/years of files from a directory of files (obviously some light editing would be required).  Add a cron job to eventually clean up the zillion files you'll generate if you do dally files.

Or generate your data by 'appending' to a file for that month, then you'll have 1/12 the number of files.

Or do your php solution and display only as-needed based on a query of the database, assuming you do it rarely.

Or write a custom extension to save hourly data to an external database you can query as needed.

Whatever you're comfortable supporting...

[Powerin]

Thanks Andrew and vince. I'll see what I can cobble together. What I want for this particular need is only temp, humidity, wind direction, speed and gust at a specific time in the past, and this could be some time last week or some time last year. But once I get that then there are any number of useful things to do with all that data sitting in the database. Temperature and rainfall trends, evaporation rates, rainfall totals over specific periods are all useful things to know when making spending decisions worth $10s of thousands in agriculture. Sure, there's probably commercial software that does the job, but not on Linux, and I would hate to go back to an inferior OS ;)

I have a spreadsheet with manually entered rainfall records for our farm which go back nearly 100 years and it's interesting doing analysis on that. I also fly so making a smartphone template with aviation formatted weather is on the list too.

[Thomas Keffer]

Can you just use the Weather Wnderground?

Or, is the problem that you have no internet connectivity?

-tk

[Powerin]

OK, bearing in mind I'm a farmer not a coder, I did some research and lot of thinking and concluded that the best way of getting a daily hour by hour summary was to bypass weewx and get the info directly from the sqlite database with a php page. With the help of vince's Bash script I've written an equivalent php script which gets the info I want from the database, converts it to metric and formats it into a html table. It's not pretty, has no error trapping, but it works. The next task is to add a javascript calendar to select the date for the script.

 Anyone want to clean it up for me? Here it is:

<html>
<body>
<table border="0" cellspacing="10">
 <tr>
  <td align="center">Time</td>
  <td align="center">Temp C</td>
  <td align="center">Pressure hPa</td>
  <td align="center">Humidity %</td>
  <td align="center">Average<br>Wind Speed km/h</td>
  <td align="center">Average<br> Wind Gust km/h</td>
  <td align="center">Average<br> Wind Direction deg</td>
  <td align="center">Total rain mm</td>
 </tr>
 
<?php

//Open the weewx database
$db = new SQLite3('/var/lib/weewx/weewx.sdb');

date_default_timezone_set('Australia/NSW');

// find the unix timestamp for a given date
$unixtime = strtotime('2015/03/28');

// set the unix timestamp start and end times for each hour of the day and then advance 1 hour (3600 seconds)
for($t=0;$t<=23;$t++)
{
$starttime = $unixtime + ($t*3600);
$endtime = $unixtime + 3600 + ($t*3600);
//then make the query
$results = $db->query("SELECT datetime(min(dateTime), 'unixepoch', 'localtime'), avg(outTemp), avg(pressure), avg(outHumidity), avg(windSpeed), avg(windGust), avg(windDir), total(rain) FROM archive WHERE dateTime>= $starttime and dateTime<= $endtime;");
//get the row of results and put it in an array
$row = $results->fetchArray();
//convert the US values to metric if necessary and round to one or zero decimal places then print
echo "<tr><td align=\"center\"> $row[0]</td>";
$metTemp = round(($row[1] -32)*5/9, 1);
echo "<td align=\"center\">$metTemp</td>";
$metPress = round($row[2]*33.86389+25.3, 1);
echo "<td align=\"center\">$metPress</td>";
$humid = round($row[3], 1);
echo "<td align=\"center\">$humid</td>";
$metWind = round($row[4]*1.60934, 1);
echo "<td align=\"center\">$metWind</td>";
$metGust = round($row[5]*1.60934, 1);
echo "<td align=\"center\">$metGust</td>";
$windDir = round($row[6], 0);
echo "<td align=\"center\">$windDir</td>"; //wind direction degrees
$metRain = round($row[7]*25.4, 0);
echo "<td align=\"center\">$metRain</td></tr>";
}

?>
</table>
</body>
</html>

[Powerin]

Yeah Thomas, I could. I can get that info off the internet from the nearest Met Station, but it is over 80km (50 miles) away and the weather can be quite different over that distance (especially wind). Our chemical applications records can be audited at any time so we need to be able to show our weather records are reasonably accurate and that we are using pesticides responsibly (and legally). It's all part of our food security and traceability systems here in Australia.

Plus it's kinda fun to customise a good software package that doesn't quite suit my needs. I just didn't want to do it if somebody else already had. ;)

P

[Chris Thompstone]

Hi Peter,
I read your posts with interest.  I'm a farmer in the UK and have to do the same thing. 
I have to fill in a form with the prescription for the field and write in the wind speed, direction, temp, humidity, start finish time etc.

My water bowser (old milk tanker) has wifi in the shed next door to it.
So I use the Pocket PWS app with a widget on my phone home page, set to refresh every 5 mins.

So I fill in the data just before leaving the yard.

This works for me, however if your short of internet connection I can appreciate that this is of no use.

Also if the weather is against me and I have little time to do the writing down of the weather conditions it would be handy to look them up.

[gjr80]

I think what Tom was referring to was if your weather station is reporting to weather underground you can actually look up your observations from any date, in tabular or graphical format, through the weather history graph/weather history table tab. Clearly you need Internet access and must be publishing to weather underground. Handy feature, but agree it's more fun doing it yourself :)

Gary

[Powerin]

Ah, I see. We only have expensive mobile/cell 3G internet access with limited downloads/uploads so I haven't considered anything which involves constant uploads...even if the actual data transfer is small. Mobile phone coverage is also patchy. My life would be so much easier with fast and reliable internet :P

[Powerin]

Hi Chris,

I guess UK and European chemical rules are probably even stricter than ours.

My biggest problem is I'm lazy and don't fill in the paperwork until later :) Also we have patchy mobile phone coverage so can't rely on being able to access the weather on the phone. Weewx has a nice smartphone formatted page so I could easily get conditions from my own weather station.

What I tend to do is enter the prescription into each job on the GPS autosteer in the tractor, then download the jobs onto a memory stick and fill in the paperwork at home at the end of the week (or sometimes month). I go back through the weather records for the start and end times of the jobs. As mentioned wfrog software does this, but it seems to crash unpredictably on my setup so I could never rely on having the records.

The ultimate solution is to stop being lazy with paperwork....but where's the fun in that?  ;)

Peter

[Powerin]

Here's what it looks like so far:

[Erwin Fiten]

This already looks good, can you post that script over here ?

It's also nice for the 'normal' user to find the temperature, etc, at a given time in the past?

Op maandag 30 maart 2015 11:38:02 UTC+2 schreef Powerin:

[Powerin]

I've now got a new station and at the same time have integrated the php daily summary script into a weewx template using the "Flat" skin/theme. It looks good but I'm having one small problem...I know this is not a weewx problem but I hope you'll indulge me.

The total rainfall for each hour shown in my daily summary does not quite add up to the same rainfall for the day as shown by both the Weewx pages and the station console itself. For instance, a couple days ago weewx and the station reported 44.4mm of rain for the day but the daily summary added up to 47.6. I can't figure out why. Can anyone see a glaring fault in the database query?

// find the unix timestamp for a given date
$unixtime = strtotime('2015/03/28');

// set the unix timestamp start and end times for each hour of the day and then advance 1 hour (3600 seconds)
for($t=0;$t<=23;$t++)
{
$starttime = $unixtime + ($t*3600);
$endtime = $unixtime + 3600 + ($t*3600);
//then make the query
$results = $db->query("SELECT datetime(min(dateTime), 'unixepoch', 'localtime'), avg(outTemp), avg(pressure), avg(outHumidity), avg(windSpeed), avg(windGust), avg(windDir), total(rain) FROM archive WHERE dateTime>= $starttime and dateTime<= $endtime;");


Here's what it looks like if anyone's interested:



On Wednesday, 11 March 2015 21:14:02 UTC+11, Powerin wrote:

Hi All,

I have weewx up and running for testing with a Fine Offset station (soon to be updated to Davis). It seems like a good stable system and I would like to move permanently to weewx but I need a daily summary of my weather records with hourly (or more) records similar to the weewx monthly or annual summaries. For example (from my wfrog setup):

                 Temp. C         Humidity %     Pressure hPa       Rain mm         Wind km/h       UV     
                max   min   avg  max  min  avg   max   min   avg   fall   rate    avg    max  dir
2015/03/11 20  24.7  23.4  24.0  25%  23%  24%  1021  1021  1021    0.0    0.0    0.0    2.5    E    -
2015/03/11 19  29.6  25.6  27.7  22%  15%  18%  1020  1020  1020    0.0    0.0    3.7   14.8    S    -
2015/03/11 18  31.1  30.0  30.6  15%  12%  14%  1020  1019  1019    0.0    0.0   10.1   24.5  SSW    -
2015/03/11 17  31.7  31.0  31.3  13%  12%  12%  1020  1019  1019    0.0    0.0   13.0   28.1   SW    -
2015/03/11 16  32.7  31.4  32.0  13%  11%  12%  1020  1020  1020    0.0    0.0   11.2   30.6  WSW    -

Is it possible to get this info from weewx? Has anyone done this? I have read a fair amount of the docs and wiki and haven't found anything yet. Please feel free to point me in the right direction bearing in mind I don't know python (simple BASH scripts are about my limit).

Why do I need daily summaries? I am a farmer. When I spray herbicides or pesticides I am legally required to record the weather conditions at the beginning and end of the spraying run (temp, humidity, wind speed, wind max and direction). It would be great if I could get weewx to provide that info. Thanks for your help.

Peter

[gjr80]

Hi,

Short answer, you are double counting some archive periods in each hour and your query needs to be:

$results = $db->query("SELECT datetime(min(dateTime), 'unixepoch', 'localtime'), avg(outTemp), avg(pressure), avg(outHumidity), avg(windSpeed), avg(windGust), avg(windDir), total(rain) FROM archive WHERE dateTime> $starttime and dateTime<= $endtime;");

ie omit the = from dateTime>= $starttime.

Explanation. The weewx archive record for a given timestamp (dateTime) holds the information for the (archive interval) period before the dateTime value. The rainfall from midnight until 1am is stored in the weewx archive records with timestamps (dateTime) from (midnight+archive interval) to (1am). The rainfall stored in the archive record with timestamp of midnight is actuall the rain that fell from (midnight-archive interval) to (midnight). The first query you do (to get rainfall from midnight to 1am) will in fact give you the rainfall from (midnight-archive interval) to (1am), your 1am to 2am query will return the rainfall from (1am-archive interval) to (2am), etc. So effectively you will double count any rainfall occurring from (archive interval) before the hour to the hour, if that makes sense.

Gary

[Powerin]

Excellent...that worked! I knew it must be double counting somewhere but I couldn't see it. Thanks for that Gary!

Peter

[Vladislav Jordanov]

сряда, 11 март 2015 г., 12:14:02 UTC+2, Powerin написа:

Hi All,

I have weewx up and running for testing with a Fine Offset station (soon to be updated to Davis). It seems like a good stable system and I would like to move permanently to weewx but I need a daily summary of my weather records with hourly (or more) records similar to the weewx monthly or annual summaries. For example (from my wfrog setup):

                 Temp. C         Humidity %     Pressure hPa       Rain mm         Wind km/h       UV     
                max   min   avg  max  min  avg   max   min   avg   fall   rate    avg    max  dir
2015/03/11 20  24.7  23.4  24.0  25%  23%  24%  1021  1021  1021    0.0    0.0    0.0    2.5    E    -
2015/03/11 19  29.6  25.6  27.7  22%  15%  18%  1020  1020  1020    0.0    0.0    3.7   14.8    S    -
2015/03/11 18  31.1  30.0  30.6  15%  12%  14%  1020  1019  1019    0.0    0.0   10.1   24.5  SSW    -
2015/03/11 17  31.7  31.0  31.3  13%  12%  12%  1020  1019  1019    0.0    0.0   13.0   28.1   SW    -
2015/03/11 16  32.7  31.4  32.0  13%  11%  12%  1020  1020  1020    0.0    0.0   11.2   30.6  WSW    -

Is it possible to get this info from weewx? Has anyone done this? I have read a fair amount of the docs and wiki and haven't found anything yet. Please feel free to point me in the right direction bearing in mind I don't know python (simple BASH scripts are about my limit).

Why do I need daily summaries? I am a farmer. When I spray herbicides or pesticides I am legally required to record the weather conditions at the beginning and end of the spraying run (temp, humidity, wind speed, wind max and direction). It would be great if I could get weewx to provide that info. Thanks for your help.

Peter

 Powerin, very interesting!

Can I ask for a final script, please!

[Mikael Fredriksson]

were there ever shared a script for this?
/Mikael

[Glenn McKechnie]

Have a look at wxobs.

I based that on Powerin's work (they started this thread)

https://github.com/glennmckechnie/weewx-wxobs

On 03/09/2020, Mikael Fredriksson <pligg...@gmail.com> wrote:
> were there ever shared a script for this?
> /Mikael
>

> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/9535d20d-f966-4ffb-b477-fe1b36323f2co%40googlegroups.com.
>


--


Cheers
Glenn

rorpi - read only raspberry pi & various weewx addons
https://github.com/glennmckechnie

[Mikael Fredriksson]

Great! I'll have a look at that. Thanks! 

/Mikael

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/cEAzvxv3T6Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAAraAzgbV0fC_6v5BYXd40WnhRzPf7N5DqpKz5t8XfJ7MuXFWg%40mail.gmail.com.

[Mikael Fredriksson]

Hi Glenn!

Can't get the conversion for database units to change.

This is how my skin.conf fill looks now (part under phpunits)

# Uncomment as appropriate...

         Only for the one/s you need.

        # Only from the group/s you need.

        # If you don't need a conversion performed then do nothing, leave them

        # all commented out.

#        temperature_convert = CtoF #degree C in database to degree F in report

         temperature_convert = FtoC # degree F in database to degree C in report

#        group_direction   = degree_compass

#        group_percent     = percent

#        group_pressure    = hPa        # Option inHg, mmHg , mbar or hPa

        group_rain        = mm         # Option are inch , cm or mm

        group_temperature = degree_C   # Option are degree_F or degree_C

        group_speed       = km_per_hour # Option are mile_per_hour km_per_hour

                                        # or meter_per_second

But still I get the values in F in the observation page.

Do you know what I could have missed?

/Mikael

Den torsdag 3 september 2020 kl. 02:11:20 UTC+2 skrev Glenn McKechnie:

Have a look at wxobs.

I based that on Powerin's work (they started this thread)

https://github.com/glennmckechnie/weewx-wxobs

On 03/09/2020, Mikael Fredriksson <plig...@gmail.com> wrote:
> were there ever shared a script for this?
> /Mikael
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to weewx...@googlegroups.com.

[Glenn McKechnie]

On 05/09/2020, Mikael Fredriksson <pligg...@gmail.com> wrote:
> Hi Glenn!
>
> Can't get the conversion for database units to change.
> This is how my skin.conf fill looks now (part under phpunits)
>
> # Uncomment as appropriate...
> Only for the one/s you need.
> # Only from the group/s you need.
> # If you don't need a conversion performed then do nothing, leave
> them
> # all commented out.
>
> # temperature_convert = CtoF #degree C in database to degree F in
> report
> temperature_convert = FtoC # degree F in database to degree C in
> report
>
> # group_direction = degree_compass
> # group_percent = percent
> # group_pressure = hPa # Option inHg, mmHg , mbar or hPa
> group_rain = mm # Option are inch , cm or mm
> group_temperature = degree_C # Option are degree_F or degree_C
> group_speed = km_per_hour # Option are mile_per_hour
> km_per_hour
> # or meter_per_second
>
> But still I get the values in F in the observation page.
>
> Do you know what I could have missed?

You've missed nothing, and I was surprised to be able to duplicate it here.

Turns out there is nothing wrong with the code, but a change that was
brought in with weewx 3.9 (02/05/2019) has finally caught up with
wxobs and the way I specified the labels. The Defaults section was
moved to [StdReport] so entries made there will apply to *all* skins
and they have precedence over anything in skin.conf

http://weewx.com/docs/customizing.htm#How_options_work

The fix is... The section within wxobs/skin.conf [Units] [[Groups]]
that refers to the Group units (which wxobs uses to generate the
labels) needs to be moved into the [[wxobs]] section within weewx.conf

Using your case above, the entries in wxobs/skin.conf...

group_rain = mm # Option are inch , cm or mm
group_temperature = degree_C # Option are degree_F or degree_C
group_speed = km_per_hour # Option are mile_per_hour

are moved to weewx.conf so that it will appear something like (your
path may be different)...
[StdReport]
[...]
[[wxobs]]
skin = wxobs
HTML_ROOT = /var/www/html/weewx/wxobs
[[[Units]]]
[[[[Groups]]]]

group_rain = mm # Option are inch , cm or mm
group_temperature = degree_C # Option are degree_F or degree_C
group_speed = km_per_hour # Option are mile_per_hour

Because these changes will now be read from weewx.conf, you will need
to restart weewx for them to take affect.

n.b. The [[PHPUnits]] section remains in wxobs/skin.conf. It does not
get moved. It is specific to wxobs.inc and weewx would probably spit
out an error if it had to parse it.

This all assumes that your database is in US units (thus why you are
converting from F to C)
and that your weewx.conf file has the section Std Reports with entries like...
[StdReport] [[Defaults]] [[[Units]]] [[[[Groups]]]]

Try it and let me know if this does indeed fix your problem. I'll then
adjust the notes / installer in wxobs to suit.

Thanks for catching it and for bringing it to my attention.

[Tom Keffer]

In addition to Glenn's comments, see the section How options work in the Customizing Guide for details on how options specified in weewx.conf override options in skin.conf.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAAraAziKtCfEpC_bGY0%2B1JeXDMfWGx6KCjUrHJTuzw%3DL%2BEcrHA%40mail.gmail.com.

[Mikael Fredriksson]

Glenn, thank you for your answer!

I did the change and moved those lines to weewx.conf, then I restarted Weewx.
But this doesn't change the values unit. It's still in US unit (F).
I hope that it was clear that I meant the values, not the labels.

I looked at the section How options work in the Customizing Guide that TK pointed at but I can't figure out if that could help me... I feel that it is a little too advanced for me at this point.

This is how it looks in weewx.conf after the change.

[[wxobs]]
HTML_ROOT = /var/www/html/weewx/wxobs
skin = wxobs
report_timing = 0 * * * *

[[[Units]]]
[[[[Groups]]]]
group_rain = mm

group_temperature = degree_C
group_speed = km_per_hour

What else can I provide to get help to sort this out?

/Mikael

[Glenn McKechnie]

Mikael,

The report_timing stanza that you currently have under the wxobs
section in weewx.conf generates the report once every hour.
That means that any changes you make to the configuration files will
not be propagated to the report pages (webserver) until the clock
chimes on the hour.
If you're not aware of that, it can make troubleshooting a real 'hair
pulling' exercise! What you change will not appear when you expect it.
Comment that line out until everything is working, then re-instate it
if you want to save some CPU cycles.

restart weewx when it's commented out.

This is not a bug that I can reproduce here (I'm running version 0.7.2).
Any temperature changes , combinations, that I make here to skin.conf
in this version are acted on and the generated page (index.php)
adjusts itself accordingly.

If you still have the temperature_convert line uncommented in
wxobs/skin.conf as you had before...

temperature_convert = FtoC # degree F in database to degree C in report

then any database values for temperature will have the FtoC conversion
applied. That conversion is found in the webservers index.php file
found on (line 103)

function FtoC($x)
{
return ($x - 32.0) * 5.0 / 9.0;
}

If that skin.conf option has been selected (as you've done) , and is
being applied to the generated index.php file then the calls to it
will be made from various locations such as found around line 711 (and
others)...

echo "<td>$metTemp</td>";
$x = FtoC ($latest_row[2]);
$metappTemp = round($x, 1);
echo "<td>$metappTemp</td>";
$x = FtoC ($latest_row[3]);
$metdew = round($x, 1);

That FtoC fuction name can be one of 3 things. FtoC, CtoF or NTC
(NoTemperatureCoversion)

If you have anything other than FtoC then the index.php script is not
being generated correctly.

As it's a skin.conf option, any changes you make will be propagated
through to index.php on the next report cycle (archive interval).

Check that what I've outlined above has been applied to your index.php
file. You can also also check it after you've changed CtoF or no
conversion (NTC) when the values should change accordingly.

If nothing is changing, then we need to dig a lot deeper and to do
that we'll take it off-list, to my address in these headers.

Send me your full syslog, the last copy of wxobs/index.php and your
weewx.conf (run that file through wee_debug and check that it's been
correctly stripped of passwords before sending).

> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to weewx-user+...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/weewx-user/3fe91512-4cc9-4fa5-b58a-b1abb45bd8c1o%40googlegroups.com.

[Mikael Fredriksson]

Glenn,

I did a test and changed the NTC to FtoC around the 207 line in index.php. And that changed the values in the tables.
Had no idea why the change in skin.conf didn't hit the tables but I now think I know what it was after I looked at the syslog!

These lines,

6 20:29:18 raspberrypi wee_reports[10137] ERROR weewx.reportengine: Failed to read skin configuration file /etc/weewx/skins/wxobs/skin.conf for report 'wxobs': Invalid line (u' Only for the one/s you need.') (matched as neither section nor keyword) at line 207.
Sep 6 20:29:18 raspberrypi wee_reports[10137] ERROR weewx.reportengine: Syntax error: Invalid line (u' Only for the one/s you need.') (matched as neither section nor keyword) at line 207.
Sep 6 20:29:18 raspberrypi wee_reports[10137] ERROR weewx.reportengine: **** Report ignored

...pointed me to a line in skin.conf where I had removed the # in that line.
I think that everything is ok now, I did a test and manually change back the FtoC to NTC and see if they change again at the archive run and they did.

So Glenn, I'm sorry that I missed this and you had to do all that writing, I will definitely first look at my syslog when things don't turn out the way they're expected to do!

Case closed.
/Mikael

[Glenn McKechnie]

:-))

Yes. You've gotta love the logs, and that weewx flagged the error.

This did highlight one bug with the configuration so we've both come
away with something.

> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/weewx-user/4acbabb6-7f08-40d5-9435-4dfb0c399ebao%40googlegroups.com.