All-Time Data different than max observed

[Mauro De Lauretis]

Hi all

Today I was looking better to my MySQL database and I was wondering why my showed all-time data is always lower than my max observed data.

Let me explain it better taking an example from UV Sensor:

In the archive table, the max observed value is 1.1 UVI and it's currently the highest value I got since I've installed the sensor. But if I take a look on archive_day_UV, the highest value from today and taken for the "All-Time" record is 0.9 UVI.

Could someone explain me why archive_day takes this value and not 1.1?

Greetings

[Tom Keffer]

No explanation. Usually the values from the daily summaries are higher than those from the archive table because they also include transient values from the LOOP packets.

But, let's double check. run these queries:

select from_unixtime(dateTime), UV from weewx.archive order by UV desc limit 1;

select from_unixtime(maxtime), max from weewx.archive_day_UV order by max desc limit 1;

Do they match?

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/85e91c5b-10ff-41bc-84b0-c1c93e5e9bf1n%40googlegroups.com.

[Mauro De Lauretis]

Hi Tom!

Thank you for your answer.

Yes, I have a match for both at

2020-12-09 07:00:00

But there I had no sensor and the values are on UVI 0 of course.

[Mauro De Lauretis]

EDIT: I’ve chosen the wrong database before. They don’t mach!

UV 0.856 on archive and 1.05 on archive_day, but on my website it’s totally the opposite

[Tom Keffer]

As I mentioned in my first note, I'm not surprised the value from the daily summary is higher --- that's because it includes transient  values from the LOOP packets.

Don't know what you mean by "on my website it's totally the opposition." How can you tell the difference?

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/389c7ce7-b21f-4b6b-ae3f-60ef97c8a68dn%40googlegroups.com.

[Mauro De Lauretis]

Sorry Tom, I got a little confused because first I thought that the value UVI 0.856 was in the archive_day_UV  and not archive and UVI 1.05 in archive instead of archive_day_UV. Therefore it's right like you told me, because I know that the LOOP packets values can be higher for daily summary.

What I don't understand is why I'm getting the UVI 0.856 for All Time and not the higher of UVI 1.05.

Is there something wrong in my formulas?

$alltime.UV.max = I get 0.856

$year.UV.max =I I get 1.05

[Tom Keffer]

Not sure. Are you sure they cover the same time period?

What do you get if you put this in a template?

  <table>
    <tr>
      <th>Start</th>
      <th>End</th>
      <th>Value</th>
    </tr>
    <tr>
      <td>$alltime.start ($alltime.start.raw)</td>
      <td>$alltime.end ($alltime.end.raw)</td>
      <td>$alltime.UV.max</td>
    </tr>
    <tr>
      <td>$year.start ($year.start.raw)</td>
      <td>$year.end ($year.end.raw)</td>
      <td>$year.UV.max</td>
    </tr>
  </table>

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0a5dec6c-c98c-41dd-8331-2b5592d062een%40googlegroups.com.

[Mauro De Lauretis]

I've solved it. Since I get this data from another data_binding and not the main one, I forgot to enable the search_list_extensions = user.stats.MyStats in the skin.conf of the second weewx instance.

[Mauro De Lauretis]

Tom,

sorry to annoying you again, but I still have a little issue.

Since I am using the Weather34 skin with multiple bindings due two different stations running (Davis and WeatherFlow, each with his own skin folder and configuration) I have this for my UV Almanac on my Davis folder:

$alltime($data_binding='weatherflow_binding').UV.max

This because my Davis it's not provided by UV Sensor, so I take it from my WeatherFlow.

Curious is the fact that by running it on the WeatherFlow itself, the showed value is the correct one (UVI 1.05).

As soon as I run it on the skin folder of my Davis and tell him to take the values from the WeatherFlow's database (weatherflow_binding), I get the wrong value (UVI 0.856).

Il giorno martedì 9 novembre 2021 alle 22:58:43 UTC+1 tke...@gmail.com ha scritto:

[gjr80]

Something to ponder and maybe worth some input from Tom.

I think the problem here is a timing issue between the time a report is being generated on one WeeWX instance and the time that records are added to (at least) two different databases. A key point to remember is that for $alltime to use the daily summaries the timespan concerned must start and end on either the first timestamp in the archive, a midnight boundary or the last timestamp in the archive. If this condition is not met the archive is used instead of the daily summaries.

The use of two different bindings implies two different databases. Whilst I don’t think we have been given a detailed description of the setup, I assume there are at least two WeeWX instances running; one with a vantage station and the other a Weatherflow each using their own database. The vantage station will be saving an archive record to database almost immediately after the end of the archive period plus archive delay. I am not familiar with the Weatherflow station, but it is quite possible the archive records are not generated until sometime after the end of the archive period plus delay, typically on receipt of the next loop packet. In such cases, if the vantage instance produces a report using the Weatherflow binding (database) the timespan used by the alltime tag will likely not match the first and last timestamps of the Weatherflow database meaning the archive will be used rather than the daily summaries. However, when the same tag (with the Weatherflow binding of course) is used on the Weatherflow system the timespan in use covers the first and last timestamps in the Weatherflow database (because the reports are run until after the archive record has been saved) so the daily summaries are used. Hence the different results for the seemingly same tag on the two different systems.

Should be easy enough to confirm by comparing the logs from the two instances.

How to fix? I’m not sure, typically the way timing differences between reports and multiple databases are handled is to use $latest in place of $current, but that won’t work here. I guess you could design some elaborate system of reports where the Weatherflow system generates/updates a file that the vantage system then uses - sounds like a lot of work. Or you could live with it. Worth noting though that you do pay a performance penalty by doing a query over the entire archive, so if you do live with it as is I would keep any eye on your report times and not go adding numerous more similar tags. 

One out of the box solution could be to increase the vantage system archive delay so that reports are not run before the Weatherflow system has saved its archive record. But this approach may have other undesirable effects, eg delaying the production of vantage reports/pages.

Then of course Tom may have a magic solution.

Gary

[Mauro De Lauretis]

Hi Gary!

Thank you for your brilliant description.

Yes, you're right. I'm running two instances of WeeWX with two different databases, skin folders and configuration files.

 

The alltime string I've posted before was quite right, but for whatever reason, the main skin folder (Davis Station) continued to take the wrong value from the database of WeatherFlow. Since the Weather34 skin has a .php file that collects all settings and processed data, including an external file as well for the sensors (called archivedata.php), I could solve it by including the archivedata.php file of the WeatherFlow as well, obviously disabling the data of the other sensors I don't need (temperature, rain, etc.). Otherwise they would "crash" together and show me fake data. 

Mauro