removed bad rainrate data but this did not reflect in totals

[Kevin Crivelli]

I removed bad rainrate data but it did not effect the raintotals field. I was able to determine the epoch date-time for the bad rainrate data. Having those I was going to delete the rain_total values for those entries but how would I have that then reflect in the following rain_total fields?

chatgpt gave me this which I feel is a start but I'm not quite there yet. anyone able to help me with this?

-- Store the rain total of the specific datetime in a variable
SET @deleted_rain_total = (SELECT rain_total FROM your_table WHERE datetime = specific_datetime);

-- Delete the rain total at the specific datetime
DELETE FROM your_table WHERE datetime = specific_datetime;

-- Decrease the rain totals for the following datetimes
UPDATE your_table
SET rain_total = rain_total - @deleted_rain_total
WHERE datetime > specific_datetime;

These are the date and times of the entries that need their raintotal nulled and then the proceeding raintotals to reflect the change

    1699559700: 2024-05-10 01:55:00
    1699560000: 2024-05-10 01:56:40
    1709221800: 2024-05-26 06:50:00
    1709222100: 2024-05-26 06:55:00
    1713436800: 2024-06-16 10:00:00
    1713437100: 2024-06-16 10:05:00
    1713780900: 2024-06-20 03:35:00
    1713781200: 2024-06-20 03:40:00
    1714771200: 2024-06-25 18:40:00
    1714771500: 2024-06-25 18:45:00
    1699559400: 2024-05-10 01:50:00
    1699560300: 2024-05-10 01:51:40
    1709221500: 2024-05-26 06:45:00
    1713437400: 2024-06-16 10:10:00
    1713781500: 2024-06-20 03:05:00
    1714771800: 2024-06-25 18:10:00

[Kevin Crivelli]

or perhaps there is a way to rebuild the raintotals fields from the current rainrate data

[vince]

You might want to consult the wiki.  That's why hundreds of hours have been spent creating it.

https://github.com/weewx/weewx/wiki/Cleaning-up-old-'bad'-data

[Kevin Crivelli]

I did everything here. It is how I was able to remove the bad rainrate data. The problem now is that while the bad rainrate data is gone, the resulting rain total data is not. I need to get the rain totals to reflect the rainrate data now. The totals are still considering the bad rainrate data that has been removed. 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b687da34-99af-42a8-a84b-490f6c902d91n%40googlegroups.com.

[michael.k...@gmx.at]

rainRate is a calculated value: https://weewx.com/docs/5.0/reference/weewx-options/stdwxcalculate/

So first you correct all the rain values and may want to delete rainRate values from your database

Second recalculate them: https://weewx.com/docs/5.0/utilities/weectl-database/#calculate-missing-derived-variables

Third drop the daily summaries https://weewx.com/docs/5.0/utilities/weectl-database/#drop-the-daily-summaries

Fourth rebuild the daily summaries https://weewx.com/docs/5.0/utilities/weectl-database/#rebuild-the-daily-summaries

If your archive values are sane, your totals will be sane then.

Consider only dropping/rebuilding the daily summaries for days with bad data, this will be faster and you won't lose the exact timestamps/value for min/max records on all the other days. See these switches in the docs:  [[--date=YYYY-mm-dd] | [--from=YYYY-mm-dd] [--to=YYYY-mm-dd]]

[gjr80]

Whilst rainRate may be a value calculated by WeeWX, the current StdWXCalculate service is limited in that it can only calculate rainRate from loop packet data. In the case of a backfill/recalculate (as with an import) the only rain data available is from archive records and hence rainRate is not calculated. In these cases rainRate must be manually calculated.

Issue #787 was raised to address this behaviour for imports. Any resolution to issue #787 would likely fix the rainRate re-calculate behaviour as well, but I have no  timeframe for when issue #787 might be addressed.

Gary

[michael.k...@gmx.at]

Good to know. I can't remember how I recalculated the rainRate some years ago, when I switched from an 1h-interval to a 15-min interval. Obviously not with --calc-missing, because it worked :)

[gjr80]

That's quite possible, when Xtypes came along in v4.0.0 the StdWXCalculate service was re-written to utilise Xtypes resulting in rainRate only being calculated for loop packets. Versions before v4.0.0 calculated rainRate for both loop packets and archive records.

But I digress.

Gary

[Martin Thoresen]

Ahh, thank you for this information! I've spent several days trying to calculate rainRate for my newly imported historic data by using weectl calc-missing, with all sorts of different settings tested out in weewx.conf. I could get all the other data I wanted, like windrun and heat index and so on. I've pulled some hairs during this period, that's for sure. But now it all makes sense: rainRate does not work on archive records, only loop data.

I could not understand from the wiki that rainRate was exempt from working on archive records, that's why I believed I was doing something wrong all this time.

You definitely have my vote if you are considering resolving issue 787.

Best regards,

Martin Thoresen