Rain Rate Calc Problem?

[James Bellanca]

I installed and have been using WeeWx 4.1.1 on a Raspberry Pi 3B since June with a WeatherFlow Tempest and rain rate calculated perfectly there. I recently got a Raspberry Pi 4 and installed WeeWx 4.2.0. I have both up and running concurrently. Rain rate is not calculating on the new 4.2.0 version, though. I even tried forcing it to "software" instead of "prefer hardware" in the config. The sensor mapping and all other config, as far as I can tell (and I've checked a few times) is all identical, except development changes made to 4.2.0, obviously. Anyone encounter this or have any ideas as to why or how to fix rain rate?

[James Bellanca]

Btw, I've even checked/compared the rain rate data in the SDB files, and all other data is populated the same between the two instances, except the blank rain rate field. Look at the data in the first 10 rows of each file for 12/21. (Ignore the fact that Excel presents some of the small rain numbers exponentially - they're right in the database, it's just an Excel thing.) You'll see that it's adding up the rain values perfectly - just off very slightly due to timing. But, rainRate's = 0 in the 4.2.0 file, so it's not calculating. Any thoughts as to why?

Google wouldn't let me upload the files here, so here they are:

https://www.dropbox.com/s/9a8wicnjs4y8r0j/rainRate_12-21_4.1.1.slk?dl=0

https://www.dropbox.com/s/qcyng5ckxoiqzmw/rainRate_12-21_4.2.0.slk?dl=0

[p q]

Any messages in the log? Did you switch python versions?

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/267f1df2-33ce-4609-92c9-bf01ce173ed4n%40googlegroups.com.

[Tom Keffer]

In addition to Peter's questions, what do you have for a sensor map?

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAA1SM20Yhv0N%3DctzQ5xkQU%3DOwbQ%2B7P6%3Dyn35qyHiuP3N_HGV9w%40mail.gmail.com.

[James Bellanca]

Both running on Python 3. 

$ ps ax | grep python

  831 ?        Sl   468:45 python3 /usr/share/weewx/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf

Attaching 3 files - the weewx.conf files from both 4.1.1 and 4.2.2, and the logs from 4.2.0 from 12/21 through today. It's been raining all morning today (12/24), and same thing - 4.1.1 calcs the rainRate, but 4.2.0 doesn't. (weewx.conf files have been renamed to include the version number.) Sensor maps between them are both the same - and it's def getting the rain value itself correct and inserting it to the database fine. I don't see anything suspicious in the log, either. I appreciate the help!

https://www.dropbox.com/s/tfvngl92flq9a4b/weewx-4.1.1.conf?dl=1

https://www.dropbox.com/s/33uakjaj8oix3yk/weewx-4.2.0.conf?dl=1

https://www.dropbox.com/s/g9egyues81u87qj/weewx-4.2.0.log?dl=1

[James Bellanca]

Here's a little more info. I added from log.debug lines in the wxxtypes.py rainRater class.

In get_scalar that calculated rainRate, right before "return rr", I added: log.debug("rr=%s", rr)

In _setup in the if not... section right before the except error trap, I added: log.debug("Log row %s %s %s.", time_ts, unit_system, rain)

Now in the logs, I'm seeing a lot of rows like:

Dec 24 10:36:56 raspberrypi weewx[18054] DEBUG weewx.wxxtypes: rr=(0.0010683464566929135, 'inch_per_hour', 'group_rainrate')

Dec 24 10:37:01 raspberrypi weewx[18054] DEBUG weewx.wxxtypes: rr=(0.0015807874015748033, 'inch_per_hour', 'group_rainrate')

...

Dec 24 10:42:04 raspberrypi weewx[18054] DEBUG weewx.wxxtypes: rr=(0.0017618897637795277, 'inch_per_hour', 'group_rainrate')

I've been watching the log for 10 mins now and I'm not seeing any "Log row..." debug messages. 

But, this means is calculating rainRate, but it's not updating the database with what it's calculating for some reason.

[Tom Keffer]

The log does not include when weewx started up, so we can't tell which weewx.conf it is using, what the sensor map was, nor what services are being loaded. Otherwise, it looks normal. Do a restart, then post the log through the first archive cycle.

If rainRate is being calculated (and your instrumented version of wxxtypes.py seems to indicate that it is), then, if it's in the schema, it should be incorporated into the database.

If you run weewxd directly from the command line, it will print out all LOOP packets and archive records. What do they show for rainRate?

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7e77eaad-44a1-4e37-be67-fea6e9051336n%40googlegroups.com.

[James Bellanca]

Appreciate the help so far. I had to wait for it to start raining again to do anything haha. And ok, so this is very strange behavior. 

I stopped the weewx service, and ran it directly (log of the terminal window is attached). rainRate calculates fine, AND updates the database fine, just like it should. I stop the direct run, and run it as a service again, and it's working fine now! HOWEVER – Those results are weird, so I did it all again from scratch - rebooted, ran at startup as a service (no rain rate in the database again, just like before!), stopped it, ran it directly (rain rate works fine), stopped it, started the service (and now it works again?!). So it calculates rain rate, but won't update the database with it, UNTIL I run it directly first, then after that, it works even not directly... until I reboot, and it's broken again.

Any idea what would cause that bizarre behavior???

Attaching three files - 

(1) weewx.log from startup (directly) through some archive cycles where it was actually raining (which you'll see near the end).

https://www.dropbox.com/s/iqmi3e61y1zn0yt/weewx-1224-1500.log?dl=1
(2) terminal.log from running weewx directly. I can see in this that it's definitely calculating rainRate. It's just not updating the field in the database with the value it's calculating.

https://www.dropbox.com/s/xtbilj0s1ar9fw7/terminal-1224-1500.log?dl=1

(3) full weewx.log from Dec 9th (when I confided this new Pi 4) with weewx startup (as a service) through some archive cycles where it was raining, including the direct starts today where it worked.

https://www.dropbox.com/s/6veouk8rets8lsg/weewx-420-full.log?dl=1

[Tom Keffer]

Hard to say without seeing the log, but if I had to hazard a guess, I'd say you're not consistently using the same copy of weewx.conf for all runs. One of the copies does not have rainRate in [StdWXCalculate].

But, that's just a guess. 

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b0db090a-8d25-460b-9c99-fc76d29d8f74n%40googlegroups.com.

[James Bellanca]

I do appreciate the guess, but I know they’re all using the stone conf file since there’s only one! :-)

Sent from my iPhone

On Dec 24, 2020, at 6:56 PM, Tom Keffer <tke...@gmail.com> wrote:



You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/zR14R_FTd4I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEAbTZ2QLxifFj0Tuwpg7GqeC8TJaE7etd7cv-qEfTu2Kw%40mail.gmail.com.

[Banana Bob]

It may be of interest to you that I had problems with the rain rate being reported as 0.00 all the time.

I eventually tracked this down to the weewx.conf file having cm/hr rather than mm/hr as the measurement required. This was in the Units Group section of Standard Report and Defaults lower down.

[Neville Davis]

Hi I have the same problem...

We have long periods without rain then it buckets down so I have only recently started to notice it is always zero.

I upgraded to 4.3 this month and had a reading of 2mm per hour after a rain period...and thought the upgrade had fixed it. Then last evening a large downpour approx 30mm in about 1 hour...rainRate = 0.0 mm per hour.

I have checked [StdWXCalculate] and rainRate was prefer hardware ( mine is a home brewed system, raspberry Pi with i2C sensors, without rainRate) I changed it to software calculation. Still zero.

I then noticed this last post, but my groups have it at mm per hour... I have noticed that the group has rainrate not rainRate, I checked some old weewx.confs back to may last year they are also rainrate??

I started weewx directly and a copy of the consul log is attached....at the beginning I also looked at old weewx.confs to see if I had made any stuff ups...so this info is also in the console log.

I also looked back into mySql db and this zero rainRate has been there for sometime..except for this one time 2.0mm per hour. 

If I set debug in weewx.conf what extra data will it show??

Any suggestions...??

            [[[[Groups]]]]

                

                group_altitude = meter    # Options are 'foot' or 'meter'

                group_degree_day = degree_C_day    # Options are 'degree_F_day' or 'degree_C_day'

                group_distance = km    # Options are 'mile' or 'km'

                group_pressure = hPa    # Options are 'inHg', 'mmHg', 'mbar', or 'hPa'

                group_rain = mm    # Options are 'inch', 'cm', or 'mm'

                group_rainrate = mm_per_hour    # Options are 'inch_per_hour', 'cm_per_hour', or 'mm_per_hour'

                group_speed = km_per_hour    # Options are 'mile_per_hour', 'km_per_hour', 'knot', or 'meter_per_second'

                group_speed2 = km_per_hour2    # Options are 'mile_per_hour2', 'km_per_hour2', 'knot2', or 'meter_per_second2'

                group_temperature = degree_C    # Options are 'degree_F' or 'degree_C'

                group_direction = degree_compass

                group_moisture = centibar

                group_percent = percent

                group_radiation = watt_per_meter_squared

                group_uv = uv_index

                group_volt = volt

                group_rpm = rpm

[StdWXCalculate]

    

    [[Calculations]]

        # How to calculate derived quantities.  Possible values are:

        #  hardware        - use the value provided by hardware

        #  software        - use the value calculated by weewx

        #  prefer_hardware - use value provide by hardware if available,

        #                      otherwise use value calculated by weewx

        

        pressure = prefer_hardware

        altimeter = prefer_hardware

        appTemp = prefer_hardware

        barometer = prefer_hardware

        beaufort = prefer_hardware

        cloudbase = prefer_hardware

        dewpoint = prefer_hardware

        ET = prefer_hardware

        heatindex = prefer_hardware

        humidex = prefer_hardware

        inDewpoint = prefer_hardware

        maxSolarRad = prefer_hardware

        rainRate = software

        windchill = prefer_hardware

        windrun = prefer_hardware

This was the first recording of a rainRate for a long time....but only once right after upgrade to 4.3.

[Tom Keffer]

Your weewx.conf seems to be configured correctly. And, it's definitely "rainRate" (not "rainrate"). Let's take a look at the database, using the tools sqlite3. 

You may have to install it:

sudo apt install sqlite3

sqlite3 /home/weewx/archive/weewx.sdb

sqlite> select dateTime,datetime(dateTime,'unixepoch','localtime'),rain,rainRate from archive where rainRate>0 order by dateTime desc limit 50;

sqlite> .quit

This will print the last 50 non-zero rainRates. 

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b52d225f-70a3-4583-8a59-9ed2ff45ba60n%40googlegroups.com.

[Neville Davis]

Hi

installed sqlite3 but when run I get nothing printed. the same query with rain I also get nothing printed, it just returns to sqlite3 prompt.

Nothing on console, user logs etc

I am going to try the same query to see if I can get the data using Sequel Pro. I use this app to check my db usually.

I mention rainrate because that is in the Groups in weewx.conf...I have not edited that part of the file, in any of my versions of weewx.conf...just looking for any possible anomaly.

Nev

[Neville Davis]

Hi

Using sequel pro

SELECT  dateTime,rain,rainRate

FROM    archive

WHERE   rainRate>0 order by dateTime desc limit 50;

[Neville Davis]

Hi using 

SELECT  dateTime,rain,rainRate

FROM    archive

WHERE   rain>0 order by dateTime desc limit 200;

the last 200 records for rain

[Tom Keffer]

Sorry, I did not notice that you are using MySQL (any particular reason?)

I'm finding your configuration very confusing. Are you using the driver PiWeather? Or, OWFS? I'm seeing PiWeather in your configuration file, but OWFS in the log. 

You have a lot of moving parts in your configuration. Here's what I'd like to see:

1. Use a single driver. Don't switch.

2. Set debug=1 in weewx.conf

3. Restart weewxd

4. Pour a bit of water in your rain bucket

5. Wait a reporting cycle or two

6. Report what is in your database.

-tk

I'd like to see a single driver in use, then a rate event, then the database. There are too many moving parts to unravel what is happening with your configuration.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1b82eb56-dcc9-4caa-b9cb-0cc65cbcfcf8n%40googlegroups.com.

[Neville Davis]

Hi

I have been using this configuration for several years......The overall hardware design is my own...I am a hardware guy more then a software type but had some experience with sql years ago (I am 76) and wanted use replication to backup to my local nas, so I migrated to Mysql when I built this system about 4 years ago....PiWeather is the driver I built to handle the data derived from my i2C sensors, Davis masthead wind gear, my FARS external temp sensor, and FARS fan speed monitor (to detect failure) etc. I could not work out the best way to get the one wire rain bucket data into weewx.....In the end it appears I do the both as drivers as you say....but it has never given me any trouble..and I didn’t know any better :).

Before I saw this post I was planning to do as you suggest ( add water and observe) and see what that gives me. We are expecting a fine day on the weekend I will do it then.

So to get only a single driver......I am not sure how to do that I will give it some thought......and that seems to be getting harder for me :).

Neville

[Neville Davis]

Hi just revisited my weewx.conf.... I believe I only have one driver PiWeather, the OWFS is configured as a service as per the instructions in owfs.py..

........

Place this file, owfs.py, in the weewx 'user' directory

or wee_extension will install it there.

To use as a driver:

[Station]

    station_type = OWFS

To use as a service:

[Engine]

    [[Service]]

        data_services = user.owfs.OWFSService

My weewx.conf has

[Engine]

    # The following section specifies which services should be run and in what order.

    [[Services]]

        prep_services = weewx.engine.StdTimeSynch

        data_services = user.owfs.OWFSService, user.aircon.AirconService

        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate

        xtype_services = weewx.wxxtypes.StdWXXTypes, weewx.wxxtypes.StdPressureCooker, weewx.wxxtypes.StdRainRater, weewx.wxxtypes.StdDelta

        archive_services = weewx.engine.StdArchive

        restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS, user.mqtt.MQTT, user.twitter.Twitter

        report_services = weewx.engine.StdPrint, weewx.engine.StdReport

In data services I also have a service  to collect temperature data from our aircon system to display within the skin as well as OWFS..

As to what is seen in the log about this, is way above my pay grade....:)

What is interesting about all this was that when I upgraded to 4.3 this month, for that day, rainRate was recorded. 

I will do the rain test as well as full restarts to see if anything pops up.

The biggest change to my system of recent months has been the addition of a MQTT broker for the Belchertown skin...I did have some issues with rain being displayed in cm not mm and there is an addition to the MQTT section in weewx.conf to correct this...at that time I had also a section for rainRate but disabled it to try and fault find this problem....Coincidence??? 

I will also disable any other skins and use just the Standard skin to ensure there is nothing happening there.

Neville

[Tom Keffer]

That's all fine, so long as the field 'rain' comes from a single source. Which one is it?

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/18cab61f-cd2e-4d58-aeab-de171f6baf3an%40googlegroups.com.

[Neville Davis]

I don't have any rain calculations or gathering in my script for PiWeather...just looks at the other sensors NOT rain.

I am currently going through a test and with only Seasons skin and running  weewx directly rainRate is calculated....could not wait for weekend :)

As I look at the scrolling data the rainRate calc is reducing (as it should be no more bucket tips) I will get back to you if I find anything significant.

Neville

[Neville Davis]

HI

At the end of testing..starting from only seasons skin, MQTT disabled etc, direct run weewx, all the way to fully operational and not once did the rainRate cease to operate???? including shutdowns reboots etc. Simply could not fault it.

We have had now rainfall last couple of days so I was able to clean the db archive and dailies for the last two days for rain and rainRate.

Just have to keep an eye on it I guess...reminds me so often as an electronics tech called to some navigation gear by an operator reporting faults and nothing found...I thought I had left that behind a couple of decades ago :) .

Neville