wxnow.txt does not contain precipitation totals

[Bill Arthur]

I recently added the cwxn extension so that I could pass the weather to my APRS/CWOP app

Its working fairly well, but it only reports the current rain rate. The 1hr and 12hr (p and P) are always 000 
We had 1.03" today so I had plenty of opportunities to test. The weewx html page shows the rain.

Here's my example:

Feb 24 2020 18:20
129/002g004t052r000p000P000h096b10067

I put "binding = archive"  in  weewx.conf

I'm missing something here, any ideas?

[gjr80]

Hi,

Looking at the cwxn service code there are three rain related values included in the output, these are (named in the code) hourRain, rain24 and dayRain. Presumably these are total rainfalls in the last hour, the last 24 hours and since midnight respectively. The cwxn service calculates hourRain by querying the database so hourRain should always be correct. However, rain24 and dayRain are not calculated by querying the database, rather they are pulled directly from fields of the same name in either the loop packets or archive records (depending on whether you bind to loop or archive) emitted by your station. Unfortunately, not all stations emit these fields and I am guessing that is the case with your station. You didn't mention what (weather) station you are using?

As for the data on your web page, if that is the Standard or Seasons skin that is shipped with WeeWX then in all likelihood you will find that the WeeWX tag system is being used and something like $day.rain.sum is being used to calculate and display the days rainfall by querying the database. Unfortunately the WeeWX tag system is only available in WeeWX reports and is not available in services such as cwxn.

Don't worry, all is not lost, if it is a case of your station not emitting rain24 and dayRain it will be a fairly straightforward job to modify the cwxn service to calculate these fields from the database just as is done with hourRain. I know the cwxn author is a bit busy with the WeeWX 4.0 release at the moment but let me see if I have some time tomorrow to make the necessary changes (or perhaps some other enthusiastic WeeWX user will come along and do the same before then) :)

Gary

[Bill Arthur]

Hi Gary,

Thanks for the quick response. This is my second week with Weewx so I assumed it was something I overlooked. And I'm glad a solution is relatively close.
I am using a Ambient Weather WS-2902 array with an Ecowitt GW1000 and using the interceptor 0.53.  cwxn is 0.4

[gjr80]

Bill,

Try the attached file in place of your existing cwxn.py.This version will calculate rain24 and dayRain by querying the database if rain24 and dayRain fields are not in the packet received from your station.  To install:

1. rename your existing cwxn.py to cwxn_orig.py (it will be in /home/weewx/bin/user or /usr/share/weewx/user)

2. download the attached cwxn.py and save in place of your current cwxn.py

3. restart WeeWX

4. monitor the log and once a report cycle is complete check wxnow.txt

If any issues post a copy of the WeeWX log from startup showing any errors. If you need to revert to the old version just delete cwxn.py and rename cwxn_orig.py back to cwxn.py and restart WeeWX.

Gary

[Bill Arthur]

A million thanks, Gary

I'll install it tonight. But It'll be a few days until we have rain again, I'll let you know as soon as it gets wet.

Bill 

[Bill Arthur]

BTW, I had to remove binding = archive. It resulted in the same data for over 24 hours.
Here is data as it is received by CWOP:  http://www.findu.com/cgi-bin/wx.cgi?call=wa4opq-1

On Tuesday, February 25, 2020 at 5:21:09 PM UTC-6, gjr80 wrote:

[gjr80]

That is interesting. Possibly because there was an error and wxnow.txt is not being updated and hence an old copy is being sent repeatedly. Did you look in the logs for any clues? I was running binding = archive here on a machine and wxnow.txt is working just fine. Running with binding = loop or with no setting (which defaults to loop) will produce wxnow.txt on arrival of every loop packet. With the extra db reads the new version has this will place a greater load on your machine but it should easily handle it.

Gary

[Bill Arthur]

At this point I think I'll work on this problem first. I don't like to add or change anything in the troubleshooting process. I'll postpone changing the cwxn.py.
I'll let you know what I find

Bill

[gjr80]

OK, just remember; the log is your friend, it will tell you most things that are going on.

Gary

[Bill Arthur]

He may be my friend but he is giving me a workout.

This is curious:
in weewx.conf  in the section for uploading data to Internet sites I have:
[CumulusWxNow]
    filename = /vat/tmp/wxnow.txt
    binding = archive

In my verbose log I have; 
cwxn: service version is 0.4
cwxn  binding is loop
cwxn  output goes to /var/tmp/wxnow.txt

Now I'm really confused.

[gjr80]

Some obvious things. You did save weewx.conf? You did restart Weewx? Is there more than one weewx.conf on your system and are you editing the right one(it happens). When you start Weewx look at the log during WeeWX startup and it will tell you the full path to weewx.conf, that is the file you need to edit.

Gary

[Bill Arthur]

Gary,

I saved and restarted, then I rebooted. I've confirmed correct /etc/weewx/weewx.conf location listed in the /var/log/syslog file. I'm not aware of any other weewx.conf file

I've set it back and I'm trying it again. Will advise

[Bill Arthur]

I've configured it (after stopping) with binding=archive in /etc/weewx/weewx.conf.. saved. viewed to confirm. restarted. syslog says binding=loop. rebooted. same results.
stopped weewx. deleted line.saved. viewed to confirm. restarted. read syslog. syslog says binding=loop. stopped. set binding=archive. saved. viewed to confirm. restarted. syslog says binding=loop.. rebooted. read syslog. no change.

I don't believe it has any effect, but i should tell you that I am not directly connected to the RasPi, I've working through VNC.

[Bill Arthur]

One more thing...
The original problem was that when I had added binding=archive my output was static for a day. Now, I have binding=archive in weewx.conf and my output is updating regularly.. But we seem to have a different problem now, with binding=archive not showing up in the syslog.

[Bill Arthur]

Instead of spinning my wheels over binding=archive, I updated cwxn.py.  It seems to be working fine. Syslog reports version 0.5a and "binding is loop"  Wxnow.txt is updating and new data is making it to findu

[Bill Arthur]

Gary,

Almost 24 hours later....  cwxn 0.5a is chugging along. It's outputting wxnow.txt every five minutes. I'm good with that.
Maybe cwxn binding is actually set for archive but it's just not displaying properly in the log. Output at exactly five minutes makes me think it is.

Now we just need some rain.

Bill

[gjr80]

Bill,

Well it is good that it is working as it should. As for binding = archive and what is displayed in the log during startup, well the code is pretty cut and dry. If binding is set to archive it should be displaying archive, so that suggests that whilst it looks like binding = archive there may be some other invisible characters in there that are interfering. What editor do you use to edit weewx.conf, is it a proper text or code editor?

I guess it is up to you if you want to look at this any further, when I see things that are working for the wrong reason I like to look closer as you don't know what else might be being masked by a potential issue. If you do want to look any further the next thing I would do would be post a wee_debug report, that will give us a clear picture of your config (which we haven't seen yet). If you do post a wee_debug report do check it for sensitive info such as user names, passwords, API keys etc before posting, wee_debug does a good job at obfuscating these things but it is not perfect.

Gary

[Bill Arthur]

Gary,

I use nano as editor.   
We're supposed to have a good rain here on Monday, so I should be able to let you know if it solved the problem.
I have another RasPi arriving in a few days and will be installing this on it. Perhaps it will not have the problem.
For now I'm staying focused on the 1hr and 12hr problem.
Thank you so much for your quick and helpful response.

Bill

[Bill Arthur]

Gary,

I was happy today as we had a light snowfall this morning; when it melted we had some "rain" to measure. The first hundredth was great. What happened at .02?

Bill

[Bill Arthur]

Some more info:
Weewx saw a total of .10
my meteobridge saw .15 and reported it to wunderground.

[Bill Arthur]

It looks like the 12 hour data is good. but I don't understand the 1 hour. Rain starts at 15:54 and I would have expected 1hr to track 12hr for the first hour. 
But it looks like we have an unexplained stoppage of data from 16:04 to 18:08. So I will have to collect data on another day. It should be raining on Monday.
And I' need to find out what caused the two hour outage.

Bill

[gjr80]

Bill,

Perhaps this has all been overtaken by your last three posts. But looking at your table of data in the first post that 2 hour gap could easily account for the figures in the table.

I'm not sure what the 2nd plot in your 2nd post shows; since one goes to 0.15 and stays there I am guessing that is your day rainfall meteobridge data on WU. What is the other, 'Prec. Rate' (I think that is what WU calls it)? Again from the meteobridge? If so that is merely rainfall in the last hour so I would expect that to go up and down and unlikely to match the 0.15in. Agree though there appears to be some inconsistency there between WeeWX and meteobridge/WU, but you would need complete WeeWX data and meteobridge/WU data from the same period to track down what is going on. It may create a lot of log chatter but setting debug = 1 just before the rain hits will give you all the raw data that the interceptor driver receives from the GW1000 as well as the parsed data passed from the interceptor driver to WeeWX.

You've mentioned the '12 hour data' a couple of times. What does that refer to? As far as I am aware you have 1 hour data, 24 hours data and day data.

As for the 2 hour odd outage, what is in the WeeWX logs? Rain went from 0.01 to 0.09 so clearly WeeWX was receiving data for at least some art of that period.

Guess we wait for Monday.

Gary

PS. As I understand it the meteobridge obtains its data from the GW1000 via the GW1000 API whereas WeeWX obtains its data from the GW1000 differently by intercepting HTTP POSTS made by the GW1000. Ideally they should provide the same data but it is possible they do not.

[Bill Arthur]

Gary,

Oh, what a trip this has been. It seems we keep finding other inconsistencies.
For both of our sanity sake I will try to keep on topic. It looks like the 24 hour data is good. (Not sure why I was calling it 12)

Thanks for suggesting the debug=1 log. I will try to get it going on time. I think it'll tell us a lot. (but it looks like cwxn doesn't log any info)

We'll see what Monday brings.

Bill

[gjr80]

Don’t get too hung up on wxnow.txt, it merely pulls data from the database/the current archive record and presents it. If the WeeWX generated plot you posted is correct then there is a discrepancy between what WeeWX is receiving and what your meteobridge is receiving. This is best tracked down by looking at the underlying data. Once we know the correct data is being received/archived by WeeWX we can follow that data through WeeWX.

Gary

[Bill Arthur]

Going through the log it looks like I may have restarted the box when I installed an app. I didn't think I did it that early in the day, but I may have. I'll take the blame.

Since then it's working flawlessly, the midnight total went away at 00:00.

I'll know more Monday when it rains.

Bill 

[Bill Arthur]

Gary,

I watched the results after the last rain. The midnight column went to 0.0 at midnight and the 24hr went down after 24 hours. Everything as expected.

It has now just started raining and all three columns are at 0.01.

I believe we can say that the problem has been resolved. Thank you for your quick and precise solution.

Bill

[Bill Arthur]

Gary,

Thanks again. Everything is working perfectly. 

BTW, I installed Weewx on a new RasPi.  "binding  = archive" works perfectly

Bill

[gjr80]

That is good news, the cwxn code is fairly basic and should have worked on your earlier system.

Gary

[David Chamberlin]

Sorry for bringing an old thread up here.  I just posted this to the related wxforum post:  https://www.wxforum.net/index.php?topic=38877

I'll copy-paste my post here:

Following this thread and some threads and references elsewhere, I just got my WX station working with direwolf and weewx running on an rpi, and I wanted to point out that there's a small issue with the cwxn.py implementation, at least for APRS use.   

At first I thought everything was going great - I was beaconing out the wxnow.txt info and looking at aprs.fi everything looked good.  But then I looked at the info in findu and noticed the dew point seemed to be messed up.  Looking at it more closely, that was because the humidity was wrong.  Instead of, for example, 92%, it was showing up as 9%.  So I took a closer look at the format of the data.  The humidity is being formatted as a 3 digit number by cwxn.  But the APRS spec and CWOP spec indicate it should be 2 digits, and 00 should be interpreted as 100%.  Apparently aprs.fi is being tolerant of the 3-digit number, but findu is not. 

I made a small change to cwxn.py to convert 100 to 0 and to format as %02d instead of %03d.  All looks good in both aprs.fi and findu now. 

BTW, in case it's relevant, I'm using the cwxn.py that was posted earlier in this thread by gjr80. 

My station:
https://tempestwx.com/station/28304
https://www.wunderground.com/dashboard/pws/KCAPACIF205
http://www.findu.com/cgi-bin/wxpage.cgi?call=AE6DC-5&last=240
https://aprs.fi/info/a/AE6DC-5

-Dave AE6DC

[gjr80]

Thanks, subsequent to the above thread and in concert with Bill I created the weewx-aprx extension which does include the correct humidity behaviour.

Gary

[David Chamberlin]

Gary,

Thanks!  I just took a look at the weewx-aprx extension.  FWIW, in addition to fixing the humidity, I also added solar radiation (luminosity).  One other thing to note which I'm likely going to try and fix shortly, and you might consider fixing in weewx-aprx, is that the nullproof probably shouldn't turn those values into 0.  To indicate "no value", I believe periods should be used - otherwise it's interpreted as a real value of 0. 

Here's the code I added for radiation:

# DWC adding solar radiation. If it's below 1000, use 'L'
# if it's above 1000, use 'l' and provide the value above 1000 

if data['radiation'] < 1000:
    fields.append("L%03d" % int(data['radiation']))
elif data['radiation'] < 2000:
    fields.apped("l%03d" % (int(data['radiation'])-1000))
else:
    loginf("radiation looks bad: %d" % int(data['radiation']))

-Dave

[tbu...@gmail.com]

Hello - 

Longtime Weewx user trying to get his station on APRS (RF).

I'm using Direwolf and, ideally, would just like to get a wxnow.txt file that it can pick up.  However, it appears that the original author of the cxwn extension may have pulled it down.

I see a reference to another plugin -  weewx-aprx  - but I'm unable to locate that as well.

Any assistance would be great!  Thanks!

[David Chamberlin]

After spending some time fixing cwxn to address the issues I found, I took a brief look at weewx-aprx.  I forget exactly why I decided not to go that route, I think it pertained to the way it was generating the packet which didn't entirely mesh with how I wanted it for direwolf.  So then I decided to leverage the CWOP support since it's basically the same format.  I enabled the CWOP support (so that it's sending directly to aprs-is) and I made a couple minor edits in restx.py to output the relevant bits of the packet to a wxnow.txt which direwolf can use.  Unfortunately I just did them as local edits - I didn't pull the weewx branch and make the changes via git.  And I basically just hard-coded the changes in.  What I should've done was add an option in the weewx.conf to write it out and then maybe submit a pull request to get that included in the weewx sources.  

Depending on your comfort level dealing with source code, if you want, I can send you my restx.py with my changes, but you'll want to compare with your restx.py in case it's not based on the same version.  

-Dave

[tbu...@gmail.com]

I do like the idea of using CWOP for the reasons you mentioned.  I'm comfortable around the Weewx ecosystem, Python, etc.  If you don't mind, send it along and I'll have a look.

No rush.  I still don't have a dedicated antenna for a packet station like I used to.  (Although I have a few things in mind)

Thanks.

[JD Burke]

I think I am in the same position...Weewx is faithfully reporting data from my weather station (Tempest) but I am having issues finding a wxnow.txt generator or way to get the data in weewx to an aprs message format.  I am comfortable with the weewx ecosystem and python as well and would value a peek at the restx.py code as well.