extraTemp1 doesn't allow Corrections?
56 views
Skip to first unread message
Jens-Jørgen Kjærgaard
May 6, 2018, 12:11:41 AM5/6/18
to weewx-user
I have for some time benn running an extraTemp1 (lake temp) in weewx. However, it is running a few centigrades high, and I would like to coorect it, but the corrections doesn't show in the output:
windSpeed = windSpeed * 1.5
windGust = windGust * 1.5
rain = rain * 1.5
extraTemp1 = extraTemp1 * 0.8
- it works fine for the other parameters, but not extraTemp1. I also tried extraTemp1 = extraTemp1 - 3.0, same negative result.
?
gjr80
May 6, 2018, 12:44:00 AM5/6/18
to weewx-user
Hi,
extraTemp1 should correct just like any other obs. One possible reason it is not working is that service StdCalibrate runs before extraTemp1 is added to your record/packets. How does weeWX receive extraTemp1, is it via a driver, a service or some other means? Once we know that your [Engine] [[Services]] stanza in weewx.conf will show us whether this is the problem. You might want to post your [Engine] [[Services]] stanza.
Gary
Jens-Jørgen Kjærgaard
May 6, 2018, 1:16:48 AM5/6/18
to weewx-user
My [Engine] [[Services]]:
prep_services = weewx.engine.StdTimeSynch
data_services = user.pond.PondService,
process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate
archive_services = weewx.engine.StdArchive
restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS
report_services = weewx.engine.StdPrint, weewx.engine.StdReport
Should I change this?
gjr80
May 6, 2018, 1:24:06 AM5/6/18
to weewx-user
I am guessing that user.pond.PondService is what is adding extraTemp1 to your data? If so there is no need to change anything, all would appear in the right order.
What diagnostics have you done to show it is not working, are you sure you are not looking at the already corrected value?
Gary
Jens-Jørgen Kjærgaard
May 6, 2018, 1:29:30 AM5/6/18
to weewx-user
Yes, user.pond.Service is adding extraTemp1.
Even if I change the correction to an extreme, say extraTemp1 = extraTemp1 - 10.0, the output remains the same.
--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/YVAkCwsY2S0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
gjr80
May 6, 2018, 1:52:34 AM5/6/18
to weewx-user
Yes that should cause a noticeable change. Just to check another obvious one, you have restarted weeWX or done a config reload after making a change to weewx.conf?
Gary
Andrew Milner
May 6, 2018, 2:21:11 AM5/6/18
to weewx-user
Just some thoughts....
a) Is pondtemp bound to loop or rec??
b) Gary - is the correction applied to loop or rec packets? (It cannot be to both!!) I suspect to loop, and I suspect pondtemp is bound to rec.
just my suspicions ….
Jens-Jørgen Kjærgaard
May 6, 2018, 2:29:49 AM5/6/18
to weewx-user
gjr80: yes, I have restarted several times.
Andrew: I don't know about loop or rec?
--
gjr80
May 6, 2018, 2:31:59 AM5/6/18
to weewx-user
The correction is always applied to loop and only to archive when using hardware record generation (otherwise you would correct twice). If the OP is using hardware record generation and the pond service is loop based, then yes that could cause the problem, but if that was the case I would not expect to see any extraTemp1 data in the archive record.
Jens-Jørgen. It might be worth posting your entire weewx.conf (with any sensitive info such as passwords/user names removed). Also worth seeing the file user/pond.py so we can see exactly what is being done by that service.
Gary
PS. There were some changes to the codebase not too long back to extract whatever useful info could be extracted from the accumulators, that could be doing something here, maybe not, would need to have a look at the code. Tom would know off the top of his head I am sure.
Jens-Jørgen. It might be worth posting your entire weewx.conf (with any sensitive info such as passwords/user names removed). Also worth seeing the file user/pond.py so we can see exactly what is being done by that service.
Gary
PS. There were some changes to the codebase not too long back to extract whatever useful info could be extracted from the accumulators, that could be doing something here, maybe not, would need to have a look at the code. Tom would know off the top of his head I am sure.
Jens-Jørgen Kjærgaard
May 6, 2018, 2:40:32 AM5/6/18
to weewx-user
Andrew Milner
May 6, 2018, 2:46:34 AM5/6/18
to weewx-user
we need to see user/pond/pondservice.py also rather than soetemp.py
Here they are:
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
Andrew Milner
May 6, 2018, 2:48:45 AM5/6/18
to weewx-user
gary - the .conf says software generation and I suspect the pondservice will be rec bound - hence the OP's problem.
gjr80
May 6, 2018, 2:48:57 AM5/6/18
to weewx-user
I think you mean user/pond.py, PondService should be a class in user/pond.py.
Gary
Andrew Milner
May 6, 2018, 2:50:47 AM5/6/18
to weewx-user
yes I did indeed!! sorry - user/pond.py we need to see
Jens-Jørgen Kjærgaard
May 6, 2018, 3:19:17 AM5/6/18
to weewx-user
These hasn't been modified by me, though:
--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/YVAkCwsY2S0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.
Andrew Milner
May 6, 2018, 3:29:13 AM5/6/18
to weewx-user
In pond.py
change
self.bind(weewx.NEW_ARCHIVE_RECORD, self.read_file)to
self.bind(weewx.NEW_LOOP_PACKET, self.read_file)
and
event.record['extraTemp1'] = float(value)
to
event.packet['extraTemp1'] = float(value)
which will insert the pond temp into every loop packet and thence to the record when generated
These hasn't been modified by me, though:
2018-05-06 8:50 GMT+02:00 Andrew Milner <andrew.s...@gmail.com>:
yes I did indeed!! sorry - user/pond.py we need to see
On Sunday, 6 May 2018 09:48:57 UTC+3, gjr80 wrote:I think you mean user/pond.py, PondService should be a class in user/pond.py.Gary
--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/YVAkCwsY2S0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
Jens-Jørgen Kjærgaard
May 6, 2018, 5:08:48 AM5/6/18
to weewx-user
Did that - ufortunately, my lake temperature (extraTemp1) now shows up as N/A ?
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.
Andrew Milner
May 6, 2018, 5:51:41 AM5/6/18
to weewx-user
Gary - I'm lost now. Was my change not correct?? - or do you also have to add the field into the rec as well as the loop
Jens - what do you see in LOOP and REC data if you run weewx directly from command line, rather than as a daemon?
Jens-Jørgen Kjærgaard
May 6, 2018, 6:01:31 AM5/6/18
to weewx-user
I think some information may be in this from syslog:
May 6 11:44:48 raspberrypi weewx[2073]: pond: found value of 52.1
May 6 11:44:48 raspberrypi weewx[2073]: engine: 2018-05-06 11:44:48 CEST (1525599888) LOOP value 'extraTemp1' 100.624 outside limits (30.0, 80.0)
May 6 11:45:48 raspberrypi weewx[2073]: pond: found value of 52.1
May 6 11:45:48 raspberrypi weewx[2073]: engine: 2018-05-06 11:45:49 CEST (1525599949) LOOP value 'extraTemp1' 100.624 outside limits (30.0, 80.0)
May 6 11:46:49 raspberrypi weewx[2073]: pond: found value of 52.1
May 6 11:46:49 raspberrypi weewx[2073]: engine: 2018-05-06 11:46:49 CEST (1525600009) LOOP value 'extraTemp1' 100.624 outside limits (30.0, 80.0)
The value of 52.1 Fahrenheit is probably the read value from the sensor, 11.1 centigrades, but neither is outside my limits of 30 and 80 (and my correction of * 0.8 not applied either). Some C/F confusion, but that does not seem to be the full explanation?
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.
Andrew Milner
May 6, 2018, 6:34:06 AM5/6/18
to weewx-user
I'm pretty sure that is the reason, but not sure how to correct it!!
the 52.1 is obviously a farenheit reading in the file - but what is actually in /var/tmp/pond.txt?? Is there a newline or other data on the end??
I'm not too sure where the 100.624 is coming from at the moment
Message has been deleted
Andrew Milner
May 6, 2018, 7:29:06 AM5/6/18
to weewx-user
you could also try and change the read() in pond.py to readline() - would really like to know what is actually in the file / the format of the file that you are reading though!!
in fact even the 52.1 is suspect - since 11.1 converts to 51.2 not 52.1!!
Jens-Jørgen Kjærgaard
May 6, 2018, 7:53:12 AM5/6/18
to weewx-user
/var/tmp/pond.txt is a simple txt file with 52.1 - nothing else. I tried the change in pond.py from read() to readline (). Will come back with result.
Omitting my limits of 30,80 gives 38.4 in the output - corresponding to 100 F. So where the 52.1 goes or comes from, I don't know.
Jens-Jørgen Kjærgaard
www.jensjk.dk
www.jensjk.dk
gjr80
May 6, 2018, 9:38:30 AM5/6/18
to weewx-user
Herein lies the problem of augmenting a loop packet or archive record when it is done 'on the cheap'. Each packet or record has a field, usUnits, that indicates the unit system used in the packet/record; in simple terms it tells us whether the temperature fields are F or C. The unit system used in a packet/record can vary for a number of reasons. The safe way to add obs to a packet/record is to check the usUnits field of the packet/record you are augmenting and make sure the obs you are adding to the packet/record is in the correct units. The cheap way to add an obs is to blindly add a field to the packet/record. This will work fine if the packet/record unit system happens to use the same units as the data you are adding. If this is not the case you tend to get nonsense data somewhere along the line as the data you added will be assumed to be in one particular unit when it is in fact in another unit and when a conversion occurs (and chances are a unit conversion will occur somewhere) it will be wrong.
In this case your service is reading a temperature of 52.1 (I assume F) from a text file and adding it to the archive record. The fousb driver emits a metric packet, so the temperatures therein are in C, and extraTemp1 is added as 52.1 and since it was added to a metric record it is now considered to be in C. Later, when the record is converted (in service StdConvert) to the unit system used by the database, in your case this is US customary which uses F for temperature, 52.1 is 'converted' to F. This results in extraTemp1 becoming 125.8F (52.1C converted to F). Now the StdCalibrate service comes along and applies any calibrations, in this case extraTemp1 is reduced by 20%, so extraTemp1 becomes 100.6. Now the QC limits are applied by service StdQC, and this is where you get the message about extraTemp1 breaking its limit.
So how to fix, the elegant way is to recode pond.py to take note of the units of the record being augmented, converting the text file value accordingly then adding it to the record. The other approach is to either modify the code that writes the text file to write a value in C or modify pond.py to convert the text file temperature to C before adding it to the record. The first approach will be able to handle future changes (to code or hardware) but requires some python skills and weeWX knowledge. The latter approach is simpler and may not work in the future (perhaps changes to the weeWX code or you get a new station) but only requires some basic python knowledge and no understanding of weeWX.
Gary
In this case your service is reading a temperature of 52.1 (I assume F) from a text file and adding it to the archive record. The fousb driver emits a metric packet, so the temperatures therein are in C, and extraTemp1 is added as 52.1 and since it was added to a metric record it is now considered to be in C. Later, when the record is converted (in service StdConvert) to the unit system used by the database, in your case this is US customary which uses F for temperature, 52.1 is 'converted' to F. This results in extraTemp1 becoming 125.8F (52.1C converted to F). Now the StdCalibrate service comes along and applies any calibrations, in this case extraTemp1 is reduced by 20%, so extraTemp1 becomes 100.6. Now the QC limits are applied by service StdQC, and this is where you get the message about extraTemp1 breaking its limit.
So how to fix, the elegant way is to recode pond.py to take note of the units of the record being augmented, converting the text file value accordingly then adding it to the record. The other approach is to either modify the code that writes the text file to write a value in C or modify pond.py to convert the text file temperature to C before adding it to the record. The first approach will be able to handle future changes (to code or hardware) but requires some python skills and weeWX knowledge. The latter approach is simpler and may not work in the future (perhaps changes to the weeWX code or you get a new station) but only requires some basic python knowledge and no understanding of weeWX.
Gary
Jens-Jørgen Kjærgaard
May 6, 2018, 10:04:09 AM5/6/18
to weewx-user
Thank you for a elaborate and understandable answer.
However, I think I am beyond using either solution - as my knowledge of python is a little less than basic :).
For your info, I add my present weewx.conf, soetemp.py and pond.py
Gary
--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/YVAkCwsY2S0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.
Andrew Milner
May 6, 2018, 10:53:04 AM5/6/18
to weewx-user
try the attached version (after renaming it of course!!)
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
Jens-Jørgen Kjærgaard
May 7, 2018, 12:15:36 AM5/7/18
to weewx-user
It works!
After running your pond.py for some hours, it works perfectly, the corrections also!Thank you for helping out in this to an amateur in python.
But I will have to use your pond.py after a future upgrade of weewx - or will it be included?
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.
Andrew Milner
May 7, 2018, 12:29:32 AM5/7/18
to weewx...@googlegroups.com
I do not know where you got the original pond.py from - but as long as it is in the user directory weewx should not change it during any upgrades. To be sure keep a copy in your personal area outside the weewx directory structure just in case!!
I took the nasty option route - so it assumes the file contains a farenheit temperature and that you have a metric producing station (like FineOffset), so changes the temp in the file to Celsius before putting it in the loop packet. IF in the future your file is generated in Celsius and you still have a metric producing station then use the previous version of pond.py. IF in the future you change to a weather station which produces Imperial output use the previous version of pond.py because in both those situations no conversion is needed!!.
Reply all
Reply to author
Forward
0 new messages