Avoid bad data with StdQC

[Alberto Sánchez]

Sometimes, in my Oregon WMR300, there are more than 8000mm rains.It is a mistake of the station, because the error also appears in the console.

I have cleaned those records from the database but I would like to avoid them by software. I read that by using http://weewx.com/docs/usersguide.htm#StdQC I could do it, but I do not know what parameters I would have to put, rain_rate?

An example in wunderground (https://www.wunderground.com/personal-weather-station/dashboard?ID=IVILLAVA2.June 13, 2018 ) because from the database I have deleted the errors.

Thanks

[Thomas Keffer]

Because the WMR300 supplies rainRate in hardware, you will need to supply limits for both rain and rainRate. It's up to you to pick sensible values, but something like this should work. 

[StdQC]

  rain = 0, 10, mm

  rainRate = 0, 100.0, mm_per_hour

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Hathaway]

I have found that I have many variables outside the StdQC values.  For example, my range for dewpoint is 5 to 95, but there are 897 rows outside this range.  The minimum is -18!

So, I have a couple questions:

• Does case count in the StdQC fields?  I have dewpoint but the variable is dewPoint.

    [[MinMax]]

        barometer = 26, 32.5, inHg

        outTemp = 0, 120, degree_F

        inTemp = 10, 120, degree_F

        outHumidity = 0, 100

        inHumidity = 0, 100

        windSpeed = 0, 120, mile_per_hour

        rain = 0, 120, inch

        dewpoint = 5, 95, degree_F

        windDir = 0, 360

        pressure = 24, 34.5, inHg

• What can I do to fix it?  I can obviously do a table UPDATE, (archive and archive_day_dewpoint), but is there a utility I have been ignoring?

Dave

[gjr80]

Hi,

Yes case counts when using variable/field names (that is what the names on the left hand side of the equal sign are). The WeeWX dewpoint field name is dewpoint, no spaces all lower case. What do you mean the 'variable is dewPoint', if something has added dewPoint then that will at the very least cause confusion and quite possibly errors. That being said, if you have a field ABCDEFG in your loop packet/archive record you can use StdQC to quality control it, just add an entry for ABCDEFG. A couple of caveats though. The field must be in the loop packet or archive record as that is all that StdQC works on, if it is just a variable somewhere in your code then you are out of luck. Secondly, if the service that creates/adds your field to the loop packet or archive record appears after StdQC in the services list in weewx.conf then StdQC will not correct it as StdQC has already done its job. That is why it is best to put services that add data to the loop packet/archive record in the data_services line under [Engine] [[Services]] in weewx.conf.

Gary

[David Hathaway]

Hi Gary,

I don't know where I saw dewPoint as a variable, but looking at my archive table, but it is dewpoint.

If I use an update query to modify the out-of-bounds dewpoint values, should I set it to the limit or NULL? I see 897 values outside the range, but 22454 that are NULL.

Dave

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/JaHGiL2mYMQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[gjr80]

Dave,

Within WeeWX a SQL NULL or python None value is used to represent an observation where there is no data or data is otherwise non-existent for that observation. So seeing NULLs in your archive is not necessarily a bad thing, just means that valid data was not available for that observation at that time. The way that StdQC works is that if it detects a value outside of its limits for the observation concerned, the value for that observation is set to None. This translates to a NULL in the database. So the correct operation for your update query would be to replace the high/low values with NULL.

Interesting you have values outside the range, they really should be cleaned up by StdQC, is this an ongoing occurrence or is it behaving itself now?

Gary

[David Hathaway]

Gary,

Thanks for your reply and guidance to use NULL for out-of-limit values.

The latest occurrence of out-of-limit dewpoint is 2016-04-08.  The span of time with out-of-limit dewpoints was 2015-01-18 to 2016-04-08, in case there is some versioning question.

Dave

[gjr80]

Ok, given the most recent anomaly is from 2016 I guess we can give it the benefit of the doubt for now. Would be more concerned if it was happening here and now. Probably not worth the effort of trying to track down what went wrong, hard enough keeping on top of current issues, let alone old ones :)

Gary