Is CWOP dying?

219 views
Skip to first unread message

Grant Wise

unread,
Apr 12, 2022, 8:36:10 PM4/12/22
to wxqc
Is CWOP dying? I've heard the admins in this group mention that the program has been on autopilot for some time... The website looks like it came from 2002, multiple links are broken, NOAA support is sparse, and there have been no added improvements for the 8 years that I have been a member... What's the deal?
Is there really so little interest from noaa in improving and expanding this project? The level of investment that this program receives really makes me wonder if NOAA actually values and uses our data as they claim. I know QC is a concern, but seriously, is our PWS data actually being used to drive mesoscale analysis, CAMs, etc, etc... 


While I'm complaining... :)
I believe this is my fifth or sixth time requesting site reactivation since I joined. This has to do with multiple factors; moving locations, hardware failures, hardware upgrades, etc, etc. I am hopeful that moving forward this improves. 
My point being, waiting for "re-activation" is a pain. It has taken multiple emails/weeks to get this done in the past. Why does the cwop database not automatically list a station as active as soon as data starts to be received? Also, I then have to re-send an email asking to be added back to the QC checks...and wait. Surely in 2022 we could automate some of these things? 


Thanks for hearing me out. I'm curious about everyone's take on the future of this program and what we can do to ensure its survival and continuous improvement. 

Regards,
Grant

Philip Gladstone

unread,
Apr 12, 2022, 9:22:53 PM4/12/22
to wxqc
The place to get support is at cwop-s...@noaa.gov . I will admit that the website is getting long in the tooth  and has not had as much love and care as it deserves. The sad truth is that the people who got this all off the ground are now 20 years older than they were.

The system has been run on a shoestring budget -- I never wanted to litter the pages with ads, so the revenue is zero. My website design skills were mediocre in 2000, and they haven't progressed significantly since then. 

If there is a volunteer who wants to spend some time and effort bringing the website up to scratch, then please contact me directly and we can see if we can work something out,

Philip

fama...@yahoo.com

unread,
Apr 12, 2022, 9:56:34 PM4/12/22
to wxqc, Grant Wise
This is coming from an operational met who also participated in CWOP…

But the QC is the biggest factor in it getting limited attention. The issues include:

Bad temp and wind data from poorly sighted stations

Bad dew point data from poor quality systems

Temp data is actually the least troublesome. The bad dew points are a bigger issue and have caused CWOP to lose credibility for mesoanalysis. It’s also not really something that we can solve… home weather station vendors just don’t put the effort in.

Wind data is a user problem because few stations have a 10 meter anemometer, which makes much CWOP wind data of little use meteorologically.

Just my $0.02 after having participated for over 10 years and having operationally forecast weather for nearly 20.
,
This is coming from an operational met who also participated in CWOP…

But the QC is the biggest factor in it getting limited attention. The issues include:

Bad temp and wind data from poorly sighted stations

Bad dew point data from poor quality systems

Temp data is actually the least troublesome. The bad dew points are a bigger issue and have caused CWOP to lose credibility for mesoanalysis. It’s also not really something that we can solve… home weather station vendors just don’t put the effort in.

Wind data is a user problem because few stations have a 10 meter anemometer, which makes much CWOP wind data of little use meteorologically.

Just my $0.02 after having participated for over 10 years and having operationally forecast weather for nearly 20.
This is coming from an operational met who also participated in CWOP…

But the QC is the biggest factor in it getting limited attention. The issues include:

Bad temp and wind data from poorly sighted stations

Bad dew point data from poor quality systems

Temp data is actually the least troublesome. The bad dew points are a bigger issue and have caused CWOP to lose credibility for mesoanalysis. It’s also not really something that we can solve… home weather station vendors just don’t put the effort in.

Wind data is a user problem because few stations have a 10 meter anemometer, which makes much CWOP wind data of little use meteorologically.

Just my $0.02 after having participated for over 10 years and having operationally forecast weather for nearly 20.
On Tuesday, April 12, 2022, 08:43:41 PM EDT, Grant Wise <grw...@gmail.com> wrote:


Is CWOP dying? I've heard the admins in this group mention that the program has been on autopilot for some time... The website looks like it came from 2002, multiple links are broken, NOAA support is sparse, and there have been no added improvements for the 8 years that I have been a member... What's the deal?
Is there really so little interest from noaa in improving and expanding this project? The level of investment that this program receives really makes me wonder if NOAA actually values and uses our data as they claim. I know QC is a concern, but seriously, is our PWS data actually being used to drive mesoscale analysis, CAMs, etc, etc... 


While I'm complaining... :)
I believe this is my fifth or sixth time requesting site reactivation since I joined. This has to do with multiple factors; moving locations, hardware failures, hardware upgrades, etc, etc. I am hopeful that moving forward this improves. 
My point being, waiting for "re-activation" is a pain. It has taken multiple emails/weeks to get this done in the past. Why does the cwop database not automatically list a station as active as soon as data starts to be received? Also, I then have to re-send an email asking to be added back to the QC checks...and wait. Surely in 2022 we could automate some of these things? 


Thanks for hearing me out. I'm curious about everyone's take on the future of this program and what we can do to ensure its survival and continuous improvement. 

Regards,
Grant

--
You received this message because you are subscribed to the Google Groups "wxqc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxqc+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wxqc/CAD_FuagsT0C4RpPs41tpgc%3DoM-XfwqH2zbSsvbVfsAWTmF6jmQ%40mail.gmail.com.
On Tuesday, April 12, 2022, 08:43:41 PM EDT, Grant Wise <grw...@gmail.com> wrote:


Is CWOP dying? I've heard the admins in this group mention that the program has been on autopilot for some time... The website looks like it came from 2002, multiple links are broken, NOAA support is sparse, and there have been no added improvements for the 8 years that I have been a member... What's the deal?
Is there really so little interest from noaa in improving and expanding this project? The level of investment that this program receives really makes me wonder if NOAA actually values and uses our data as they claim. I know QC is a concern, but seriously, is our PWS data actually being used to drive mesoscale analysis, CAMs, etc, etc... 


While I'm complaining... :)
I believe this is my fifth or sixth time requesting site reactivation since I joined. This has to do with multiple factors; moving locations, hardware failures, hardware upgrades, etc, etc. I am hopeful that moving forward this improves. 
My point being, waiting for "re-activation" is a pain. It has taken multiple emails/weeks to get this done in the past. Why does the cwop database not automatically list a station as active as soon as data starts to be received? Also, I then have to re-send an email asking to be added back to the QC checks...and wait. Surely in 2022 we could automate some of these things? 


Thanks for hearing me out. I'm curious about everyone's take on the future of this program and what we can do to ensure its survival and continuous improvement. 

Regards,
Grant

--
You received this message because you are subscribed to the Google Groups "wxqc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxqc+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wxqc/CAD_FuagsT0C4RpPs41tpgc%3DoM-XfwqH2zbSsvbVfsAWTmF6jmQ%40mail.gmail.com.

Coyote Boy

unread,
Apr 12, 2022, 11:26:58 PM4/12/22
to wx...@googlegroups.com

    My station KMELIVER4, a Davis Vantage Pro2 located for many years on Sanders Road, Livermore, is sited correctly and everything used to be OK on wind direction and speed. At some time in the past that suddenly changed to being horrible. Nothing at my station has ever changed and is working as it always did. My anemometer is 10 meters above ground and my temp/humidity sensor is 5' over a grassy area, etc. I have always used Cumulus software version 1.9.4 (1099) which I think is the reason for my erroneous wind data. That software can only send wind data in miles per hour and I suspect that somewhere along the line, something in CWOP has changed, possibly changed to kilometers per hour. I can't make that change and I think that is why my wind data suddenly went askew on CWOP. I would like to know.

To view this discussion on the web visit https://groups.google.com/d/msgid/wxqc/1051669015.77859.1649814967492%40mail.yahoo.com.
-- 

"Arguing that you don't care about privacy because 
you have nothing to hide, is no different than saying 
you don't care about free speech, because you have 
nothing to say." 

 - Edward Snowden, 2016

googl...@tedlum.com

unread,
Apr 13, 2022, 2:15:40 AM4/13/22
to wxqc
> Is CWOP dying?
Not that I'm aware of


> I've heard the admins in this group mention that the program has been on autopilot for some time...
I have not heard this, but autopilot sounds about right. Contributors submit data to NOAA, and that's been happening non-stop for over 20 years. What more is there?


> The website looks like it came from 2002, multiple links are broken,
That's because it is from about then, and was put together by non-web developers who volunteer their time to this non-revenue generating enterprise.

> NOAA support is sparse
CWOP isn't NOAA and NOAA isn't CWOP. You have to contact NOAA for NOAA support... and aside from putting your name in a file, what else is there?


> and there have been no added improvements for the 8 years that I have been a member... What's the deal?
I'm not sure that's true. The two servers at mesowest were added recently, but I can't recall if it was in the last 8 years... time flies :-)


> Is there really so little interest from noaa in improving and expanding this project?
It's not really a NOAA project. NOAA happily accepts data from various providers to their MAIDS system. MADIS currently has well over 100 providers and CWOP remains the single largest, and is continuing to expand.

> The level of investment that this program receives really makes me wonder if NOAA actually values and uses our data as they claim. I know QC is a concern, but seriously, is our PWS data actually being used to drive mesoscale analysis, CAMs, etc, etc...
The vast majority of the investment sits with the users themselves. What's the average cost of a station, $500? Multiply that by say, 15,500 currently active stations, and you have a total of $7.75 million in station hardware. Then you have the various individuals and universities who donate their time and resources just so there will be an ARPSWXNET to send data to. Then you have one man, Steve Dimse who's donation supported operation site at the center of everything, aggregating, archiving, and shipping 3.3 million observations per day over to MADIS.

The total NWS owned stations is probably just shy of 1,000, yet MADIS claims 66,000+ stations overall. While many of those are certainly duplicates coming in through multiple providers, NWS wouldn't have much coverage if not for all the other providers, and while ASOS reports hourly, other providers support much higher temporal frequencies. So the density in terms of location as well as data frequency comes mostly from 3rd party providers. It dosen't end at NWS either, as that's one of the sources AccuWeather, as well as many others, access and use.


> While I'm complaining... :)
> I believe this is my fifth or sixth time requesting site reactivation since I joined. This has to do with multiple factors; moving locations, hardware failures, hardware upgrades, etc, etc. I am hopeful that moving forward this improves.
> My point being, waiting for "re-activation" is a pain. It has taken multiple emails/weeks to get this done in the past. Why does the cwop database not automatically list a station as active as soon as data starts to be received?

The APRSWXNET mesonet does distribute and aggregate CWOP data at findu as soon as you start to transmit it. But those packets alone are not sufficient to build a users station record, which is required in order for findu to know to put it in the batch file that's sent to MADIS. On the other end, MADIS needs to know what it's receiving, otherwise it's just orphan data.

> Also, I then have to re-send an email asking to be added back to the QC checks...and wait. Surely in 2022 we could automate some of these things?
Are you offering to write and maintain the code? Willing to donate the AWS server instance to run it on?


> Thanks for hearing me out. I'm curious about everyone's take on the future of this program and what we can do to ensure its survival and continuous improvement.
Maybe do like those who have come before... make a personal donation of your time and money? That's how it got this far.

On Tuesday, April 12, 2022 at 8:36:10 PM UTC-4 grw...@gmail.com wrote:

tucso...@gmail.com

unread,
Apr 13, 2022, 10:38:51 AM4/13/22
to wxqc
Hello Grant 
If you are EW5634, it seems you are reporting a humidity of 100% for days now. Are you working to correct this???
Thanks,
James B
DW4536

Bob Hassell

unread,
Apr 13, 2022, 10:41:19 AM4/13/22
to Philip Gladstone, wxqc
I give you props for everything you have done and if I were a more skilled younger person with deep pockets, I would take you up on helping to maintain the current site. Many do not realize that even the "simple fixes" require digging thru lines of code that may take hours or days...I know!   First I have to remember (a real struggle) what I did...how...and why...just to get going!
Kudos Phillip for a website that has served us well and set the bar high for any successor. 

From: wx...@googlegroups.com <wx...@googlegroups.com> on behalf of Philip Gladstone <philip...@gladstonefamily.net>
Sent: Tuesday, April 12, 2022 8:22:53 PM
To: wxqc <wx...@googlegroups.com>
Subject: [wxqc] Re: Is CWOP dying?
 
--
You received this message because you are subscribed to the Google Groups "wxqc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxqc+uns...@googlegroups.com.

Lucy Hancock

unread,
Apr 13, 2022, 12:12:23 PM4/13/22
to Philip Gladstone, wxqc
For what it is worth, I think it is collecting the best big free database of weather.  I think Wunderground charges.  Everything is there in the CWOP archive.  And that is raw data, but we have presented at AMS a couple of times our technical reports on the database, which have detailed directions on preparation of a parsed database (Redshift).  And we actually have such a database, which anyone is invited to clone from AWS if they want to.

It was last updated in 2020 but that is because after that I got to work on the presentation just made at AMS in January.  

It is a fantastic database and afaik there is no other that is free that is equally good.  That is especially so in the solar radiation database, which has been collecting globally since Feb 2009.

Sometimes we are just quietly collecting but ... once the moment has gone by, there is no way to regather what was lost.... It is invaluable that we have this... Lively or not.  It is a huge gorgeous database.

--

James Ziebarth

unread,
Apr 13, 2022, 2:00:25 PM4/13/22
to Grant Wise, wx...@googlegroups.com
Somebody's paying attention to CWOP.  My weather station at my house got used for some data in this paper https://doi.org/10.1175/MWR-D-20-0412.1 after a macroburst hit our town almost two years ago, and that was because they found my station reporting data into CWOP right up until I took some pretty heavy damage and went off-line (for some reason, the pole and mounting pipe that I had my anemometer and two-way radio antenna mounted to didn't appreciate being there in 102+ MPH winds).  So, the front end may not seem like a lot is happening, but there are definitely folks out there looking at and using this data, so I keep feeding into it, and I'm trying to get my employer to set up a few more weather stations in our service territory that can also be used to feed into CWOP, in addition to helping us internally.

From: wx...@googlegroups.com <wx...@googlegroups.com> on behalf of Grant Wise <grw...@gmail.com>
Sent: Tuesday, April 12, 2022 6:35 PM
To: wxqc <wx...@googlegroups.com>
Subject: [wxqc] Is CWOP dying?
 
--
You received this message because you are subscribed to the Google Groups "wxqc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxqc+uns...@googlegroups.com.

Jim

unread,
Apr 13, 2022, 2:44:47 PM4/13/22
to Lucy Hancock, Philip Gladstone, wxqc

Well, I an thinking that no one is paying attention to the locations. I have done everything I can to change my location, following their instructions and still reporting at my old location. I just wish someone would update it. N8JE (AR391)

 

Jim, N8JE

 

Sent from Mail for Windows

Mike Dvorak

unread,
Apr 13, 2022, 2:47:33 PM4/13/22
to James Ziebarth, Grant Wise, wx...@googlegroups.com
I was inspired by the weather.gladstonefamily.net website to create a real-time weather verification app. Assuming your station passed MADIS quality control, you can see how your station compares to the GFS, NAM, and GEM forecast at https://app.weathertactics.io/

Here's a sample of NAM forecast verification from central California:
image.png
I'd be interested in getting involved in a new QC effort for CWOP. It's an incredibly unique dataset.


To view this discussion on the web visit https://groups.google.com/d/msgid/wxqc/DS7PR19MB4504B5D0172BB39C0317BBC0B7EC9%40DS7PR19MB4504.namprd19.prod.outlook.com.

tucso...@gmail.com

unread,
Apr 13, 2022, 3:43:53 PM4/13/22
to wxqc
WOW, great example!!!
Thanks fir posting,
James B
DW4536

bhutch...@gmail.com

unread,
Apr 14, 2022, 11:02:16 AM4/14/22
to James Ziebarth, Grant Wise, wx...@googlegroups.com

Seeing no problems here. My station has been showing up on the NWS observations page for years along with showing on findu.com.

I was the first station to begin reporting local WX via amateur packet radio back in around 1984.

 

Bill KB6CYS

AR-144

Message has been deleted

Ted Lum

unread,
Apr 15, 2022, 11:17:30 AM4/15/22
to wxqc
Jim:

Can you be more specific where you see the wrong location please. Location data comes from different places depending on where you're accessing it. Then, it would also be helpful to know which instruction you followed so we can make sure you have the right instructions for the values that need to change.
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Roy Lamberton Gmail

unread,
Apr 15, 2022, 11:17:31 AM4/15/22
to wx...@googlegroups.com
Since there is no "official" temperature station in our town, how do you determine an error in temperature?

rsl

Jim Bayer

unread,
Apr 15, 2022, 12:26:50 PM4/15/22
to wxqc
I have given up on trying to get everything to pass QC. Humidity is always the biggy. There are some other stations in the area but they are in the high prairie which is mostly open land. My location is midway down the north facing slope of a deep canyon and is heavily forested. I would need a 60' tower to get above the trees and it would still not be comparable as the air movement, when there is any,  tends to swirl in the canyon and of course the humidity is higher due to the trees. But it works for us in our little micro climate. The station was rebuilt by Davis and calibrated two years ago so it is what it is.

FW9819

Jim

unread,
Apr 16, 2022, 10:15:41 AM4/16/22
to Paul G, wxqc

I have attached a screen shot of what I am seeing from CWOP

 

Jim, N8JE

 

Sent from Mail for Windows

 

From: Paul G
Sent: Friday, April 15, 2022 11:18 AM
To: wxqc
Subject: [wxqc] Re: Is CWOP dying?

 

 

Just to be clear -- are you talking about https://weather.gladstonefamily.net, http://www.wxqa.com, http://www.findu.com or something else? I think this 'Group' is primarily about weather.gladstonefamily.net but I've been wrong before.

 

On Tuesday, April 12, 2022 at 8:36:10 PM UTC-4 Grant wrote:

Is CWOP dying? I've heard the admins in this group mention that the program has been on autopilot for some time... The website looks like it came from 2002, multiple links are broken, NOAA support is sparse, and there have been no added improvements for the 8 years that I have been a member

 

--

You received this message because you are subscribed to the Google Groups "wxqc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxqc+uns...@googlegroups.com.

Location.PNG

Paul G

unread,
Apr 16, 2022, 11:37:00 AM4/16/22
to wxqc
The service provided by Philip (weather.gladstonefamily.net) is related to but is not CWOP (wxqa.com). Although the location error that shows at weather.gladstonefamily.net suggests moving the pointer and clicking, both CWOP and Philip (in the second post) say send an email to cwop-s...@noaa.gov. Note that there are 725 sites with notable location errors which is much better than the 1,251 sites with elevation error. I don't know if MADIS incorporates position errors in any products so perhaps it's irrelevant. Certainly the procedure (as I understand it) for "moving" a station suggests getting a new ID so the historical data associated with the previous ID remains "valid".

Jim

unread,
Apr 16, 2022, 12:15:12 PM4/16/22
to Paul G, wxqc

OK, well I have done what it 3 times and nothing. This now explains everything. I guess I got confused. I have been reporting weather to CWOP for many years and just had problems with it sense my move back in July.

 

Jim, N8JE

 

Sent from Mail for Windows

 

--

You received this message because you are subscribed to the Google Groups "wxqc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxqc+uns...@googlegroups.com.

googl...@tedlum.com

unread,
Apr 16, 2022, 1:06:11 PM4/16/22
to wxqc
The "Registered Location" is the coordinates that you registered the station as when you applied for, or updated, the CWOP MADIS ID. To change that you should contact cwop-s...@noaa.gov.

Now the red location error comes from a discrepancy between the coordinates in the "Registered Location" and the coordinates being transmitted in the weather packets by the station. All wxqa can tell you is there is a discrepancy between the two. You'll need to figure out which is incorrect and adjust it to agree with the correct coordinates.

Paul G

unread,
Apr 18, 2022, 1:22:56 PM4/18/22
to wxqc
If you want to do your own QC presentation:

mrof...@gmail.com

unread,
May 3, 2022, 8:57:53 PM5/3/22
to wxqc
Couldn't agree more. I've seen data from my station show up in HGX local information statements numerous times for rainfall, wind, and temperature. Although not identified by call sign, I can verify it's my station based on the general location provided combined with the data value and corresponding time of the event. Trust me y'all, the NWS is using your data!

I sincerely appreciate the efforts of Ted, Phillip and others over the years. Maybe some young whippersnapper will step up and volunteer some time like they did. 

Grant Wise

unread,
May 3, 2022, 9:36:23 PM5/3/22
to mrof...@gmail.com, wxqc
Thanks to everyone for the thoughts, corrections, and history on this project. I echo the statements of many, thank you to Philip and Ted for all you have done to manage and develop this program. I certainly did not realize just how much a private partnership this was, even on the maintenance and creation side of things. I had assumed that CWOP was a wholly NOAA run and maintained program. 

It is also certainly encouraging to hear the reports of our data being used in research, etc! 


(James, yes my hunidity sensor had been down for a few days. Just got a much better one installed! )


Regards,
Grant


Reply all
Reply to author
Forward
0 new messages