Solar Radiation > Theoretical Max

[t...@tom.org]

Looks like I might have unlocked some secret of the universe because my solar radiation numbers are regularly above the "theoretical max" as shown in the Belchertown skin.

I see this on two different stations in two different places, so I doubt it is my station.

Does anyone know how to adjust this max or have another recommendation?

Or have I unlocked the secrets of the universe?

Thanks in advance.

[t...@tom.org]

By the way:

http://www.tom.org/

http://vtwx.tom.org/

[vince]

On Saturday, December 19, 2020 at 10:18:43 AM UTC-8 t...@tom.org wrote:

Looks like I might have unlocked some secret of the universe because my solar radiation numbers are regularly above the "theoretical max" as shown in the Belchertown skin.

Many stations emit readings like this (WeatherFlow to name one).

I guess I'd suggest making sure you have pyephem installed and that your lat/lon in weewx.conf is correct.

 

[Greg Troxel]

"t...@tom.org" <t...@tom.org> writes:

> Looks like I might have unlocked some secret of the universe because my
> solar radiation numbers are regularly above the "theoretical max" as shown
> in the Belchertown skin.

The theoretical max is for sun on a completely clear day.

When there are clouds and the sun goes in and out from behind them, I
see values over theoretical max. But I never see that when it's clear.

> I see this on two different stations in two different places, so I doubt it
> is my station.
>
> Does anyone know how to adjust this max or have another recommendation?

Don't adjust: I suspect it is right. My recommendation is to accept
that with a cloud edge as a lens, there is a temporary focusing.

Now, actually looking at your data, things are not making sense. Are
you graphing the max radiation in a 5 min archive interval, or the
average? Your graphs look peaky and irregular and when looking at
average (non-Belchertown) I tend to see a nice approximation of a sine
wave, on clear-sky days.

Today my measured high radiation was 364 W/m^2 (but I'm down south, at
about 42.5).

527 just doesn't seem right. I see values like that at times, but
mostly around 250 these days.

Also cross check your console vs weewx.

> Or have I unlocked the secrets of the universe?

I don't see hwo to turn this into a perpetual motion machine.

But you have noticed something that very few people understand. Maybe
not even me :-)

[t...@tom.org]

Thanks for the thoughtful responses. Inlined..

On Saturday, December 19, 2020 at 2:37:10 PM UTC-5 Greg Troxel wrote:

"t...@tom.org" <t...@tom.org> writes:

> Looks like I might have unlocked some secret of the universe because my
> solar radiation numbers are regularly above the "theoretical max" as shown
> in the Belchertown skin.

The theoretical max is for sun on a completely clear day.

When there are clouds and the sun goes in and out from behind them, I
see values over theoretical max. But I never see that when it's clear.

I see it under both circumstances.

 

> I see this on two different stations in two different places, so I doubt it
> is my station.
>
> Does anyone know how to adjust this max or have another recommendation?

Don't adjust: I suspect it is right. My recommendation is to accept
that with a cloud edge as a lens, there is a temporary focusing.

Now, actually looking at your data, things are not making sense. Are
you graphing the max radiation in a 5 min archive interval, or the
average? Your graphs look peaky and irregular and when looking at
average (non-Belchertown) I tend to see a nice approximation of a sine
wave, on clear-sky days.

The weewx graph and Belchertown are the same. I am graphing the radiation field against the maxSolarRad. My archive interval is 5 mins, and I am not sure whether weewx stores the average or the high from the logger. I assume the high, but I assume that for no reason.

Today my measured high radiation was 364 W/m^2 (but I'm down south, at
about 42.5).  

527 just doesn't seem right. I see values like that at times, but
mostly around 250 these days.

Where are you seeing 527? I think I got to 461 in MA (42.5) and 332 in VT at 44.49 but in VT, it was cloudier mid-day so I didn't get a true max.

Also cross check your console vs weewx.

Good idea. I'll do that tomorrow. 

> Or have I unlocked the secrets of the universe?

I don't see hwo to turn this into a perpetual motion machine.

Read my mind. Anything outside the laws of physics is promising!

[t...@tom.org]

Double-checked on both counts.

[t...@tom.org]

OK, update on this..

I made sure the consoles and weewx had the exact same lat/long. They were very close but not exact.

I'll just use one station for this today, which is the one in MA at 42.5 latitude.

It is completely overcast today, at least so far.

I ran this query against my archive table:

SELECT datetime, convert_tz(from_unixtime(datetime),'GMT', 'US/Eastern') as Date, radiation, round(maxSolarRad,0) as maxRad, round(radiation - maxSolarRad, 0) as delta FROM weewx_ma.archive where radiation > 0 order by datetime desc limit 288;

Here are the results as of now:

'1608471000','2020-12-20 08:30:00','38','82','-44'

'1608470700','2020-12-20 08:25:00','39','72','-33'

'1608470400','2020-12-20 08:20:00','47','63','-16'

'1608470100','2020-12-20 08:15:00','44','53','-9'

'1608469800','2020-12-20 08:10:00','38','44','-6'

'1608469500','2020-12-20 08:05:00','32','36','-4'

'1608469200','2020-12-20 08:00:00','29','28','1'

'1608468900','2020-12-20 07:55:00','26','21','5'

'1608468600','2020-12-20 07:50:00','26','15','11'

'1608468300','2020-12-20 07:45:00','20','10','10'

'1608468000','2020-12-20 07:40:00','16','6','10'

'1608467700','2020-12-20 07:35:00','12','3','9'

'1608467400','2020-12-20 07:30:00','10','1','9'

'1608467100','2020-12-20 07:25:00','5','0','5'

'1608466800','2020-12-20 07:20:00','4','0','4'

So I am having this issue on a cloudy day with lat and long aligned.

Is anyone else seeing this in their environment??

[Cameron D]

The main cause of higher than expected insolation is scattered light being added to direct light.

Thin clouds are a good source of scattered light, but if you have nearby structures, such as walls, above the sensor then they can also contribute.

Your very high ratio early morning is almost certainly indirect light, and it might be that the "theoretical" calculation only gives a direct light value (I haven't looked at the code).

Finally, theoretical calculations still have to take into account atmospheric absorption by minor gases and particles that are not known locally. If the calculations assume a moderate level of pollution but you have very clear air then you will see higher than expected insolation, but your readings look too high to be explained by just that.

[Greg Troxel]

I had a thought about this last night.

There are two entirely separable questions. One is whether the
maxSolarRad value is correct, and the other is what your sensor is
measuring. I think the "is the max correct" can be answered (and should
be) without considering your data at all.

First, I think we're talking about radiation incident on a horizontal
surface. I didn't level my sensor carefully, but it's within a few
degrees.


I see in the defautl config file that maxSolarRad is set to
prefer_hardware, but I am guessing VP2 does not actually output that.


For theory, there seem to be multiple approaches, and notions include
not only the in-space radiation of 1353 W/m^2, but some contribution
from diffusion as well as air-mass attenuation and pollution (but
northeast air quality is pretty good)

https://en.wikipedia.org/wiki/Air_mass_(solar_energy)

{The rest of the specific numbers I mention in this post assume 42.5
latitude, which is more or less where Tom and I are.)

At 42.5 degrees and the winter solstice, the sun elevation is 20. In
the summer, it's 65. So that's 70 and 25 in terms of zenith angle.

Interpolating by guessing from the table, that leads to

1017 W/m^w summer solstice
710 W/m^2 winter solstice

but as I understand it, that is radiation at the arrival angle, not
radiation horizontally. It just does not make sense to me that there
would be 710 W/m^2 on horizontal panel when the sun is 20 degrees up -
but my intuition has not been working well so far on this subject.

Tom's graph shows 461 for yesterday's peak theoretical
and I'd expect 243 horizontal for 710 at 20 degrees. Except that the
diffuse component would not be reduced, just the direct, from the angle.

For 1017 incident , basically discounting air mass and keeping the 10%
gain, I'd expect 347.

So I have no idea where that 461 is coming from.

Looking back near the solstice at a cloud-free day (June 17), I see a nice
sine-ish curve over the day peaking at about 941 W/m^2.
If I take 1017 and multiple by sin(65deg) I get 922.


So I guess the big point is that this is much harder than I thought it
was going to be when I started looking it up!



This may be useful, but it seems to be about daily energy
https://naldc.nal.usda.gov/download/34980/PDF

This might have some data from calibrated instruments:
https://www.ncdc.noaa.gov/data-access/land-based-station-data/land-based-datasets/solar-radiation

[Greg Troxel]

Can someone share how to add maxSolarRad (when it is in the db) to the
graphs for the traditional skin? Can I graph radiation, max (observed),
and theory all at once, having three?

I think the answer is the same for seasons and traditional.

I only just today updated to 4.2.0 (well, git master from this morning),
and it is running fine (NetBSD 9, RPI3, python 2.7). I also changed my
db to the extended schema following the really great documentation at:
https://github.com/weewx/weewx/wiki/Switching-to-the-new-wview_extended-schema

[Greg Troxel]

Greg Troxel <g...@lexort.com> writes:

> Can someone share how to add maxSolarRad (when it is in the db) to the
> graphs for the traditional skin? Can I graph radiation, max (observed),
> and theory all at once, having three?

The answer is to just add it and label it; it comes out in green after
radiation in blue and max in red. Pro Tip: add it after radiation_max,
which is the max of local observations, and don't stick the line after
the radiation_max header and the 4 lines defining how max should be.


[[[dayradiation]]]
[[[[radiation]]]]
[[[[radiation_max]]]]
data_type = radiation
aggregate_type = max
aggregate_interval = 3600
label = max
[[[[maxSolarRad]]]]
label = theory

[Karen K]

I found that thread interesting, so I added the column "maxSolarRad", too. 

Readings of the console and the WeatherLinkLive device are quite the same. And the readings of "radiation" are higher than "maxSolarRad". The values I upload to the local weather network are well in the range of other stations nearby.

[t...@tom.org]

kk, glad you corroborated my observations. I am no expert in this for sure. I am just tired of all of the comments I get from visitors to the website about how my readings exceed theoretical max. I could remove the max, but that doesn't seem fun.

I do not have the expertise to validate the way weewx calculates it nor am I even competent in Python, but for those who may, here is a link to the code:

https://github.com/weewx/weewx/blob/d91635f3bc429f906d1f084c6a6bc8ee09fa1a27/bin/weewx/wxformulas.py#L332

[Tom Keffer]

Well, if someone has a better source for calculating maximum solar radiation, that matches the radiation spectrum of popular sensors, I'm all ears.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8b44e185-9f77-4cfd-b024-a7df7f79b617n%40googlegroups.com.

[Karen K]

First, I see 2 different formulas in WeeWX. Additionally there is a factor that can be between 0.7 and 0.91. It may be an idea to adjust those values. (But in the moment I found no place to put "atc" and "maxSolarRad_algo" into weewx.conf.

[Tom Keffer]

Selection of the algorithm is in there --- just not documented. (I often do that with configurations that are not quite settled.)

Here's the default. It will be documented in V4.3.

[StdWXCalculate]
  [[WXXTypes]]
    [[[maxSolarRad]]]
      algorithm = rs
      atc = 0.8
      nfac = 2

Adjust as you please.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/bddae2eb-83d0-48f6-a4ae-7b996a49fe45n%40googlegroups.com.

[Karen K]

http://www.michler-fischer.privat.t-online.de/Bulletin68_83_94.pdf

(from Switzerland, unfortunately in german)

[moth...@gmail.com]

These are the mine measurements in the Netherlands today 21-12-2020.

Ton

Op maandag 21 december 2020 om 20:24:13 UTC+1 schreef kk44...@gmail.com:

[Greg Troxel]

"t...@tom.org" <t...@tom.org> writes:

> kk, glad you corroborated my observations. I am no expert in this for sure.
> I am just tired of all of the comments I get from visitors to the website
> about how my readings exceed theoretical max. I could remove the max, but
> that doesn't seem fun.

I have added maxSolarRad to my db and am now looking at it. This
morning, I saw values that exceeded theory right at dawn, then going
under theory after a while. Two thoughts from this:

Instruments are typically specified at something like 5% of full
scale. I just looked up the VP2 and that is exactly what it says.
So that's 5% of 1800 W/m^2, or 90 W/m^2. That's huge compared to
morning levels.

The theory is presumably for a sensor that is somewhat off the ground
in an open area with not only no trees but no buildings. A
light-colored building nearby might reflect light from direct sun.

> I do not have the expertise to validate the way weewx calculates it nor am
> I even competent in Python, but for those who may, here is a link to the
> code:
>
> https://github.com/weewx/weewx/blob/d91635f3bc429f906d1f084c6a6bc8ee09fa1a27/bin/weewx/wxformulas.py#L332

Thanks. That seems to refer to fairly old work (which doesn't mean it's
wrong, but it seems there has been a lot more attention paid to this
more recently).

With any luck someone can go over the various theory papers and
implement a few more algorithms.

This looks like the state of the art:
https://pvpmc.sandia.gov/modeling-steps/1-weather-design-inputs/irradiance-and-insolation-2/irradiance-data-sources-for-performance-modeling/national-solar-radiation-database/

but my guess is that using the Wikipedia article is going to be easier.

[Cameron D]

So the weewx python code says it is using the Ryan and Stolzenbach model, which has a few approximations that don't work well in some cases, and it looks like this is one of them.

I've attached a plot comparing 3 insolation models predicting global horizontal irradiation.

Bird and Hulstrom 1991

Bras 1992

and  Ryan and Stolzenbach  1972

B&H has a lot more parameters to account for, but I have just thrown in the date/location for Boston, using whatever parameters were in the spreadsheet, and come up with the following comparison.

I used the B&H predictions for modelling my solar PV system and found it gives very close results - or at least it did before my system got a bit older.  However R&S also gives results that aren't too bad in my location.

I have the code in php, but no spare time at the moment to convert to python.

[Cameron D]

I should have looked a bit more closely before posting!  I used the solrad excel code from Uni Washington.

I had adjusted my B&H parameters to represent quite clean air but forgot to match the Bras and RS code from default.

If I:

1. reset the BH turbidity params to default,
2. adjust the Bras param down to 1.6 and
3. adjust the RS param to 0.84,

 then the curves are close to normalised at peak.  The RS curve is still a somewhat poor representation at dawn and dusk, while Bras is probably close enough to BH that it's not worth the extra effort.  

The main advantage with B&H is the atmospheric moisture, which I think contributes significantly, but differently from the atmospheric turbidity parameters and changes the shape of the curve. So a single parameter cannot account for all variables.

The safest bet might be to adjust the parameter for clear skies and then say that is an upper limit.

[Cameron D]

And then I forgot the attachment!

[Tom Keffer]

Very useful, Cameron.

It looks like the R-S curve is quite a bit lower, maybe as much as 20% lower, at dawn and dusk, which seems to be when people are experiencing values higher than maxSolarRad.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ef4b8a0b-110b-4e48-bb2e-50ebae8c8277n%40googlegroups.com.

[Cameron D]

That is part of it, but some of the early morning examples are up to 500% different, although with very small values.  In those cases I still suspect there is extra scattered radiation from the surroundings, whether it be trees or buildings.

I wonder also if the sensor type may change results slightly.

There may also be a calibration error, due to the different spectral energy distribution comparing dawn and midday, or perhaps even simple nonlinearity near zero output. These are not research-grade instruments that most of us are using.

If you add all that in with ozone, water vapour and particulates of different sizes, all of which change seasonally and daily, you will never expect precise agreement.

Here's another example - I left the RS and Bras models with the same values as previous, and changed the BH model to ignore just ozone, water vapour and particulates.

That gave the solid line. Then I set albedo for the full snow cover and the result was  the dashed line.

So, the "theoretical curve" for around Boston is anywhere in that range of curves, or it could be even lower in moderately polluted skies.