TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

[runge....@googlemail.com]

Hi Jim,

i have today a strange error.

I do not know why this happend.

Can you check this?

I will check my rawdata if something is messed up:

2023-10-10 13:14:04:pywws.regulartasks:Templating ws3600.txt
2023-10-10 13:14:04:pywws.livelog:unsupported operand type(s) for -: 'NoneType' and 'float'
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pywws/livelog.py", line 89, in live_log
    tasks.do_tasks()
  File "/usr/local/lib/python3.9/dist-packages/pywws/regulartasks.py", line 213, in do_tasks
    self._do_common(now, sections)
  File "/usr/local/lib/python3.9/dist-packages/pywws/regulartasks.py", line 147, in _do_common
    self.services[name].upload(live_data=live_data, options=options)
  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 340, in upload
    if self.queue_data(timestamp, data):
  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 307, in queue_data
    OK = super(CatchupDataService, self).queue_data(timestamp, data)
  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 281, in queue_data
    prepared_data = self.prepare_data(data)
  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 290, in prepare_data
    data_str = self.templater.make_text(self.template_file, data)
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 580, in make_text
    for text in self.process(live_data, template_file):
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 462, in process
    x = eval(command[1])
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 339, in rain_hour
    return max(0.0, self.hour_diff(data, 'rain'))
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 336, in hour_diff
    return data[key] - hour_ago[key]
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

Thanks and best regards,

Stefan

[runge....@googlemail.com]

Ok i found the root cause.

My weather station had the glitch with providing wrong data and my calib removed the values.

2023-10-10 11:28:05:pywws.service.underground:1 record sent
2023-10-10 11:28:50:calib:temp_out spike > 4.0 from median removed? [19.6, 19.7, 19.7, 19.8, 20, 20.7] 36.7
2023-10-10 11:28:50:calib:rain spike > 400 removed? 65.1 11057.4

The removal of the rain spike seems to have a negative impact on the raw data so that it can not be templated.

Can you help, how to prevent this?

2023-10-10 11:28:50:pywws.regulartasks:doing task sections ['live']
2023-10-10 11:28:50:pywws.regulartasks:Templating ws3600.txt
2023-10-10 11:28:50:pywws.livelog:unsupported operand type(s) for -: 'NoneType' and 'WSFloat'

Traceback (most recent call last):

  File "/usr/local/lib/python3.9/dist-packages/pywws/livelog.py", line 91, in live_log
    tasks.do_live(data)
  File "/usr/local/lib/python3.9/dist-packages/pywws/regulartasks.py", line 203, in do_live
    self._do_common(now, sections, live_data=calib_data)

  File "/usr/local/lib/python3.9/dist-packages/pywws/regulartasks.py", line 147, in _do_common
    self.services[name].upload(live_data=live_data, options=options)

  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 343, in upload
    self.queue_data(live_data['idx'], live_data)

  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 307, in queue_data
    OK = super(CatchupDataService, self).queue_data(timestamp, data)
  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 281, in queue_data
    prepared_data = self.prepare_data(data)
  File "/usr/local/lib/python3.9/dist-packages/pywws/service/__init__.py", line 290, in prepare_data
    data_str = self.templater.make_text(self.template_file, data)
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 580, in make_text
    for text in self.process(live_data, template_file):
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 462, in process
    x = eval(command[1])
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 339, in rain_hour
    return max(0.0, self.hour_diff(data, 'rain'))
  File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line 336, in hour_diff
    return data[key] - hour_ago[key]

TypeError: unsupported operand type(s) for -: 'NoneType' and 'WSFloat'

Thanks and best regards,

Stefan

[Jim Easterbrook]

On 10/10/2023 12:37, 'runge....@googlemail.com' via pywws wrote:
>
> i have today a strange error.
> I do not know why this happend.
> Can you check this?
> I will check my rawdata if something is messed up:
>
> 2023-10-10 13:14:04:pywws.regulartasks:Templating ws3600.txt
> 2023-10-10 13:14:04:pywws.livelog:unsupported operand type(s) for -:
> 'NoneType' and 'float'

[snip]

>   File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line
> 339, in rain_hour
>     return max(0.0, self.hour_diff(data, 'rain'))
>   File "/usr/local/lib/python3.9/dist-packages/pywws/template.py", line
> 336, in hour_diff
>     return data[key] - hour_ago[key]
> TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

Ah, this is a consequence of your data spike filter replacing an
incorrect rain value with 'None'. I should have realised that would be a
problem.

If you change your user calibration module to replace a rain spike with
the previous value instead (and then regenerate the affected calibrated
data) then all should be OK. There shouldn't be any need to test for
rain being 'None' because it should always have a value.

before = self.raw_data.before(result['idx'])
if result['rain'] - self.raw_data[before]['rain'] > 500:
result['rain'] = self.raw_data[before]['rain']

--
Jim Easterbrook <http://www.jim-easterbrook.me.uk/>

[runge....@googlemail.com]

Thanks that sounds good, i adapted my CALIB,

But i still get the error,

Where do i have to clean up data?

I tried in RAW data for today but it did not help.

Do i have to check all other files too (Hourly, Daily, Monthly, ...)?

Thanks,

 Stefan

[Jim Easterbrook]

On 10/10/2023 12:58, 'runge....@googlemail.com' via pywws wrote:
> Thanks that sounds good, i adapted my CALIB,
> But i still get the error,
> Where do i have to clean up data?

Stop pywws, then delete 'calib' files from your data directory working
backwards from today to where the problem is, then restart pywws. The
deleted calib files will be automatically regenerated using your new
calibration module.

> I tried in RAW data for today but it did not help.

Don't change the raw data, it's the calib data that gets the corrected
data, so needs to be regenerated if the calibration didn't do the right
thing.

> Do i have to check all other files too (Hourly, Daily, Monthly, ...)?

You shouldn't need to as pywws will detect they are out of date with
respect to the regenerated calib files.

[runge....@googlemail.com]

Hi Jim,

you saved my day again!

Thank you so much for your help!

Best regards,

Stefan