USBTimeoutError after live_data lost sync

[runge....@googlemail.com]

Hi Jim,

i know this is a very old topic and i hab no problem with it at all the last years.

But now i get this problem nearly every day, after i have cleaned my outside sensors.

I have a raspberry pi 4,

I already read all the discussions and tried the fix with:

dwc_otg.fiq_split_enable=0 dwc_otg.lpm_enable=0 in cmdline.txt

but it did not help.

So i checked the logs more in detail and read your documentation.

I get: 

 2024-01-19 15:59:06:pywws.weatherstation:live_data lost sync -3.39785

and after that i loose the USB connection.

I adapted 

usb activity margin = 3.9

but also this seems not to help.

You write:

" The clock drifts are measured at approximately 24 hour intervals. If pywws loses synchronisation with your station it will measure them again. Doing this measurement increases the risk of causing a USB lockup, so if pywws often loses synchronisation you should try and find out why it’s happening."

But how can i find out why it is happening? Is it just bad reception? I can move my base station to a better place if this might help.

Do you have an idea what i can check or do?

Here is the interesting part of the log from today where i lost the USB connection,

interesting that two times after each other it looses the sync.

2024-01-19 15:54:03:pywws.regulartasks:Templating ws3600.txt
2024-01-19 15:54:04:pywws.service.copy:1 upload
2024-01-19 15:54:20:pywws.weatherstation:live_data lost sync -3.29565
2024-01-19 15:54:20:pywws.weatherstation:old data {'illuminance': 2329, 'uv': 0, 'delay': 0, 'hum_in': 62, 'temp_in': 20.1, 'hum_out': 75, 'temp_out': 0.4, 'abs_pressure': 997.2, 'wind_ave': 0, 'wind_gust': 0, 'wind_dir': 8, 'rain': 78, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-19 15:54:20:pywws.weatherstation:new data {'illuminance': 0, 'uv': 0, 'delay': 5, 'hum_in': 71, 'temp_in': 19.7, 'hum_out': 86, 'temp_out': 8.6, 'abs_pressure': 989.7, 'wind_ave': 1, 'wind_gust': 1.7, 'wind_dir': 4, 'rain': 22.5, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-19 15:54:22:pywws.weatherstation:setting sensor clock 46.0917
2024-01-19 15:54:22:pywws.regulartasks:doing task sections ['live']
2024-01-19 15:54:22:pywws.regulartasks:Templating ws3600.txt
2024-01-19 15:54:22:pywws.service.underground:1 record sent
2024-01-19 15:54:24:pywws.service.copy:1 upload
2024-01-19 15:55:13:pywws.regulartasks:doing task sections ['live']
2024-01-19 15:55:13:pywws.regulartasks:Templating ws3600.txt
2024-01-19 15:55:15:pywws.service.underground:1 record sent
2024-01-19 15:55:16:pywws.service.copy:1 upload
2024-01-19 15:56:03:pywws.regulartasks:doing task sections ['live']
2024-01-19 15:56:03:pywws.regulartasks:Templating ws3600.txt
2024-01-19 15:56:04:pywws.service.copy:1 upload
2024-01-19 15:56:07:pywws.service.underground:1 record sent
2024-01-19 15:56:49:pywws.regulartasks:doing task sections ['live']
2024-01-19 15:56:49:pywws.regulartasks:Templating ws3600.txt
2024-01-19 15:56:52:pywws.service.copy:1 upload
2024-01-19 15:57:42:pywws.regulartasks:doing task sections ['live']
2024-01-19 15:57:42:pywws.regulartasks:Templating ws3600.txt
2024-01-19 15:57:44:pywws.service.copy:1 upload
2024-01-19 15:57:46:pywws.service.underground:2 records sent
2024-01-19 15:58:28:pywws.weatherstation:live_data missed
2024-01-19 15:59:03:pywws.weatherstation:live_data new ptr: 008404
2024-01-19 15:59:03:pywws.process:Generating summary data
2024-01-19 15:59:03:pywws.regulartasks:doing task sections ['logged']
2024-01-19 15:59:03:pywws.regulartasks:Templating ws3600.txt
2024-01-19 15:59:04:pywws.service.copy:1 upload
2024-01-19 15:59:06:pywws.service.underground:1 record sent
2024-01-19 15:59:06:pywws.weatherstation:live_data lost sync -3.39785
2024-01-19 15:59:06:pywws.weatherstation:old data {'illuminance': 2004.7, 'uv': 0, 'delay': 0, 'hum_in': 62, 'temp_in': 20.1, 'hum_out': 75, 'temp_out': 0.3, 'abs_pressure': 997.4, 'wind_ave': 0, 'wind_gust': 0, 'wind_dir': 8, 'rain': 78, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-19 15:59:06:pywws.weatherstation:new data {'illuminance': 0, 'uv': 0, 'delay': 5, 'hum_in': 71, 'temp_in': 19.7, 'hum_out': 86, 'temp_out': 8.6, 'abs_pressure': 989.9, 'wind_ave': 1.4, 'wind_gust': 2, 'wind_dir': 14, 'rain': 22.5, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-19 15:59:18:pywws.livelog:[Errno 110] Operation timed out
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pywws/livelog.py", line 83, in live_log
    for data, logged in datalogger.live_data(
  File "/usr/local/lib/python3.9/dist-packages/pywws/logdata.py", line 252, in live_data
    for data, ptr, logged in self.ws.live_data(logged_only=logged_only):
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 549, in live_data
    new_ptr = self.current_pos()
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 711, in current_pos
    self._read_fixed_block(0x0020), self.lo_fix_format['current_pos'])
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 771, in _read_fixed_block
    result += self._read_block(mempos)
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 759, in _read_block
    new_block = self.cusb.read_block(ptr)
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 346, in read_block
    return self.dev.read_data(32)
  File "/usr/local/lib/python3.9/dist-packages/pywws/device_pyusb1.py", line 104, in read_data
    result = self.dev.read(0x81, size, timeout=1200)
  File "/usr/local/lib/python3.9/dist-packages/usb/core.py", line 1029, in read
    ret = fn(
  File "/usr/local/lib/python3.9/dist-packages/usb/backend/libusb1.py", line 864, in intr_read
    return self.__read(self.lib.libusb_interrupt_transfer,
  File "/usr/local/lib/python3.9/dist-packages/usb/backend/libusb1.py", line 954, in __read
    _check(retval)
  File "/usr/local/lib/python3.9/dist-packages/usb/backend/libusb1.py", line 602, in _check
    raise USBTimeoutError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBTimeoutError: [Errno 110] Operation timed out

Thanks,

Stefan

[Jim Easterbrook]

On 19/01/2024 16:09, 'runge....@googlemail.com' via pywws wrote:
>
> But now i get this problem nearly every day, after i have cleaned my
> outside sensors.

I wonder what changed?

> So i checked the logs more in detail and read your documentation.
> I get:
>  2024-01-19 15:59:06:pywws.weatherstation:live_data lost sync -3.39785
> and after that i loose the USB connection.
> I adapted
> usb activity margin = 3.9
> but also this seems not to help.

The number shown in the "lost sync" message isn't really an indicator of
what margin is required. I think it will always be larger than your
current margin.

If you have an old backup of your data file you could have a look in
status.ini at the "sensor drift" value. The activity margin needs to
exceed the larger of the (absolute) drift values by at least 0.5 seconds.

You could try setting the margin to something larger, e.g. 6 seconds,
and see if things get any more stable.

> You write:
> " The clock drifts are measured at approximately 24 hour intervals. If
> pywws loses synchronisation with your station it will measure them
> again. Doing this measurement increases the risk of causing a USB
> lockup, so if pywws often loses synchronisation you should try and find
> out why it’s happening."

Yeah, that's not very helpful is it?

With hindsight I think I was wrong to try and synchronise to the
station's clocks. It was an OK thing to do before stations started
locking up but now it seems to be more trouble than it's worth.

Taking one reading every 15 seconds and declaring it to be the current
weather is probably good enough. (It's what the EasyWeather software
does.) Reading multiple times to see when it changes (what pywws does)
doesn't seem so clever now.

> But how can i find out why it is happening? Is it just bad reception? I
> can move my base station to a better place if this might help.

Probably not due to bad reception.

> Do you have an idea what i can check or do?

If you can bear not having 'live' logging then you could set "logdata
sync" to zero and run pywws-hourly at twice your logging interval. This
should poll the station much less often and reduce your chances of lockup.

> Here is the interesting part of the log from today where i lost the USB
> connection,
> interesting that two times after each other it looses the sync.
>

> 2024-01-19 15:54:20:pywws.weatherstation:live_data lost sync -3.29565
> 2024-01-19 15:54:20:pywws.weatherstation:old data {'illuminance': 2329,
> 'uv': 0, 'delay': 0, 'hum_in': 62, 'temp_in': 20.1, 'hum_out': 75,
> 'temp_out': 0.4, 'abs_pressure': 997.2, 'wind_ave': 0, 'wind_gust': 0,
> 'wind_dir': 8, 'rain': 78, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
> 2024-01-19 15:54:20:pywws.weatherstation:new data {'illuminance': 0,
> 'uv': 0, 'delay': 5, 'hum_in': 71, 'temp_in': 19.7, 'hum_out': 86,
> 'temp_out': 8.6, 'abs_pressure': 989.7, 'wind_ave': 1, 'wind_gust': 1.7,
> 'wind_dir': 4, 'rain': 22.5, 'status': {'lost_connection': False,
> 'rain_overflow': False}}

This is why it loses sync - nothing to do with clock drift but because
of an unexpected data change. Note that all the readings change, with an
abrupt change in air pressure and a decrease in the total rain. This
looks like picking up very old data, maybe from reading the wrong memory
address.

> 2024-01-19 15:59:06:pywws.weatherstation:live_data lost sync -3.39785
> 2024-01-19 15:59:06:pywws.weatherstation:old data {'illuminance':
> 2004.7, 'uv': 0, 'delay': 0, 'hum_in': 62, 'temp_in': 20.1, 'hum_out':
> 75, 'temp_out': 0.3, 'abs_pressure': 997.4, 'wind_ave': 0, 'wind_gust':
> 0, 'wind_dir': 8, 'rain': 78, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
> 2024-01-19 15:59:06:pywws.weatherstation:new data {'illuminance': 0,
> 'uv': 0, 'delay': 5, 'hum_in': 71, 'temp_in': 19.7, 'hum_out': 86,
> 'temp_out': 8.6, 'abs_pressure': 989.9, 'wind_ave': 1.4, 'wind_gust': 2,
> 'wind_dir': 14, 'rain': 22.5, 'status': {'lost_connection': False,
> 'rain_overflow': False}}

This is almost the same wrong data, probably the next wrong address to
the previous wrong one.

The two readings are five minutes apart. I assume that's your logging
interval. When the logging interval is reached the station increments
the "current data" pointer and pywws starts reading data from the new
address. In your case it looks as if the station hasn't yet written new
data to to the new current data pointer so you get old data. I've not
seen this behaviour before.

If you run live logging with high verbosity (in a terminal rather than a
background task writing to log files) you should see it log when the
data pointer changes.

It might be possible to modify pywws to ignore the "new" data if it's
obviously wrong. In this case the 'delay' value jumping from 0 to 5 is
proof of wrong data.

--
Jim Easterbrook <http://www.jim-easterbrook.me.uk/>

[Jim Easterbrook]

On 20/01/2024 10:07, Jim Easterbrook wrote:
> On 19/01/2024 16:09, 'runge....@googlemail.com' via pywws wrote:
>
>> 2024-01-19 15:59:06:pywws.weatherstation:live_data lost sync -3.39785
>> 2024-01-19 15:59:06:pywws.weatherstation:old data {'illuminance':
>> 2004.7, 'uv': 0, 'delay': 0, 'hum_in': 62, 'temp_in': 20.1, 'hum_out':
>> 75, 'temp_out': 0.3, 'abs_pressure': 997.4, 'wind_ave': 0,
>> 'wind_gust': 0, 'wind_dir': 8, 'rain': 78, 'status':
>> {'lost_connection': False, 'rain_overflow': False}}
>> 2024-01-19 15:59:06:pywws.weatherstation:new data {'illuminance': 0,
>> 'uv': 0, 'delay': 5, 'hum_in': 71, 'temp_in': 19.7, 'hum_out': 86,
>> 'temp_out': 8.6, 'abs_pressure': 989.9, 'wind_ave': 1.4, 'wind_gust':
>> 2, 'wind_dir': 14, 'rain': 22.5, 'status': {'lost_connection': False,
>> 'rain_overflow': False}}
>

> The two readings are five minutes apart. I assume that's your logging
> interval. When the logging interval is reached the station increments
> the "current data" pointer and pywws starts reading data from the new
> address. In your case it looks as if the station hasn't yet written new
> data to to the new current data pointer so you get old data. I've not
> seen this behaviour before.

Thinking about this some more (insomnia has its benefits) I realise that
my experience is limited to 1080-type stations (i.e. without solar data)
and you have a 3080-type machine.

As I understand it, the solar data is sent separately from the other
external sensor data, and at 60 second intervals instead of 48 seconds.
Maybe the arrival of solar data triggers the station to advance the
"current pointer" to the next memory slot, but that data doesn't get
filled in until normal external sensor data arrives.

Have any other 3080-type owner suffered similar problems?

I'll try modifying the pywws live log loop to not use the new pointer
until the 'delay' value has been reset, which I hope will reliably
signal that the data is now valid.

[richar...@gmail.com]

Hello,

I have a 3080 station and I regularly get this as well, here is an example from this morning:

2024-01-21 07:17:13:pywws.weatherstation:live_data lost sync -12.2477
2024-01-21 07:17:13:pywws.weatherstation:old data {'illuminance': 111.6, 'uv': 0, 'delay': 0, 'hum_in': 50, 'temp_in': 15.3, 'hum_out': 95, 'temp_out': 10.1, 'abs_pressure': 1001, 'wind_ave': 0.7, 'wind_gust': 1.7, 'wind_dir': 12, 'rain': 187.5, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-21 07:17:13:pywws.weatherstation:new data {'illuminance': 111.6, 'uv': 0, 'delay': 0, 'hum_in': 50, 'temp_in': 15.2, 'hum_out': 95, 'temp_out': 10.1, 'abs_pressure': 1001.1, 'wind_ave': 0.7, 'wind_gust': 1.4, 'wind_dir': 12, 'rain': 187.5, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-21 07:18:14:pywws.weatherstation:setting sensor clock 38.1175

To be honest it doesn't seem to affect the data and graphing, so I just ignored it and put it down to a quirk of the system.

Hope this helps figure out if there is something to fix, or not.

[Jim Easterbrook]

On 21/01/2024 09:17, richar...@gmail.com wrote:
>
> I have a 3080 station and I regularly get this as well, here is an
> example from this morning:
>
> 2024-01-21 07:17:13:pywws.weatherstation:live_data lost sync -12.2477
> 2024-01-21 07:17:13:pywws.weatherstation:old data {'illuminance': 111.6,
> 'uv': 0, 'delay': 0, 'hum_in': 50, 'temp_in': 15.3, 'hum_out': 95,
> 'temp_out': 10.1, 'abs_pressure': 1001, 'wind_ave': 0.7, 'wind_gust':
> 1.7, 'wind_dir': 12, 'rain': 187.5, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
> 2024-01-21 07:17:13:pywws.weatherstation:new data {'illuminance': 111.6,
> 'uv': 0, 'delay': 0, 'hum_in': 50, 'temp_in': 15.2, 'hum_out': 95,
> 'temp_out': 10.1, 'abs_pressure': 1001.1, 'wind_ave': 0.7, 'wind_gust':
> 1.4, 'wind_dir': 12, 'rain': 187.5, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
> 2024-01-21 07:18:14:pywws.weatherstation:setting sensor clock 38.1175

That's a loss of sync but it's not picking up old data. Comparing the
new and old shows they are very similar.

> To be honest it doesn't seem to affect the data and graphing, so I just
> ignored it and put it down to a quirk of the system.

As long as it doesn't cause your station to lock up when it resyncs.

> Hope this helps figure out if there is something to fix, or not.

I'm getting closer to doing a total rewrite with no attempt to
synchronise to the station!

[runge....@googlemail.com]

Hi Jim,

thank you very much for looking again at this problem.

Yes i am also wondering what changed.

It was running for years now with only very rare hang ups. (Perhaps every 6 month).

Beside of cleaning the sensors (Solar and Temp), 

i also changed to a new USB 3.1 hub, and a new SSD enclosure.

The USB hub and the enclosure are now devices which are listed as well known and good working with raspberry PI.

My problems with SSD and other USB devices went away.

But now i have the problems with the station again.

I have tried to connect to the USB 2.0 hub of the raspberry and also to other hubs connected to the USB 2.0 hub of the raspberry.

But it seems not to change the situation.

But i have a few more external hubs i can try. :-)

Today i had again:

2024-01-21 11:15:08:pywws.weatherstation:live_data lost sync -4.07431
2024-01-21 11:15:08:pywws.weatherstation:old data {'illuminance': 12158.8, 'uv': 1, 'delay': 0, 'hum_in': 60, 'temp_in': 16.2, 'hum_out': 78, 'temp_out': -0.7, 'abs_pressure': 1002, 'wind_ave': 0, 'wind_gust': 0.3, 'wind_dir': 8, 'rain': 80.4, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-21 11:15:08:pywws.weatherstation:new data {'illuminance': 0, 'uv': 0, 'delay': 5, 'hum_in': 71, 'temp_in': 19.8, 'hum_out': 81, 'temp_out': 6.9, 'abs_pressure': 981.1, 'wind_ave': 2, 'wind_gust': 3.1, 'wind_dir': 12, 'rain': 23.7, 'status': {'lost_connection': False, 'rain_overflow': False}}

I checked the status.ini:

[clock]
station = 1705820939.7095635
station drift = 0.9106067314525064
sensor = 1705829088.2602959
sensor drift = 3.4194341854769776
solar = 1705755337.9066527
solar drift = -1.7212174246220397

I also have old backups. Why do you write i should use a backup and how old should it be?

I increased now usb activity margin to 6.0.

Lets see if this makes a difference.

Yes my logging interval is 5 mins. Would it help to increase this?

I use wunderground with live logging, but sure i could try pywws-hourly at twice my logging interval.

But before i do this i want to try everything else.

Again thank you for looking into this and trying a code fix.

Best regards,

Stefan

[richar...@gmail.com]

Dear Jim,

Your point on whether the loss of sync to the station could cause a lock-up got me thinking, as my station does lock up the USB from time to time and I use your script which I've adapted to email me the log file and restart the station If the error is a USB ERROR. Looking back at the last few instances of a lock-up of the USB was immediately preceded by a loss of station sync, such as this one:

2024-01-17 03:21:10:pywws.weatherstation:live_data lost sync -12.2497 

2024-01-17 03:21:10:pywws.weatherstation:old data {'illuminance': 0, 'uv': 0, 'delay': 0, 'hum_in': 45, 'temp_in': 13.7, 'hum_out': 81, 'temp_out': -1.5, 'abs_pressure': 985.2, 'wind_ave': 0, 'wind_gust': 0, 'wind_dir': 14, 'rain': 184.8, 'status': {'lost_connection': False, 'rain_overflow': False}} 

2024-01-17 03:21:10:pywws.weatherstation:new data {'illuminance': 0, 'uv': 0, 'delay': 0, 'hum_in': 45, 'temp_in': 13.8, 'hum_out': 81, 'temp_out': -1.5, 'abs_pressure': 985, 'wind_ave': 0, 'wind_gust': 0, 'wind_dir': 14, 'rain': 184.8, 'status': {'lost_connection': False, 'rain_overflow': False}} 

2024-01-17 03:22:11:pywws.livelog:LIBUSB_ERROR_TIMEOUT [-7]

[Jim Easterbrook]

On 21/01/2024 12:28, 'runge....@googlemail.com' via pywws wrote:
>
> Today i had again:
> 2024-01-21 11:15:08:pywws.weatherstation:live_data lost sync -4.07431
> 2024-01-21 11:15:08:pywws.weatherstation:old data {'illuminance':
> 12158.8, 'uv': 1, 'delay': 0, 'hum_in': 60, 'temp_in': 16.2, 'hum_out':
> 78, 'temp_out': -0.7, 'abs_pressure': 1002, 'wind_ave': 0, 'wind_gust':
> 0.3, 'wind_dir': 8, 'rain': 80.4, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
> 2024-01-21 11:15:08:pywws.weatherstation:new data {'illuminance': 0,
> 'uv': 0, 'delay': 5, 'hum_in': 71, 'temp_in': 19.8, 'hum_out': 81,
> 'temp_out': 6.9, 'abs_pressure': 981.1, 'wind_ave': 2, 'wind_gust': 3.1,
> 'wind_dir': 12, 'rain': 23.7, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
>
> I checked the status.ini:
> [clock]
> station = 1705820939.7095635
> station drift = 0.9106067314525064
> sensor = 1705829088.2602959
> sensor drift = 3.4194341854769776

This suggests your margin should be 4 seconds instead of the usual 3.

> solar = 1705755337.9066527
> solar drift = -1.7212174246220397
>
> I also have old backups. Why do you write i should use a backup and how
> old should it be?
> I increased now usb activity margin to 6.0.
> Lets see if this makes a difference.
>
> Yes my logging interval is 5 mins. Would it help to increase this?

No, it shouldn't be a factor.

One thing you could try is to edit weatherstation.py and change line 756
from

elif next_live and data_time < next_live - self.margin:

to

elif (new_ptr == old_ptr
and next_live and data_time < next_live - self.margin):

This should stop a pointer change triggering a loss of sync. I'm testing
it at the moment and it doesn't seem to have any unwanted side effects.

[runge....@googlemail.com]

Hi Jim,

ok thanks. 4 seconds was my setting. I was playing around and had it at 3.9 and other values. 

At the moment i am trying with 6, but will change it back to 4.

Ok, i wanted to make the code change but line 756 is different.

I find the code in line 576.

Is it a typo or did i pull something wrong?

When i check in git it is also 576, so most likely you have a typo i will change the code and try to build and compile.

I will report any findings.

Thanks and best regards,

Stefan

[runge....@googlemail.com]

Ok, normally i install with pip.

Now i tried getting the source from git do the adaptations and build.

All worked well until

 sudo python setup.py install

There i get (most likely i have to update python or pip?):

running install
/usr/local/lib/python3.9/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.9/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(

Traceback (most recent call last):

  File "/home/pi/Programs/weather/pywws/setup.py", line 170, in <module>
    setup(name = 'pywws',
  File "/usr/local/lib/python3.9/dist-packages/setuptools/__init__.py", line 108, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.9/dist-packages/setuptools/dist.py", line 1213, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.9/dist-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/usr/local/lib/python3.9/dist-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.ensure_finalized()  # finalize before bdist_egg munges install cmd
  File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
    self.finalize_options()
  File "/usr/local/lib/python3.9/dist-packages/setuptools/command/easy_install.py", line 311, in finalize_options
    self.local_index = Environment(self.shadow_path + sys.path)
  File "/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py", line 1044, in __init__
    self.scan(search_path)
  File "/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py", line 1077, in scan
    self.add(dist)
  File "/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py", line 1096, in add
    dists.sort(key=operator.attrgetter('hashcmp'), reverse=True)
  File "/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py", line 2631, in hashcmp
    self.parsed_version,
  File "/usr/local/lib/python3.9/dist-packages/pkg_resources/__init__.py", line 2685, in parsed_version
    raise packaging.version.InvalidVersion(f"{str(ex)} {info}") from None
pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '1.14.0-unknown' (package: gpg)

 

Best regards,

Stefan

[Jim Easterbrook]

On 21/01/2024 16:36, 'runge....@googlemail.com' via pywws wrote:
>
> Ok, i wanted to make the code change but line 756 is different.
> I find the code in line 576.
> Is it a typo or did i pull something wrong?
> When i check in git it is also 576, so most likely you have a typo i
> will change the code and try to build and compile.

Yes, a typo. Sorry.

[Jim Easterbrook]

On 21/01/2024 16:42, 'runge....@googlemail.com' via pywws wrote:
> Ok, normally i install with pip.
> Now i tried getting the source from git do the adaptations and build.
>
> All worked well until
>  sudo python setup.py install
>
> There i get (most likely i have to update python or pip?):

You probably could have edited your pip installed file. Try 'sudo pip
install .' instead of 'python setup.py build' followed by 'sudo python
setup.py install'

The Python packaging recommended tools have changed a lot in recent
years and I haven't updated pywws accordingly. With many users using old
platforms and old versions of Python (e.g. 2.7) it's not obvious what to do.

[runge....@googlemail.com]

Hi Jim,

thanks.

I installed it now with  pip install .

Let's see i will report if this solves the issue.

Thanks and best regards,

Stefan

[runge....@googlemail.com]

Ok,

i see now from time to time this loglines.

2024-01-23 17:11:52:pywws.weatherstation:live_data lost sync -6.22083
2024-01-23 17:11:52:pywws.weatherstation:old data {'illuminance': 0, 'uv': 0, 'delay': 0, 'hum_in': 66, 'temp_in': 21.3, 'hum_out': 71, 'temp_out': 8.1, 'abs_pressure': 1003, 'wind_ave': 2, 'wind_gust': 3.1, 'wind_dir': 6, 'rain': 86.7, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-23 17:11:52:pywws.weatherstation:new data {'illuminance': 0, 'uv': 0, 'delay': 0, 'hum_in': 66, 'temp_in': 21.3, 'hum_out': 71, 'temp_out': 8.1, 'abs_pressure': 1003, 'wind_ave': 1.7, 'wind_gust': 2.4, 'wind_dir': 6, 'rain': 86.7, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-23 17:12:46:pywws.weatherstation:setting sensor clock 46.0562

The data in old and new is the same. 

Is this the proof that it is working?

Thanks and best regards,

Stefan

[Jim Easterbrook]

On 23/01/2024 17:29, 'runge....@googlemail.com' via pywws wrote:
>
> i see now from time to time this loglines.
> 2024-01-23 17:11:52:pywws.weatherstation:live_data lost sync -6.22083
> 2024-01-23 17:11:52:pywws.weatherstation:old data {'illuminance': 0,
> 'uv': 0, 'delay': 0, 'hum_in': 66, 'temp_in': 21.3, 'hum_out': 71,
> 'temp_out': 8.1, 'abs_pressure': 1003, 'wind_ave': 2, 'wind_gust': 3.1,
> 'wind_dir': 6, 'rain': 86.7, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
> 2024-01-23 17:11:52:pywws.weatherstation:new data {'illuminance': 0,
> 'uv': 0, 'delay': 0, 'hum_in': 66, 'temp_in': 21.3, 'hum_out': 71,
> 'temp_out': 8.1, 'abs_pressure': 1003, 'wind_ave': 1.7, 'wind_gust':
> 2.4, 'wind_dir': 6, 'rain': 86.7, 'status': {'lost_connection': False,
> 'rain_overflow': False}}
> 2024-01-23 17:12:46:pywws.weatherstation:setting sensor clock 46.0562
>
> The data in old and new is the same.
> Is this the proof that it is working?

Maybe not proof, but it's definitely an improvement. As you see, the old
and new data are consistent, so this is "real" new data. Let me know how
it's behaving in a day or two. The sensor clock drift gets averaged over
a few days and you've probably had some bad values in the past because
of the sync loss problem.

[runge....@googlemail.com]

Yes i think so too,

i am pretty sure that some bad values where caused by this.

Since the problem occured i had a hang up every day.

So i will monitor this for the next 2 days and inform you.

Thanks,

Stefan

[runge....@googlemail.com]

Hi Jim,

no hangup till now. 

But when checking the logs i found a very interesting fact i do not understand.

This section of missed live data is now always happening at a exact interval of 20 minutes.

2024-01-24 13:11:15:pywws.weatherstation:live_data missed
2024-01-24 13:11:15:pywws.weatherstation:live_data new ptr: 00e9f8
2024-01-24 13:11:15:pywws.process:Generating summary data
2024-01-24 13:11:15:pywws.regulartasks:doing task sections ['logged']
2024-01-24 13:11:15:pywws.regulartasks:Templating ws3600.txt
2024-01-24 13:11:16:pywws.service.copy:1 upload
2024-01-24 13:11:51:pywws.weatherstation:live_data lost sync -6.20055
2024-01-24 13:11:51:pywws.weatherstation:old data {'illuminance': 9755, 'uv': 1, 'delay': 0, 'hum_in': 67, 'temp_in': 18.6, 'hum_out': 61, 'temp_out': 13.1, 'abs_pressure': 998.2, 'wind_ave': 2, 'wind_gust': 3.1, 'wind_dir': 14, 'rain': 95.4, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-24 13:11:51:pywws.weatherstation:new data {'illuminance': 9106.4, 'uv': 1, 'delay': 0, 'hum_in': 67, 'temp_in': 18.6, 'hum_out': 61, 'temp_out': 13.1, 'abs_pressure': 998.4, 'wind_ave': 1.7, 'wind_gust': 2.7, 'wind_dir': 14, 'rain': 95.4, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-24 13:12:45:pywws.weatherstation:setting sensor clock 45.1373

Do you have any idea why this might happen like this.

2024-01-24 13:11:15:pywws.weatherstation:live_data missed

2024-01-24 13:31:15:pywws.weatherstation:live_data missed

2024-01-24 13:51:15:pywws.weatherstation:live_data missed

2024-01-24 14:11:15:pywws.weatherstation:live_data missed

2024-01-24 14:31:15:pywws.weatherstation:live_data missed

2024-01-24 14:51:15:pywws.weatherstation:live_data missed

2024-01-24 15:11:15:pywws.weatherstation:live_data missed

2024-01-24 15:31:15:pywws.weatherstation:live_data missed

2024-01-24 15:51:15:pywws.weatherstation:live_data missed

2024-01-24 16:11:15:pywws.weatherstation:live_data missed

2024-01-24 16:31:15:pywws.weatherstation:live_data missed

2024-01-24 16:51:15:pywws.weatherstation:live_data missed

2024-01-24 17:11:15:pywws.weatherstation:live_data missed

2024-01-24 17:31:15:pywws.weatherstation:live_data missed

2024-01-24 17:51:15:pywws.weatherstation:live_data missed

This can not be coincidence.

There are also some other live data missed events in between but without new pointer or any other reaction of the coding like this one:

2024-01-24 17:46:35:pywws.service.underground:1 record sent
2024-01-24 17:46:37:pywws.service.copy:1 upload
2024-01-24 17:47:15:pywws.weatherstation:live_data missed
2024-01-24 17:48:06:pywws.regulartasks:doing task sections ['live']
2024-01-24 17:48:06:pywws.regulartasks:Templating ws3600.txt
2024-01-24 17:48:08:pywws.service.underground:1 record sent

Nevertheless data looks fine and still no hang up.

Thanks,

Stefan

[Jim Easterbrook]

On 24/01/2024 17:27, 'runge....@googlemail.com' via pywws wrote:
>
> But when checking the logs i found a very interesting fact i do not
> understand.
>
> This section of missed live data is now always happening at a exact
> interval of 20 minutes.
> 2024-01-24 13:11:15:pywws.weatherstation:live_data missed
> 2024-01-24 13:11:15:pywws.weatherstation:live_data new ptr: 00e9f8

I think what's happening is this: live data (48 second intervals) is
expected just before the pointer change (5 minute intervals). To avoid
hangups pywws doesn't poll the weather station for a few seconds either
side of when either of these events is due. Once polling is resumed the
pointer has changed and we now ignore the data as it might be old. So
the expected live data was missed. You should still have logged data
from that time, so any services etc that run on both live and logged
should be OK.

> Do you have any idea why this might happen like this.
> 2024-01-24 13:11:15:pywws.weatherstation:live_data missed
> 2024-01-24 13:31:15:pywws.weatherstation:live_data missed
> 2024-01-24 13:51:15:pywws.weatherstation:live_data missed

20 minutes == (25 * 48 seconds) == (4 * 5 minutes).

Eventually your sensor clock and station clock will drift apart and
these missed data events will stop.

> There are also some other live data missed events in between but without
> new pointer or any other reaction of the coding like this one:
> 2024-01-24 17:46:35:pywws.service.underground:1 record sent
> 2024-01-24 17:46:37:pywws.service.copy:1 upload
> 2024-01-24 17:47:15:pywws.weatherstation:live_data missed
> 2024-01-24 17:48:06:pywws.regulartasks:doing task sections ['live']
> 2024-01-24 17:48:06:pywws.regulartasks:Templating ws3600.txt
> 2024-01-24 17:48:08:pywws.service.underground:1 record sent
>
> Nevertheless data looks fine and still no hang up.

It can also be logged as missed if the new data is identical to the old.
Usually the 'delay' value changes, but 48 seconds is less than a minute
so two successive live data sets can have the same 'delay'.

[runge....@googlemail.com]

Hi Jim, 

ok i see  20 minutes == (25 * 48 seconds) == (4 * 5 minutes).

Unfortunately i had a hung up at the moment.

It was running longer than before, nearly 2 days but it hung up now.

The hung up happened exactly at this 20 min interval:

2024-01-25 15:31:49:pywws.weatherstation:live_data lost sync -6.29283
2024-01-25 15:31:49:pywws.weatherstation:old data {'illuminance': 11243.2, 'uv': 1, 'delay': 0, 'hum_in': 65, 'temp_in': 20.8, 'hum_out': 58, 'temp_out': 11.6, 'abs_pressure': 1002.9, 'wind_ave': 1, 'wind_gust': 1.4, 'wind_dir': 8, 'rain': 95.4, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-25 15:31:49:pywws.weatherstation:new data {'illuminance': 11314.6, 'uv': 1, 'delay': 0, 'hum_in': 65, 'temp_in': 20.8, 'hum_out': 58, 'temp_out': 11.5, 'abs_pressure': 1002.9, 'wind_ave': 1, 'wind_gust': 1.4, 'wind_dir': 4, 'rain': 95.4, 'status': {'lost_connection': False, 'rain_overflow': False}}
2024-01-25 15:32:26:pywws.weatherstation:unexpected solar clock change
2024-01-25 15:32:26:pywws.weatherstation:setting solar clock 26.5591
2024-01-25 15:32:44:pywws.livelog:[Errno 110] Operation timed out

Traceback (most recent call last):

  File "/usr/local/lib/python3.9/dist-packages/pywws/livelog.py", line 83, in live_log
    for data, logged in datalogger.live_data(
  File "/usr/local/lib/python3.9/dist-packages/pywws/logdata.py", line 252, in live_data
    for data, ptr, logged in self.ws.live_data(logged_only=logged_only):

  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 553, in live_data
    new_data = self.get_data(old_ptr, unbuffered=True)
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 705, in get_data
    result = _decode(self.get_raw_data(ptr, unbuffered),
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 696, in get_raw_data
    self._data_block = self._read_block(self._data_pos)
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 760, in _read_block

    new_block = self.cusb.read_block(ptr)
  File "/usr/local/lib/python3.9/dist-packages/pywws/weatherstation.py", line 346, in read_block
    return self.dev.read_data(32)
  File "/usr/local/lib/python3.9/dist-packages/pywws/device_pyusb1.py", line 104, in read_data
    result = self.dev.read(0x81, size, timeout=1200)
  File "/usr/local/lib/python3.9/dist-packages/usb/core.py", line 1029, in read
    ret = fn(
  File "/usr/local/lib/python3.9/dist-packages/usb/backend/libusb1.py", line 864, in intr_read
    return self.__read(self.lib.libusb_interrupt_transfer,
  File "/usr/local/lib/python3.9/dist-packages/usb/backend/libusb1.py", line 954, in __read
    _check(retval)
  File "/usr/local/lib/python3.9/dist-packages/usb/backend/libusb1.py", line 602, in _check
    raise USBTimeoutError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBTimeoutError: [Errno 110] Operation timed out

I assume that again the questioning of the base station happened exactly at the moment when the sensor was sending the data.

I now increased the usb activity margin from 6 to 8.

Let's see.

Best regards,

Stefan

[Jim Easterbrook]

On 25/01/2024 15:36, 'runge....@googlemail.com' via pywws wrote:
>
> Unfortunately i had a hung up at the moment.
> It was running longer than before, nearly 2 days but it hung up now.
> The hung up happened exactly at this 20 min interval:
> 2024-01-25 15:31:49:pywws.weatherstation:live_data lost sync -6.29283
> 2024-01-25 15:31:49:pywws.weatherstation:old data {'illuminance':
> 11243.2, 'uv': 1, 'delay': 0, 'hum_in': 65, 'temp_in': 20.8, 'hum_out':
> 58, 'temp_out': 11.6, 'abs_pressure': 1002.9, 'wind_ave': 1,
> 'wind_gust': 1.4, 'wind_dir': 8, 'rain': 95.4, 'status':
> {'lost_connection': False, 'rain_overflow': False}}
> 2024-01-25 15:31:49:pywws.weatherstation:new data {'illuminance':
> 11314.6, 'uv': 1, 'delay': 0, 'hum_in': 65, 'temp_in': 20.8, 'hum_out':
> 58, 'temp_out': 11.5, 'abs_pressure': 1002.9, 'wind_ave': 1,
> 'wind_gust': 1.4, 'wind_dir': 4, 'rain': 95.4, 'status':
> {'lost_connection': False, 'rain_overflow': False}}
> 2024-01-25 15:32:26:pywws.weatherstation:unexpected solar clock change
> 2024-01-25 15:32:26:pywws.weatherstation:setting solar clock 26.5591

This time it's the solar data changing at an unexpected time. Solar data
is sent separately from the other external data, at 60 second intervals.
If both types of data are sent at the same time I expect the station
ignores both. I've never had a 3080 type station to study. I assume it's
another clock to track and avoid USB activity when it's expected.

> I assume that again the questioning of the base station happened exactly
> at the moment when the sensor was sending the data.
> I now increased the usb activity margin from 6 to 8.

I'm not sure that's going to help. It might be better to reduce it to 3
or 4 again.

[runge....@googlemail.com]

Ok thanks,

Is the usb activity margin the time added to the 48 sec to get the point in time to query the base station over USB?

So what i think to understand is:
The sensor sends every 48 sec + / - eventually clock drift.
The solar sensor sends every 60 sec +/- eventually clock drift.

The USB query must be outside of the 48 and 60 sec.

If i understand it correct the code just cares about the 48 sec because 1080 has no solar sensor.
Do you calculate the 48 sec interval always new, factoring in the clock drift you found from the last queries?
It seems really not easy to sync with 2 clocks and a drift to be considered.

Nevertheless the code fix seems to help, especially with the data which looks now very consistent.

I would like to solve the problem programmatically, but the raspberry pi4 does not support powering down single USB ports.
If i can not solve the problem at all i am thinking about adding a relay to the USB + cable and
disconnecting and connecting the positive wire by GPIO with help of the relay.

But let's first see if the problem can be reduced to a minimum with a good usb activity margin.

I am still wondering what changed the timing of the clocks.
Is it possible that the replacement of the 1,5V alkaline rechargeable batteries because one was leaking with
1,2V rechargeable once also make the clocks tick differently now?

Thanks and best regards,
Stefan

[Jim Easterbrook]

On 25/01/2024 17:16, 'runge....@googlemail.com' via pywws wrote:
>
> Is the usb activity margin the time added to the 48 sec to get the point
> in time to query the base station over USB?

It's the number of seconds either side of the time a reading is expected
to arrive when pywws ceases all USB activity. Suppose a reading is due
at 30 seconds and your margin is 3, pywws would then do this:

26.0 seconds - read data from station, expecting no change
26.5 seconds - read data from station, expecting no change
27.0 seconds - do nothing until 33.0
33.0 seconds - read data from station, expecting a change. If it's
changed then go to sleep until a few seconds before the next data is
expected.

This same avoidance strategy is used for the 48 seconds sensor clock,
the 60 seconds solar data clock and the 5 minute logged data clock. Your
status.ini file stores the three clock times and drift rates.

When data changes at an unexpected time the clock is invalidated and
pywws samples the data every half second until it changes, giving a new
clock time to avoid in future. This is when USB hangups are most likely.

> So what i think to understand is:
> The sensor sends every 48 sec + / - eventually clock drift.
> The solar sensor sends every 60 sec +/- eventually clock drift.
>
> The USB query must be outside of the 48 and 60 sec.

Yes.

> If i understand it correct the code just cares about the 48 sec because
> 1080 has no solar sensor.
> Do you calculate the 48 sec interval always new, factoring in the clock
> drift you found from the last queries?
> It seems really not easy to sync with 2 clocks and a drift to be considered.

It tracks three different clocks and avoids all of them.

> I would like to solve the problem programmatically, but the raspberry
> pi4 does not support powering down single USB ports.

Powering down the port won't make any difference, unless you are running
your station without batteries I suppose. I keep batteries in mine so it
keeps going if I power down my Pi.

> I am still wondering what changed the timing of the clocks.
> Is it possible that the replacement of the 1,5V alkaline rechargeable
> batteries because one was leaking with
> 1,2V rechargeable once also make the clocks tick differently now?

I've not tried rechargeable batteries. Normal AA cells last two years or
more in my outside sensors.

[runge....@googlemail.com]

Ok, thanks a lot for the explanation.

So you already consider all 3 clocks.

>When data changes at an unexpected time the clock is invalidated and
>pywws samples the data every half second until it changes, giving a new
>clock time to avoid in future. This is when USB hang ups are most likely.

Is it necessary to read every half second?

Can i try and adapt this to 1 second?

Yes i have no battery in the base station since my pi runs 24/7 and should always provide power.

When restarting it is only shortly interrupting the power.

>Normal AA cells last two years or more in my outside sensors.

I thought this is a problem since my station is charging the batteries with the solar panel. 

So i use rechargeable batteries.

But i can check to get 1,5V special ons, they can still be found.

Thanks,

Stefan

[Jim Easterbrook]

On 25/01/2024 19:09, 'runge....@googlemail.com' via pywws wrote:
> Ok, thanks a lot for the explanation.
> So you already consider all 3 clocks.
>
> >When data changes at an unexpected time the clock is invalidated and
> >pywws samples the data every half second until it changes, giving a new
> >clock time to avoid in future. This is when USB hang ups are most likely.
>
> Is it necessary to read every half second?
> Can i try and adapt this to 1 second?

Yes, edit weatherstation.py and change the value of 'min_pause'.

[runge....@googlemail.com]

Ok, thanks.

I guess i have to do some testing now.

I will report my progress if there a news.

The change in line 576 was perfect for me. 

Now let's see if i can get the  USB connection stable with the various parameters.

Thanks,

Stefan

[runge....@googlemail.com]

Hi Jim,

it is running now since 3 days without problem.

But i have changed two things in parallel.

I changed usb activity margin back from 6.0 to 4.0.
But i do not think this did the trick.

I also changed min_pause in line 457 from 0.5 to 2.
I used 2 to first have a bigger step to see effects faster.

After this change the logfile changed a lot.

I have nearly no "live_data lost sync" anymore. Before i had it every 0 mins.

No i have it 5 times in the last day.

What i recognize that there are more live_data missed:

2024-01-28 20:00:48:pywws.weatherstation:live_data missed

2024-01-28 20:01:36:pywws.weatherstation:live_data missed

2024-01-28 20:02:25:pywws.weatherstation:live_data missed
2024-01-28 20:03:12:pywws.weatherstation:live_data missed
2024-01-28 20:04:05:pywws.weatherstation:live_data missed

But it always solved itself and it was not too long.

I think this is perfectly what was to be expected with this change.

But i am not the expert and it could also be coincident with something else.

What do you think?

Can i test other values for you?

My plan would be to change min_pause to 1 second and see if it is also fine.

What do you think about being able to change min_pause also in wether.ini if necessary like in my case?

Thanks and best regards,

Stefan

[Jim Easterbrook]

On 29/01/2024 17:30, 'runge....@googlemail.com' via pywws wrote:
> it is running now since 3 days without problem.
>
> But i have changed two things in parallel.
>
> I changed usb activity margin back from 6.0 to 4.0.
> But i do not think this did the trick.
>
> I also changed min_pause in line 457 from 0.5 to 2.
> I used 2 to first have a bigger step to see effects faster.

I think that's too large - it reduces the accuracy of clock drift
measurements so might lead to more problems than it solves. But I can't
be sure.

> After this change the logfile changed a lot.
> I have nearly no "live_data lost sync" anymore. Before i had it every 0
> mins.
> No i have it 5 times in the last day.
>
> What i recognize that there are more live_data missed:
> 2024-01-28 20:00:48:pywws.weatherstation:live_data missed
> 2024-01-28 20:01:36:pywws.weatherstation:live_data missed
> 2024-01-28 20:02:25:pywws.weatherstation:live_data missed
> 2024-01-28 20:03:12:pywws.weatherstation:live_data missed
> 2024-01-28 20:04:05:pywws.weatherstation:live_data missed

With fewer readings being taken and a wider range of times to avoid it's
more likely that a change will be missed. But those are every 48 seconds
(roughly) which doesn't look good.

> But it always solved itself and it was not too long.
>
> I think this is perfectly what was to be expected with this change.
> But i am not the expert and it could also be coincident with something else.
> What do you think?

There are just too many variables in the system to be certain of anything.

> Can i test other values for you?
>
> My plan would be to change min_pause to 1 second and see if it is also fine.
> What do you think about being able to change min_pause also in
> wether.ini if necessary like in my case?

I don't think a general setting is needed. The current 0.5 seconds seems
to work OK in most cases - your station does seem to behave differently
from others.

Another factor is that as the various clocks drift with respect to each
other the system may become more or less likely to have problems. This
makes testing any changes quite difficult as you can't reproduce the
exact conditions that previously caused problems.

[runge....@googlemail.com]

Ok,

thanks for your answers.

Yes you are right it is very hard to be certain with so many variables.

But the last month it was never running for more than a day without hung up and now it runs already for 3,5 days.

So i will now change the min_pause to 1, i think this should be ok for my system and monitor it for a longer time.

You are right, even for me 0,5 worked worked fine the last years for me.

Thanks for all the help, i will try to work around on my system and i will update once this was running for a month or more without problem.

[runge....@googlemail.com]

Hi Jim,

thanks for the new fixed version.

And sorry that i have again a question.

In the meantime i played around a lot with the values.

For all tests i use usb activity margin = 4.0.

I still have the strange behaviour:

if i use min_pause = 0.5 or 1 or 1.5 the USB locks up very fast, but at least after 24 hours, it never worked longer without hang up.
As soon as i enter min_pause = 2 it seems to work fine for at least 48 hours.

I now want to rule out dumb configuration errors from me in my wether.ini.

I checked the docu and think i did something not standard.

I use  /usr/local/bin/pywws-livelog-daemon

I have 2 services MQTT and UNDERGROUND.

Where do i have to use them?

Only in the [LIVE] section?

I had underground in all sections [live], [logged], [hourly], [12 hourly], [daily] and MQTT in [live] and [logged].

If my understanding is correct this just caused duplicate entries in the backend at the logged, hourly, 12 hourly and daily intervall.

I think it makes no sense to call in my case the services in more than the live section.

Is my understanding correct?

Thanks and best regards,
Stefan

[Jim Easterbrook]

On 01/02/2024 15:52, 'runge....@googlemail.com' via pywws wrote:
>
> I still have the strange behaviour:
> if i use min_pause = 0.5 or 1 or 1.5 the USB locks up very fast, but at
> least after 24 hours, it never worked longer without hang up.
> As soon as i enter min_pause = 2 it seems to work fine for at least 48
> hours.

If it works for you then stick with it!

> I have 2 services MQTT and UNDERGROUND.
> Where do i have to use them?
> Only in the [LIVE] section?
>
> I had underground in all sections [live], [logged], [hourly], [12
> hourly], [daily] and MQTT in [live] and [logged].
> If my understanding is correct this just caused duplicate entries in the
> backend at the logged, hourly, 12 hourly and daily intervall.
> I think it makes no sense to call in my case the services in more than
> the live section.
> Is my understanding correct?

Yes, but having them in the [logged] section as well will allow pywws to
upload past data if you have a network outage or similar and want to
fill in the gap in your data. This is useful for Weather Underground but
probably not for MQTT, which I assume is only interested in current
conditions.