Summertime settings did something here

[Joakim Thulin]

Getting the following error messages since summertime change this morning, pywws service restarts automatically every minute now. Could someone lead me inthe right direction to resolve this:

2021-03-28 07:57:15:pywws.livelog:2021-03-28 02:28:01

Traceback (most recent call last):

  File "/usr/local/lib/python3.7/dist-packages/pywws/livelog.py", line 91, in live_log

    tasks.do_live(data)

  File "/usr/local/lib/python3.7/dist-packages/pywws/regulartasks.py", line 211, in do_live

    self._do_common(now, sections, live_data=calib_data)

  File "/usr/local/lib/python3.7/dist-packages/pywws/regulartasks.py", line 149, in _do_common

    self.services[name].upload(live_data=live_data, options=options)

  File "/usr/local/lib/python3.7/dist-packages/pywws/service/__init__.py", line 340, in upload

    if self.queue_data(timestamp, data):

  File "/usr/local/lib/python3.7/dist-packages/pywws/service/__init__.py", line 307, in queue_data

    OK = super(CatchupDataService, self).queue_data(timestamp, data)

  File "/usr/local/lib/python3.7/dist-packages/pywws/service/__init__.py", line 281, in queue_data

    prepared_data = self.prepare_data(data)

  File "/usr/local/lib/python3.7/dist-packages/pywws/service/__init__.py", line 290, in prepare_data

    data_str = self.templater.make_text(self.template_file, data)

  File "/usr/local/lib/python3.7/dist-packages/pywws/template.py", line 578, in make_text

    for text in self.process(live_data, template_file):

  File "/usr/local/lib/python3.7/dist-packages/pywws/template.py", line 460, in process

    x = eval(command[1])

  File "<string>", line 1, in <module>

  File "/usr/local/lib/python3.7/dist-packages/pywws/template.py", line 343, in rain_day

    midnight = timezone.local_midnight(data['idx'])

  File "/usr/local/lib/python3.7/dist-packages/pywws/timezone.py", line 131, in local_midnight

    return self.local_replace(dt, hour=0, minute=0, second= 0)

  File "/usr/local/lib/python3.7/dist-packages/pywws/timezone.py", line 107, in local_replace

    dst_offset = self.dst(local_time)

  File "/usr/local/lib/python3.7/dist-packages/pywws/timezone.py", line 68, in dst

    return self.local.dst(dt)

  File "/usr/local/lib/python3.7/dist-packages/pytz/tzinfo.py", line 461, in dst

    dt = self.localize(dt, is_dst)

  File "/usr/local/lib/python3.7/dist-packages/pytz/tzinfo.py", line 341, in localize

    raise NonExistentTimeError(dt)

pytz.exceptions.NonExistentTimeError: 2021-03-28 02:28:01

2021-03-28 07:57:17:pywws.service.metoffice:OK

[bee...@gmail.com]

Same problem on my system. I can't offer a solution, just the thought that you're not alone <grin>.

[richar...@gmail.com]

Hello,

I had the same issue, I fixed it by rolling back to the previous version, 

sudo pip3 install pywws==20.1.0

That'll give time for Jim to see what is going on.

[Joakim Thulin]

Thanks a lot, feels good not to be alone :-)

Great tip to rollback to 20.1

[Jim Easterbrook]

This NonExistentTimeError exception was originally cured by passing
'is_dst=False' to some of the timezone twiddling library functions. When
I tried using Python 3.6 I found the 'is_dst' option had been removed as
it was now the default. Maybe the change isn't only in Python's datetime
library but also in the pytz library. Try updating pytz.

In pywws 21.3.0 running the pywws.timezone module from the command line
(no need to stop pywws logging) should test the timezone conversions
around DST changes. Unfortunately I only get a proper test of all this
stuff once (or maybe twice) a year. (-:

--
Jim Easterbrook <http://www.jim-easterbrook.me.uk/>

[richar...@gmail.com]

Dear Jim,

Thanks for the advice, I updated pytz (and all the other dependencies too) and ran pywws.timezone , and get the following output:

using timezone Europe/London

current time

2021-03-28 09:02:35 UTC

2021-03-28 10:02:35+01:00 BST

2021-03-28 09:02:35+00:00 UTC

most recent 9am 2021-03-28 08:00:00 UTC

most recent 9pm 2021-03-27 21:00:00 UTC

Ambiguous / missing times in "Europe/London"

2020-03-28 23:15:00 UTC = 2020-03-28 23:15:00+00:00 GMT

2020-03-29 00:15:00 UTC = 2020-03-29 00:15:00+00:00 GMT

2020-03-29 01:15:00 UTC = 2020-03-29 02:15:00+01:00 BST

2020-03-29 02:15:00 UTC = 2020-03-29 03:15:00+01:00 BST

2020-10-24 23:15:00 UTC = 2020-10-25 00:15:00+01:00 BST

2020-10-25 00:15:00 UTC = 2020-10-25 01:15:00+01:00 BST

2020-10-25 01:15:00 UTC = 2020-10-25 01:15:00+00:00 GMT

2020-10-25 02:15:00 UTC = 2020-10-25 02:15:00+00:00 GMT

DST transitions in "America/St_Johns"

midnight 2018-03-11 03:30:00 UTC 2018-03-11 00:00:00-03:30 NST

midnight 2018-03-12 02:30:00 UTC 2018-03-12 00:00:00-02:30 NDT

midnight 2018-11-04 02:30:00 UTC 2018-11-04 00:00:00-02:30 NDT

midnight 2018-11-05 03:30:00 UTC 2018-11-05 00:00:00-03:30 NST

Is this correct, not sure about the Ambiguous / missing times in "Europe/London"

[Jim Easterbrook]

It's exactly correct (and doesn't throw an exception). It just shows
that in UK time 2020-10-25 01:15:00 has two possible UTC values and
2020-03-29 01:15:00 doesn't exist.

[richar...@gmail.com]

Thanks for the help Jim, I can confirm that pywws is successfully running using version 21.3.0 post updating the dependancies.

Is there a way to automatically update the various dependencies when pywws is updated?

[Jim Easterbrook]

On 28/03/2021 10:15, richar...@gmail.com wrote:
> Thanks for the help Jim, I can confirm that pywws is successfully
> running using version 21.3.0 post updating the dependancies.
>
> Is there a way to automatically update the various dependencies when
> pywws is updated?

I think any dependencies listed in setup.py get updated if you do 'pip
install -U pywws', but most dependencies aren't in setup.py for good
reasons:
1/ You have a choice of usb libraries
2/ You might have installed some from your operating system package
manager and do not want to break anything else.
3/ Many dependencies are optional anyway.

[Geeky Tim]

How can I get pywws restarted? 

[richar...@gmail.com]

Hello,

Run the following command:

sudo pip3 install --upgrade pytz

It should fix everything, you can confirm this by running:

python3 -m pywws.timezone

If it runs and doesn't crash everything is OK.

If you are still having issues, ensure all the dependencies are up to date, a list of them can be found here: pywws ReadTheDocs Dependancies

[Tim Richardson]

Perfect! Thank you :-)

Thanks

Tim Richardson

Twitter: @Geeky_Tim

--
You received this message because you are subscribed to the Google Groups "pywws" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pywws+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pywws/fbbed115-0a4b-48af-bbad-6b7ea453aa71n%40googlegroups.com.

[Jim Easterbrook]

On 28/03/2021 09:50, Jim Easterbrook wrote:
> On 28/03/2021 08:35, richar...@gmail.com wrote:
>>
>> I had the same issue, I fixed it by rolling back to the previous version,
>>
>> sudo pip3 install pywws==20.1.0
>>
>> That'll give time for Jim to see what is going on.
>
> This NonExistentTimeError exception was originally cured by passing
> 'is_dst=False' to some of the timezone twiddling library functions. When
> I tried using Python 3.6 I found the 'is_dst' option had been removed as
> it was now the default. Maybe the change isn't only in Python's datetime
> library but also in the pytz library. Try updating pytz.

The change isn't (directly) because of Python 3.6, it's to do with
tzlocal. I somehow managed to have tzlocal version 3.0b1 on my desktop
machine (Python 3.6) but tzlocal version 2.1 on my Pi (Python 3.5) which
is where I run pywws.

pip shouldn't normally install beta versions, so I'm not sure how I came
to have tzlocal version 3.0b1. However, it has alerted me to the fact
that tzlocal will eventually drop use of pytz.

For now I recommend sticking with pywws version 20.1.0 and tzlocal
version 2.1, regardless of your Python version.

[Jim Easterbrook]

On 30/03/2021 07:36, Jim Easterbrook wrote:
>
> The change isn't (directly) because of Python 3.6, it's to do with
> tzlocal. I somehow managed to have tzlocal version 3.0b1 on my desktop
> machine (Python 3.6) but tzlocal version 2.1 on my Pi (Python 3.5) which
> is where I run pywws.
>
> pip shouldn't normally install beta versions, so I'm not sure how I came
> to have tzlocal version 3.0b1. However, it has alerted me to the fact
> that tzlocal will eventually drop use of pytz.
>
> For now I recommend sticking with pywws version 20.1.0 and tzlocal
> version 2.1, regardless of your Python version.

I've just committed to GitHub a new time_zone object (to replace
timezone) and changes to all the modules that used timezone. If anyone
is in a position to test this I'd welcome any feedback. It seems to work
OK for me.