My experience upgrading from Weewx 3.92 to Weewx 4.0.0b12 on a system with both Python-2.7.17 and Python-3.8.1

[Joel Bion]

My system still has Python2 installed. I also have Python-3.8.1. Here's what I had to do to get weewx 4.0.0b12 working on my system, which supports both python2 and python3 simultaneously:

1. Since "python" on my system resolves to Python2 - I had to change the executable scripts in /home/weewx/bin to have hashbangs that referenced /usr/bin/python3
2. I had to install PyMySQL for Python3. MySQLdb/MySQL-Python just didn't work for me. PyMySQL worked like a charm.
3. I had to install Cheetah3.
4. I had to install Pillow for images support. The very old "imaging" package (I think v1.1.7?) didn't work for me with Python3.
5. As to other Python packages needed for weewx under Python3, I haven't checked. I had a good number installed already. The above (Cheetah3, PyMySQL, Pillow) were the ones I had to install because I did not already have them under Python3.
   
6. I wondered how old the stuff was in the bin/user directory - and there were a couple of extensions I no longer used, which I just removed, and then I upgraded my versions of __init__.py and extensions.py from the 4.0.0b12 source tree. To get the 'purpleair' extension working in Python3, I had to make a couple of syntax edits (print is a function, not a statement) and (exception handling uses the 'as' keyword). When I see the 'purpleair' extension working for a week or so, I will send these trivial edits up-stream.
   

The above got me running on a system supporting both Python2 and Python3. Total work was probably about 60-90 minutes; not bad! It was nice to see how relatively easy it was upgrading to WeeWx 4.0, and all of the changes I had to make had nothing to do with the Weewx code.

I'm scanning through the weewx doc to see if it references the newer Python3 modules, but with the work-week beginning, that may not happen right away.

At some point, when GCC 10.0 comes out, it will be time to rebuild my system from the ground up. (I use linuxfromscratch, which is sort of a 'build-it-yourself step-by-step distribution.) When I do that, I am going to try to have a python3-only system under which to try to run Weewx.

-Joel

[Thomas Keffer]

Hi, Joel

Thanks for your feedback!

Question: did you look through the install instructions in docs/setup.htm? How do they compare with your experience? Are they pretty much on target?

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/dd3aefaa-b407-4d35-808d-542acdeb3d80%40googlegroups.com.

[John Kline]

Assuming you used setup.py to install, a lot of this is covered in the documentation.  For example, you’ll find the python3 packages you need to install.  You’ll also see the steps to install are:

python3 ./setup.py build
sudo python3 ./setup.py install

With the above steps, you won’t need to hand edit the scripts to use python3.

To view the WeeWX 4 setup guide in HTML, click the link below:

https://htmlpreview.github.io/?https://raw.githubusercontent.com/weewx/weewx/master/docs/setup.htm

On Feb 17, 2020, at 6:02 AM, Joel Bion <jpb...@gmail.com> wrote:



[Joel Bion]

Thank you both for your swift replies!

Here's where my mistake was rooted: I assumed weewx 4.0 only supported Python3, and missed the point that it was written to support both Python2 and Python3 simultaneously. Therefore, I just followed the instructions in 'upgrading.htm' - and assumed that the upgrade steps would upgrade my weewx to one supporting Python3.

But, since weewx supports both, just executing "./setup.py ..." as mentioned in 'upgrading.htm' chooses the python based on the "env python" in the first line of the setup.py script, it meant that a Python2 based upgrade happened, creating binaries that referenced "/usr/bin/python" vs "/usr/bin/python3".

Since the errors reported upon weewx startup (and later, report generation) were so trivial to resolve in a couple of minutes (by just pulling in Cheetah3, PyMysql and Pillow with pip3), I never dug more deeply into the documentation to see what was wrong. It was so easy to resolve.

Most of my time was spent scanning through the weewx-purpleair extension. Since it had been written to support only Python2, I didn't want to make just the syntactic changes, but convince myself that the rest of the script would actually do what was intended. It's a short script - but the reason I am taking a while to send the changes upstream is that I want to see if the divisions (/)  in the script should be changed to floor divisions (//) to mimic Python2's behavior - and I want to make sure the results look reasonable over a period of time before suggesting them upstream.

Yes, if I were installing weewx from scratch, the few minutes spent at the start would not have been needed, as the setup.htm documentation does indeed mention python3 compatible packages. :)

Overall, it was a very easy experience, with weewx working flawlessly, with the only possible suggestion from me being this: that there be some sort of text in upgrading.htm about upgrading weewx from python2 to python3.

-Joel

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-de...@googlegroups.com.

[Thomas Keffer]

The other option would be to make python3 the default in setup.py.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/910cd84f-c1a6-4127-bb67-e623d6f486d0%40googlegroups.com.

[Vince Skahan]

On Monday, February 17, 2020 at 7:23:26 AM UTC-8, Joel Bion wrote:

Most of my time was spent scanning through the weewx-purpleair extension. Since it had been written to support only Python2, I didn't want to make just the syntactic changes, but convince myself that the rest of the script would actually do what was intended.

I run that one too.  Works fine in both python versions after running it through "2to3 -w" to clean up the syntax.

What I did was a little simpler than how you battled it, I think:

• ran python3 setup.py (to get a python3 based installation of weewx)
• installed all the extensions I use
• checked each of the extensions with "2to3" which so far has done a great job finding python3 edits needed
• ran "2to3 -w" for the extensions needing tweaking
• restarted weewx and verified everything worked (or at least didn't crash out due to python3)
• provided pull requests upstream for anything I found that didn't work as-is in both python2+3

So far the only thing I can't get to do anything is the forecast extension.  I got it to not blow up, but it isn't downloading the forecasts successfully.   This one was a low enough priority for me that I'm just going to wait until the official version catches up someday.

[John Kline]

I have weewx-forecast working with WeeWx 4 and Python2 and 3.  It does correctly download forecasts.

https://github.com/chaunceygardiner/weewx-forecast 

There were a couple of Python3 gotchas, but the big one is this:

def __init__(self, target, *args):
	self._target = target
	self._args = args
	threading.Thread.__init__(self)

The threading.Thread.__init__(self) line is wiping out the two assignments above it.

Move that __init__  up to the first line.

On Feb 17, 2020, at 8:18 AM, Vince Skahan <vince...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/55c86fc3-3c08-4b7c-8314-3b329735eb4b%40googlegroups.com.

[Joel Bion]

2to3 is a nice tool! The only thing it didn't cover was the 'division' operations in the file. Python2 vs Python3 handle division differently. So, I want to think about whether to force floor division with '//' or not in the couple places division is used. The answer should be pretty clear; I just haven't had the uninterrupted time to think about it in these 2 cases.

-Joel

[Vince Skahan]

On Monday, February 17, 2020 at 11:51:47 AM UTC-8, John Kline wrote:

I have weewx-forecast working with WeeWx 4 and Python2 and 3.  It does correctly download forecasts.

https://github.com/chaunceygardiner/weewx-forecast 

Works great.  Thanks !

Did you send a PR to Matthew to catch up the master copy ?

[John Kline]

Did you send a PR to Matthew to catch up the master copy ?

I got the feeling Matthew wasn’t interested.  I’ll try again. 

On Feb 17, 2020, at 7:17 PM, Vince Skahan <vince...@gmail.com> wrote:



--

You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/206d0b99-51be-4585-8ecb-5fb769ace666%40googlegroups.com.

[John Kline]

PR sent.

On Feb 17, 2020, at 8:53 PM, John Kline <jo...@johnkline.com> wrote:



[mwall]

On Monday, February 17, 2020 at 11:53:29 PM UTC-5, John Kline wrote:

Did you send a PR to Matthew to catch up the master copy ?

I got the feeling Matthew wasn’t interested.  I’ll try again. 

not at all the case. i've been swamped with real life and real work, and trying to get weewx-core stuff sorted before cleaning up all of my extensions.

so your changes are MOST welcome - it will save me massive amounts of time.

its just taking me awhile to get to the merging of them

m

[John Kline]

Got it.  In addition to the PR for this,  I just sent you an ip100 PR with a bug fix and updates for WeeWX 4/Python 3.  Like the change for forecast, these changes will require the extension to run under WeeWX 4.  (I’m in the, “use the previous version of the extension for WeeWX 3” camp.) You’ll have to decide how you feel about that.

On Feb 22, 2020, at 6:33 AM, mwall <mw...@users.sourceforge.net> wrote:



--

You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/16c146ff-a54c-4cdb-8aed-6178bcc19470%40googlegroups.com.