Move to Python 3?
290 views
Skip to first unread message
WindnFog
Jun 3, 2020, 9:24:20 AM6/3/20
to weewx-user
I updated my Pi 4 to Raspberry Pi OS and weewx to version 4.1.1. Everything seems A-OK. Great job, fellows!
To move entirely into the 21st century, I want to start using python 3 (already installed) instead of python 2.7. Where do I find the documentation to make the switch? Any "gotchas" I of which I need to be aware?
- Paul VE1DX
WindnFog
Jun 3, 2020, 10:21:30 AM6/3/20
to weewx-user
Well, an Internet search suggests this is not just flipping a pointer or symlink!
Seeing as my system is up to date from an O/S and weewx standpoint, and I have written no extensions or drivers, etc., would the following work?
# # required apt-get install python3-configobj apt-get install python3-pil apt-get install python3-cheetah apt-get install python3-usb apt-get install python3-serial # optional apt-get install python3-ephem apt-get install mariadb-client apt-get install python3-mysqldb # weewx apt-get install python3-weewx
Seems to be a bit risky and not for the faint of heart. What say, gurus? I can leave it at python 2.7.16 for now, although at some point that's going away . . .
vince
Jun 3, 2020, 1:36:04 PM6/3/20
to weewx-user
On Wednesday, June 3, 2020 at 7:21:30 AM UTC-7, WindnFog wrote:
Seems to be a bit risky and not for the faint of heart. What say, gurus? I can leave it at python 2.7.16 for now, although at some point that's going away . . .
Not risky in the least, but also no 'technical' reason to switch at this time if you don't want to.
If the original poster updated their Raspbian and used the dpkg installer version, they're already using python3 automagically almost certainly.
WindnFog
Jun 3, 2020, 4:01:23 PM6/3/20
to weewx-user
Hi Vince,
To clarify, I am the original poster. After the first post, I did some digging with Google. I found the github discussion by Matthew Wall et al. about the permutations and combinations of various Linux distros combined with either Python 2.7 and Python 3. I did do the upgrade of my Pi to Raspberry Pi O/S from Raspian Buster 10. As you suggest, it automagically upgraded weewx to 4.1.1 at the same time. Everything seems to be working well. On my system I have both versions of Python:
$ python --version
Python 2.7.16
$ python3 --version
Python 3.7.3
What's confusing me is the log says python2 at the beginning of each line:
Jun 03 16:30:16 rhubarb python2[1164]: weewx[1164] INFO weewx.manager: Added record 2020-06-03 16:30:00 ADT (1591212600) to database 'weewx.sdb'
Jun 03 16:30:16 rhubarb python2[1164]: weewx[1164] INFO weewx.manager: Added record 2020-06-03 16:30:00 ADT (1591212600) to daily summary in 'weewx.sdb'
Jun 03 16:30:19 rhubarb weewxd[1164]: restx: Weather365: Data uploaded to https://channel1.weather365.net/stations/index.php is: (dew2m=6.0&datum=202006031630&humidex=8.5&lat=44.73794&soiltemp=-66.7&prec_time=60&et=0.00000&alt=85.0&soilmoisture=4.0&t2m=8.5&leafwetness=4.0&soiltemp2=-67.8&soiltemp3=-66.7&soiltemp4=-67.8&long=-63.37766&rainh=0.00&rxsignal=96&soilmoisture2=8.0&soilmoisture3=4.0&soilmoisture4=40.0&windrun=0.3&utcstamp=1591212600&wchill=8.5&txbattery=0&winddir=158&windspeed=0.9&windgust=2.7&press=1003.658&relhum=84&appTemp=6.9&heat=8.5&stationid=RN5CDP36Y3&leafwetness2=8.0&raind=10.00&rainrate=0.00&cloudbase=336.5)
Jun 03 16:30:19 rhubarb python2[1164]: weewx[1164] INFO weewx.restx: WOW: Published record 2020-06-03 16:30:00 ADT (1591212600)
Jun 03 16:30:19 rhubarb python2[1164]: weewx[1164] INFO weewx.restx: AWEKAS: Published record 2020-06-03 16:30:00 ADT (1591212600)
Jun 03 16:30:19 rhubarb python2[1164]: weewx[1164] INFO weewx.restx: Weather365: Published record 2020-06-03 16:30:00 ADT (1591212600)
Jun 03 16:30:19 rhubarb python2[1164]: weewx[1164] INFO weewx.restx: WeatherCloud: Published record 2020-06-03 16:30:00 ADT (1591212600)
Jun 03 16:30:20 rhubarb python2[1164]: weewx[1164] INFO weewx.restx: PWSWeather: Published record 2020-06-03 16:30:00 ADT (1591212600)
Jun 03 16:30:20 rhubarb python2[1164]: weewx[1164] INFO weewx.cheetahgenerator: Generated 14 files for report StandardReport in 0.92 seconds
Jun 03 16:30:20 rhubarb python2[1164]: weewx[1164] INFO weewx.imagegenerator: Generated 13 images for report StandardReport in 0.46 seconds
Jun 03 16:30:20 rhubarb python2[1164]: weewx[1164] INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx
Jun 03 16:30:20 rhubarb python2[1164]: weewx[1164] INFO paramiko.transport: Connected (version 2.0, client OpenSSH_7.2p2)
Jun 03 16:30:21 rhubarb python2[1164]: weewx[1164] INFO paramiko.transport: Authentication (publickey) successful!
Jun 03 16:30:21 rhubarb python2[1164]: weewx[1164] INFO paramiko.transport.sftp: [chan 0] Opened sftp connection (server version 3)
Jun 03 16:30:26 rhubarb python2[1164]: weewx[1164] INFO paramiko.transport.sftp: [chan 0] sftp session closed.
Jun 03 16:30:26 rhubarb python2[1164]: weewx[1164] INFO user.sftp: sftpgenerator: transferred 29 files in 5.77 seconds
Jun 03 16:30:28 rhubarb python2[1164]: weewx[1164] INFO weeutil.rsyncupload: rsync'd 30 files (1,217,636 bytes) in 2.05 seconds
Jun 03 16:30:39 rhubarb python2[1164]: weewx[1164] INFO weewx.restx: CWOP: Published record 2020-06-03 16:30:00 ADT (1591212600)
That python2[1164] reference
might be a hiccup in the log. If my system is using both of the new
versions pf python and Cheetah, then all is good. If it's using python
2.7, then I might have inadvertently created some kind of hybrid. The
reference to paramiko is new, and that's because I am using Matthew's sftp extension. I was doing so well before before the upgrade, and the log didn't mention
paramiko. That's probably a red herring in the sense that I went from
weewx 3.9.3 to 4.1.1 and likely has nothing to do with the python
version weewx is using. I don't want to have created something that
will break on future upgrades, and that's my only concern. Is there an easy way to tell which version of python and Cheetah that weewx is using?
On Wednesday, June 3, 2020 at 10:24:20 AM UTC-3, WindnFog wrote:I updated my Pi 4 to Raspberry Pi OS and weewx to version 4.1.1. Everything seems A-OK. Great job, fellows!
To
move entirely into the 21st century, I want to start using python 3
(already installed) instead of python 2.7. Where do I find the
documentation to make the switch? Any "gotchas" I of which I need to be
aware?
- Paul VE1DX
On Wednesday, June 3, 2020 at 10:24:20 AM UTC-3, WindnFog wrote:
gjr80
Jun 3, 2020, 4:19:41 PM6/3/20
to weewx-user
Simple, look at the log....
The first few lines after a WeeWX startup tell you exactly what python version is being used by WeeWX, eg:
Jun 3 11:15:55 weewx weewx[2712] INFO __main__: Initializing weewx version 4.1.1
Jun 3 11:15:55 weewx weewx[2712] INFO __main__: Using Python 3.8.2 (default, Apr 27 2020, 15:53:34) #012[GCC 9.3.0]
Jun 3 11:15:55 weewx weewx[2712] INFO __main__: Platform Linux-5.4.0-33-generic-x86_64-with-glibc2.29
Gary
Neville Davis
Jun 3, 2020, 4:29:35 PM6/3/20
to weewx-user
When I recently built up a new Pi3 system I defaulted to python3 so that any call to just python uses python3. This removed any confusion :). The issue I had with this was that some python testing scripts I used for my home brew weather station had a number of syntax (primarily printing) issues..fixed these and as far as python concerned no issues with weewxV4.0,
I only did it to remain current not for any other reason.
Neville
WindnFog
Jun 3, 2020, 6:30:05 PM6/3/20
to weewx-user
Thanks, Neville. It's using Python 2.7.16. How do I switch the default to Python 3, and of I do, is that likely to cause any problems? My guess is no, but "if it ain't broke, don't fix it" might apply here.
On Wednesday, June 3, 2020 at 10:24:20 AM UTC-3, WindnFog wrote:
Daniel Rich
Jun 3, 2020, 6:52:09 PM6/3/20
to weewx-user
I posted the issue I ran into converting to Python 3 a few days back, but never received any replies. It may work, but you may also run into issues if you have any add-ons with Python 2.x specific code.
| Dan Rich <dr...@employees.org> | http://www.employees.org/~drich/ |
| "Step up to red alert!" "Are you sure, sir? | |
| It means changing the bulb in the sign..." | |
| - Red Dwarf (BBC) |
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/26d4b49f-0478-450c-8879-0c1ba52e63a9%40googlegroups.com.
Neville Davis
Jun 3, 2020, 6:55:16 PM6/3/20
to weewx-user
Read this
if you use the alternative option it is system-wide. This is the procedure I followed. But is it needed......I did it just to be current it only caused minor problems for me but no issues with operational python scripts I had
Neville
WindnFog
Jun 4, 2020, 3:28:04 PM6/4/20
to weewx-user
That worked, Neville:
python --version
Python 3.7.3
However, weewx still grabs python 2.7 . . . perhaps it's a bit too early (new Raspberry Pi O/S and a new weewx version 4.1.1 for someone at my level of expertise to monkey around with this yet. It's working, and while I have a "just made" backup on a separate SD card, there seem to be a lot of threads discussing this. I don't run anything but weewx on the Pi (i.e. no other python scripts), but experience has taught me not to change too many things at the same time. I suspect this is more related to the O/S that's only a week or so old as opposed to weewx. 2.7 seems to be working OK. I don't understand why weewx is not picking up 3.7.3, but there's no rush.
python --version
Python 3.7.3
However, weewx still grabs python 2.7 . . . perhaps it's a bit too early (new Raspberry Pi O/S and a new weewx version 4.1.1 for someone at my level of expertise to monkey around with this yet. It's working, and while I have a "just made" backup on a separate SD card, there seem to be a lot of threads discussing this. I don't run anything but weewx on the Pi (i.e. no other python scripts), but experience has taught me not to change too many things at the same time. I suspect this is more related to the O/S that's only a week or so old as opposed to weewx. 2.7 seems to be working OK. I don't understand why weewx is not picking up 3.7.3, but there's no rush.
Paul VE1DX
Tom Keffer
Jun 4, 2020, 4:06:45 PM6/4/20
to weewx-user
Depends on how weewxd is executed. If you (or the daemon init script) run the simple command
weewxd /etc/weewx/weewx.conf
then the "shebang" is put into use. What is a shebang? If the first line of a file starts with the characters '#!', that's a shebang. It instructs the shell how to run the file. For weewx v3.x, it looks something like this:
#!/usr/bin/env python
It means, use the program "python" to run the file. Which version of python will depend on the defaults of your system. In your case, this is probably python 2.7.
On the other hand, if you (or your daemon init script) start weewxd with an explicit python interpreter:
python3 weewxd /etc/weewx/weewx.conf
then that's the version of Python that gets used. The shebang does not come into play. Personally, this is the approach I use, because I always know what version of Python I am getting.
Hope this helps.
-tk
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7e72ef46-3368-4f26-ad42-ac3cc89c92aao%40googlegroups.com.
mwall
Jun 4, 2020, 4:07:11 PM6/4/20
to weewx-user
On Thursday, June 4, 2020 at 3:28:04 PM UTC-4, WindnFog wrote:
That worked, Neville:
python --version
Python 3.7.3
However, weewx still grabs python 2.7 . . . perhaps it's a bit too early (new Raspberry Pi O/S and a new weewx version 4.1.1 for someone at my level of expertise to monkey around with this yet. It's working, and while I have a "just made" backup on a separate SD card, there seem to be a lot of threads discussing this. I don't run anything but weewx on the Pi (i.e. no other python scripts), but experience has taught me not to change too many things at the same time. I suspect this is more related to the O/S that's only a week or so old as opposed to weewx. 2.7 seems to be working OK. I don't understand why weewx is not picking up 3.7.3, but there's no rush.
if you install weewx using setup.py, then the python will be whatever python you used when you did 'xxx setup.py install'
if you install weewx using apt/yum/zypper, then the python will default to whatever python is specified in the package you used. for example, the python3-weewx deb package uses python3. however, for these installations you can easily change the python by modifying /etc/default/weewx.
if you do that, beware that you might not have the correct dependencies. if that is the case, then just manually install the dependencies.
this is independent of whatever mechanism the operating system might have for specifying which python is 'default'
so in your case, assuming that you installed the python-weewx package using apt, you must manually install the python3 dependencies (you listed the apt commands for that earlier in this thread), then modify the python entry in /etc/default/weewx
m
WindnFog
Jun 5, 2020, 3:40:53 PM6/5/20
to weewx-user
Thanks for the pointers, etc. Matthew. I did what you suggested with the dependencies, etc., and all went well (as in no errors upgrading reported.) Also, as you said, I changed /etc/weewx/default from:
WEEWX_PYTHON=python2
WEEWX_BINDIR=/usr/share/weewx
WEEWX_BIN=/usr/bin/weewxd
WEEWX_CFG=/etc/weewx/weewx.conf
--- to --
WEEWX_PYTHON=python3
WEEWX_BINDIR=/usr/share/weewx
WEEWX_BIN=/usr/bin/weewxd
WEEWX_CFG=/etc/weewx/weewx.conf
However, starting it with:
weewxd /etc/weewx/weewx.conf
Gives the following error:
Traceback (most recent call last):
File "/usr/share/weewx/weewxd", line 261, in <module>
main()
File "/usr/share/weewx/weewxd", line 148, in main
engine = weewx.engine.StdEngine(config_dict)
File "/usr/share/weewx/weewx/engine.py", line 75, in __init__
self.loadServices(config_dict)
File "/usr/share/weewx/weewx/engine.py", line 138, in loadServices
obj = weeutil.weeutil.get_object(svc)(self,config_dict)
File "/usr/share/weewx/weeutil/weeutil.py", line 1093, in get_object
mod = __import__(module)
File "/usr/share/weewx/user/wcloud.py", line 134
except KeyError, e:
^
SyntaxError: invalid syntax
I *think* I am missing something subtle, but I can't figure it out. Any further suggestions? I went with your post instead of Tom's for now regarding starting it because I didn't want to introduce yet another variable. Either way, does that error tell you anything? I didn't touch my config files or anything else. I can post them if you think they'd be useful. Probably unrelated, I'm using your sftp extension. Everything works fine with Python 2.7.
WEEWX_PYTHON=python2
WEEWX_BINDIR=/usr/share/weewx
WEEWX_BIN=/usr/bin/weewxd
WEEWX_CFG=/etc/weewx/weewx.conf
--- to --
WEEWX_PYTHON=python3
WEEWX_BINDIR=/usr/share/weewx
WEEWX_BIN=/usr/bin/weewxd
WEEWX_CFG=/etc/weewx/weewx.conf
However, starting it with:
weewxd /etc/weewx/weewx.conf
Gives the following error:
Traceback (most recent call last):
File "/usr/share/weewx/weewxd", line 261, in <module>
main()
File "/usr/share/weewx/weewxd", line 148, in main
engine = weewx.engine.StdEngine(config_dict)
File "/usr/share/weewx/weewx/engine.py", line 75, in __init__
self.loadServices(config_dict)
File "/usr/share/weewx/weewx/engine.py", line 138, in loadServices
obj = weeutil.weeutil.get_object(svc)(self,config_dict)
File "/usr/share/weewx/weeutil/weeutil.py", line 1093, in get_object
mod = __import__(module)
File "/usr/share/weewx/user/wcloud.py", line 134
except KeyError, e:
^
SyntaxError: invalid syntax
I *think* I am missing something subtle, but I can't figure it out. Any further suggestions? I went with your post instead of Tom's for now regarding starting it because I didn't want to introduce yet another variable. Either way, does that error tell you anything? I didn't touch my config files or anything else. I can post them if you think they'd be useful. Probably unrelated, I'm using your sftp extension. Everything works fine with Python 2.7.
I put it back to weewx 4.1.1 using python2 from my SD backup. All works fine, except that I'm no farther ahead . . . still using python 2.7.
Paul VE1DX
Tom Keffer
Jun 5, 2020, 3:46:07 PM6/5/20
to weewx-user
Your version of wcloud has not been ported to Python 3. The current version has. Just upgrade it.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/678b895a-5097-4e86-be6b-98af8dbbe003o%40googlegroups.com.
vince
Jun 5, 2020, 3:55:37 PM6/5/20
to weewx-user
On Friday, June 5, 2020 at 12:40:53 PM UTC-7, WindnFog wrote:
except KeyError, e:
^
SyntaxError: invalid syntax
This is becoming our most frequently asked question.
Anytime you see this, it says you are running old python2 code vs. the python3 interpreter. The code needs updating, or you need to stick with python2 until the author updates their extension/driver/skin to be python3-compatible.
WindnFog
Jun 5, 2020, 5:13:18 PM6/5/20
to weewx-user
Progress! One issue left:
python3[8081]: weewx[8081] INFO user.sftp: sftpgenerator: upload not possible: No module named 'pysftp'
Any thoughts on how to fix this?
Paul VE1DX
To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.
vince
Jun 5, 2020, 6:11:10 PM6/5/20
to weewx-user
On Friday, June 5, 2020 at 2:13:18 PM UTC-7, WindnFog wrote:
Progress! One issue left:python3[8081]: weewx[8081] INFO user.sftp: sftpgenerator: upload not possible: No module named 'pysftp'Any thoughts on how to fix this?
Try googling the error you got - like everything after the word 'sftpgenerator:' above.
WindnFog
Jun 6, 2020, 9:26:37 AM6/6/20
to weewx-user
Matthew/Tom/Vince:
Thank you all for you help.
On Thursday, June 4, 2020 at 5:07:11 PM UTC-3, mwall wrote:
Reply all
Reply to author
Forward
0 new messages