Migration from 4.10.2 (setup.py) to 5.0 (pip install) problems

[bgra...@umw.edu]

Hello,

As an "old" user of weewx (much appreciated for the software!), I'm having problems with migration. Not being much of a programmer, can someone point out my error below? Thanks in advance. Bob

(weewx-venv) bg@n4mrv:~$ weewxd --config=/home/weewx/weewx.conf
Traceback (most recent call last):
  File "/home/bg/weewx-venv/bin/weewxd", line 8, in <module>
    sys.exit(main())
  File "/home/bg/weewx-venv/lib/python3.10/site-packages/weewxd.py", line 151, in main
    engine = weewx.engine.StdEngine(config_dict)
  File "/home/bg/weewx-venv/lib/python3.10/site-packages/weewx/engine.py", line 90, in __init__
    self.loadServices(config_dict)
  File "/home/bg/weewx-venv/lib/python3.10/site-packages/weewx/engine.py", line 158, in loadServices
    obj = weeutil.weeutil.get_object(svc)(self, config_dict)
  File "/home/bg/weewx-venv/lib/python3.10/site-packages/weeutil/weeutil.py", line 1352, in get_object
    mod = __import__(module)
  File "/home/weewx/bin/user/purple.py", line 25, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

[Tom Keffer]

One of your extensions, purple, needs the Python package "requests" in order to operate. You can install it like this:

source ~/weewx-venv/bin/activate

python3 -m pip install requests

This will install the missing "requests" package inside the virtual environment.

Incidentally, this is a good example of why it's a good thing to keep the number of packages that an extension requires down to a minimum.

[vince]

I'd strongly recommend that folks keep track of what they install during the transition so that the 'next' time they do a clean installation they have a paper trail of things they add to core weewx for their station.   Most people are likely going to miss reinstalling additional modules like 'requests' and 'paho-mqtt' at least once or twice....

Just activate your venv and run a 'python3 -m pip list' and save the output.  Super simple.

[bgra...@umw.edu]

Thanks Vince and Tom. Below is the listing of packages from a pip listing. ( bg@n4mrv:~$ python3 -m pip list  followed by  (weewx-venv) bg@n4mrv:~$ python3 -m pip list ) I  guess the problem is that I have lots but they need to be inside venv. Sorry, I don't know much about virtual sessions.

Is there a way to move these over or do I need to reinstall one by one? When I migrate to venv do I reinstall all the extensions? 

After installing "requests" and running weewxd I get the following error:

ModuleNotFoundError: No module named 'dateutil'
(weewx-venv) bg@n4mrv:~$ python3 -m pip install dateutil
ERROR: Could not find a version that satisfies the requirement dateutil (from versions: none)
ERROR: No matching distribution found for dateutil

Thanks

Bob

grattans.org/wx
 

bg@n4mrv:~$ python3 -m pip list
Package                    Version
-------------------------- ----------------
appdirs                    1.4.4
apturl                     0.5.2
attrs                      21.2.0
Babel                      2.13.0
bcrypt                     3.2.0
beautifulsoup4             4.10.0
beniget                    0.4.1
blinker                    1.4
Brlapi                     0.8.3
Brotli                     1.0.9
certifi                    2020.6.20
chardet                    4.0.0
charset-normalizer         3.3.0
checkbox-support           0.22
Cheetah3                   3.2.6
chrome-gnome-shell         0.0.0
cliapp                     1.20180812.1
click                      8.0.3
cmdtest                    0.32+git
colorama                   0.4.4
command-not-found          0.3
configobj                  5.0.6
cryptography               3.4.8
CT3                        3.3.2
cupshelpers                1.0
cycler                     0.11.0
dbus-python                1.2.18
decorator                  4.4.2
defer                      1.0.6
distro                     1.7.0
distro-info                1.1+ubuntu0.1
dnspython                  2.1.0
duplicity                  0.8.21
ephem                      4.1.3
fasteners                  0.14.1
feedparser                 6.0.8
fonttools                  4.29.1
fs                         2.4.12
future                     0.18.2
gast                       0.5.2
geographiclib              1.52
geopy                      2.2.0
ghp-import                 2.1.0
gpg                        1.16.0
html5lib                   1.1
httplib2                   0.20.2
idna                       3.3
importlib-metadata         4.6.4
iotop                      0.6
jeepney                    0.7.1
Jinja2                     3.0.3
keyring                    23.5.0
keyrings.alt               4.1.0
kiwisolver                 1.3.2
language-selector          0.1
launchpadlib               1.10.16
lazr.restfulclient         0.14.4
lazr.uri                   1.0.6
lockfile                   0.12.2
louis                      3.20.0
lxml                       4.8.0
lz4                        3.1.3+dfsg
macaroonbakery             1.3.1
Mako                       1.1.3
Markdown                   3.3.6
MarkupSafe                 2.0.1
matplotlib                 3.5.1
mergedeep                  1.3.4
mkdocs                     1.5.3
mkdocs-material            9.4.6
mkdocs-material-extensions 1.2
monotonic                  1.6
more-itertools             8.10.0
mpmath                     0.0.0
netifaces                  0.11.0
numpy                      1.21.5
oauthlib                   3.2.0
onboard                    1.4.1
ovs                        2.17.7
packaging                  21.3
padme                      1.1.1
paginate                   0.5.6
paho-mqtt                  1.5.1
paramiko                   2.9.3
pathspec                   0.11.2
pexpect                    4.8.0
Pillow                     9.0.1
pip                        22.0.2
platformdirs               3.11.0
ply                        3.11
protobuf                   3.12.4
ptyprocess                 0.7.0
pyasn1                     0.4.8
pycairo                    1.20.1
pycryptodomex              3.11.0
pycups                     2.0.1
pycurl                     7.44.1
pyephem                    9.99
pyephem-sunpath            0.2.2
Pygments                   2.16.1
PyGObject                  3.42.1
PyJWT                      2.3.0
pymacaroons                0.13.0
pymdown-extensions         10.3
PyNaCl                     1.5.0
pyparsing                  2.4.7
pyRFC3339                  1.1
pyserial                   3.5
python-apt                 2.4.0+ubuntu2
python-dateutil            2.8.1
python-debian              0.1.43+ubuntu1.1
pythran                    0.10.0
pytz                       2022.1
pyusb                      1.2.1.post1
pyxdg                      0.27
PyYAML                     5.4.1
pyyaml_env_tag             0.1
regex                      2023.10.3
reportlab                  3.6.8
requests                   2.31.0
requests-toolbelt          0.9.1
scipy                      1.8.0
scour                      0.38.2
SecretStorage              3.3.1
setuptools                 59.6.0
sgmllib3k                  1.0.0
six                        1.16.0
sortedcontainers           2.1.0
soupsieve                  2.3.1
ssh-import-id              5.11
sympy                      1.9
system-service             0.3
systemd-python             234
ttystatus                  0.38
ubuntu-advantage-tools     8001
ubuntu-drivers-common      0.0.0
ufoLib2                    0.13.1
ufw                        0.36.1
unattended-upgrades        0.1
unicodedata2               14.0.0
unity-lens-photos          1.0
unity-scope-calculator     0.1
unity-scope-devhelp        0.1
unity-scope-manpages       0.1
unity-scope-tomboy         0.1
unity-scope-virtualbox     0.1
unity-scope-zotero         0.1
urllib3                    1.26.5
usb-creator                0.3.7
wadllib                    1.3.6
watchdog                   3.0.0
webencodings               0.5.1
wheel                      0.37.1
xdg                        5
xdiagnose                  3.8.10
xkit                       0.0.0
xlrd                       1.2.0
XlsxWriter                 3.0.2
zipp                       1.0.0
_____________________________________________

(weewx-venv) bg@n4mrv:~$ python3 -m pip list
Package    Version
---------- --------
configobj  5.0.8
CT3        3.3.2
ephem      4.1.5
Pillow     10.1.0
pip        22.0.2
PyMySQL    1.1.0
pyserial   3.5
pyusb      1.2.1
setuptools 59.6.0
six        1.16.0
weewx      5.0.0b13

[vince]

You should 'install' into the venv using pip after activating the venv.

The weewx venv needs only the modules needed for your weewx setup to work, 'not' all the modules you might have installed at the os level.  Everybody's answer might differ based on what extensions/skins etc. they have added to weewx itself.

But just as an example, for my weewx v5 setup here running 5.0.0b12.....

(weewx-venv) pi@pi4:~$ pip list
Package            Version
------------------ --------
certifi            2023.5.7
charset-normalizer 3.2.0
configobj          5.0.8
CT3                3.3.1
ephem              4.1.4
idna               3.4
paho-mqtt          1.6.1
Pillow             9.5.0
pip                20.3.4
pkg-resources      0.0.0

PyMySQL            1.1.0
pyserial           3.5
pyusb              1.2.1

requests           2.31.0
setuptools         44.1.1
six                1.16.0
urllib3            2.0.4
weewx              5.0.0b12

And my extension list, excluding skins like Belchertown that I have installed as well....

(weewx-venv) pi@pi4:~$ weectl extension list
Extension Name    Version   Description
MQTTSubscribe     2.2.2     Augment WeeWX records or packets with MQTT data.
StackedWindRose   3.0.1     Stacked windrose image generator for WeeWX.
forecast          3.4.0b12  Generate and display weather and tide forecasts.
mqtt              0.24      Upload weather data to MQTT server.
purpleair         0.1       Collect Purple Air air quality data.
xaggs             0.8       XTypes extension that calculates historical highs and lows for a date, or days above or below a mean value

[Tom Keffer]

Try installing the date-util package.

python3 -m pip install date-util

It would be nice if you showed more of the stack trace. It would tell us which extension uses date-util. There may be others.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/cf1b01ab-4434-42b3-a22c-ce0a166ed936n%40googlegroups.com.

[bgra...@umw.edu]

I found the name was actually "python-dateutil" so with that installed in venv I'm not getting the error when I run weewxd using /home/weewx/weewx.conf.

On to the next step in migration. Thanks Vince and Tom--I'm sure to have more questions...