New Install of WeeWx on Raspbian Bookworm - No module named 'sortedcontainers'

[Marty b]

Hello,

Just built a new SD card with Raspbian bookworm 64 bit and WeeWx.  When starting weewx I get an error "No module named 'sortedcontainers'.  Please let me know how to resolve this.

Thanks!

Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.engine: StdConvert target unit is 0x1
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.engine: Archive will use data binding wx_binding
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.engine: Record generation will be attempted in 'hardware'
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.engine: Using archive interval of 300 seconds (specified by hardware)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.restx: StationRegistry: Registration not requested.
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.restx: Wunderground: Posting not enabled.
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.restx: PWSweather: Posting not enabled.
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.restx: CWOP: Posting not enabled.
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.restx: WOW: Posting not enabled.
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.restx: AWEKAS: Posting not enabled.
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: INFO weewx.engine: 'pyephem' detected, extended almanac data is available
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__: Caught unrecoverable exception:
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****  No module named 'sortedcontainers'
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****  Traceback (most recent call last):
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd.py", line 121, in main
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****      engine = weewx.engine.StdEngine(config_dict)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 89, in __init__
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****      self.loadServices(config_dict)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 157, in loadServices
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****      obj = weeutil.weeutil.get_object(svc)(self, config_dict)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "/usr/share/weewx/weeutil/weeutil.py", line 1404, in get_object
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****      module = importlib.import_module(module_name)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****      return _bootstrap._gcd_import(name[level:], package, level)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: Traceback (most recent call last):
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:   File "/usr/share/weewx/weewxd.py", line 226, in <module>
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap_external>", line 940, in exec_module
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:     main()
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:   File "/usr/share/weewx/weewxd.py", line 121, in main
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:     engine = weewx.engine.StdEngine(config_dict)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:   File "/usr/share/weewx/weewx/engine.py", line 89, in __init__
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****    File "/etc/weewx/bin/user/loopdata.py", line 30, in <module>
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:     self.loadServices(config_dict)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:   File "/usr/share/weewx/weewx/engine.py", line 157, in loadServices
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:     obj = weeutil.weeutil.get_object(svc)(self, config_dict)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****      from sortedcontainers import SortedDict
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:   File "/usr/share/weewx/weeutil/weeutil.py", line 1404, in get_object
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****  ModuleNotFoundError: No module named 'sortedcontainers'
Jul 28 23:12:10 WeeWx2024 weewxd[2632]: CRITICAL __main__:     ****  Exiting.
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:     module = importlib.import_module(module_name)
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
Jul 28 23:12:10 WeeWx2024 weewxd[2632]:     return _bootstrap._gcd_import(name[level:], package, level)

[Marty b]

Should point out that I did try installing the missing module, but got the following error:

 $ pip3 install sortedcontainers
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.

[Karen K]

You should state whether this is a package installation or a pip installation. What you have to do differs between the two.

[Tom Keffer]

The user is doing a package install (the clue is paths such as '/usr/share/weewx/...'

My thinking is that you are using an extension that needs 'sortedcontainers'.

Please post your configuration file, /etc/weewx/weewx.conf. Make sure you look it over and remove any passwords or other sensitive information before posting. 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/3cb59936-0567-4c41-a53a-62e752879775n%40googlegroups.com.

[bell...@gmail.com]

This is ‘due’ to pep 668 and a ‘push’ for users to use virtual environments. It is warning the person installing the python module, sortedcontainers, that they  are installing into an ‘global’ (externally-managed-environment) location. There are a few ways to solve this.
Create a virtual environment, activate it, and install the module. Since WeeWX was a package install, this is not reasonable.
As the message states, override the warning using the ‘--break-system-packages’ option. It would look something like this, ‘pip3 install sortedcontainers --break-system-packages’.
If it exists, install the OS package. It would be something like this, ‘sudo apt install python3-sortedcontainers’

Additional information:
http://rptl.io/venv
https://peps.python.org/pep-0668/
https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments

rich

[John Kline]

I looks like he is using an extension of mine (weewx-loopdata) that requires sorted containers since weewx-loopdata v3.0.  The README states that sortedcontainers must be installed and the weewx 5 install instructions, step 1, states:

1. 
   

   Activate the virtual environment (actual syntax varies by type of WeeWX install):/home/weewx/weewx-venv/bin/activate

I’m asssuming weewx 5.  If it is weewx 4, follow the WeeWX instructions for weewx 4 installs.

On Jul 29, 2024, at 9:51 AM, bell...@gmail.com <bell...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d8a08cdb-d5b1-4db2-9f9f-54855503f901n%40googlegroups.com.

[Tom Keffer]

Thanks, John. I figured it was something like that.

Marty: you can probably fix by installing using apt:

sudo apt install python3-sortedcollections

Rich: the OP used a package install, not pip install, so the best way to fix is a package install of sortedcollections.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/39F7996D-675F-4B5D-AB45-8660C1F1E6C4%40johnkline.com.

[Marty b]

Hello everyone,

Thank you all for the replies!

I tried installing the module using both methods (apt & pip).  Both gave errors. I tried a bunch of other things and couldn't get it to work.  So, I started working on some other things on the Pi and noticed that the language was set for EN-GB-UTF8.  I change this to EN-US-UTF8 and tried install the module again (via apt) and was successful.  This make absolutely no sense, but that was the last thing that changed. 

I did try activating the virtual environment and could not make it work. 

I am new to the virtual environment of python - and very willing to learn/understand it.

I will further try to figure it out since I want to use the mqtt plug-in (https://github.com/weewx/weewx/wiki/mqtt) and it requires a package that does not appear to be installable via apt (paho-mqtt).

[John Kline]

sudo apt install python3-sortedcollections is the instruction I give for WeeWX 4.  Do I need to give that instruction for package installs of WeeWX 5?

On Jul 29, 2024, at 10:49 AM, Tom Keffer <tke...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEBg-NS-pLr02CZ6Vh37XDzHGkeVHhKxRKQau04Cfv6ZTQ%40mail.gmail.com.

[Tom Keffer]

On Mon, Jul 29, 2024 at 8:57 AM 'John Kline' via weewx-user <weewx...@googlegroups.com> wrote:

sudo apt install python3-sortedcollections is the instruction I give for WeeWX 4.  Do I need to give that instruction for package installs of WeeWX 5?

I would think so.

[bell...@gmail.com]

Tom,
I'd say maybe... Sometimes the package installs are very behind the pip installs. I recently ran into this with paho mqtt client. That is why I gave the OP the option of either a package install or pip install.
For me since I have WeeWX running on dedicated pi (essentially a self contained environment) I prefer using pip to install any required modules. But as always YMMV...
rich

[Warren Gill]

This worked for me:

$ sudo apt install python3-paho-mqtt

While you're there might as well install mosquitto-clients so you can test if it's working.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b7dc5644-6ab1-4e74-9f8f-60580487ca13n%40googlegroups.com.

[bell...@gmail.com]

Depending on your preference, either of these should work for the paho mqtt client.
sudo apt install python-paho-mqtt
pip install paho-mqtt --break-system-packages

rich

[bell...@gmail.com]

Oops, sudo apt install python3-paho-mqtt

[Marty b]

Thank you for all the responses.  I ended up doing another fresh install of the Raspberry Pi OS and WeeWx and everything worked like a charm on the 1st try - based on the feedback provided.  I do believe there was something wrong with my 1st fresh install of the RaspiOS that caused problems not being able to initially install the ' sortedcontainers' module.

Thank you again - the assistance was appreciated!

Marty