Driver permission error when starting Weewx

[Tomasz Lewicki]

I got problem with running Weewx with old/new station. Recently I bought new station and moved it to previous remote location. Old station will be installed in my home. It is clone of FineOffset HP1000 so both HP1000 driver from AussieSusan (https://github.com/AussieSusan/HP1000) and Interceptor from Matthew Wall (https://github.com/matthewwall/weewx-interceptor) should work.

I installed both interceptor and HP1000 drivers by weectl. But after starting Weewx I got critical errors in my log (see below). I suppose it is something easy to fix, some permissions to file and/or directory but I don't know where.

Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Initializing weewxd version 5.0.0
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Command line: /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Using Python 3.7.3 (default, Oct 11 2023, 09:51:27) #012[GCC 8.3.0]
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Located at /usr/bin/python3
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Platform Linux-5.10.103-v7+-armv7l-with-debian-10.13
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Locale: 'pl_PL.UTF-8'
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Entry path: /usr/share/weewx/weewxd.py
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: WEEWX_ROOT: /etc/weewx
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Configuration file: /etc/weewx/weewx.conf
Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: User module: /etc/weewx/bin/user

Jan 21 18:14:17 FR24 weewxd[14285]: INFO __main__: Debug: 1
Jan 21 18:14:17 FR24 weewxd[14285]: DEBUG __main__: Initializing engine
Jan 21 18:14:17 FR24 weewxd[14285]: INFO weewx.engine: Loading station type Interceptor (user.interceptor)
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: driver version is 0.60
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: device type: observer
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: hardware name: weatherstation via interceptor
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: mode is listen
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: listen on :80
Jan 21 18:14:17 FR24 weewxd[14285]: ERROR weewx.engine: Import of driver failed: [Errno 13] Brak dostępu (<class 'PermissionError'>)
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****  Traceback (most recent call last):
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/usr/share/weewx/weewx/engine.py", line 115, in setupStation
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      self.console = loader_function(config_dict, self)
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/etc/weewx/bin/user/interceptor.py", line 316, in loader
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      return InterceptorDriver(**config_dict[DRIVER_NAME])
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/etc/weewx/bin/user/interceptor.py", line 2529, in __init__
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      self._device = self.DEVICE_TYPES.get(self._device_type)(**stn_dict)
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/etc/weewx/bin/user/interceptor.py", line 1286, in __init__
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      Observer.Parser(), handler=Observer.Handler, **stn_dict)
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/etc/weewx/bin/user/interceptor.py", line 430, in __init__
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      self._server = Consumer.TCPServer(address, port, handler)
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/etc/weewx/bin/user/interceptor.py", line 585, in __init__
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      TCPServer.__init__(self, (address, int(port)), handler)
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/usr/lib/python3.7/socketserver.py", line 452, in __init__
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      self.server_bind()
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****    File "/usr/lib/python3.7/socketserver.py", line 466, in server_bind
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****      self.socket.bind(self.server_address)
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL weewx.engine:     ****  PermissionError: [Errno 13] Brak dostępu
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL __main__: Unable to load driver: [Errno 13] Brak dostępu
Jan 21 18:14:17 FR24 weewxd[14285]: CRITICAL __main__:     ****  Exiting...

[matthew wall]

On Sunday, January 21, 2024 at 12:23:34 PM UTC-5 Tomasz Lewicki wrote:

Jan 21 18:14:17 FR24 weewxd[14285]: INFO weewx.engine: Loading station type Interceptor (user.interceptor)
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: driver version is 0.60
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: device type: observer
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: hardware name: weatherstation via interceptor
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: mode is listen
Jan 21 18:14:17 FR24 weewxd[14285]: INFO user.interceptor: listen on :80

if you listen on port 80, then the process must run as root (only root can listen on lower ports).

so either run weewxd as root, or configure interceptor (and the station) to communicate on a higher port.

 

[Tomasz Lewicki]

I run weewx as root:

sudo systemctl start weewx

If I set higher port (8080), weewx starts but I have empty queue for interceptor.

[gary....@gmail.com]

sudo only issues the command as root.

What is in the service file for user and group?

That is who weewx is running as.

[Tomasz Lewicki]

OK, I changed user and group to root in /etc/weewx/systemd/weewx.service, weewx now starts with port 80 and interceptor driver. But nevermind which  option I use - observer or wu-client - I always get:

weewxd[27966]: DEBUG user.interceptor: empty queue

But I found this wiki article -> https://github.com/weewx/weewx/wiki/gw1000-recipe and started interceptor driver from command line:

sudo PYTHONPATH=/usr/share/weewx python /etc/weewx/bin/user/interceptor.py --port=80 --debug --device=observer --mode=listen

Then - exactly  as this part of article says -> https://github.com/weewx/weewx/wiki/gw1000-recipe#verify-that-the-interceptor-can-receive-data I pasted to browser such query:

http://192.168.0.106/data/report?PASSKEY=XXX&stationtype=GW1000B_V1.5.5&dateutc=2019-12-29+16:27:27&tempinf=67.1&humidityin=39&baromrelin=30.138&baromabsin=30.138&freq=915M&model=GW1000

And I got an answer in terminal:

raw data: PASSKEY=XXXX&stationtype=GW1000B_V1.5.5&dateutc=2019-12-29+16:27:27&tempinf=67.1&humidityin=39&baromrelin=30.138&baromabsin=30.138&freq=915M&model=GW1000
raw packet: {'dateTime': 1577636847, 'usUnits': 1}
mapped packet: {'dateTime': 1577636847, 'usUnits': 1}
raw data:
raw packet: {'dateTime': 1706118216, 'usUnits': 1}
mapped packet: {'dateTime': 1706118216, 'usUnits': 1}

So it seems that driver is working when called explicitly from command line. So why I got "empty queue" when started by 'sudo systemctl start weewx'?

[vince]

Lets see your systemd service file.....

[Tomasz Lewicki]

Here it comes:

# systemd service configuration file for WeeWX

[Unit]
Description=WeeWX
Documentation=https://weewx.com/docs
Requires=time-sync.target
After=time-sync.target

[Service]
ExecStart=weewxd /etc/weewx/weewx.conf
StandardOutput=null
StandardError=journal+console
RuntimeDirectory=weewx
RuntimeDirectoryMode=775
User=root
Group=root

[Install]
WantedBy=multi-user.target

[Tomasz Lewicki]

Previously I had strange problems with Python but finally I realized that I have Python2 and Python3 installed. When I called commands begining with 'python', v2 was started. But 'sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10' has made the trick:

pi@meteo:~ $ python --version
Python 3.7.3

[vince]

Just a thought, but interceptor is 'listening' on port 80.  Don't you need to configure your hardware to 'send' to port 80 on your weewx computer ?

[Tomasz Lewicki]

By hardware do you mean WiFi console from my weather station? If yes, it has very simple (almost none) configuration. It sends weather data to WU only. But I hijacked DNS on my router:

pi@meteo:~ $ rtupdate.wunderground.com

PING rtupdate.wunderground.com (192.168.0.106) 56(84) bytes of data.
64 bytes from rtupdate.wunderground.com (192.168.0.106): icmp_seq=1 ttl=64 time=0.130 ms
64 bytes from rtupdate.wunderground.com (192.168.0.106): icmp_seq=2 ttl=64 time=0.128 ms

pi@meteo:~ $ dig rtupdate.wunderground.com

; <<>> DiG 9.11.5-P4-5.1+deb10u9-Raspbian <<>> rtupdate.wunderground.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48940
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;rtupdate.wunderground.com. IN A

;; ANSWER SECTION:
rtupdate.wunderground.com. 0 IN A 192.168.0.106

;; Query time: 170 msec
;; SERVER: 192.168.0.10#53(192.168.0.10)
;; WHEN: śro sty 24 20:59:32 CET 2024
;; MSG SIZE  rcvd: 70

I even updated console's firmware to the latest available version from Ambient Weather website. In server.ini file I see only this:

[Menu]
default=www.wunderground.com
web=1
web1=www.wunderground.com

[www.wunderground.com]
Server=rtupdate.wunderground.com
ServerType=php
ServerPort=80
UploadType=Customize
ReadOnly=true

[vince]

You need something to send traffic to intercept.  Run wireshark on your weewx computer and listen on port 80 and see if your computer is hearing anything (and what).  If it's http the payload should be decipherable.

[Tomasz Lewicki]

I used command line version of Wireshark - Tshark - and collected packets for 15 minutes. Then loaded the PCAP file into Wireshark and filtered by source IP of weather station  console. Every 2 seconds I see:

0000   b8 27 eb dd 86 34 08 d8 33 53 93 a9 08 00 45 00   .'...4..3S....E.
0010   00 90 1b 12 00 00 80 06 9d 2a c0 a8 00 71 c0 a8   .........*...q..
0020   00 6a c0 08 19 64 e3 81 0c 05 7e 63 ac 14 50 18   .j...d....~c..P.
0030   00 44 fc 9d 00 00 48 50 32 30 30 30 00 00 57 52   .D....HP2000..WR
0040   49 54 45 00 00 00 4e 4f 57 52 45 43 4f 52 44 00   ITE...NOWRECORD.
0050   00 00 00 00 00 00 68 00 00 00 8a 00 00 00 c5 00   ......h.........
0060   2f 3d 33 33 b3 41 33 d3 73 44 00 c0 7d 44 9a 99   /=33.A3.sD..}D..
0070   e9 40 9a 99 99 3e 9a 99 e9 40 00 00 00 00 00 00   .@...>...@......
0080   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0090   00 00 00 00 00 00 00 00 00 00 00 ff 00 00         ..............

But I don't know if it makes any sense :)   

[Tomasz Lewicki]

My mistake. These packets were captured when Weewx was running with HP1000 driver. When I changed the driver to interceptor, I didn't captured *any* packets from WiFi console. I don't understand it...

środa, 24 stycznia 2024 o 21:06:53 UTC+1 vince napisał(a):

[vince]

Stop weewx.  Just run the packet capture and post the .pcap file so others can take a look please.  Be sure to capture only tcp/80.   Compress the pcap file if it's large or post it someplace online where others can download it.

The gui version of wireshark is much easier to understand.

[Tomasz Lewicki]

I attach PCAP file with packets captured for 120 seconds. TCP/80 only. Weewx was shut down. No traffic on this port until I manually have done the operation described in one of me previous posts (browser query): https://groups.google.com/g/weewx-user/c/O8DTjzeeQcg/m/psrDaMHoAAAJ 

[Tomasz Lewicki]

Just one more thought. Maybe interceptor driver is "too new" for my WiFi console? Maybe I should try some older version? But unfortunately I don't see any releases on driver's GitHub page (https://github.com/matthewwall/weewx-interceptor).

środa, 24 stycznia 2024 o 22:22:37 UTC+1 vince napisał(a):

[vince]

Difficult to answer with no info from you on exactly 'what' command you ran for wireshark and whether your listening computer is wifi, ethernet, or both.  What kind of computer are you running on ?  What os ?  What version ?  Which interfaces ?  What was your 'exact' wireshark command ?

But I see nothing basically in that 6-second pcap.  If running a sniffer on your computer sees no traffic being redirected from the station, then there is nothing for interceptor to intercept on the weewx computer.

Again, when you say "But I hijacked DNS on my router" that (to me) does not cause any traffic from your station to wunderground to be redirected to your weewx system unless I'm not understanding what you're saying.  Perhaps you should tell everybody what your system config is so those who do interceptor can try to help more. 

[Tomasz Lewicki]

Mystery solved. 

But answering to vince question, my system is rather typical - Raspbian on Raspberry Pi, only WLAN interface is active. Weewx was unwillingly updated from 4.10.2 to 5.0.0. I checked all point of failure: Python version, permissions (thank you Gary!), network traffic. As I wrote before, rtupdate.wunderground.com was hijacked - local DNS redirected it to Weewx. So I deleted this bypass, allowing console to send data to real WU server. But still I couldn't see any traffic on my router. Total silence. It was abnormal (and it explains why PCAP file captured by Tshark was empty on port 80). But I didn't check WU settings in WiFi console. Station ID was empty, password was obfuscated by asterisks. I don't use WU website at all, I just needed credentials for conversation between console and Weewx. I entered ID and password - and then console started send data to real WU. So I redirected  network traffic on my DNS again, and Weewx started to receive data from WiFi console via interceptor driver :)

Thank you to everyone who patiently read my writings and tried to help.

[Chris McLeod]

@Tomasz, you stated above that ' I installed both interceptor and HP1000 drivers by weectl. " Can you provide some insight into how you did this? I have a fresh install of 5.0.1 and when I run 'weectl extension install weewx-interceptor.zip (fresh download) I get an error stack below. I am posting here just in case I am missing something obvious, if not I will start a new thread.

Thanks,

Chris

:~$ weectl extension install weewx-interceptor.zip
Using configuration file /etc/weewx/weewx.conf
Install extension 'weewx-interceptor.zip' (y/n)? y
Extracting from zip archive weewx-interceptor.zip

Traceback (most recent call last):

  File "/usr/share/weewx/weectl.py", line 74, in <module>
    main()
  File "/usr/share/weewx/weectl.py", line 66, in main
    namespace.func(namespace)
  File "/usr/share/weewx/weectllib/__init__.py", line 121, in dispatch
    namespace.action_func(config_dict, namespace)
  File "/usr/share/weewx/weectllib/extension_cmd.py", line 116, in install_extension
    ext.install_extension(namespace.source, no_confirm=namespace.yes)
  File "/usr/share/weewx/weecfg/extension.py", line 138, in install_extension
    extension_name = self._install_from_file(extension_path, filetype)
  File "/usr/share/weewx/weecfg/extension.py", line 168, in _install_from_file
    extension_name = self.install_from_dir(extension_dir)
  File "/usr/share/weewx/weecfg/extension.py", line 185, in install_from_dir
    self._install_files(installer['files'], extension_dir)
  File "/usr/share/weewx/weecfg/extension.py", line 269, in _install_files
    shutil.copy(source_path, destination_path)
  File "/usr/lib/python3.8/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/bin/user/interceptor.py'

[Tomasz Lewicki]

That's strange message. When you manualy decompress ZIP (I suppose downloaded from GitHub) and look into install.py, last line says:

files=[('bin/user', ['bin/user/interceptor.py'])]

I don't know why in your case there is / (slash) in error message:

FileNotFoundError: [Errno 2] No such file or directory: '/bin/user/interceptor.py'

But I think you can manually copy just one file - interceptor.py  - to /etc/weewx/bin/user directory and make changes in weewx.conf, then restart weewx, it should work. There are additiional files in util directory but install.py omits them so they are not necessary. 

[gjr80]

You have almost certainly encountered the bug mentioned in this post. The bug does not bite (ie potentially delete system files) unless you uninstall the extension using weectl extension - so my strong advice to you is do not attempt to uninstall the extension. A manual uninstall (or install) will be fine. But I would further suggest waiting a few days for v5.0.1 which fixes this bug, its release should not be too far off.

Gary

[Chris McLeod]

Gary,

Thank you very much for the quick reply. I will stand by for the new release.

Thanks,

Chris

[Chris McLeod]

Tomasz,

Thank you very much for the steps on how to manually install the driver!

Chris