Fwd: [weewx-user] Re: Pulling my hair - one weird issue after the other [4]

76 views
Skip to first unread message

Rainer Lang

unread,
Aug 3, 2021, 8:29:57 AM8/3/21
to weewx-user

-------- Forwarded Message --------
Subject: [weewx-user] Re: Pulling my hair - one weird issue after the other [4]
Date: Tue, 3 Aug 2021 14:28:28 +0200
From: Rainer Lang <lang....@gmail.com>
To: Konrad Skeri Ekblad <kon...@skeri.com>


I think this side effect is due to a recent change of Interceptor.py to show also the hardware name in the "about this weather station" section - the change fell somewhat too short. I had already been searching myself for a solution, but gave up due to lack of time and decided to bear with this message before every archiving cycle.

If you want to get rid of this log message, you would have to walk through the Interceptor.py coding and add some additional filtering.
Without that hardware name change the PASSKEY info was filtered out. Now the program flow is different.
It will probably have to be filtered out a 2nd time or only at some other place now.

On 03.08.2021 10:55, Konrad Skeri Ekblad wrote:
Actually, the console tries to post even without a path entered, it just doesn't specify a request-URI in the HTTP POST header causing an error 400 on the server side.
I never changed the path setting from when it was working, so it must have been a side effect from a firmware update done during the months my WeeWX was down. Good thing to know for people having the same trouble in the future. I also changed the path from /path? as suggested in the GitHub discussion I found to just / and that works as well. However, in both cases I get the following line in the message log every time the station sends an update:

Aug  3 10:51:33 localhost journal[2825]: weewx[2825] INFO user.interceptor: unrecognized parameter b'PASSKEY=9879…

While not interfering with function it clutters up the log making it harder to read. Guess that's a topic for the GitHub issue tracker though.

Konrad

tisdag 3 augusti 2021 kl. 01:35:43 UTC+2 skrev lang....@googlemail.com:

By mistake I sent this reply only to the OPer - now forward also to the group for information and documentation

It's a common mistake when using the custom server function of a Fine Offset (clone) console in ecowitt protocol
to leave the path entry in the WSView app empty.
Even though weewx (the interceptor driver) doesn't care about any path, the console still does.

If there is no entry at all, the console will not post !!!

Therefore at least "/" without "" has to be entered into the path field in the WSView app.

This is for weewx.

When posting via custom server function to a web site, the path has to be meaningful and point to the HTML or www root of the server respectively the path from there where the posting will be picked up e.g. by some index.php file.

The posting in Ecowitt protocol will always look like

http://IP-or-URL/path-from-wsview-app/index.php&PASSKEY=xxxxxxxxxxxxxx&stationtype= .....

the interceptor driver filters out the PASSKEY, stationtype and other not observation related parts of the posted string and takes only the observation related portions e.g. windgustmph=5.82



-------- Forwarded Message --------
Subject: [weewx-user] Re: Pulling my hair - one weird issue after the other [3]
Date: Mon, 2 Aug 2021 16:05:30 +0200
From: Rainer Lang <lang....@gmail.com>
To: Konrad Skeri Ekblad <kon...@skeri.com>


Did you enter a path in the WSView app ?
There must be at least a "/" - even though weewx doesn't care, the console does.
If there is no path entered, it doesn't post via the custom server function.

On 02.08.2021 14:55, Konrad Skeri Ekblad wrote:
Running WeeWX directly instead the output in /var/log/messages is this:

Aug  2 14:36:59 localhost systemd[1]: Starting SYSV: start and stop the weewx weather system...
Aug  2 14:36:59 localhost journal[2697]: weewx[2697] INFO __main__: Initializing weewx version 4.5.1
Aug  2 14:36:59 localhost journal[2697]: weewx[2697] INFO __main__: Using Python 3.6.8 (default, Mar 19 2021, 05:13:41) #012[GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]
Aug  2 14:36:59 localhost journal[2697]: weewx[2697] INFO __main__: Platform Linux-4.18.0-305.10.2.el8_4.x86_64-x86_64-with-centos-8.4.2105
Aug  2 14:36:59 localhost journal[2697]: weewx[2697] INFO __main__: Locale is 'sv_SE.UTF-8'
Aug  2 14:36:59 localhost journal[2697]: weewx[2697] INFO __main__: PID file is /var/run/weewx.pid
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO __main__: Using configuration file /etc/weewx/weewx.conf
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO __main__: Debug is 1
Aug  2 14:36:59 localhost weewx[2688]: Starting weewx: [     OK     ]
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.engine: Loading station type Interceptor (user.interceptor)
Aug  2 14:36:59 localhost systemd[1]: Started SYSV: start and stop the weewx weather system.
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO user.interceptor: driver version is 0.53
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO user.interceptor: device type: ecowitt-client
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO user.interceptor: mode is listen
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO user.interceptor: listen on :8000
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO user.interceptor: sensor map: {'pressure': 'pressure', 'barometer': 'barometer', 'outHumidity': 'humidity_out', 'inHumidity': 'humidity_in', 'outTemp': 'temperature_out', 'inTemp': 'temperature_in', 'windSpeed': 'wind_speed', 'windGust': 'wind_gust', 'windDir': 'wind_dir', 'windGustDir': 'wind_gust_dir', 'radiation': 'solar_radiation', 'dewpoint': 'dewpoint', 'windchill': 'windchill', 'rain': 'rain', 'rainRate': 'rain_rate', 'UV': 'uv', 'txBatteryStatus': 'battery', 'extraTemp1': 'temperature_1', 'extraTemp2': 'temperature_2', 'extraTemp3': 'temperature_3', 'extraHumid1': 'humidity_1', 'extraHumid2': 'humidity_2', 'soilTemp1': 'soil_temperature_1', 'soilTemp2': 'soil_temperature_2', 'soilMoist1': 'soil_moisture_1', 'soilMoist2': 'soil_moisture_2', 'soilMoist3': 'soil_moisture_3', 'soilMoist4': 'soil_moisture_4', 'leafWet1': 'leafwetness_1', 'leafWet2': 'leafwetness_2', 'pm2_5': 'pm2_5', 'extraTemp4': 'temperature_4', 'extraTemp5': 'temperature_5', 'extraTemp6': 'temperature_6', 'extraTemp7': 'temperature_7', 'extraTemp8': 'temperature_8', 'extraHumid3': 'humidity_3', 'extraHumid4': 'humidity_4', 'extraHumid5': 'humidity_5', 'extraHumid6': 'humidity_6', 'extraHumid7': 'humidity_7', 'extraHumid8': 'humidity_8', 'soilTemp3': 'soil_temperature_3', 'soilTemp4': 'soil_temperature_4'}
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.engine: StdConvert target unit is 0x1
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.engine: Archive will use data binding wx_binding
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.engine: Record generation will be attempted in 'hardware'
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.engine: Using archive interval of 300 seconds (specified in weewx configuration)
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.restx: StationRegistry: Registration not requested.
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.restx: Wunderground: Posting not enabled.
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.restx: PWSweather: Posting not enabled.
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.restx: CWOP: Posting not enabled.
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.restx: WOW: Posting not enabled.
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.restx: AWEKAS: Posting not enabled.
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO __main__: Starting up weewx version 4.5.1
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.manager: Starting backfill of daily summaries
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.manager: Daily summaries up to date
Aug  2 14:36:59 localhost journal[2701]: weewx[2701] INFO weewx.engine: Starting main packet loop.

After that there are no more log entries. Been running about 10 minutes now.

systemctr status weewx gives the following info:
● weewx.service - SYSV: start and stop the weewx weather system
   Loaded: loaded (/etc/rc.d/init.d/weewx; generated)
   Active: active (running) since Mon 2021-08-02 14:36:59 CEST; 10min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 2671 ExecStop=/etc/rc.d/init.d/weewx stop (code=exited, status=0/SUCCESS)
  Process: 2688 ExecStart=/etc/rc.d/init.d/weewx start (code=exited, status=0/SUCCESS)
    Tasks: 2 (limit: 11377)
   Memory: 13.5M
   CGroup: /system.slice/weewx.service
           └─2701 python3 /usr/share/weewx/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf

aug 02 14:45:29 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:45:39 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:45:49 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:45:59 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:46:09 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:46:19 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:46:29 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:46:39 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:46:49 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue
aug 02 14:46:59 localhost.localdomain python3[2701]: weewx[2701] DEBUG user.interceptor: empty queue

According to Wireshark the weather station connects to WeeWX every minute, but receives the 400 error code.
Stopping WeeWX, Wireshark now shows that the weather station instead of the SYN ACK response to the low level TCP handshake instead receives a RST ACK, indicating that the port now is closed for communication.

Konrad
måndag 2 augusti 2021 kl. 13:02:50 UTC+2 skrev gjr80:
Have a look in the other log files on your system, it should be there somewhere. When running the interceptor driver directly the (WeeWX) logging setup is somewhat abbreviated so it could be different to when WeeWX is run.

Gary

On Monday, 2 August 2021 at 20:47:01 UTC+10 Konrad Skeri Ekblad wrote:
If I start WeeXW (still using simulator driver) I get its log messages in /var/log/messages, however, running the interceptor driver directly does not produce any output in this log file.

Konrad

måndag 2 augusti 2021 kl. 12:22:05 UTC+2 skrev gjr80:
When you just ran the interceptor driver directly there should have been output to the console as well as log output in the WeeWX log, not sure what log file it will be in as I am not a Centos user. Can you please post the log output exactly as it appears in the log.

Gary

On Monday, 2 August 2021 at 19:17:12 UTC+10 Konrad Skeri Ekblad wrote:
I did a complete reinstallation of the server with Centos 8. Fresh install of WeeWX from repository using yum as described on web page. Installed interceptor driver according to set up instructions.
Running PYTHONPATH=/usr/share/weewx python3 user/interceptor.py --debug --port=8000 --device=ecowitt-client for almost an hour only gives the following output:

raw data:
raw packet: {'dateTime': 1627891880, 'usUnits': 1}
mapped packet: {'dateTime': 1627891880, 'usUnits': 1}

Using Wireshark to monitor the network traffic I notice this:
Weatherstation to WeeWX server:
POST  HTTP/1.1
HOST: 192.168.129.10
Connection: Close
Content-Type: application/x-www-form-urlencoded
Content-Length: 415
PASSKEY=**replaced**&stationtype=EasyWeatherV1.6.0&dateutc=2021-08-02+08:11:21&tempinf=77.5&humidityin=42&baromrelin=29.894&baromabsin=29.374&tempf=59.9&humidity=76&winddir=25&windspeedmph=2.7&windgustmph=3.4&maxdailygust=6.9&rainratein=0.000&eventrainin=0.000&hourlyrainin=0.000&dailyrainin=0.000&weeklyrainin=0.000&monthlyrainin=0.000&totalrainin=53.673&solarradiation=355.26&uv=3&model=WS2900

Response from WeeWX server:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html;charset=utf-8">
        <title>Error response</title>
    </head>
    <body>
        <h1>Error response</h1>
        <p>Error code: 400</p>
        <p>Message: Bad HTTP/0.9 request type ('POST').</p>
        <p>Error code explanation: HTTPStatus.BAD_REQUEST - Bad request syntax or unsupported method.</p>
    </body>
</html>

According to WireShark there is this communication about once each minute, but I have only one entry in the terminal.
onsdag 28 juli 2021 kl. 02:24:31 UTC+2 skrev gjr80:
You have changed many things in your setup and even though you appear to have reverted to your original working configuration I would be starting from scratch as per the interceptor drive readme.I would be running the driver directly and then making sure you use the —debug command line option. Post the console output here. Let’s see where that gets us.

Gary
On Wednesday, 28 July 2021 at 08:20:26 UTC+10 Konrad Skeri Ekblad wrote:
I have been using weewx on a Raspberry Pi for about two years now. After my old Fine Offset died soon after I moved from pywws I bought a Eurochron WS2900 which I got running using the interceptor driver, and that worked fine until the weather station stopped giving temperature readings this winter. I was able to fix that last week (bad connector on the cable from the temperature sensor to the circuit board in the sensor array) but when I reconnected it to weewx I started to experience various odd things. The unit shows correct data on its display and sends data to weewx, and the webpages rendered by weewx are correct (e.g. temperature ranging between 16°C and 28°C the other day) but the temperatures sent to Windy (and others) by weewx varies from -2°C to 21°C for the same day. Thinking that there might be some API change or something during the half year my station was offline, and that I for some time have had thoughts of moving it to a virtual server on my ESXi station anyway, I set up a Centos 8 machine and installed weewx on it. Simulator works fine, but using the same interceptor settings as on my Raspberry Pi I don't get any data (IP updated in the weather station settings).

Using
PYTHONPATH=/usr/share/weewx python3 -m user.interceptor --debug --mode=listen --port=8000 --device=ecowitt-client
I was able to once capture this after listening maybe one hour:
raw data:
raw packet: {'dateTime': 1627416663, 'usUnits': 1}
mapped packet: {'dateTime': 1627416663, 'usUnits': 1}
Yay! A timestamp!

Trying to connect to port 8000 from another computer in the network shows that the port is open.

Thinking there is some problem with the weather station I have reseted it, but no change. The device is by itself directly uploading to Weather Underground and some other sites, and that works.

Frustrated I one again change the local IP the weather station should send data to to the Raspberry Pi, but now I don't get any data there at all.

This is the interceptor settings of weewx.conf:

…
    station_type = Interceptor
…
##############################################################################

[Interceptor]
    # This section is for the network traffic interceptor driver.

    # The driver to use:
    driver = user.interceptor

    # Specify the hardware device to capture.  Options include:
    #   acurite-bridge - acurite internet bridge, smarthub, or access
    #   observer - fine offset WH2600/HP1000/HP1003, ambient WS2902
    #   lw30x - oregon scientific LW301/LW302
    #   lacrosse-bridge - lacrosse GW1000U/C84612 internet bridge
    #   wu-client - any hardware that uses the weather underground protocol
#    device_type = observer
    device_type = ecowitt-client
    port = 8000

##############################################################################

I'm not sure what more is relevant logs to post.
I see no dropped packages originating from the weather station in the firewall log.

Does anyone have any idea how I should proceed or what I should try?

regards
Konrad Skeri Ekblad
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/aeab950d-a2c4-4039-8750-b03b9b21fa64n%40googlegroups.com.


--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/fd503c01-014a-4fe9-bd99-db3de185272dn%40googlegroups.com.


Konrad Skeri Ekblad

unread,
Aug 3, 2021, 6:14:49 PM8/3/21
to weewx-user
Someone actually posted a working solution in issue #80 on GitHub https://github.com/matthewwall/weewx-interceptor/issues/80#issuecomment-699638053
It works for me!
Reply all
Reply to author
Forward
0 new messages