interceptor packet

[Nah Yi Jie]

I'm a new user for weewx, currently using a WH2950 with WSTOOLS app to transmit packets to my weewx host pc.

i did a netcat scan for the packet on port 8000:

nc -l 8000

GET/ID=xxx&PASSWORD=xxx&indoortempf=85.8&tempf=85.8&dewptf=74.1&windchillf=85.8&indoorhumidity=66&humidity=68&windspeedmph=2.9&windgustmph=4.9&winddir=188&absbaromin=29.740&baromin=30.062&rainin=0.000&dailyrainin=0.000&weeklyrainin=0.035&monthlyrainin=0.224&solarradiation=0.00&UV=0&dateutc=2020-03-20%2009:09:50&softwaretype=EasyWeatherV1.4.7&action=updateraw&realtime=1&rtfreq=5 HTTP/1.0

Host: 192.168.43.216

Accept:*/*

Connection: Close

but running interceptor.py directly on debug doesnt give me any useful data:

sudo PYTHONPATH=/usr/share/weewx python /usr/share/weewx/user/interceptor.py --device=observer --mode=listen --port 8000 --debug

raw data: 

raw packet: {'usUnits': 16, 'dateTime': 1584695729}

mapped packet: {'usUnits': 16, 'dateTime': 1584695729}

is there something in interceptor or weewx.conf that i am missing? Cheers

[mwall]

On Friday, March 20, 2020 at 5:38:49 AM UTC-4, Nah Yi Jie wrote:

is there something in interceptor or weewx.conf that i am missing? Cheers

which python are you using?  which weewx?

could you post the log output? 

m

[Nah Yi Jie]

python version: 2.7.17

weewx version: 3.9.2

syslog:

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Initializing weewx version 3.9.2

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Using Python 2.7.17 (default, Nov  7 2019, 10:07:09) #012[GCC 7.4.0]

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Platform Linux-5.3.0-42-generic-x86_64-with-Ubuntu-18.04-bionic

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Locale is 'en_SG.UTF-8'

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Using configuration file /etc/weewx/weewx.conf

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Debug is 1

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Initializing engine

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading station type Interceptor (user.interceptor)

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: MainThread: driver version is 0.53

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: MainThread: device type: observer

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: MainThread: mode is listen

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: MainThread: listen on 192.168.43.216:8000

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: MainThread: sensor map: {'extraTemp2': 'temperature_2', 'pm2_5': 'pm2_5', 'soilTemp1': 'soil_temperature_1', 'soilTemp2': 'soil_temperature_2', 'soilTemp3': 'soil_temperature_3', 'soilTemp4': 'soil_temperature_4', 'txBatteryStatus': 'battery', 'outTemp': 'temperature_out', 'windSpeed': 'wind_speed', 'windGust': 'wind_gust', 'outHumidity': 'humidity_out', 'windDir': 'wind_dir', 'radiation': 'solar_radiation', 'inTemp': 'temperature_in', 'windGustDir': 'wind_gust_dir', 'barometer': 'barometer', 'windchill': 'windchill', 'dewpoint': 'dewpoint', 'rain': 'rain', 'extraHumid6': 'humidity_6', 'pressure': 'pressure', 'extraHumid4': 'humidity_4', 'extraHumid5': 'humidity_5', 'extraHumid2': 'humidity_2', 'extraHumid3': 'humidity_3', 'extraTemp8': 'temperature_8', 'extraHumid1': 'humidity_1', 'extraTemp6': 'temperature_6', 'extraTemp7': 'temperature_7', 'extraTemp4': 'temperature_4', 'extraTemp5': 'temperature_5', 'rainRate': 'rain_rate', 'extraTemp3': 'temperature_3', 'extraHumid8': 'humidity_8', 'extraTemp1': 'temperature_1', 'leafWet1': 'leafwetness_1', 'leafWet2': 'leafwetness_2', 'extraHumid7': 'humidity_7', 'UV': 'uv', 'soilMoist3': 'soil_moisture_3', 'soilMoist2': 'soil_moisture_2', 'soilMoist1': 'soil_moisture_1', 'inHumidity': 'humidity_in', 'soilMoist4': 'soil_moisture_4'}

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: ServerThread: start tcp server

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.engine.StdTimeSynch

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.engine.StdTimeSynch

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.engine.StdConvert

Mar 20 22:59:12 yijie-home weewx[1901]: engine: StdConvert target unit is 0x1

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.engine.StdConvert

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.engine.StdCalibrate

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.engine.StdCalibrate

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.engine.StdQC

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.engine.StdQC

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.wxservices.StdWXCalculate

Mar 20 22:59:12 yijie-home weewx[1901]: wxcalculate: The following values will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, heatindex=prefer_hardware, maxSolarRad=prefer_hardware, humidex=prefer_hardware, pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, cloudbase=prefer_hardware

Mar 20 22:59:12 yijie-home weewx[1901]: wxcalculate: The following algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.wxservices.StdWXCalculate

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.engine.StdArchive

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Archive will use data binding wx_binding

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Record generation will be attempted in 'hardware'

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Using archive interval of 300 seconds (specified in weewx configuration)

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Use LOOP data in hi/low calculations: 1

Mar 20 22:59:12 yijie-home weewx[1901]: manager: Daily summary version is 2.0

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Using binding 'wx_binding' to database 'weewx.sdb'

Mar 20 22:59:12 yijie-home weewx[1901]: manager: Starting backfill of daily summaries

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.engine.StdArchive

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.restx.StdStationRegistry

Mar 20 22:59:12 yijie-home weewx[1901]: restx: StationRegistry: Registration not requested.

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.restx.StdStationRegistry

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.restx.StdWunderground

Mar 20 22:59:12 yijie-home weewx[1901]: restx: Wunderground: Posting not enabled.

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.restx.StdWunderground

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.restx.StdPWSweather

Mar 20 22:59:12 yijie-home weewx[1901]: restx: PWSweather: Posting not enabled.

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.restx.StdPWSweather

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.restx.StdCWOP

Mar 20 22:59:12 yijie-home weewx[1901]: restx: CWOP: Posting not enabled.

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.restx.StdCWOP

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.restx.StdWOW

Mar 20 22:59:12 yijie-home weewx[1901]: restx: WOW: Posting not enabled.

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.restx.StdWOW

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.restx.StdAWEKAS

Mar 20 22:59:12 yijie-home weewx[1901]: restx: AWEKAS: Posting not enabled.

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.restx.StdAWEKAS

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.engine.StdPrint

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.engine.StdPrint

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Loading service weewx.engine.StdReport

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Finished loading service weewx.engine.StdReport

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Starting up weewx version 3.9.2

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Station does not support reading the time

Mar 20 22:59:12 yijie-home weewx[1901]: engine: Starting main packet loop.

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: ServerThread: GET:

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: MainThread: raw data:

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: MainThread: raw packet: {'usUnits': 16, 'dateTime': 1584716353}

weewxd console display:

LOOP:   2020-03-20 23:05:34 +08 (1584716734) dateTime: 1584716734, maxSolarRad: None, rainRate: 0, usUnits: 1

REC:    2020-03-20 23:05:00 +08 (1584716700) dateTime: 1584716700.0, ET: None, interval: 5, maxSolarRad: None, rainRate: 0.0, usUnits: 1

perhaps i have to map the sensor device addresses on weewx.conf?

[mwall]

On Friday, March 20, 2020 at 11:28:56 AM UTC-4, Nah Yi Jie wrote:

python version: 2.7.17

weewx version: 3.9.2

syslog:

...

Mar 20 22:59:12 yijie-home weewx[1901]: interceptor: ServerThread: GET:

this is the line i'm looking for.  the interceptor is receive an http GET request, but there is nothing in it. 

please re-run with the attached interceptor.  do not post the output directly - this version of interceptor will expose any passwords, so you will have to manually obfuscate them from the output.

m

[Nah Yi Jie]

got it, looks like the same issue here. log file below

[mwall]

On Friday, March 20, 2020 at 1:35:31 PM UTC-4, Nah Yi Jie wrote:

got it, looks like the same issue here. log file below

thanks for that.

apparently your station is sending a GET request with no query part.

it is doing:

/ID=xxx&PASSWORD=xxx&...

rather than:

/...?ID=xxx&PASSWORD=xxx&...

interceptor was using urlparse to get to the query part, but now the 'query' part is the URL itself.

looks like your station is reporting itself as 'EasyWeatherV1.4.7'.  the most recent actual data i have for interceptor was with EasyWeatherV1.3.9.  so somewhere along the line the firmware changed, and they started using a different structure in their GET requests.

please try the attached interceptor 0.54b

m

[n yj]

i'll get back to you on this! right as i tried the new interceptor file the wifi module on the console broke? i cant get it to turn on at the moment... let me try to get this fixed first.

[n yj]

On Saturday, March 21, 2020 at 2:05:25 AM UTC+8, mwall wrote:

please try the attached interceptor 0.54b

managed to get my console replaced, running v1.4.9 EasyWeather on WSView. 0.54b did the trick and its now reading wu-protocol correctly. i was playing around with 0.53 with an ecowitt protocol and it seemed to work as well...

but this time, i'm receiving repeated timestamps from raw data. its probably not a software side issue, but is it possible to prefer to use computer timestamps instead?

[n yj]

i have resolved the issue. in the interceptor file under class Observer(Consumer):

        def parse(self, s):

            pkt = dict()

            try:

                data = _cgi_to_dict(s)

                pkt['dateTime'] = self.decode_datetime(

                    data.pop('dateutc', int(time.time() + 0.5)))

i have replaced the decode function to be simply time.time(). required this fix as my packet upload frequency is 16s, but timestamps from the receiver does not update until a minute later even though other data updates fine. weird...

thanks matthew for your help!