Interceptor Driver - rainin dailyrain - Acurite 5 in 1
326 views
Skip to first unread message
Brad Tucker
Nov 4, 2016, 10:42:37 PM11/4/16
to weewx-user
For some reason Rain is not being logged to the system w/ the interceptor model. I accidentally tripped the rain señor while working on the unit today, and the acurite display is reporting .55 inches however my plots show 0 as do the stats. I took a look at the database to make sure nothing is being logged and there are all zeros in the rain fields. Does anyone have any ideas?
Nov 4 19:20:56 weather weewx[1133]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=1&winddir=0&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&windspeedmph=1&winddir=0&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&humidity=50&tempf=62.2&dewptf=43&baromin=29.35&battery=normal&rssi=3
Nov 4 19:20:56 weather weewx[1133]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=1&winddir=0&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&windspeedmph=1&winddir=0&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&humidity=50&tempf=62.2&dewptf=43&baromin=29.35&battery=normal&rssi=3
Nov 4 19:20:56 weather weewx[1133]: interceptor: MainThread: ignored parameter dailyrainin=0.55
Nov 4 19:20:56 weather weewx[1133]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 0.0, 'temperature.00002179.24C86E06B15C': 62.2, 'rssi.00002179.24C86E06B15C': 0.75, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'rainfall.00002179.24C86E06B15C': 0.0, 'dateTime.00002179.24C86E06B15C': 1478312457, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.35, 'dateTime': 1478312457, 'windspeed.00002179.24C86E06B15C': 1.0, 'humidity.00002179.24C86E06B15C': 50.0, 'usUnits': 1}
Nov 4 19:20:56 weather weewx[1133]: interceptor: MainThread: mapped packet: {'barometer': 29.35, 'outHumidity': 50.0, 'rain': 0.0, 'dateTime': 1478312457, 'windDir': 0.0, 'outTemp': 62.2, 'windSpeed': 1.0, 'usUnits': 1}
Hers is a copy of my sensor map in case that is the issue...
[[sensor_map]]
# bridge pressure = pressure..* inTemp = temperature..* inHumidity = humidity..* barometer = barometer.*.*
# 5n1 outTemp = temperature.00002179.* outHumidity = humidity.00002179.* windSpeed = windspeed.00002179.* windDir = winddir.00002179.* rain = rainfall.00002179.* # dewptf = dewpoint.00002179.*
# living room tower extraTemp1 = temperature.00012694.* extraHumid1 = humidity.00012694.*
# second tower extraTemp2 = temperature.00008384.* extraHumid2 = humidity.00008384.*Brad
mwall
Nov 5, 2016, 12:26:05 AM11/5/16
to weewx-user
On Friday, November 4, 2016 at 10:42:37 PM UTC-4, Brad Tucker wrote:
For some reason Rain is not being logged to the system w/ the interceptor model. I accidentally tripped the rain señor while working on the unit today, and the acurite display is reporting .55 inches however my plots show 0 as do the stats. I took a look at the database to make sure nothing is being logged and there are all zeros in the rain fields. Does anyone have any ideas?
brad,
is the output you posted from immediately after the bucket tip, or more than an hour after the bucket tip?
if you do some bucket tips, do they show up for both rainin and dailyrainin, or only dailyrainin?
wu defines the 'rainin' parameter is amount of rain in the past hour - a sliding window. it defines the 'dailyrainin' parameter is the amout of rain since the start of the day. i assume that the acurite bridge is using the same definitions, since it is posting to wu.
weewx expects 'rain' to be the amount of rain in the last interval, whether that interval is a LOOP packet or an archive record. so it needs to calculate the difference between consecutive dailyrainin values.
the interceptor was using rainin instead of dailyrainin for the delta. this has been fixed at commit 231fa14.
btw, your POSTs contain duplicate data. it is harmless, but you might want to figure out where that is coming from.
m
Brad Tucker
Nov 5, 2016, 12:45:30 AM11/5/16
to weewx-user
Mwall,
Thanks for getting back to me so quickly.
Im not sure if the bucket tips where within an hour of these logs. I went up to work on the sense and triggered the tips. I came back in and started fiddling with weewx after. I think it was roughly an hour later... Cant be exact though.
Ill have to check if the bucket tips show up in "rainin" I have to assume they do, however I didn't have debug on to see them. The problem is they aren't making there way to the database. They make there way to weather underground and to myacurite so I think it safe to say they are being broadcast. Happy to bring the station in and do bucket tips while I check the logs if you'd like to me in the morning. Don't want to get back up on the roof tonight in the dark ;)
"dailyrainin" is being triggered by the bucket drops. I hit it a few times and thats why you are seeing the .55" in the previous logs.
Im not sure where the duplicate data is coming from. One says "ServerThread: POST:" and the other says "MainThread: raw data:" Are those really duplicate or just 2 ways of displaying the same data?
Thanks again for your help,
Brad
Message has been deleted
Brad Tucker
Nov 5, 2016, 12:22:17 PM11/5/16
to weewx-user
Edited my last post.... It has been deleted.
GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=113&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=59.2&dewptf=41&baromin=29.35&battery=normal&rssi=3 HTTP/1.1Host: hubapi.myacurite.comUser-Agent: Hub/224Connection: close
GET /weatherstation/updateweatherstation.php?ID=KCATHOUS110&PASSWORD=XXXXXXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=113&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=59.2&dewptf=41&baromin=29.35&battery=normal&rssi=3 HTTP/1.1Connection: closeBrad Tucker
Nov 5, 2016, 12:46:27 PM11/5/16
to weewx-user
Good Morning Mwall,
I performed some bucket drops this morning. The logs follow below. It appears the information is being transmitted in both "rainin" and "dailyrainin" however they are not making there way into the database. I suspect maybe I have an error in my sensormap so I have included that as well. Both the TCPdumps and the Syslogs should be from the same time period, both happening within the hour of the bucket drops.
Hope this is the information you where looking for last night and thanks again!
Brad
tcpdump output:
E....f..d.E7....&f...c.PT^J.gz..P...{C..GET /weatherstation/updateweatherstation.php?ID=KCATHOUS110&PASSWORD=XXXXXXXX&dateutc=now&action=updateraw&realtime=1E..2.g..d.E.....&f...c.PT^Jzgz..P...Q...&rtfreq=36E..R.h..d.E.....&f...c.PT^J.gz..P....K..&id=24C86E06B15C&mt=5N1x31&sensor=00002179E..C.i..d.E.....&f...c.PT^J.gz..P.......&windspeedmph=2&winddir=135E..f.j..d.Ej....&f...c.PT^J.gz..P.......&rainin=0.13&dailyrainin=0.20&humidity=37&tempf=69.4&dewptf=41E..L.k..d.E.....&f...c.PT^K.gz..P....W..&baromin=29.36&battery=normal&rssi=3E..9.l..d.E.....&f...c.PT^K+gz..P....... HTTP/1.1E..A.m..d.E.....&f...c.PT^K<gz..P.......rtupdate.wunderground.com
E..x....d..o....4,.h.i.P..s[....P....3..GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1E..R....d.......4,.h.i.P..s.....P...\z..&id=24C86E06B15C&mt=5N1x31&sensor=00002179E..C....d.......4,.h.i.P..s.....P.......&windspeedmph=2&winddir=135E..f....d..~....4,.h.i.P..s.....P.......&rainin=0.13&dailyrainin=0.20&humidity=37&tempf=69.4&dewptf=41E..L....d.......4,.h.i.P..t.....P.......&baromin=29.36&battery=normal&rssi=3
E..x....d..8....4....m.P.z^nQ.pJP...e...GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1E..R....d..]....4....m.P.z^.Q.pJP.......&id=24C86E06B15C&mt=5N1x38&sensor=00002179E..C....d..k....4....m.P.z^.Q.pJP...2...&windspeedmph=2&humidity=39E..3....d..z....4....m.P.z_.Q.pJP.......&tempf=69.0E..L....d..`....4....m.P.z_.Q.pJP.......&baromin=29.36&battery=normal&rssi=3
E..x....d.%O....46._.k.P.u......P....]..GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1E..Q....d.%u....46._.k.P.u......P...^...&id=24C86E06B15C&mt=tower&sensor=00012694E..?....d.%.....46._.k.P.u.'....P.......&humidity=53&tempf=69.8E..L....d.%x....46._.k.P.u.>....P...l...&baromin=29.36&battery=normal&rssi=3
E..x....d..(....4....l.P..g.....P...N...GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1E..Q....d..N....4....l.P..g.....P....r..&id=24C86E06B15C&mt=tower&sensor=00008384E..?....d.._....4....l.P..h.....P...YQ..&humidity=35&tempf=80.5E..L....d..Q....4....l.P..h%....P.......&baromin=29.36&battery=normal&rssi=2syslog output:
Nov 5 09:18:40 weather weewx[4699]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=135&humidity=37&tempf=69.4&dewptf=41&baromin=29.36&battery=normal&rssi=3Nov 5 09:18:40 weather weewx[4699]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=135&humidity=37&tempf=69.4&dewptf=41&baromin=29.36&battery=normal&rssi=3Nov 5 09:18:40 weather weewx[4699]: interceptor: MainThread: ignored parameter realtime=1Nov 5 09:18:40 weather weewx[4699]: interceptor: MainThread: unrecognized parameter dewptf=41Nov 5 09:18:40 weather weewx[4699]: interceptor: MainThread: ignored parameter action=updaterawNov 5 09:18:40 weather weewx[4699]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 135.0, 'temperature.00002179.24C86E06B15C': 69.4, 'dateTime.00002179.24C86E06B15C': 1478362721, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'dateTime': 1478362721, 'humidity.00002179.24C86E06B15C': 37.0, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.36, 'rssi.00002179.24C86E06B15C': 0.75, 'windspeed.00002179.24C86E06B15C': 2.0, 'usUnits': 1}Nov 5 09:18:40 weather weewx[4699]: interceptor: MainThread: mapped packet: {'barometer': 29.36, 'outHumidity': 37.0, 'dateTime': 1478362721, 'windDir': 135.0, 'outTemp': 69.4, 'windSpeed': 2.0, 'usUnits': 1}
Nov 5 09:18:41 weather weewx[4699]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=135&humidity=37&tempf=69.4&dewptf=41&baromin=29.36&battery=normal&rssi=3Nov 5 09:18:41 weather weewx[4699]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=135&humidity=37&tempf=69.4&dewptf=41&baromin=29.36&battery=normal&rssi=3Nov 5 09:18:41 weather weewx[4699]: interceptor: MainThread: ignored parameter realtime=1Nov 5 09:18:41 weather weewx[4699]: interceptor: MainThread: unrecognized parameter dewptf=41Nov 5 09:18:41 weather weewx[4699]: interceptor: MainThread: ignored parameter action=updaterawNov 5 09:18:41 weather weewx[4699]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 135.0, 'temperature.00002179.24C86E06B15C': 69.4, 'dateTime.00002179.24C86E06B15C': 1478362722, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'dateTime': 1478362722, 'humidity.00002179.24C86E06B15C': 37.0, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.36, 'rssi.00002179.24C86E06B15C': 0.75, 'windspeed.00002179.24C86E06B15C': 2.0, 'usUnits': 1}Nov 5 09:18:41 weather weewx[4699]: interceptor: MainThread: mapped packet: {'barometer': 29.36, 'outHumidity': 37.0, 'dateTime': 1478362722, 'windDir': 135.0, 'outTemp': 69.4, 'windSpeed': 2.0, 'usUnits': 1}
Nov 5 09:18:59 weather weewx[4699]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=2&humidity=39&tempf=69.0&baromin=29.36&battery=normal&rssi=3Nov 5 09:18:59 weather weewx[4699]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=2&humidity=39&tempf=69.0&baromin=29.36&battery=normal&rssi=3Nov 5 09:18:59 weather weewx[4699]: interceptor: MainThread: ignored parameter realtime=1Nov 5 09:18:59 weather weewx[4699]: interceptor: MainThread: ignored parameter action=updaterawNov 5 09:18:59 weather weewx[4699]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x38', 'sensor_id.00002179.24C86E06B15C': '00002179', 'temperature.00002179.24C86E06B15C': 69.0, 'dateTime.00002179.24C86E06B15C': 1478362740, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'dateTime': 1478362740, 'humidity.00002179.24C86E06B15C': 39.0, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.36, 'rssi.00002179.24C86E06B15C': 0.75, 'windspeed.00002179.24C86E06B15C': 2.0, 'usUnits': 1}Nov 5 09:18:59 weather weewx[4699]: interceptor: MainThread: mapped packet: {'barometer': 29.36, 'outHumidity': 39.0, 'dateTime': 1478362740, 'outTemp': 69.0, 'windSpeed': 2.0, 'usUnits': 1}
Nov 5 09:19:00 weather weewx[4699]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=53&tempf=69.8&baromin=29.36&battery=normal&rssi=3Nov 5 09:19:00 weather weewx[4699]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=53&tempf=69.8&baromin=29.36&battery=normal&rssi=3Nov 5 09:19:00 weather weewx[4699]: interceptor: MainThread: ignored parameter realtime=1Nov 5 09:19:00 weather weewx[4699]: interceptor: MainThread: ignored parameter action=updaterawNov 5 09:19:00 weather weewx[4699]: interceptor: MainThread: raw packet: {'dateTime.00012694.24C86E06B15C': 1478362741, 'sensor_type.00012694.24C86E06B15C': 'tower', 'rssi.00012694.24C86E06B15C': 0.75, 'battery.00012694.24C86E06B15C': 0, 'dateTime': 1478362741, 'bridge_id.00012694.24C86E06B15C': '24C86E06B15C', 'usUnits': 1, 'humidity.00012694.24C86E06B15C': 53.0, 'usUnits.00012694.24C86E06B15C': 1, 'barometer.00012694.24C86E06B15C': 29.36, 'temperature.00012694.24C86E06B15C': 69.8, 'sensor_id.00012694.24C86E06B15C': '00012694'}Nov 5 09:19:00 weather weewx[4699]: interceptor: MainThread: mapped packet: {'extraHumid1': 53.0, 'extraTemp1': 69.8, 'barometer': 29.36, 'usUnits': 1, 'dateTime': 1478362741}
Nov 5 09:19:09 weather weewx[4699]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=35&tempf=80.5&baromin=29.36&battery=normal&rssi=2Nov 5 09:19:09 weather weewx[4699]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=35&tempf=80.5&baromin=29.36&battery=normal&rssi=2Nov 5 09:19:09 weather weewx[4699]: interceptor: MainThread: ignored parameter realtime=1Nov 5 09:19:09 weather weewx[4699]: interceptor: MainThread: ignored parameter action=updaterawNov 5 09:19:09 weather weewx[4699]: interceptor: MainThread: raw packet: {'humidity.00008384.24C86E06B15C': 35.0, 'dateTime.00008384.24C86E06B15C': 1478362750, 'battery.00008384.24C86E06B15C': 0, 'rssi.00008384.24C86E06B15C': 0.5, 'sensor_type.00008384.24C86E06B15C': 'tower', 'dateTime': 1478362750, 'usUnits.00008384.24C86E06B15C': 1, 'sensor_id.00008384.24C86E06B15C': '00008384', 'temperature.00008384.24C86E06B15C': 80.5, 'usUnits': 1, 'bridge_id.00008384.24C86E06B15C': '24C86E06B15C', 'barometer.00008384.24C86E06B15C': 29.36}Nov 5 09:19:09 weather weewx[4699]: interceptor: MainThread: mapped packet: {'extraTemp2': 80.5, 'barometer': 29.36, 'extraHumid2': 35.0, 'usUnits': 1, 'dateTime': 1478362750}Sensor Map:
[[sensor_map]] # bridge pressure = pressure..* inTemp = temperature..* inHumidity = humidity..* barometer = barometer.*.*
# 5n1 outTemp = temperature.00002179.* outHumidity = humidity.00002179.* windSpeed = windspeed.00002179.* windDir = winddir.00002179.* rain = rainfall.00002179.* # dewptf = dewpoint.00002179.*
# living room tower extraTemp1 = temperature.00012694.* extraHumid1 = humidity.00012694.*
# second tower extraTemp2 = temperature.00008384.* extraHumid2 = humidity.00008384.*On Friday, November 4, 2016 at 9:26:05 PM UTC-7, mwall wrote:
mwall
Nov 5, 2016, 12:55:47 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 12:46:27 PM UTC-4, Brad Tucker wrote:
I performed some bucket drops this morning. The logs follow below. It appears the information is being transmitted in both "rainin" and "dailyrainin" however they are not making there way into the database. I suspect maybe I have an error in my sensormap so I have included that as well. Both the TCPdumps and the Syslogs should be from the same time period, both happening within the hour of the bucket drops.
brad, thanks for that. are those tests done with interceptor v0.14?
m
Brad Tucker
Nov 5, 2016, 1:00:35 PM11/5/16
to weewx-user
Yes, that is correct. I have installed the latest off GitHub.
DRIVER_NAME = 'Interceptor'DRIVER_VERSION = '0.14'mwall
Nov 5, 2016, 1:25:30 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 12:46:27 PM UTC-4, Brad Tucker wrote:
syslog output:
brad,
i need to see log output like this, but for a POST that includes rainin and dailyrainin
none of the syslog snippets you posted have rainin or dailyrainin in the POST line
m
Brad Tucker
Nov 5, 2016, 1:46:46 PM11/5/16
to weewx-user
Strange... I looked through todays logs and none of them display a "POST" with the results. The info is in the TCPdump but not the syslog after processing. Nothing has changed on the system other then the new version of the interceptor driver. Not sure where to go from here... Open to suggestions...
Here is a log from last night that has the POST info you request. This was on the previous version of the driver. Not sure why it doesn't show up today?
Nov 4 19:19:02 weather weewx[1133]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=338&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&windspeedmph=0&winddir=338&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&humidity=50&tempf=62.2&dewptf=43&baromin=29.34&battery=normal&rssi=3Nov 4 19:19:02 weather systemd[1]: Starting Cleanup of Temporary Directories...Nov 4 19:19:02 weather weewx[1133]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=338&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&windspeedmph=0&winddir=338&rainin=0.00&dailyrainin=0.55&humidity=50&tempf=62.2&dewptf=43&humidity=50&tempf=62.2&dewptf=43&baromin=29.34&battery=normal&rssi=3Nov 4 19:19:02 weather weewx[1133]: interceptor: MainThread: ignored parameter realtime=1Nov 4 19:19:02 weather weewx[1133]: interceptor: MainThread: unrecognized parameter dewptf=43Nov 4 19:19:02 weather weewx[1133]: interceptor: MainThread: ignored parameter dailyrainin=0.55Nov 4 19:19:02 weather weewx[1133]: interceptor: MainThread: ignored parameter action=updaterawNov 4 19:19:02 weather weewx[1133]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 338.0, 'temperature.00002179.24C86E06B15C': 62.2, 'rssi.00002179.24C86E06B15C': 0.75, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'rainfall.00002179.24C86E06B15C': 0.0, 'dateTime.00002179.24C86E06B15C': 1478312343, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.34, 'dateTime': 1478312343, 'windspeed.00002179.24C86E06B15C': 0.0, 'humidity.00002179.24C86E06B15C': 50.0, 'usUnits': 1}Nov 4 19:19:02 weather weewx[1133]: interceptor: MainThread: mapped packet: {'barometer': 29.34, 'outHumidity': 50.0, 'rain': 0.0, 'dateTime': 1478312343, 'windDir': 338.0, 'outTemp': 62.2, 'windSpeed': 0.0, 'usUnits': 1}mwall
Nov 5, 2016, 2:04:26 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 1:46:46 PM UTC-4, Brad Tucker wrote:
Strange... I looked through todays logs and none of them display a "POST" with the results. The info is in the TCPdump but not the syslog after processing. Nothing has changed on the system other then the new version of the interceptor driver. Not sure where to go from here... Open to suggestions...Here is a log from last night that has the POST info you request. This was on the previous version of the driver. Not sure why it doesn't show up today?
brad,
i am pretty sure i made no changes to the interceptor that would change how it reports the POST requests that it receives.
the 'ServerThread: POST:' messages reflect what the intercept receives, before it does any processing
so if there is no rainin or dailyrainin in a post, it must be lost before it gets to the interceptor
what commands/process are you now using to feed data to the interceptor?
m
Brad Tucker
Nov 5, 2016, 2:14:55 PM11/5/16
to weewx-user
Mwall, Im using tcpdump and a perl script by radar attached below. Could this be a sensor map issue???
sudo tcpdump -A -n -p -l -i eth0 -s0 -W tcp dst port 80 | stdbuf -oL strings -n8 | stdbuf -oL grep "&" | ./acurite-lwp2.pl &
Im also using a perl script to process the file and make it readable from RADAR. I just installed a fresh copy in case I messed up somehow between last night and today.
#!/usr/bin/perl #===============================================================================## FILE: acurite-lwp.pl## USAGE: ./acurite-lwp.pl ## DESCRIPTION: ## OPTIONS: ---# REQUIREMENTS: ---# BUGS: ---# NOTES: ---# AUTHOR: Radar (), radar@localhost# ORGANIZATION: # VERSION: 0.1# CREATED: 10/08/2016 07:16:48 PM# Last modified: Mon Oct 31, 2016 10:49PM# REVISION: 1#===============================================================================
use strict;use warnings;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;$ua->agent("MyApp/0.1 ");
my $req = HTTP::Request->new(POST => 'http://192.168.1.7:8080');my $res;
my $newline = '';my $build = 0;my $obs_line = 'dateutc=now&action=updateraw&realtime=1';
#-------------------------------------------------------------------------------# the 5n1 sends two packets# 5N1x31 packet has windspeedmph & winddir & rainin & dailyrainin & # baromin & battery & rssi## 5N1x38 packet has windspeedmph & humidity & tempf & baromin & battery & rssi#-------------------------------------------------------------------------------while ( my $line = <> ) { if ( defined $line ) { if ($line =~ /GET /) { $build = 1; $newline = ''; }#-------------------------------------------------------------------------------# id & mt & sensor# id = smarthub mac address# mt sensor type# sensor number#------------------------------------------------------------------------------- if (($line =~ /&id=/) && ($build == 1)) { $line =~ s/.*&id=/&id=/xm; chomp $line; $newline = $line; }#-------------------------------------------------------------------------------# windspeedmph & humidity for 5n1 from 5N1x38 packet;# 5N1x31 packet has windspeedmph & winddir#------------------------------------------------------------------------------- if (($build == 1) && ($line =~ /&windspeedmph=/)) { $line =~ s/.*&windspeedmph=/&windspeedmph=/xm; chomp $line; $newline = $newline . $line; }#-------------------------------------------------------------------------------# look for 5N1x38 if found grab the tempf#------------------------------------------------------------------------------- if (($build == 1) && ($newline =~ /5N1x38/)) { if (($build == 1) && ($line =~ /&tempf=/)) { $line =~ s/.*&tempf=/&tempf=/xm; chomp $line; $newline = $newline . $line; }#-------------------------------------------------------------------------------# if not found grab humidity & tempf#------------------------------------------------------------------------------- } elsif (($build == 1) && ($line =~ /&humidity=/)) { $line =~ s/.*&humidity=/&humidity=/xm; chomp $line; $newline = $newline . $line; }#-------------------------------------------------------------------------------# rainin & dailyrainin for rain gauge and 5n1#------------------------------------------------------------------------------- if (($build == 1) && ($line =~ /&rainin=/)) { $line =~ s/.*&rainin=/&rainin=/xm; chomp $line; $newline = $newline . $line; }#-------------------------------------------------------------------------------# ProIn sensor# indoorhumidity & indoortempf#------------------------------------------------------------------------------- if (($build == 1) && ($line =~ /&indoorhumidity=/)) { $line =~ s/.*&indoorhumidity=/&indoorhumidity=/xm; chomp $line; $newline = $newline . $line; }#-------------------------------------------------------------------------------# the extra probs for a ProIn & ProOut sensors# probe & check & water (Water Detector)# probe & check & ptempf (Liquide and or Soil Temp)# probe & check & ptempf & phumidity (Spot Check Temperature and Humidity)#------------------------------------------------------------------------------- if (($build == 1) && ($line =~ /&probe=/)) { $line =~ s/.*&probe=/&probe=/xm; chomp $line; $newline = $newline . $line; }#-------------------------------------------------------------------------------# baromin & battery & rssi# baromin is from the smarthub#------------------------------------------------------------------------------- if (($build == 1) && ($line =~ /&baromin=/)) { $line =~ s/.*&baromin=/&baromin=/xm; chomp $line; $build = 2; $newline = $newline . $line; $obs_line = $obs_line . $newline; $newline = ''; }
if ($build == 2) { $build = 0; print "$obs_line\n"; $req->content("$obs_line"); $res = $ua->request($req); $obs_line = 'dateutc=now&action=updateraw&realtime=1'; if ( $res->is_success ) { print $res->content, "\n"; } else { print $res->status_line, "\n"; } } } #end of if definde $line} # end of while loop
mwall
Nov 5, 2016, 2:58:03 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 2:14:55 PM UTC-4, Brad Tucker wrote:
Mwall, Im using tcpdump and a perl script by radar attached below. Could this be a sensor map issue???
brad, before we can determine if it is a sensor_map issue, we need to see what is actually getting to the interceptor.
please try the attached combine-lines.pl
use it something like this:
tcpdump -i eth0 src X.X.X.X and port 80 | combine-lines.pl | curl http://localhost:9999 -s -d
first do the tcpdump piped to combine-lines to be sure that we're getting what we need. once that works, add the curl to connect it to the interceptor.
tcpdump captures the traffic and spits it out in multiple lines, combine-lines puts them together and strips off anything that is not cgi args, then curl posts those args to the interceptor web server.
m
Brad Tucker
Nov 5, 2016, 5:32:52 PM11/5/16
to weewx-user
Hello Matt,
as soon as I attempt to send it to curl it stops working. It doesn't seem to like the curl syntax provided. has a problem with -d...
It then just sits there with no output. I tried reading the man page but I don't understand what parameter its looking for me to put in.
Thanks for the new perl script. I have spent some time trying to get it to work and unfortunately it doesn't seem to work with the syntax given. Just for background I am running a raspberry pi w/ the latest version of raspian. It has two ethernet devices running in bridge mode. The acurite bridge is connected to eth1 and the router is plugged into eth0. I have attached an ifconfig -a so you can see the network setup:
br0 Link encap:Ethernet HWaddr 00:1f:f3:8b:40:f4 inet addr:192.168.1.7 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::21f:f3ff:fe8b:40f4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:397718 errors:0 dropped:0 overruns:0 frame:0 TX packets:101956 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:48136637 (45.9 MiB) TX bytes:53601798 (51.1 MiB)
eth0 Link encap:Ethernet HWaddr b8:27:eb:ab:27:9a inet addr:169.254.173.171 Bcast:169.254.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:616661 errors:0 dropped:0 overruns:0 frame:0 TX packets:308956 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:78114140 (74.4 MiB) TX bytes:72282022 (68.9 MiB)
eth1 Link encap:Ethernet HWaddr 00:1f:f3:8b:40:f4 inet addr:169.254.197.59 Bcast:169.254.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:205873 errors:0 dropped:0 overruns:0 frame:0 TX packets:455102 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:12516899 (11.9 MiB) TX bytes:63064200 (60.1 MiB)Closest thing I could get is with this:
sudo tcpdump -A -n -p -l -i eth0 -s0 -w - tcp dst port 80 | stdbuf -oL strings -n8 | ./combine.pl
output looks like this:
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=31&tempf=80.2&baromin=29.30&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=225&rainin=0.00&dailyrainin=0.20&humidity=20&tempf=87.6&dewptf=42&baromin=29.30&battery=normal&rssi=3hubapi.myacurite.comID=KCATHOUS110&PASSWORD=XXXXXXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=225&rainin=0.00&dailyrainin=0.20&humidity=20&tempf=87.6&dewptf=42&baromin=29.30&battery=normal&rssi=3rtupdate.wunderground.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=31&tempf=80.2&baromin=29.30&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=31&tempf=80.2&baromin=29.30&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=49&tempf=71.2&baromin=29.30&battery=normal&rssi=3hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=270&rainin=0.00&dailyrainin=0.20&humidity=20&tempf=87.6&dewptf=42&baromin=29.30&battery=normal&rssi=3hubapi.myacurite.comID=KCATHOUS110&PASSWORD=XXXXXXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=270&rainin=0.00&dailyrainin=0.20&humidity=20&tempf=87.6&dewptf=42&baromin=29.30&battery=normal&rssi=3rtupdate.wunderground.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=31&tempf=80.2&baromin=29.30&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1#E&id=24C86E06B15C&mt=tower&sensor=00012694#E&humidity=49&tempf=71.2#E&baromin=29.30&battery=normal&rssi=3#Edateutc=now&action=updateraw&realtime=1&E&id=24C86E06B15C&mt=5N1x38&sensor=00002179&E&windspeedmph=1&humidity=22&E&tempf=87.1&E&baromin=29.30&battery=normal&rssi=3&Edateutc=now&action=updateraw&realtime=1,E&id=24C86E06B15C&mt=tower&sensor=00008384,E&humidity=31&tempf=80.2,E&baromin=29.30&battery=normal&rssi=2,Edateutc=now&action=updateraw&realtime=14E&id=24C86E06B15C&mt=tower&sensor=000126944E&humidity=49&tempf=71.24E&baromin=29.30&battery=normal&rssi=44Edateutc=now&action=updateraw&realtime=1CE&id=24C86E06B15C&mt=5N1x31&sensor=00002179CE&windspeedmph=3&winddir=113CE&rainin=0.00&dailyrainin=0.20&humidity=22&tempf=87.1&dewptf=43CE&baromin=29.30&battery=normal&rssi=3CE
If I would try to to use something similar to the syntax provided I would get acks and seq w/o and info in the dump:
sudo tcpdump -i eth1 port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes14:29:45.633996 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [S], seq 458494429, win 400, options [mss 536], length 014:29:45.708706 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [S.], seq 825064456, ack 458494430, win 17922, options [mss 1460], length 014:29:45.710630 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 1, win 400, length 014:29:45.712456 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 1:81, ack 1, win 400, length 8014:29:45.788645 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [.], ack 81, win 17922, length 014:29:45.792628 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 81:122, ack 1, win 400, length 4114:29:45.795496 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 122:145, ack 1, win 400, length 2314:29:45.801577 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 145:181, ack 1, win 400, length 3614:29:45.803026 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 181:198, ack 1, win 400, length 1714:29:45.804895 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 198:239, ack 1, win 400, length 4114:29:45.868009 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [.], ack 122, win 17922, length 014:29:45.871427 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [.], ack 145, win 17922, length 014:29:45.873747 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 239:262, ack 1, win 400, length 2314:29:45.875715 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [.], ack 181, win 17922, length 014:29:45.879692 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [.], ack 198, win 17922, length 014:29:45.879807 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [.], ack 239, win 17922, length 014:29:45.947971 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [.], ack 262, win 17922, length 014:29:45.989474 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [P.], seq 1:401, ack 262, win 17922, length 40014:29:45.996607 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 401, win 400, length 014:29:46.072238 IP ec2-52-4-188-138.compute-1.amazonaws.com.http > 192.168.1.18.3868: Flags [FP.], seq 401:412, ack 262, win 17922, length 1114:29:46.073935 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 413, win 389, length 014:29:46.074997 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 413, win 400, length 014:29:46.076078 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [F.], seq 262, ack 413, win 400, length 014:29:46.076624 IP 192.168.1.18.3868 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [R.], seq 262, ack 413, win 400, length 0pi@new-host-9:~ $ sudo tcpdump -A -n -p -l -i eth0 -s0 -w - tcp dst port 80 | stdbuf -oL strings -n8 | ./combine.pl | curl http://192.168.1.7:8080 -s -dcurl: option -d: requires parameterIt then just sits there with no output. I tried reading the man page but I don't understand what parameter its looking for me to put in.
-d, --data <data> (HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. To post data purely binary, you should instead use the --data-binary option. To URL-encode the value of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command line, the data pieces specified will be merged together with a separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter @, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data @foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out.Ive spent a couple hours now trying, but Im just not getting anywhere. I am defeated :(
Thanks,
Brad
mwall
Nov 5, 2016, 5:52:36 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 5:32:52 PM UTC-4, Brad Tucker wrote:
Thanks for the new perl script. I have spent some time trying to get it to work and unfortunately it doesn't seem to work with the syntax given.
take it one step at a time.
what is the output you get from a simple tcpdump?
tcpdump -i eth0 dst port 80
then what is the output from tcpdump with all of the other options you have thrown in?
tcpdump -A -n -p -l -i eth0 -s0 -w -tcp dst port 80
post those then we can figure out why combine-lines.pl is not stripping off the spurious hubapi.acurite.com
m
m
Brad Tucker
Nov 5, 2016, 6:00:00 PM11/5/16
to weewx-user
sudo tcpdump -i eth0 dst port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes14:53:48.501215 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [S], seq 62518866, win 400, options [mss 536], length 014:53:48.582768 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 3179448309, win 400, length 014:53:48.584594 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 0:80, ack 1, win 400, length 8014:53:48.677787 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 80:121, ack 1, win 400, length 4114:53:48.680555 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 121:144, ack 1, win 400, length 2314:53:48.686696 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 144:180, ack 1, win 400, length 3614:53:48.688129 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 180:197, ack 1, win 400, length 1714:53:48.689983 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 197:238, ack 1, win 400, length 4114:53:48.758710 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [P.], seq 238:261, ack 1, win 400, length 2314:53:48.900334 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 401, win 400, length 014:53:48.979087 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 413, win 389, length 014:53:48.980138 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [.], ack 413, win 400, length 014:53:48.981227 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [F.], seq 261, ack 413, win 400, length 014:53:48.981757 IP 192.168.1.18.4102 > ec2-52-4-188-138.compute-1.amazonaws.com.http: Flags [R.], seq 261, ack 413, win 400, length 0^C14 packets captured14 packets received by filter0 packets dropped by kerneltcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes<<&�8vB$�n�E(n�d�o��4��/P&}��|EPP����UX���&�8vB$�n�Exn�d���4��/P&}��|EPP��GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1�UX�M``&�8vB$�n�ERn�d�C��4��/P&}�!|EPP�sI&id=24C86E06B15C&mt=5N1x31&sensor=00002179�UXkVQQ&�8vB$�n�ECn�d�Q��4��/P&}�K|EPP�ΐ&windspeedmph=1&winddir=203�UXFltt&�8vB$�n�Efn�d�-��4��/P&}�f|EPP�Ŧ&rainin=0.00&dailyrainin=0.20&humidity=30&tempf=83.2&dewptf=48�UX�ZZ&�8vB$�n�ELn�d�F��4��/P&}ؤ|EPP��S&baromin=29.28&battery=normal&rssi=3�UXP�GG&�8vB$�n�E9n�d�X��4��/P&}��|EPP��� HTTP/1.1Host: �UXl�__&�8vB$�n�EQod�?��4��/P&}��|EPP��jhubapi.myacurite.comUser-Agent: Hub/224�UXG�MM&�8vB$�n�E?od�P��4��/P&}�|EPP�2<Connection: close
�UX0�<<&�8vB$�n�E(od�f��4��/P&}�|EQ�P����UX�<<&�8vB$�n�E(od�e��4��/P&}�|EQ�P����UX�<<&�8vB$�n�E(od�d��4��/P&}�|EQ�P����UX3�<<&�8vB$�n�E(od�c��4��/P&}�|EQ�P����UXA�<<&�8vB$�n�E(od�b��4��/P&}�|EQ�P����UX�%<<&�8vB$�n�E,dv��&f��0P=�v�`�c �UX�s<<&�8vB$�n�E(o dv ��&f��0P=�v�i��P���UX�}��&�8vB$�n�E�oGET /weatherstation/updateweatherstation.php?ID=KCATHOUS110&PASSWORD=ofsglckk&dateutc=now&action=updateraw&realtime=1�UX� @@&�8vB$�n�E2o du���&f��0P=�wqi��P�m�&rtfreq=36�UX�``&�8vB$�n�ERo du���&f��0P=�w{i��P��du���&f��0P=�w�i��P�,]&windspeedmph=1&winddir=203�UXV�tt&�8vB$�n�Efodu���&f��0P=�w�i��P�#s&rainin=0.00&dailyrainin=0.20&humidity=30&tempf=83.2&dewptf=48�UX:ZZ&�8vB$�n�ELodu���&f��0P=�w�i��P��&baromin=29.28&battery=normal&rssi=3�UX�GG&�8vB$�n�E9odu���&f��0P=�x"i��P�� HTTP/1.1Host: �UX�OO&�8vB$�n�EAodu���&f��0P=�x3i��P�ƒrtupdate.wunderground.com�UX]MM&�8vB$�n�E?odu���&f��0P=�xLi��P��Connection: close
�UX�GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1�UX��__&�8vB$�n�EQodV<��46�_1P�p{rM�!P��w&id=24C86E06B15C&mt=tower&sensor=00012694�UX�MM&�8vB$�n�E?odVM��46�_1P�p{�M�!P�O&humidity=48&tempf=70.8�UX�ZZ&�8vB$�n�ELodV?��46�_1P�p{�M�!P���&baromin=29.28&battery=normal&rssi=3�UX\#GG&�8vB$�n�E9odVQ��46�_1P�p{�M�!P��� HTTP/1.1Host: �UX�*__&�8vB$�n�EQodV8��46�_1P�p{�M�!P��hubapi.myacurite.comUser-Agent: Hub/224�UXMM&�8vB$�n�E?o dVI��46�_1P�p|M�!P�CnConnection: close
^C44 packets captured44 packets received by filter0 packets dropped by kernel�UX�*<<&�8vB$�n�E(o!dV_��46�_1P�p|'M�P��#�UX�f<<&�8vB$�n�E(o"dV^��46�_1P�p|'M�P��"�UX k<<&�8vB$�n�E(o#dV]��46�_1P�p|'M�P���UXRo<<&�8vB$�n�E(o$dV\��46�_1P�p|'M�P���UXuq<<&�8vB$�n�E(o%dV[��46�_1P�p|'M�P��mwall
Nov 5, 2016, 6:02:25 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 5:32:52 PM UTC-4, Brad Tucker wrote:
Just for background I am running a raspberry pi w/ the latest version of raspian. It has two ethernet devices running in bridge mode. The acurite bridge is connected to eth1 and the router is plugged into eth0. I have attached an ifconfig -a so you can see the network setup:
good. since you have a bridge, you can simply sniff the packets; we do not have to worry about relaying. if you had a dns hijack configuration then you would have to configure a repeater/relay to get the packets to hubapi.acurite.com.
(there is not yet a weewx uploader for acurite.com. i am inclined to not bother with uploaders for cloudy services that are specific to a single type of hardware)
(it would be interesting to see what happens if you configure weewx to upload to wunderground in addition to the wunderground pass-through from the acurite bridge)
mwall
Nov 5, 2016, 6:33:59 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 6:00:00 PM UTC-4, Brad Tucker wrote:
sudo tcpdump -i eth0 dst port 80
sudo tcpdump -A -n -p -l -i eth0 -s0 -w - tcp dst port 80 (Please note the - tcp not -tcp) There is a space. w/o it I get 0 output.
good. neither of those alone is what we want, but tcpdump is doing what is supposed to do.
now post the output from this:
sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n8
the 'strings -n8' simply filters out any lines that are shorter than 8 characters
the -Anpl gives us ascii, no name lookups, non-promiscuous, line-buffered output from tcpdump. the -w - sends to stdout instead of file, and controls the number of bytes snarfed (probably not an issue here).
m
Brad Tucker
Nov 5, 2016, 6:47:45 PM11/5/16
to weewx-user
this is looking yummy :)
sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n8
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1l_&id=24C86E06B15C&mt=tower&sensor=00008384l_&humidity=30&tempf=79.3l_&baromin=29.29&battery=normal&rssi=2l_ HTTP/1.1Host: l_User-Agent: Hub/224l_Connection: closeGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1o_&id=24C86E06B15C&mt=tower&sensor=00012694o_&humidity=46&tempf=71.5o_&baromin=29.29&battery=normal&rssi=3o_ HTTP/1.1Host: o_User-Agent: Hub/224p_Connection: closeGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1q_&id=24C86E06B15C&mt=5N1x31&sensor=00002179q_&windspeedmph=1&winddir=113q_&rainin=0.00&dailyrainin=0.20&humidity=37&tempf=79.4&dewptf=50q_&baromin=29.29&battery=normal&rssi=3q_ HTTP/1.1Host: q_User-Agent: Hub/224q_Connection: closeGET /weatherstation/updateweatherstation.php?ID=KCATHOUS110&PASSWORD=ofsglckk&dateutc=now&action=updateraw&realtime=1r_&rtfreq=36r_&id=24C86E06B15C&mt=5N1x31&sensor=00002179r_&windspeedmph=1&winddir=113r_&rainin=0.00&dailyrainin=0.20&humidity=37&tempf=79.4&dewptf=50r_&baromin=29.29&battery=normal&rssi=3r_ HTTP/1.1Host: r_rtupdate.wunderground.comr_Connection: close^C114 packets captured114 packets received by filter0 packets dropped by kernelI went a head and ran it through the combine.pl as well:
sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n8 | ./combine.pl
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=79.3&baromin=29.28&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=79.3&baromin=29.28&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=1&humidity=37&tempf=78.8&baromin=29.28&battery=normal&rssi=3hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=46&tempf=71.5&baromin=29.28&battery=normal&rssi=3hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=79.3&baromin=29.28&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=1&winddir=90&rainin=0.00&dailyrainin=0.20&humidity=37&tempf=78.8&dewptf=51&baromin=29.28&battery=normal&rssi=3hubapi.myacurite.comID=KCATHOUS110&PASSWORD=ofsglckk&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=1&winddir=90&rainin=0.00&dailyrainin=0.20&humidity=37&tempf=78.8&dewptf=51&baromin=29.28&battery=normal&rssi=3rtupdate.wunderground.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=46&tempf=71.5&baromin=29.28&battery=normal&rssi=3hubapi.myacurite.com^C117 packets captured117 packets received by filter0 packets dropped by kernelmwall
Nov 5, 2016, 7:23:22 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 6:47:45 PM UTC-4, Brad Tucker wrote:
this is looking yummy :)
getting close. i don't know where the spurious l_, q_ etc are coming from in your tcpdump output, and i am unable to duplicate the behavior. could you send the tcpdump output to file and post that file?
sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 > dump.txt
meanwhile, i tightened up the combiner a bit. please try version 0.2 of combine-lines.pl from the interceptor repository:
https://raw.githubusercontent.com/matthewwall/weewx-interceptor/master/util/combine-lines.pl
that should prevent the hostnames from being appended to the cgi args
m
mwall
Nov 5, 2016, 7:25:15 PM11/5/16
to weewx-user
sorry, this is where to find the combiner now:
https://github.com/matthewwall/weewx-interceptor/blob/master/util/combine/combine-lines.pl
m
https://github.com/matthewwall/weewx-interceptor/blob/master/util/combine/combine-lines.pl
m
Brad Tucker
Nov 5, 2016, 7:39:30 PM11/5/16
to weewx-user
Here is the latest out put with the new combine.pl. I rebooted the pi as I was getting strange outputs as well. Once I reboot this new output was looking mighty fine...
-B
pi@weather:~ $ sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n8 | ./combine.pltcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=51&tempf=71.9&baromin=29.29&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=79.6&baromin=29.29&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=90&rainin=0.00&dailyrainin=0.20&humidity=51&tempf=71.9&dewptf=53&baromin=29.29&battery=normal&rssi=3hubapi.myacurite.comID=KCATHOUS110&PASSWORD=XXXXXXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=90&rainin=0.00&dailyrainin=0.20&humidity=51&tempf=71.9&dewptf=53&baromin=29.29&battery=normal&rssi=3rtupdate.wunderground.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=45&tempf=71.2&baromin=29.29&battery=normal&rssi=3hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=79.8&baromin=29.29&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=51&tempf=71.9&baromin=29.29&battery=normal&rssi=2hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=45&tempf=71.2&baromin=29.29&battery=normal&rssi=3hubapi.myacurite.comdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=79.8&baromin=29.29&battery=normal&rssi=2hubapi.myacurite.com^C131 packets captured131 packets received by filter0 packets dropped by kernelBrad Tucker
Nov 5, 2016, 7:46:58 PM11/5/16
to weewx-user
and a raw tcpdump:
sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n8
pi@weather:~ $ sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n8tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=1&winddir=293&rainin=0.00&dailyrainin=0.20&humidity=55&tempf=71.1&dewptf=54&baromin=29.29&battery=normal&rssi=3 HTTP/1.1User-Agent: Hub/224Connection: closeGET /weatherstation/updateweatherstation.php?ID=KCATHOUS110&PASSWORD=XXXXXXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=1&winddir=293&rainin=0.00&dailyrainin=0.20&humidity=55&tempf=71.1&dewptf=54&baromin=29.29&battery=normal&rssi=3 HTTP/1.1Connection: closeGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=80.0&baromin=29.29&battery=normal&rssi=2 HTTP/1.1GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=45&tempf=71.5&baromin=29.29&battery=normal&rssi=3 HTTP/1.1^C58 packets captured58 packets received by filter0 packets dropped by kernelmwall
Nov 5, 2016, 7:48:30 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 7:39:30 PM UTC-4, Brad Tucker wrote:
Here is the latest out put with the new combine.pl. I rebooted the pi as I was getting strange outputs as well. Once I reboot this new output was looking mighty fine...
use -n5 instead of -n8 as the argument to strings. that will ensure that the hostnames do not get appended.
at that point you should be able to connect to the interceptor using curl and xargs:
sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n5 | ./combine-lines.pl | xargs -n 1 curl http://localhost:9999 -s -d
that is all one line
m
Brad Tucker
Nov 5, 2016, 8:06:26 PM11/5/16
to weewx-user
Unfortunately once I add curl into the mix nothing happens:
pi@weather:~ $ sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n5
pi@weather:~ $ sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n5tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=45&tempf=71.7&baromin=29.28&battery=normal&rssi=3 HTTP/1.1Host:User-Agent: Hub/224Connection: closeGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=56&tempf=69.6&baromin=29.28&battery=normal&rssi=3 HTTP/1.1Host:User-Agent: Hub/224Connection: closeGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=80.2&baromin=29.28&battery=normal&rssi=2 HTTP/1.1Host:User-Agent: Hub/224Connection: closeGET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=45&tempf=71.7&baromin=29.28&battery=normal&rssi=3 HTTP/1.1Host:58 packets received by filter0 packets dropped by kernelpi@weather:~ $ sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n5 | ./combine-lines.pl tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesdateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=80.2&baromin=29.28&battery=normal&rssi=2dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=45&tempf=71.7&baromin=29.28&battery=normal&rssi=3dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=23&rainin=0.00&dailyrainin=0.20&humidity=56&tempf=69.6&dewptf=53&baromin=29.28&battery=normal&rssi=3ID=KCATHOUS110&PASSWORD=XXXXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=23&rainin=0.00&dailyrainin=0.20&humidity=56&tempf=69.6&dewptf=53&baromin=29.28&battery=normal&rssi=3dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=30&tempf=80.2&baromin=29.28&battery=normal&rssi=2^C72 packets captured72 packets received by filter0 packets dropped by kernelpi@weather:~ $ sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n5 | ./combine-lines.pl | xargs -n 1 curl http://192.168.1.7:8080 -s -dtcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytesNov 5 17:02:36 weather weewx[769]: interceptor: MainThread: empty queueNov 5 17:02:46 weather weewx[769]: interceptor: MainThread: empty queueNov 5 17:02:56 weather weewx[769]: interceptor: MainThread: empty queueThanks,
Brad
mwall
Nov 5, 2016, 8:21:00 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 8:06:26 PM UTC-4, Brad Tucker wrote:
Unfortunately once I add curl into the mix nothing happens:
so close! i'll bet the perl script is buffering.
try putting this after the 'use strict' line in the perl script:
$| = 1;
(that is supposed to be the 'less cryptic' way of doing it! gotta love perl :)
m
Brad Tucker
Nov 5, 2016, 8:38:48 PM11/5/16
to weewx-user
This appears to work!!!! I'm vpn'ed in from my phone and doing it at dinner while with the family. I'm getting dirty looks from the wife ;) I'll let it run and send you some logs as soon as I get home. I did notice some small errors but I see things parsing! Thanks for your help and your patience!!!
-Brad
Brad Tucker
Nov 5, 2016, 10:02:50 PM11/5/16
to weewx-user
Hello Matt,
Things seem to go ok for a few min but then the tcpdump stops going. It just holds and displays nothing more... It probably gets 50 or 60 of the "success": 1, "checkversion": "126" messages... We are getting really close. Still havnt seen the rain show up in the graphed plots but the logs for the 5N1 have rain entries in the POST statement.
sudo tcpdump -Anpl -s0 -w - -i eth0 dst port 80 | stdbuf -oL strings -n5 | ./combine-lines.pl | xargs -n 1 curl http://192.168.1.7:8080 -s -d
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }{ "success": 1, "checkversion": "126" }
tail -f /var/log/syslog |grep POST
pi@weather:/home/weewx $ tail -f /var/log/syslog |grep POSTNov 5 18:56:31 weather weewx[2574]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=86&tempf=59.0&baromin=29.28&battery=normal&rssi=3Nov 5 18:56:40 weather weewx[2574]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=48&tempf=71.9&baromin=29.28&battery=normal&rssi=3Nov 5 18:56:44 weather weewx[2574]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=31&tempf=81.4&baromin=29.28&battery=normal&rssi=2Nov 5 18:57:01 weather weewx[2574]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=90&rainin=0.00&dailyrainin=0.20&humidity=86&tempf=59.0&dewptf=54&baromin=29.28&battery=normal&rssi=3Nov 5 18:57:01 weather weewx[2574]: interceptor: ServerThread: POST: ID=KCATHOUS110&PASSWORD=XXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=90&rainin=0.00&dailyrainin=0.20&humidity=86&tempf=59.0&dewptf=54&baromin=29.28&battery=normal&rssi=3Nov 5 18:57:09 weather weewx[2574]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=86&tempf=59.0&baromin=29.28&battery=normal&rssi=2tail -f /var/log/syslog
Nov 5 18:45:15 weather weewx[2331]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1*&id=24C86E06B15C&mt=tower&sensor=00008384*&humidity=31&tempf=81.1*&baromin=29.28&battery=normal&rssi=2*Nov 5 18:45:15 weather rsyslogd-2007: action 'action 17' suspended, next retry is Sat Nov 5 18:46:45 2016 [try http://www.rsyslog.com/e/2007 ]Nov 5 18:45:15 weather weewx[2331]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1*&id=24C86E06B15C&mt=tower&sensor=00008384*&humidity=31&tempf=81.1*&baromin=29.28&battery=normal&rssi=2*Nov 5 18:45:15 weather weewx[2331]: interceptor: MainThread: parse failed for dateutc=now&action=updateraw&realtime=1*&id=24C86E06B15C&mt=tower&sensor=00008384*&humidity=31&tempf=81.1*&baromin=29.28&battery=normal&rssi=2*: invalid literal for float(): 81.1*Nov 5 18:45:15 weather weewx[2331]: interceptor: MainThread: raw packet: {'usUnits..': 1, 'dateTime..': 1478396715, 'usUnits': 1, 'dateTime': 1478396715}Nov 5 18:45:15 weather weewx[2331]: interceptor: MainThread: mapped packet: {'usUnits': 1, 'dateTime': 1478396715}Nov 5 18:45:15 weather weewx[2331]: reportengine: Running reports for latest time in the database.Nov 5 18:45:15 weather weewx[2331]: reportengine: Running report StandardReportNov 5 18:45:15 weather weewx[2331]: reportengine: Found configuration file /home/weewx/skins/Sofaskin/skin.conf for report StandardReportNov 5 18:45:16 weather weewx[2331]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1+&id=24C86E06B15C&mt=5N1x38&sensor=00002179+&windspeedmph=0&humidity=85+&tempf=59.9+&baromin=29.28&battery=normal&rssi=3+Nov 5 18:45:16 weather weewx[2331]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1+&id=24C86E06B15C&mt=5N1x38&sensor=00002179+&windspeedmph=0&humidity=85+&tempf=59.9+&baromin=29.28&battery=normal&rssi=3+Nov 5 18:45:16 weather weewx[2331]: interceptor: MainThread: parse failed for dateutc=now&action=updateraw&realtime=1+&id=24C86E06B15C&mt=5N1x38&sensor=00002179+&windspeedmph=0&humidity=85+&tempf=59.9+&baromin=29.28&battery=normal&rssi=3+: invalid literal for float(): 59.9+Nov 5 18:45:16 weather weewx[2331]: interceptor: MainThread: raw packet: {'usUnits..': 1, 'dateTime..': 1478396716, 'usUnits': 1, 'dateTime': 1478396716}Nov 5 18:45:16 weather weewx[2331]: interceptor: MainThread: mapped packet: {'usUnits': 1, 'dateTime': 1478396716}Nov 5 18:45:16 weather weewx[2331]: cheetahgenerator: using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']Nov 5 18:45:25 weather weewx[2331]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=14&id=24C86E06B15C&mt=tower&sensor=000126944&humidity=48&tempf=71.94&baromin=29.28&battery=normal&rssi=34Nov 5 18:45:25 weather weewx[2331]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=14&id=24C86E06B15C&mt=tower&sensor=000126944&humidity=48&tempf=71.94&baromin=29.28&battery=normal&rssi=34Nov 5 18:45:25 weather weewx[2331]: interceptor: MainThread: ignored parameter realtime=14Nov 5 18:45:25 weather weewx[2331]: interceptor: MainThread: ignored parameter action=updaterawNov 5 18:45:25 weather weewx[2331]: interceptor: MainThread: raw packet: {'barometer.000126944.24C86E06B15C': 29.28, 'sensor_id.000126944.24C86E06B15C': '000126944', 'dateTime.000126944.24C86E06B15C': 1478396725, 'usUnits.000126944.24C86E06B15C': 1, 'rssi.000126944.24C86E06B15C': 8.5, 'bridge_id.000126944.24C86E06B15C': '24C86E06B15C', 'dateTime': 1478396725, 'humidity.000126944.24C86E06B15C': 48.0, 'sensor_type.000126944.24C86E06B15C': 'tower', 'battery.000126944.24C86E06B15C': 0, 'usUnits': 1, 'temperature.000126944.24C86E06B15C': 71.94}Nov 5 18:45:25 weather weewx[2331]: interceptor: MainThread: mapped packet: {'barometer': 29.28, 'usUnits': 1, 'dateTime': 1478396725}Nov 5 18:45:32 weather weewx[2331]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1;&id=24C86E06B15C&mt=tower&sensor=00008384;&humidity=31&tempf=81.1<&baromin=29.28&battery=normal&rssi=2<Nov 5 18:45:32 weather weewx[2331]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1;&id=24C86E06B15C&mt=tower&sensor=00008384;&humidity=31&tempf=81.1<&baromin=29.28&battery=normal&rssi=2<Nov 5 18:45:32 weather weewx[2331]: interceptor: MainThread: parse failed for dateutc=now&action=updateraw&realtime=1;&id=24C86E06B15C&mt=tower&sensor=00008384;&humidity=31&tempf=81.1<&baromin=29.28&battery=normal&rssi=2<: invalid literal for float(): 81.1<Nov 5 18:45:32 weather weewx[2331]: interceptor: MainThread: raw packet: {'usUnits..': 1, 'dateTime..': 1478396732, 'usUnits': 1, 'dateTime': 1478396732}Nov 5 18:45:32 weather weewx[2331]: interceptor: MainThread: mapped packet: {'usUnits': 1, 'dateTime': 1478396732}Nov 5 18:45:35 weather weewx[2331]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1>&id=24C86E06B15C&mt=5N1x31&sensor=00002179>&windspeedmph=0&winddir=45>&rainin=0.00&dailyrainin=0.20&humidity=85&tempf=59.9&dewptf=55>&baromin=29.28&battery=normal&rssi=3>Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1>&id=24C86E06B15C&mt=5N1x31&sensor=00002179>&windspeedmph=0&winddir=45>&rainin=0.00&dailyrainin=0.20&humidity=85&tempf=59.9&dewptf=55>&baromin=29.28&battery=normal&rssi=3>Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: ignored parameter realtime=1>Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: unrecognized parameter dewptf=55>Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: ignored parameter rainin=0.00Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: parse failed for dateutc=now&action=updateraw&realtime=1>&id=24C86E06B15C&mt=5N1x31&sensor=00002179>&windspeedmph=0&winddir=45>&rainin=0.00&dailyrainin=0.20&humidity=85&tempf=59.9&dewptf=55>&baromin=29.28&battery=normal&rssi=3>: invalid literal for float(): 45>Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: skipping rain measurement of 0.2: no last rainNov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: raw packet: {'dateTime': 1478396735, 'usUnits..': 1, 'temperature..': 59.9, 'windspeed..': 0.0, 'dateTime..': 1478396735, 'usUnits': 1, 'rainfall..': None, 'barometer..': 29.28}Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: mapped packet: {'barometer': 29.28, 'inTemp': 59.9, 'usUnits': 1, 'dateTime': 1478396735}Nov 5 18:45:35 weather weewx[2331]: interceptor: ServerThread: POST: ID=KCATHOUS110&PASSWORD=XXXX&dateutc=now&action=updateraw&realtime=1>&rtfreq=36>&id=24C86E06B15C&mt=5N1x31&sensor=00002179>&windspeedmph=0&winddir=45>&rainin=0.00&dailyrainin=0.20&humidity=85&tempf=59.9&dewptf=55>&baromin=29.28&battery=normal&rssi=3>Nov 5 18:45:35 weather weewx[2331]: interceptor: MainThread: raw data: ID=KCATHOUS110&PASSWORD=00000000&dateutc=now&action=updateraw&realtime=1>&rtfreq=36>&id=24C86E06B15C&mt=5N1x31&sensor=00002179>&windspeedmph=0&winddir=45>&rainin=0.00&dailyrainin=0.20&humidity=85&tempf=59.9&dewptf=55>&baromin=29.28&battery=normal&rssi=3>I did notice some of the posts that have the Weather Underground PASSWORD have XXXX in them. I wrote out the one that had the password with 00000000. Not sure if you where hiding it in the logs or if something else is going on...
Thanks,
Brad
Brad Tucker
Nov 5, 2016, 10:06:39 PM11/5/16
to weewx-user
Hey Matt,
to get going again, I have to restart weewx completely...
This is in syslog when the tcpdump chokes...
Nov 5 18:58:55 weather weewx[2574]: interceptor: MainThread: parse failed for dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=31&tempf=81.4&baromin=29.28&battery=normal&rssi=2#012dateutc=now&action=updateraw&realtime=1^&id=24C86E06B15C&mt=tower&sensor=00012694^&humidity=48&tempf=71.9^&baromin=29.28&battery=normal&rssi=3^: dictionary update sequence element #10 has length 3; 2 is requiredNov 5 18:58:55 weather weewx[2574]: engine: Main loop exiting. Shutting engine down.Nov 5 18:58:55 weather weewx[2574]: engine: Shutting down StdReport threadNov 5 18:58:55 weather weewx[2574]: engine: StdReport thread has been terminatedNov 5 18:58:55 weather weewx[2574]: interceptor: MainThread: shutting down server threadNov 5 18:58:55 weather weewx[2574]: engine: Caught unrecoverable exception in engine:Nov 5 18:58:55 weather weewx[2574]: **** 'dateTime'Nov 5 18:58:55 weather weewx[2574]: **** Traceback (most recent call last):Nov 5 18:58:55 weather weewx[2574]: **** File "/home/weewx/bin/weewx/engine.py", line 844, in mainNov 5 18:58:55 weather weewx[2574]: **** engine.run()Nov 5 18:58:55 weather weewx[2574]: **** File "/home/weewx/bin/weewx/engine.py", line 186, in runNov 5 18:58:55 weather weewx[2574]: **** for packet in self.console.genLoopPackets():Nov 5 18:58:55 weather weewx[2574]: **** File "/home/weewx/bin/user/interceptor.py", line 1447, in genLoopPacketsNov 5 18:58:55 weather weewx[2574]: **** pkt = self._device.parser.parse(data)Nov 5 18:58:55 weather weewx[2574]: **** File "/home/weewx/bin/user/interceptor.py", line 477, in parseNov 5 18:58:55 weather weewx[2574]: **** return self.parse_wu(s)Nov 5 18:58:55 weather weewx[2574]: **** File "/home/weewx/bin/user/interceptor.py", line 513, in parse_wuNov 5 18:58:55 weather weewx[2574]: **** return self.add_identifiers(pkt)Nov 5 18:58:55 weather weewx[2574]: **** File "/home/weewx/bin/user/interceptor.py", line 568, in add_identifiersNov 5 18:58:56 weather weewx[2574]: **** packet = {'dateTime': pkt['dateTime'], 'usUnits': pkt['usUnits']}Nov 5 18:58:56 weather weewx[2574]: **** KeyError: 'dateTime'Nov 5 18:58:56 weather weewx[2574]: **** Exiting.to get going again, I have to restart weewx completely...
sudo service weewx restart
-Brad
mwall
Nov 5, 2016, 10:57:52 PM11/5/16
to weewx-user
On Saturday, November 5, 2016 at 10:06:39 PM UTC-4, Brad Tucker wrote:
Hey Matt,This is in syslog when the tcpdump chokes...
there is the root cause - the ^ and #012 and double && indicate problems in the cgi string formatting. it is probably due to dangling characters at the end of tcpdump strings (like the p_ r_ and other trailing garbage we saw earlier in this thread).
you'll want to run the tcpdump and stdbuf part of the chain to see what makes tcpdump stick random characters on the end of the strings
fwiw, i've been running the interceptor for months using the apache/nginx cgi proxy approach. i've been keen to get the tcpdump/tcpflow/xargs approach to work reliably for systems where you cannot run a proxy (and for another way to test).
also, i'm giving some thought to integrating jerome's pcap implementation - as he points out, it is a cleaner approach - but it would still be nice to get the tcpdump/tcpflow approach to work.
m
Brad Tucker
Nov 5, 2016, 11:45:15 PM11/5/16
to weewx-user
I have an extra SD card. Gonna try a fresh install of everything and see what happens. This pi seems buggy lately. I installed a real time clock the other day. Im wondering if thats part of the issue. Ill reload and report back in the morning. Ive never set up a proxy server, the bridge just seemed to be the easiest and make the most since to me when reading the options. It also gave me the ability to update the acurite bridge w/o hassle.
Thanks for all the help today.
Brad
Brad Tucker
Nov 6, 2016, 2:25:55 AM11/6/16
to weewx-user
Hello Matt!
I think I have some good news. After I did the clean install tcpdump worked exactly as it had before. However this time I tried to give tcpflow a shot. It has a great output and then once I run it through your perl script it looks really pretty and all on one line. It will run for a good 10 min then it crashes. I don't believe the crash is because of bad characters in the tcpdump this time though. Its as clean as Ive ever seen! This looks the most promising. Please take a look and let me know what you think.
Thanks,
Brad
sudo tcpflow -C -s tcp dst port 80
GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=180&rainin=0.00&dailyrainin=0.00&humidity=93&tempf=51.5&dewptf=49&baromin=29.30&battery=normal&rssi=3GET /weatherstation/updateweatherstation.php?ID=KCATHOUS110&PASSWORD=00000000&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=180&rainin=0.00&dailyrainin=0.00&humidity=93&tempf=51.5&dewptf=49&baromin=29.30&battery=normal&rssi=3 HTTP/1.1Host:
Connection: close
GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=93&tempf=51.5&baromin=29.30&battery=normal&rssi=3 HTTP/1.1Host:
User-Agent: Hub/224
Connection: close
GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.6&baromin=29.30&battery=normal&rssi=4 HTTP/1.1Host:User-Agent: Hub/224
Connection: close
GET /weatherstation/updateweatherstation?dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=34&tempf=81.4&baromin=29.30&battery=normal&rssi=2sudo tcpflow -C -s tcp dst port 80 | ./combine-lines.pl
tcpflow: listening on eth0
dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=34&tempf=81.4&baromin=29.30&battery=normal&rssi=2
dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.6&baromin=29.30&battery=normal&rssi=4
dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=90&rainin=0.00&dailyrainin=0.00&humidity=93&tempf=51.5&dewptf=49&baromin=29.30&battery=normal&rssi=3
ID=KCATHOUS110&PASSWORD=00000000dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=90&rainin=0.00&dailyrainin=0.00&humidity=93&tempf=51.5&dewptf=49&baromin=29.30&battery=normal&rssi=3
dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=93&tempf=51.4&baromin=29.30&battery=normal&rssi=3syslog:
Nov 6 00:00:35 weather weewx[19425]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=180&rainin=0.00&dailyrainin=0.00&humidity=95&tempf=51.4&dewptf=50&baromin=29.30&battery=normal&rssi=3Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=180&rainin=0.00&dailyrainin=0.00&humidity=95&tempf=51.4&dewptf=50&baromin=29.30&battery=normal&rssi=3Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter realtime=1Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter dewptf=50Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter rainin=0.00Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter action=updaterawNov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 180.0, 'temperature.00002179.24C86E06B15C': 51.4, 'rssi.00002179.24C86E06B15C': 0.75, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'rainfall.00002179.24C86E06B15C': 0.0, 'dateTime.00002179.24C86E06B15C': 1478415635, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.3, 'dateTime': 1478415635, 'windspeed.00002179.24C86E06B15C': 0.0, 'humidity.00002179.24C86E06B15C': 95.0, 'usUnits': 1}Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: mapped packet: {'barometer': 29.3, 'outHumidity': 95.0, 'rain': 0.0, 'dateTime': 1478415635, 'windDir': 180.0, 'outTemp': 51.4, 'windSpeed': 0.0, 'usUnits': 1}Nov 6 00:00:35 weather weewx[19425]: interceptor: ServerThread: POST: ID=KCATHOUS110&PASSWORD=XXXX&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=180&rainin=0.00&dailyrainin=0.00&humidity=95&tempf=51.4&dewptf=50&baromin=29.30&battery=normal&rssi=3Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: raw data: ID=KCATHOUS110&PASSWORD=00000000&dateutc=now&action=updateraw&realtime=1&rtfreq=36&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=0&winddir=180&rainin=0.00&dailyrainin=0.00&humidity=95&tempf=51.4&dewptf=50&baromin=29.30&battery=normal&rssi=3Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter realtime=1Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter dewptf=50Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter rainin=0.00Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter rtfreq=36Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter action=updaterawNov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter PASSWORD=00000000Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: ignored parameter ID=KCATHOUS110Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 180.0, 'temperature.00002179.24C86E06B15C': 51.4, 'dateTime.00002179.24C86E06B15C': 1478415636, 'rssi.00002179.24C86E06B15C': 0.75, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'rainfall.00002179.24C86E06B15C': 0.0, 'humidity.00002179.24C86E06B15C': 95.0, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.3, 'dateTime': 1478415636, 'windspeed.00002179.24C86E06B15C': 0.0, 'usUnits': 1}Nov 6 00:00:35 weather weewx[19425]: interceptor: MainThread: mapped packet: {'barometer': 29.3, 'outHumidity': 95.0, 'rain': 0.0, 'dateTime': 1478415636, 'windDir': 180.0, 'outTemp': 51.4, 'windSpeed': 0.0, 'usUnits': 1}Nov 6 00:00:41 weather weewx[19425]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.8&baromin=29.30&battery=normal&rssi=4Nov 6 00:00:41 weather weewx[19425]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.8&baromin=29.30&battery=normal&rssi=4Nov 6 00:00:41 weather weewx[19425]: interceptor: MainThread: ignored parameter realtime=1Nov 6 00:00:41 weather weewx[19425]: interceptor: MainThread: ignored parameter action=updaterawNov 6 00:00:41 weather weewx[19425]: interceptor: MainThread: raw packet: {'dateTime.00012694.24C86E06B15C': 1478415641, 'sensor_type.00012694.24C86E06B15C': 'tower', 'rssi.00012694.24C86E06B15C': 1.0, 'battery.00012694.24C86E06B15C': 0, 'dateTime': 1478415641, 'bridge_id.00012694.24C86E06B15C': '24C86E06B15C', 'usUnits': 1, 'humidity.00012694.24C86E06B15C': 51.0, 'usUnits.00012694.24C86E06B15C': 1, 'barometer.00012694.24C86E06B15C': 29.3, 'temperature.00012694.24C86E06B15C': 70.8, 'sensor_id.00012694.24C86E06B15C': '00012694'}Nov 6 00:00:41 weather weewx[19425]: interceptor: MainThread: mapped packet: {'extraHumid1': 51.0, 'extraTemp1': 70.8, 'barometer': 29.3, 'usUnits': 1, 'dateTime': 1478415641}Nov 6 00:00:44 weather weewx[19425]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=34&tempf=81.4&baromin=29.30&battery=normal&rssi=2Nov 6 00:00:44 weather weewx[19425]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00008384&humidity=34&tempf=81.4&baromin=29.30&battery=normal&rssi=2Nov 6 00:00:44 weather weewx[19425]: interceptor: MainThread: ignored parameter realtime=1Nov 6 00:00:44 weather weewx[19425]: interceptor: MainThread: ignored parameter action=updaterawNov 6 00:00:44 weather weewx[19425]: interceptor: MainThread: raw packet: {'humidity.00008384.24C86E06B15C': 34.0, 'dateTime.00008384.24C86E06B15C': 1478415644, 'battery.00008384.24C86E06B15C': 0, 'rssi.00008384.24C86E06B15C': 0.5, 'sensor_type.00008384.24C86E06B15C': 'tower', 'dateTime': 1478415644, 'usUnits.00008384.24C86E06B15C': 1, 'sensor_id.00008384.24C86E06B15C': '00008384', 'temperature.00008384.24C86E06B15C': 81.4, 'usUnits': 1, 'bridge_id.00008384.24C86E06B15C': '24C86E06B15C', 'barometer.00008384.24C86E06B15C': 29.3}Nov 6 00:00:44 weather weewx[19425]: interceptor: MainThread: mapped packet: {'extraTemp2': 81.4, 'barometer': 29.3, 'extraHumid2': 34.0, 'usUnits': 1, 'dateTime': 1478415644}Nov 6 00:00:54 weather weewx[19425]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=93&tempf=51.5&baromin=29.30&battery=normal&rssi=3Nov 6 00:00:54 weather weewx[19425]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=0&humidity=93&tempf=51.5&baromin=29.30&battery=normal&rssi=3Nov 6 00:00:54 weather weewx[19425]: interceptor: MainThread: ignored parameter realtime=1Nov 6 00:00:54 weather weewx[19425]: interceptor: MainThread: ignored parameter action=updaterawNov 6 00:00:54 weather weewx[19425]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x38', 'sensor_id.00002179.24C86E06B15C': '00002179', 'temperature.00002179.24C86E06B15C': 51.5, 'dateTime.00002179.24C86E06B15C': 1478415654, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'dateTime': 1478415654, 'humidity.00002179.24C86E06B15C': 93.0, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.3, 'rssi.00002179.24C86E06B15C': 0.75, 'windspeed.00002179.24C86E06B15C': 0.0, 'usUnits': 1}Nov 6 00:00:54 weather weewx[19425]: interceptor: MainThread: mapped packet: {'barometer': 29.3, 'outHumidity': 93.0, 'dateTime': 1478415654, 'outTemp': 51.5, 'windSpeed': 0.0, 'usUnits': 1}Nov 6 00:00:58 weather weewx[19425]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.8&baromin=29.30&battery=normal&rssi=4Nov 6 00:00:58 weather weewx[19425]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.8&baromin=29.30&battery=normal&rssi=4Nov 6 00:00:58 weather weewx[19425]: interceptor: MainThread: ignored parameter realtime=1Nov 6 00:00:58 weather weewx[19425]: interceptor: MainThread: ignored parameter action=updaterawNov 6 00:00:58 weather weewx[19425]: interceptor: MainThread: raw packet: {'dateTime.00012694.24C86E06B15C': 1478415658, 'sensor_type.00012694.24C86E06B15C': 'tower', 'rssi.00012694.24C86E06B15C': 1.0, 'battery.00012694.24C86E06B15C': 0, 'dateTime': 1478415658, 'bridge_id.00012694.24C86E06B15C': '24C86E06B15C', 'usUnits': 1, 'humidity.00012694.24C86E06B15C': 51.0, 'usUnits.00012694.24C86E06B15C': 1, 'barometer.00012694.24C86E06B15C': 29.3, 'temperature.00012694.24C86E06B15C': 70.8, 'sensor_id.00012694.24C86E06B15C': '00012694'}Nov 6 00:00:58 weather weewx[19425]: interceptor: MainThread: mapped packet: {'extraHumid1': 51.0, 'extraTemp1': 70.8, 'barometer': 29.3, 'usUnits': 1, 'dateTime': 1478415658}Syslog Crash:
Nov 6 00:01:13 weather weewx[19425]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 270.0, 'temperature.00002179.24C86E06B15C': 51.5, 'dateTime.00002179.24C86E06B15C': 1478415673, 'rssi.00002179.24C86E06B15C': 0.75, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'rainfall.00002179.24C86E06B15C': 0.0, 'humidity.00002179.24C86E06B15C': 93.0, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.3, 'dateTime': 1478415673, 'windspeed.00002179.24C86E06B15C': 0.0, 'usUnits': 1}Nov 6 00:01:13 weather weewx[19425]: interceptor: MainThread: mapped packet: {'barometer': 29.3, 'outHumidity': 93.0, 'rain': 0.0, 'dateTime': 1478415673, 'windDir': 270.0, 'outTemp': 51.5, 'windSpeed': 0.0, 'usUnits': 1}Nov 6 00:01:13 weather weewx[19425]: ftpupload: Uploaded file /weekbarometer.pngNov 6 00:01:13 weather weewx[19425]: ftpupload: Uploaded file /monthinside.pngNov 6 00:01:13 weather weewx[19425]: ftpupload: Uploaded file /monthtempchill.pngNov 6 00:01:13 weather weewx[19425]: ftpupload: Uploaded file /dayradiation.pngNov 6 00:01:13 weather weewx[19425]: ftpupload: Uploaded file /yearwind.pngNov 6 00:01:13 weather weewx[19425]: ftpupload: Uploaded file /yearuv.pngNov 6 00:01:13 weather weewx[19425]: ftpupload: Uploaded file /yearrx.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /weekwinddir.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /weekrain.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /dayrx.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /daybarometer.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /monthrain.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /weektempchill.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /yearwinddir.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /daywind.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /year.htmlNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /week.htmlNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /daytempdew.pngNov 6 00:01:14 weather weewx[19425]: ftpupload: Uploaded file /weektempdew.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /yeartempdew.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /weekpond.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /yearbarometer.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /monthradiation.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /index.htmlNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /monthwind.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /yearradiation.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /weekinside.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /weekrx.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /dayuv.pngNov 6 00:01:15 weather weewx[19425]: ftpupload: Uploaded file /yearhilow.pngNov 6 00:01:16 weather weewx[19425]: ftpupload: Uploaded file /yearwindvec.pngNov 6 00:01:16 weather weewx[19425]: ftpupload: Uploaded file /RSS/weewx_rss.xmlNov 6 00:01:16 weather weewx[19425]: ftpupload: Uploaded file /NOAA/NOAA-2016.txtNov 6 00:01:16 weather weewx[19425]: ftpupload: Uploaded file /NOAA/NOAA-2016-11.txtNov 6 00:01:16 weather weewx[19425]: reportengine: ftp'd 56 files in 6.19 secondsNov 6 00:01:16 weather weewx[19425]: reportengine: Running report RSYNCNov 6 00:01:16 weather weewx[19425]: reportengine: Found configuration file /home/weewx/skins/Rsync/skin.conf for report RSYNCNov 6 00:01:16 weather weewx[19425]: reportengine: rsync upload not requested. Skipped.Nov 6 00:01:23 weather weewx[19425]: interceptor: MainThread: empty queueNov 6 00:01:25 weather weewx[19425]: interceptor: ServerThread: POST: /weewx/index.htmlNov 6 00:01:25 weather weewx[19425]: interceptor: MainThread: raw data: /weewx/index.htmlNov 6 00:01:25 weather weewx[19425]: engine: Main loop exiting. Shutting engine down.Nov 6 00:01:25 weather weewx[19425]: engine: Shutting down StdReport threadNov 6 00:01:25 weather weewx[19425]: engine: StdReport thread has been terminatedNov 6 00:01:25 weather weewx[19425]: interceptor: MainThread: shutting down server threadNov 6 00:01:25 weather weewx[19425]: interceptor: ServerThread: POST: /weewx/js/modernizr-2.6.2.min.jsNov 6 00:01:25 weather weewx[19425]: engine: Caught unrecoverable exception in engine:Nov 6 00:01:25 weather weewx[19425]: **** need more than 1 value to unpackNov 6 00:01:25 weather weewx[19425]: **** Traceback (most recent call last):Nov 6 00:01:25 weather weewx[19425]: **** File "/home/weewx/bin/weewx/engine.py", line 844, in mainNov 6 00:01:25 weather weewx[19425]: **** engine.run()Nov 6 00:01:25 weather weewx[19425]: **** File "/home/weewx/bin/weewx/engine.py", line 186, in runNov 6 00:01:25 weather weewx[19425]: **** for packet in self.console.genLoopPackets():Nov 6 00:01:25 weather weewx[19425]: **** File "/home/weewx/bin/user/interceptor.py", line 1447, in genLoopPacketsNov 6 00:01:25 weather weewx[19425]: **** pkt = self._device.parser.parse(data)Nov 6 00:01:25 weather weewx[19425]: **** File "/home/weewx/bin/user/interceptor.py", line 478, in parseNov 6 00:01:25 weather weewx[19425]: **** return self.parse_chaney(s)Nov 6 00:01:25 weather weewx[19425]: **** File "/home/weewx/bin/user/interceptor.py", line 522, in parse_chaneyNov 6 00:01:25 weather weewx[19425]: **** (n, v) = x.split('=')Nov 6 00:01:25 weather weewx[19425]: **** ValueError: need more than 1 value to unpackNov 6 00:01:25 weather weewx[19425]: **** Exiting.Nov 6 00:04:27 weather rsyslogd-2007: action 'action 17' suspended, next retry is Sun Nov 6 00:05:57 2016 [try http://www.rsyslog.com/e/2007 ]Nov 6 00:04:29 weather systemd[1]: Stopping LSB: weewx weather system...Nov 6 00:04:29 weather weewx[19670]: Stopping weewx weather system: weewx not running....Nov 6 00:04:29 weather systemd[1]: Stopped LSB: weewx weather system.Brad Tucker
Nov 6, 2016, 2:32:46 AM11/6/16
to weewx-user
Decided to turn off the FTP skin and see if it changed anything. I still got the crash and here is the syslog:
Nov 6 00:28:32 weather weewx[19819]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.6&baromin=29.29&battery=normal&rssi=3Nov 6 00:28:32 weather weewx[19819]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=tower&sensor=00012694&humidity=51&tempf=70.6&baromin=29.29&battery=normal&rssi=3Nov 6 00:28:32 weather weewx[19819]: interceptor: MainThread: ignored parameter realtime=1Nov 6 00:28:32 weather weewx[19819]: interceptor: MainThread: ignored parameter action=updaterawNov 6 00:28:32 weather weewx[19819]: interceptor: MainThread: raw packet: {'dateTime.00012694.24C86E06B15C': 1478417312, 'sensor_type.00012694.24C86E06B15C': 'tower', 'rssi.00012694.24C86E06B15C': 0.75, 'battery.00012694.24C86E06B15C': 0, 'dateTime': 1478417312, 'bridge_id.00012694.24C86E06B15C': '24C86E06B15C', 'usUnits': 1, 'humidity.00012694.24C86E06B15C': 51.0, 'usUnits.00012694.24C86E06B15C': 1, 'barometer.00012694.24C86E06B15C': 29.29, 'temperature.00012694.24C86E06B15C': 70.6, 'sensor_id.00012694.24C86E06B15C': '00012694'}Nov 6 00:28:32 weather weewx[19819]: interceptor: MainThread: mapped packet: {'extraHumid1': 51.0, 'extraTemp1': 70.6, 'barometer': 29.29, 'usUnits': 1, 'dateTime': 1478417312}Nov 6 00:28:34 weather weewx[19819]: interceptor: ServerThread: POST: /weewx/index.htmlNov 6 00:28:34 weather weewx[19819]: interceptor: MainThread: raw data: /weewx/index.htmlNov 6 00:28:34 weather weewx[19819]: engine: Main loop exiting. Shutting engine down.Nov 6 00:28:34 weather weewx[19819]: interceptor: MainThread: shutting down server threadNov 6 00:28:34 weather weewx[19819]: engine: Caught unrecoverable exception in engine:Nov 6 00:28:34 weather weewx[19819]: **** need more than 1 value to unpackNov 6 00:28:34 weather weewx[19819]: **** Traceback (most recent call last):Nov 6 00:28:34 weather weewx[19819]: **** File "/home/weewx/bin/weewx/engine.py", line 844, in mainNov 6 00:28:34 weather weewx[19819]: **** engine.run()Nov 6 00:28:34 weather weewx[19819]: **** File "/home/weewx/bin/weewx/engine.py", line 186, in runNov 6 00:28:34 weather weewx[19819]: **** for packet in self.console.genLoopPackets():Nov 6 00:28:34 weather weewx[19819]: **** File "/home/weewx/bin/user/interceptor.py", line 1447, in genLoopPacketsNov 6 00:28:34 weather weewx[19819]: **** pkt = self._device.parser.parse(data)Nov 6 00:28:34 weather weewx[19819]: **** File "/home/weewx/bin/user/interceptor.py", line 478, in parseNov 6 00:28:34 weather weewx[19819]: **** return self.parse_chaney(s)Nov 6 00:28:34 weather weewx[19819]: **** File "/home/weewx/bin/user/interceptor.py", line 522, in parse_chaneyNov 6 00:28:34 weather weewx[19819]: **** (n, v) = x.split('=')Nov 6 00:28:34 weather weewx[19819]: **** ValueError: need more than 1 value to unpackNov 6 00:28:34 weather weewx[19819]: **** Exiting.mwall
Nov 6, 2016, 7:47:09 AM11/6/16
to weewx-user
On Sunday, November 6, 2016 at 2:32:46 AM UTC-5, Brad Tucker wrote:
Decided to turn off the FTP skin and see if it changed anything. I still got the crash and here is the syslog:
tcpdump is picking up the traffic from your web server/browser and the weewx web pages.
you need to adjust the tcpdump filter so that it only captures traffic from the acurite bridge
try adding a src clause something like this:
tcpdump ... src X.X.X.X and dst port 80
where X.X.X.X is the address of the bridge
or just listen on the eth1 interface (the one to which the bridge is connected) instead of the eth0 interface (the one to which the rest of the network is connected)
m
Brad Tucker
Nov 6, 2016, 11:26:13 AM11/6/16
to weewx-user
Hello Matt,
Here is my working command:
sudo tcpflow -C -i eth1 -s tcp dst port 80 | ./combine-lines.pl | xargs -n 1 curl http://192.168.1.19:9999 -s -d
Thanks again for all the hard work on this. I would have given up days ago w/o you :)
If you'd like me to test the pcap code you implemented last night Im happy to do so once I test the rain sensor.
Thanks
-Brad
Brad Tucker
Nov 6, 2016, 12:14:08 PM11/6/16
to weewx-user
Matt,
One last question... How would you recommend implementing the tcpflow command so it runs with the service. Currently Im running it manually from the command line with an "&" to send it to the background but Id like to automate this.
Photos attached
-Brad
Brad Tucker
Nov 6, 2016, 12:15:34 PM11/6/16
to weewx-user
And.. Just in case you'd like to see the syslog I've included it as well:
Nov 6 08:50:17 weather weewx[20110]: interceptor: ServerThread: POST: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=180&rainin=0.03&dailyrainin=0.03&humidity=50&tempf=70.3&dewptf=51&baromin=29.38&battery=normal&rssi=3
Nov 6 08:50:17 weather weewx[20110]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x31&sensor=00002179&windspeedmph=2&winddir=180&rainin=0.03&dailyrainin=0.03&humidity=50&tempf=70.3&dewptf=51&baromin=29.38&battery=normal&rssi=3
Nov 6 08:50:17 weather weewx[20110]: interceptor: MainThread: ignored parameter realtime=1
Nov 6 08:50:17 weather weewx[20110]: interceptor: MainThread: ignored parameter dewptf=51
Nov 6 08:50:17 weather weewx[20110]: interceptor: MainThread: ignored parameter rainin=0.03
Nov 6 08:50:17 weather weewx[20110]: interceptor: MainThread: ignored parameter action=updateraw
Nov 6 08:50:17 weather weewx[20110]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x31', 'sensor_id.00002179.24C86E06B15C': '00002179', 'winddir.00002179.24C86E06B15C': 180.0, 'temperature.00002179.24C86E06B15C': 70.3, 'rssi.00002179.24C86E06B15C': 0.75, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'rainfall.00002179.24C86E06B15C': 0.0, 'dateTime.00002179.24C86E06B15C': 1478451017, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.38, 'dateTime': 1478451017, 'windspeed.00002179.24C86E06B15C': 2.0, 'humidity.00002179.24C86E06B15C': 50.0, 'usUnits': 1}
Nov 6 08:50:17 weather weewx[20110]: interceptor: MainThread: mapped packet: {'barometer': 29.38, 'outHumidity': 50.0, 'rain': 0.0, 'dateTime': 1478451017, 'windDir': 180.0, 'outTemp': 70.3, 'windSpeed': 2.0, 'usUnits': 1}
Nov 6 08:50:17 weather weewx[20110]: manager: added record 2016-11-06 08:50:00 PST (1478451000) to database 'weewx.sdb'
Nov 6 08:50:17 weather weewx[20110]: manager: added record 2016-11-06 08:50:00 PST (1478451000) to daily summary in 'weewx.sdb'
Nov 6 08:50:18 weather weewx[20110]: reportengine: Running reports for latest time in the database.
Nov 6 08:50:18 weather weewx[20110]: reportengine: Running report StandardReport
Nov 6 08:50:18 weather weewx[20110]: engine: garbage collected 17356 objectsmwall
Nov 6, 2016, 12:49:17 PM11/6/16
to weewx-user
On Sunday, November 6, 2016 at 12:14:08 PM UTC-5, Brad Tucker wrote:
And here is the celebration dance... Hope Im not doing this too early but I have a couple hours of tcpflow going and I see data in both my database and on my skin. PIZZA PARTY!
thank *you* for being patient. nice work!
One last question... How would you recommend implementing the tcpflow command so it runs with the service. Currently Im running it manually from the command line with an "&" to send it to the background but Id like to automate this.
put the one-liner in a shell script, say /usr/local/bin/wxcapture.sh:
#!/bin/sh
sudo tcpflow -C -i eth1 -s tcp dst port 80 | ./combine-lines.pl | xargs -n 1 curl http://192.168.1.19:9999 -s -d
then make the script executable:
sudo chmod 755 /usr/local/bin/wxcapture.sh
then invoke capture.sh in your init system somewhere. one approach is to invoke it in /etc/rc.local. another approach is to invoke it in the start section of the weewx init script. another approach is to create a systemd unit file for it.
m
Message has been deleted
mwall
Nov 6, 2016, 12:52:18 PM11/6/16
to weewx-user
btw, you'll need the full path to combine-lines.pl, so you might want to put combine-lines.pl in /usr/local/bin and invoke it like this:
#!/bin/sh
tcpflow -C -i eth1 -s tcp dst port 80 | /usr/local/bin/combine-lines.pl | xargs -n 1 curl http://192.168.1.19:9999 -s -d
Brad Tucker
Nov 6, 2016, 2:44:25 PM11/6/16
to weewx-user
Great. I was thinking to add it to the initd startup but I like the idea of adding if it to the weewx initd instead. Thanks for the thoughts.
Thanks,
Brad
Brad Tucker
Nov 7, 2016, 12:08:35 AM11/7/16
to weewx-user
Good Evening Matt,
I thought tcpflow would go forever but forever happened to only be 12 hours ;) After running for a very long time everything crashed. Ive been keeping a mindful eye on it so I caught it pretty early... Attached is a syslog. Let me know if you see anything I should look for. The error codes are the same as the last crash. Not ure that helps any...
Thanks,
Brad
Nov 6 20:47:52 weather weewx[20110]: interceptor: MainThread: raw packet: {'dateTime.00012694.24C86E06B15C': 1478494073, 'sensor_type.00012694.24C86E06B15C': 'tower', 'rssi.00012694.24C86E06B15C': 0.75, 'battery.00012694.24C86E06B15C': 0, 'dateTime': 1478494073, 'bridge_id.00012694.24C86E06B15C': '24C86E06B15C', 'usUnits': 1, 'humidity.00012694.24C86E06B15C': 51.0, 'usUnits.00012694.24C86E06B15C': 1, 'barometer.00012694.24C86E06B15C': 29.36, 'temperature.00012694.24C86E06B15C': 70.8, 'sensor_id.00012694.24C86E06B15C': '00012694'}Nov 6 20:47:52 weather weewx[20110]: interceptor: MainThread: mapped packet: {'extraHumid1': 51.0, 'extraTemp1': 70.8, 'barometer': 29.36, 'usUnits': 1, 'dateTime': 1478494073}Nov 6 20:48:02 weather weewx[20110]: interceptor: MainThread: empty queueNov 6 20:48:05 weather weewx[20110]: interceptor: ServerThread: POST: /Nov 6 20:48:05 weather weewx[20110]: interceptor: MainThread: raw data: /Nov 6 20:48:05 weather weewx[20110]: engine: Main loop exiting. Shutting engine down.Nov 6 20:48:05 weather weewx[20110]: engine: Shutting down StdReport threadNov 6 20:48:05 weather weewx[20110]: engine: StdReport thread has been terminatedNov 6 20:48:05 weather weewx[20110]: interceptor: MainThread: shutting down server threadNov 6 20:48:05 weather weewx[20110]: engine: Caught unrecoverable exception in engine:Nov 6 20:48:05 weather weewx[20110]: **** need more than 1 value to unpackNov 6 20:48:05 weather weewx[20110]: **** Traceback (most recent call last):Nov 6 20:48:05 weather weewx[20110]: **** File "/home/weewx/bin/weewx/engine.py", line 844, in mainNov 6 20:48:05 weather weewx[20110]: **** engine.run()Nov 6 20:48:05 weather weewx[20110]: **** File "/home/weewx/bin/weewx/engine.py", line 186, in runNov 6 20:48:05 weather weewx[20110]: **** for packet in self.console.genLoopPackets():Nov 6 20:48:05 weather weewx[20110]: **** File "/home/weewx/bin/user/interceptor.py", line 1447, in genLoopPacketsNov 6 20:48:05 weather weewx[20110]: **** pkt = self._device.parser.parse(data)Nov 6 20:48:05 weather weewx[20110]: **** File "/home/weewx/bin/user/interceptor.py", line 478, in parseNov 6 20:48:05 weather weewx[20110]: **** return self.parse_chaney(s)Nov 6 20:48:05 weather weewx[20110]: **** File "/home/weewx/bin/user/interceptor.py", line 522, in parse_chaneyNov 6 20:48:05 weather weewx[20110]: **** (n, v) = x.split('=')Nov 6 20:48:05 weather weewx[20110]: **** ValueError: need more than 1 value to unpackNov 6 20:48:05 weather weewx[20110]: **** Exiting.mwall
Nov 7, 2016, 10:57:39 AM11/7/16
to weewx-user
On Monday, November 7, 2016 at 12:08:35 AM UTC-5, Brad Tucker wrote:
Good Evening Matt,I thought tcpflow would go forever but forever happened to only be 12 hours ;) After running for a very long time everything crashed. Ive been keeping a mindful eye on it so I caught it pretty early... Attached is a syslog. Let me know if you see anything I should look for. The error codes are the same as the last crash. Not ure that helps any...
hi brad,
please try the latest interceptor.py. i added rejection of unexpected cgi formats at commit 2ad77b7
m
Brad Tucker
Nov 7, 2016, 11:09:41 AM11/7/16
to weewx-user
It has been loaded. Ill report back in the morning if all goes well.
Thanks Matt,
Brad
Brad Tucker
Nov 8, 2016, 10:12:45 AM11/8/16
to weewx-user
It's been running for 24 hours :)
I'll let it go for a few more days and let you know my findings.
I'll let it go for a few more days and let you know my findings.
Thanks Matt!
B
Brad Tucker
Nov 11, 2016, 12:24:37 AM11/11/16
to weewx-user
Still going strong! Thanks Matt!
B
Brad Tucker
Nov 13, 2016, 12:42:56 AM11/13/16
to weewx-user
Hey Matt,
I had the interceptor crash this morning. I have attached the current logs.
Thanks,
Brad
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: raw data: dateutc=now&action=updateraw&realtime=1&id=24C86E06B15C&mt=5N1x38&sensor=00002179&windspeedmph=1&humidity=43&tempf=58.8&baromin=29.39&battery=normal&rssi=3
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: ignored parameter realtime=1
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: ignored parameter action=updateraw
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: raw packet: {'sensor_type.00002179.24C86E06B15C': '5N1x38', 'sensor_id.00002179.24C86E06B15C': '00002179', 'temperature.00002179.24C86E06B15C': 58.8, 'dateTime.00002179.24C86E06B15C': 1478962358, 'usUnits.00002179.24C86E06B15C': 1, 'battery.00002179.24C86E06B15C': 0, 'dateTime': 1478962358, 'humidity.00002179.24C86E06B15C': 43.0, 'bridge_id.00002179.24C86E06B15C': '24C86E06B15C', 'barometer.00002179.24C86E06B15C': 29.39, 'rssi.00002179.24C86E06B15C': 0.75, 'windspeed.00002179.24C86E06B15C': 1.0, 'usUnits': 1}
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: mapped packet: {'barometer': 29.39, 'outHumidity': 43.0, 'dateTime': 1478962358, 'outTemp': 58.8, 'windSpeed': 1.0, 'usUnits': 1}
Nov 12 06:52:37 weather weewx[725]: interceptor: ServerThread: POST: /
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: raw data: /
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: raw packet: {}
Nov 12 06:52:37 weather weewx[725]: engine: Main loop exiting. Shutting engine down.
Nov 12 06:52:37 weather weewx[725]: engine: Shutting down StdReport thread
Nov 12 06:52:37 weather weewx[725]: engine: StdReport thread has been terminated
Nov 12 06:52:37 weather weewx[725]: interceptor: MainThread: shutting down server thread
Nov 12 06:52:38 weather weewx[725]: engine: Caught unrecoverable exception in engine:
Nov 12 06:52:38 weather weewx[725]: **** 'dateTime'
Nov 12 06:52:38 weather weewx[725]: **** Traceback (most recent call last):
Nov 12 06:52:38 weather weewx[725]: **** File "/home/weewx/bin/weewx/engine.py", line 844, in main
Nov 12 06:52:38 weather weewx[725]: **** engine.run()
Nov 12 06:52:38 weather weewx[725]: **** File "/home/weewx/bin/weewx/engine.py", line 186, in run
Nov 12 06:52:38 weather weewx[725]: **** for packet in self.console.genLoopPackets():
Nov 12 06:52:38 weather weewx[725]: **** File "/home/weewx/bin/user/interceptor.py", line 1597, in genLoopPackets
Nov 12 06:52:38 weather weewx[725]: **** pkt = self._device.parser.map_to_fields(pkt, self._obs_map)
Nov 12 06:52:38 weather weewx[725]: **** File "/home/weewx/bin/user/interceptor.py", line 701, in map_to_fields
Nov 12 06:52:38 weather weewx[725]: **** return Consumer.Parser.map_to_fields(pkt, sensor_map)
Nov 12 06:52:38 weather weewx[725]: **** File "/home/weewx/bin/user/interceptor.py", line 355, in map_to_fields
Nov 12 06:52:38 weather weewx[725]: **** packet = {'dateTime': pkt['dateTime'], 'usUnits': pkt['usUnits']}
Nov 12 06:52:38 weather weewx[725]: **** KeyError: 'dateTime'
Nov 12 06:52:38 weather weewx[725]: **** Exiting.
Brad Tucker
Nov 13, 2016, 12:55:46 AM11/13/16
to weewx-user
Hello Matt,
I starting thinking about the time stamp on the log I just sent, and that was right around the time I typed in the IP address to find the boot version and firmware version of the Acurite Bridge for RADAR. That got me thinking... So I went in and viewed that website again and sure enough thats whats killing interceptor/weewx... If you log onto the http server of the Acurite Bride it takes it down...
Guess it makes since when you think about it... Not sure what could be done about it, but at least I know what not to do anymore ;)
Take Care,
Brad
Liz
Nov 13, 2016, 1:17:47 AM11/13/16
to weewx...@googlegroups.com
On Sat, 12 Nov 2016 21:55:45 -0800 (PST)
Brad Tucker <br...@makeitrad.com> wrote:
> Hello Matt,
>
> I starting thinking about the time stamp on the log I just sent, and
> that was right around the time I typed in the IP address to find the
> boot version and firmware version of the Acurite Bridge for RADAR.
> That got me thinking... So I went in and viewed that website again
> and sure enough thats whats killing interceptor/weewx... If you log
> onto the http server of the Acurite Bride it takes it down...
>
> Guess it makes since when you think about it... Not sure what could
> be done about it, but at least I know what not to do anymore ;)
>
> Take Care,
> Brad
What port does the bridge use to communicate with their server?
Brad Tucker <br...@makeitrad.com> wrote:
> Hello Matt,
>
> I starting thinking about the time stamp on the log I just sent, and
> that was right around the time I typed in the IP address to find the
> boot version and firmware version of the Acurite Bridge for RADAR.
> That got me thinking... So I went in and viewed that website again
> and sure enough thats whats killing interceptor/weewx... If you log
> onto the http server of the Acurite Bride it takes it down...
>
> Guess it makes since when you think about it... Not sure what could
> be done about it, but at least I know what not to do anymore ;)
>
> Take Care,
> Brad
Is there a possibility to block that port at your router?
Liz
mwall
Nov 13, 2016, 1:24:33 AM11/13/16
to weewx-user
On Sunday, November 13, 2016 at 12:55:46 AM UTC-5, Brad Tucker wrote:
Hello Matt,I starting thinking about the time stamp on the log I just sent, and that was right around the time I typed in the IP address to find the boot version and firmware version of the Acurite Bridge for RADAR. That got me thinking... So I went in and viewed that website again and sure enough thats whats killing interceptor/weewx... If you log onto the http server of the Acurite Bride it takes it down...
brad,
you can fix it by tweaking your capture command. are you still using this:
sudo tcpflow -C -i eth1 -s tcp dst port 80 | ./combine-lines.pl | xargs -n 1 curl http://192.168.1.19:9999 -s -d
if so, then try indicating the source something like this:
that way you will capture only the traffic from the bridge, not from your web browser too.
anyway, i added a fix to the interceptor so it will just log such traffic instead of dying.
m
sudo tcpflow -C -i eth1 -s tcp src X.X.X.X and dst port 80 | ./combine-lines.pl | xargs -n 1 curl http://192.168.1.19:9999 -s -d
that way you will capture only the traffic from the bridge, not from your web browser too.
anyway, i added a fix to the interceptor so it will just log such traffic instead of dying.
m
Brad Tucker
Nov 13, 2016, 1:26:01 AM11/13/16
to weewx-user
The Acurite Bridge is running it own web server on port 80. It basically just shows you the MAC address of the Bridge along with the firmware and hardware versions. Since my TCPflow command is sniffing any traffic and trying to parse it from that IP address on port 80 it grabs the traffic and kills weewx. For most uses its not a big deal. I have no need to see the firmware version or the hardware version.
Blocking the port 80 at the router would not work in my circumstance. I have a raspberry pi running wth 2 ethernet cards acting in bridge mode. It is between the accurate bridge and the router. It sniffs all traffic being sent from the acurite bride on port 80 that is headed to myacurite and parses that information, along with sending it off to Myacurite so I can still access that interface.
Thanks,
Brad
Brad Tucker
Nov 13, 2016, 1:29:08 AM11/13/16
to weewx-user
Thanks for the tip Matt. I never changed the command because it seemed to be working great... If it aint broke why fix it ;) Ill give the new line a shot though. Thanks for help yet again!
B
Brad Tucker
Nov 13, 2016, 1:52:45 AM11/13/16
to weewx-user
Hey Matt,
Thanks again,
Tried your fix and it worked great. I had to remove the "tcp" modifier on my line, but I did test browsing the web site on the bridge an no more crashes.
sudo tcpflow -C -i eth1 -s src host 192.168.1.18 and dst port 80 | ./combine-lines.pl | xargs -n 1 curl http://192.168.1.19:9999 -s -d
Thanks again,
Brad
On Saturday, November 12, 2016 at 10:24:33 PM UTC-8, mwall wrote:
Reply all
Reply to author
Forward
0 new messages