help with ambient weather 2902 / interceptor / pihole

[Christopher Lanus]

hello there, i recently got an ambient weather 2902 station, and am interested in storing my data locally, which brought me to weewx, and eventually here.

i've done a fair bit of reading/experimenting, and am still unable to intercept my weather data. i'll do my best describing what my setup looks like:

- weather station up and running, connected to wifi,
  - reporting data to ambient weather and wunderground.
  - have wunderground station id
  - have a record of this station's local ip in my network
- weewx installed on rpi3 running raspian (via [[http://weewx.com/docs/debian.htm][debian instructions]])
  - this pi is also running owncloud, served by nginx on ports 80/443
  - this pi has a static ip
- interceptor driver installed via [[https://github.com/matthewwall/weewx-interceptor][its github]]
- pihole running on a pi zero
  - this pi has a static ip
  - my router (asus rt-ac66u) uses this pi as its dns server
  - i have noted that my station makes regular calls to `time.windows.com`
    - and only infrequent to `rtupdate.wunderground.com`
  - i have updated the `/etc/hosts`, adding records to resolve both `time.windows.com` and `rtupdate.wunderground.com` to the ip running weewx
- i have configured weewx.conf

  station_type = Interceptor
...
  [Interceptor]
      driver = user.interceptor
      device_type = observer
      port = 9000
...
  [[Wunderground]]
    enable = true
..

- i have started/restarted weewx daemon
- i have manually run interceptor
  - do not see any traffic
  - however, when i manually hit my weewx machine's ip on port 9000, i see it logged in my terminal
- running `journalctl -u weewx.service`, i've seen the following messages:

Jan 21 21:32:28 cql-server weewx[5408]: restx: Wunderground-PWS: Data for station KILLAGRA32 will be posted
Jan 21 21:32:28 cql-server weewx[5408]: restx: PWSweather: Posting not enabled.
Jan 21 21:32:28 cql-server weewx[5408]: restx: CWOP: Posting not enabled.
Jan 21 21:32:28 cql-server weewx[5408]: restx: WOW: Posting not enabled.
Jan 21 21:32:28 cql-server weewx[5408]: restx: AWEKAS: Posting not enabled.
Jan 21 21:32:28 cql-server weewx[5408]: engine: Starting up weewx version 3.9.2
Jan 21 21:32:28 cql-server weewx[5408]: engine: Starting main packet loop.
Jan 21 21:47:16 cql-server weewx[5408]: cheetahgenerator: Generated 8 files for report SeasonsReport in 3.75 seconds
Jan 21 21:47:17 cql-server weewx[5408]: imagegenerator: Generated 14 images for SeasonsReport in 1.04 seconds
Jan 21 21:47:17 cql-server weewx[5408]: copygenerator: copied 5 files to /var/www/html/weewx
Jan 21 21:50:04 cql-server weewx[5408]: manager: Added record 2020-01-21 21:50:00 CST (1579665000) to database 'weewx.sdb'
Jan 21 21:50:04 cql-server weewx[5408]: manager: Added record 2020-01-21 21:50:00 CST (1579665000) to daily summary in 'weewx.sdb'
Jan 21 21:50:06 cql-server weewx[5408]: cheetahgenerator: Generated 8 files for report SeasonsReport in 1.26 seconds
Jan 21 21:50:07 cql-server weewx[5408]: imagegenerator: Generated 14 images for SeasonsReport in 0.98 seconds
Jan 21 21:50:07 cql-server weewx[5408]: copygenerator: copied 0 files to /var/www/html/weewx
Jan 21 21:50:21 cql-server weewx[5408]: restx: Wunderground-PWS: Failed to publish record 2020-01-21 21:50:00 CST (1579665000): Failed upload after 3 tries

at this point, not sure what else i should be doing - any help would be much appreciated.

[galfert]

You need to do one of two things...

- put the WS-2902A console behind the WeeWx system so that it can see the WU packets and actually intercept them. This is done by creating a separate network hotspot for the WS-2902A to connect to.

- or you need d to do DNS poisoning so that the WS-2902A sends the data to the WeeWx system instead of to WU servers. The WS-2902A will think it is sending data to WU but the DNS resolution will be changed so that it resolves to the WeeWx system's IP address instead.

Yet another alternative is to forget using the WS-2902A console and instead use the Ecowitt GW1000 to pick up your sensors and then the GW1000 can be configured to send the data to the WeeWx system using its "Customized" server settings. This is an option that the WS-2902A console device does not have hence is a reason adding the GW1000 is appealing. The other benefit to using the GW1000 is that you'll be able to add more sensors that the WS-2902A can't pick up (extra temp/hum, soil moisture, lightning, PM2.5, and more). You can then use the GW1000 with Ecowitt.net protocol instead of WU protocol to send these extra sensors to WeeWx.

[Christopher Lanus]

Thanks for the input! My impression was that I was attempting to perform the latter by updating my pihole's /etc/hosts file. I added records to resolve both `time.windows.com` and `rtupdate.wunderground.com` to the ip running weewx (those domains are the domains regularly being requested by the WS-2902A, according to pihole).

Is there a better way to to do this?

That being said, I like the idea of Ecowitt GW1000 - I'll have to explore that route, as well. Thanks again!

[galfert]

Yes, you are correct. I missed the line where you mentioned changing the host file on your DNS server. That should have worked. Test that DNS is really being served by using a different system on the network to ping the WU server name and see if it resolves to the IP address of the WeeWx system.

Still the GW1000 is a better way. No need for DNS trickery.

[Christopher Lanus]

I can confirm that the DNS is resolving to my weewx ip. This leads me to believe that the problem lies with either my nginx configuration or my weewx interceptor configuration.

I mentioned that the pi that's running weewx is also running owncloud, exposing ports 80/443 in an nginx webserver. I understand that the interceptor driver listens on port 80 by default, so, according to its directions, i set up a reverse proxy to port 9000 (where i configured interceptor to be listening). I think I failed to mention the reverse proxy.

In any case, I've run interceptor directly on port 9000, then hit the endpoint from a browser, resulting in the following terminal response

mapped packet: {'usUnits': 16, 'dateTime': 1579808958}

I've also updated my nginx config to stop listening on port 80, and tried running interceptor again on its default port, with the same respnose as above.

So...

1. I can resolve rtupdate.wunderground.com to correct weewx ip

2. I can run/listen witth interceptor driver, controlling for ports

3. I can confirm reverse proxying is working as expected

Still can't figure out where my data is! Might have to grab myself a GW1000, unless someone else has insights that I'm missing.

[G Hammer]

My experience with getting the data using Interceptor is that it's a pain. I tried the DNS spoofing, setting up an AP on the weewx machine and sniffing the data (worked best), iptables in my router. None were really good as there were too many moving parts. I then used the SDR method and it worked fine once the receiver was in just the right place. Then, as happens with radio, a sensor was lost and never regained.

I found the GW1000 method and have not looked back. Dead simple. MWall has written a driver for it.

If you really just want to use the Interceptor network method, I'd suggest simplifying your overall design then introduce additional items one at a time.

Me? I'm firmly in the simple camp with the GW1000.

[Christopher Lanus]

Good to know! I ordered the GW1000 yesterday - sounds like that's the best route.

And yes, using interceptor does clearly introduce too many moving parts, creating a fragile system. Guess I'll just have to be ok with not knowing what exactly was causing my issue. In any case, silver lining is that I learned a bunch in the process!

Thanks all for the help and advice

[KnC]

Christopher Lanus: This was a great thread to follow, because I sounds like I have a similar setup.  I am distracted at the moment with another project, but it would be very cool if you share your experience with setting up the GW1000, as I am considering going the same route.  Thanks all for the thoughtful feedback and willing to share an alternate solution in the process.

[Christopher Lanus]

I can report that the rumors are true - the GW1000 is dead simple! Nearly plug and play.

I followed the directions from here, and got my data displaying in weewx in less than a half hour.

Now, finally, I can get to work customizing the thing! Thanks all for the advice

[Tolip Wen]

I use interceptor and I have my router do the internet portion of the equation. The way I implimented it is with firewall rules on my router.

When my WS-1400-IP sends his data to wunderground my router makes a copy of the packets and sends them to my weewx computer.

If your WS2902 communicates with wunderground directly from your router over WiFi then the router is the logical first opportunity to duplicate and redirect the copy of the data. I have no idea if the Asus rt-a66u running stock software has provisions for 'ip tables' but you can load DD-WRT software on that model and it can do it. Switching the OS that is running on your router IS NOT TRIVIAL!

At any rate the firewall rules I use with DD-WRT are as follows,

iptables -t nat -A PREROUTING -s 10.10.100.72 -p tcp --dport 80 -j DNAT --to-destination 10.10.100.13:8099
iptables -t nat -A POSTROUTING -j MASQUERADE


My WS-1400-IP device has IP address 10.10.100.72
My weewx computer has IP address 10.10.100.13
My router changes the destination IP and port, my examples use 8099.
My weewx.conf file has the following interceptor settings,

[Interceptor]
    # This section is for the network traffic interceptor driver.
    
    # The driver to use:
    driver = user.interceptor
    
    # Specify the hardware device to capture.  Options include:
    #   acurite-bridge - acurite internet bridge
    #   observer - fine offset WH2600/HP1000/HP1003, aka 'observer'
    #   lw30x - oregon scientific LW301/LW302
    #   lacrosse-bridge - lacrosse GW1000U/C84612 internet bridge
    device_type = observer
    port = 8099

There is another way to use interceptor in sniff mode but I think it requires different firewall rules.
If your stock router uses 'iptables' you can try the lines I used just replace the IP addresses with whatever you are using.
Change the device_type in your weewx.conf to whatever is apropriate for your device.

Beware! the firewall rules use fixed IP addresses. 
If you use DHCP on your router it is necessary to make sure your router always gives the same IP to your 2902 and your PC running weewx.
The solution is to add Static Leases to your DHCP server on your router for the 2902 and weewx respectively.
If you don't do this and one of the devices gets a different address it can be "level 3 fun" figuring out why something stopped working when you did not change anything.

If you choose this method and it does not work right away you can use wireshark on the weewx PC and sniff packets to verify that the router is working as expected. Don't forget to verify that the data is still going to a remote destination if that is what you want.

It is also possible with firewall rules to NOT send the data directly to wunderground etc.

That way weewx can do whatever you choose with the data and send that out. :-)

Hope this helps! :-)