continuation of discussion on wxMesh MQTT

[Bill Morrow]

This conversation has been copied over from the weewx-user group (https://groups.google.com/d/msg/weewx-user/zhl4I7oRtt8/5pwhOAioBgAJ). I believe it is more appropriate to continue here.

Some imbedded comments below, as we discuss the design of receiving weather observations from an MQTT broker.

On Tuesday, 21 February 2017 06:06:40 UTC-4, wysiwyg wrote:

Hello !

Regarding websockets: 

Just found the howto I was talking about:

http://www.frugalprototype.com/installation-de-votre-propre-broker-mqtt/

it's in french (very convenient for me, sorrry guys ;-).

But as a summary, it says the websocket function of mosquitto can be enabled just with a small config file.

I will try this (quick & easy to test).

Also, just to summarise my setup:

I just received last week a brand new Orange Pi PC plus (kind of low cost raspberry which as a nice advantage imho: it has integrated emmc 8GB memory).

It is setup with Armbiam Ubuntu Xenial (server), v5.25.  (I am more familiar to ubuntu so it's easier for me too)

I just setup mosquitto from depot ( apt-get install mosquitto ).

I setup weewx using the quick start guide with depot ( http://www.weewx.com/docs/apt-get.htm )

So it's all fresh install.

I only modified mosquitto config to use "user/password" and play a bit with weewx and wxMesh as you know.

finally, I have a general question regarding how the driver works:

let me know if my understanding is correct or wrong

the "on_message" stuff is kind of asynchronous

then when a message is available on the broker, this function will be trigged  and will load the message content in payload.

Yes, when a message is received, the on_message callback is triggered.

 

then the genLoopPackets is synchronous (I mean executed every n seconds, n=poll interval defined in weewx.conf)

So every n seconds, it checks if payload as some content and if yes, it will process the data and deliver the "_packet" to next weewx process

Correct. I originally planned to build the loop packet in the on_message call back, but on_message actually runs in a separate thread I think, and I could not figure out how to easily get the data back to the weewx thread.  generating the loop packet only when a subscription arrives would be more efficient. I do not understand weewx or python quite well enough to design this yet.

If this is correct, does it mean that if several mqtt messages are delivered inside the n seconds interval, maybe only the last one will be processed ?

I believe any number of "loop" packets can arrive within n seconds, especially for n>1. Read the Data Architecture section here: http://www.weewx.com/docs/customizing.htm

The background of the question is that I'm thinking if it may be possible to use a sub topic structure for mqtt

The simplest idea I have in mind is to have:  weather/<weewx_schemas_label>:

weather/outTemp
weather/outHumidity
....



Then each of these topics may be sent at any time:  Temp, humidity, barometer may be send only every few minutes.  Wind may be sent  every 1 or few seconds,   rain may be send only when it's raining...

weewx driver may subscribe to /weather/# to get them all and process whenever they arrive ?

-Ok- I speak but don't have a clue on how to do that :-) not sure it is realistic.

That was exactly my original idea on how to build the driver. However, if a datum does not get set within the time interval of a RECORD (see the setting in the [StdArchive] section of weewx.conf), then it will be set to None, I think. I don't think this is what we want. Depending on the type of observation (wind, rain) we may want to keep the last value, or enter a zero.

wxMesh is definitely a work in progress. It may very well grow away from the original names

The advantage could that it stick better to what I have seen from mqtt usage, then same published topics could be used by other subscribers (home-assistant, automatic watering for the garden or whatever)

Best regards,

Le mardi 21 février 2017 00:05:11 UTC+1, mwall a écrit :

i found it helpful to configure the broker so that it publishes using websockets.  then you can use a web browser to see what is happening.  this is especially useful when you're trying to decode someone else's topic hierarchy, or to figure out why your own topics are not showing up where you think they should be.

the hivemq folks wrote one of the first ones, which many people have now embedded:

http://www.hivemq.com/blog/full-featured-mqtt-client-browser

http://www.espert.io/mqtt/index.html

but many other browser-based clients are out there:

http://mitsuruog.github.io/what-mqtt/

https://www.cloudmqtt.com/docs-websocket.html

if you're using the mosquitto broker, you'll have to compile it with websocket compatibility.  but once you do that, enabling websockets is just a matter of making the broker listen on the websocket port.

m

[mwall]

bill,

you might want a separate thread in the wxMesh driver - one thread that receives mqtt data and puts it on a queue, then the main thread that runs genLoopPackets and picks data off the queue as soon as they arrive.

see the ws3000 driver for a pretty simple example implementation.  it has one thread that writes/reads the usb, then the main thread that picks data off the queue.

https://github.com/matthewwall/weewx-ws3000

if you find that the broker sends data faster than weewx can process it, then you will need some queue management too.

this way you do not need a polling interval - the loop packet generator will emit data as fast as they come in from the mqtt broker.

m

[Wysiwyg]

Hi

Not sure if it's a good or helpful example but wmr100 driver, as far as I understood, receive data asynchronously from the Wmr100 station (one packet for each sensor with different occurrence depending on the sensor)

Code is here
https://github.com/weewx/weewx/blob/master/bin/weewx/drivers/wmr100.py

Also weewx customizing Guide said some stations works this way (so called partial package):

LOOP packets are the raw data generated by the device driver. They get their name from the Davis Instruments documentation. For some devices they are generated at rigid intervals, such as every 2 seconds for the Davis Vantage series, for others, irregularly, every 20 or 30 seconds or so. LOOP packets may or may not contain all the data types. For example, a packet may contain only temperature data, another only barometric data, etc.. This kind of packet is called a partial record packet. For other types of hardware (notably the Vantage series), every LOOP packet contains every data type.

[Bill Morrow]

I'll look at changing my software which publishes the MQTT message so that the messages are more MQTT-like: weather topic, and individual subtopics, which potentially will be published at different intervals. I like the idea, so that publications are only done when there is something worth publishing.

A couple of other tasks to get done first. If anyone is interested in working on this, I'm happy to accept your work.

[Wysiwyg]

Hi Bill,

I tried hard to understand how the drivers are working, but I lack to much python knowledge to help in coding. One of my collegue is good at this, I will try to ask for some idea.

but alone, I only roughly understand the principle of the driver (and see little things like this user/password).

But I'm ready to help with some tests !

It can be easy to manually simulate the sensors based on apps like mqtt dashboard on android (you can setup slide bars or button that send a value to a specified topic when you activate them).

Regarding the mqtt topic format. I would suggest  to have a "base" topic like "weather" or "weewx" or anything defined in weewx.conf ? and then sensor labels that could use by default weewx schema (outTemp, outHumidity, WindGust etc....) or alternate labels from weewx.conf (just like you already did (HUMT, RHUM...)).

But that's only a suggestion,open for discussion.

The only issue I see so far is that if we have:

weather/WindDir

weather/WindSpeed.

The 2 topics will be send more or less at same time from the sensor but it should be checked that they are matching same (or very close) reception time.

It may not be that critical for average wind, but if we want to do something not to inaccurate for gusts, WindGustDir and WindGust shall be considered at same moment.

It may mean it should be checked that both data are available before sending a valid packet to weewx.

Let me know if I'm not clear.

Best regards,

[Bill Morrow]

I understand your points about wind, Frederic. There's an interesting discussion underway in another thread in the user group: https://groups.google.com/d/msg/weewx-user/x2RYkjwyNjc/Ilcqr3-7BgAJ. "tempus" makes what I think is a good point: you may be able to sense wind direction, even if the wind speed is below the sensitivity of the wind speed sensor. So reporting wind direction and "None" for wind speed is a valid data scenario.

I like your idea of the sensor-specific subtopics. Let me think about implementation. 

[mwall]

On Wednesday, February 22, 2017 at 12:42:59 PM UTC-5, Bill Morrow wrote:

I understand your points about wind, Frederic. There's an interesting discussion underway in another thread in the user group: https://groups.google.com/d/msg/weewx-user/x2RYkjwyNjc/Ilcqr3-7BgAJ. "tempus" makes what I think is a good point: you may be able to sense wind direction, even if the wind speed is below the sensitivity of the wind speed sensor. So reporting wind direction and "None" for wind speed is a valid data scenario.

regarding wind speed and direction:

if at all possible, the driver should emit data as close as possible to what it receives from the sensors.  the driver should not cache.

policies such as 'wind direction is None when wind speed is zero' is applied at a higher level (in the StdWXCalculate service)

in the case of the 'wind direction is None' policy, a user can disable that policy with a boolean flag in the StdWXCalculate section of weewx.conf

regarding one-observation-per-message vs many-per-message:

sometimes you want data as fast and fine-grained as possible.  other times you want some aggregation.  one way to address this is to send both.  for example, do a 'loop' topic under which there is a message for each observation, as soon as the observation changes. each message is simply the value from the sensor (or possibly the value plus the units).

/telemetry/stationX/loop

with payloads like this:

37.5

or like this:

{'outTemp': '37.5', 'units': 'degree_C'}

have a separate 'archive' topic under which you send full records.  for example, each message would contain every element in an archive record, typically in json format.

/telemetry/stationX/archive

with payloads like this:

{'dateTime': 'xxxxxx', 'outTemp': 'xxxx', 'outHumidity': 'xxxx', ...}

then the subscriber can decide which kind of data to subscribe to.  for example, subscribe to all loop observations:

/telemetry/+/loop

m

[Wysiwyg]

Hello Matthew,

I'm not fully sure about this idea (if I undestand correctly your point), let me explain:

As far as I understood, whatever the quantity of loop packet, weewx will average or compute the data to build record packet (what is stored in database).

Then, if you need higher or lower rate of data, it could managed at emitter side by increasing or decreasing the rate of mqtt publisher(s)

Example: I see no use (for me) of humidity measurement every 2 seconds, but if someone need it, he can just make the mqtt value available every 2 seconds.

2nd and main issue I see:  as a main difference to standard weather station, the great interest (imho) of MQTT is that you can distribute your sensors as you wish without links between each other.

You main setup your windsensor 4m above your roof, while Temp/Humidity sensor will be correctly installed 1.5m over the grass at the end of the garden, while barometer will be inside house while whatever :-)

You can have (at extreme) as many publisher as sensors.

Then there is no way to build the archive mqtt message (unlesh you build a special mqtt client to build & publish it, but that sounds a bit strange).

I am just thinking that an interesting enhancement (for future) could be to set the sensors measurement rate in weewx.conf and the driver may publish it as mqtt message :-)

for example:  /telemetry/outTemperature/emit_interval  that would contain a number of seconds.  

Then sensors may get this value as mqtt subscriber and adjust their emissions accordingly

but forget it for the moment, just an open idea for future.. :-)

Best regards,

Frederic

[Bill Morrow]

I discovered a possible problem with this approach working well with other parts of weewx. Last night I was attempting to get the Realtime Gauge-Data (https://github.com/gjr80/weewx-realtime_gauge-data) extension going, and ran in to a problem because not each packet the my wxMesh driver subscribes to has the same data. Here, an indoor station packet arrives, is parsed and the temperature (INTE) and humidity (INHU) yielded to weewx, which causes rtgd to do some work. rtgdthread exits on a KeyError, because the packet does not contain outTemp. 

The opposite can also happen, if an outdoor packet arrives, rtgdthread will exit because if cannot find the inTemp keyword.

Feb 22 16:20:46 walrus weewx[29153]: wxMesh: Working on payload : TIME:0,INTE:22.60,INHU:32.59
Feb 22 16:20:46 walrus weewx[29153]: wxMesh: key: TIME value: 1487794846
Feb 22 16:20:46 walrus weewx[29153]: wxMesh: key: INTE value: 22.60
Feb 22 16:20:46 walrus weewx[29153]: wxMesh: key: INHU value: 32.59
Feb 22 16:20:46 walrus weewx[29153]: rtgd: queued loop packet: {'barometer': None, 'windchill': None, 'dewpoint': None, 'humidex': None, 'maxSolarRad': None, 'pressure': None, 'altimeter': None, 'usUnits': 17, 'appTemp': None, 'rainRate': 0.0, 'heatindex': None, 'dateTime': 1487794846.0, 'inHumidity': 32.59, 'inTemp': 22.600000000000005, 'cloudbase': None, 'inDewpoint': 41.58132605144329}
Feb 22 16:20:46 walrus weewx[29153]: rtgdthread: received packet: {'barometer': None, 'windchill': None, 'dewpoint': None, 'humidex': None, 'maxSolarRad': None, 'pressure': None, 'altimeter': None, 'usUnits': 17, 'appTemp': None, 'rainRate': 0.0, 'heatindex': None, 'dateTime': 1487794846.0, 'inHumidity': 32.59, 'inTemp': 22.600000000000005, 'cloudbase': None, 'inDewpoint': 41.58132605144329}
Feb 22 16:20:46 walrus weewx[29153]: rtgdthread: Unexpected exception of type <type 'exceptions.KeyError'>
Feb 22 16:20:46 walrus weewx[29153]: *** Traceback (most recent call last):
Feb 22 16:20:46 walrus weewx[29153]: ***   File "/usr/share/weewx/user/rtgd.py", line 773, in run
Feb 22 16:20:46 walrus weewx[29153]: ***     self.process_packet(_package['payload'])
Feb 22 16:20:46 walrus weewx[29153]: ***   File "/usr/share/weewx/user/rtgd.py", line 790, in process_packet
Feb 22 16:20:46 walrus weewx[29153]: ***     self.buffer.setLowsAndHighs(packet)
Feb 22 16:20:46 walrus weewx[29153]: ***   File "/usr/share/weewx/user/rtgd.py", line 1614, in setLowsAndHighs
Feb 22 16:20:46 walrus weewx[29153]: ***     outTemp = packet_d['outTemp']
Feb 22 16:20:46 walrus weewx[29153]: *** KeyError: 'outTemp'
Feb 22 16:20:46 walrus weewx[29153]: rtgdthread: Thread exiting. Reason: 'outTemp'

Should the wxMesh driver set all possible data to None before parsing the incoming MQTT message? Or is this a problem for rtgdthread to deal with? Or StdWXCalculate in weewx?

I'm asking not so much for these two specific modules, but as a general design guideline for my driver.

[Wysiwyg]

Hi Bill,

Not sure it helps, but I output some loop packet of my Oregon station (WMRS200 using WMR100 driver provided with weewx:

LOOP:   2017-02-22 12:33:40 CET (1487763220) altimeter: None, appTemp: None, barometer: None, cloudbase: None, dateTime:

1487763220, dewpoint: None, heatindex: None, humidex: None, inDewpoint: 41.5373706516, inHumidity: 35.0, inTemp: 70.52,

inTempBatteryStatus: 0, maxSolarRad: 559.235507813, pressure: None, rainRate: 0, usUnits: 1, windchill: None

LOOP:   2017-02-22 12:33:46 CET (1487763226) altimeter: None, appTemp: None, barometer: None, cloudbase: None, dateTime:

1487763226, dewpoint: None, heatindex: None, hourRain: 0.0, humidex: None, inDewpoint: None, maxSolarRad: 559.294551306

, pressure: None, rain: None, rain24: 0.0, rainBatteryStatus: 0, rainRate: 0.0, totalRain: 94.72, usUnits: 1, windchill:

None

LOOP:   2017-02-22 12:33:48 CET (1487763228) altimeter: None, appTemp: None, barometer: None, cloudbase: None, dateTime:

1487763228, dewpoint: None, heatindex: None, humidex: None, inDewpoint: None, maxSolarRad: 559.314190756, pressure: Non

e, rainRate: 0, usUnits: 1, windBatteryStatus: 0, windchill: None, windDir: None, windGust: 0.0, windGustDir: None, wind

Speed: 0.0

LOOP:   2017-02-22 12:33:51 CET (1487763231) altimeter: None, appTemp: None, barometer: None, cloudbase: None, dateTime:

1487763231, dewpoint: None, heatindex: None, humidex: None, inDewpoint: 41.5373706516, inHumidity: 35.0, inTemp: 70.52,

inTempBatteryStatus: 0, maxSolarRad: 559.343713308, pressure: None, rainRate: 0, usUnits: 1, windchill: None

LOOP:   2017-02-22 12:33:51 CET (1487763231) altimeter: None, appTemp: None, barometer: None, cloudbase: None, dateTime:

1487763231, dewpoint: None, heatindex: None, humidex: None, inDewpoint: None, maxSolarRad: 559.343713308, pressure: Non

e, rainRate: 0, usUnits: 1, windBatteryStatus: 0, windchill: None, windDir: None, windGust: 0.0, windGustDir: None, wind

Speed: 0.0

LOOP:   2017-02-22 12:33:56 CET (1487763236) altimeter: None, appTemp: None, barometer: None, cloudbase: 9154.78830959,

dateTime: 1487763236, dewpoint: 19.7668618938, heatindex: 58.46, humidex: 58.46, inDewpoint: None, maxSolarRad: 559.3926

82737, outHumidity: 22.0, outTemp: 58.46, outTempBatteryStatus: 1, pressure: None, rainRate: 0, usUnits: 1, windchill: N

one

I have not tested this realtime gauge, but for sure the loop packet does not include measurement: outTemp is only on 1 packet, inTEmp only on 2 (for 6 packets displayed here).

[mwall]

On Thursday, February 23, 2017 at 7:59:13 AM UTC-5, Bill Morrow wrote:

I discovered a possible problem with this approach working well with other parts of weewx. Last night I was attempting to get the Realtime Gauge-Data (https://github.com/gjr80/weewx-realtime_gauge-data) extension going, and ran in to a problem because not each packet the my wxMesh driver subscribes to has the same data. Here, an indoor station packet arrives, is parsed and the temperature (INTE) and humidity (INHU) yielded to weewx, which causes rtgd to do some work. rtgdthread exits on a KeyError, because the packet does not contain outTemp. 

The opposite can also happen, if an outdoor packet arrives, rtgdthread will exit because if cannot find the inTemp keyword.

...

Should the wxMesh driver set all possible data to None before parsing the incoming MQTT message? Or is this a problem for rtgdthread to deal with? Or StdWXCalculate in weewx?

I'm asking not so much for these two specific modules, but as a general design guideline for my driver.

partial packets are ok - the driver should emit data as soon as it receives data, and only for observations that it actually knows about.

so do not fill a loop packet with a bunch of None values - use None only to indicate that the observation returned a bogus value, not to indicate that the driver is capable of returning that value but just happened to not return a value for that sensor this time around.

you will find many downstream consumers that do not handle partial packets properly.  that is a bug in the downstream consumer, and the downstream consumer needs to be fixed.

m

[gjr80]

Yes, this 'downstream consumer' does not handle partial packets properly. It's on the to-do list.

Gary

[Robert Mantel]

When you guys said "downstream consumer" I thought you were referring to us....lol.

[Wysiwyg]

Hi !

I just set a very very very simple bash script to generate some random mqtt publisher.

here it is, just in case it helps for testing

#!/bin/sh
while [ true ]
do
        outTemp=$((20+ $(od -An -N1 -i /dev/random) % 5))
        mosquitto_pub -t weewx/outTemp -m $outTemp -u <broker_username> -P <broker_password> -h <broker_ip>
        barometer=$((1010+ $(od -An -N1 -i /dev/random) % 10))
        mosquitto_pub -t weewx/barometer -m $barometer -u <broker_username> -P <broker_password> -h <broker_ip>
        sleep 2
done

[Wysiwyg]

Hi Bill, 

I just wanted to share I received my ESP8266... 30min later it send some mqtt message every minute for outTemp & Humidity... Well it is still dummy values because the sensor is still on the way. But it's very simple to connect to WiFi then mosquitto. 

I think I will open a github.com page to share the code for sensor side. 

Best regards, 

[Bill Morrow]

On Tuesday, 28 February 2017 17:18:44 UTC-4, Wysiwyg wrote:

Hi Bill, 

I just wanted to share I received my ESP8266... 30min later it send some mqtt message every minute for outTemp & Humidity... Well it is still dummy values because the sensor is still on the way. But it's very simple to connect to WiFi then mosquitto. 

I think I will open a github.com page to share the code for sensor side. 

I haven't got mine to the point where I wanted to share it that formally, but here it is for inspiration. It is mostly cut-paste from examples:

/***************************************************
  Adafruit MQTT Library ESP8266 Example


  Must use ESP8266 Arduino from:
    https://github.com/esp8266/Arduino


  Works great with Adafruit's Huzzah ESP board & Feather
  ----> https://www.adafruit.com/product/2471
  ----> https://www.adafruit.com/products/2821


  Adafruit invests time and resources providing this open source code,
  please support Adafruit and open-source hardware by purchasing
  products from Adafruit!


  Written by Tony DiCola for Adafruit Industries.
  MIT license, all text above must be included in any redistribution
 ****************************************************/
#include <Wire.h>
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include "Adafruit_HTU21DF.h"


const char* ssid = "XXX";
const char* password = "XXX";
const char* mqtt_server = "XXX";


/************************* MQTT Setup *********************************/


#define MQTT_USERNAME    "YYY"
#define MQTT_PASSWD      "YYY"


/************ Global State (you don't need to change this!) ******************/


// Create an ESP8266 WiFiClient class to connect to the MQTT server.
WiFiClient wifiClient;
PubSubClient mqttClient(wifiClient);


Adafruit_HTU21DF htu = Adafruit_HTU21DF();


void setup() {
  Serial.begin(9600);
  delay(10);


  Serial.println(F("Adafruit MQTT publish of HTU21D"));


  if (!htu.begin()) {
    Serial.println("Couldn't find sensor!");
    while (1);
  }


  setup_wifi();
  mqttClient.setServer(mqtt_server, 1883);
  mqttClient.setCallback(callback);
}


long lastMsg = 0;


void setup_wifi() {


  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);


  WiFi.begin(ssid, password);


  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}


void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (unsigned int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();


  // Switch on the LED if an 1 was received as first character
  if ((char)payload[0] == '1') {
    digitalWrite(BUILTIN_LED, LOW);   // it is active low on the ESP-01)
  } else {
    digitalWrite(BUILTIN_LED, HIGH);  // Turn the LED off by making the voltage HIGH
  }


}


void reconnect() {
  // Loop until we're reconnected
  while (!mqttClient.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (mqttClient.connect("ESP8266Client", "YYY", "YYY")) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      mqttClient.publish("outTopic", "hello world");
      // ... and resubscribe
      mqttClient.subscribe("weather");
    } else {
      Serial.print("failed, rc=");
      Serial.print(mqttClient.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}


void loop() {
  char message[64];
  char temperatureStr[6], humidityStr[6];


  if (!mqttClient.connected()) {
    reconnect();
  }
  mqttClient.loop();


  long now = millis();
  if (now - lastMsg > 10000) {
    lastMsg = now;
   
    dtostrf(htu.readTemperature(), 4, 2, temperatureStr);
    dtostrf(htu.readHumidity(), 4, 2, humidityStr);
 
    sprintf( message, "TIME:0,INTE:%s,INHU:%s", temperatureStr, humidityStr);
    Serial.print("Publish message: ");
    Serial.println(message);
    mqttClient.publish("weather", message);
  }


}

 

[Wysiwyg]

Hi Bill,

I think we started from same source, but I've done a bit more clean up (removing the subscribe part) but also I modified it to use deep sleep mode to reduce power consumption (target: use it with battery+ solar panel).

But I still have some clean up to do before publishing.

What are the news regarding the weewx driver ?

[Bill Morrow]

No progress, too many intrusions from real life!

I have been thinking about the subtopic change.

[Wysiwyg]

I can understand :-) free time is rare !

I tried a bit to understand the python code, may I have a suggestion...but it may be completely stupid :)

     # The callback for when a PUBLISH message is received from the server.

    def on_message(self, client, userdata, msg):

self.payload = str(msg.payload)

self.topic = str(msg.topic)                                       # Modification to record also the topic of the message .

logdbg("Got message %s" % str(msg.payload))

    def genLoopPackets(self):

        while True:

   self.client.loop()

            # read whatever values we can get from the MQTT broker

            logdbg("Working on payload : %s" % self.payload)

   if self.payload != "Empty" :

data = {}       ## start of modified part

received_topic = self.topic

                key =  mystring.split('/')[-1]   # found on a forum, it is supposed to output last string after separator, ex: if topic is 'weewx/outHumidity', it shall return outHumidity  

value = self.payload

data[key] = value

# end of modified part 

# map the data into a weewx loop packet

_packet = {'usUnits': weewx.METRIC}

for vname in data:

   _packet[self.label_map.get(vname, vname)] = _get_as_float(data, vname)

yield _packet

       self.payload = "Empty"

    

I try to made it simple:

let's assume that last part of topic match weewx dictionnary for measurements (outTemp, outHumidity etc...)

also, it is supposed to provide 1 value per LOOP packet.

and I have no idea what happen when more than 1 mqtt packet arrive at same time...

it is missing the correct subscription to get for example all message with topics starting by weewx/xxxx 

[Wysiwyg]

Hi all !

I worked a bit on this python, not easy but maybe I have something not so bad...

here it is:

    def on_message(self, client, userdata, msg):

        self.payload = str(msg.payload)

        string_topic = str(msg.topic)

        key =  string_topic.split('/')[-1] 

        self.receive_buffer[key] = str(msg.payload)

    def closePort(self):

        self.client.disconnect()

    def genLoopPackets(self):

        self.client.loop_start()

        while True:

            time.sleep(self.poll_interval)

            data = self.receive_buffer

            _packet = {'dateTime': int(time.time() + 0.5),'usUnits': weewx.METRIC}

            logdbg("dateTime %s" % _packet["dateTime"])

            for vname in data:

                _packet[self.label_map.get(vname, vname)] = _get_as_float(data, vname)

                logdbg("buffer content for key: %s =  %s" %(vname, data[vname]))

            yield _packet

            self.receive_buffer.clear()

            data.clear()

I'm really not sure of what I am doing with those self.xxx (I talk about my receive buffer), but it seems to works not so bad.

here is the log I get:

Mar 11 21:46:51 localhost weewx[11116]: engine: Starting up weewx version 3.6.2

Mar 11 21:46:51 localhost weewx[11116]: engine: Station does not support reading the time

Mar 11 21:46:51 localhost weewx[11116]: engine: Starting main packet loop.

Mar 11 21:47:01 localhost weewx[11116]: wxMesh: dateTime 1489265221

Mar 11 21:47:01 localhost weewx[11116]: wxMesh: buffer content for key: outHumidity =  73

Mar 11 21:47:01 localhost weewx[11116]: wxMesh: buffer content for key: pressure =  1005.31

Mar 11 21:47:01 localhost weewx[11116]: wxMesh: buffer content for key: barometer =  1018.59

Mar 11 21:47:01 localhost weewx[11116]: wxMesh: buffer content for key: inTemp =  24.98

Mar 11 21:47:01 localhost weewx[11116]: wxMesh: buffer content for key: outTemp =  22

Mar 11 21:47:11 localhost weewx[11116]: wxMesh: dateTime 1489265232

Mar 11 21:47:21 localhost weewx[11116]: wxMesh: dateTime 1489265242

Mar 11 21:47:21 localhost weewx[11116]: wxMesh: buffer content for key: outHumidity =  70

Mar 11 21:47:21 localhost weewx[11116]: wxMesh: buffer content for key: outTemp =  23

Mar 11 21:47:31 localhost weewx[11116]: wxMesh: dateTime 1489265252

Mar 11 21:47:41 localhost weewx[11116]: wxMesh: dateTime 1489265262

the most difficult thing, (imho) what the "loop_start()" trick.

As far as I understood, it allows to run the "on_message" in background, even during the time.sleep.

then when time.sleep is ended, we just have to get the buffer and empty data if there are some, to build the loop packet.

what do you think ?

by the way, I also added a in init:

   self.topic = ('weewx/#')

   self.receive_buffer = {}

my mqtt topics are :

weewx/inTemp

weewx/pressure

....

[Wysiwyg]

Hello!

The code has worked all night pretty well (I have nice barometer curve :-) ).

I added a "if" to output packet only when data are available, otherwise, there was some empty packet with only dateTime inside.

and I also clean up a few things that was hardcoded (topic, username and password), now it should take values from weewx.conf.

Also FYI, currently my label are same as in weewx.

mean in weewx.conf in label map, I have "inTemp = inTemp,  barometer = barometer " etc...

here is the full driver:

#!/usr/bin/python

#

# weewx driver that reads data from MQTT subscription

#

# This program is free software: you can redistribute it and/or modify it under

# the terms of the GNU General Public License as published by the Free Software

# Foundation, either version 3 of the License, or any later version.

#

# This program is distributed in the hope that it will be useful, but WITHOUT

# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS

# FOR A PARTICULAR PURPOSE.

#

# See http://www.gnu.org/licenses/

#

# The units must be weewx.US:   (this comment need to be updated)

#   degree_F, inHg, inch, inch_per_hour, mile_per_hour

#

# To use this driver, put this file in the weewx user directory, then make

# the following changes to weewx.conf:

#

# [Station]

#     station_type = wxMesh

# [wxMesh]

#     host = localhost           # MQTT broker hostname

#     topic = weather            # topic, mqtt topics should have format "topic/label_map", ex: weather/humi

#     driver = user.wxMesh

#

# If the variables in the file have names different from those in weewx, then

# create a mapping such as this:

#

# [wxMesh]

#     ...

#     [[label_map]]

#         temp = outTemp

#         humi = outHumidity

#         in_temp = inTemp

#         in_humid = inHumidity

from __future__ import with_statement

import syslog

import time

import paho.mqtt.client as mqtt

import weewx.drivers

DRIVER_VERSION = "0.1"

def logmsg(dst, msg):

    syslog.syslog(dst, 'wxMesh: %s' % msg)

def logdbg(msg):

    logmsg(syslog.LOG_DEBUG, msg)

def loginf(msg):

    logmsg(syslog.LOG_INFO, msg)

def logerr(msg):

    logmsg(syslog.LOG_ERR, msg)

def _get_as_float(d, s):

    v = None

    if s in d:

        try:

            v = float(d[s])

        except ValueError, e:

            logerr("cannot read value for '%s': %s" % (s, e))

    return v

def loader(config_dict, engine):

    return wxMesh(**config_dict['wxMesh'])

class wxMesh(weewx.drivers.AbstractDevice):

    """weewx driver that reads data from a file"""

    def __init__(self, **stn_dict):

        # where to find the data file

        self.host = stn_dict.get('host', 'localhost')

        # subscribe to all sub-topic of the topic define in weewx.conf 

        self.topic = stn_dict.get('topic', 'weather') + "/#"

self.username = stn_dict.get('username', 'XXX')

        self.password = stn_dict.get('password', 'password')

        # how often to poll the weather data file, seconds

        self.poll_interval = float(stn_dict.get('poll_interval', 5.0))

        # mapping from variable names to weewx names

        self.label_map = stn_dict.get('label_map', {})

        loginf("host is %s" % self.host)

        loginf("topic is %s" % self.topic)

        loginf("polling interval is %s" % self.poll_interval)

        loginf('label map is %s' % self.label_map)

        

        self.payload = "Empty"

self.receive_buffer = {}

        #self.payloadList = [payload]

        self.client = mqtt.Client(client_id="XXX", protocol=mqtt.MQTTv31)

#self.client.on_connect = self.on_connect

self.client.on_message = self.on_message

        self.client.username_pw_set(self.username, self.password)

self.client.connect(self.host, 1883, 60)

self.client.subscribe(self.topic, qos=0)

    # The callback for when a PUBLISH message is received from the server.

    def on_message(self, client, userdata, msg):

self.payload = str(msg.payload)

string_topic = str(msg.topic)

key =  string_topic.split('/')[-1] 

        self.receive_buffer[key] = str(msg.payload)

    def closePort(self):

        self.client.disconnect()

    def genLoopPackets(self):

        self.client.loop_start()   # enable to receive ('on_message') in background

        while True:

            time.sleep(self.poll_interval) # wait for some MQTT data to be published

            data = self.receive_buffer

            if data:       # if data is not empty then prepare loop packet

                _packet = {'dateTime': int(time.time() + 0.5),'usUnits': weewx.METRIC}

                logdbg("dateTime %s" % _packet["dateTime"])

                for vname in data:

                    _packet[self.label_map.get(vname, vname)] = _get_as_float(data, vname)

                    logdbg("buffer content for key: %s =  %s" %(vname, data[vname]))

                yield _packet

                self.receive_buffer.clear()

                data.clear()

    @property

    def hardware_name(self):

        return "wxMesh"

 

[Wysiwyg]

Hi,

I have a question:

If I use a mqtt topic that does not exist in label_map, it will anyway put it in the LOOP packet.

For example, I tried to publish a mqtt "weewx/baro" with 1015 inside

I get it in the LOOP like this:

LOOP:   2017-03-12 11:41:15 CET (1489315275) altimeter: None, appTemp: None, baro: 1015.0, barometer: None, cloudbase: None, dateTime: 1489315275, dewpoint: None, heatindex: None, humidex: None, inDewpoint: None, maxSolarRad: None, pressure: None, rainRate: 0, usUnits: 1, windchill: None

This has no meaning and I guess it would be better to filter.

also it may happens that some mqtt topics may be not for weewx but for other personnal use.

I would like to check if a key is existing in weewx schema before put it in the packet, but no clue how to do that.

I mean, as far as I understood, it's easy to check if a key exist in dictionnary :

if 'key1' in dict.keys():

But what's the name of the weewx expected dictionnary? (I assume it may exist a dictionnary matching the database schema?

Best regards,

[Wysiwyg]

No help for the dictionary question?

[mwall]

On Sunday, March 12, 2017 at 6:49:17 AM UTC-4, Wysiwyg wrote:

This has no meaning and I guess it would be better to filter.

also it may happens that some mqtt topics may be not for weewx but for other personnal use.

I would like to check if a key is existing in weewx schema before put it in the packet, but no clue how to do that.

you do not have to check the schema; any fields in the loop packet that are not in the schema will be ignored.

m

[Frederic Stuyk]

That's great! Thanks for the information! 

So I don't see much more to say, any opinions about the code? I'm a noob in python and may have missed some important stuff. 

So far I was considering to clear the "receive buffer" right after data = self.receive_buffer

Then if a mqtt message arrives during the processing of loop packet, it will be processed on next loop.

Also I think poll interval does not mean so much now, it should be smaller than the quicker sensor to avoid missing some data. And at the end, loop packet rate will more or less follows the quickest sensor. 

Best regards, 

[Dan Verbus]

Would you mind if I used your code as a basis for a weewx service that will augment loop packets from an existing weather station with information from an MQTT queue? I'm working on that now and would be happy to share when I'm done.

Thanks,

Dan

[Wysiwyg]

Hello Dan,

If you talk about my code, answer is NO ! 

:-)

just kidding !

my code is just a modificaton of Bill code which is a modifcation of the fileparser which is free sofware

from my side, feel free to use !

if you see some improvements that can be done, let me know !

Best regards,

[Wysiwyg]

Hi All,

I modified slightly the code and uploaded it on my github here (as no recent news from Bill)

https://github.com/bonjour81/ESP8266-MQTT-WEEWX/blob/master/wxMesh.py

Bill, I hope it's no offence, I just wanted to put my modifications somewhere. 

Modifications are the following:

add loop_stop() for mqtt, as we launch a background thread for mqtt reception with loop_start(), it must be ended somewhere.

without it, I noticed some activity in mosquitto logs even when weewx was stopped.

I also move a bit the receive_buffer clear earlier, to make sure we dont loose any mqtt data during packet processing.  it's not clear for me if could happen before, but I like to have my "data" dict "frozen" during the packet construction.

Matthew, I have a big question for better understanding of how weewx deals with drivers !

There is an infinite loop in the driver (while true:).

So my understanding is that weewx launch the driver as a kind of background thread that output packet with that yield command ?

So it's just never go out of the while true loop as long as weewx run.

Then in case of weewx stop, weewx will force exit this infinite loop and launch the "close_port()" procedure ?

I may use wrong wording, but I hope my idea is clear, let me know if not :-)

by the way, if my understanding is correct, does it mean I shall check regularly if mqtt connexion is still valid and reconnect if needed ?  

The connexion is in the __init__ then, if disconnected for any reason, we cannot recover, am I right ?

[Thomas Keffer]

Matthew, I have a big question for better understanding of how weewx deals with drivers !

There is an infinite loop in the driver (while true:).

So my understanding is that weewx launch the driver as a kind of background thread that output packet with that yield command ?

So it's just never go out of the while true loop as long as weewx run.

Then in case of weewx stop, weewx will force exit this infinite loop and launch the "close_port()" procedure ?

 

​Yes, the drivers should emit LOOP packets indefinitely, but the engine ​calling them does not. At the end of an archive period, it raises exception

​

​

engine.BreakLoop, which breaks the loop. After processing the archive record, it starts the loop again with a call to

​

​

genLoopPackets

​()

​

.

As you say, it's also true that at program termination it also breaks the loop.

​by the way, if my understanding is correct, does it mean I shall check regularly if mqtt connexion is still valid and reconnect if needed ?  

The connexion is in the __init__ then, if disconnected for any reason, we cannot recover, am I right ?​

If your lose the mqtt connection, you should raise an exception of type weewx.WeeWxIOError, or a subclass of weewx.WeeWxIOError. The engine will catch this exception and, by default, after 60 seconds do a restart. This will cause your driver to reload, giving it an opportunity to reconnect with the broker.

-tk

[Bill Morrow]

Hi all,

Back from a business trip where I was ignoring anything here.

Feel free to use and improve my code in any form you want. We should standardize at one github location, any suggestions on how to do that? Tom, how do we go about lobbying for inclusion of a new module in the weewx baseline?

[Thomas Keffer]

Hi, Bill,

Good work.

Using an mqtt broker is a specialized business. It would be difficult to test as part of the regular release process.

It's a perfect candidate for a weewx extension. Package it up, then post a link in the Wiki. Don't worry: it will get used!

-tk

[Wysiwyg]

Hi Tom,

Thanks for advices !

After a deeper look in python mqtt documentation, it seems that "loop_start()" thing take care to make sure we are still connected to broker.

if not, it will reconnect.

but it may make sense to check before sending anyway.

also during the weekend I noticed that at reboot, as weewx start before my ethernet connexion is fully setup, it seems I get a crash (the driver is not able to connect).

Is it possible to use this weewx.WeeWxIOError during the __init__ of the driver ?

Best regards,

[mwall]

On Monday, March 20, 2017 at 11:12:55 AM UTC-4, Wysiwyg wrote:

also during the weekend I noticed that at reboot, as weewx start before my ethernet connexion is fully setup, it seems I get a crash (the driver is not able to connect).

Is it possible to use this weewx.WeeWxIOError during the __init__ of the driver ?

this situation is handled by the loop_on_init=True option in the weewx configuration file.

weewx is designed to fail hard on startup when system resources are not available.  for example, if the driver expects a usb device, but that usb device is not plugged in, then the driver fails and weewx exits.  similarly, if a driver needs the network, but there is no nic or the network subsystem is not up and running, then the driver fails and weewx exits.

once weewx is running, these kinds of failures are handled by the weewx engine - the engine detects the driver failure, waits 60 seconds, then starts over.

the intent is that the user should be made aware of the problem during startup, but once weewx is running, the engine should keep things running.

lets call this the 'interactive' use case - a person is interactively getting weewx to start up.

however, there is another use case that has opposite goal from this - it wants weewx to start running whether or not the things it needs are attached/available.  lets call this the 'set and forget' use case - you configure a system that may or may not have all of the resources available for weewx, and you want weewx to start working as soon as those resources are available.  for example, you might start up weewx when no usb devices are plugged in, but you want weewx to automatically detect the devices and start working as soon as they are plugged in.  or you might have a raspberry pi that takes a long time for networking to come up, but you want weewx to start up anyway, then start working as soon as the network is available.

one way to do this would be to use udev rules/scripts to start/stop weewx.  another way is to use systemd (or init.d) dependencies.

but the loop_on_init=True approach is much easier, and works for many use cases.

m

[Wysiwyg]

Hi Matthew and thanks for details informations !

I tried the loop_on_init, works perfectly !

Mar 23 10:02:46 localhost weewx[786]: import of driver failed: [Errno 101] Le réseau n'est pas accessible (<class 'socket.error'>)

Mar 23 10:02:46 localhost weewx[786]: engine: Unable to load driver: [Errno 101] Le réseau n'est pas accessible

Mar 23 10:02:46 localhost weewx[786]:     ****  Waiting 60 seconds then retrying...

Mar 23 10:04:04 localhost weewx[786]: engine: retrying...

Mar 23 10:04:04 localhost weewx[786]: engine: Using configuration file /etc/weewx/weewx.conf

after 1 minute, weewx started correctly.

[Wysiwyg]

Hello Tom, Matthew (or anybody who knows) :-)

How can I get the default measurement dictionnary in the driver ?

Currently, I have define in weewx.conf a stupid dictionnary:

    [[label_map]]

        barometer   = barometer  

        pressure    = pressure   

        altimeter   = altimeter  

        inTemp      = inTemp     

        outTemp     = outTemp    

        inHumidity  = inHumidity 

        ....

With all weewx expected values (db schema)

left side should be the mqtt topic  (ex :  weewx/outTemp  )

this label map allow custom topics  ( ex weewx/OT ) is ok with:

    [[label_map]]

                     OT    = outTemp    

But I think it could be also easy just to have no label map in driver config in weewx.conf and use default dict matching the database schema.

I dont know how I can get this dict from driver. any suggestions ?

[Robert Mantel]

Bill, I have a question about the format of the MQTT message.  I have my raspberry pi running weewx, I also have that unit using mathew's mqtt publisher to push data to my mosquitto server.  I set up another virtual machine running weewx with your driver, but for the life of me cannot get your driver to pick up the info from it.  It keeps saying the payload is empty, but I have mqtt dashboard running on my android phone that I tested with and get updates just fine.  My goal is to actually migrate from my fileparse setup on my PI to an mqtt based server on the freenas box.  Presumably Mathiew's mqtt implimentation will use the standard weewx labels so I would assume I wouldn't even need a label map for the wxMesh driver.  Unless I'm way off on that.

[mwall]

On Friday, March 24, 2017 at 8:13:57 PM UTC-4, Robert Mantel wrote:

Bill, I have a question about the format of the MQTT message.  I have my raspberry pi running weewx, I also have that unit using mathew's mqtt publisher to push data to my mosquitto server.  I set up another virtual machine running weewx with your driver, but for the life of me cannot get your driver to pick up the info from it.  It keeps saying the payload is empty, but I have mqtt dashboard running on my android phone that I tested with and get updates just fine.  My goal is to actually migrate from my fileparse setup on my PI to an mqtt based server on the freenas box.  Presumably Mathiew's mqtt implimentation will use the standard weewx labels so I would assume I wouldn't even need a label map for the wxMesh driver.  Unless I'm way off on that.

the weewx-mqtt uploader publishes to the topic specified in [[MQTT]], which is 'weather' by default.

so if you subscribe to /weather/+ you should see everything

other mqtt uploader options:

[StdRESTful]
    [[MQTT]]
        topic = weather
        append_units_label = True # should the units name be appended to the observation name?
        retain = False
        aggregation = individual, aggregate # upload each observation individually, and/or all in a single message
        unit_system = None # convert to the indicated unit system 'US', 'METRIC', or 'METRICWX'
        binding = archive # how often to emit data, either 'loop' or 'archive'

you can also specify units and formatting for each observation, if necessary.

m

[Robert Mantel]

Ok, so using Bill's driver for the topic I am using "weather" and it's not finding anything in the payload then it must be something I have configured wrong in his driver?  Bill, any comments?  Like I said I have an mqtt dashboard app on my phone that is having no problem grabbing the individual readings.  I guess Bill's driver needs a fully healed tuple to parse?

[Wysiwyg]

Hello Robert,

Bill driver need a mqtt topic (like "weather").

Payload should be a string with all measurement.

Ex:

AMBT: 7.23,BARP:1001.97,RHUM:70.92,HUMT: 4.44,IRRA:12,BATV:974,PHOV:673,SYST:33.20,WIND:  0.0,WDIR:  0.0

Labels like AMBT, BARP shall be defined in weewx.conf to match weewx expected labels.

My modification of the driver works with single topic per measurement like:

weather/barometer   (ex payload = 1013.25 )

weather/outTemp (ex payload = 23) 

....

You may set debug=1 in weewx.conf and monitor weewx log file.

(ex tail -100f /var/log/weewx/weewx.log)

And you can simulate sensors by publishing topics with your phone  (I use MQTT dash and MQTT dashboard on android)

This is what I have done, it works well.

You should see something in the log when you publish the values

[Robert Mantel]

So I tried changing the topic to weather/loop because that appears to have the full json tuple format with the paired label:values that the wxMesh driver needs but I get this error:

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: engine: RealTimeGaugeData will generate gauge-data.txt. min_interval is 120 seconds

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: engine: Finished loading service user.rtgd.RealtimeGaugeData

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: engine: Starting up weewx version 3.7.1

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: engine: Station does not support reading the time

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: engine: Starting main packet loop.

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: wxMesh: Working on payload : Empty

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: wxMesh: Sleeping for 5

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: manager: Daily summary version is 2.0

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: manager: Daily summary version is 2.0

Mar 25 15:35:47 ubuntu-freenas weewx[5297]: rtgdthread: Zambretti is installed: False

Mar 25 15:35:52 ubuntu-freenas weewx[5297]: wxMesh: Working on payload : Empty

Mar 25 15:35:52 ubuntu-freenas weewx[5297]: wxMesh: Sleeping for 5

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: Got message {"soc": "68.8047", "altimeter": "30.4916869968", "outHumidity": "63.0", "dayRain": "0.0", "rainin": "0.0", "voltage": "3.9688", "dailyrainin": "0.132", "windspeedmph": "4.4383", "kilopascals": "100.941", "heatindex": "29.534", "rainmm": "0.0", "windspeedmps": "1.9841", "hpa": "1009.4", "hourRain": "0.0", "windGustDir": "90.0", "tempc": "-1.37", "barometer": "30.5439450745", "windchill": "29.534", "dewpoint": "18.4883", "windrun": "32.3367", "windspdmps_avg2m": "1.3076", "windspeedkmph": "7.1427", "humidex": "29.534", "winddir": "90.0", "pressure": "29.8079", "rain": "0.0", "rainRate": "0.0", "dewptc": "-7.5065", "usUnits": "1.0", "altm": "33.1085", "appTemp": "22.7487311772", "altf": "108.6236", "interval": "2.0", "dateTime": "1490455920.0", "windDir": "53.0", "outTemp": "29.534", "windSpeed": "2.9333", "rain24": "0.363", "windGust": "11.8354", "rssi": "-71.0", "cloudbase": "3141.49934286"}

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: Working on payload : {"soc": "68.8047", "altimeter": "30.4916869968", "outHumidity": "63.0", "dayRain": "0.0", "rainin": "0.0", "voltage": "3.9688", "dailyrainin": "0.132", "windspeedmph": "4.4383", "kilopascals": "100.941", "heatindex": "29.534", "rainmm": "0.0", "windspeedmps": "1.9841", "hpa": "1009.4", "hourRain": "0.0", "windGustDir": "90.0", "tempc": "-1.37", "barometer": "30.5439450745", "windchill": "29.534", "dewpoint": "18.4883", "windrun": "32.3367", "windspdmps_avg2m": "1.3076", "windspeedkmph": "7.1427", "humidex": "29.534", "winddir": "90.0", "pressure": "29.8079", "rain": "0.0", "rainRate": "0.0", "dewptc": "-7.5065", "usUnits": "1.0", "altm": "33.1085", "appTemp": "22.7487311772", "altf": "108.6236", "interval": "2.0", "dateTime": "1490455920.0", "windDir": "53.0", "outTemp": "29.534", "windSpeed": "2.9333", "rain24": "0.363", "windGust": "11.8354", "rssi": "-71.0", "cloudbase": "3141.49934286"}

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key: {"soc" value:  "68.8047"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "altimeter" value:  "30.4916869968"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "outHumidity" value:  "63.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "dayRain" value:  "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "rainin" value:  "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "voltage" value:  "3.9688"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "dailyrainin" value:  "0.132"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windspeedmph" value:  "4.4383"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "kilopascals" value:  "100.941"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "heatindex" value:  "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "rainmm" value:  "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windspeedmps" value:  "1.9841"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "hpa" value:  "1009.4"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "hourRain" value:  "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windGustDir" value:  "90.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "tempc" value:  "-1.37"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "barometer" value:  "30.5439450745"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windchill" value:  "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "dewpoint" value:  "18.4883"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windrun" value:  "32.3367"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windspdmps_avg2m" value:  "1.3076"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windspeedkmph" value:  "7.1427"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "humidex" value:  "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "winddir" value:  "90.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "pressure" value:  "29.8079"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "rain" value:  "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "rainRate" value:  "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "dewptc" value:  "-7.5065"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "usUnits" value:  "1.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "altm" value:  "33.1085"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "appTemp" value:  "22.7487311772"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "altf" value:  "108.6236"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "interval" value:  "2.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "dateTime" value:  "1490455920.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windDir" value:  "53.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "outTemp" value:  "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windSpeed" value:  "2.9333"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "rain24" value:  "0.363"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "windGust" value:  "11.8354"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "rssi" value:  "-71.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: key:  "cloudbase" value:  "3141.49934286"}

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windrun"': could not convert string to float: "32.3367"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windGust"': could not convert string to float: "11.8354"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windchill"': could not convert string to float: "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "rainmm"': could not convert string to float: "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "voltage"': could not convert string to float: "3.9688"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "altm"': could not convert string to float: "33.1085"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windspdmps_avg2m"': could not convert string to float: "1.3076"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "altimeter"': could not convert string to float: "30.4916869968"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "barometer"': could not convert string to float: "30.5439450745"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "pressure"': could not convert string to float: "29.8079"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windspeedmps"': could not convert string to float: "1.9841"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "rainRate"': could not convert string to float: "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windspeedkmph"': could not convert string to float: "7.1427"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windspeedmph"': could not convert string to float: "4.4383"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "altf"': could not convert string to float: "108.6236"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "dewpoint"': could not convert string to float: "18.4883"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windGustDir"': could not convert string to float: "90.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windSpeed"': could not convert string to float: "2.9333"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "outTemp"': could not convert string to float: "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "dateTime"': could not convert string to float: "1490455920.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "outHumidity"': could not convert string to float: "63.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "dewptc"': could not convert string to float: "-7.5065"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "hpa"': could not convert string to float: "1009.4"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "interval"': could not convert string to float: "2.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "winddir"': could not convert string to float: "90.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "kilopascals"': could not convert string to float: "100.941"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "rain24"': could not convert string to float: "0.363"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "heatindex"': could not convert string to float: "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "appTemp"': could not convert string to float: "22.7487311772"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "usUnits"': could not convert string to float: "1.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "rssi"': could not convert string to float: "-71.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "tempc"': could not convert string to float: "-1.37"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "dayRain"': could not convert string to float: "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "dailyrainin"': could not convert string to float: "0.132"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "humidex"': could not convert string to float: "29.534"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "windDir"': could not convert string to float: "53.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "cloudbase"': could not convert string to float: "3141.49934286"}

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "hourRain"': could not convert string to float: "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for '{"soc"': could not convert string to float: "68.8047"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "rainin"': could not convert string to float: "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: wxMesh: cannot read value for ' "rain"': could not convert string to float: "0.0"

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: engine: Main loop exiting. Shutting engine down.

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: rtgd: Shut down Thread-1 thread.

Mar 25 15:35:57 ubuntu-freenas weewx[5297]: engine: Caught unrecoverable exception in engine:

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****  'dateTime'

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****  Traceback (most recent call last):

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****    File "/usr/share/weewx/weewx/engine.py", line 871, in main

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****      engine.run()

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****    File "/usr/share/weewx/weewx/engine.py", line 190, in run

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****      self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****    File "/usr/share/weewx/weewx/engine.py", line 223, in dispatchEvent

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****      callback(event)

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****    File "/usr/share/weewx/weewx/wxservices.py", line 45, in new_loop_packet

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****      self.calc.do_calculations(event.packet, 'loop')

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****    File "/usr/share/weewx/weewx/wxservices.py", line 189, in do_calculations

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****      getattr(self, 'calc_' + obs)(data_us, data_type)

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****    File "/usr/share/weewx/weewx/wxservices.py", line 306, in calc_maxSolarRad

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****      data['dateTime'], self.atc)

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****  KeyError: 'dateTime'

Mar 25 15:35:57 ubuntu-freenas weewx[5297]:     ****  Exiting.

Seems like it gets part way reading the payload, but then dies on trying to convert the values?  Not sure Bill, is what I'm trying to do here possible?  

On Friday, 24 March 2017 20:23:37 UTC-4, mwall wrote:

[Robert Mantel]

I would prefer the single measurement idea as potentially you could have different update rates for variables from different devices in different locations.  How did you modify the driver for this?

[Robert Mantel]

I'm using Mathews MQTT publisher on my raspberry pi and then on another instance of weewx on my freenas box in a ubuntu vm I'm subscribing with Bill's driver, so given using Mathews publisher with defaults it should be using the weewx standard labels should it not?

On Saturday, 25 March 2017 10:36:24 UTC-4, Wysiwyg wrote:

[Bill Morrow]

I haven't died, just busy with work.

Robert, my driver does not read JSON, it's only simple key-value pairs in a string. So for example, it doesn't know to strip the quotes off the key or the value here:

"altimeter": "30.4916869968"

It wants

altimeter: 30.4916869968

I am planning on adopting WYSIWYG (Frederic's) approach of a separate MQTT topic for each value.

[Robert Mantel]

Ok, I will stand pat until you update the driver.  I'm not ready to switch to an mqtt setup yet anyways.  I want to swap out the particle P1 for a moteino for my sensor front end.  I'm going to convert the P1 to act as my rf to wifi gateway, or I may do this with my raspberry pi, not quite sure yet.

[Andy]

I have this up and running with a wemos D1 R1 using a slightly modified sketch from here: https://github.com/bonjour81/ESP8266-MQTT-WEEWX. Only using the simulated temp and humidity values for now. Weewx is 3.7.1 on a Raspberry Pi2. Seems ok for the last few hours. https://www.andylittle.us/weewxuno. I have a dht11 and Davis rain/wind sensors to use with the wemos D1.

Only issue I had was the paho-mqtt needed to be installed with pip. Thanks for all this.

Andy

[Wysiwyg]

Thx for feedback Andy !!

Nice to hear it works fine !

the MQTT driver works fine also on my side, with barometer & inTemp sensors.

Still slowly but not stopping developping the other sensors

[Wysiwyg]

Hi There!

I just wanted to share a bit the news on this project.

The mqtt driver works since a few monthes now with pressure, barometer & intemp like a charm !

Since a few days, I introduced outTemp, outHumidity, rain & UV. it works perfectly !

Best regards,

Frederic

[Neil Trimboy]

I am wanting to get some remote sensors data into weewx and was considering mqtt.

It seems there are currently at least 2 threads of development on drivers/services to load data into weewx ?

Could someone summarize their differences and statuses please?. Are they forks and work fundamentally differently, or is one a more mature version ?

thanks

[Wysiwyg]

Hello,

The initial driver from Bill is working (as far as I know).

It is subscribing to a single mqtt topic which should contain a long string looking like this:

TIME:0,AMBT: 7.23,BARP:1001.97,RHUM:70.92,HUMT: 4.44,IRRA:12,BATV:974,PHOV:673,SYST:33.20,WIND:  0.0,WDIR:  0.0

In the conf file, it is set the dictionnary to convert labels like "HUMT" to weewx database schema 'outHumidity'.

My own modification if based on Bill's job, but I modified a bit to use separated mqtt topics for each sensor.

for example I have the following mqtt topics:  "weewx/outTempature" , "weewx/barometer" etc...

the sensors publish the measurement value a string in the topics. for example "23.42" for temperature (when it's 23.42°C).

weewx is subscribing to all topics below "weewx/" (but you can choose something else as root topic).

I have done this modification because my sensors are at different locations and different rates.

for example, temperature is published  every few minutes, rain is published only when it's raining.

to make it simple, as I am a python noob, I use as right part of the mqtt topic exactly weewx database schema : 

for example, in the topic " weewx/outTemperature"  the "outTemperature" is the same label as weewx use.

in case you publish some topics not available in the schema, weewx will just not consider them.

For example, I have published some topics like "www/Vsolar" which is the voltage of the solar panel supplying the sensor. 

This value Vsolar is rejected by weewx unless you add it in the schema.

This is working pretty well for some time now on my setup (barometer, inTemperature, outTemperature, outHumidity, rain & UV are working well.)

I hope that helps ?

[Neil Trimboy]

Yes it does, thanks

[Neil Trimboy]

Frederic
Thanks for your reply.
I have forked your code and am in the process of making it also work as a service rather than just a driver.
Currently running fine for several days suppling my solar readings in as a service to weewx running OWFS driver via mosquito running on same platform (Pi)
Neil

[Frederic Stuyk]

Hi Neil !

Nice to hear it works for you too!

I am in same configuration for mosquitto (also running on same device: an orange pi).

So you have both owfs and mqtt as data inputs for weewx ?

[Neil Trimboy]

Correct.... The standard OWFS running as a driver for my main inputs and my modified version of yours that now runs as a service to obtain additional data from some other sensors

[Robert Mantel]

Hey Fredric, I implemented your version of the wxMesh driver, but sometimes I'm getting this error and I'm not sure if it's the realtime gauge data issue or the wxMesh issue but sometimes I'll get multiple cycles of this:

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: **** Traceback (most recent call last):

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: ****   File "/usr/share/weewx/user/rtgd.py", line 875, in process_packet

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: ****     data = self.calculate(cached_packet)

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: ****   File "/usr/share/weewx/user/rtgd.py", line 1030, in calculate

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: ****     self.db_manager, ts - 3600, 300)

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: ****   File "/usr/share/weewx/user/rtgd.py", line 1962, in calc_trend

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: ****     return now - then

Oct 24 21:48:22 raspberrypi weewx[1298]: rtgdthread: **** TypeError: unsupported operand type(s) for -: 'float' and 'NoneType'

Now I'm getting updates from my Moteino powered weather station about every 3-4 seconds and have wxmesh setup to take loop packets as quickly as it gets them, I'm wondering if it's getting out of sync somehow?  Maybe someone else has seen this?  Over all this works great.  I love having the information available in mqtt which makes it usable not just for weewx but for anything else where you'd like to use that data.

[gjr80]

The error below is due to a bug in the trend calculation code in the realtime gauge-data extension. In this case the historical observation being used in the trend calculation is None and that is throwing the error. I will raise an issue and fix it in the next realtime gauge-data extension release. In the meantime if you are up to it you can make the following change to bin/user/rtgd.py line 1956 fix the problem:

    if then_record is None:
        return None
    else:
        if obs_type not in then_record or then_record[obs_type] is None:
            return None
        else:
            then_vt = weewx.units.as_value_tuple(then_record, obs_type)
            now = convert(now_vt, group).value
            then = convert(then_vt, group).value
            return now - then

You will need to restart weeWX after saving the change.

It looks as if you are using v0.2.10 of the extension, the current release is v0.3.0 which fixes a number of issues (not the one above) and adds a number of new features.

Gary

[Robert Mantel]

Ok, I will try this, and yes I should update the extension....thanks for the heads up.

[Robert Mantel]

That seems to have fixed the problem, thanks Gary.

[Andreas Landixus]

Can you please answer me some questions.

I run a weewx with fine offset driver (WS 2080) running well for years on a RPi3.

Now i want to receive more date for example a BME280 in the attic and DS18B20 for Ground temperature.

I have 2 Nodemcus (ESP 8266) running and they send on topic /weather the data with MQTT (1 for BME280 and 1 for DS18B20)

My questions is to receive the data, do i need a second instance, or can i include the wxMesh Driver from morrowwm @github into the current running fine offset?

When i need a second instance then i have 2 databases? And when yes, how i collect and show the data on 1 site on local server?

I am not really sure how to include it.

Thanks for help.

Am Dienstag, 21. Februar 2017 14:23:33 UTC+1 schrieb Bill Morrow:

This conversation has been copied over from the weewx-user group (https://groups.google.com/d/msg/weewx-user/zhl4I7oRtt8/5pwhOAioBgAJ). I believe it is more appropriate to continue here.

 

[Andreas Landixus]

Hmmm ok if i run a second instance i have 2 confs:

ws2080.conf

mqttmesh.conf

then i have 2 databases (ws2080.sdb and mqttmesh.sdb)

then i can read both databases?

Like:

[Databases]
    # This section binds to the actual database to be used

    [[archive_sqlite1]]
        database_type = SQLite
        database_name = ws2080.sdb

    [[archive_sqlite2]]
        database_type = SQLite
        database_name = mqttmesh.sdb
      

[gjr80]

Certainly can, if you've not already been there have a look at Using multiple bindings in the Customization Guide. It covers both tags in reports and images.

Gary

[Andreas Landixus]

Ah i understand deeper.

So i can use the same database too. Because i have control what i send with my nodemcu.

I send over mqtt to the driver for example:

[[label_map]]
SEN1 = Sensor1Meter
SEN2 = Sensor2Meter
SEN3 = Sensor3Meter
SEN4 = Sensor4Meter

 

and "TIME = dateTime" for the key,

Because when i have different named values it does not affect the "current" data and i use "latest" values? :)

I hope i thinking it right.

[Andreas Landixus]

When i read through the complete documetation of weewx 

*puh that much...

i try to create a service/extension that reads mqtt lines, i add archive and values "premade" to the database like mqtt_sources.sdb

create new tables sensor001... sensor999 where i can fill the custom values.

later grab data with custom names for the templates.

So anybody can add custom cheap sensors to weewx.

I think easyesp as a mqtt publish source is user friendly enough to make it work easier.

or is there an easier way? Not that i dev what already exists :)

[Ralph Underwood]

Hi Neil

I have been attempting to get the mqtt stuff working and wonder if you have made any progress?  (I have a post about problems on the user group)

Thanks

[Bill Morrow]

On Monday, 21 August 2017 08:25:59 UTC-3, Neil Trimboy wrote:

Correct.... The standard OWFS running as a driver for my main inputs and my modified version of yours that now runs as a service to obtain additional data from some other sensors

Neil, is your re-implementation as a service available anywhere? It seems like the preferred way to go. Most people have a driver to commercial weather instrument, and want to augment it with the odd datum from some other sensor. So it the MQTT subscriber driver (wxMesh is not a good name anymore) turns in to a service, that becomes possible.

 

[Neil Trimboy]

Ralph/Bill

Good job you pinged me privately as I was not subscribed to this thread for some reason.....anyway....

My MQTT fork has been running as a service along with OWFS driver for about 6 months with no big issues. Most of my problems are due to my One-wire system hanging.

I have a separate piece of code polling a yoctopuce USB analogue to digital converter connected to my apogee SP-110 pyranometer, this posts to mosquito.

My mqtt service pulls in this data from mosquito

The rest of my system is one-wire running as a driver

I have not dabbled with my MQTT code since I shoe stringed it together so what I have is dirty and not really something I'd want to brag about, it still needs work to be robust and kind to failures/exceptions.

The code is on github, the commit comment is "Messy but functional. No exception catches"

https://github.com/nellyt/ESP8266-MQTT-WEEWX/commit/26127c5da24fd814b5871640e2c896a395fee0a0

This was I recall a fork of bonjour81's fork of Bill's.

I recall I was using this because of the way i structured my MQTT data

I think if you look at some of the standard include drivers/services from Mwalll/Tom you will see that they support both driver or service. By looking at a simple one of these you can understand how to convert your code to a service....I think I based mine on the deltas between service/driver in Mwalls OWFS plugin. There may well be more elegant coding ways of achieving this now...i was running code from at least a year ago  when i did this.

Also relevant is a read of http://www.weewx.com/docs/customizing.htm#service_engine

and http://www.weewx.com/docs/customizing.htm#Adding_2nd_source

I would suggest looking at what I did and then incorporating similar changes in your code (Bill), certainly dont fork mine as I think you are multiple bug fixes ahead of me!

I will keep track of thread but afraid I have little time available at present. :(

Neil

[Franz]

Hello,
I have a problem with weewx in connection with MQTT.
I only use sensors that transmit the data via MQTT to the broker Mosquitto (Rasberrypi 2) with the ESP8266 (no commercial weather station). Unfortunately, the data is not transferred to weewx.
I attach to logfile, my weewx.conf, wxMesh.py and a copy of Mosquitto data from the broker.
Can someone help me?
weewx is perfect for me to record my data from my self-made weather station.
thanks in advance!
Greetings Franz

-----

weewx.service - LSB: weewx weather system
   Loaded: loaded (/etc/init.d/weewx)
   Active: active (exited) since So 2018-05-20 19:13:31 CEST; 12s ago
  Process: 1203 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS)
  Process: 1275 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

Mai 20 19:13:31 raspberrypi weewx[1289]: ****      self.setupStation(config_dict)
Mai 20 19:13:31 raspberrypi weewx[1289]: ****    File "/home/weewx/bin/weewx/engine.py", line 95, in setupStation
Mai 20 19:13:31 raspberrypi weewx[1289]: ****      __import__(driver)
Mai 20 19:13:31 raspberrypi weewx[1289]: ****    File "/home/weewx/bin/user/wxMesh.py", line 48
Mai 20 19:13:31 raspberrypi weewx[1289]: ****          syslog.syslog(dst, 'wxMesh: %s' % msg)
Mai 20 19:13:31 raspberrypi weewx[1289]: ****      ^
Mai 20 19:13:31 raspberrypi weewx[1289]: ****  SyntaxError: invalid syntax
Mai 20 19:13:31 raspberrypi weewx[1289]: ****  Exiting.
Mai 20 19:13:31 raspberrypi systemd[1]: Started LSB: weewx weather system.
Mai 20 19:13:31 raspberrypi weewx[1275]: Starting weewx weather system: weewx.




----------------------------------


# WEEWX CONFIGURATION FILE
#
# Copyright (c) 2009-2015 Tom Keffer <tke...@gmail.com>
# See the file LICENSE.txt for your rights.

##############################################################################

# This section is for general configuration information.

# Set to 1 for extra debug info, otherwise comment it out or set to zero
debug = 1

# Root directory of the weewx data file hierarchy for this station
WEEWX_ROOT = /home/weewx

# How long to wait before timing out a socket (FTP, HTTP) connection
socket_timeout = 20

# Do not modify this. It is used when installing and updating weewx.
version = 3.8.0

##############################################################################

#   This section is for information about the station.
[Station]
   station_type = wxMesh

[wxMesh]
    host = 192.168.1.30
    username = xxxxxx
    password = yyyyyy
    topic = weewx/+
    driver = user.wxMesh
    poll_interval = 1
    loop_on_init = true

    [[label_map]]
       TIME = dateTime
       HUMT = outTemp
       RHUM = outHumidity

    # Description of the station location
    location = Wetter Rheinbach Voreifel
   
    # Latitude and longitude in decimal degrees
    latitude = 50.3552
    longitude = 6.5346
   
    # Altitude of the station, with unit it is in. This is downloaded from
    # from the station if the hardware supports it.
    altitude = 270, meter    # Choose 'foot' or 'meter' for unit
   
    # Set to type of station hardware. There must be a corresponding stanza
    # in this file with a 'driver' parameter indicating the driver to be used.
    # station_type = Simulator
   
    # If you have a website, you may specify an URL
    #station_url = http://www.example.com
   
    # The start of the rain year (1=January; 10=October, etc.). This is
    # downloaded from the station if the hardware supports it.
    rain_year_start = 1
   
    # Start of week (0=Monday, 6=Sunday)
    week_start = 6

##############################################################################

[Simulator]
    # This section is for the weewx weather station simulator
   
    # The time (in seconds) between LOOP packets.
    loop_interval = 2.5
   
    # The simulator mode can be either 'simulator' or 'generator'.
    # Real-time simulator. Sleep between each LOOP packet.
    mode = simulator
    # Generator.  Emit LOOP packets as fast as possible (useful for testing).
    #mode = generator
   
    # The start time. Format is YYYY-mm-ddTHH:MM. If not specified, the default
    # is to use the present time.
    #start = 2011-01-01T00:00
   
    # The driver to use:
    # driver = weewx.drivers.simulator

##############################################################################

#   This section is for uploading data to Internet sites

[StdRESTful]
   
  [[MQTT]]
       topic = weewx
       append_units_label = True

       retain = False
       aggregation = individual

       unit_system = METRIC
       binding = loop
 
  [[StationRegistry]]
        # To register this weather station with weewx, set this to true
        register_this_station = false
   
    [[AWEKAS]]
        # This section is for configuring posts to AWEKAS.
       
        # If you wish to do this, set the option 'enable' to true,
        # and specify a username and password.
        enable = false
        username = replace_me
        # To guard against parsing errors, put your password in quotes:
        password = replace_me
   
    [[CWOP]]
        # This section is for configuring posts to CWOP.
       
        # If you wish to do this, set the option 'enable' to true,
        # and specify the station ID (e.g., CW1234).
        enable = false
        station = replace_me
   
    # If this is an APRS (radio amateur) station, uncomment
    # the following and replace with a passcode (e.g., 12345).
    #passcode = replace_me (APRS stations only)
   
    [[PWSweather]]
        # This section is for configuring posts to PWSweather.com.
       
        # If you wish to do this, set the option 'enable' to true,
        # and specify a station and password.
        enable = false
        station = replace_me
        # To guard against parsing errors, put your password in quotes:
        password = replace_me
   
    [[WOW]]
        # This section is for configuring posts to WOW.
       
        # If you wish to do this, set the option 'enable' to true,
        # and specify a station and password.
        enable = false
        station = replace_me
        # To guard against parsing errors, put your password in quotes:
        password = replace_me
   
    [[Wunderground]]
        # This section is for configuring posts to the Weather Underground.
       
        # If you wish to do this, set the option 'enable' to true,
        # and specify a station (e.g., 'KORHOODR3') and password.
        enable = false
        station = replace_me
        # To guard against parsing errors, put your password in quotes:
        password = replace_me
       
        # Set the following to True to have weewx use the WU "Rapidfire"
        # protocol. Not all hardware can support it. See the User's Guide.
        rapidfire = False

##############################################################################

#   This section specifies what reports, using which skins, to generate.

[StdReport]
   
    # Where the skins reside, relative to WEEWX_ROOT
    SKIN_ROOT = skins
   
    # Where the generated reports should go, relative to WEEWX_ROOT
    HTML_ROOT = public_html
   
    # The database binding indicates which data should be used in reports.
    data_binding = wx_binding
   
    # Each of the following subsections defines a report that will be run.
   
    [[StandardReport]]
        # See the customizing guide to change the units, plot types and line
        # colors, modify the fonts, display additional sensor data, and other
        # customizations. Many of those changes can be made here by overriding
        # parameters, or by modifying templates within the skin itself.
       
        # The StandardReport uses the 'Standard' skin, which contains the
        # images, templates and plots for the report.
        skin = Standard
        [[[Units]]]
            [[[[Groups]]]]
                group_altitude = meter
                group_speed2 = meter_per_second2
                group_pressure = mbar
                group_rain = mm
                group_rainrate = mm_per_hour
                group_temperature = degree_C
                group_degree_day = degree_C_day
                group_speed = meter_per_second
   
    [[FTP]]
        # FTP'ing the results to a webserver is treated as just another report,
        # albeit one with an unusual report generator!
        skin = Ftp
       
        # If you wish to use FTP, uncomment and fill out the next four lines.
        #user = replace with the ftp username
        #password = replace with the ftp password; put in quotes to guard against parsing errors.
        #server = replace with the ftp server name, e.g, www.threefools.org
        #path = replace with the ftp destination directory (e.g., /weather)
       
        # Set to True for an FTP over TLS (FTPS) connection. Not all servers
        # support this.
        secure_ftp = False
       
        # To upload files from something other than what HTML_ROOT is set
        # to above, specify a different HTML_ROOT here.
        #HTML_ROOT = public_html
       
        # Most FTP servers use port 21
        port = 21
       
        # Set to 1 to use passive mode, zero for active mode
        passive = 1
   
    [[RSYNC]]
        # rsync'ing to a webserver is treated as just another report
        skin = Rsync
       
        # If you wish to use rsync, you must configure passwordless ssh using
        # public/private key authentication from the user account that weewx
        # runs as to the user account on the remote machine where the files
        # will be copied.
        #
        # The following three lines determine where files will be sent.
        #server = replace with the rsync server name, e.g, www.threefools.org
        #path = replace with the rsync destination directory (e.g., /weather)
        #user = replace with the rsync username
       
        # Rsync can be configured to remove files from the remote server if
        # they don't exist under HTML_ROOT locally. USE WITH CAUTION: if you
        # make a mistake in the remote path, you could could unintentionally
        # cause unrelated files to be deleted. Set to 1 to enable remote file
        # deletion, zero to allow files to accumulate remotely.
        delete = 0

##############################################################################

#   This service acts as a filter, converting the unit system coming from
#   the hardware to a unit system in the database.

[StdConvert]
   
    # The target_unit affects only the unit system in the database. Once
    # chosen it cannot be changed without converting the entire database.
    # Modification of target_unit after starting weewx will result in
    # corrupt data - the database will contain a mix of US and METRIC data.
    #
    # The value of target_unit does not affect the unit system for
    # reporting - reports can display US, Metric, or any combination of units.
    #
    # In most cases, target_unit should be left as the default: US
    #
    # In particular, those migrating from a standard wview installation
    # should use US since that is what the wview database contains.
   
    # DO NOT MODIFY THIS VALUE UNLESS YOU KNOW WHAT YOU ARE DOING!
    target_unit = US    # Options are 'US', 'METRICWX', or 'METRIC'

##############################################################################

#   This section can adjust data using calibration expressions.

[StdCalibrate]
   
    [[Corrections]]
        # For each type, an arbitrary calibration expression can be given.
        # It should be in the units defined in the StdConvert section.
        # Example:
        foo = foo + 0.2

##############################################################################

#   This section is for quality control checks. If units are not specified,
#   values must be in the units defined in the StdConvert section.

[StdQC]
   
    [[MinMax]]
        barometer = 26, 32.5, inHg
        outTemp = -40, 120, degree_F
        inTemp = 10, 120, degree_F
        outHumidity = 0, 100
        inHumidity = 0, 100
        windSpeed = 0, 120, mile_per_hour
        pressure = 24, 34.5, inHg

##############################################################################

#   This section controls the origin of derived values.

[StdWXCalculate]
   
    [[Calculations]]
        # Derived quantities are calculated by this service. Possible values are:
        #  hardware        - use the value provided by hardware
        #  software        - use the value calculated by weewx
        #  prefer_hardware - use value provide by hardware if available,
        #                      otherwise use value calculated by weewx
       
        pressure = prefer_hardware
        barometer = prefer_hardware
        altimeter = prefer_hardware
        windchill = prefer_hardware
        heatindex = prefer_hardware
        dewpoint = prefer_hardware
        inDewpoint = prefer_hardware
        rainRate = prefer_hardware

##############################################################################

#   For hardware that supports it, this section controls how often the
#   onboard clock gets updated.

[StdTimeSynch]
   
    # How often to check the weather station clock for drift (in seconds)
    clock_check = 14400
   
    # How much it can drift before we will correct it (in seconds)
    max_drift = 5

##############################################################################

#   This section is for configuring the archive service.

[StdArchive]
   
    # If the station hardware supports data logging then the archive interval
    # will be downloaded from the station. Otherwise, specify it (in seconds).
    archive_interval = 300
   
    # If possible, new archive records are downloaded from the station
    # hardware. If the hardware does not support this, then new archive
    # records will be generated in software.
    # Set the following to "software" to force software record generation.
    record_generation = hardware
   
    # Whether to include LOOP data in hi/low statistics
    loop_hilo = True
   
    # The data binding used to save archive records
    data_binding = wx_binding

##############################################################################

#   This section binds a data store to a database.

[DataBindings]
   
    [[wx_binding]]
        # The database must match one of the sections in [Databases].
        # This is likely to be the only option you would want to change.
        database = archive_sqlite
        # The name of the table within the database
        table_name = archive
        # The manager handles aggregation of data for historical summaries
        manager = weewx.wxmanager.WXDaySummaryManager
        # The schema defines the structure of the database.
        # It is *only* used when the database is created.
        schema = schemas.wview.schema

##############################################################################

#   This section defines various databases.

[Databases]
   
    # A SQLite database is simply a single file
    [[archive_sqlite]]
        database_type = SQLite
        database_name = weewx.sdb
   
    # MySQL
    [[archive_mysql]]
        database_type = MySQL
        database_name = weewx

##############################################################################

#   This section defines defaults for the different types of databases.

[DatabaseTypes]
   
    # Defaults for SQLite databases
    [[SQLite]]
        driver = weedb.sqlite
        # Directory in which the database files are located
        SQLITE_ROOT = %(WEEWX_ROOT)s/archive
   
    # Defaults for MySQL databases
    [[MySQL]]
        driver = weedb.mysql
        # The host where the database is located
        host = localhost
        # The user name for logging in to the host
        user = weewx
        # The password for the user name. Put in quotes to guard against parsing errors.
        password = weewx

##############################################################################

#   This section configures the internal weewx engine.

[Engine]
   
    [[Services]]
        # This section specifies the services that should be run. They are
        # grouped by type, and the order of services within each group
        # determines the order in which the services will be run.
        prep_services = weewx.engine.StdTimeSynch
        data_services = ,
        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate
        archive_services = weewx.engine.StdArchive
        restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS
        report_services = weewx.engine.StdPrint, weewx.engine.StdReport



----------------




#!/usr/bin/python
#
# weewx driver that reads data from MQTT subscription
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free Software
# Foundation, either version 3 of the License, or any later version.
#
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE.
#
# See http://www.gnu.org/licenses/

#
# The units must be weewx.US:   (this comment need to be updated)
#   degree_F, inHg, inch, inch_per_hour, mile_per_hour
#
# To use this driver, put this file in the weewx user directory, then make
# the following changes to weewx.conf:
#
# [Station]
#     station_type = wxMesh
# [wxMesh]
#     host = localhost           # MQTT broker hostname
#     topic = weewx              # topic, mqtt topics should have format "topic/label_map", ex: weather/humi
#     driver = user.wxMesh
#
# If the variables in the file have names different from those in weewx, then
# create a mapping such as this:
#
# [wxMesh]
#     ...
#     [[label_map]]
#         temp = outTemp
#         humi = outHumidity
#         in_temp = inTemp
#         in_humid = inHumidity
from __future__ import with_statement
import syslog
import time
import paho.mqtt.client as mqtt
import weewx.drivers

DRIVER_VERSION = "0.1"

def logmsg(dst, msg):
    syslog.syslog(dst, 'wxMesh: %s' % msg)

def logdbg(msg):
    logmsg(syslog.LOG_DEBUG, msg)

def loginf(msg):
    logmsg(syslog.LOG_INFO, msg)

def logerr(msg):
    logmsg(syslog.LOG_ERR, msg)

def _get_as_float(d, s):
    v = None
    if s in d:
        try:
            v = float(d[s])
        except ValueError, e:
            logerr("cannot read value for '%s': %s" % (s, e))
    return v

def loader(config_dict, engine):
    return wxMesh(**config_dict['wxMesh'])

class wxMesh(weewx.drivers.AbstractDevice):
    """weewx driver that reads data from a file"""

    def __init__(self, **stn_dict):
        # where to find the data file
        self.host = stn_dict.get('host', 'localhost')
        # subscribe to all sub-topic of the topic define in weewx.conf
        self.topic = stn_dict.get('topic', 'weather') + "/#"
    self.username = stn_dict.get('username', 'XXX')
        self.password = stn_dict.get('password', 'password')
        # how often to poll the weather data file, seconds
        self.poll_interval = float(stn_dict.get('poll_interval', 5.0))
        # mapping from variable names to weewx names
        self.label_map = stn_dict.get('label_map', {})

        loginf("host is %s" % self.host)
        loginf("topic is %s" % self.topic)
        loginf("polling interval is %s" % self.poll_interval)
        loginf('label map is %s' % self.label_map)
       
        self.payload = "Empty"
    self.receive_buffer = {}
        #self.payloadList = [payload]
        self.client = mqtt.Client(client_id="XXX", protocol=mqtt.MQTTv31)

    #self.client.on_connect = self.on_connect
    self.client.on_message = self.on_message

        self.client.username_pw_set(self.username, self.password)
    self.client.connect(self.host, 1883, 60)
    self.client.subscribe(self.topic, qos=0)

    # The callback for when a PUBLISH message is received from the server.
    def on_message(self, client, userdata, msg):
    self.payload = str(msg.payload)
    string_topic = str(msg.topic)
    key =  string_topic.split('/')[-1]
        self.receive_buffer[key] = str(msg.payload)

    def closePort(self):
        self.client.disconnect()

    def genLoopPackets(self):
        self.client.loop_start()   # enable to receive ('on_message') in background
        while True:
            time.sleep(self.poll_interval) # wait for some MQTT data to be published
            data = self.receive_buffer
            if data:       # if data is not empty then prepare loop packet
                _packet = {'dateTime': int(time.time() + 0.5),'usUnits': weewx.METRIC}
                logdbg("dateTime %s" % _packet["dateTime"])
                for vname in data:
                    _packet[self.label_map.get(vname, vname)] = _get_as_float(data, vname)
                    logdbg("buffer content for key: %s =  %s" %(vname, data[vname]))
                yield _packet
                self.receive_buffer.clear()
                data.clear()

    @property
    def hardware_name(self):
        return "wxMesh"


----------------
pi@raspberrypi ~ $ mosquitto_sub -d -u Axxxx -P yyyyyy -t "weewx/#"
Client mosqsub|1024-raspberryp sending CONNECT
Client mosqsub|1024-raspberryp received CONNACK
Client mosqsub|1024-raspberryp sending SUBSCRIBE (Mid: 1, Topic: weewx/#, QoS: 0)
Client mosqsub|1024-raspberryp received SUBACK
Subscribed (mid: 1): 0
Client mosqsub|1024-raspberryp sending PINGREQ
Client mosqsub|1024-raspberryp received PINGRESP
Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outHumidity', ... (5 bytes))
48.80
Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outTemp', ... (5 bytes))
18.90
Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outHumidity', ... (5 bytes))
48.80
Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outTemp', ... (5 bytes))
18.90

===================

[Bill Morrow]

Franz, it looks like some of the indentation in the wxMesh.py driver has been corrupted. python uses indentation to understand the structure of the code. Could you check the lines I have indicated below? The error might be right - do you have some special character on line 48?.

You are using wysiwig's fork of my original driver, and your MQTT publication looks correct for it.

Also, paste source code in and use the {} to "Highlight code syntax". This will help understand where the issue is.

    def __init__(self, **stn_dict):
        # where to find the data file
        self.host = stn_dict.get('host', 'localhost')
        # subscribe to all sub-topic of the topic define in weewx.conf
        self.topic = stn_dict.get('topic', 'weather') + "/#"

    self.username = stn_dict.get('username', 'XXX') <---- Is this line not indented in your wxMesh.py file?

        self.password = stn_dict.get('password', 'password')
        # how often to poll the weather data file, seconds
        self.poll_interval = float(stn_dict.get('poll_interval', 5.0))
        # mapping from variable names to weewx names
        self.label_map = stn_dict.get('label_map', {})

        loginf("host is %s" % self.host)
        loginf("topic is %s" % self.topic)
        loginf("polling interval is %s" % self.poll_interval)
        loginf('label map is %s' % self.label_map)
       
        self.payload = "Empty"

    self.receive_buffer = {} <---- This line should be indented
        #self.payloadList = [payload]

        self.client = mqtt.Client(client_id="XXX", protocol=mqtt.MQTTv31)

    #self.client.on_connect = self.on_connect
    self.client.on_message = self.

on_message

        self.client.username_pw_set(self.username, self.password) <---- Too much indent, probably harmless
    self.client.connect(self.host, 1883, 60)

    self.client.subscribe(self.topic, qos=0)

    # The callback for when a PUBLISH message is received from the server.
    def on_message(self, client, userdata, msg):
    self.payload = str(msg.payload)
    string_topic = str(msg.topic)
    key =  string_topic.split('/')[-1]

        self.receive_buffer[key] = str(msg.payload)<----Too much indent, probably harmless

[Franz]

Hello Bill,
Thank you for the quick help! The errors were caused by copying (mac RTF - TXT). Am beginners ...
In the file were still many formatting errors for this reason, I copied from the github the zip file weewxMQTT / bin / user / wxMesh.py from January 19th in the Raspberry. I hope it is the right wxMesh file. If not, please refer to the correct wxMesh.py file.
Unfortunately I get another error here, see logfile.
What am I doing wrong?
Thank you for your help!
Greetings Franz

● weewx.service - LSB: weewx weather system
   Loaded: loaded (/etc/init.d/weewx)
   Active: active (exited) since Mo 2018-05-21 19:50:32 CEST; 40min ago
  Process: 1520 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS)
  Process: 1680 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

Mai 21 19:50:32 raspberrypi weewx[1694]: ****    File "/home/weewx/bin/user/...4
Mai 21 19:50:32 raspberrypi weewx[1694]: ****      except ValueError, e:
Mai 21 19:50:32 raspberrypi weewx[1694]: ****           ^
Mai 21 19:50:32 raspberrypi weewx[1694]: ****  SyntaxError: invalid syntax
Mai 21 19:50:32 raspberrypi weewx[1694]: ****  Exiting.
Mai 21 19:50:32 raspberrypi systemd[1]: Started LSB: weewx weather system.
Mai 21 19:50:32 raspberrypi weewx[1680]: Starting weewx weather system: weewx.
Mai 21 20:11:02 raspberrypi systemd[1]: Started LSB: weewx weather system.
Mai 21 20:15:25 raspberrypi systemd[1]: Started LSB: weewx weather system.
Mai 21 20:30:30 raspberrypi systemd[1]: Started LSB: weewx weather system.
Hint: Some lines were ellipsized, use -l to show in full.

[Bill Morrow]

I am guessing that you still have some problems with how wxMesh.py was copied in to place.

Try copying wxMesh.py as simple text by going to this link

https://raw.githubusercontent.com/bonjour81/ESP8266-MQTT-WEEWX/master/weewx/wxMQTT.py

right-click and "Save As" to your .../bin/user/wxMesh.py file.

.. when I examine the code, it looks like there are some errors, though. You could try the attached copy, where I have fixed some indentation. I have not tested it myself.

[Franz]

Hello Bill,
Thanks for the wxMesh.py file. I have copied this file unchanged to / home / weewx / bin / user. With sudo chmod 777 wxMesh.py rights changed. Subsequently, with the Python 2 IDLE formatting error sought and removed. When checking with the IDLE: Run / Check Module -> "There's an error in your program: *** 'yield' outside funcion (wxMesh.py, line 124).
Maybe wxMesh.py is not compatible with weewx-3.8.0. Enclosed the current log file. Thanks again for your effort!
Franz
——
pi@raspberrypi ~ $ sudo /etc/init.d/weewx start
[ ok ] Starting weewx (via systemctl): weewx.service.
pi@raspberrypi ~ $ sudo /etc/init.d/weewx status  -l

● weewx.service - LSB: weewx weather system
   Loaded: loaded (/etc/init.d/weewx)

   Active: active (exited) since Di 2018-05-22 16:04:18 CEST; 3s ago
  Process: 2524 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS)
  Process: 2636 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

Mai 22 16:04:18 raspberrypi weewx[2650]: ****    File "/home/weewx/bin/weewx/engine.py", line 71, in __init__
Mai 22 16:04:18 raspberrypi weewx[2650]: ****      self.setupStation(config_dict)
Mai 22 16:04:18 raspberrypi weewx[2650]: ****    File "/home/weewx/bin/weewx/engine.py", line 95, in setupStation
Mai 22 16:04:18 raspberrypi weewx[2650]: ****      __import__(driver)
Mai 22 16:04:18 raspberrypi weewx[2650]: ****    File "/home/weewx/bin/user/wxMesh.py", line 124
Mai 22 16:04:18 raspberrypi weewx[2650]: ****      yield _packet
Mai 22 16:04:18 raspberrypi weewx[2650]: ****  SyntaxError: 'yield' outside function
Mai 22 16:04:18 raspberrypi weewx[2650]: ****  Exiting.
Mai 22 16:04:18 raspberrypi systemd[1]: Started LSB: weewx weather system.
Mai 22 16:04:18 raspberrypi weewx[2636]: Starting weewx weather system: weewx.

Hint: Some lines were ellipsized, use -l to show in full.

pi@raspberrypi ~ $ sudo /etc/init.d/weewx stop
[ ok ] Stopping weewx (via systemctl): weewx.service.
pi@raspberrypi ~ $

[Wysiwyg]

Hi Franz,

If you use my variant of the driver (the link provided by Bill), I noticed a few differences in your weewx.conf (but it may not explain the issue):

you have:

[wxMesh]

    ....
    topic = weewx/+

I have:

[wxMesh]

   .....

   topic = weewx

   client = weewx_mqtt   #(whatever you like) => should be no issue.

I publish my sensors with topic "weewx/<weewx data label>", for example "weewx/outTemp",  the data content is raw values, for example "23.2"  (degrees)

I use weewx 3.8.0 and python 2.7.12  

My label map include anything my weewx can take.  Also, to keep it simple, I use same labels for mqtt & weewx (after a while playing with templates, I know them almost by heart :-D).

So my label map looks a bit stupid, just like this:

[label_map]

     outTemp = outTemp

     barometer = barometer

     .....

I put all my database schematic there, except dateTime !

An other thing that might cause issue :  you don't need dateTime in label or in mqtt topics

The driver will just pick the current time from the weewx device and set this as timestamp.

I assume it's good enought as poll interval is 1 or 2 seconds, see here:

_packet = {'dateTime': int(time.time() + 0.5),'usUnits': weewx.METRIC}

I'm not very good at python (I was barely able to modify Bill driver for my need ;-)), but don't hesitate to ask again in case I'm not clear.

One last thing: I move (very very slowly) my github from https://github.com/bonjour81/ESP8266-MQTT-WEEWX  to  https://github.com/bonjour81/station_meteo

I try (very very slowly) to learn how to use git a little more like it should be

Also, if you like, I would be very happy to discuss about your project ! (separately from this discussion). it looks like we have a more or less similar project (ESP8266 + mqtt + weewx).

Best regards,

[Franz]

Hello Wysiwyg,
thank you for the interest. I changed the file weewx.conf accordingly. [WxMesh]
   topic = weewx
and under [[label]]
removed the line with Time. Unfortunately, no success! On my rasberrypi 2 I use Python 2.7.9 and weewx 3.8.0
-
To my weather station. I printed with fusion360 for the wind speed, wind direction and rain gauge (rocker) sensors. Other sensors measure the temperature at different altitudes, humidity and particulate matter (PM10 and PM2.5). The ESP8266webserver sends in my private network. The data is displayed per html on a page. I would like to archive the data with weewx and present it more beautifully. Unfortunately I am a beginner in Python and have problems with the configuration of weewx in connection with mqtt. Bill is also trying to help me.
The simplest solution would be if you provided me with a working weewx.conf and a wxMseh.py. That would be nice!

Franz

---

pi@raspberrypi ~ $ sudo /etc/init.d/weewx status -l
● weewx.service - LSB: weewx weather system
   Loaded: loaded (/etc/init.d/weewx)

   Active: active (exited) since Di 2018-05-22 19:16:06 CEST; 13s ago
  Process: 1437 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS)
  Process: 1629 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

Mai 22 19:16:06 raspberrypi weewx[1643]: ****    File "/home/weewx/bin/weewx/engine.py", line 71, in __init__
Mai 22 19:16:06 raspberrypi weewx[1643]: ****      self.setupStation(config_dict)
Mai 22 19:16:06 raspberrypi weewx[1643]: ****    File "/home/weewx/bin/weewx/engine.py", line 95, in setupStation
Mai 22 19:16:06 raspberrypi weewx[1643]: ****      __import__(driver)
Mai 22 19:16:06 raspberrypi weewx[1643]: ****    File "/home/weewx/bin/user/wxMesh.py", line 124
Mai 22 19:16:06 raspberrypi weewx[1643]: ****      yield _packet
Mai 22 19:16:06 raspberrypi weewx[1643]: ****  SyntaxError: 'yield' outside function
Mai 22 19:16:06 raspberrypi weewx[1643]: ****  Exiting.
Mai 22 19:16:06 raspberrypi systemd[1]: Started LSB: weewx weather system.
Mai 22 19:16:06 raspberrypi weewx[1629]: Starting weewx weather system: weewx.
pi@raspberrypi ~ $ ^C
pi@raspberry

[Bill Morrow]

It's very strange, Franz. The errors you are getting do seem to be from python indenting errors. You copied the wxMesh.py file in place without changes, so the yield statement should be on line 127. Your error message indicates it is on line 124.

Try going in to your bin/user/ directory and run these commands at your shell prompt

wget https://raw.githubusercontent.com/bonjour81/ESP8266-MQTT-WEEWX/master/weewx/wxMQTT.py
mv wxMQTT.py wxMesh.py

You do not need to make the file mode 777.

[Vince Skahan]

On Tuesday, May 22, 2018 at 10:45:58 AM UTC-7, Bill Morrow wrote:

Try going in to your bin/user/ directory and run these commands at your shell prompt

wget https://raw.githubusercontent.com/bonjour81/ESP8266-MQTT-WEEWX/master/weewx/wxMQTT.py
mv wxMQTT.py wxMesh.py

You might also check the checksums on the file to verify it's the same as the master copy.

Mine looks like:

pi@pi3:/tmp $ sha1sum wxMQTT.py

826d007e040b540cb5c6092842667c9966852db9  wxMQTT.py

pi@pi3:/tmp $ sha256sum wxMQTT.py

e99e86a15f3315eb1dd0db38f4ce759c4af2cb4965f71e031000a380c087d32b  wxMQTT.py

[Franz]

Hello Vince,
with wget https: // ... copied the file wxMQTT.py. Checksum is ok.
Are the wxMesh.py and wxMQTT.py files the same?
The error messages have changed (see log file). I suspect it's because of the MQTT data. MQTT sends float values with me. I have to change the Arduino program. Many thanks for the help. I'm probably a little further.

Franz

---

pi@raspberrypi /home/weewx/bin/user $ ls
extensions.py   __init__.py   installer  wxMesh.pyc
extensions.pyc  __init__.pyc  wxMesh.py
pi@raspberrypi /home/weewx/bin/user $ sudo sha1sum wxMesh.py
826d007e040b540cb5c6092842667c9966852db9  wxMesh.py
pi@raspberrypi /home/weewx/bin/user $
----

pi@raspberrypi ~ $ sudo /etc/init.d/weewx start
[ ok ] Starting weewx (via systemctl): weewx.service.
pi@raspberrypi ~ $ sudo /etc/init.d/weewx status -l
● weewx.service - LSB: weewx weather system
   Loaded: loaded (/etc/init.d/weewx)

   Active: active (exited) since Mi 2018-05-23 11:52:38 CEST; 6s ago
  Process: 1396 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS)
  Process: 1591 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

Mai 23 11:52:38 raspberrypi weewx[1605]: ****    File "/home/weewx/bin/weewx/engine.py", line 865, in main
Mai 23 11:52:38 raspberrypi weewx[1605]: ****      engine = engine_class(config_dict)
Mai 23 11:52:38 raspberrypi weewx[1605]: ****    File "/home/weewx/bin/weewx/engine.py", line 74, in __init__
Mai 23 11:52:38 raspberrypi weewx[1605]: ****      self.preLoadServices(config_dict)
Mai 23 11:52:38 raspberrypi weewx[1605]: ****    File "/home/weewx/bin/weewx/engine.py", line 115, in ...ices
Mai 23 11:52:38 raspberrypi weewx[1605]: ****      self.stn_info = weewx.station.StationInfo(self.cons...n'])
Mai 23 11:52:38 raspberrypi weewx[1605]: ****    File "/home/weewx/bin/weewx/station.py", line 35, in ...it__
Mai 23 11:52:38 raspberrypi weewx[1605]: ****      self.altitude_vt = weewx.units.ValueTuple(float(alt...de")
Mai 23 11:52:38 raspberrypi weewx[1605]: ****  TypeError: float() argument must be a string or a number
Mai 23 11:52:38 raspberrypi weewx[1605]: ****  Exiting.

Hint: Some lines were ellipsized, use -l to show in full.
pi@raspberrypi ~ $

-----------

[Bill Morrow]

Hi Franz, progress! It looks like you have fixed the file corruption issue.

Some comments below...

On Wednesday, 23 May 2018 08:17:13 UTC-3, Franz wrote:

Hello Vince,
with wget https: // ... copied the file wxMQTT.py. Checksum is ok.
Are the wxMesh.py and wxMQTT.py files the same?

I think you have configuration in weewx.conf such that your file should be named wxMesh.py. Namely these configuration parameters::

    station_type = wxMesh
...
[wxMesh]
    driver = user.wxMesh

The error messages have changed (see log file). I suspect it's because of the MQTT data. MQTT sends float values with me. I have to change the Arduino program. Many thanks for the help. I'm probably a little further.

Consider editing your weewx.conf file and changing the debug setting from

# Set to 1 for extra debug info, otherwise comment it out or set to zero.
debug = 0

to

# Set to 1 for extra debug info, otherwise comment it out or set to zero.
debug =1

and then restart weewx. Watch the log file with

tail -f /var/log/syslog

This should give more information on how the driver is attempting to turn some part of the MQTT payload to a floating point number. 

[Franz]

Hello Bill,
here current log file (tail -f / var / sys / syslog) note.
I change my Arduino / ESP program and then try again.
Is the index.html (if the data type is String) created automatically?
Many thanks for your help!

Franz

----------

pi@raspberrypi ~ $ tail -f /var/log/syslog
May 23 13:20:14 raspberrypi kernel: [   21.063198] cfg80211: Calling CRDA to update world regulatory domain
May 23 13:20:17 raspberrypi kernel: [   24.223222] cfg80211: Calling CRDA to update world regulatory domain
May 23 13:53:08 raspberrypi systemd[710]: Time has been changed
May 23 13:53:08 raspberrypi systemd[1]: Time has been changed
May 23 13:53:10 raspberrypi kernel: [   27.383344] cfg80211: Calling CRDA to update world regulatory domain
May 23 13:53:13 raspberrypi kernel: [   30.543240] cfg80211: Calling CRDA to update world regulatory domain
May 23 13:53:16 raspberrypi kernel: [   33.703274] cfg80211: Calling CRDA to update world regulatory domain
May 23 13:53:19 raspberrypi kernel: [   36.863249] cfg80211: Calling CRDA to update world regulatory domain
May 23 13:53:23 raspberrypi kernel: [   40.023286] cfg80211: Calling CRDA to update world regulatory domain
May 23 13:53:26 raspberrypi kernel: [   43.183347] cfg80211: Exceeded CRDA call max attempts. Not calling CRDA
May 23 13:54:40 raspberrypi systemd[1]: Stopping LSB: weewx weather system...
May 23 13:54:40 raspberrypi weewx[1076]: Stopping weewx weather system: weewx not running....
May 23 13:54:40 raspberrypi systemd[1]: Stopped LSB: weewx weather system.
May 23 13:54:48 raspberrypi systemd[1]: Starting LSB: weewx weather system...
May 23 13:54:49 raspberrypi weewx[1138]: engine: Initializing weewx version 3.8.0
May 23 13:54:49 raspberrypi weewx[1138]: engine: Using Python 2.7.9 (default, Sep 17 2016, 20:26:04) #012[GCC 4.9.2]
May 23 13:54:49 raspberrypi weewx[1138]: engine: Platform Linux-4.1.7-v7+-armv7l-with-debian-8.0
May 23 13:54:49 raspberrypi weewx[1138]: engine: Locale is 'de_DE.UTF-8'
May 23 13:54:49 raspberrypi weewx[1138]: engine: pid file is /var/run/weewx.pid
May 23 13:54:49 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Wed May 23 13:55:19 2018 [try http://www.rsyslog.com/e/2007 ]
May 23 13:54:49 raspberrypi weewx[1128]: Starting weewx weather system: weewx.
May 23 13:54:49 raspberrypi systemd[1]: Started LSB: weewx weather system.
May 23 13:54:49 raspberrypi weewx[1142]: engine: Using configuration file /home/weewx/weewx.conf
May 23 13:54:49 raspberrypi weewx[1142]: engine: debug is 1
May 23 13:54:49 raspberrypi weewx[1142]: engine: Initializing engine
May 23 13:54:49 raspberrypi weewx[1142]: engine: Loading station type wxMesh (user.wxMesh)
May 23 13:54:49 raspberrypi weewx[1142]: wxMesh: host is 192.168.1.30
May 23 13:54:49 raspberrypi weewx[1142]: wxMesh: topic is weewx/#
May 23 13:54:49 raspberrypi weewx[1142]: wxMesh: polling interval is 1.0
May 23 13:54:49 raspberrypi weewx[1142]: wxMesh: label map is {'HUMT': 'outTemp', 'RHUM': 'outHumidity', 'location': 'Wetter Rheinbach Voreifel', 'latitude': '50.3552', 'longitude': '6.5346', 'altitude': ['270', 'meter'], 'rain_year_start': '1', 'week_start': '6'}
May 23 13:54:49 raspberrypi weewx[1142]: engine: Caught unrecoverable exception in engine:
May 23 13:54:49 raspberrypi weewx[1142]:     ****  float() argument must be a string or a number
May 23 13:54:49 raspberrypi weewx[1142]:     ****  Traceback (most recent call last):
May 23 13:54:49 raspberrypi weewx[1142]:     ****    File "/home/weewx/bin/weewx/engine.py", line 865, in main
May 23 13:54:49 raspberrypi weewx[1142]:     ****      engine = engine_class(config_dict)
May 23 13:54:49 raspberrypi weewx[1142]:     ****    File "/home/weewx/bin/weewx/engine.py", line 74, in __init__
May 23 13:54:49 raspberrypi weewx[1142]:     ****      self.preLoadServices(config_dict)
May 23 13:54:49 raspberrypi weewx[1142]:     ****    File "/home/weewx/bin/weewx/engine.py", line 115, in preLoadServices
May 23 13:54:49 raspberrypi weewx[1142]:     ****      self.stn_info = weewx.station.StationInfo(self.console, **config_dict['Station'])
May 23 13:54:49 raspberrypi weewx[1142]:     ****    File "/home/weewx/bin/weewx/station.py", line 35, in __init__
May 23 13:54:49 raspberrypi weewx[1142]:     ****      self.altitude_vt = weewx.units.ValueTuple(float(altitude_t[0]), altitude_t[1], "group_altitude")
May 23 13:54:49 raspberrypi weewx[1142]:     ****  TypeError: float() argument must be a string or a number
May 23 13:54:49 raspberrypi weewx[1142]:     ****  Exiting.
^[May 23 13:55:36 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Wed May 23 13:56:06 2018 [try http://www.rsyslog.com/e/2007 ]
May 23 13:55:36 raspberrypi systemd[1]: Stopping LSB: weewx weather system...
May 23 13:55:36 raspberrypi weewx[1206]: Stopping weewx weather system: weewx not running....
May 23 13:55:36 raspberrypi systemd[1]: Stopped LSB: weewx weather system.

--------------------

[Wysiwyg]

Hi There!

Sorry for confusion between wxMQTT.py & wxMesh.py.

 wxMesh.py => Original naming from Bill's driver.

When I discussed with Bill and modified his driver for my own use, Bill mention the name should be updated.

I named it wxMQTT.py, but I didnt finalized the job correctly  :-(

I shall change the name in weewx.conf and also I think there are some mention in the python code.

Anyway, if all is name wxMesh. it shall work.

regarding "float".

My arduino/ESP8266 code also publish some float. you can see here: https://github.com/bonjour81/station_meteo/blob/master/sensors/MQTT_TH_UV_rain/src/main.cpp.

The values are truncated at 2 digit after dot.

I also added a bit of checks in my arduino code to confirm validity of values before publishing

You can use an other subscriber to check the content of your messages in parallel to weewx (like mosquitto_sub), but for more comfortable use, you can try mqtt dash on android. ( https://play.google.com/store/apps/details?id=net.routix.mqttdash&hl=fr ).

You just have to setup your mqtt server once, and you can define some tiles for each topic.

I use "text" tiles for my weewx topics (ex: weewx/outTemp). 

It also display since how much time last publish was received.

it's pretty nice for debugging.

in parallel to 

tail -f /var/log/syslog

you may also try:

tail -f /var/log/mosquitto/mosquitto.conf

[Bill Morrow]

wxMesh: label map is {'HUMT': 'outTemp', 'RHUM': 'outHumidity', 'location': 'Wetter Rheinbach Voreifel', 'latitude': '50.3552', 'longitude': '6.5346', 'altitude': ['270', 'meter'], 'rain_year_start': '1', 'week_start': '6'}

Try taking the units of measure off the altitude entry in your label map in weewx.conf, and restarting.

I am not sure why you are reading some of these values periodically. Is the station mobile, such that the geo-location changes?

[Franz]

Hello Bill,
the data comes from the initial setup of weewx. These are my geodata from weewx. Should I delete those from the weewx.conf?
Franz

[Franz]

Hello Wysiwyg, hello Bill,
Thank you for the hints. I used the framework for my ESP8266 from Wysiwyg. That is well suited. For the test run, I deactivated all sensors and the deep sleep function. I use the AM2320 for temperature and humidity. In the loop, I send the data to the broker. Unfortunately wewx has problems with the data. I convert float to char in the ESP (see code example) or manually text like "39.5" returns the same error message. It takes a lot of patience ...
As a control, I use MQTT.fx on a Mac. Works perfectly. All data is displayed. In the configuration of MQTT.fx I entered the client ID (a020a60cc73b) of the ESP. Could this be a cause for the little problem? According to Bill, all location data has been deleted (see weewx.conf). Nevertheless problems.
Many thanks for the help!!!
Franz

ESP8266
----------------
void loop() {
// not used due to deepsleep
//
char tmp[20];   // String temp
char dtmpst[20];
float temp;
//
//
char tmh[20];   // String temp
char dthust[20];
float humi;

temp = am2320.readTemperature();
dtostrf(temp, 2, 2, dtmpst);   //float in str

humi = am2320.readHumidity();
dtostrf(humi, 2, 2, dthust);




if ((temp > -40) && (temp < 80)) {
 //  outTemp_pub.publish(dtmpst);
 outTemp_pub.publish("39.5");      //TEST manuell
 //  Serial.println(dtmpst);       // Normal
   delay(2000);
}



//if ( humi >= 0 && humi <= 100 ) {
//   outHumidity_pub.publish(dthust);
//   Serial.println(dthust);
//   delay(2000);
//}


  }



mosquitto -log
------------------
pi@raspberrypi ~ $ sudo tail -f /var/log/mosquitto/mosquitto.log
1527169098: mosquitto version 1.4.15 terminating
1527169103: mosquitto version 1.4.15 (build date Sat, 07 Apr 2018 11:13:41 +0100) starting
1527169103: Config loaded from /etc/mosquitto/mosquitto.conf.
1527169105: Opening ipv4 listen socket on port 1883.
1527169105: Opening ipv6 listen socket on port 1883.
1527169178: New connection from ::1 on port 1883.
1527169178: New client connected from ::1 as mosqsub|1092-raspberryp (c1, k60, u'AX700').
1527169191: New connection from 192.168.1.20 on port 1883.
1527169191: New client connected from 192.168.1.20 as b825d276-27a1-4692-9fe3-0c3db0f850ca (c1, k300, u'AX700').
1527169216: New connection from 192.168.1.30 on port 1883.
1527169216: New client connected from 192.168.1.30 as weewx_mqttc (c1, k60, u'AX700').
1527169216: Socket error on client weewx_mqttc, disconnecting.
1527169471: New connection from 192.168.1.30 on port 1883.

weewx- log
---------------------

pi@raspberrypi ~ $ tail -f /var/log/syslog

May 24 15:40:16 raspberrypi weewx[1196]:     ****      engine = engine_class(config_dict)
May 24 15:40:16 raspberrypi weewx[1196]:     ****    File "/home/weewx/bin/weewx/engine.py", line 74, in __init__
May 24 15:40:16 raspberrypi weewx[1196]:     ****      self.preLoadServices(config_dict)
May 24 15:40:16 raspberrypi weewx[1196]:     ****    File "/home/weewx/bin/weewx/engine.py", line 115, in preLoadServices
May 24 15:40:16 raspberrypi weewx[1196]:     ****      self.stn_info = weewx.station.StationInfo(self.console, **config_dict['Station'])
May 24 15:40:16 raspberrypi weewx[1196]:     ****    File "/home/weewx/bin/weewx/station.py", line 35, in __init__
May 24 15:40:16 raspberrypi weewx[1196]:     ****      self.altitude_vt = weewx.units.ValueTuple(float(altitude_t[0]), altitude_t[1], "group_altitude")
May 24 15:40:16 raspberrypi weewx[1196]:     ****  TypeError: float() argument must be a string or a number
May 24 15:40:16 raspberrypi weewx[1196]:     ****  Exiting.
May 24 15:41:35 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Thu May 24 15:42:05 2018 [try http://www.rsyslog.com/e/2007 ]
May 24 15:43:26 raspberrypi wpa_supplicant[461]: wlan0: WPA: Group rekeying completed with 6c:70:9f:e2:fe:56 [GTK=CCMP]
May 24 15:43:26 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Thu May 24 15:43:56 2018 [try http://www.rsyslog.com/e/2007 ]
May 24 15:44:21 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Thu May 24 15:44:51 2018 [try http://www.rsyslog.com/e/2007 ]
May 24 15:44:21 raspberrypi systemd[1]: Stopping LSB: weewx weather system...
May 24 15:44:21 raspberrypi weewx[1291]: Stopping weewx weather system: weewx not running....
May 24 15:44:21 raspberrypi systemd[1]: Stopped LSB: weewx weather system.
May 24 15:44:30 raspberrypi systemd[1]: Starting LSB: weewx weather system...
May 24 15:44:31 raspberrypi weewx[1354]: engine: Initializing weewx version 3.8.0
May 24 15:44:31 raspberrypi weewx[1354]: engine: Using Python 2.7.9 (default, Sep 17 2016, 20:26:04) #012[GCC 4.9.2]
May 24 15:44:31 raspberrypi weewx[1354]: engine: Platform Linux-4.1.7-v7+-armv7l-with-debian-8.0
May 24 15:44:31 raspberrypi weewx[1354]: engine: Locale is 'de_DE.UTF-8'
May 24 15:44:31 raspberrypi weewx[1354]: engine: pid file is /var/run/weewx.pid
May 24 15:44:31 raspberrypi weewx[1358]: engine: Using configuration file /home/weewx/weewx.conf
May 24 15:44:31 raspberrypi weewx[1358]: engine: debug is 1
May 24 15:44:31 raspberrypi weewx[1358]: engine: Initializing engine
May 24 15:44:31 raspberrypi weewx[1358]: engine: Loading station type wxMesh (user.wxMesh)
May 24 15:44:31 raspberrypi systemd[1]: Started LSB: weewx weather system.
May 24 15:44:31 raspberrypi weewx[1344]: Starting weewx weather system: weewx.
May 24 15:44:31 raspberrypi weewx[1358]: wxMesh: host is 192.168.1.30
May 24 15:44:31 raspberrypi weewx[1358]: wxMesh: topic is weewx//#
May 24 15:44:31 raspberrypi weewx[1358]: wxMesh: polling interval is 3.0
May 24 15:44:31 raspberrypi weewx[1358]: wxMesh: label map is {'HUMT': 'outTemp'}
May 24 15:44:31 raspberrypi weewx[1358]: engine: Caught unrecoverable exception in engine:
May 24 15:44:31 raspberrypi weewx[1358]:     ****  float() argument must be a string or a number
May 24 15:44:31 raspberrypi weewx[1358]:     ****  Traceback (most recent call last):
May 24 15:44:31 raspberrypi weewx[1358]:     ****    File "/home/weewx/bin/weewx/engine.py", line 865, in main
May 24 15:44:31 raspberrypi weewx[1358]:     ****      engine = engine_class(config_dict)
May 24 15:44:31 raspberrypi weewx[1358]:     ****    File "/home/weewx/bin/weewx/engine.py", line 74, in __init__
May 24 15:44:31 raspberrypi weewx[1358]:     ****      self.preLoadServices(config_dict)
May 24 15:44:31 raspberrypi weewx[1358]:     ****    File "/home/weewx/bin/weewx/engine.py", line 115, in preLoadServices
May 24 15:44:31 raspberrypi weewx[1358]:     ****      self.stn_info = weewx.station.StationInfo(self.console, **config_dict['Station'])
May 24 15:44:31 raspberrypi weewx[1358]:     ****    File "/home/weewx/bin/weewx/station.py", line 35, in __init__
May 24 15:44:31 raspberrypi weewx[1358]:     ****      self.altitude_vt = weewx.units.ValueTuple(float(altitude_t[0]), altitude_t[1], "group_altitude")
May 24 15:44:31 raspberrypi weewx[1358]:     ****  TypeError: float() argument must be a string or a number
May 24 15:44:31 raspberrypi weewx[1358]:     ****  Exiting.
May 24 15:44:42 raspberrypi wpa_supplicant[461]: wlan0: WPA: Group rekeying completed with 6c:70:9f:e2:fe:56 [GTK=CCMP]
May 24 15:47:27 raspberrypi rsyslogd-2007: action 'action 17' suspended, next retry is Thu May 24 15:47:57 2018 [try http://www.rsyslog.com/e/2007 ]
May 24 15:47:56 raspberrypi wpa_supplicant[461]: CTRL_IFACE: Detach monitor /tmp/libdhcpcd-wpa-933.1\x00 that cannot receive messages

mosquitto
------------------------
pi@raspberrypi ~ $ mosquitto_sub -d -u AX700 -P 84N7601 -t "weewx/#" -v
Client mosqsub|1398-raspberryp sending CONNECT
Client mosqsub|1398-raspberryp received CONNACK
Client mosqsub|1398-raspberryp sending SUBSCRIBE (Mid: 1, Topic: weewx/#, QoS: 0)
Client mosqsub|1398-raspberryp received SUBACK
Subscribed (mid: 1): 0
Client mosqsub|1398-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outTemp', ... (4 bytes))
weewx/outTemp 39.5
Client mosqsub|1398-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outTemp', ... (4 bytes))
weewx/outTemp 39.5

weewx.conf

-------------------
# WEEWX CONFIGURATION FILE
#
# Copyright (c) 2009-2015 Tom Keffer <tke...@gmail.com>
# See the file LICENSE.txt for your rights.

##############################################################################

# This section is for general configuration information.

# Set to 1 for extra debug info, otherwise comment it out or set to zero
debug = 1

# Root directory of the weewx data file hierarchy for this station
WEEWX_ROOT = /home/weewx

# How long to wait before timing out a socket (FTP, HTTP) connection
socket_timeout = 20

# Do not modify this. It is used when installing and updating weewx.
version = 3.8.0

##############################################################################

#   This section is for information about the station.
[Station]
   station_type = wxMesh

[wxMesh]
    host = 192.168.1.30

    username = AX700
    password = 84N7601
    topic = weewx/
    driver = user.wxMesh
    poll_interval = 3
    loop_on_init = true

    [[label_map]]
       HUMT = outTemp
      # RHUM = outHumidity

    #Description of the station location
    #location = Wetter Rheinbach Voreifel
    #
    #Latitude and longitude in decimal degrees
    #latitude = 50.3552
    #longitude = 6.5346

   
    # Altitude of the station, with unit it is in. This is downloaded from
    # from the station if the hardware supports it.

    # altitude = 270, meter    # Choose 'foot' or 'meter' for unit

   
    # Set to type of station hardware. There must be a corresponding stanza
    # in this file with a 'driver' parameter indicating the driver to be used.
    # station_type = Simulator
   
    # If you have a website, you may specify an URL
    #station_url = http://www.example.com
   
    # The start of the rain year (1=January; 10=October, etc.). This is
    # downloaded from the station if the hardware supports it.

    # rain_year_start = 1

   
    # Start of week (0=Monday, 6=Sunday)

    # week_start = 6

[Wysiwyg]

Hi Franz,

It's also pretty frustrating for us to that we cannot find :(, but let's try again !

I see 2 things that may help or at least rise some questions :

1/ 

in label map you have:   [[label_map]]
       HUMT = outTemp

if you use my version of driver, This mean the MQTT topic shall look like this:  weewx/HUMT

but in the mean time in mosquitto sub I see: 

Client mosqsub|1398-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outTemp', ... (4 bytes)) 

I'm not sure it cause the crash, because, as far as I remember, the driver shall just ignore the unknown topics (for example if you publish in weewx/spiderman, it shall be ignored by the driver) (it may need to be checked)

2/ I'm not sure at all, but I'm confused by the crash message you have: 

May 24 15:44:31 raspberrypi weewx[1358]:     ****    File "/home/weewx/bin/weewx/station.py", line 35, in __init__
May 24 15:44:31 raspberrypi weewx[1358]:     ****      self.altitude_vt = weewx.units.ValueTuple(float(altitude_t[0]), altitude_t[1], "group_altitude") 

The crash is not in the driver anymore ? I have no clue why, but it might be something completely different ? 

You may try to set back the station_type to simulator in weewx.conf to check if you still have the crash. If yes, something else than the driver is wrong.

Last comment: it may not be critical as it's on your LAN, but you disclosed your mqtt broker user & password...

Best regards,

Frederic.

[Franz]

Hello Frederic, hello Bill,
1. with the standard weewx.conf with simulator -> no error messages. Index.html is generated - perfect. weewx works ok.
2. Thanks for the reference to [[label_map]] and my password. I will change it and keep trying ......
Thanks for your time!
Franz

[Franz]

Hello Frederic, hello Bill,
very good news: It works great and my test values Outside Temperature and Humidity are represented by index.html.
I am happy!
It was due to the [[label_map]] incorrect mapping problem solved.
Thank you very much !!!!

Franz

Am Donnerstag, 24. Mai 2018 17:51:52 UTC+2 schrieb Wysiwyg:

[Wysiwyg]

Great news! \o/

It seems there is some margin to improve robustness of the driver ;).

[Bill Morrow]

I think you have inserted the wxMesh driver configuration in the middle of the Station configuration. Try changing this in weewx.conf

to

#   This section is for information about the station.
[Station]
   station_type = wxMesh

    #Description of the station location

    location = Wetter Rheinbach Voreifel
    #
    #Latitude and longitude in decimal degrees

    latitude = 50.3552

    longitude = 6.5346

   
    # Altitude of the station, with unit it is in. This is downloaded from
    # from the station if the hardware supports it.

    altitude = 270, meter    # Choose 'foot' or 'meter' for unit

   
    # Set to type of station hardware. There must be a corresponding stanza
    # in this file with a 'driver' parameter indicating the driver to be used.
    # station_type = Simulator
   
    # If you have a website, you may specify an URL
    #station_url = http://www.example.com
   
    # The start of the rain year (1=January; 10=October, etc.). This is
    # downloaded from the station if the hardware supports it.

    # rain_year_start = 1

... additional configuration of {Station]

# new section for configuring wxMesh driver
[wxMesh]
    host = 192.168.1.30

    username = AX700
    password = 84N7601
    topic = weewx/
    driver = user.wxMesh
    poll_interval = 3
    loop_on_init = true

    [[label_map]]
       HUMT = outTemp

      RHUM = outHumidity

et cetera

[Franz]

Hello Bill,
attached my working weewx.conf file.
My topics are: weewx / outTemp and weewx / outHumidity and do not need any assignment in the [[label_map]] (tip from Frederic).
Thanks again for the help !!!!
Franz
——

# WEEWX CONFIGURATION FILE
#
# Copyright (c) 2009-2015 Tom Keffer <tke...@gmail.com>
# See the file LICENSE.txt for your rights.

##############################################################################

# This section is for general configuration information.

# Set to 1 for extra debug info, otherwise comment it out or set to zero
debug = 1

# Root directory of the weewx data file hierarchy for this station
WEEWX_ROOT = /home/weewx

# How long to wait before timing out a socket (FTP, HTTP) connection
socket_timeout = 20

# Do not modify this. It is used when installing and updating weewx.
version = 3.8.0

##############################################################################

#   This section is for information about the station.

[Station]
    station_type = wxMesh

    #
    # Description of the station location

    location = Wetter Rheinbach Voreifel
   

    # Latitude and longitude in decimal degrees

    latitude = 50.3552
    longitude = 6.5346
   
    # Altitude of the station, with unit it is in. This is downloaded from
    # from the station if the hardware supports it.
    altitude = 270, meter    # Choose 'foot' or 'meter' for unit
   
    # Set to type of station hardware. There must be a corresponding stanza
    # in this file with a 'driver' parameter indicating the driver to be used.
    # station_type = Simulator
   
    # If you have a website, you may specify an URL
    #station_url = http://www.example.com
   
    # The start of the rain year (1=January; 10=October, etc.). This is
    # downloaded from the station if the hardware supports it.

    rain_year_start = 1
   

    # Start of week (0=Monday, 6=Sunday)
    week_start = 6

##############################################################################

[wxMesh]
   host = 192.168.1.30
   username = AX700
   password = 84N7601
   topic = weewx

   driver = user.wxMesh
   poll_interval = 3
   loop_on_init = true

#
#
#[[label_map]]

 
[Simulator]
    # This section is for the weewx weather station simulator
   
    # The time (in seconds) between LOOP packets.

    # loop_interval = 2.5

   
    # The simulator mode can be either 'simulator' or 'generator'.
    # Real-time simulator. Sleep between each LOOP packet.

    # mode = simulator

    # Generator.  Emit LOOP packets as fast as possible (useful for testing).
    #mode = generator
    # The start time. Format is YYYY-mm-ddTHH:MM. If not specified, the default
    # is to use the present time.
    #start = 2011-01-01T00:00
    # The driver to use:
    # driver = weewx.drivers.simulator

##############################################################################

#   This section is for uploading data to Internet sites

[Franz]

Hello,
Ask a question about wxMesh / MQTT / Rain Sensor.
My home rain sensor sends one pulse per spoon (3.4 ml) of rain over MQTT to weewx.
-
The ESP8266 ... sends ...
Adafruit_MQTT_Publish rain_pub = Adafruit_MQTT_Publish (& mqtt, "weewx / rain", 1);
...
rain_pub.publish (dregm); // value as char "0.034"

About 3 spoons correspond to 1 mm of rain per m2. My hardware does not provide hourly or daily rainfall. The skin (blue leather sofa) displays the following values ​​for rain on an impulse:
current weather
    Rain - rate 0.9 mm / h - wrong
Highs and lows of the day
   Today's rainfall: 0.3 mm - correct!
   Max.Rain rate: 1.4 mm / h 18:17:57 - wrong
One day later, in the "Rain (total hour)", 35 mm of rain is shown.

Question: How can I positively influence the automatic summation of rainfall?
Where can I change the internal formula for summing the Rain Rate, Max. Rain Rate?
or what value do I have to send per impulse?

Many thanks for the help!
Franz

PS. All other sensors (outTemp, outHumidity, barometer, windDir, windSpeed) are displayed correctly.

[mwall]

On Saturday, June 2, 2018 at 3:01:58 AM UTC-4, Franz wrote:

Hello,
Ask a question about wxMesh / MQTT / Rain Sensor.
My home rain sensor sends one pulse per spoon (3.4 ml) of rain over MQTT to weewx.
-
The ESP8266 ... sends ...
Adafruit_MQTT_Publish rain_pub = Adafruit_MQTT_Publish (& mqtt, "weewx / rain", 1);
...
rain_pub.publish (dregm); // value as char "0.034"

About 3 spoons correspond to 1 mm of rain per m2. My hardware does not provide hourly or daily rainfall. The skin (blue leather sofa) displays the following values ​​for rain on an impulse:
current weather
    Rain - rate 0.9 mm / h - wrong
Highs and lows of the day
   Today's rainfall: 0.3 mm - correct!
   Max.Rain rate: 1.4 mm / h 18:17:57 - wrong
One day later, in the "Rain (total hour)", 35 mm of rain is shown.

what values do you expect to see for rain rate?

the default algorithm is to use the rainfall in the past time period, normalized to a one hour interval.  the default time period is 15 minutes.

 

Question: How can I positively influence the automatic summation of rainfall?
Where can I change the internal formula for summing the Rain Rate, Max. Rain Rate?
or what value do I have to send per impulse?

rain rate is calculated by StdWXCalculate. 

to change the default time period used for normalization, do this:

[StdWXCalculate]

    rain_period = 900 # seconds

to change the algorithm, derive your own service from StdWXCalculate then install it as a weewx service *after* StdWXCalculate.

 

[Franz]

Hello,
Thank you for the tips. I will test.
Franz

[Wysiwyg]

Hi there!

It's a long time I didn't visit here.

I have a question regarding MQTT QoS.

Even if, in a normal setup (with correct home wifi connexion), this shall never be necessary, I wonder how to set QoS.

My concern is related to rain: it's the only measurement that should be sent one time and only one time to weewx (to avoid any risks of couting 2 times a value).

Today, I have QoS set to 0 for rain (sensor to broker, broker to Weewx)...only risk might be to miss some packet in case of poor transmission.

I wonder if we could improve this:

Sensor side: 

Adafruit mqtt does not provide QoS2 in the library: we could use QoS1

On WeeWX driver side, we currrently use QoS 0, but maybe we shall use QoS 2 ?

Could we associate QoS1 on pub + QoS2 on sub ? Does this work ?

If we use QoS 1 on weewx side, is there any risks to receive a rain packet more than 1 time?

[Bill Morrow]

I have no experience with MQTT quality of service. What you say sounds like a good idea. Otherwise, perhaps the weewx driver, or something like the weewx StdQC module could make sure only one rain measurement is sent.

[Cyril D]

Hello Frederic,

I currently use the FileParse driver (v 0.7) and am thinking of switching to MQTT.

Would it be possible to add the management of cumulative rain (interpret rain as a cumulative value) in your wxMesh.py driver?

Is this one the last one:

https://github.com/bonjour81/station_meteo/blob/master/weewx/driver/wxMesh.py ?

Thanks for the driver

[Wysiwyg]

Hi Cyril,

Currently my driver does not do any processing.

To simplify, I use mqtt topic name same as weewx variables.

So, any data published to topic "weewx/xxxxx" will be transfered to weewx to feed "xxxxx".

For example, any data published on topic "weewx/outTemp" will be given to weewx to feed "outTemp".

Then, regarding rain: as far as I know, weewx deals with rain measurement, but not with already made cumulative value.

If you cannot have rain samples data, It may be possible to add a suitable processing either in weewx driver or service... or externally with a script ? I'm not sure what is the easiest...

What is your data source that provide only cumulative ?

[Cyril D]

It is the PCR800 from Oregon.

The cumulative rain is handle by the FileParse driver, may be you can easily copy/paste it in your driver? You would need an extra parameter in the weewx.conf file.

[Cyril D]

Hi

I am trying the driver and I get these lines:

[code]LOOP:   2019-12-10 18:03:05 CET (1575997385) dateTime: 1575997385, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windBatteryStatus: 0.0, windDir: 247.5, windGust: 6.03974300024, windGustDir: 247.5, windSpeed: 5.14496625946
LOOP:   2019-12-10 18:03:20 CET (1575997400) dateTime: 1575997400, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windBatteryStatus: 0.0, windDir: 135.0, windGust: 2.46063603713, windGustDir: 135.0, windSpeed: 4.47388370388
LOOP:   2019-12-10 18:04:55 CET (1575997495) dateTime: 1575997495, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windBatteryStatus: 0.0, windDir: 180.0, windGust: 9.17146159295, windGustDir: 180.0, windSpeed: 7.15821392621
LOOP:   2019-12-10 18:05:10 CET (1575997510) dateTime: 1575997510, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windBatteryStatus: 0.0, windDir: 157.5, windGust: 4.02649533349, windGustDir: 157.5, windSpeed: 6.48713137062
LOOP:   2019-12-10 18:06:05 CET (1575997565) dateTime: 1575997565, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windBatteryStatus: 0.0, windDir: 247.5, windGust: 13.6453452968, windGustDir: 247.5, windSpeed: 5.36866044465
REC:    2019-12-10 18:05:00 CET (1575997500) dateTime: 1575997500.0, ET: None, interval: 5, maxSolarRad: 0.0, rainRate: 0.0, usUnits: 1, windBatteryStatus: 0.0, windDir: 187.370394522, windGust: 9.17146159295, windGustDir: 180.0, windrun: 0.466029552487, windSpeed: 5.59235462985
LOOP:   2019-12-10 18:06:39 CET (1575997599) dateTime: 1575997599, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windSpeed: 6.71082555582
LOOP:   2019-12-10 18:06:44 CET (1575997604) dateTime: 1575997604, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windBatteryStatus: 0.0, windGust: 6.26343718543
LOOP:   2019-12-10 18:06:49 CET (1575997609) dateTime: 1575997609, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windDir: 135.0, windGustDir: 135.0
LOOP:   2019-12-10 18:06:54 CET (1575997614) dateTime: 1575997614, maxSolarRad: 0.0, rainRate: 0, usUnits: 1, windBatteryStatus: 0.0, windDir: 337.5, windGust: 4.47388370388, windGustDir: 337.5, windSpeed: 6.26343718543[/code]

I am surprise, I don't have a Solar device. The rain sensor does not send data so often.

There was a problem you mentioned and it happened here: the values from the wind sensor are sent in different LOOPs

Any idea of what is wrong here?

[Wysiwyg]

Hi Cyril,

Which driver gives those logs ?

[Cyril D]

Hi,

Le mardi 10 décembre 2019 19:04:57 UTC+1, Wysiwyg a écrit :

Hi Cyril,

Which driver gives those logs ?

I think its yours:

https://raw.githubusercontent.com/bonjour81/station_meteo/master/weewx/driver/wxMesh.py

[Wysiwyg]

Hi,

Sorry for late answer.

I do not have anything like this in my logs.

I only have the "wxMesh" packet (= when it receive an MQTT data) and then when weewx add sdb record every 5 min.

Dec 13 10:41:11 meteo weewx[1013]: wxMesh: packet content: outTemp = 11.05

Dec 13 10:41:11 meteo weewx[1013]: wxMesh: packet content: Vbat = 3.52

Dec 13 10:41:11 meteo weewx[1013]: wxMesh: packet content: outHumidity = 91.60

Dec 13 10:41:11 meteo weewx[1013]: wxMesh: packet content: Isolar = 0.008

Dec 13 10:41:11 meteo weewx[1013]: wxMesh: packet content: UV = 0.44

Dec 13 10:41:11 meteo weewx[1013]: wxMesh: packet content: rain = 0.00

Dec 13 10:41:11 meteo weewx[1013]: wxMesh: packet content: Vsolar = 4.64

Dec 13 10:41:11 meteo weewx[1013]: manager: Added record 2019-12-13 10:40:00CET (1576230000) to database 'weewx.sdb'

Dec 13 10:41:11 meteo weewx[1013]: manager: Added record 2019-12-13 10:40:00CET (1576230000) to daily summary in 'weewx.sdb'

Dec 13 10:41:11 meteo weewx[1013]: reportengine: Running reports for latest time in the database.

Dec 13 10:41:11 meteo weewx[1013]: reportengine

This maxSolarRad is very weird too as this is not a weewx data I know (there is radiation).

Did you add the needed section in weewx.conf to configure the mqtt connexion ?

What are the topics you use to publish the data ?

[Wysiwyg]

This is what I have at the end of my weewx.conf :

# This section configures the mqtt driver

[wxMesh]

driver = user.wxMesh

# MQTT specifics

host = 192.168.1.181 #localhost

client = weewx_mqtt

username = replace_with_your_broker_user

password = replace_with_your_broker_password

topic = weewx

poll_interval = 2

[[label_map]]

barometer = barometer

pressure = pressure

altimeter = altimeter

inTemp = inTemp

outTemp = outTemp

inHumidity = inHumidity

outHumidity = outHumidity

windSpeed = windSpeed

windDir = windDir

windGust = windGust

windGustDir = windGustDir

rainRate = rainRate

rain = rain

dewpoint = dewpoint

windchill = windchill

heatindex = heatindex

ET = ET

radiation = radiation

UV = UV

extraTemp1 = extraTemp1

extraTemp2 = extraTemp2

extraTemp3 = extraTemp3

soilTemp1 = soilTemp1

soilTemp2 = soilTemp2

soilTemp3 = soilTemp3

soilTemp4 = soilTemp4

leafTemp1 = leafTemp1

leafTemp2 = leafTemp2

extraHumid1 = extraHumid1

extraHumid2 = extraHumid2

soilMoist1 = soilMoist1

soilMoist2 = soilMoist2

soilMoist3 = soilMoist3

soilMoist4 = soilMoist4

leafWet1 = leafWet1

leafWet2 = leafWet2

rxCheckPercent = rxCheckPercent

txBatteryStatus = txBatteryStatus

consBatteryVoltage = consBatteryVoltage

hail = hail

hailRate = hailRate

heatingTemp = heatingTemp

heatingVoltage = heatingVoltage

supplyVoltage = supplyVoltage

referenceVoltage = referenceVoltage

windBatteryStatus = windBatteryStatus

rainBatteryStatus = rainBatteryStatus

outTempBatteryStatus = outTempBatteryStatus

inTempBatteryStatus = inTempBatteryStatus

Note you have to set your username/password for mqtt connexion (if you have configured one).

if you don't use username/password for mqtt, maybe you need to comment some lines in the driver, I'm not sure.

if the connexion to broker is successful, you should see it in mosquitto log  (/var/log/mosquitto/mosquitto.log )

I publish my sensors on:

weewx/outTemp

weewx/outHumidity

etc....

The topic prefix "weewx" is defined in weewx.conf  (you can see below, just above "poll interval")

You can change the labels also, if you wish to use different mqtt topics.

For example, if you want to use topic  "weewx/out_T", you have to set   outTemp = out_T  in label map

But to keep it simple, I don't use this, I prefer keep same labels in mqtt topic than in weewx.

I hope that helps ?