Manually add min and max data

[Thomas Carlin]

Good evening!

I have extended my weewx instance with several sensors build on the ESP8266 platform with help from many in this community.  I have built a driver based on the custom service documentation provided by TK and crew.  This has been working wonderfully, and I am really enjoying the extra temperature sensors, as well as things like electricity monitors, irrigation system monitors etc.  The only issue that I have with this system is that I am only adding a single piece of data into the archive each time the archive interval is reached.  I would like to know if there is a way to as part of the additional service, add into the records the minimum and maximum values recorded by the sensors, and the dates and times.  Obviously, the sensors will have to record and transmit this data on the archive interval, and the driver will need to interpret this data.  I just need to know how to insert the min and max data into the database.  Is this possible with the current architecture?

For Example, the contents of the an archive for outTemp and one of my custom sensors.  Notice the precision of the time of the outTemp, and the general time of the l2_volt, all on the 5 minute archive interval of my system.  

sqlite> select * from archive_day_outTemp order by dateTime desc limit 1;
dateTime  |min             |mintime   |max |maxtime   |sum             |count|wsum            |sumtime
1536040800|55.1999999999999|1536065400|87.5|1536097806|18741.1237726958|263  |5622337.13180874|78900

sqlite> select * from archive_day_l2_volt order by dateTime desc limit 10;
dateTime  |min   |mintime   |max   |maxtime   |sum     |count|wsum     |sumtime
1536040800|115.15|1536117900|125.74|1536079200|32513.28|263  |9753984.0|78900

Hopefully this request makes sense, and thank you for your help!

http://carlincomputing.duckdns.org/weewx/index.html

[Andrew Milner]

If you bind your service to loop rather than record then

weewx will build an archive record at the archive interval, with the value being the average value for the period

if you specify use loop for hilo in weewx.conf then weewx will populate the max and min from the loop data and the archive record will contain the average value for the archive period

if you do not specify to use loop hilo then min and max will be the highest archived value for the day or the lowest archived value for the day

does this answer your question though??

oh yes, the min and max times will always be on archive boundaries unless use loop for hilow is specified since the source of the data is of course the archive record.

so the way to specify the times of the max and min is for the data to be in loop packets rather than rec packets

[Thomas Keffer]

Andrew pretty much has the right idea.

The whole value of weewx is in its archiving abilities. If your driver(s) don't emit LOOP packets, then your missing out on half the reason to use the program! 

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeff A. D.]

If I may chime in on the subject, but there are various reasons I don't have a full time connection to my wx station (Davis VP), and therefore rely on the archive records to download from the data logger periodically.  Even using a fairly long archive interval of 30 minutes, the archive records are (would be) more than adequate for me as they do contain the highs and lows for that interval.  However, for whatever reason, even though as I understand it, weewx, and I think wview, where originally started for the Davis stations, they neglected to record these values in the data base.

So maybe in the view of some, I'm missing out on half the reason to use the program, as I use it only for archive purposes and not to publish to the web in real time; but in my view the program is lacking in an area where it could otherwise be complete.

Please don't misunderstand, I do love weewx and am thankful for it, as it seems to be the best alternative for Linux.  I just wish it didn't have this one shortcoming.

[Thomas Keffer]

Thanks for your comments, Jeff, but you've got me curious. If you don't use WeeWX for its ability to accumulate LOOP packets and make records out of them, and you don't publish to the web, what do you use it for? Internal LAN publishing? Something else?

-tk

[Thomas Carlin]

Thank you for your replies. 

A couple points of clarification: The main datasource for my station is a Vantage Pro2, and it is using weewx as designed, LOOP packets, etc.  I have added a second data source as described here http://www.weewx.com/docs/customizing.htm#Adding_2nd_source, which specifically discusses adding data to the record dictionary.

Weewx is currently configured to get the hilo data from the loop packets as described, and this is working for the Pro2.  My question is specifically about the additional custom service that has been added to the data_services section as described in the doc linked above.

Does that clarify anything?

Based on the comments it sounds like there is no way to add the hilo data manually as part of the additional service, correct?

Is there a way to have my custom service run as part of a loop, rather than the record without reinventing the wheel, or disrupting the Pro2 and its data?

Thomas

[Jeff A. D.]

Hi Thomas,

I use it mainly for the NOAA Climatological monthly and yearly summaries, and also compile yearly and all time highs and lows from the database, and appreciate having the database available for any other kind of record manipulation or graphing.  As i mentioned, I don't keep a full time connection to a computer, as I have never had a computer I had on for 24/7, or necessarily kept my console near the computer, and I still use an original VP (not VP2) which seems to sometimes cause interference problems if I do keep it near and attached to the computer, have never experimented with a website or a Raspberry Pi.

Anyway, weeWX has works great downloading my archive data from the logger to the weeWX database, other than the loss of granularity by not reading the available hi/lo data for the archive interval. If that extra bit of data could also be taken from the logged archive data, I would see no reason I would need to accumulate LOOP packet data as I'm not publishing in real time, just keeping old-fashioned (more or less) local data records. (Each month I also check the highs and lows as recorded on the console to adjust the weeWX reports if needed, and save them elsewhere.)

It also seems to me that the ability to record the archive hi/lo s to the db would also be a plus for those who do rely on the LOOP data, but have unexpected computer downtime or dropped connections, and need to download the archived data from the logger to fill in the gaps.

I used WeatherLink exactly the same way before I dumped Windows all together; for periodic downloads, reports, and graphing; except it would record the archive interval hi/lo s.

I hope that makes sense, please let me know if I can clarify something, or if I'm just using weeWX wrong, or misunderstanding something, or if maybe there's something else I could be doing (I'm not going to pretend I'm an expert on weeWX or computers).

Thanks for your work!  WeeWX does most everything I want (and more), except just that one little thing.  I also hope I didn't hijack the other Thomas' thread, but if I understand it it sounds similar to what I wanted.

[Jeff A. D.]

BTW, I missed Thomas Carlin's last reply before posting.  My apologies for getting his thread a little sidetracked.

[gjr80]

Hi,

It all comes down to how your second source service (I'll refer to it from now on as the service) is providing data. Your VP2 will be providing loop data every 2 odd seconds, if your service adds its data to the loop packet then you should have a similar time resolution for data from your service as for your VP2 data. This is what Andrew meant by binding your service to loop. Your service may be augmenting loop packets but only doing it every so often, say every 60 seconds, in this case the best time resolution you will see for data from your service is 60 seconds. The other way your service may be working is to bind to the archive record, in this case your service is adding data to the archive record and then, depending on your service, it is quite likely that the time resolution for data from your service will be your archive period (lets say 5 minutes). As long as (1) your service is bound to loop (2) you have loop_hilo = True set in weewx.conf and (3) your service is included in the service list at the bottom of weewx.conf before StdArchive then loop data from your service will be used to come up with highs and lows. The thing to note though is that the best time resolution you can get for data from your service is the shortest period between it providing data to WeeWX.

Its a complex arrangement with many variables, the best way for us to let you know exactly what you should expect would be to know what station you are using (provided - VP2), some info on how it is configured (provided - loop_hilo = True) and how is your service coded (not yet provided). If you are prepared to show us the code your are using for your service we can then give you some definitive advice.

Gary

[Thomas Carlin]

Hi Gary,

All that makes sense, thank you.  My service is coded directly from the documentation linked above, with the addition of a function to "somehow downloads the data", and I have included it below.  One thing to note, I am not a programmer, I dabble as a hobby, so if there is something done poorly or improperly, I am open to suggestions, recomendations and things that can be done better.  I have also included the relevant bit of code from weewx.config about how I am referencing my service, and the configuration to help explain things.  Currently my service is adding data to the archive record, and in order to augment the loop packets, I would need to change the structure to speed up the process of getting data.  Currently, with 4 separate sensors, it takes a measurable amount of time to get the data from all of them, and I would want to convert to a queue structure, maybe a local MQTT instance.  (I have already toyed with this idea, and this would give me a good excuse.) Let me know if there is anything else that I can answer or provide that would help.  2 second resolution would be great, but I would be happy with 30 or 60 seconds.  Currently, my archive interval is set to 5 minutes.

THank you,

Custom Service: (esp8266.py)

#!/usr/bin/env python                                                                                                                                  

import weewx
import syslog
from weewx.engine import StdService
import schemas.wview
import requests

class add_esp_records(StdService):

    def __init__(self, engine, config_dict):

# Initialize my superclass first:   
       super(add_esp_records, self).__init__(engine, config_dict)

        # Bind to any new archive record events:                                                                                                       
        self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_packet)

        try:
            # Dig the needed options out of the configuration dictionary.                                                                              
            # If a critical option is missing, an exception will be raised and                                                                         
            # the alarm will not be set.                                                                                                               
            self.devices    = config_dict['ESP8266']

            #print "Devices: "                                                                                                                         
            #print self.devices                                                                                                                        


            syslog.syslog(syslog.LOG_INFO, "ESP8266: Setup for devices")

        except KeyError as e:
            syslog.syslog(syslog.LOG_INFO, "ESP8266: Not configured.  Missing parameter: %s" % e)


    def new_archive_packet(self, event):

        for device, value in self.devices.iteritems():
            self.url = device
            #print "URL: "                                                                                                                             
            #print self.url                                                                                                                            
            self.sensors = value
            #print "Sensors: "                                                                                                                         
            #print self.sensors                                                                                                                        

            #Pass the URL and the expected values to the read function.                                                                                
            esp_records = read_esp_json(self.url);

            if esp_records != 0:

                #print self.sensors['mac']                                                                                                             
                #print esp_records['UID']                                                                                                              

                if self.sensors['mac'] == esp_records['UID']:
                    #print "Sensor ID matches, continue"                                                                                               

                    for sensor in self.sensors:
                        if sensor != 'mac':
                            print "DB Entry: " + self.sensors[sensor]
                            try:
                                value = esp_records[sensor]

                            except KeyError:
                                 value = None

                            print "Value: " + str(value)
                            event.record[self.sensors[sensor]] = value

#Reads the data from the ESP and pass it back to be added to the database.
def read_esp_json(url):

    #Fetch the data, read, and do some basic sanitization on the output.                                                                               
    try:
        response = requests.get(url, timeout=2)
    except (requests.exceptions.RequestException) as err:
        syslog.syslog(syslog.LOG_INFO, "ESP8266: Unable to access " + url + ": %s" %err)
        return 0

    data = response.json()

    print data

#    print data['KWh']                                                                                                                                 

    result = {}
    for key in data:
#        print key                                                                                                                                     
        if key == 'UID':
            result[key] = data[key]
        else:
            if data[key] == 'nan':
                result[key] = None
            else:
                result[key] = float(data[key])

    return result

#Add extra fields to the schema.
schema_with_esp = schemas.wview.schema + [('DoorOpen1', 'REAL'), ('DoorClosed1', 'REAL'), ('DoorOpen2', 'REAL'), ('DoorClosed2', 'REAL'), ('IrrigationFlow', 'REAL'), ('IrrigationPressure', 'REAL'),('esp1Signal', 'REAL'), ('esp2Signal', 'REAL'),  ('esp3Signal', 'REAL'), ('esp3LIPOVoltage', 'REAL'), ('esp3LIPOCurrent', 'REAL'), ('esp3SolarVoltage', 'REAL'), ('esp3SolarCurrent', 'REAL'), ('esp3OutputVoltage', 'REAL'), ('esp3OutputCurrent', 'REAL'), ('extraHumid3', 'REAL'), ('esp4Signal', 'REAL'), ('esp4LIPOVoltage', 'REAL'), ('l1_watt', 'REAl'), ('l1_volt', 'REAl'), ('l1_amp', 'REAl'), ('l2_watt', 'REAl'), ('l2_volt', 'REAl'), ('l2_amp', 'REAl'), ('kwh', 'REAL')]

Weewx.conf

##############################################################################                                                                         

#   This section configures the internal weewx engine.                                                                                                 

[Engine]

    [[Services]]
        # This section specifies the services that should be run. They are                                                                             
        # grouped by type, and the order of services within each group                                                                                 
        # determines the order in which the services will be run.                                                                                      
        prep_services = weewx.engine.StdTimeSynch
        data_services = user.esp8266.add_esp_records,
        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate, user.cmon.Computer\
Monitor
        archive_services = weewx.engine.StdArchive
        restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.St\
dWOW, weewx.restx.StdAWEKAS
        report_services = weewx.engine.StdPrint, weewx.engine.StdReport

##############################################################################                                                                         

#   This section is for information about the additional ESP8266 Sensors that are in use.                                                              

[ESP8266]
    [[http://192.168.2.104]]
        mac = 5C:CF:7F:02:0C:17
        Temp = extraTemp1
        Humid = extraHumid1
        doorOpen = DoorOpen1
        doorClosed = DoorClosed1
        Signal = esp1Signal

    [[http://192.168.2.107]]
        mac = 5C:CF:7F:07:5B:90
        Temp = extraTemp2
        Humid = extraHumid2
        doorOpen = DoorOpen2
        doorClosed = DoorClosed2
        Signal = esp2Signal

    [[http://192.168.2.105]]
        mac = F8:F0:05:E7:A2:7B
        Flow = IrrigationFlow
        Pressure = IrrigationPressure
        Signal = esp3Signal
        LIPO_Voltage = esp3LIPOVoltage
        LIPO_Current = esp3LIPOCurrent
        Solar_Voltage = esp3SolarVoltage
        Solar_Current = esp3SolarCurrent
        Output_Voltage = esp3OutputVoltage
        Output_Current = esp3OutputCurrent

    [[http://192.168.2.112]]
        mac = F8:F0:05:E7:A2:89
        Signal = esp4Signal
        L1_apparentPower = l1_watt
        L1_volt = l1_volt
        L1_amp = l1_amp
        L2_apparentPower = l2_watt
        L2_volt = l2_volt
        L2_amp = l2_amp
        KWh = kwh

##############################################################################                                                                         

#  Options for extension 'cmon'                                                                                                                        
[ComputerMonitor]
    data_binding = cmon_binding

[gjr80]

Hi Thomas,

If it is any consolation I am not a programmer either, I get by Googling, trying things out, seeing how existing things work etc and am always learning new and better ways to code.

Your service is bound to the NEW_ARCHIVE_RECORD event so each time a new archive record appears your service is called and it obtains the various values from your sensor and adds them to the archive record. Since your service is in the date_services list it is called before the process_services that convert, calibrate, quality check and ultimately save the data to the database. That is all good but as you have found with only one data value being recorded per sensor per archive period the time resolution of your lows/highs for your extra sensors is your archive interval. If you want to have better resolution then you need to get more frequent values into WeeWX. If you changed your service to bind to NEW_LOOP_PACKET you could then pick up values from your extra sensors at least every 2.5 odd seconds (your station loop interval) or longer if you wanted (say every 10 or 30 seconds).

The changes to your code to do this are quite trivial; however, the risk that you run with your existing code is that you may find that if a sensor is not available, your network is down or something else prevents one of more sensors being read your service may end up blocking WeeWX. When bound to NEW_ARCHIVE_RECORD you are only reading your sensors once per archive period (say 5 minutes), so if there is a delay for a few seconds while a sensor is read or it times out it likely does not matter too much. Though if you now work in a NEW_LOOP_PACKET context, where loop packets come in every 2.5 seconds, that 1 or 2 second delay can cause WeeWX to miss loop packets. This is why it is often not good practice to have WeeWX services that directly interrogate an internet address for data, it just needs a slow internet connection or a server that has disappeared and it can cause WeeWX quite a few problems. There are ways to deal with this, here are a few:

1. You can make sure that any reads you do over the network have a timeout such that if they do fail the timeout kicks in and you don't miss the next loop packet. Remember though that you are reading devices sequentially so your timeouts would be cumulative.
2. Use something like the fileparse driver (but you would need to adapt it to be a service - easily done) whereby it reads pre-formatted data from a text file. If the text file is on your network or WeeWX machine you should be safe. You then run a separate program (could be based on your existing service) that polls your sensors every so often and writes the sensor data to the file.
3. Write a service that uses a separate thread to obtain the data from your sensors and pass it back to your WeeWX service via a python queue where your service adds it to the loop packet when/if data is available. This has the advantage of interrogating your sensors in a separate thread so if there are any delays/problems reading sensors it does not bring down WeeWX. A number of drivers operate with threads as do a number of services. The forecasting service uses a number of threads as does the netatmo driver; basically any service or driver that obtains data from the internet probably uses threads.
   

A threaded solution is probably the more elegant but it is also the most complex and difficult to troubleshoot, maybe you might want to consider using the fileparse approach, it would not be to difficult and would keep WeeWX from being brought down by misbehaving networks or sensors.

I am sure others will have other views/suggestions.

Gary

[Thomas Carlin]

Perfect Thank you Gary!  That gets me what I need.  Now I just need to reprogram all my sensors and my driver, or create another service to collect and queue data.  I'm leaning towards MQTT, for its simplicity, speed, and scalability.  This sounds more like a winter project to me!

A couple quick questions before we wrap this up:

I have seen rumblings about MQTT drivers in the forums, but some of them look like they are so Weewx provides MQTT data for something else.  Is there anything that you are aware of that could act as a second service with MQTT being the data source?  Has anything been blessed by the weewx development crew? I would hate to reinvent the wheel if someone else figured it out for me!

Second, just for my piece of mind, did i miss the NEW_LOOP_PACKET in the documentation, or was it not mentioned?  If it is the latter, I think this would be a valuable addition.

Thank you all for your help, excellent as always!

Thomas

[gjr80]

On Wednesday, 12 September 2018 13:36:00 UTC+10, Thomas Carlin wrote:

Perfect Thank you Gary!  That gets me what I need.  Now I just need to reprogram all my sensors and my driver, or create another service to collect and queue data.  I'm leaning towards MQTT, for its simplicity, speed, and scalability.  This sounds more like a winter project to me!

That's usually when the best work is done!

 

A couple quick questions before we wrap this up:

I have seen rumblings about MQTT drivers in the forums, but some of them look like they are so Weewx provides MQTT data for something else.  Is there anything that you are aware of that could act as a second service with MQTT being the data source?  Has anything been blessed by the weewx development crew? I would hate to reinvent the wheel if someone else figured it out for me!

There is the weewxMQTT driver that is included under drivers in the wiki. Have never used it but it allows an MQTT broker to be a data source for WeeWX. Can't say it is endorsed any more or less than any other driver in the wiki but I understand a number of folks are using it. It is a driver and not a service so if you have an existing station whose data you wish to augment you will need to adapt it to be a service. A driver and a data service that both obtain data from a given source are quite the similar, usually they use exactly the same process to get the data, where the difference starts is that the driver has a method genLoopPackets() that just sits there and spits out loop packets and WeeWX and its services then do the rest. On the other hand a data service waits for loop packets and/or archive records and augments the packet/record with data, it may continue to read data from its source but it will not do anything more with WeeWX until (potentially) the next loop packet/archive record comes along. In this manner you can have WeeWX aggregate data from many sources; one driver and as many services as you like.

 

Second, just for my piece of mind, did i miss the NEW_LOOP_PACKET in the documentation, or was it not mentioned?  If it is the latter, I think this would be a valuable addition.

Now you had me looking. The best I could find was the comments in bin/weewx/__init__.py, though probably not easily found. There is the Notes for Developers document, though unfortunately it does not cover the WeeWX events. It probably should but as is commonly the case there are too many things to do and not enough people or time to do them.

Gary

[Thomas Carlin]

Hi Gary, 

I've started working on this service, and I have run into an issue.  

A quick refresher so you don't have to read through the whole thread:  I am writing an MQTT service to add data to the loop packets.  You pointed me in the right direction with the NEW_LOOP_PACKET, and I'm 99% of the way there.  My service is pulling data with the loop packets, processing and normalizing it, and I'm now to the point that I am ready to stuff it into the loop packet.  

Since I am modifying an existing service that I wrote, I am trying to use the line event.record[key] = value. that I found in the documentation.  When I try this, I get the following trackback.  I'm sure that this is because event.record is for archive packets, and I'm now working with loop packets.

Traceback (most recent call last):

  File "/usr/bin/weewxd", line 64, in <module>

    weewx.engine.main(options, args)

  File "/usr/share/weewx/weewx/engine.py", line 877, in main

    engine.run()

  File "/usr/share/weewx/weewx/engine.py", line 191, in run

    self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))

  File "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent

    callback(event)

  File "/usr/share/weewx/user/esp8266.py", line 105, in new_loop_packet

    event.record[self.subscriptions[topic]] = value

AttributeError: 'Event' object has no attribute 'record'

Any thoughts on what I can do to correct this?  I have verified that the variable self.subscriptions[topic] provides the expected value of extraTemp1, and value contains 74.30.

Also, related to this discussion, I would like to provide a simplified version of code, and an explanatory blurb to be added to the documentation to hopefully help others in a similar situation.  Is that something that the development team would be open to?

[gjr80]

Hi Thomas,

It sounds like you should be using event.packet rather than event.record. Archive records are found in event.record, loop packets are found in event.packet. It's a subtle but important distinction. You have probably moved on from the code you posted above, and I probably should have mentioned it earlier, but I would avoid method names such as def new_archive_packet(). There is nothing wrong with using that name per se, but since we have loop packets and archive records it does tend to create a little confusion, something like  def new_archive_record() and def new_loop_packet() would be better.

Gary

[Thomas Carlin]

Perfect!  thank you Gary!  I knew it had to be something simple like that.  Everything is up and running now, and I'll just need to catch the necessary exceptions, and optimize the collection as much as possible.  Is there a target time I should shoot for, or just try to make it as fast as possible?

[Thomas Carlin]

Re: the new_archive_packet() recommendation, I didn't think about it, but I agree.  It is posted written in the "second data source" documentation with the function new_archive_packet().

On Thursday, October 4, 2018 at 7:52:37 PM UTC-6, gjr80 wrote:

[gjr80]

On Friday, 5 October 2018 11:58:44 UTC+10, Thomas Carlin wrote:

Perfect!  thank you Gary!  I knew it had to be something simple like that.  Everything is up and running now, and I'll just need to catch the necessary exceptions, and optimize the collection as much as possible.  Is there a target time I should shoot for, or just try to make it as fast as possible?

Not quite sure what you mean by a target time. If you are still using the Vantage as the heart of your system then the Vantage loop interval will set the rate at which NEW_LOOP_PACKETS events occur, ie every 2.5 odd seconds, so that will be the fastest rate at which WeeWX will receive data. If you are now subscribing to an MQTT topic to get your data that you augment the Vantage loop packets with then I guess how your service works with MQTT will depend on how you are using MQTT; are you subscribing to a topic, caching values etc. Perhaps if you show us your service as it stands now.

Gary

[gjr80]

Um, ah yes. Will mention that one to Tom.

Gary

[Thomas Carlin]

I am augmenting the VP2 data, and looking for an idea what the maximum time my code could run to without interrupting or missing data from my VP2.  Right now, on my 'production' system I expect additional code to run about 3/4 second, with most of it spent connecting to MQTT, but I know that I can optimize that by moving a couple things to the init function rather than being part of the loop.  Do you thing that 3/4 or even 1 second will cause issues?  I'm planning on setting my sensors up to report data every 10 seconds, so every 4th cycle will be slightly longer.

[gjr80]

OK, understand. Ideally you want all of the processing of a loop packet (ie all of the services that do something with or to the loop packet, eg StdCalibrate, StdQC, StdArchive, your service etc - all those with a binding to NEW_LOOP_PACKET) to execute within a loop period. The loop period varies with station, in your case for your Vantage it is 2.5 odd seconds. If all of those services take longer than the loop period (remember they are executed sequentially) then you risk missing loop packets or other bad things occurring. The problem you have when you need to poll or otherwise query an external server is that issues beyond your control (eg network delays, latency, timeouts etc) may mean that your service blocks for some extended period while it obtains the external data or the request times out. This could mean your overall execution time for all services pushes out beyond the next loop packet arriving and it is missed.

You may recall in an earlier post I presented three options, the final one was a service running in its own thread. There is actually two parts to such a service, firstly some code is run in a separate thread that handles the external server/system and obtains the necessary data. Once this thread has some data it passes the data to a more traditional WeeWX type service (running in the same thread as WeeWX) via a python queue. When this traditional type service is run it takes the data it has received from the queue (if in fact it has received any) and performs its operation on the loop packet. If there is no data it may do nothing. This way any delays in dealing with the external server are confined the the non-WeeWX thread and the portion of your service running in the WeeWX thread can execute very quickly and not block. The forecast extension works in this way (it queries WU, NWS etc), as does the forecast aspects of my realtime SteelSeries Gauges extension (it queries WU and Darksky) as do a number of WeeWX drivers (they query the station).

If it were me I would get your service running now as you have it. Do what you can to make sure it does not block WeeWX ie make sure you can handle the MQTT broker being unavailable, keep any timeouts to fairly low values compared to the loop interval. If you don't overload your system with a multitude of additional services you should have no problems. A longer term goal can be to rework your service to be a threaded service. You can take your time to build your python skills, research and understand how a threaded service works and ultimately write your own. It is not that hard to do but they can be a challenge to debug at times.

Hope that helps.

Gary