fileparse polling interval

[jf42]

Hi All,

I have weewx running with the fileparse driver. In weewx.conf the polling interval ist set to 5 secs,

however, the records are added to the database with 1 min updates. I want it to have as fast

as possible. What I'm doing wrong?

Hints are welcome.

[jf42]

Addition:

I found my mistake: archive_interval was 60 in weewx.conf ... But:

Values below 60 sec cause a crash of the software.

Any ideas for that?

[Andrew Milner]

If you have set archive interval too small there is probably too little time to 

a) store archive record

b) generate HTML pages

c) generate NOAA reports

d) do any uploads

etc etc

Normally the loop data (obtained at small time intervals) is NOT written to the dstabase, or used to trigger reports etc - the loop data is just accumulated until the archive interval and then the archive record is written to the database.

So as to what are you doing wrong, my response would be to suggest you are attempting to do something with weewx which it is not designed to do ,,,,,

[Thomas Keffer]

In V3.x, the option archive_interval must divide evenly into an hour. Version 4.X will remove that restriction. See Issue #469.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/6c8e638d-7aa5-4ae9-9a82-bcad43dcd6e7%40googlegroups.com.

[jf42]

Thanks for the info.

Seems that with Version 4 archive intervals less than 60 secs would be possible.

I guess, my computer is fast enough to handle the neccessary tasks ...

[Thomas Keffer]

In theory, yes, you will be able to use an archive interval of less than a minute. But, some weather stations emit data only every 30-60 seconds or so and, even then, not all data.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/dcc3a091-dcd0-4934-9250-6e3f62cfa7ba%40googlegroups.com.

[jf42]

My station is a german "MWS 55V", which is read out by a Python program, writing the data to a tcp port. Another computer

collects the data, does a quality check, many other stuff, and writes a file. Weewx uses the file parser to get the data.

The station provides the data every 2 secs, the file resolution is about 5 secs. We use the data for a lidar instrument

(light radar) for weather monitoring during atmospheric measurements. Thus: the faster the better ;-)

That's for the background of my question.

[Thomas Keffer]

Sounds like an interesting project.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/beae49e6-f7d2-472a-8a6b-efcab5d9fc0d%40googlegroups.com.

[Bill Burton]

Hello,

So it sounds like you are using the WeeWX report generation or archive records creation to send data to the lidar?

Normally, the archive interval is configured to match reporting requirements that will work for a day/week/month/year which is why the default is 300 seconds/5 minutes. Also as some weather stations send data less frequently, this gives time to collect most if not all the data of each type. The archive interval is a different requirement than how fast a WeeWX driver receives data.

Since you are trying to get data to the lidar as fast as possible, have you considered using MQTT to transfer data? The message based publish-subscribe model may be more suitable for event based data transfer than polling for files. For instance:

• The program that writes the data file after quality control could instead publish it to an MQTT topic.
• WeeWX would be configured to use the MQTT Subscribe Driver and receive these messages as loop packets. At the appropriate time, WeeWX would write the archive record and generate the reports.
• You would implement another (Python) program that subscribes to the same MQTT topic and sends that information to the lidar as fast as data is received.
  

With this approach, the lidar would receive the data almost immediately after the QC machine publishes it without going through the WeeWX archival and report generation process. This way, the archive interval could be configured at the right amount to meet your reporting requirements would likely be much less frequent then how often data is received. However, if your requirement is that all data sent to the lidar must be logged, then you either have to log it elsewhere or use a short archive interval. However, the MQTT driver can be configured to publish archive data which should make it available more quickly than other methods that provide the archive data.

The page, How to setup your own MQTT Broker is a nice overview on setting up MQTT to work with WeeWX even if you don't use the Belchertown skin. However, the MQTT extension for WeeWX mentioned in that post is used to publish loop packets after WeeWX receives them, it cannot act as a driver to subscribe to an MQTT topic for receiving data.

Hope this helps,

-Bill

[jf42]

Hi Bill,

in fact I have such solution. The computer doing the quality check dumps the checked data in highest resolution to the disk

and additionally publishes it to a MQTT server, where other clients can access it.

I have a Vantage Pro2 station running with weewx at home and the idea was, to use weewx for nice looking visualization

of the weather data, current as well as historical, for the lidar station too.

Anyway, if time allows I will upgrade to weewx v4 and see how fast it will be.

Thanks a lot,

Jens