Weatherdata (rain) from own Weatherstation (Arduino)

[Johann Schneider]

Hello everybody!

I have a little problem. Sorry for my very bad english. I come from Germany and my English is not the best. My experience with weewx is also very limited. I built a weather station 3 years ago. It consists of an Arduino Mega and several sensors.

Among other things, a rain sensor that reports 0.16mm of rain per seesaw

The values ​​are pushed to iobroker (similar to openhab) every 15 seconds. Here is an excerpt from my Arduino codes:

/*

Auswertung Regenmengensensor mit Regenwippe

- Counter kann durch reed-Kontakt oder Hallsensor erfolgen

- i2c eeprom AT24C256

- i2c Adresse EEPROM (default: 0x50)

*/

#include <Wire.h>

//#include <I2C_EEPROM.h>

#//include <avr/eeprom.h>

/********************************************/

/*                PIN-Section               */

/*      Hier Sensor-Pins definieren         */

/********************************************/

//#define RAIN_COUNT_PIN 7

#define RAIN_COUNT_PIN 30

/********************************************/

/*           Variablen-Section              */

/********************************************/

long lastSecond; // Original - The millis counter to see when a second rolls by

unsigned int minutesSinceLastReset; //Used to reset variables after 24 hours. Imp should tell us when it's midnight, this is backup.

byte secondsRain; //When it hits 60, increase the current minute

byte minutesRain; //Keeps track of where we are in various arrays of data

volatile float rainHour[60]; //60 floating numbers to keep track of 60 minutes of rain

volatile unsigned long raintime, rainlast, raininterval, rain;

volatile unsigned long last_micros1;

long debouncing_time1 = 3; // in millis -> eleminiert evtl REED-Kontakt-Prellen

boolean RainTimeReset = false;

//struct Configuration

//{

//  float regenTag;

//  float regenStunde;

//}eepconfig EEMEM;

//AT24C256<0x50> eep; // Das EEProm auf der china üblichen RTC3231 

/********************************************/

/*             Interrupt-Section            */

/********************************************/

void rainIRQ(unsigned long now){

  if((long)(micros() - last_micros1) >= debouncing_time1 * 1000) {

    rainToday += 0.1618; // Each dump is 0.1618mm of water

    rainHour[minutesRain] += 0.1618; // Increase this minute's amount of rain

  last_micros1 = micros();

  }

}

/********************************************/

/*              Setup-Section               */

/********************************************/

void setupRain(unsigned long now){

  Serial.begin(115200);

  Wire.begin();

#ifdef DEBUG_WS

  Serial.println("rain gauge online!");

#endif

// if(eep.ready()){

//  eep.get(eepconfig.regenTag,rainToday);

//#ifdef DEBUG_WS

//  Serial.print("EEPROM rainToday : ");

//  Serial.println(rainToday);

//#endif

//  }

//  else

//  {

  midnightReset(now); //Reset rain totals

  secondsRain = 0;

  lastSecond = now;

//  }

  rainLastHour = 0;

  pinMode(RAIN_COUNT_PIN, INPUT_PULLUP);

  attachInterrupt(RAIN_COUNT_PIN, rainIRQ, RISING);

  reportWeather();

}

/********************************************/

/*               Loop-Section               */

/********************************************/

void loopRain(unsigned long now){

  //Keep track of which minute it is

  if(now - lastSecond >= 1000)

  {

    lastSecond += 1000;

    reportWeather(); //Print the current readings. Takes 172ms.

    //If we roll over 60 seconds then update the arrays for rain

    if(++secondsRain > 59)

    {

      secondsRain = 0;

      if(++minutesRain > 59) minutesRain = 0;

      

      rainHour[minutesRain] = 0; //Zero out this minute's rainfall amount

      minutesSinceLastReset++; //It's been another minute since last night's midnight reset

    }

  }

    if((TimeStunde == 0) && (! RainTimeReset))

    {

      midnightReset(now); //Reset a bunch of variables like rain and daily total rain

      RainTimeReset = true;

    }

    else if((TimeStunde != 0) && (RainTimeReset))

    {

      RainTimeReset = false;

    }

  //If we go for more than 24 hours without a midnight reset then force a reset

  //24 hours * 60 mins/hr = 1,440 minutes + 10 extra minutes. We hope that Imp is doing it.

  if(minutesSinceLastReset > (1440 + 10))

  {

    midnightReset(now); //Reset a bunch of variables like rain and daily total rain

  }

}

//When the imp tells us it's midnight, reset the total amount of rain and gusts

void midnightReset(unsigned long now)

{

  rainToday = 0; //Reset daily amount of rain

//  if(eep.ready()){

//      eep.put(eepconfig.regenTag,rainToday);

//  }

  minutesRain = 0; //Reset minute tracker

  secondsRain = 0;

  lastSecond = now; //Reset variable used to track minutes

  minutesSinceLastReset = 0; //Zero out the backup midnight reset variable

}

void calcWeather()

{

  rainLastHour = 0;

  for(int i = 0 ; i < 60 ; i++)

    rainLastHour += rainHour[i];

}

void reportWeather()

{

  calcWeather(); //Go calc all the various sensors

#ifdef INFO_WS

  Serial.print("rainLastHour : ");

  Serial.print(rainLastHour, 2);

  Serial.print(" , rainToday : ");

  Serial.println(rainToday, 2);

#endif

//  if(eep.ready()){

//  eep.put(eepconfig.regenStunde,rainLastHour); //Nicht nutzen

//  eep.put(eepconfig.regenTag,rainToday);

//  }

}

The rain data are given every 15 seconds.

I get the values ​​for rain of the last hour and the total amount since midnight.

Now I have tried to collect the data using fileparse.

Iobroker writes the data to a file every minute.

See example:

outTemp=43.8

outHumidity=74.7

pressure=29.896

rain=0.00

windSpeed=0.8

Fileparse evaluates the data and pushes it onto weewx.

Unfortunately, the data is summed up every few minutes and the values ​​no longer fit.

For example, I have 0.15mm of rain in the last hour, then the value after each loop adds up until the hour is over.

/*

Auswertung Regenmengensensor mit Regenwippe

- Counter kann durch reed-Kontakt oder Hallsensor erfolgen

- i2c eeprom AT24C256

- i2c Adresse EEPROM (default: 0x50)

*/

#include <Wire.h>

//#include <I2C_EEPROM.h>

#//include <avr/eeprom.h>

/********************************************/

/*                PIN-Section               */

/*      Hier Sensor-Pins definieren         */

/********************************************/

//#define RAIN_COUNT_PIN 7

#define RAIN_COUNT_PIN 30

/********************************************/

/*           Variablen-Section              */

/********************************************/

long lastSecond; // Original - The millis counter to see when a second rolls by

unsigned int minutesSinceLastReset; //Used to reset variables after 24 hours. Imp should tell us when it's midnight, this is backup.

byte secondsRain; //When it hits 60, increase the current minute

byte minutesRain; //Keeps track of where we are in various arrays of data

volatile float rainHour[60]; //60 floating numbers to keep track of 60 minutes of rain

volatile unsigned long raintime, rainlast, raininterval, rain;

volatile unsigned long last_micros1;

long debouncing_time1 = 3; // in millis -> eleminiert evtl REED-Kontakt-Prellen

boolean RainTimeReset = false;

//struct Configuration

//{

//  float regenTag;

//  float regenStunde;

//}eepconfig EEMEM;

//AT24C256<0x50> eep; // Das EEProm auf der china üblichen RTC3231 

/********************************************/

/*             Interrupt-Section            */

/********************************************/

void rainIRQ(unsigned long now){

  if((long)(micros() - last_micros1) >= debouncing_time1 * 1000) {

    rainToday += 0.1618; // Each dump is 0.1618mm of water

    rainHour[minutesRain] += 0.1618; // Increase this minute's amount of rain

  last_micros1 = micros();

  }

}

/********************************************/

/*              Setup-Section               */

/********************************************/

void setupRain(unsigned long now){

  Serial.begin(115200);

  Wire.begin();

#ifdef DEBUG_WS

  Serial.println("rain gauge online!");

#endif

// if(eep.ready()){

//  eep.get(eepconfig.regenTag,rainToday);

//#ifdef DEBUG_WS

//  Serial.print("EEPROM rainToday : ");

//  Serial.println(rainToday);

//#endif

//  }

//  else

//  {

  midnightReset(now); //Reset rain totals

  secondsRain = 0;

  lastSecond = now;

//  }

  rainLastHour = 0;

  pinMode(RAIN_COUNT_PIN, INPUT_PULLUP);

  attachInterrupt(RAIN_COUNT_PIN, rainIRQ, RISING);

  reportWeather();

}

/********************************************/

/*               Loop-Section               */

/********************************************/

void loopRain(unsigned long now){

  //Keep track of which minute it is

  if(now - lastSecond >= 1000)

  {

    lastSecond += 1000;

    reportWeather(); //Print the current readings. Takes 172ms.

    //If we roll over 60 seconds then update the arrays for rain

    if(++secondsRain > 59)

    {

      secondsRain = 0;

      if(++minutesRain > 59) minutesRain = 0;

      

      rainHour[minutesRain] = 0; //Zero out this minute's rainfall amount

      minutesSinceLastReset++; //It's been another minute since last night's midnight reset

    }

  }

    if((TimeStunde == 0) && (! RainTimeReset))

    {

      midnightReset(now); //Reset a bunch of variables like rain and daily total rain

      RainTimeReset = true;

    }

    else if((TimeStunde != 0) && (RainTimeReset))

    {

      RainTimeReset = false;

    }

  //If we go for more than 24 hours without a midnight reset then force a reset

  //24 hours * 60 mins/hr = 1,440 minutes + 10 extra minutes. We hope that Imp is doing it.

  if(minutesSinceLastReset > (1440 + 10))

  {

    midnightReset(now); //Reset a bunch of variables like rain and daily total rain

  }

}

//When the imp tells us it's midnight, reset the total amount of rain and gusts

void midnightReset(unsigned long now)

{

  rainToday = 0; //Reset daily amount of rain

//  if(eep.ready()){

//      eep.put(eepconfig.regenTag,rainToday);

//  }

  minutesRain = 0; //Reset minute tracker

  secondsRain = 0;

  lastSecond = now; //Reset variable used to track minutes

  minutesSinceLastReset = 0; //Zero out the backup midnight reset variable

}

void calcWeather()

{

  rainLastHour = 0;

  for(int i = 0 ; i < 60 ; i++)

    rainLastHour += rainHour[i];

}

void reportWeather()

{

  calcWeather(); //Go calc all the various sensors

#ifdef INFO_WS

  Serial.print("rainLastHour : ");

  Serial.print(rainLastHour, 2);

  Serial.print(" , rainToday : ");

  Serial.println(rainToday, 2);

#endif

//  if(eep.ready()){

//  eep.put(eepconfig.regenStunde,rainLastHour); //Nicht nutzen

//  eep.put(eepconfig.regenTag,rainToday);

//  }

}

Is there a solution?

greeting

Johnnny

[Tom Keffer]

Hello Johnny,

Unfortunately, the data is summed up every few minutes and the values no longer fit.
For example, I have 0.15mm of rain in the last hour, then the value after each loop adds up until the hour is over.

If I understand you correctly, your software emits the total rain measured so far in an hour, which then gets reset at the end of the hour.

If that that is the case, then, no, that would not work well with weewx. In weewx, observation type 'rain' is the total amount of rain measured for that packet, and only for that packet. Typically, it is a very small number. This is explained in a little more detail in the section genLoopPackets in the Customizing Guide.

Or, maybe I am misunderstanding your problem.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/a9156731-4c86-4498-9546-0481365ff600%40googlegroups.com.

[John Kline]

Hi Tom,

Apologies in advance if I have misunderstood the question and have made you read what I believe you already know.

If the question is can a driver be written to make use of rain emitted every 15 seconds and resetting every hour, I believe such a driver can easily be written.

The RainWise CC3000 reports total rain and resets once a day.

The delta is calculated with the following call in the cc3000 driver:

return weewx.wxformulas.calculate_rain(rain_total, last_rain)

If the the rain total is less than the last rain total recorded, it is assumed that the counter reset.  On the CC3000, the reset is at midnight; but this would work just fine if the reset occurs every hour.

I suspect other drivers do the same in this regard (convert total rain to incremental).

On May 4, 2020, at 3:58 PM, Tom Keffer <tke...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEASnBsOhS3pMcsP-h0u1G-T8eo9cTrOrvvE%2BU1Ngq5WCA%40mail.gmail.com.

[Tom Keffer]

Hi, John,

Yes, a driver can be made to do this. But, he's using the fileparse driver, which has no such capability. So, he can either write a bespoke weewx driver, or change his arduino code to emit what we call 'rain'. I just think the latter is easier.

In the long run, the right way to do this is to implement issue #491.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/19B03C71-EB79-482D-A190-E31C996753F9%40johnkline.com.

[John Kline]

I missed the “fileparse driver” and thought he was asking about writing a driver.  Sorry about that.

On May 4, 2020, at 6:30 PM, Tom Keffer <tke...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEBmDc4swE2pvOTkk7inCso11fUmKGXzWvNZhXLSrU8k9A%40mail.gmail.com.

[Johann Schneider]

Hello everybody!

Thank you for your answers.

Sorry for my difficult to understand question.

So my weather station sends the measured values every 15 seconds.

If it sends rain, I get the hourly rain value and the rain value for the whole day. The rain value for the whole day is cumulative and is set to 0 at 00:00. The hourly rain value at the beginning of the rain is always calculated for one hour.

I thought I could simply import the data into weewx with "fileparser". This also works with the other data. Just not with the rain values.

John Kline's suggestion sounds great. That's how I would have imagined it.

How can I implement something for the rain value?

kind regards

Johnny

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/a9156731-4c86-4498-9546-0481365ff600%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[John Kline]

If you have some experience programming, you might consider writing a driver for your weather station (as opposed to hacking something just for rain).

See:

http://www.weewx.com/docs/customizing.htm#porting

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7cf18d09-e9e0-460f-b2a5-40a45f86c923%40googlegroups.com.

[mihec]

While looking for solution to my issue, I came around this post.

I am using fileparser to log data from my remote weather station (Ecowitt) and everything works fine except the rain. I have implemented my own pre-processing where I log the delta-rain but that doesn't work for some reason (might my timing issue in the scripts). I am logging the dailyRain and other values, too. Just in case. I don't know how to e.g. use the dailyRain in the NOAA report.

As I read this: https://github.com/weewx/weewx/issues/491

I assume this could resolve it. Are there any more details on the use?

sreda, 6. maj 2020 ob 00:00:03 UTC+2 je oseba John Kline napisala:

[gjr80]

Provided you are running WeeWX v4.2.0 or later you need to make the following changes to the [StdWXCalculate] stanza in weewx.conf:

1. under [[Calculations]] add an entry as follows:

        rain = prefer_hardware

2. add a new stanza [[Delta]] after the [StdWXCalculate] [[Calculations]] stanza:

    [[Delta]]

        [[[rain]]]

            input = dailyRain

where dailyRain is the name of your cumulative rain field to be used to calculate WeeWX field rain. If you have more than one cumulative rain field use the one that resets least frequently, eg use day rain in preference to hour rain, year rain in preference to month rain etc.

 your [StdWXCalculate] stanza should look something like:

[StdWXCalculate]

    [[Calculations]]

        ....

        rain = prefer_hardware

    [[Delta]]

        [[[rain]]]

            input = dailyRain

After making the changes save weewx.conf and restart WeeWX. WeeWX should now correctly populate the WeeWX field rain.

Gary

[mihec]

I am running version 5.1.0. I have edited the weewx.conf file and waiting for some rain events now.

Many thanks for your reply.

nedelja, 6. april 2025 ob 13:53:47 UTC+2 je oseba gjr80 napisala:

[mihec]

Gary,

I have edited the weewx.conf with proposed configuration. I am still not getting the values correct. I do have: rainDay, rainWeek, rainMonth and rainYear values in the weewx's database.

Maybe I don't understand the logic behind the [[Delta]].

I have few questions:

1. Does the logging interval matter? I am polling the data every minute and store it to the fileparser's compatible format. However, looking at the log file, I see that the records in the weewx.sdb do not update every minute. There are occasions where a minute is skipped (say 1 time per 10 minutes). I don't know the reason.

2. To avoid midnight time discrepancies, I am using the dateTime value provided by the data source (Ecowitt cloud) and provide the "dateTime=<>" string to the fileparser. Is this valid to do? I have also tried without this. Explanation: when polling for the data few seconds over 00:00, I get data from 23:59. If I let weewx use the actual OS time, the "rainDay" would be wrongly logged to the next day.

Why is using e.g. rainMonth or rainYear better than rainDay?

Thank you.

nedelja, 6. april 2025 ob 13:53:47 UTC+2 je oseba gjr80 napisala:

Provided you are running WeeWX v4.2.0 or later you need to make the following changes to the [StdWXCalculate] stanza in weewx.conf:

[gjr80]

Some comments below.

Gary

On Wednesday, 16 April 2025 at 18:13:29 UTC+10 mihec wrote:

Gary,

I have edited the weewx.conf with proposed configuration. I am still not getting the values correct. I do have: rainDay, rainWeek, rainMonth and rainYear values in the weewx's database.

Just one point of clarification. The cumulative field used in a [[Delta]] stanza need only appear in loop packets/archive records, there is no need for it to be saved to the WeeWX database. It won't hurt if it is, but there is no requirement.

Maybe I don't understand the logic behind the [[Delta]].

I have few questions:

1. Does the logging interval matter? I am polling the data every minute and store it to the fileparser's compatible format. However, looking at the log file, I see that the records in the weewx.sdb do not update every minute. There are occasions where a minute is skipped (say 1 time per 10 minutes). I don't know the reason.

Not sure what you mean by 'the logging interval'. When using the fileparse driver there are a few intervals/periods of interest. First you have the interval at which the fileparse source data file is updated. This is typically completely external to WeeWX. Then you have the fileparse driver polling interval, this is the interval used by the fileparse driver to read the source data file and generate a loop packet. In effect the fileparse driver will emit a loop packet every polling_interval seconds). Ideally the driver polling interval is shorter than the source data file update interval. How much shorter is somewhat subjective, if it was me I would have my polling interval 1/3 (or less) of the source data file update interval. If your source data file update interval and polling interval are the same you risk missing data (ie the source data file is updated twice before fileparse polls it). The final interval of concern in the WeeWX archive interval, this is the interval at which (in the case of the fileparse driver) WeeWX generates an archive record and saves the record to database. The WeeWX archive interval should be greater than the driver polling interval, if they are the same or the archive interval is less than the polling interval you will have archive periods where WeeWX does not receive any loop packets from the driver and consequently no archive record is generated and no archive record would be saved to database.

As an example, if you were updating your source data file every minute you might set the fileparse driver polling interval to 10 seconds and the WeeWX archive interval to five minutes.

 

2. To avoid midnight time discrepancies, I am using the dateTime value provided by the data source (Ecowitt cloud) and provide the "dateTime=<>" string to the fileparser. Is this valid to do? I have also tried without this. Explanation: when polling for the data few seconds over 00:00, I get data from 23:59. If I let weewx use the actual OS time, the "rainDay" would be wrongly logged to the next day.

I am not sure exactly what you mean. Are you obtaining data from Ecowitt.net to produce your source data file? If so you need to be very careful of latency. For example, when your station posts data to Ecowitt.net there is a delay while the data is processed by Ecowitt.net and then made available via the Ecowitt.net API. There will then likely be a delay until you poll the Ecowitt.net API, obtain the data and generate the source data file. You then may have a delay before the fileparse driver polls the source data file and generates a loop packet. The fileparse driver timestamps each loop packet with the time the loop packet is generated, not the time the original data was uploaded to Ecowitt.net, so there could well be cases where a loop packet may contain data from a previous loop period rather than the current loop period. This could cause discrepancies between data on Ecowitt.net and WeeWX. This would especially be the case for obs such as rain.

 

Why is using e.g. rainMonth or rainYear better than rainDay?

The per-period or 'delta' values are obtained by taking the difference between a current cumulative value and the last cumulative value. Cumulative fields such as rain usually reset to zero after some time (eg each hour, day, week, month, year) or when the counter rolls overs (perhaps for an all-time or total rain value). When the cumulative value jumps to zero the delta value for that period cannot be determined, so if any rain fell in that period it would be lost. When using a hour based cumulative value that reset to zero happens once per hour or 24 time per day, whereas for a year based cumulative value the reset to zero happens once per year and there is much less chance of any rainfall occurring in a reset period. If no rain ever falls in the reset period there will be no problem/discrepancy, but eventually it will. Best to minimise the risk by maximising the reset period of the cumulative value used. Hence the preference for year rain over day rain or hour rain.

[mihec]

Gary, many thanks for the detailed explanation. This helped me to track the issue and resolve it immediately. My answer is delayed because I was waiting for the first rain event.

I confirm the rain is logged correctly and polling/archive interval is perfectly on time.

There were three issues with my setup/configuration:

* external script was running every 1 minute with 10 second fixed delay (cronjob) --> too rare; I've increased this to every 30 seconds

* the polling interval in weewx.conf was set to 60 seconds --> not good as it could still get old data. I set the polling to 8 seconds or so

* I did log the "dateTime" value from Ecowitt cloud and made it available to weewx through fileparser --> not good. As soon as I removed it, the weewx's started archiving data constantly every 1 minute with no gap

Overall, I'm very happy with how the weewx works and all support you provide.

petek, 18. april 2025 ob 11:53:35 UTC+2 je oseba gjr80 napisala: