Migration Plan

[Wayne]

I have been runnning weewx on a Raspberry Pi 4 in simulator mode for over a week gaining confidence in it's (and my) ability. I am able to generate reports, transfer files to a web server and successfully imported a Davis WeatherLink (.wlk) file. The next step is to cut-over to my actual weather station hardware and go "live".

My weather station hardware is an older Davis Weather Monitor II which will talk to the Pi over an RS-232 link via an RS-232 HAT. The Weather Monitor II has the Davis data logger hardware which is capable of storing upwards of 1,500 archive records.

My plan to migrate from Davis WeatherLink to weewx is as follows:

1. Delete the current weewx.sdb file, change drivers from simulator to Vantage, connect the weather station serial line and start weewx.

2. After some number of archive data captures, stop weewx and let the Monitor II continue to store observation records to its internal data logger.

3. Import historical data into the new weewx.sdb database.

4. Restart weewx.

For this plan to work, the weewx driver needs to be smart enough to recognize and read multiple timestamped records from the station data logger (not just the current record). Depending on the length of time the station is "offline" there could be tens or even hundreds of stored logger records.

Is the Vantage driver able to read multiple such saved logger records which weewx then loads into the database? I have not seen it explicitly stated that the driver has the ability to retrieve multiple stored records from a logger. My migration plan depends upon this capability.

Thanks for any comments the group may offer.

[Greg Troxel]

Wayne <nm73wee...@fastmail.com> writes:

> I have been runnning weewx on a Raspberry Pi 4 in simulator mode for
> over a week gaining confidence in it's (and my) ability. I am able to
> generate reports, transfer files to a web server and successfully
> imported a Davis WeatherLink (.wlk) file. The next step is to
> cut-over to my actual weather station hardware and go "live".

Sounds like you are doing the right level of planning.

You did not explain what you are doing now to process and stre
observations. Skins and web servers are good to make work, but as I can
tell you understand it is all about safely storing the data in the
database.

I am assuming that a web publishing outage is not a big deal and the
point is not to lose data.

> My weather station hardware is an older Davis Weather Monitor II which
> will talk to the Pi over an RS-232 link via an RS-232 HAT. The
> Weather Monitor II has the Davis data logger hardware which is capable
> of storing upwards of 1,500 archive records.

Are you sure that the logger the same protocol as the loggers that go
with the Vantage Pro and Pro2? Is it the very same logger model#? To
me this is the biggest risk in your approach.

> My plan to migrate from Davis WeatherLink to weewx is as follows:
>

0. Double check time setting, time zone, and other settings on the
station. Replan if not ok. Probably you are using 5 minute interval;
if not think about what interval and why. It's not magic, but it's
pretty normal to use 5 minutes.

> 1. Delete the current weewx.sdb file, change drivers from simulator to Vantage, connect the weather station serial line and start weewx.

1.A. Watch the log file for reading the archive records. Expect it to
take a while (minutes not 4 hours) to process them.

> 2. After some number of archive data captures, stop weewx and let the
> Monitor II continue to store observation records to its internal data
> logger.

2.A. Check the weewx.sdb file to see if archive records have been stored
and look at the values. Look at the skins, more to check the archive
records than the skins.

> 3. Import historical data into the new weewx.sdb database.

weewx should and I think will insert every record in the logger when it starts.
So you need to either prepare an import file cut to the right place or
use an import method that gets this right. wee_import probably does,
given that weewx does things right :-)

Also you could import and start with no connection and see if things
seems ok other than "can't talk to station" errors.

Beware of imported records being exactly the same or not as the ones you
retrieve form the logger. I think it's just about timestamp, but think
about this.

> 4. Restart weewx.
>
> For this plan to work, the weewx driver needs to be smart enough to
> recognize and read multiple timestamped records from the station data
> logger (not just the current record). Depending on the length of time
> the station is "offline" there could be tens or even hundreds of
> stored logger records.

Yes, for the logger with VP/VP2 this is normal. It's one of the reasons
why they are such great stations: after you get power back N days later
you process those records and your history is continuous. Mine does
this right every power failure (VP2).

> Is the Vantage driver able to read multiple such saved logger records
> which weewx then loads into the database? I have not seen it
> explicitly stated that the driver has the ability to retrieve multiple
> stored records from a logger. My migration plan depends upon this
> capability.

Yes, for VP/VP2 logger. I am not sure your logger is the same. But I
have not reason to think it is different either. I just haven't used
any pre-VP hardware and only got a logger in December 2017.

[vince]

Set debug=1, make sure you have quality system time (hint - run ntpd) and just figure it all out experimentally

• which driver does your ancient station need ?
• does weewx wake up your console ok via your RS232 HAT ?
• does it try to download historical records from the logger ?
• if so you're set.  Your syslog will tell you what it's doing (or not doing)

The 'Vantage' driver will backfill whatever the logger has in its storage.   No idea if any other variants for your old hardware do the same.  A quick search of the group archives seems to say that somebody was trying to write a driver for your station a couple years ago, but the thread is a bit skeletal.

[Wayne]

Great comments all, thank you.

I think there has been an "ahh haa" moment for me here. I was mainly relying on the information provided in the Supported Hardware list located here:

https://weewx.com/hardware.html

...which lists my Davis Weather Monitor as a weewx compatible model. There is another page in the Weewx Hardware Guide located here:

https://weewx.com/docs/hardware.htm

...which lists the Davis Vantage models but not the Monitor/Wizard/Perception models. Upon further research I discovered that the data logger for the Vantage models is Davis PN 6510. The data logger for the older Monitor, Wizard and Perception models is Davis PN 7862. According to the Davis data sheets the Vantage logger generates a 52-byte archive record whereas the Monitor logger generates a 21-byte archive record.

So the big question is, does the weewx Vantage driver support the model 6510 logger as well as the model 7862 logger? Based on the information in the one weewx hardware documentation page I was assuming the answer is 'yes". Now I am not so sure. If I'm not mistaken, there is only one weewx driver for Davis hardware.

Are there any weewx users out there using Davis Weather Monitor II hardware with a model 7862 logger?

On Wed, Jan 25, 2023, at 11:17 AM, vince wrote:
> Set debug=1, make sure you have quality system time (hint - run ntpd)
> and just figure it all out experimentally

> * which driver does your ancient station need ?
> * does weewx wake up your console ok via your RS232 HAT ?
> * does it try to download historical records from the logger ?
> * if so you're set. Your syslog will tell you what it's doing (or not

> doing)
> The 'Vantage' driver will backfill whatever the logger has in its
> storage. No idea if any other variants for your old hardware do the
> same. A quick search of the group archives seems to say that somebody
> was trying to write a driver for your station a couple years ago, but
> the thread is a bit skeletal.
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to weewx-user+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/ac12c4c0-77df-4ecd-9f5a-353a698b09bcn%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/ac12c4c0-77df-4ecd-9f5a-353a698b09bcn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Tom Keffer]

I'm not sure what Matthew had in mind when he wrote that hardware guide. The WMII is not "supported", although there is a 3rd party driver for it: https://github.com/jardiamj/wmII

I have no idea how well it works. Just browsing the code it seems reasonably well written. Worth a try.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4b074225-148d-4a1f-88d1-33b8937c8b8d%40app.fastmail.com.

[Wayne]

Fantastic! Thank you, Tom. I am sure glad I posted to the group. I will try the wmII driver and report back the outcome. Give me a day or two to jump off that cliff...

>> > an email to weewx-user+...@googlegroups.com <mailto:weewx-user%2Bunsu...@googlegroups.com>.

>> > To view this discussion on the web visit
>> > https://groups.google.com/d/msgid/weewx-user/ac12c4c0-77df-4ecd-9f5a-353a698b09bcn%40googlegroups.com
>> > <https://groups.google.com/d/msgid/weewx-user/ac12c4c0-77df-4ecd-9f5a-353a698b09bcn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "weewx-user" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com <mailto:weewx-user%2Bunsu...@googlegroups.com>.

>> To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4b074225-148d-4a1f-88d1-33b8937c8b8d%40app.fastmail.com.
>

> --
> You received this message because you are subscribed to the Google
> Groups "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to weewx-user+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/weewx-user/CAPq0zEARtW88bp5bxV0Qo2tFzK_9q36_FYC6DcUzOVO-t1EbQg%40mail.gmail.com
> <https://groups.google.com/d/msgid/weewx-user/CAPq0zEARtW88bp5bxV0Qo2tFzK_9q36_FYC6DcUzOVO-t1EbQg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Wayne]

Before I pulled the trigger and switched drivers from Simulator to Davis Weather Monitor II (wmII.py) I took a look at the code. Upon inspection it appears that the wmII driver can not access the hardware logger archive records since there is no genArchiveRecords method, just genLoopPackets. So unless I'm mistaken my migration plan won't work because any records stored in the logger while weewx is stopped will be ignored and lost. Admittedly my understanding of the weewx architecture is beginner level so if I am wrong please let me know.

So I can either (a) live with loop packets only, (b) modify the wmII driver, or (c) migrate my historical data on-the-fly. Or perhaps I am putting too much emphasis on the logger capability. I just think it is a nice feature to utilize if it is provided in the hardware.

> https://groups.google.com/d/msgid/weewx-user/5d2188f5-d137-4c9d-a3a7-bca22f73103b%40app.fastmail.com.

[vince]

Really depends on how much you personally have an itch to scratch there.  The driver has one one-line change in 18 months, so it's uncertain how actively the author is working on this one currently.

Low blood pressure approach if you can already import your .wlk data is to just go with (a) and live without the logger unless it's a need to have.  You could I guess take a shot at a cut+paste job from the Vantage driver, but that might be harder than it sounds as you're on untrodden ground there.

Once you get your old data imported, it might come down to how often you'll lose power on your setup and how important never missing any readings is to you.