Export Meteobridge db as csv, import into WeeWX as csv

[William B]

I have a couple years worth of data that is stored on my Meteobridge NanoSD's database. I want to export that data and import it into WeeWX's database.

I have created a custom export for Meteobridge (https://pastebin.com/dg7weFe7) that exports everything I could find that has matching columns in the WeeWX DB. Sample of the CSV that was created (https://pastebin.com/gkvWRBwP). I have also created a custom WeeWX CSV import config (https://pastebin.com/UL7iFex0) to import all of that.

Just want someone to double check that that looks good before I import years of data in 1 minute intervals.

[steepleian]

William,

You appear to have mixed US customary and metric values in the data you wish to export, namely barometer and pressure in metric. I am by no means as expert on the WeeWX database as others are here but thought it worth flagging that up as something that may need addressing.

On 10 Apr 2024, at 05:25, William B <lightmas...@gmail.com> wrote:



I have a couple years worth of data that is stored on my Meteobridge NanoSD's database. I want to export that data and import it into WeeWX's database.

I have created a custom export for Meteobridge (https://pastebin.com/dg7weFe7) that exports everything I could find that has matching columns in the WeeWX DB. Sample of the CSV that was created (https://pastebin.com/gkvWRBwP). I have also created a custom WeeWX CSV import config (https://pastebin.com/UL7iFex0) to import all of that.

Just want someone to double check that that looks good before I import years of data in 1 minute intervals.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/58df2a8e-1653-43d1-ba5b-063bbdd06143n%40googlegroups.com.

[gjr80]

Looks fine to me. A couple of comments:

- Importing data in mixed unit systems (ie mixed US customary, Metric and Metric WX) should be fine provided your field map units are correctly specified.

- Deriving the interval value from the imported data involves taking the difference between the timestamp of successive records. This works fine when the import data is complete with no missing records, but can be problematic when there are missing records. If you have a fixed interval you might find it safer to set interval to a fixed number, in your case interval = 1. 

- Rather than importing all of your data in one go you might want to just try importing one or a few few days to start with, you can do this by using the --date or --from and --to command line arguments.

Gary

[William B]

So turns out my MB was set to discard minute data after 180 days, so I have minute data back to Oct 13th, and hourly data back till May 2022. I did end up setting the interval to 1 for the minute data and 60 for the hourly data. Also, I did break it up into 1 month per CSV.

In order to make sure that I could easily remove any imported data should something go wrong, I added a leafWet1 reading of 1 to every record (using a quick python script to add the column to each CSV and set that column to 1. I also added the leafWet1 sensor to the .conf. Then once all of the data was in the database, I created a column called "meteobridge" and ran an SQL query that set "meteobridge" equal to "leafWet1", then "leafWet1" equal to NULL for all records where "leafWet1" was set to 1. Now I can easily filter out all of the imported records. The last rolling month charts look good with the new data, just waiting for the yearly charts to generate whenever they do (I assume midnight either my time or GMT).