Datenbank zusammen legen.

[Günther Wrana]

Hallo

ich möchte Versuchen zwei Datenbanken von weewx.sdb zusammen zu fügen.

Ich habe einen Rechner mit Ubuntu und kann die Datenbank mit DB Browser for SQLite öffnen.

Die beiden Datenbanken beginnen und enden zu folgenden Zeiten.

1573144800 07.11.2019 17:40:00

1573149300 07.11.2019 18:55:00

1573249500 08.11.2019 22:45:00

1574009700 17.11.2019 17:55:00

Nun habe ich noch nicht herausgefunden wie ich das mit DB Browser for SQLite mache.

Ich weis auch nicht ob das das Richtige Programm für mein Vorhaben ist.

Darum hoffe ich das Ihr mir sagen könnt wie oder mit welchem Programm ich das tun soll.

Das ganze ist einmal ein Versuch danach würde ich dann gerne meine alten Wetterdaten die ich seit November 2014 aufgezeichnet habe auch noch einfügen.

Für alle Tipps bin ich dankbar also dann

Grüße Günther

[Xant]

Hallo Gunther

I dont know if possible to merge 2 databases.... which it seems can open a "pandora box" and lead to much conflict.

On the other side in a recent posting, Thomas has pointed that there is a whole section in the Customization Guide, on how to address multiple bindings (please, verify):

https://groups.google.com/d/msg/weewx-user/XE9Qqjzqqak/J-bKEd29AQAJ

That said, always verify the User's Guide, do a Forum search prior to new posting, and English is the primary language of WeeWX Forum (not been harsh here, just been clear if you expecting replies).

Best, X

[Andrew Milner]

I would merge only the archive table and then use the database utility to drop the daily summary and rebuild the summaries for the combined database.

On Tuesday, 26 November 2019 06:33:06 UTC+2, J D wrote:

Hi,

Yes you can merge two databases I have done it a number of times. You need to use SQLite commands. At a high level you attached to the two databases and then use the update command. If you google merge sqlite databases you can find a number of ways to do it.

Jerry 

[Günther Wrana]

Womit bearbeite ich die Archivtabelle und welche Datenbankdienstprogramm verwende ich dafür?

[vince]

On Wednesday, November 27, 2019 at 5:21:43 AM UTC-8, Günther Wrana wrote:

Womit bearbeite ich die Archivtabelle und welche Datenbankdienstprogramm verwende ich dafür?

https://github.com/weewx/weewx/wiki/Cleaning-up-old-'bad'-data

 

[Günther Wrana]

Mit welchem Programm soll ich das durchführen die Archivtabelle zusammen führen?

Soll ich die Beiden Tabellen als CSV zusamen führen und dann wieder Importieren?

[Günther Wrana]

Ich habe nun schon die ganze Seite 

WeeWX Utilities Guide

mir durchgelesen.

Aber ich verstehe immer noch nicht wie ich die alten Werte in die neue Datenbank bekomme.

Beide Datenbanken wurden mit weewx erstellt und von der selben Wetterstation mit Werten befüllt.

Der einzige Unterschied ist eine Zeit lang war die Wetterstation an einen PC mit Ubuntu angeschlossen, dann an einen Raspberry.

Es gibt keine Zeit Überschneidung oder sonstiges.

Ich möchte nur alle Werte in einer Datenbank haben das ich die Messwerte mir über längere Zeit zurück ansehen kann.

Beide Datenbanken heißen weewx.sdb nur mir unterschiedlichen Größen und Zeitstempeln.

Wie lege ich das an das ich diese beiden Datenbanken zusammen führe, oder besser gesagt was verstehe ich nicht.

wee_database

wee_import

wee_reports

I apologize but my English is not yellow from the egg so I write German.

[gjr80]

Hallo Günther,

Ihr Englisch ist wahrscheinlich besser als mein Deutsch, aber lassen Sie uns sehen, wie wir gehen. Ich habe sowohl Englische als auch Google-Deutsche Wörter eingefügt.

Wenn ich das richtig verstehe, haben Sie zwei WeeWX SQLite-Datenbanken, die Sie zusammenführen möchten. Leider gibt es im Utilities Guide nichts, was Ihnen das leicht machen könnte. Dies ist ein paar Mal vorgekommen und wenn Sie das Forum nach Beiträgen mit "merge sqlite datbase" durchsuchen, werden Sie einige Beiträge finden, die Ihnen helfen werden. Ich würde dieses versuchen (https://groups.google.com/d/msg/weewx-user/T2VuiIkcAEM/wZzemNzWknQJ) und das ist das, was ich als Basis für das verwendet habe:

1. Erstellen Sie eine Sicherungskopie Ihrer beiden Datenbanken

2. Installieren Sie das Dienstprogramm sqlite3, falls es noch nicht installiert ist:

$ sudo apt-get install sqlite3

3. Wählen Sie eine Ihrer Datenbanken als endgültige Datenbank aus und benennen Sie die andere in db1.sdb um. Sie sollten jetzt zwei Datenbanken haben, eine mit dem Namen weewx.sdb und eine andere mit dem Namen db1.sdb. Um die Dinge später zu vereinfachen, stellen Sie sicher, dass sich weewx.sdb in dem Verzeichnis befindet, in dem Sie Ihre WeeWX-Datenbank dauerhaft speichern möchten (die Standardverzeichnisse sind /home/weewx/archive für eine setup.py-Installation oder /var/lib/weewx für alle andere installiert). Ich werde /home/weewx/archive in dieser Anleitung verwenden

4. Kopieren Sie die Datenbank db1.sdb in eine Datei:

$ sqlite3 /path/to/db1.sdb
sqlite> .output /var/tmp/db1.dump
sqlite> .dump
sqlite> .quit

5. Importieren Sie nun die gedumpten Daten in die endgültige Datenbank weewx.sdb:

$ sqlite3 /home/weewx/archive/weewx.sdb
sqlite> .read /var/tmp/db1.dump

Sie werden wahrscheinlich Beschwerden über 'existing tables' oder 'unique constraint failed' sehen. Diese können ignoriert werden. Es muss jedoch etwas Ernsthafteres betrachtet werden.

sqlite> .quit

6. Der nächste Schritt besteht darin, die täglichen Zusammenfassungen neu zu erstellen. Dies erfolgt mit dem Dienstprogramm wee_database und erfordert, dass Sie eine WeeWX-Datenbindung konfigurieren, um die jetzt konsolidierte Datenbank weewx.sdb zu verwenden. Bearbeiten Sie die Datei weewx.conf und suchen Sie die Zeilengruppe [Databases]. Überprüfen Sie, ob Sie einen Datenbankeintrag für weewx.sdb haben. Dieser sollte ungefähr so ​​aussehen:

[Databases]

    # A SQLite database is simply a single file
    [[archive_sqlite]]
        database_name = weewx.sdb
        database_type = SQLite

Wenn sich weewx.sdb in den Verzeichnissen home/weewx/archive oder /var/lib/weewx befindet, ist das oben Genannte in Ordnung. Beachten Sie den Eintrag in [[ ]], in diesem Fall archive_sqlite.

Überprüfen Sie nun die Zeilengruppe [DataBindings] in der Datei weewx.conf.

[DataBindings]

    [[wx_binding]]
        # The database must match one of the sections in [Databases].
        # This is likely to be the only option you would want to change.
        database = archive_sqlite
        # The name of the table within the database
        table_name = archive
        # The manager handles aggregation of data for historical summaries
        manager = weewx.wxmanager.WXDaySummaryManager
        # The schema defines the structure of the database.
        # It is *only* used when the database is created.
        schema = schemas.wview.schema

Sie müssen überprüfen, ob die Datenbankeinstellung in der Zeilengruppe [[wx_binding]] den Eintrag aus der Zeilengruppe [[ ]] verwendet, den wir zuvor in der Zeilengruppe [Databases] notiert haben, in diesem Fall archive_sqlite.

Sie können nun mit wee_database die täglichen Zusammenfassungen löschen und neu erstellen.

7. Legen Sie die täglichen Zusammenfassungen ab und erstellen Sie sie neu:

$ wee_database --drop-daily
$ wee_database --rebuild-daily

Beantworten Sie die Aufforderungen nach Bedarf

8. Ihre konsolidierte Datenbank sollte nun vollständig und WeeWX gesetzt, es zu benutzen. Der letzte Schritt vor dem Neustart von WeeWX besteht darin, alle Berichte im NOAA-Format zu löschen, damit sie mit den neuen konsolidierten Daten neu generiert werden. Suchen Sie die NOAA-Format Berichte über Ihre WeeWX Maschine und löschen sie alle, Sie werden wahrscheinlich die Berichte im /home/weewx/public_html/NOAA oder /var/www/html/weewx/NOAA-Verzeichnis je nach WeeWX Art installieren finden.

9. Starten Sie WeeWX neu

Überprüfen Sie Ihre Protokolle beim Start von WeeWX auf Fehler. Überprüfen Sie Ihre Ausgabe, um festzustellen, ob die neuen konsolidierten Daten verwendet werden.

Gary

Your English is probably better than my German but let us see how we go. I have included both English and Google-German words.

If I understand correctly you have two WeeWX SQLite databases that you wish to merge. Unfortunately there is nothing in the Utilities Guide that will easily do this for you. This has come up a few times before and if you search the forum for posts with 'merge sqlite database' you will find a few posts that will help. I would try this one (https://groups.google.com/d/msg/weewx-user/T2VuiIkcAEM/wZzemNzWknQJ) and that is the one I have used as the basis for the following steps:

1. Make a backup copy of both your databases

2. Install the sqlite3 utility if it is not already installed:

$ sudo apt-get install sqlite3

3. Select one of your databases as the final database and rename the other to say db1.sdb. You should now have two databases, one named weewx.sdb and another named db1.sdb. To simplify things later make sure that weewx.sdb is located in the directory where you wish to permanently store your WeeWX database (the default locations are /home/weewx/archive for a setup.py install or /var/lib/weewx for all other installs). I will use /home/weewx/archive in these instructions

4. Dump the db1.sdb database to a file:

$ sqlite3 /path/to/db1.sdb
sqlite> .output /var/tmp/db1.dump
sqlite> .dump
sqlite> .quit

5. Now import the dumped data into the final database weewx.sdb:

$ sqlite3 /home/weewx/archive/weewx.sdb
sqlite> .read /var/tmp/db1.dump

you will likely see some complaints about 'existing tables' or 'unique constraint failed', these can be ignored. Though anything more serious needs to be looked at.

sqlite> .quit

6. The next step is to rebuild the daily summaries, this is done with the wee_database utilitiy and requires that you configure a WeeWX data binding to use the now consolidated database weewx.sdb. Edit weewx.conf and locate the [Databases] stanza. Check you have a database entry there for weewx.sdb, it should look something like:

[Databases]

    # A SQLite database is simply a single file
    [[archive_sqlite]]
        database_name = weewx.sdb
        database_type = SQLite

if weewx.sdb is in the home/weewx/archive or /var/lib/weewx directories then the above is fine. Take note of the entry in the [[ ]], in this case archive_sqlite.

Now check the [DataBindings] stanza in weewx.conf, it will likely look like this:

[DataBindings]

    [[wx_binding]]
        # The database must match one of the sections in [Databases].
        # This is likely to be the only option you would want to change.
        database = archive_sqlite
        # The name of the table within the database
        table_name = archive
        # The manager handles aggregation of data for historical summaries
        manager = weewx.wxmanager.WXDaySummaryManager
        # The schema defines the structure of the database.
        # It is *only* used when the database is created.
        schema = schemas.wview.schema

You need to check that the database setting in the [[wx_binding]] stanza uses the entry from the [[ ]] stanza that we previously noted in the [Databases] stanza, in this case archive_sqlite.

You can now use wee_database to drop and rebuild the daily summaries.

7. Drop and rebuild the daily summaries:

$ wee_database --drop-daily
$ wee_database --rebuild-daily

answering any prompts as required

8. Your consolidated database should now be complete and WeeWX set to use it. The final step before restarting WeeWX is to delete all the NOAA format reports so that they are re-generated with the new consolidated data. Locate the NOAA format reports on your WeeWX machine and delete them all, you will likely find the reports in the /home/weewx/public_html/NOAA or /var/www/html/weewx/NOAA directory depending on your WeeWX install type.

9. Restart WeeWX

Check your logs from WeeWX startup for errors. Check your output to see that the new consolidated data is being used (checking the NOAA reorts is a good and simple check for this)

[Günther Wrana]

Hallo wenn ich folgende Befehle im Terminal eingebe wie oben beschrieben.

Bekomme ich folgende Fehlermeldungen.

Es wir irgend etwas in der Datenbank durchgeführt nur wird sie dadurch nicht grösser.

Wo liegt das Problem, oder was mache ich falsch?

daham@daham-p7-1011de:~$ sqlite3 /var/tmp/db1.sdb

SQLite version 3.22.0 2018-01-22 18:45:57

Enter ".help" for usage hints.

sqlite> .output/var/tmp/db1.dump

daham@daham-p7-1011de:~$ sqlite3 /media/daham/Neu1601/Eigene\ Dateien/Wetterstation/weewx/271220192133/weewx.sdb

SQLite version 3.22.0 2018-01-22 18:45:57

Enter ".help" for usage hints.

sqlite> .read /var/tmp/db1.dump

Error: near line 3: table archive already exists

Error: near line 40840: table archive_day_barometer already exists

Error: near line 40984: table archive_day_pressure already exists

Error: near line 41128: table archive_day_altimeter already exists

Error: near line 41256: table archive_day_inTemp already exists

Error: near line 41400: table archive_day_outTemp already exists

Error: near line 41544: table archive_day_inHumidity already exists

Error: near line 41688: table archive_day_outHumidity already exists

Error: near line 41832: table archive_day_windSpeed already exists

Error: near line 41976: table archive_day_windDir already exists

Error: near line 42120: table archive_day_windGust already exists

Error: near line 42264: table archive_day_windGustDir already exists

Error: near line 42408: table archive_day_rainRate already exists

Error: near line 42552: table archive_day_rain already exists

Error: near line 42696: table archive_day_dewpoint already exists

Error: near line 42840: table archive_day_windchill already exists

Error: near line 42984: table archive_day_heatindex already exists

Error: near line 43128: table archive_day_ET already exists

Error: near line 43272: table archive_day_radiation already exists

Error: near line 43416: table archive_day_UV already exists

Error: near line 43560: table archive_day_extraTemp1 already exists

Error: near line 43704: table archive_day_extraTemp2 already exists

Error: near line 43848: table archive_day_extraTemp3 already exists

Error: near line 43992: table archive_day_soilTemp1 already exists

Error: near line 44136: table archive_day_soilTemp2 already exists

Error: near line 44280: table archive_day_soilTemp3 already exists

Error: near line 44424: table archive_day_soilTemp4 already exists

Error: near line 44568: table archive_day_leafTemp1 already exists

Error: near line 44712: table archive_day_leafTemp2 already exists

Error: near line 44856: table archive_day_extraHumid1 already exists

Error: near line 45000: table archive_day_extraHumid2 already exists

Error: near line 45144: table archive_day_soilMoist1 already exists

Error: near line 45288: table archive_day_soilMoist2 already exists

Error: near line 45432: table archive_day_soilMoist3 already exists

Error: near line 45576: table archive_day_soilMoist4 already exists

Error: near line 45720: table archive_day_leafWet1 already exists

Error: near line 45864: table archive_day_leafWet2 already exists

Error: near line 46008: table archive_day_rxCheckPercent already exists

Error: near line 46152: table archive_day_txBatteryStatus already exists

Error: near line 46296: table archive_day_consBatteryVoltage already exists

Error: near line 46440: table archive_day_hail already exists

Error: near line 46584: table archive_day_hailRate already exists

Error: near line 46728: table archive_day_heatingTemp already exists

Error: near line 46872: table archive_day_heatingVoltage already exists

Error: near line 47016: table archive_day_supplyVoltage already exists

Error: near line 47160: table archive_day_referenceVoltage already exists

Error: near line 47304: table archive_day_windBatteryStatus already exists

Error: near line 47448: table archive_day_rainBatteryStatus already exists

Error: near line 47592: table archive_day_outTempBatteryStatus already exists

Error: near line 47736: table archive_day_inTempBatteryStatus already exists

Error: near line 47880: table archive_day__metadata already exists

Error: near line 47881: UNIQUE constraint failed: archive_day__metadata.name

Error: near line 47882: UNIQUE constraint failed: archive_day__metadata.name

Error: near line 47883: table archive_day_wind already exists

sqlite> 

[gjr80]

Hallo

Die angezeigten Fehlermeldungen werden erwartet und können ignoriert werden.

Die Dateigröße kann manchmal ein nützlicher Maßstab sein, liefert jedoch nicht immer ein genaues Bild. Der wahre Test besteht darin, die Daten in der neuen zusammengeführten Datenbank zu überprüfen oder zu verwenden. Haben Sie die letzten Schritte befolgt und WeeWX tatsächlich dazu gebracht, die neue zusammengeführte Datenbank zu verwenden? Wenn ja, was zeigen die Berichte jetzt?

Sie können auch die neue Datenbank abfragen und die frühesten und neuesten Zeitstempel finden. Wenn Ihre Datenbank /home/weewx/archive/weewx.sdb lautet, kann die folgende Befehlsfolge verwendet werden:

$ sqlite3 /home/weewx/archive/weewx.sdb
sqlite> SELECT MIN(dateTime),MAX(dateTime),COUNT(dateTime) FROM archive;

Dies zeigt die frühesten und neuesten Zeitstempel, gefolgt von der Anzahl der Archivdatensätze.

1486989000|1578345600|300759

oder um die Zeitstempel in lesbarer Form anzuzeigen:

sqlite> SELECT strftime('%d-%m-%Y %H:%M:%S', datetime(MIN(dateTime), 'unixepoch', 'localtime')),strftime('%d-%m-%Y %H:%M:%S', datetime(MAX(dateTime), 'unixepoch', 'localtime')),COUNT(dateTime) FROM archive;

was so etwas zeigen wird:

13-02-2017 22:30:00|07-01-2020 07:25:00|300760

Sie können die obigen Überprüfungen auch für die beiden Datenbanken vor dem Zusammenführen durchführen. Wenn der früheste und der späteste Zeitstempel und Datensatz übereinstimmen, haben Sie ein gewisses Maß an Sicherheit, dass die Zusammenführung wie erwartet erfolgt ist. Eine umfassendere Überprüfung besteht darin, dass WeeWX die zusammengeführte Datenbank verwendet und anschließend neu generierte NOAA-Berichte überprüft. Dies sollte auf einen Blick zeigen, ob die Zusammenführung funktioniert hat.

Gary

The error messages you see are expected, they can be ignored.

File size can sometimes be a useful yardstick but does not always give an accurate picture. The true test is to check or use the data in the new merged database. Did you follow the last few steps and actually get WeeWX to use the new merged database? If so what do the reports show now?

Another check you can do is to query the new database and find the earliest and latest timestamps. If your database is /home/weewx/archive/weewx.sdb the following sequence of commands can be used:

$ sqlite3 /home/weewx/archive/weewx.sdb
sqlite> SELECT MIN(dateTime),MAX(dateTime),COUNT(dateTime) FROM archive;

this will show the earliest and latest timestamps followed by the archive record count, something like:

1486989000|1578345600|300759

or to view the timstamps in human readable form:

sqlite> SELECT strftime('%d-%m-%Y %H:%M:%S', datetime(MIN(dateTime), 'unixepoch', 'localtime')),strftime('%d-%m-%Y %H:%M:%S', datetime(MAX(dateTime), 'unixepoch', 'localtime')),COUNT(dateTime) FROM archive;

which will show something like:

13-02-2017 22:30:00|07-01-2020 07:25:00|300760

You can also perform the above checks on the two databases before the merge, if the earliest and latest timestamps and record counts tally then you have a degree of confidence that the merge occurred as expected. A more complete check is having WeeWX use the merged database and then checking re-generated NOAA reports. This should show at a glance if the merge worked.

[Günther Wrana]

Das wird gemeldet wenn ich die alte Datenbank mit den oben angeführten Befehlen abfrage.

Ist da noch etwas zu machen oder ist die alte Datenbank defekt und ich kann sie löschen.

Grüße Günther

daham@hp-pc:/tmp$ sudo sqlite3 /tmp/db1.sdb

SQLite version 3.22.0 2018-01-22 18:45:57

Enter ".help" for usage hints.

sqlite> SELECT MIN(dateTime),MAX(dateTime),COUNT(dateTime) FROM archive;

Error: database disk image is malformed

sqlite> SELECT strftime('%d-%m-%Y %H:%M:%S', datetime(MIN(dateTime), 'unixepoch', 'localtime')),strftime('%d-%m-%Y %H:%M:%S', datetime(MAX(dateTime), 'unixepoch', 'localtime')),COUNT(dateTime) FROM archive;

Error: database disk image is malformed

sqlite> 

[Günther Wrana]

https://drive.google.com/open?id=1q_ILd6m6M_KT0HThnv6X7JHxbaq8zMfA

If i open this merged database on Ubuntu with SQLite there is no problem.

But when i send this datatbase on the raspberry where weewx is runing weewx is out of order.

Where is the problem?

I know that the database on the raspberry is named weewx.sdb

I hope anybody can help me and understand my terrible english.

Thanks

[Thomas Keffer]

Günther, 

I get "database disk image is malformed" on both my Ubuntu system, and on a Raspberry Pi. 

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/87c9cfc6-2b06-4924-8479-c996405e1d74%40googlegroups.com.

[Günther Wrana]

Sorry i have it on Ubuntu with DB Browser for SQLite opend.

And with this sofware i can take a look on the database.

But i dont know were is the problem.
Is this the problem that i opned the database with this software and not in terminal?

Günther

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[Thomas Keffer]

I doubt it, but it is easy enough to check. What happens if you use sqlite3?

sqlite3 db3.sdb

sqlite> select datetime(dateTime,'unixepoch','localtime'), outTemp from archive order by dateTime desc limit 10;

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ff73bff7-949e-4266-9964-62844b78556d%40googlegroups.com.

[Günther Wrana]

daham@hp-pc:/tmp$ sudo sqlite3 /tmp/db3.sdb

[sudo] Passwort für daham: 

sqlite> select datetime(dateTime,'unixepoch','localtime'), outTemp from archive order by dateTime desc limit 10;

2020-01-16 17:40:00|33.2514285714286

2020-01-16 17:35:00|33.3028571428572

2020-01-16 17:30:00|33.701

2020-01-16 17:25:00|33.7914285714286

2020-01-16 17:20:00|33.9371428571429

2020-01-16 17:15:00|33.9542857142857

2020-01-16 17:10:00|34.0914285714286

2020-01-16 17:05:00|34.34

2020-01-16 17:00:00|34.4685714285714

2020-01-16 16:55:00|34.5628571428571

sqlite> 

This is the answer if i open db3.sdb in terminal on my linux computer.

I want to know, how the syntax looks like, if i want to look for the mistake.

Danke Günther

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ff73bff7-949e-4266-9964-62844b78556d%40googlegroups.com.

[Thomas Keffer]

You can check the integrity of the database using the pragma "integrity_check".

sqlite3 db3.sdb

sqlite> pragma integrity_check;

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c8e26b1c-5cf4-42ec-b1d7-dc40c07703ac%40googlegroups.com.

[Günther Wrana]

sqlite> pragma integrity_check;

*** in database main ***

On tree page 554 cell 75: invalid page number 1879769860

Multiple uses for byte 1072 of page 554

On tree page 957 cell 28: Rowid 18740937184 out of order

On tree page 957 cell 25: Rowid 18749066592 out of order

On tree page 957 cell 23: Rowid 1561684576 out of order

On tree page 957 cell 19: Rowid 1568679008 out of order

On tree page 957 cell 17: Rowid 1560117600 out of order

On tree page 957 cell 16: Rowid 1593585632 out of order

row 9507 missing from index sqlite_autoindex_archive_1

row 9508 missing from index sqlite_autoindex_archive_1

row 9509 missing from index sqlite_autoindex_archive_1

row 9510 missing from index sqlite_autoindex_archive_1

row 9511 missing from index sqlite_autoindex_archive_1

row 9512 missing from index sqlite_autoindex_archive_1

row 9513 missing from index sqlite_autoindex_archive_1

row 9514 missing from index sqlite_autoindex_archive_1

row 9515 missing from index sqlite_autoindex_archive_1

row 9516 missing from index sqlite_autoindex_archive_1

row 9517 missing from index sqlite_autoindex_archive_1

row 9518 missing from index sqlite_autoindex_archive_1

row 9519 missing from index sqlite_autoindex_archive_1

row 9520 missing from index sqlite_autoindex_archive_1

row 9521 missing from index sqlite_autoindex_archive_1

row 9522 missing from index sqlite_autoindex_archive_1

row 9523 missing from index sqlite_autoindex_archive_1

row 9524 missing from index sqlite_autoindex_archive_1

row 9525 missing from index sqlite_autoindex_archive_1

row 9526 missing from index sqlite_autoindex_archive_1

row 9527 missing from index sqlite_autoindex_archive_1

row 9528 missing from index sqlite_autoindex_archive_1

row 9529 missing from index sqlite_autoindex_archive_1

row 9530 missing from index sqlite_autoindex_archive_1

row 9531 missing from index sqlite_autoindex_archive_1

row 9532 missing from index sqlite_autoindex_archive_1

row 9533 missing from index sqlite_autoindex_archive_1

row 9534 missing from index sqlite_autoindex_archive_1

row 9535 missing from index sqlite_autoindex_archive_1

row 9536 missing from index sqlite_autoindex_archive_1

row 9537 missing from index sqlite_autoindex_archive_1

row 9538 missing from index sqlite_autoindex_archive_1

row 9539 missing from index sqlite_autoindex_archive_1

row 9540 missing from index sqlite_autoindex_archive_1

row 9541 missing from index sqlite_autoindex_archive_1

row 9542 missing from index sqlite_autoindex_archive_1

row 9543 missing from index sqlite_autoindex_archive_1

row 9544 missing from index sqlite_autoindex_archive_1

row 9545 missing from index sqlite_autoindex_archive_1

row 9546 missing from index sqlite_autoindex_archive_1

row 9547 missing from index sqlite_autoindex_archive_1

row 9548 missing from index sqlite_autoindex_archive_1

row 9549 missing from index sqlite_autoindex_archive_1

row 9550 missing from index sqlite_autoindex_archive_1

row 9551 missing from index sqlite_autoindex_archive_1

row 9552 missing from index sqlite_autoindex_archive_1

row 9553 missing from index sqlite_autoindex_archive_1

row 9554 missing from index sqlite_autoindex_archive_1

row 9555 missing from index sqlite_autoindex_archive_1

row 9556 missing from index sqlite_autoindex_archive_1

row 9557 missing from index sqlite_autoindex_archive_1

row 9558 missing from index sqlite_autoindex_archive_1

row 9559 missing from index sqlite_autoindex_archive_1

row 9560 missing from index sqlite_autoindex_archive_1

row 9561 missing from index sqlite_autoindex_archive_1

row 9562 missing from index sqlite_autoindex_archive_1

row 9563 missing from index sqlite_autoindex_archive_1

row 9564 missing from index sqlite_autoindex_archive_1

row 9565 missing from index sqlite_autoindex_archive_1

row 9566 missing from index sqlite_autoindex_archive_1

row 9567 missing from index sqlite_autoindex_archive_1

row 9568 missing from index sqlite_autoindex_archive_1

row 9569 missing from index sqlite_autoindex_archive_1

row 9570 missing from index sqlite_autoindex_archive_1

row 9571 missing from index sqlite_autoindex_archive_1

row 9572 missing from index sqlite_autoindex_archive_1

row 9573 missing from index sqlite_autoindex_archive_1

row 9574 missing from index sqlite_autoindex_archive_1

Error: database or disk is full

sqlite> 

This is the answer and how can i repair the defect datacell?

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c8e26b1c-5cf4-42ec-b1d7-dc40c07703ac%40googlegroups.com.

[Thomas Keffer]

The best solution is to replace with your backup.

If you do not have a backup, well, ... you can try dumping it to a file, then restoring. The steps are (NOT TESTED):

sqlite3 db3.sdb

sqlite> .output db3.sql

sqlite> .dump archive

sqlite> .quit

Check the file db3.sql, then use it to restore to a new database:

sqlite3 db3_restored.sdb < db3.sql

Hopefully the new file, db3_restored.sdb, will be OK.

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/555d9cbb-eaac-4be7-a334-cad72f604725%40googlegroups.com.

[Günther Wrana]

Was habe ich nun kaputt gemacht?

pi@raspberrypi:~/Desktop $ tail -f /var/log/syslog

Feb 26 20:33:39 raspberrypi systemd[1]: Starting LSB: weewx weather system...

Feb 26 20:33:39 raspberrypi weewx[8560]: engine: Initializing weewx version 3.9.2

Feb 26 20:33:39 raspberrypi weewx[8560]: engine: Using Python 2.7.16 (default, Apr  6 2019, 01:42:57) #012[GCC 8.2.0]

Feb 26 20:33:39 raspberrypi weewx[8560]: engine: Platform Linux-4.19.75-v7+-armv7l-with-debian-10.1

Feb 26 20:33:39 raspberrypi weewx[8560]: engine: Locale is 'de_AT.UTF-8'

Feb 26 20:33:39 raspberrypi weewx[8560]: engine: pid file is /var/run/weewx.pid

Feb 26 20:33:39 raspberrypi weewx[8549]: Starting weewx weather system: weewx.

Feb 26 20:33:39 raspberrypi systemd[1]: Started LSB: weewx weather system.

Feb 26 20:33:39 raspberrypi weewx[8564]: engine: Error while parsing configuration file /home/weewx/weewx.conf

Feb 26 20:33:39 raspberrypi weewx[8564]: ****    Reason: 'Parsing failed with several errors.#012First error at line 9.'

Feb 26 20:33:42 raspberrypi dhcpcd[389]: wlan0: Router Advertisement from fe80::9ada:c4ff:feb9:eeda

Feb 26 20:33:42 raspberrypi dhcpcd[389]: wlan0: soliciting a DHCPv6 lease

Feb 26 20:33:42 raspberrypi dhcpcd[389]: wlan0: dropping DHCPv6 due to no valid routers

Feb 26 20:33:52 raspberrypi dhcpcd[389]: wlan0: Router Advertisement from fe80::9ada:c4ff:feb9:eeda

Feb 26 20:33:52 raspberrypi dhcpcd[389]: wlan0: soliciting a DHCPv6 lease

Feb 26 20:33:52 raspberrypi dhcpcd[389]: wlan0: dropping DHCPv6 due to no valid routers

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/555d9cbb-eaac-4be7-a334-cad72f604725%40googlegroups.com.

[gjr80]

Vielleicht Zeile neun in weewx.conf?

Gary

[Günther Wrana]

Ich habe die weewx.conf aus meiner Datensicherung neu eingespielt nun kommt diese Meldung.

Soll ich das ganze neu installieren oder was ist nun defekt?

Feb 27 05:53:21 raspberrypi weewx[19161]: engine: pid file is /var/run/weewx.pid

Feb 27 05:53:21 raspberrypi weewx[19173]: engine: Using configuration file /home/weewx/weewx.conf

Feb 27 05:53:21 raspberrypi weewx[19173]: engine: Loading station type WH23xx (user.wh23xx)

Feb 27 05:53:21 raspberrypi weewx[19150]: Starting weewx weather system: weewx.

Feb 27 05:53:21 raspberrypi systemd[1]: Started LSB: weewx weather system.

Feb 27 05:53:21 raspberrypi weewx[19173]: engine: Caught unrecoverable exception in engine:

Feb 27 05:53:21 raspberrypi weewx[19173]:     ****  bad marshal data (unknown type code)

Feb 27 05:53:21 raspberrypi weewx[19173]:     ****  Traceback (most recent call last):

Feb 27 05:53:21 raspberrypi weewx[19173]:     ****    File "/home/weewx/bin/weewx/engine.py", line 888, in main

Feb 27 05:53:21 raspberrypi weewx[19173]:     ****    File "/home/weewx/bin/weewx/engine.py", line 72, in __init__

Feb 27 05:53:28 raspberrypi dhcpcd[389]: wlan0: Router Advertisement from fe80::9ada:c4ff:feb9:eeda

Feb 27 05:53:28 raspberrypi dhcpcd[389]: wlan0: soliciting a DHCPv6 lease

Feb 27 05:53:28 raspberrypi dhcpcd[389]: wlan0: dropping DHCPv6 due to no valid routers

Feb 27 05:53:32 raspberrypi dhcpcd[389]: wlan0: Router Advertisement from fe80::9ada:c4ff:feb9:eeda

Feb 27 05:53:32 raspberrypi dhcpcd[389]: wlan0: soliciting a DHCPv6 lease

Feb 27 05:53:32 raspberrypi dhcpcd[389]: wlan0: dropping DHCPv6 due to no valid routers

Feb 27 05:53:36 raspberrypi dhcpcd[389]: wlan0: Router Advertisement from fe80::9ada:c4ff:feb9:eeda

Feb 27 05:53:37 raspberrypi dhcpcd[389]: wlan0: soliciting a DHCPv6 lease

Feb 27 05:53:37 raspberrypi dhcpcd[389]: wlan0: dropping DHCPv6 due to no valid routers

[Günther Wrana]

It is done i have the database merged but wen i copy the database to /var/lib/weewx on my Ubuntu System come this messages in terminal. 

Feb 28 12:15:32 hp-pc weewx[13721]: engine: retrying...

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Using configuration file /etc/weewx/weewx.conf

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Loading station type WH23xx (user.wh23xx)

Feb 28 12:15:32 hp-pc weewx[13721]: wh23xx: driver version is 0.14

Feb 28 12:15:32 hp-pc weewx[13721]: wh23xx: usb info: pyusb_version=1.0.0

Feb 28 12:15:32 hp-pc weewx[13721]: wh23xx: poll interval is 15

Feb 28 12:15:32 hp-pc weewx[13721]: wh23xx: Found device on USB bus= device=

Feb 28 12:15:32 hp-pc kernel: [65563.123761] usb 2-1.7: reset full-speed USB device number 14 using ehci-pci

Feb 28 12:15:32 hp-pc weewx[13721]: engine: StdConvert target unit is 0x1

Feb 28 12:15:32 hp-pc weewx[13721]: wxcalculate: The following values will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, heatindex=prefer_hardware, maxSolarRad=prefer_hardware, humidex=prefer_hardware, pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, cloudbase=prefer_hardware

Feb 28 12:15:32 hp-pc weewx[13721]: wxcalculate: The following algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Archive will use data binding wx_binding

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Record generation will be attempted in 'software'

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Using archive interval of 300 seconds (software record generation)

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Using binding 'wx_binding' to database 'weewx.sdb'

Feb 28 12:15:32 hp-pc weewx[13721]: manager: Starting backfill of daily summaries

Feb 28 12:15:32 hp-pc weewx[13721]: restx: StationRegistry: Station will be registered.

Feb 28 12:15:32 hp-pc weewx[13721]: restx: Wunderground-PWS: Data for station IMATZENR2 will be posted

Feb 28 12:15:32 hp-pc weewx[13721]: restx: PWSweather: Posting not enabled.

Feb 28 12:15:32 hp-pc weewx[13721]: restx: CWOP: Posting not enabled.

Feb 28 12:15:32 hp-pc weewx[13721]: restx: WOW: Posting not enabled.

Feb 28 12:15:32 hp-pc weewx[13721]: restx: AWEKAS: Data will be uploaded for user Gilli

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Starting up weewx version 3.9.2

Feb 28 12:15:32 hp-pc weewx[13721]: engine: Starting main packet loop.

Feb 28 12:15:33 hp-pc weewx[13721]: wh23xx: get_current: failed attempt 1 of 5: [Errno 110] Operation timed out

Feb 28 12:15:44 hp-pc weewx[13721]: wh23xx: get_current: failed attempt 2 of 5: [Errno 110] Operation timed out

Feb 28 12:15:55 hp-pc weewx[13721]: wh23xx: get_current: failed attempt 3 of 5: [Errno 110] Operation timed out

Feb 28 12:16:06 hp-pc weewx[13721]: wh23xx: get_current: failed attempt 4 of 5: [Errno 110] Operation timed out

Feb 28 12:16:17 hp-pc weewx[13721]: wh23xx: get_current: failed attempt 5 of 5: [Errno 110] Operation timed out

Feb 28 12:16:27 hp-pc weewx[13721]: wh23xx: read failed: max retries (5) exceeded

Feb 28 12:16:27 hp-pc weewx[13721]: engine: Main loop exiting. Shutting engine down.

Feb 28 12:16:27 hp-pc weewx[13721]: engine: Caught WeeWxIOError: read failed: max retries (5) exceeded

Feb 28 12:16:27 hp-pc weewx[13721]:     ****  Waiting 60 seconds then retrying...

Where is the problem?

When i delete the database and weewx makes the database new then somtimes it works.

But somtimes the failed messages cames also.

Thanks Günther

Danke Günther

[Günther Wrana]

Hello and thanks i have succeeded my tow database to merged.

daham@hp-pc:/tmp$ sudo sqlite3 /media/daham/Neu1601/Eigene\ Dateien/Wetterstation/weewx/010320201357/weewx.sdb

SQLite version 3.22.0 2018-01-22 18:45:57

Enter ".help" for usage hints.

sqlite> SELECT strftime('%d-%m-%Y %H:%M:%S', datetime(MIN(dateTime), 'unixepoch', 'localtime')),strftime('%d-%m-%Y %H:%M:%S', datetime(MAX(dateTime), 'unixepoch', 'localtime')),COUNT(dateTime) FROM archive;

24-05-2019 21:10:00|01-03-2020 14:25:00|61866

sqlite> 

This is the begin data 24-05-2019 and the open end 01-03-2020.

Tanks for all the have me supports.

Günther 

[Günther Wrana]

Hello, since I've merged the databases, weewx has also worked. Only since then has the following been written in the log files.

Mar 12 07:50:27 raspberrypi weewx[29291]: manager: Added record 2020-03-12 07:50:00 CET (1583995800) to database 'weewx.sdb'

Mar 12 07:50:27 raspberrypi weewx[29291]: manager: Added record 2020-03-12 07:50:00 CET (1583995800) to daily summary in 'weewx.sdb'

Mar 12 07:50:29 raspberrypi weewx[29291]: restx: AWEKAS: Failed to publish record 2020-03-12 07:50:00 CET (1583995800): server returned ' - UV Wert administrativ gesperrt  - Transfer OK'

Mar 12 07:50:30 raspberrypi weewx[29291]: restx: Wunderground-PWS: Published record 2020-03-12 07:50:00 CET (1583995800)

Mar 12 07:50:31 raspberrypi weewx[29291]: cheetahgenerator: Generated 9 files for report SeasonsReport in 1.66 seconds

Mar 12 07:50:49 raspberrypi weewx[29291]: reportengine: Caught unrecoverable exception in generator 'weewx.imagegenerator.ImageGenerator'

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****  unsupported operand type(s) for -: 'list' and 'float'

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****  Traceback (most recent call last):

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 204, in run

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****      obj.start()

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 300, in start

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****      self.run()

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****    File "/home/weewx/bin/weewx/imagegenerator.py", line 33, in run

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****      self.genImages(self.gen_ts)

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****    File "/home/weewx/bin/weewx/imagegenerator.py", line 188, in genImages

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****      new_data_vec_t = self.converter.convert(data_vec_t)

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****    File "/home/weewx/bin/weewx/units.py", line 784, in convert

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****      new_val_t = convert(val_t, new_unit_type)

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****    File "/home/weewx/bin/weewx/units.py", line 1138, in convert

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****      new_val = conversion_func(val_t[0]) if val_t[0] is not None else None

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****    File "/home/weewx/bin/weewx/units.py", line 33, in FtoC

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****      return (x - 32.0) * 5.0 / 9.0

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****  TypeError: unsupported operand type(s) for -: 'list' and 'float'

Mar 12 07:50:49 raspberrypi weewx[29291]:         ****  Generator terminated

Mar 12 07:50:49 raspberrypi weewx[29291]: copygenerator: copied 17 files to /var/weewx/reports

Is that a problem or do I have to set something in the database?

I just noticed that some of the annual reports are not displayed.

This is my page www.guentherwrana.ddns.net/weewx

[Günther Wrana]

This is my page http://guentherwrana.ddns.net/weewx/

[Meteo Oberwallis]

Hallo zusammen.

Ich wollte ebenfalls meine beiden .sdb Datenbanken zusammenlegen. Sie stammen beide aus weewx. Nun bekomme ich immer diese Fehlermeldung:

root@raspberrypi:~# sqlite3 weewx.sdb_new_new
SQLite version 3.16.2 2017-01-06 16:32:41

Enter ".help" for usage hints.

sqlite> .read weewx.sdb_new
Error: cannot open "weewx.sdb_new"
sqlite> .read /home/pi/Desktop/weewx.sdb_new
Error: near line 1: near "SQLite": syntax error
Error: near line 403: unrecognized token: " "
Error: near line 5261: near "@n�": syntax error
Error: near line 5337: unrecognized token: " "
Error: near line 6332: unrecognized token: " "
Error: near line 6793: unrecognized token: " "
Error: near line 7575: near "@rH": syntax error
Error: near line 7831: near "@n�": syntax error
sqlite>

Hatte das jemand auch?

[Günther Wrana]

Hallo

bei mir hat das dann funktioniert nach dem ich die alte Datenbank so wie oben beschrieben in eine .sql Datenbank umgewandelt habe.

sqlite3 db3.sdb

sqlite> .output db3.sql

sqlite> .dump archive

sqlite> .quit

und danach haben sich die beiden Datenbanken zusammmen legen lassen.

Fehlermeldungen sind auch bei mir gekommen nur wenn man am Schluss die Datenbank mit dem checkt.

sqlite3 db3.sdb

sqlite> pragma integrity_check;

und es O.K ausgibt dann passt das.

Grüße Günther 

[Meteo Oberwallis]

Hallo Günther.

Darf ich dich fragen, ob du vorher auch mal WsWin benutzt hast? Ich habe wetterdaten seit 2005, welche aber natürlich jetzt in weewx fehlen. Es müsste doch möglich sein, die von WsWin erstellten Monatsdateien, also .csv in die weewx Datenbank zu implementieren? Das wäre natürlich genial. anonsten verliere ich die ganzen jahre.

Gruss und danke für eure Hilfe

[Günther Wrana]

Hallo Meteo

ich habe das Problem auch meine Wetterdaten von 2014 bis zum Mai 2019 als .csv Dateien.

Mit dem weewx_import funktioniert das irgend wie. Ich Versuche das auch gerade aber angeblich soll das dann bei weewx 4.0 dann einfacher oder überhaupt funktionieren.

Ich bin auch erst seit voriges Jahr im Mai mit dem Programm so ein wenig vertraut.

Aber es sind hier alle sehr hilfsbereit bei Fragen.

Nur ist es besser wenn du in Englisch die Fragen stellst dann bekommst du schneller Antworten.

Ich mache das immer mit dem Google Übersetzer denn in Englisch bin ich nicht so versiert.

Aber lies dir die Dokumentation von weewx durch dort findest du oft Hilfen bei manchen Rätseln dieses Programmes.

Sonst funktioniert es sehr gut und ich bin zufrieden den nun kann ich meine Wetterdaten von allen meinen Geräten im Netzwerk zuhause und im Internet ansehen.

Grüße Günther  

[gjr80]

Hallo,

Ich entschuldige mich für das Google-Deutsche :)

Abhängig vom Format Ihrer WsWin-CSV-Datei sollte es möglich sein, Ihre CSV-Daten mit dem Dienstprogramm wee_import zu importieren. Für einen CSV-Import erfordert wee_import, dass die Datei eine Kopfzeile und dann einen Datensatz pro Zeile enthält. Es muss auch ein einzelnes Feld vorhanden sein, das Datum und Uhrzeit des Datensatzes enthält. Wenn Ihre Daten also separate Felder für Datum und Uhrzeit haben, müssen diese zusammengeführt werden. Außerdem kann wee_import jeweils nur aus einer einzelnen CSV-Datei importieren. Wenn Sie also mehrere CSV-Dateien haben, müssen Sie mehrere wee_import-Sitzungen ausführen oder die Dateien auf andere Weise zu einer zusammenführen.

Wenn Sie eine Ihrer WsWin-CSV-Dateien veröffentlichen möchten, schaue ich sie mir gerne an und prüfe, ob wee_import mit der Datei funktioniert und welche Einstellungen / Änderungen möglicherweise erforderlich sind.

In English:

My apologies for the Google-German :)

Depending on the format of your WsWin .csv file it should be possible to import your csv data using the wee_import utility. For a csv import wee_import requires the file contain a header line and then one record per line. There must also be a single field that contains the date and time of the record, so if your data has separate fields for date and time these would need to be merged. Also, wee_import can only import from a single csv file at a time, so if you have multiple csv files you would need to run multiple wee_import sessions or otherwise merge the files into one.

If you wish to post one of your WsWin csv files I am happy to look at it and see if wee_import will work with the file and what settings/changes may be required.

gruß,

Gary

[Günther Wrana]

Hello,

I would have my .csv file here that I would like to import. How should it look like? Do I have to insert the column names listed in the documentation in the header?

Time,Temp,Dewpoint,Press,WindDir,WindSpeed,WindGust,Hum,dailyrain,SolarRad

Date and time are in one column, do they also have to have this UNIX date format?

Thank you Günther

[gjr80]

Hallo Günther,

A few observations on your data:

1. The column names can be whatever you want, the only requirements are they are unique and you can enter them in the [[FieldMap]] portion of the import config file. In theory, your German field names should work but all we need is some sort of unicode failure somewhere in the processing chain and the import will fail. If it were me I would remove the units in brackets and try the import with your fields names via a dry run (using --dry-run). That will show you if it will fail without affecting your WeeWX data.

2. You will need to remove the 'M' from the Intervall field.

3. WeeWX records rain on a per-interval basis whereas your rain data is cumulative. That is fine, wee_import can do the calculations but you need to tell wee_import your rain data is cumulative and you need to select one of the rain fields to use (you have five). I would use the longest period rain field, in your case Gesamter Niederschlag. So in your import config file you will need (untested):

rain = cumulative

and in your field map:

rain = 'Gesamter Niederschlag', mm

4. Your Zeit (date-time) field format is fine, no need for a unix epoch timestamp, wee_import will do that provided you come up with a suitable date-time format string that wee_import can use to recognise the different portions of the date-time. This is covered by the raw_datetime_format setting in the import config file, the wee_import section of the Utilities Guide explains it's use. in your case I suspect that '%d.%m.%Y  %H:%M:%S' will do the job (untested). Again the dry run should be used to check.

Gary

[Meteo Oberwallis]

Hello Gary.

First of all, thank you for exploring the cvs file. Sounds good that you can import and save them. Now I have to know something else. you talk about adapting the wee_import? How does that work? I also wanted to ask because I didn't quite understand it with rain. Weewx needs the cumulative amount of rain. Wouldn't it be easier if I simply deleted all other rain data from the table? And the last question. You wrote that (--dry-run) can be used to test whether everything is ok. How should I do that? Sorry for the many questions. But is not quite clear ;-)

Many thanks and best regards

[William Burton]

Hallo Günther,

On Sunday, April 12, 2020 at 10:35:24 AM UTC-4, Meteo Oberwallis wrote:

Hello Gary.

First of all, thank you for exploring the cvs file. Sounds good that you can import and save them. Now I have to know something else. you talk about adapting the wee_import? How does that work? I also wanted to ask because I didn't quite understand it with rain. Weewx needs the cumulative amount of rain. Wouldn't it be easier if I simply deleted all other rain data from the table? And the last question. You wrote that (--dry-run) can be used to test whether everything is ok. How should I do that? Sorry for the many questions. But is not quite clear ;-)

See the instructions at http://www.weewx.com/docs/utilities.htm#wee_import_utility for the wee_import utility.

From your WeeWX install location such as /home/weewx, copy the file utils/import/csv-example.conf and use as a template to make the changes.

Hope this helps,

-Bill

[gjr80]

Hallo Günther,

Now I have to know something else. you talk about adapting the wee_import? How does that work?

How wee_import processes/interprets your data is controlled by the import config file. As Bill points out there is a template CSV import config file provided with WeeWX, you will find it at /home/weewx/util/import/csv-example.conf or /etc/weewx/import/csv-example.conf depending on your WeeWX install type.

I also wanted to ask because I didn't quite understand it with rain. WeeWX needs the cumulative amount of rain. Wouldn't it be easier if I simply deleted all other rain data from the table?

You can certainly delete the other rain columns in your import data if you want but there is no need, wee_import will only import data in the columns included in the field map ([[FieldMap]] stanza in the import config file) and all other columns are ignored (there is likely a small processing overhead for having the extra unused columns included but I expect this overhead would be negligible).

 

And the last question. You wrote that (--dry-run) can be used to test whether everything is ok. How should I do that?

When you run wee_import you can add the --dry-run option to the command line to run the import but any saves to your archive are skipped, so in effect the import runs as normal (as configured in you import config file) but your database is left untouched. Have a read of Option --dry-run in the Utilities Guide.

 

Sorry for the many questions. But is not quite clear ;-)

No problems, most of what you want to know is covered in the wee_import section of the Utilities Guide. You are interested in a CSV import whereas the wee_import section of the Utilities Guide covers imports from other sources (Weatherunderground, Cumulus etc) as well. You can skip these sections paying attention only to the Actions and options section and the CSV specific sections here and here.

 

If language is a problem or you just don't follow, let me know and I will put together a suitable import config file for you.

Gary

[Meteo Oberwallis]

Hello.

I have now worked exactly according to the instructions. Unfortunately I always get the same error message when importing the data. I have the relevant data as an attachment here.

wee_import --import-config=/var/tmp/csv.conf --dry-run
Starting wee_import...
A CSV import from source file '/home/pi/Desktop/EXP09_17.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb_new'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
This is a dry run, imported data will not be saved to archive.
**** Unable to map source data.
**** Field 'timestamp' not found in source data.
**** Nothing done, exiting.

Can someone help me please?
Thx

[Andrew Milner]

Your csv file does not contain a field call timestamp, which your .conf file says should be imported to dateTime.  You need to combine Datum and Zeit into a field called timestamp, and then tell wee_import the format of that field.

[Meteo Oberwallis]

Hello Andrew.

Thank you for your message. What do you mean exactly? Sorry for the question, but this is completely new.
What exactly do I have to do and where do I have to adjust it?

Thank you for your help

[rich T]

Are you viewing your csv file in excel or openoffice?  If so, add an additional column between Zeit and Temp.Temperatur. In cell C2 add the following formula: =TEXT(A2,"m/dd/yy ")& " "&TEXT(B2,"hh:mm").

[Meteo Oberwallis]

Hello.

I've done that now. Unfortunately I still get the error message that the "timestamp" is missing. wee_import --import-config=/var/tmp/csv.conf --dry-run

Starting wee_import...
A CSV import from source file '/home/pi/Desktop/EXP09_17.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb_new'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
This is a dry run, imported data will not be saved to archive.
**** Unable to map source data.
**** Field 'timestamp' not found in source data.
**** Nothing done, exiting.

[rich T]

Looks like you put all the fields in one column.

Attached is an updated CSV File with timestamp column

Attached is an updated CVS Field map.  Need to verify units are correct.

Make sure to make backup copy of database before proceeding.

Hopefully this helps.

[William Burton]

Hello,

The wee_import CSV support appears to only support a comma (,) as a field delimiter or at least it cannot detect a semi-colon (;) field delimiter so you need to take the version of the file with the timestamp fix as suggested by rich T, and from OpenOffice/LibreOffice/Excel, re-save or Save As the standard CSV format so it uses a comma delimiter.

-Bill

[Günther Wrana]

Hello

I have now adjusted the csv.conf and the datad.csv so that no warnings are given when running dry.

I can't say which units are behind the "," in [[FieldMap]] and whether they are correct.

I only entered them on suspicion and hope that they are correct.

I also entered the inside temperature although they are not shown in these NOAA files but are inside the database.

Can I actually import all the data in my CSV file that is also in the database?

 [[FieldMap]]

        dateTime    = Zeit, unix_epoch

        usUnits     =

        interval    =

        barometer   = Absoluter Luftdruck, inHg

        pressure    =

        altimeter   =

        inTemp      = Innentemperatur, degree_C

        outTemp     = Aussentemperatur, degree_C

        inHumidity  = Innenluftfeuchtigkeit, percent

        outHumidity = Aussenluftfeuchtigkeit, percent

        windSpeed   = Windgeschwindigkeit, km_per_hour

        windDir     = Windrichtung, degree_compass

        windGust    = Windboeen, km_per_hour

        windGustDir = Windrichtung, degree_compass

        rainRate    = 

        rain        = 24-Stunden-Niederschlag, mm

        dewpoint    =

        windchill   =

        heatindex   =

        ET          =

        radiation   =

        UV          =

Thanks Günther

[William Burton]

Hallo Günther,

Gary and many others have much more experience with this than myself but since I saw a few issues, thought I would reply before it becomes too late for you.

It looks like you changed the CSV file to use the Meteo export file instead of the raw data file which is a good idea. I tried to find where it was documented but was not successful.

On Tuesday, April 14, 2020 at 9:58:19 AM UTC-4, Günther Wrana wrote:

Hello

I have now adjusted the csv.conf and the datad.csv so that no warnings are given when running dry.

I can't say which units are behind the "," in [[FieldMap]] and whether they are correct.

For the Zeit column, if you look closely at the data, there are two spaces between the date and time:

1,01.05.2019  00:02:00,5, 20.9,51, 12.2,74, 985.2, 988.3, 17.3, 20.9,315, 7.7, 7.1, 0.0, 17.1, 0.0, 369.6, 706.5, 0.0,0,0

so:

    raw_datetime_format = %d.%m.%Y %H:%M:%S
might best be changed to:
    raw_datetime_format = %d.%m.%Y  %H:%M:%S

The field map:

        barometer   = Absoluter Luftdruck, inHg

should be changed to use hectopascals:
        barometer   = Absoluter Luftdruck, hPa

change:
        UV          =
to:
        UV          = UV(uW/m2)

For rain, the field specified is supposed to have a value that's reset each midnight when the "rain = cumulative" option is used. So I don't know if the 
field "24-Stunden-Niederschlag" (24-hour-precipitation) will work or not.

I only entered them on suspicion and hope that they are correct.

I also entered the inside temperature although they are not shown in these NOAA files but are inside the database.

Can I actually import all the data in my CSV file that is also in the database?

You have to make sure you make a backup of the database first if you already have the station collecting data that you don't want to lose in case something goes wrong. Note that once wee_import loads the file, if something isn't correct and you want to try again, it requires reverting to a backup database or manually deleting the data just imported as wee_import doesn't have an option to overwrite existing data. If wee_import finds existing archive table records that match the ones in the import file, it will skip over them.

Once you import the data, you need to run wee_report and then carefully check in your WeeWX skin reports and graphs to ensure the data has been imported correctly. In the case of rain, make sure to find several days and verify them.

I hesitate to list the exact steps for the import process, etc. as I'm newer to WeeWX and don't want to lead you in the wrong direction.

 [[FieldMap]]

        dateTime    = Zeit, unix_epoch

        usUnits     =

        interval    =

        barometer   = Absoluter Luftdruck, inHg

        pressure    =

        altimeter   =

        inTemp      = Innentemperatur, degree_C

        outTemp     = Aussentemperatur, degree_C

        inHumidity  = Innenluftfeuchtigkeit, percent

        outHumidity = Aussenluftfeuchtigkeit, percent

        windSpeed   = Windgeschwindigkeit, km_per_hour

        windDir     = Windrichtung, degree_compass

        windGust    = Windboeen, km_per_hour

        windGustDir = Windrichtung, degree_compass

        rainRate    = 

        rain        = 24-Stunden-Niederschlag, mm

        dewpoint    =

        windchill   =

        heatindex   =

        ET          =

        radiation   =

        UV          =

Thanks Günther

Hope this helps,

-Bill

[Meteo Oberwallis]

Hello.

I was now able to retroactively import my data into weewx. It worked wonderfully. Now I have another question. 
After I have written the data into the database, I have to execute another command so that it is also on the homepage, so. e.g. in Belchertown under "Reports"? So far nothing is there, not even after midnight.

Thanks

[Günther Wrana]

Hallo

hast du die ganze 2MB csv auf einmal in die Datenbank gespielt?

Mit all den Daten und in der Konfiguration?

Grüße Günther

[Meteo Oberwallis]

Hallo.

Ich mache mittlerweile Jahres Daten und lade das so hoch. Dies klappt, dauert aber rund 3-4 Stunden pro Jahr. Die csv hat dann bei mir rund 24 MB. Leider muss man eine neue Spalte "timestamp" machen, da es sonst nicht klappt.

Gruss Stefan

[Günther Wrana]

Hallo

Bei mir funktioniert das Importieren nur im Trockenlauf. Wenn ich die .csv Datei dann bei gestopptem weewx importiere und weewx wieder starte funktioniert nichts mehr. Es werden folgende Zeilen im log Feil angezeigt. Wenn ich dann die gesicherte Datenbank wieder einspiele und weewx neustarte funktioniert es wieder. Was mache ich falsch oder habe ich übersehen?

Danke Günther

Hello

For me, the import only works in dry running. If I import the .csv file with weewx stopped and weewx start again, nothing works. The following lines are displayed in the log file. If I then import the saved database again and restart weewx, it works again. What am I doing wrong or have I overlooked?

Thanks Günther

[Günther Wrana]

Hello

Maybe you can do more with that than with the log file.

Thanks Günther

daham@hp-pc:~$ ssh p...@192.168.1.104

p...@192.168.1.104's password: 

Linux raspberrypi 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l

The programs included with the Debian GNU/Linux system are free software;

the exact distribution terms for each program are described in the

individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent

permitted by applicable law.

Last login: Wed Apr 15 17:21:47 2020 from 192.168.1.103

pi@raspberrypi:~ $ cd /home/weewx/archive

pi@raspberrypi:/home/weewx/archive $ dir

weewx.sdb

pi@raspberrypi:/home/weewx/archive $ sudo /etc/init.d/weewx stop

[ ok ] Stopping weewx (via systemctl): weewx.service.

pi@raspberrypi:/home/weewx/archive $ sudo cp weewx.sdb /var/tmp

pi@raspberrypi:/home/weewx/archive $ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/csv.conf --dry-run

Starting wee_import...

A CSV import from source file '/var/tmp/datad.csv' has been requested.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

This is a dry run, imported data will not be saved to archive.

Starting dry run import ...

288 records identified for import.

Records processed: 250; Unique records: 250; Last timestamp: 2019-05-01 20:47:00Records processed: 288; Unique records: 288; Last timestamp: 2019-05-01 23:57:00 CEST (1556747820)

Finished dry run import. 288 records were processed and 288 unique records would have been imported.

pi@raspberrypi:/home/weewx/archive $ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/csv.conf

Starting wee_import...

A CSV import from source file '/var/tmp/datad.csv' has been requested.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

Starting import ...

288 records identified for import.

Proceeding will save all imported records in the WeeWX archive.

Are you sure you want to proceed (y/n)? y

Records processed: 288; Unique records: 288; Last timestamp: 2019-05-01 23:57:00 CEST (1556747820)

Finished import. 288 raw records resulted in 288 unique records being processed in 10.23 seconds.

Those records with a timestamp already in the archive will not have been

imported. Confirm successful import in the WeeWX log file.

pi@raspberrypi:/home/weewx/archive $ sudo /etc/init.d/weewx start

[ ok ] Starting weewx (via systemctl): weewx.service.

pi@raspberrypi:/home/weewx/archive $ sudo tail -f /var/log/syslog

Apr 15 20:14:35 raspberrypi weewx[14619]: wh23xx: Found device on USB bus= device=

Apr 15 20:14:35 raspberrypi kernel: [10925.373718] usb 1-1.5: reset full-speed USB device number 4 using dwc_otg

Apr 15 20:14:35 raspberrypi weewx[14619]: engine: StdConvert target unit is 0x1

Apr 15 20:14:35 raspberrypi weewx[14619]: wxcalculate: The following values will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, heatindex=prefer_hardware, maxSolarRad=prefer_hardware, humidex=prefer_hardware, pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, cloudbase=prefer_hardware

Apr 15 20:14:35 raspberrypi weewx[14619]: wxcalculate: The following algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS

Apr 15 20:14:35 raspberrypi weewx[14619]: engine: Archive will use data binding wx_binding

Apr 15 20:14:35 raspberrypi weewx[14619]: engine: Record generation will be attempted in 'hardware'

Apr 15 20:14:35 raspberrypi weewx[14619]: engine: Using archive interval of 300 seconds (specified in weewx configuration)

Apr 15 20:14:35 raspberrypi weewx[14619]: engine: Using binding 'wx_binding' to database 'weewx.sdb'

Apr 15 20:14:35 raspberrypi weewx[14619]: manager: Starting backfill of daily summaries

Apr 15 20:15:57 raspberrypi weewx[14619]: engine: Caught unrecoverable exception in engine:

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****  accum: ScalarStats.addHiLo expected float or int, got type '<type 'unicode'>' ('')

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****  Traceback (most recent call last):

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/engine.py", line 888, in main

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      engine = engine_class(config_dict)

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/engine.py", line 78, in __init__

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      self.loadServices(config_dict)

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/engine.py", line 142, in loadServices

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/engine.py", line 500, in __init__

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      self.setup_database(config_dict)

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/engine.py", line 617, in setup_database

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      _nrecs, _ndays = dbmanager.backfill_day_summary() # @UnusedVariable

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/manager.py", line 1473, in backfill_day_summary

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      day_accum.addRecord(rec, weight=weight)

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/accum.py", line 260, in addRecord

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      func(self, record, obs_type, add_hilo, weight)

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/accum.py", line 318, in add_value

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      self[obs_type].addHiLo(val, record['dateTime'])

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****    File "/home/weewx/bin/weewx/accum.py", line 78, in addHiLo

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****      "got type '%s' ('%s')" % (type(val), val))

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****  ValueError: accum: ScalarStats.addHiLo expected float or int, got type '<type 'unicode'>' ('')

Apr 15 20:15:57 raspberrypi weewx[14619]:     ****  Exiting.

^C

pi@raspberrypi:/home/weewx/archive $ sudo /etc/init.d/weewx stop

[ ok ] Stopping weewx (via systemctl): weewx.service.

pi@raspberrypi:/home/weewx/archive $ cd /var/tmp

pi@raspberrypi:/var/tmp $ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/csv.conf

Starting wee_import...

A CSV import from source file '/var/tmp/datad.csv' has been requested.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

Starting import ...

288 records identified for import.

Proceeding will save all imported records in the WeeWX archive.

Are you sure you want to proceed (y/n)? y

Records processed: 288; Unique records: 288; Last timestamp: 2019-05-01 23:57:00 CEST (1556747820)

Finished import. 288 raw records resulted in 288 unique records being processed in 4.05 seconds.

Those records with a timestamp already in the archive will not have been

imported. Confirm successful import in the WeeWX log file.

pi@raspberrypi:/var/tmp $ sudo /etc/init.d/weewx start

[ ok ] Starting weewx (via systemctl): weewx.service.

pi@raspberrypi:/var/tmp $ sudo tail -f /var/log/syslog

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****    File "/home/weewx/bin/weewx/manager.py", line 1473, in backfill_day_summary

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****      day_accum.addRecord(rec, weight=weight)

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****    File "/home/weewx/bin/weewx/accum.py", line 260, in addRecord

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****      func(self, record, obs_type, add_hilo, weight)

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****    File "/home/weewx/bin/weewx/accum.py", line 318, in add_value

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****      self[obs_type].addHiLo(val, record['dateTime'])

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****    File "/home/weewx/bin/weewx/accum.py", line 78, in addHiLo

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****      "got type '%s' ('%s')" % (type(val), val))

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****  ValueError: accum: ScalarStats.addHiLo expected float or int, got type '<type 'unicode'>' ('')

Apr 15 20:17:07 raspberrypi weewx[14938]:     ****  Exiting.

^C

pi@raspberrypi:/var/tmp $ sudo /etc/init.d/weewx reload

[ ok ] Reloading weewx configuration (via systemctl): weewx.service.

pi@raspberrypi:/var/tmp $ sudo /etc/init.d/weewx restart

[ ok ] Restarting weewx (via systemctl): weewx.service.

pi@raspberrypi:/var/tmp $ sudo tail -f /var/log/syslog

Apr 15 20:19:17 raspberrypi weewx[15205]: wh23xx: Found device on USB bus= device=

Apr 15 20:19:17 raspberrypi kernel: [11207.753810] usb 1-1.5: reset full-speed USB device number 4 using dwc_otg

Apr 15 20:19:17 raspberrypi weewx[15205]: engine: StdConvert target unit is 0x1

Apr 15 20:19:17 raspberrypi weewx[15205]: wxcalculate: The following values will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, heatindex=prefer_hardware, maxSolarRad=prefer_hardware, humidex=prefer_hardware, pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, cloudbase=prefer_hardware

Apr 15 20:19:17 raspberrypi weewx[15205]: wxcalculate: The following algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS

Apr 15 20:19:17 raspberrypi weewx[15205]: engine: Archive will use data binding wx_binding

Apr 15 20:19:17 raspberrypi weewx[15205]: engine: Record generation will be attempted in 'hardware'

Apr 15 20:19:17 raspberrypi weewx[15205]: engine: Using archive interval of 300 seconds (specified in weewx configuration)

Apr 15 20:19:17 raspberrypi weewx[15205]: engine: Using binding 'wx_binding' to database 'weewx.sdb'

Apr 15 20:19:17 raspberrypi weewx[15205]: manager: Starting backfill of daily summaries

Apr 15 20:19:22 raspberrypi weewx[15205]: engine: Caught unrecoverable exception in engine:

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****  accum: ScalarStats.addHiLo expected float or int, got type '<type 'unicode'>' ('')

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****  Traceback (most recent call last):

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/engine.py", line 888, in main

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      engine = engine_class(config_dict)

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/engine.py", line 78, in __init__

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      self.loadServices(config_dict)

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/engine.py", line 142, in loadServices

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/engine.py", line 500, in __init__

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      self.setup_database(config_dict)

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/engine.py", line 617, in setup_database

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      _nrecs, _ndays = dbmanager.backfill_day_summary() # @UnusedVariable

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/manager.py", line 1473, in backfill_day_summary

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      day_accum.addRecord(rec, weight=weight)

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/accum.py", line 260, in addRecord

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      func(self, record, obs_type, add_hilo, weight)

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/accum.py", line 318, in add_value

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      self[obs_type].addHiLo(val, record['dateTime'])

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/accum.py", line 78, in addHiLo

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      "got type '%s' ('%s')" % (type(val), val))

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****  ValueError: accum: ScalarStats.addHiLo expected float or int, got type '<type 'unicode'>' ('')

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****  Exiting.

^C

pi@raspberrypi:/var/tmp $ sudo tail -f /var/log/syslog

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/manager.py", line 1473, in backfill_day_summary

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      day_accum.addRecord(rec, weight=weight)

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/accum.py", line 260, in addRecord

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      func(self, record, obs_type, add_hilo, weight)

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/accum.py", line 318, in add_value

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      self[obs_type].addHiLo(val, record['dateTime'])

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****    File "/home/weewx/bin/weewx/accum.py", line 78, in addHiLo

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****      "got type '%s' ('%s')" % (type(val), val))

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****  ValueError: accum: ScalarStats.addHiLo expected float or int, got type '<type 'unicode'>' ('')

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****  Exiting.

^C

pi@raspberrypi:/var/tmp $ sudo /etc/init.d/weewx stop

[ ok ] Stopping weewx (via systemctl): weewx.service.

pi@raspberrypi:/var/tmp $ sudo cp /home/weewx/archive/weewx.sdb /tmp/

pi@raspberrypi:/var/tmp $ sudo cp weewx.sdb /home/weewx/archive

pi@raspberrypi:/var/tmp $ sudo /etc/init.d/weewx start

[ ok ] Starting weewx (via systemctl): weewx.service.

pi@raspberrypi:/var/tmp $ sudo tail -f /var/log/syslog

Apr 15 20:23:19 raspberrypi weewx[15651]: engine: Using binding 'wx_binding' to database 'weewx.sdb'

Apr 15 20:23:19 raspberrypi weewx[15651]: manager: Starting backfill of daily summaries

Apr 15 20:23:19 raspberrypi weewx[15651]: restx: StationRegistry: Station will be registered.

Apr 15 20:23:19 raspberrypi weewx[15651]: restx: Wunderground-PWS: Data for station IMATZENR2 will be posted

Apr 15 20:23:19 raspberrypi weewx[15651]: restx: PWSweather: Posting not enabled.

Apr 15 20:23:19 raspberrypi weewx[15651]: restx: CWOP: Posting not enabled.

Apr 15 20:23:19 raspberrypi weewx[15651]: restx: WOW: Posting not enabled.

Apr 15 20:23:19 raspberrypi weewx[15651]: restx: AWEKAS: Data will be uploaded for user Gilli

Apr 15 20:23:19 raspberrypi weewx[15651]: engine: Starting up weewx version 3.9.2

Apr 15 20:23:19 raspberrypi weewx[15651]: engine: Starting main packet loop.

Apr 15 20:24:13 raspberrypi systemd[1]: Started Session c3 of user pi.

Apr 15 20:24:14 raspberrypi systemd[1]: Started Session c4 of user pi.

Apr 15 20:25:20 raspberrypi weewx[15651]: manager: Added record 2020-04-15 20:25:00 CEST (1586975100) to database 'weewx.sdb'

Apr 15 20:25:20 raspberrypi weewx[15651]: manager: Added record 2020-04-15 20:25:00 CEST (1586975100) to daily summary in 'weewx.sdb'

Apr 15 20:25:22 raspberrypi weewx[15651]: restx: AWEKAS: Failed to publish record 2020-04-15 20:25:00 CEST (1586975100): server returned ' - UV Wert administrativ gesperrt  - Transfer OK'

Apr 15 20:25:23 raspberrypi weewx[15651]: restx: StationRegistry: Published record 2020-04-15 20:25:00 CEST (1586975100)

Apr 15 20:25:24 raspberrypi weewx[15651]: restx: Wunderground-PWS: Published record 2020-04-15 20:25:00 CEST (1586975100)

Apr 15 20:25:28 raspberrypi weewx[15651]: cheetahgenerator: Generated 9 files for report SeasonsReport in 5.58 seconds

Apr 15 20:25:46 raspberrypi weewx[15651]: reportengine: Caught unrecoverable exception in generator 'weewx.imagegenerator.ImageGenerator'

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****  unsupported operand type(s) for -: 'list' and 'float'

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****  Traceback (most recent call last):

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 204, in run

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****      obj.start()

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 300, in start

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****      self.run()

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****    File "/home/weewx/bin/weewx/imagegenerator.py", line 33, in run

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****      self.genImages(self.gen_ts)

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****    File "/home/weewx/bin/weewx/imagegenerator.py", line 188, in genImages

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****      new_data_vec_t = self.converter.convert(data_vec_t)

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****    File "/home/weewx/bin/weewx/units.py", line 784, in convert

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****      new_val_t = convert(val_t, new_unit_type)

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****    File "/home/weewx/bin/weewx/units.py", line 1138, in convert

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****      new_val = conversion_func(val_t[0]) if val_t[0] is not None else None

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****    File "/home/weewx/bin/weewx/units.py", line 33, in FtoC

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****      return (x - 32.0) * 5.0 / 9.0

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****  TypeError: unsupported operand type(s) for -: 'list' and 'float'

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****  Generator terminated

Apr 15 20:25:46 raspberrypi weewx[15651]: copygenerator: copied 34 files to /var/weewx/reports

^C

pi@raspberrypi:/var/tmp $ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/csv.conf --dry-run

Starting wee_import...

A CSV import from source file '/var/tmp/datad.csv' has been requested.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'

Destination table 'archive' unit system is '0x01' (US).

Missing derived observations will be calculated.

This is a dry run, imported data will not be saved to archive.

Starting dry run import ...

288 records identified for import.

Records processed: 288; Unique records: 288; Last timestamp: 2019-05-01 23:57:00 CEST (1556747820)

Finished dry run import. 288 records were processed and 288 unique records would have been imported.

pi@raspberrypi:/var/tmp $ 

[gjr80]

Hallo Günther,

I have this morning used your csv.conf and datad.csv with wee_import under WeeWX v3.9.2 and the import works fine, though your barometer (Absoluter Luftdruck) source data is in hPa/mbar not inHg so need to change the barometer field map under [[FieldMap]] in your csv.conf as follows:

        barometer   = Absoluter Luftdruck, hPa

As for the errors in your syslog. The error:

Apr 15 20:19:22 raspberrypi weewx[15205]:     ****  ValueError: accum: ScalarStats.addHiLo expected float or int, got type '<type 'unicode'>' ('')

looks like somehow you have ended up with a unicode null string in your archive. That could have happened with some failed imports. Running wee_database with --check-strings followed by --fix-strings may fix the issue.

I gather you then reverted to a previous database backup that now gives the following error:

Apr 15 20:25:46 raspberrypi weewx[15651]:         ****  TypeError: unsupported operand type(s) for -: 'list' and 'float'

I am a bit stumped with this, the trace seems to imply val_t[0] is not a list but then the error states it is a list. Perhaps Tom can help with this one.

Gary

[Günther Wrana]

Hello Gary

But when I run wee_database --fix-strings the following appears.

I also stopped weewx before.

pi@raspberrypi:/var/tmp $ sudo /home/weewx/bin/wee_database --fix-strings

Using configuration file /home/weewx/weewx.conf

Using database binding 'wx_binding', which is bound to database 'archive_sqlite'

Preparing Null String Fix, this may take a while...

^CTraceback (most recent call last):019-11-09 12:50:00 CET (1573300200))

  File "/home/weewx/bin/wee_database", line 794, in <module>

    main()

  File "/home/weewx/bin/wee_database", line 171, in main

    check_strings(config_dict, db_binding, options, fix=True)

  File "/home/weewx/bin/wee_database", line 669, in check_strings

    dbmanager.updateValue(record[0], obs_list[icol], corrected_value)

  File "/home/weewx/bin/weewx/manager.py", line 382, in updateValue

    (self.table_name, obs_type), (new_value, timestamp))

  File "/home/weewx/bin/weedb/sqlite.py", line 29, in guarded_fn

    return fn(*args, **kwargs)

  File "/home/weewx/bin/weedb/sqlite.py", line 138, in execute

    self.connection.execute(sql_string, sql_tuple)

KeyboardInterrupt

After a few minutes I canceled with CTRL + C. Is it normal that this takes so long or is there an error in my database that I have to correct manually.

I tried a few times but it always stays on the same line.

Thanks Günther

[gjr80]

Hello Günther,

Did you try —check-strings first? And to be fair when you run —fix-strings it does say on the console ‘Preparing Null String Fix, this may take a while...’ :)

Gary

[Günther Wrana]

Hello Gary

This is the answer when i start --check-strings. This is the answer when i start --check-strings.

32972 null strings were found.

Recommend running --fix-strings to fix these strings.

Completed Null String Check in 7.59 seconds.

I'm curious how long it will take.

Thanks Günther

[gjr80]

Hallo Günther,

So there are definitely some to fix. —fix-strings will work through every archive record and then every column in each record so it may take while. Many factors affect how long it will take. You should see a progress message on screen saying how many records have been checked and what timestamp —fix-strings is up to. This message updates every 1000 records processed. Does that update message not appear?

Gary

[Günther Wrana]

Hello Gary

Yes, it is displayed, but with these two it stops.

Checking record: 41000; Timestamp: 2019-11-09 12:50:00 CET (1573300200))

Thanks Günther

[gjr80]

Hallo Günther,

I would leave it go for a while, it could be that at first it finds no null strings but then comes across a large number. Once a null string is found a corrected value is written to archive and these writes do slow things down.

Gary

[Meteo Oberwallis]

Hallo Günther.

Hast du mal probiert, in der csv.conf die Oprion "conf" anstatt der Option "derive" zu nehmen. Das hat bei mir die Lösung gebracht. Auch habe ich bemerkt, das man nicht mehr Daten als ein halbes Jahr am Stück hochladen sollte. 6 Monate ergeben ca. 23 MB. Das hält das System noch aus. Alles was länger ist kann zu Problemen führen. Ich habe 45 Systeme am laufen. 4 sind jetzt Offline. Leider. Ich habe überall den Teamviewer am laufen. Das erleichtert die Sachen zum hin und her schieben.

Gruss

Stefan

[Günther Wrana]

Hallo Leute

nach weiß ich nicht genau wie vielen Minuten oder ein zwei Stunden war das --fix-strings fertig. Nun sind auch die Fehlermeldungen beim erstellen neuer Reports weg. Ich werde es dann noch versuchen ob es nun funktioniert mit dem importieren von .csv Dateien aber danke vorerst einmal.

Günther

Hi Guys

after i don't know exactly how many minutes or an hour the --fix-strings was done. Now the error messages when creating new reports are gone. I will then try to import .csv files but thanks for now.

Günther

[Günther Wrana]

Hallo Leute

ich habe es geschafft das ich meine .csv Datei in meine Datenbank importiert habe und weewx nach dem Import auch weiter läuft.

Nun würde ich noch gerne Wissen den das habe ich in der ganzen Dokumentation von weewx nicht heraus gefunden.

Wie bringe ich weewx dazu die NOAA Datein neu zu erstellen.

Den dann wen ich das richtig verstehe müsste in der neue NOAA Datei vom 2019-05 ja der 01 Tag ja ach dabei sein weil ich ja nur den 01.05.2019 in meine Datenbank dazu importiert haben.

Habe ich da richtig gedacht oder fehlt mir da noch etwas um diese NOAA Datei zu vervollständigen?

Danke schön Günther

hi Guys

I have managed to import my .csv file into my database and weewx continues to run after the import.

Now I would like to know that I have not found out in the entire weewx documentation.

How do I get weewx to recreate the NOAA files.

Then who I understand correctly should be in the new NOAA file from 2019-05 yes the 01 day yes oh because I only imported the 01.05.2019 into my database.

Did I think correctly or am I missing something to complete this NOAA file?

Thank you Günther

[gjr80]

Hallo Günther,

Good to hear you have imported your data and WeeWX is running without error. You need WeeWX to recreate all of your NOAA format reports, to do this just delete the reports on your WeeWX machine and they will be recreated (including your earlier ones) on the next report cycle.

Gary

[Meteo Oberwallis]

Hello.
I have now deleted the data in the folder "/var/www/html/weewx/NOAA/". These came again after the next interval. Unfortunately without data. Do I have to delete it anywhere else?
Thank you for your top help

[Günther Wrana]

Hello

I deleted the file NOAA-2019-05.txt, NOAA-2019.txt and it worked perfectly. For me is now the 01.05.2015 that I imported. The fact that there is nothing inside has happened to me even though I have only deleted the NOAA.txt. Maybe you have to restart or reload weewx.

Günther

[Schnidrig Stefan]

Hallo Günther

In welchem Ordner hast du das gelöscht? 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8fc609f0-224a-4205-98d4-410a7acc759f%40googlegroups.com.

[gjr80]

Hallo Stefan,

Sie müssen die dateien von dem speicherort löschen, an dem WeeWX sie erstellt hat (im gegensatz zu dem speicherort, an dem WeeWX sie möglicherweise kopiert hat). Dieser speicherort hängt von Ihrem WeeWX installationstyp ab. Sofern sie es nicht geändert haben, ist dies /home/weewx/public_html/NOAA oder /var/www/html/weewx/NOAA.

Wenn dies nicht dazu führt, dass Ihre importierten daten angezeigt werden, müssen wir Ihre datenbank genauer untersuchen, um zu überprüfen, ob die daten tatsächlich importiert wurden.

You need to delete the files from the location where WeeWX created them (as distinct from where WeeWX may have copied them). This location depends on your WeeWX install type. Unless you have changed it this will be /home/weewx/public_html/NOAA or /var/www/html/weewx/NOAA.

If this does not cause your imported data to appear then we need to look further at your database to check that the data was in fact imported.

Gary

On Sunday, 19 April 2020 01:45:24 UTC+10, Meteo Oberwallis wrote:

Hallo Günther

In welchem Ordner hast du das gelöscht? 

On Sat, Apr 18, 2020 at 5:34 PM +0200, "Günther Wrana" <kawa...@gmail.com> wrote:

Hello

I deleted the file NOAA-2019-05.txt, NOAA-2019.txt and it worked perfectly. For me is now the 01.05.2015 that I imported. The fact that there is nothing inside has happened to me even though I have only deleted the NOAA.txt. Maybe you have to restart or reload weewx.

Günther

Am Samstag, 18. April 2020 14:56:21 UTC+2 schrieb Meteo Oberwallis:

Hello.
I have now deleted the data in the folder "/var/www/html/weewx/NOAA/". These came again after the next interval. Unfortunately without data. Do I have to delete it anywhere else?
Thank you for your top help

Am Freitag, 17. April 2020 23:42:39 UTC+2 schrieb gjr80:

Hallo Günther,

Good to hear you have imported your data and WeeWX is running without error. You need WeeWX to recreate all of your NOAA format reports, to do this just delete the reports on your WeeWX machine and they will be recreated (including your earlier ones) on the next report cycle.

Gary

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[Günther Wrana]

Hallo

Ich habe die Dateien im /var/weewx/reports/NOAA gelöscht aber ich habe diesen Pfad bei mir vorher so geändert damit diese Dateien nicht auf die SD-Karte sondern in den RAM Speicher geschrieben werden.

Grüße Günther

Hello

I deleted the files in /var/weewx/reports/NOAA but I changed this path for me beforehand so that these files are not written to the SD card but to the RAM memory.

Regards Günther

Am Samstag, 18. April 2020 17:45:24 UTC+2 schrieb Meteo Oberwallis:

Hallo Günther

In welchem Ordner hast du das gelöscht? 

On Sat, Apr 18, 2020 at 5:34 PM +0200, "Günther Wrana" <kawa...@gmail.com> wrote:

Hello

I deleted the file NOAA-2019-05.txt, NOAA-2019.txt and it worked perfectly. For me is now the 01.05.2015 that I imported. The fact that there is nothing inside has happened to me even though I have only deleted the NOAA.txt. Maybe you have to restart or reload weewx.

Günther

Am Samstag, 18. April 2020 14:56:21 UTC+2 schrieb Meteo Oberwallis:

Hello.
I have now deleted the data in the folder "/var/www/html/weewx/NOAA/". These came again after the next interval. Unfortunately without data. Do I have to delete it anywhere else?
Thank you for your top help

Am Freitag, 17. April 2020 23:42:39 UTC+2 schrieb gjr80:

Hallo Günther,

Good to hear you have imported your data and WeeWX is running without error. You need WeeWX to recreate all of your NOAA format reports, to do this just delete the reports on your WeeWX machine and they will be recreated (including your earlier ones) on the next report cycle.

Gary

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

[sschn...@bluewin.ch]

Hallo Günther

 

Nun. Das finde ich so nirgends in meinem weewx Ordner. Ist das der Ordner, bevor es via FTP hochgeladen wird?

 

Gruss

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f02b56bc-8550-45f7-95ae-3e58054284d5%40googlegroups.com.

[Günther Wrana]

Hallo

wenn du die Standard Installation hast ist das /home/weewx/public_html/NOAA oder /var/www/html/weewx/NOAA so wie es Gary geschrieben hat.

Grüße

Hello

if you have the standard installation this is /home/weewx/public_html/NOAA or /var/www/html/weewx/NOAA as Gary wrote it.

greetings

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f02b56bc-8550-45f7-95ae-3e58054284d5%40googlegroups.com.

[Meteo Oberwallis]

Good Morning

I have a question regarding the SQLite database. I updated weewx and now use sunshine_time instead of sunshine_hours. Now of course the problem is that the old sunshine times are in the sunshine_hours column and not in the sunshine_time. How can I fix the fact that the old sunshine_hours data is now being sent to the sunshine_time splate?

Thanks and Greetings

[Tom Keffer]

If I understand correctly, you're trying to rename a database type from sunshine_hours to sunshine_time.

If so, use the utility wee_database with the --rename-column option:

wee_database --rename-column=sunshine_hours --to-name=sunshine_time

Make a backup first.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f7df793b-6e6b-43f8-95e8-799af2218b3en%40googlegroups.com.

[sschn...@bluewin.ch]

Hey Tom.

 

Thx for your Help.

In the new weewx version, "schema = schemas.wview_extended.schema" is used with the table. In the previous version of weewx, however, this was "schema = user.radiationhours.schema_with_sunshine_hours". What exactly do I have to do if I transfer my old weewx.sdb to the program? Do I then have to adapt these with sudo wee_database weewx.conf --reconfigure to the ""schema = schemas.wview_extended.schema"? I think that's where my problem lies.

 

Currently I did the following: Old weewx.sdb in the /var/lib/weewx/ folder. Then adjust the column names with "wee_database --rename-column=sunshine_hours --to-name=sunshine_time" and finally adjust the new Shmea from weewx with sudo wee_database weewx.conf --reconfigure. After I did that, the "sunshine_time" column was gone. Only after "sudo wee_database --add-column=sunshine_time" is it back, but unfortunately without all the data from the last few years.

 

Where is my mistake?

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/86KPd-dyvD4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEBx1WzUgoiV%2Bj07vMVqW7seYe6jM-3WCmks2ZmREeVmCQ%40mail.gmail.com.

[Meteo Oberwallis]

Hello, everyone.

The error really seems to be with the database, i.e. SQLite, since the "user.sunduration.schema_with_sunshine_time" will have a different structure. But what can I do now so that I can use my old database from the previous weewx version again with the latest version and thus overwrite the sunshine hours (old name in the old weewx.sdb) in the new one?

Many thanks for your help

[Meteo Oberwallis]

Hallo zusammen.
Ich probiere nun schon seit geraumer Zeit, die weewx.sdb der Version 3.9.0 mit der neuen weewx.sdb der aktuellen Version 4.7.0 zusammen zu legen. Dies klappte bis anhin leider nicht. Nun vermute ich aber, das dies daran liegt, das ab der Version 4.7 ein anderes Shema benutzt wird? Interessanterweise konnte ich die alte weewx.sdb mit wee_database weewx.conf --reconfigure in das richtige format bringen. Nun aber fehlen mit die "sunshine_hours". Die neuste Version von weewx hat ja ebenfalls ein neues "sunshine_time" format. Nun wollte die mittels "rename" einfach die Tabllenespalte anpassen. Dies blieb aber erfolglos. Kann ich die "sunshine_hours" über einen anderen weg ind die neue weewx.sdb in dessen "sunshine_time" spalte kopieren?

Ich weiss wirklich nicht mehr weiter und würde mich über Hilfen freuen. Vielen Dank

Der Fehler bei der Datenbank zusammenführung:

Error: near line 333827: table archive has 114 columns but 53 values were supplied
Error: near line 333828: table archive has 114 columns but 53 values were supplied
Error: near line 333829: table archive has 114 columns but 53 values were supplied
Error: near line 333830: table archive has 114 columns but 53 values were supplied
Error: near line 333831: table archive has 114 columns but 53 values were supplied
Error: near line 333832: table archive has 114 columns but 53 values were supplied
Error: near line 333833: table archive has 114 columns but 53 values were supplied
Error: near line 333834: table archive has 114 columns but 53 values were supplied
Error: near line 333835: table archive has 114 columns but 53 values were supplied
Error: near line 333836: table archive has 114 columns but 53 values were supplied
Error: near line 333837: table archive has 114 columns but 53 values were supplied
Error: near line 333838: table archive has 114 columns but 53 values were supplied
Error: near line 333839: table archive has 114 columns but 53 values were supplied
Error: near line 333840: table archive has 114 columns but 53 values were supplied
Error: near line 333841: table archive has 114 columns but 53 values were supplied
Error: near line 333842: table archive has 114 columns but 53 values were supplied
Error: near line 333843: table archive has 114 columns but 53 values were supplied
Error: near line 333844: table archive has 114 columns but 53 values were supplied
Error: near line 333845: table archive has 114 columns but 53 values were supplied
Error: near line 333846: table archive has 114 columns but 53 values were supplied
Error: near line 333847: table archive has 114 columns but 53 values were supplied
Error: near line 333848: table archive has 114 columns but 53 values were supplied
Error: near line 333849: table archive has 114 columns but 53 values were supplied
Error: near line 333850: table archive has 114 columns but 53 values were supplied
Error: near line 333851: table archive has 114 columns but 53 values were supplied
Error: near line 333852: table archive has 114 columns but 53 values were supplied

Hello, everyone.
For some time now I've been trying to merge the weewx.sdb of version 3.9.0 with the new weewx.sdb of the current version 4.7.0. Unfortunately, this has not worked so far. But now I assume that this is due to the fact that a different shema is used from version 4.7 onwards? Interestingly, I was able to get the old weewx.sdb into the right format with wee_database weewx.conf --reconfigure. But now I'm missing the "sunshine_hours". The latest version of weewx also has a new "sunshine_time" format. Now she just wanted to adjust the table column using "rename". But this was unsuccessful. Can I use another way to copy the "sunshine_hours" into the new weewx.sdb in its "sunshine_time" column?
I really don't know what to do and would appreciate any help. Thanks very much

[Meteo Oberwallis]

Hallo.

Ich möchte die Sonnenscheinzeit meiner Wetterstation vom Jahr 2006 ab nachführen in weewx. In der weewx.sdb befindet sich die Spalte "sunshine_time". Nun habe ich probiert, die Daten hoch zu laden über folgenden Code:

    [[FieldMap]]
        dateTime    = Zeit, unix_epoch
        usUnits     =
        interval    =
        barometer   = Luftdruck, hPa
        pressure    =
        altimeter   =
        inTemp      =
        outTemp     = Temp. A., degree_C
        inHumidity  =
        outHumidity = Feuchte A., percent
        windSpeed   = Wind, km_per_hour
        windDir     = Richtung, degree_compass
        windGust    = Windboe, km_per_hour
        windGustDir =
        rainRate    =
        rain        = Regen, mm
        dewpoint    =
        windchill   =
        heatindex   =
        ET          =
        radiation   = Solar, watt_per_meter_squared
        UV          = UV, uv_index
        sunshine_time = Sonnenschein, hour

Leider ergibt dies immer einen Fehler:

/home/technik# sudo wee_import --import-config=/var/tmp/csv.conf
Using WeeWX configuration file /etc/weewx/weewx.conf
Starting wee_import...
A CSV import from source file '/home/technik/Schreibtisch/Grächen/EXP_2006.CSV' has been requested.

Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.

All WeeWX UV fields will be set to None.
Starting import ...
Warning: Import field 'UV' is mapped to WeeWX field 'UV' but the
         import field 'UV' could not be found in one or more records.
         WeeWX field 'UV' will be set to 'None' in these records.

Traceback (most recent call last):

  File "/usr/share/weewx/wee_import", line 899, in <module>
    main()
  File "/usr/share/weewx/wee_import", line 829, in main
    source_obj.run()
  File "/usr/share/weewx/weeimport/weeimport.py", line 410, in run
    _mapped_data = self.mapRawData(_raw_data, self.archive_unit_sys)
  File "/usr/share/weewx/weeimport/weeimport.py", line 941, in mapRawData
    weewx.units.obs_group_dict[_field])
  File "/usr/share/weewx/weeutil/weeutil.py", line 1501, in __getitem__
    return self.__missing__(key)
  File "/usr/share/weewx/weeutil/weeutil.py", line 1493, in __missing__
    raise KeyError(key)
KeyError: u'sunshine_time'

Woran könnte das liegen?

Danke für eure Hilfe