Transfering db from Python 2 to 3 with sqlite ...

[Yves Martin]

Hi,

I've upgraded the Debian distribution (from version 7 to 10) and Python 2.7 to 3.8.

I've copied the sqlite database in /var/lib/weewx/ to the new distribution.

Unfortunately, when I start weewx, I have this error :

Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/manager.py", line 130, in __init__
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****      self.sqlkeys = self.connection.columnsOf(self.table_name)
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****    File "/usr/share/weewx/weedb/sqlite.py", line 187, in columnsOf
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****      column_list = [row[1] for row in self.genSchemaOf(table)]
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****    File "/usr/share/weewx/weedb/sqlite.py", line 187, in <listcomp>
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****      column_list = [row[1] for row in self.genSchemaOf(table)]
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****    File "/usr/share/weewx/weedb/sqlite.py", line 176, in genSchemaOf
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****      for row in self.connection.execute("""PRAGMA table_info(%s);""" % table):
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****  sqlite3.DatabaseError: database disk image is malformed
Jul 04 15:17:09 weewx4-raspi python3[377]: weewx[377] CRITICAL __main__:     ****  Exiting.

I cannot find here this kind of error.

Any help or suggestion will be appreciated.

Regards,

Yves

YMartin.com/meteo

[p q]

The error says the disk image is malformed. Can you open the database file with another tool, such as DB Browser or SQLite itself?

If it's true and the database is messed up, then you may need to get a back up if you have one.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/61339c2c-7deb-4c69-88c8-f2e3a0edcfd2n%40googlegroups.com.

--

Peter Quinn
(415)794-2264

[Tom Keffer]

I'd say somewhere along the way your sqlite file got damaged. Did you change from 32-bit to 64-bit when you did the OS upgrade?

Do you have a backup?

If not, you may be able to use the ".recover" command in the utility sqlite3. See https://www.sqlite.org/cli.html#recover_data_from_a_corrupted_database

-tk

On Tue, Jul 4, 2023 at 2:04 PM Yves Martin <ve2...@gmail.com> wrote:

--

[Yves Martin]

Hi Tom,

I use the 32 bit distribution (the version of Raspberry Pi 2 B I have do not support the 64 bits).

In fact I have 2 raspberry Pi 2B, one is working live for now with the old version of Debian distribution, under Python 2.7 and another one I'm using for the upgrade with the new distribution and using Python 3.7.3 ...

What I'm doing is copying the sqlite database in /var/lib/weewx/weewx.sdb from the old board to the new one ... using "mc". I love this tool, this is really usefull.

Of course, before I stop weewx. It seems the database is corrupted when I copy it, but don't worry, this is just the copy and I hardly learned when I lost my database in 2014 ... with the years to do backup every night with a small script.

How Can I check if the database is correct when I copy it? I'm not very familiar with sqlite ...

YM

[Yves Martin]

Here are some information:

sqlite> .show
        echo: off
         eqp: off
     explain: auto
     headers: off
        mode: list
   nullvalue: ""
      output: stdout
colseparator: "|"
rowseparator: "\n"
       stats: off
       width:
    filename: /var/lib/weewx/weewx.sdb
sqlite> .dbinfo ?DB?
unable to read database header
sqlite> .dbinfo /var/lib/weewx/weewx.sdb
unable to read database header
sqlite> .dbinfo weewx.sdb
unable to read database header

(...)

What is the best way to copy this database from one raspberry Pi to another one?

YM

[Tom Keffer]

I usually use rsync.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d556a0b8-1387-40f8-b13e-6dddb74c079fn%40googlegroups.com.