Linux issues: pysqlite2.dbapi2.DatabaseError: file is encrypted or is not a database

[Patrick Olsen]

I'm having some issues with Firefox and pysqlite2. Both versions are running Python 2.7 from what I can tell.

If I run this on Linux:

$ python firefox.py -d /home/<username>/Desktop/ -b Firefox -p bookmark

Traceback (most recent call last):

  File "barff_0_rev3_firefox.py", line 241, in <module>

    main()

  File "barff_0_rev3_firefox.py", line 237, in main

    bookmark = browser.get_bookmarks()

  File "barff_0_rev3_firefox.py", line 154, in get_bookmarks

    cursor.execute('SELECT datetime(moz_bookmarks.dateAdded/1000000, \'unixepoch\') AS \'Date Added\', moz_bookmarks.title AS \'Title\', moz_places.url AS \'URL\', moz_places.visit_count AS \'Count\' FROM moz_bookmarks, moz_places WHERE moz_places.id = moz_bookmarks.fk ORDER BY moz_bookmarks.dateAdded ASC')

If I run it on Windows:

>firefox.py -b Firefox -d C:\Users\<username>\Desktop -p bookmark

2012-11-22 20:58:53 | Getting Started | http://www.mozilla.com/en-US/firefox/central/ | 0

2012-11-22 20:58:53 | Help and Tutorials | http://www.mozilla.com/en-US/firefox/help/ | 0

2012-11-22 20:58:53 | Customize Firefox | http://www.mozilla.com/en-US/firefox/customize/ | 0

2012-11-22 20:58:53 | Get Involved | http://www.mozilla.com/en-US/firefox/community/ | 0

Here is the code doing the parsing:

# Firefox browser class    

class Firefox(browserCommon):

    def __init__(self, profile_directory):

        browserCommon.__init__(self, profile_directory)

        

    def get_bookmarks(self):

        res = []

        (conn, cursor) = self.get_conn_cursor('places.sqlite')

        cursor.execute('SELECT datetime(moz_bookmarks.dateAdded/1000000, \'unixepoch\') AS \'Date Added\', moz_bookmarks.title AS \'Title\', moz_places.url AS \'URL\', moz_places.visit_count AS \'Count\' FROM moz_bookmarks, moz_places WHERE moz_places.id =      moz_bookmarks.fk ORDER BY moz_bookmarks.dateAdded ASC')

        for (dateAdded, title, url, visit_count) in cursor.fetchall():

            res.append(bookmarksEntry(dateAdded, title, url, visit_count))

        return res

Program Imports:

import datetime, time

import sys, getopt, os

from cStringIO import StringIO

from pysqlite2 import dbapi2 as sqlite

from django.utils.encoding import smart_str

[Roger Binns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 26/11/12 05:43, Patrick Olsen wrote:
> I'm having some issues with Firefox and pysqlite2. Both versions are
> running Python 2.7 from what I can tell.
>

> *If I run this on Linux:*

>
> $ python firefox.py -d /home/<username>/Desktop/ -b Firefox -p
> bookmark Traceback (most recent call last): File
> "barff_0_rev3_firefox.py", line 241, in <module> main() File
> "barff_0_rev3_firefox.py", line 237, in main bookmark =
> browser.get_bookmarks() File "barff_0_rev3_firefox.py", line 154, in
> get_bookmarks cursor.execute('SELECT
> datetime(moz_bookmarks.dateAdded/1000000, \'unixepoch\') AS \'Date
> Added\', moz_bookmarks.title AS \'Title\', moz_places.url AS \'URL\',
> moz_places.visit_count AS \'Count\' FROM moz_bookmarks, moz_places
> WHERE moz_places.id = moz_bookmarks.fk ORDER BY moz_bookmarks.dateAdded
> ASC')

You left out the rest of the stack trace. From the subject line I'd guess
this is because the file is not a database. The alternative is you are
using an old version of SQLite.

> for (dateAdded, title, url, visit_count) in cursor.fetchall():

There is no need for the fetchall. The cursor is an iterator already.

Roger

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlCzqp4ACgkQmOOfHg372QQF5wCgtaH/WS1RoYk3GCR/BaW3KRxe
87AAoJhN+L/glNX6UcfxZ0Ym4JgMXGxq
=8yqW
-----END PGP SIGNATURE-----

[Patrick Olsen]

Roger,

It's indeed a sqlite database. I used the same database for the Windows example as I did the Linux version. Windows worked, Linux failed. The code only fails against the Firefox places.sqlite database. It works against Google Chrome on both Linux and Windows.

As for the stack trace, that's all that was "spit out". I'll play with the fetchall portion.

[Roger Binns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 26/11/12 18:26, Patrick Olsen wrote:
> As for the stack trace, that's all that was "spit out".

It wasn't. A stack trace includes the actual exception which you didn't show.

> The alternative is you are using an old version of SQLite.

You missed this possibility. WAL was added to SQLite a while back, and it
is possible that the Linux version of SQLite being used predates WAL which
will result in the error.

Another possibility is that a journal file is missing, or present and
incorrect.

A full list of ways to get the error is at
http://www.sqlite.org/lockingv3.html#how_to_corrupt

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlC0d3oACgkQmOOfHg372QTFJQCfWXQ51jzlkD0985CfW2a2h/B+
F8MAoJNhK/hacrmC6nfVQKsWBs4u+dtq
=T0DZ
-----END PGP SIGNATURE-----