On 10/15/2010 08:12 PM, faldridge wrote:
> My question to the developer of pysqlite consequently is: if you don't
> want to support combining pysqlite and apsw,
Note that you can still load APSW and pysqlite in the same process - the
only thing you won't be able to do with that change is get them to share the
same database handle. (ie you'd have to open the database file once with
each library which will work fine for everything except :memory: databases.)
There is a request for APSW to support sqlite database handles from outside:
http://code.google.com/p/apsw/issues/detail?id=79
However this is very difficult to do if you want to be 100% reliable as
SQLite does not provide a good object model (eg reference counting rarely
used, no notification on closes) so it would be possible to cause process
crashes.
> would you be interested in a patch for pysqlite that implements the
> online backup API in a similar manner to apsw?
The APSW license is very liberal so the code can just be copied over and
lightly edited, and I am also happy to assign copyright etc.
The issue for pysqlite is that it would require SQLite 3.6.11 or newer
(released Feb 2009) whereas the current pysqlite will work with far older
versions. I don't know how Gerhard handles that sort of thing.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAky5H1cACgkQmOOfHg372QTgMwCfYW1rD1JEnx6VLtqe1rmb9NqK
gSQAnRpi+Rdp/+2LzV+0cqJYNw1ze2OA
=jgf8
-----END PGP SIGNATURE-----
On 10/16/2010 01:49 PM, faldridge wrote:
> This won't work for me as my database files for this particular
> application are sufficiently large and write-heavy that attempting to
> use the backup API without sharing the database handle will (as far as
> I can tell from the testing I have done) cause the backup process to
> endlessly restart and never finish as per the SQLite Online Backup API
> documentation:
Does WAL mode help at all? In theory it means the database is not being
written to at all, and instead writes are diverted to the WAL until a
checkpoint is done. You should be able to backup the database then.
> Having found the dynamic library that pysqlite is linked against using
> ldd, is there anyway to force apsw to build using that particular
> library?
With both you can create a setup.cfg with include_dirs and library_dirs
pointing to the same place which will then ensure they use the same header
and libraries.
> Or you do have any other suggestions for getting the two to
> work together?
You could switch to APSW completely :-)
Also for the record I did not change nor did I ever plan to change anything
in APSW to make providing its handles to pysqlite stop working.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAky6ImIACgkQmOOfHg372QQLDgCfWtTC0m6FE/MJ3Gs3u5CcwuNz
VMcAmQHcZ7OApHUiA/XtjMEqnutk12I1
=A9L/
-----END PGP SIGNATURE-----
>
> You could switch to APSW completely :-)
>
Hi, I just cut this remark out of a large context. Swtiching between
the two wrappers is not such a big deal if you add an extra layer
around APSW to make it appear like pysqlite. I did this for my own
project. The resulting module, apsw_dbapi2, offers most of the
Pysqlite interface but should also allow to address APSW features.
I'd be happy happy to mail it. A limitation, in its current version,
is that isolation_level must be set to None, i.e. it does not do
automatic transactions. But I may dig up an older version where that
was included as well. Best regards, Edzard Pasma
On 10/17/2010 08:34 AM, Edzard Pasma wrote:
> The resulting module, apsw_dbapi2, offers most of the Pysqlite interface but should also
> allow to address APSW features.
Do you have the source somewhere (eg Google code)? There may be things I
can do to APSW to make your code easier and smaller.
> does not do automatic transactions.
I must admit I have never understood why anyone would want them.
> But I may dig up an older version where
> that [automatic transactions] was included as well.
How did you implement that? pysqlite actually parses the SQL text which
seems a little yucky to me. In theory it should be possible to do with
authorisers although you'll hit cases like "insert ... (select ...)".
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAky7NP4ACgkQmOOfHg372QQH+QCg1Qcje2680YCP3czRl9W8L5+G
T7EAoIw91FQWHuohpApYCZG2BZqY9GcT
=/w9C
-----END PGP SIGNATURE-----
On 10/17/2010 08:34 AM, Edzard Pasma wrote:The resulting module, apsw_dbapi2, offers most of the Pysqlite interface but should alsoallow to address APSW features.Do you have the source somewhere (eg Google code)? There may be things Ican do to APSW to make your code easier and smaller.does not do automatic transactions.I must admit I have never understood why anyone would want them.But I may dig up an older version wherethat [automatic transactions] was included as well.How did you implement that? pysqlite actually parses the SQL text whichseems a little yucky to me. In theory it should be possible to do withauthorisers although you'll hit cases like "insert ... (select ...)".Roger
On 10/26/2010 08:12 AM, Gerhard H�ring wrote:
> I was kinda busy lately. But now I've created a added backup functionality
> to pysqlite (fetch the trunk Mercurial version from Google Code and see
> attached example). It may be a bit rough still, and I haven't even looked
> into the APSW implementation for now.
You should have :-) At least you can make the API the same:
http://apidoc.apsw.googlecode.com/hg/backup.html
You seem to have picked the same API except finish is missing, as is done.
finish really should be exposed. At the moment that is called by the
destructor (which can't raise an exception) and you have no control over
when or which thread the destructor is called in anyway.
The parameters to create the backup object are also in a different order
where you have optimized for convenience and I have optimized for explicit
is better than implicit :-)
We also put the backup method on different objects. I put it on the
destination database and you put it on the source. I don't remember why I
picked the order I did - best guess is that matches the SQLite API. Also
you have to make sure that no regular API calls are made to the destination
database handle. Quoting from the SQLite docs:
However, the application must guarantee that the destination
database connection is not passed to any other API (by any thread)
after sqlite3_backup_init() is called and before the corresponding
call to sqlite3_backup_finish(). SQLite does not currently check
to see if the application incorrectly accesses the destination
database connection and so no error code is reported, but the
operations may malfunction nevertheless. Use of the destination
database connection while a backup is in progress might also also
cause a mutex deadlock.
Since APSW is thread safe (and thread correct) I keep track of usage and can
make that guarantee.
Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkzHkOIACgkQmOOfHg372QRZuQCfZozqU+YNvph+sCXZHO5aXGLx
pJUAnjS9yZurupsF8AU1IET1jq/6LUF+
=zxV5
-----END PGP SIGNATURE-----