testing dbsqlite in Pygr
Christopher Lee
the endless problems with dbm style interfaces that people run into on different Python installs (especially Mac) persuade me that we need to take action on the dbsqlite dbm-style interface to sqlite as we had previously discussed. It seems like this would really solve the problem, which has wasted so much of our time already. Looking at issue 3783 in the Python bugs database, it looks like no further progress has occurred there, so if we want action on this we will have to take it up ourselves.
I took a look at the dbsqlite code, which suggested the following:
- we would need to implement basic handling of the c / n / w / r mode flags.
- how do you think we should test this prior to pushing it into a Pygr maintenance release (such as 0.8.3)?
One obvious initial test is to run the Pygr test suite using this on mac 10.6 apple-supplied Python 2.6, which lacks any decent dbm style interface (only dbm works, and it lacks any way of iterating...). Running the full Pygr test suite using dbsqlite would at least test it in a wide variety of areas including sequence databases, worldbase, etc. This would at least give us a basic sense of whether it solves our relevant Pygr needs...
What are your thoughts on this idea of trying to use sqlite as a dbm / shelve replacement? It's been a long time since we've discussed it.
Yours,
Chris
C. Titus Brown
Hey Chris,
good idea.
since then I've used the dbsqlite code in two separate projects, tagnabbit
and pony-build, whtout any hitches. While I don't remember the specific issues
driving these patches, I made two modifications:
https://github.com/ctb/tagnabbit/commit/26db7053f05f579d1f2e801336ad759ac3468ed8
https://github.com/ctb/pony-build/commit/baa93ff5f0b014e8b01b0a1f0941b5f2b947363f
The latter patch is important for proper iterator behavior, I believe. The
first patch is for good emacs support.
The general persistence system in pygr is due for a radical simplification
and overhaul, but that's probably too ambitious for now. I agree with your
proposal for testing.
I'm on vacation right now, which means I may or may not get a chance to work
on this. Ping me privately if you start working on it so we don't duplicate
work.
Note, for the SequenceFileDB fix, screed may be a good option: see
https://github.com/ctb/screed/blob/master/screed/tests/test_pygr_api.py
for an example of how to use it. Briefly,
from screed.pygr_api import ScreedSequenceDB
should give you a drop-in replacement for SequenceFileDB, although you'll
need to index the database separately; see
https://github.com/ctb/screed/blob/master/doc/example.txt
cheers,
--titus