alternative engines, dev problems
Paul Miller
> I plan on extending this to work with SQLite and PostgreSQL in the next
> release. Oracle, Sybase, and other engines will come later.
Where is this being discussed? Is it in the group somewhere and I'm
missing it? or is this private thinking? I'm very much interested in
the way this is going and I'd like to read about what is and is not
working for you.
I poked around the groups a little, but didn't really see anything
related.
Paul Miller
On Jan 1, 8:15 am, Paul Miller <jett...@gmail.com> wrote:
> Where is this being discussed?
I didn't really finish my thought. Of particular interest to me is
why the locking won't work in InnoDB and will it work in SQLite? Why
was the performance bad in MySQL? ... things like that.
Rob Kinyon
>> Where is this being discussed?
As of yet, it hadn't been discussed. This is the merging of a patch
provided to me by Michael Frager (a columbus.pm colleague). He did the
initial legwork and I took it and ran with it. The big thing, imho,
was the push to refactor the innards to make it easy(-er) to add new
storage engines. One of the biggest obstacles to using DBM::Deep in
many situations was the inability to use this safely across multiple
machines. This development effort (indirectly) addresses that concern.
> I didn't really finish my thought. Of particular interest to me is
> why the locking won't work in InnoDB and will it work in SQLite? Why
> was the performance bad in MySQL? ... things like that.
Backstory: I've always been a proponent of MySQL. I recognize that
Oracle (et al) have better characteristics in the high-end, but I've
always felt that MySQL's implementation of ACID transactions (in the
InnoDB engine) was just as good as Oracle's, if potentially less
performant under heavy load.
You can imagine my surprise when I found out that DBM::Deep's
implementation of transactions is actually *BETTER* (as in more
correct) than InnoDB's. The problem I ran into is in how InnoDB
manages auto-increment columns and foreign keys. You can see the
problem by adding some code in DBM::Deep::Engine::DBI to delegate
begin_work/commit/rollback to $self->storage->$method, making ::DBI
support transactions, then running t/33_transactions.t. Either you run
into a problem where you are seeing commits by other processes
(transaction isolation mode is too low) or you're blocking on a lock
on the auto-increment of refs or a lock on the index for the foreign
key (transaction isolation mode is too high). There's no middle
ground.
I haven't taken the time to explore other RDBMS engines; most of this
work has been done in stolen moments over my winter vacation (while my
wife wasn't looking! *grins*).
For 1.0019_002, my plans are:
1. Add support for SQLite
a. See if SQLite and MySQL differ enough to support different
DBI engines (sub-engines?)
b. Pray that SQLite's transactions aren't as broken as InnoDB's
c. Pray that someone sees a way around the problem described above
2. Start addressing the overflow of bug reports that have been
building up in my inbox for the past 2 years(!!)
a. See if the DBI engine fixes other bugs
3. Start refactoring the documentation
4. See if adding support for BDB is easy enough to do
a. See if I can't leverage BDB's hashing instead of being
indirect as I have to do for databases
5. Beg for someone to start looking at performance characteristics
Any takers?
Rob
Paul Miller
> Any takers?
I'm sorely tempted to work on the SQLite stuff, but I don't have
any more time than anyone else.
The nice thing about git is that I can work on it and post it and
you can choose to pull, or not, ... whatever.
Thanks for the responses to my questions,
-Paul
--
If riding in an airplane is flying, then riding in a boat is swimming.
114 jumps, 47.2 minutes of freefall, 90.4 freefall miles.
Rob Kinyon
> On Fri, Jan 01, 2010 at 11:27:51AM -0500, Rob Kinyon wrote:
>> Any takers?
>
> I'm sorely tempted to work on the SQLite stuff, but I don't have
> any more time than anyone else.
I've already started the SQLite work. Everything is done for the same
level of support as in MySQL (i.e., no DBM::Deep transactions). I'll
be seeing if I can't get transactions working. Also, I need to rethink
how I do one of the tests in t/19_crossref.t. The whole feature may
require some rethinking.
Rob