Sharing an In memory database with other libraries

[Daniel Ribeiro]

Is there a way to share the same in memory sqlite3 database? I am
using Sequel for raw queries, where DM is really lacking, but they
can't work toguehter unless they share a physical database (which
pumps our tests from 7 seconds to unbearable 47).

Regards,
- Daniel

[Daniel Ribeiro]

It essentially means I want to set up a Repository Adapter with a
given SQLite3::Database instance.

[Markus Schirp]

Sequel supports DataObjects while DataMapper does too. You should be
able to hook into Sequel or DataMamapper and inject eigther DataMappers
do instance in Sequel or the other way round...

Regards,

Markus

[Daniel Ribeiro]

Yes, I've noticed this while I've debugged my way into the guts of
both. They kinda instatiate their databases very deep down, and it
will require a bit of hacking around to make them talk to each other.
DataObjectsAdapter from dm-do-adapter/adapter.rb should do it on the
DM-side.

We'll see how this is needed. After making only the few tests that
need both frameworks use it the disk database, I managed to bring back
the full test time down to 13 seconds. Hacking Sequel will probably
need more justification than 4 seconds.

[Ted Han]

Hey Guys,

So DataObjects refers to things via the URI of the datastore.  So you should be able to pass that single key into both systems (i'm hypothesizing here) in order to get access to the store.

-T

--
You received this message because you are subscribed to the Google Groups "DataMapper" group.
To post to this group, send email to datam...@googlegroups.com.
To unsubscribe from this group, send email to datamapper+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/datamapper?hl=en.

[Daniel Ribeiro]

Actually, for in memory databases *only*, each one of them create a
different in memory database.

> > datamapper+...@googlegroups.com<datamapper%2Bunsu...@googlegroups.com>
> > .

[David Masover]

Hack: Run it on a RAMdisk or equivalent. I recommend tmpfs on Linux.

[Daniel Ribeiro]

Yes, this is the ultimate hack. It is quite fragile, and setup
intensive, as all team members must have a ramdisk enabled, and have
the same path. Or, what is worse, configurable ones (breaking our
"don't make me think" test rule). Hacking sequel seems a lot less
work. And it might pay off, as truncating all databases is far more
performatic than automigrate! (for clearing up tests).

On one of my colegues notebook, the disk is so bad that the tests take
147 seconds!

[David Masover]

On Sunday, September 19, 2010 08:52:18 pm Daniel Ribeiro wrote:
> Yes, this is the ultimate hack. It is quite fragile, and setup
> intensive, as all team members must have a ramdisk enabled, and have
> the same path.

While true, it's also not as bad as you'd think. If you're running a modern
Linux, it's as simple as:

sudo mount tmpfs /wherever/i/want/a/ramdisk

...and done! You could easily put that in a script.

I have a tmpfs mounted somewhere convenient all the time, because I find it
generally useful.

> Or, what is worse, configurable ones (breaking our
> "don't make me think" test rule).

Symlinks.

> Hacking sequel seems a lot less
> work.

The above was a one-liner.

Now, if you're on Windows or OS X, I don't know how you'd do that. It also may
violate other rules, like "Don't make me run things as root". I just don't see
it being harder than hacking sequel, certainly not if you're on a Linux.

> And it might pay off, as truncating all databases is far more
> performatic than automigrate! (for clearing up tests).

Have you run automigrate on anything not disk-backed?

By all means, don't let me stop you from doing this a cleaner way, if you can.
I just like the one-liner, dirt-simple, generic solution.

[Richard Conroy]

You got any more links on this? You don't really see RAMdisk options

considered enough in my experience. Are people generally satisfied

enough with using DBs like sqlite3 in-memory capability that they dont

look into RAMdisk options.

I am mostly looking for peoples experiences, blog posts. Especially 

if people manage to run something wierd, like MySQL from a RAMDisk,

rather than sqlite3 

--

http://richardconroy.blogspot.com | http://twitter.com/RichardConroy

[Daniel Ribeiro]

Great points David. Your post makes this hack seems way less fun, and
yet, way more productive.

About automigrate: Yes. We still use in memory databases for 140 of
the tests (more than 90%). I'll refrain from speculating how much less
time it would take without testing. The test setup gets a bit more
compicated and that would really need to pay off to make it.

About the ramdisk: we don't touch files anywhere else, as good unit
tests do. But, the trick should really go a mile for NOSQL databases,
should
we need them (don't spect to be soon, but we rely on data, not
expectations).


Regards,
[]

[Daniel Ribeiro]

Using Sequel's support on DataObjects, and hacking
DataObjects::Connection to make sure it is a singleton, they both play
very nice toguether. I've uploaded a gist with the details:
http://gist.github.com/644328

Regards,
[]