Minimum Viable Database Component

[Andy Chambers]

Hi All,

I'm trying to follow the "component" architecture for an app I'm working on and wondered if I was missing something. In the "Just enough structure" talk, one of the examples Stuart presents is a DB component that contains just a small selection of DB related functions (i.e. insert, and query IIRC) so that when you need to mock it out for your tests, you don't have to implement the entire JDBC interface.

This makes sense but I'm wondering if anyone has released such a subset (possibly expanded to include things like transactions, and maybe a few utility query builders) as open source ideally with a corresponding mock implementation. With the popularity of the component library, I'm surprised not to find ready made components I can just plug into my app.

If there's nothing like this already, then I guess I have an idea for a new project. Anyone think this is a good idea or would everyone's ideal DB component look a little different? Look forward to hearing your thoughts.

Cheers,

Andy

[Malcolm Sparks]

I think it can be a dangerous road trying to create abstractions over data sources. It can be a lot of work maintaining a stub/mock implementation to work alongside a database, and sometimes it's not that much more work (or less work) to install Postgres, SQLite, or whatever you're using, locally.

One of the pain points of working with databases is managing their state, and this is one of the reasons why stub/mock implementations initially look attractive. However, there are solid alternative approaches to making the job of managing databases a lot easier.

For example, https://github.com/juxt/joplin is a good choice for managing database schema migrations, seeding databases with state (for test setups) and has a good range of supported data sources. (Disclosure: I work for juxt)

[Jeroen van Dijk]

Hi Andy,

I'm not aware of such a project, but I can confirm that it is an approach that we apply internally at AdGoji and has been working well for us.  For instance, we have (simple) components that mock Kafka and Redis in a way that make them feature compatible to these services in the context of our platform. For Kafka we have one mock component for testing (so you can can test what has been pushed to Kafka) and we have one for development that just logs to the file system (in a human-readable) form. 

Although it doesn't replace real testing with these external services, it does make the development flow a lot faster as boot-time is quicker, easier cleanup etc. 

Cheers,

Jeroen

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Stuart Sierra]

Hi Andy,

That was just an example I made up for the slides, and not a very good one.

Usually, the functions/protocols you implement for your components are specific to your application.

–S

On Thursday, April 30, 2015 at 4:54:38 AM UTC+1, Andy Chambers wrote:

[Derek Troy-West]

Hi Andy, 

 We use component at SMX, and do follow an approach similar to the one you hint at but as Stuart suggests we tend to have protocols that are more application-centric.

 

 For instance, we use Cassandra extensively and our general interaction with that is captured by the protocol:

(defprotocol Connection

  (bind [this stmt vals] "bind values to a prepared statement")

  (prepare [this query] "provide a prepared statement from a query")

  (execute [this stmt] [this stmt opts] "execute a query or statement")

  (execute-batch [this stmts] [this stmts opts] "execute a batch of statements")

That isn't an attempt to create an abstraction of Cassandra, it captures one way in which a number of our applications use Cassandra.

In certain test cases that allows us to use a stub-component which returns canned responses, in other cases we integrate with a test Cassandra cluster, both have merits.

One further advantage is this protocol would allow us to swap the underlying implementation (which is currently based on Alia - https://github.com/mpenet/alia) with something else if we ever wanted to without impacting consumers of that protocol. That wasn't the driver for it though, stubbed tests was.