When to add the database

[Jop van Raaij]

When would you add a database to your application?

Using TDD, possible results of methods to test business logic always come from memory. This makes the testing super fast. I've heard(and tried) about the FitNess testing framework, of which Uncle Bob states the database is still a deferred subject, as saving files to disk is sufficient (I believe there is a MySQL implementation because of an client' request). So persistence itself is not forcing to use a database.

What user story, not explicitly mentioning the database, could force usage of a database? Is it only when considering a performance related use-case of a story, like "given there are 2 unpaid invoices of a total of 100,000 invoices, the admin should see the list of unpaid invoices in less than 1 second"?

[Uncle Bob]

I would add the database as early as possible, and as late as possible.  <giggle>

Early implementations would be mocked.  I want the data management API in place, but I don't need the back end implementation of the database itself.  I can get a lot working with a mocked data management API.

Later I'd add a ram based implementation to the data management API.  This would allow me to "store" entities over the short term; but would not give those entities true persistence.

Later I'd find some way to add persistence to the data management API.  In FitNesse we added a flat file approach which we thought would be a stop-gap measure on the way to MySql.  

Finally, If necessary, I'd implement the data management API with a longer term solution.  

You asked what storie would force these decisions.  Sometimes these can be as simple as: "Use Oracle", or "Remember data across power cycles."  Sometimes you just use your own judgement.  The point is to always assume you will be using some external data management system, and then try to defer implementing it for as long as possible.  

[Andrzej Krzywda]

That's a very good explanation, thanks!

Sometimes, though, you're limited by the technology choice. If you're
starting your app with Rails (I know, I know, we shouldn't....), then
there's almost no way of using ram-based implementation, as almost all
of the 'mainstream' Rails deployments are process-based.

Any idea how to handle this kind of situations?

> --
> The only way to go fast is to go well.
> ---
> You received this message because you are subscribed to the Google Groups
> "Clean Code Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clean-code-discu...@googlegroups.com.
> To post to this group, send email to clean-code...@googlegroups.com.
> Visit this group at http://groups.google.com/group/clean-code-discussion.

[Uncle Bob]

You can start your application with Rails, and you can have a database technology already selected, and you can still decouple your use cases from the database.  The first rule is not to use Active Record in your interactors and entities.  Active Record belongs in the implementation code of your database gateways.

[Andreas Altendorfer]

I'll have to start a new project in a few weeks and therefore I'm currently doing an experiment where I define a module "Store" which is used instead of ActiveRecord/MongoId in a Rails4 application.

The Store-module uses simple PStore from ruby standard library. It defines all methods needed within "Rails-Models" and "Rails-Controllers".

Next I will get rid of ActiveModel as well, thus I can build a gem which is independent from Rails before I start our new project. 

Then I will do the same on the view-side. If my experiment succeed, I'm able to start –with my pair– a simple Command-line-ruby-app and focus on the application and business logic.

Another pair will begin with a  Sinatra-app to implement the web based frontend.

 

Should a day come where we have to "scale", it will be easy to switch to AR/Mongo/Datamapper and from Sinatra to Rails without touching the application (except in a few LOC).

If you're interested in this experiment, it's available at Github and http://yarb.iboard.cc

[Jop van Raaij]

Yeah, I made Uncle Bob giggle!

A user story like 'Use an Oracle DB' is discouraged, according to  'User Stories Applied', by Mike Cohn. This story does not represent any customer value (most of the time).  Maybe they already got a license for an Oracle DB and would like to use it because they paid for it. It is value, but it's a bad thing to force using something just because you have it. It might not be the best suited DB.

Plugging in something like a persistence, or a GUI, at a later time, gives me focus to the business logic to implement given the user stories. The client will not see a lot of progress however. He wants to see the application working from end-to-end. I've seen discussion in this group about a 'walking skeleton': get all bits and pieces working, from the GUI to the database for one of the more simple stories. This proves everything is working and integrating OK, but it is hard to defer many of the decisions, like DB, GUI etc. Or you have to do a lot of mocking.
In the book 'Growing Object-Oriented Software, Guided by Tests', they use the walking skeleton approach. But I think this first story they implement results in many lines of code for one acceptance test (written in Java). To get to a passing acceptance test, they do not seem to need any TDD cycles, which would create unit tests. To me this feels wrong. I think you'll miss out the guidance the test in TDD provide when developing the application step by step.

I could use a database from the beginning and decouple by using something like the Boundary Interactor Entity pattern. This will add complexity from the beginning and I'll implement a pattern not guided by tests, but guided by what I think will be useful in the (near) future.

I guess I'll have to combine the strategies on a case by case bases :).

[Uncle Bob]

While it's true that "Use Oracle" is not a good user story in theory; it often _is_ a user story in practice.  The reason for this is that there can be business value in using Oracle.  For example, the application may be taking advantage of an enterprise data model housed in an Oracle DB.  

The notion of a walking skeleton is a good one.  Users _do_ want to see end to end progress early.  However, this doesn't mean that the choice of UI and DB in the walking skeleton is going to be the same as for the final product.  For example, you might toss up a quick website using ~frames~ <yuk> for the first few iterations, so that the customer can see the walking skeleton.  It might also use CouchDB, or Linda, or even flat files on the DB backend.  Later on, as the customers demand better UI and DB behavior, you can shift towards Ajax and Oracle, etc.

[Jop van Raaij]

I agree on all points. Thank you for taking the time to answer/discuss. These discussions really complement the talks I have with my stuffed tux (a little cooler as a stuffed bear, but also a bit more nerdy).