In-memory PostgreSQL database
Ben Finney
I'm using PostgreSQL as the server for several of my applications, but
unit testing the application by setting up and SQLite in-memory
database during the test case. This is workable, and certainly better
than the unacceptable overhead of connecting to a PostgreSQL server
when running the unit tests.
This has limitations, though: there are many PostgreSQL features that
aren't supported in SQLite, so the unit test fixtures have to fake a
lot of it for the sake of the PostgreSQL-specific application code.
What options are there for creating and connnecting to a PostgreSQL
database that exists only in memory, similar to the same thing in
SQLite (and, I believe, MySQL)?
--
\ “Leave nothing to chance. Overlook nothing. Combine |
`\ contradictory observations. Allow yourself enough time.” |
_o__) —Hippocrates |
Ben Finney
Laurenz Albe
> I'm using PostgreSQL as the server for several of my applications, but
> unit testing the application by setting up and SQLite in-memory
> database during the test case. This is workable, and certainly better
> than the unacceptable overhead of connecting to a PostgreSQL server
> when running the unit tests.
>
> This has limitations, though: there are many PostgreSQL features that
> aren't supported in SQLite, so the unit test fixtures have to fake a
> lot of it for the sake of the PostgreSQL-specific application code.
>
> What options are there for creating and connnecting to a PostgreSQL
> database that exists only in memory, similar to the same thing in
> SQLite (and, I believe, MySQL)?
There is no "embedded database" in PostgreSQL.
I am very surprised that you even consider running unit tests with
anything else than the database you want to use in production. Doesn't
that take away much of the benefits of unit testing?
If it is the overhead of establishing server connections that worries
you, why don't you write your unit tests so that the connection is kept
open?
If you are worried about concurrent unit tests thrashing each other's data,
you could have one database per developer in one PostgreSQL cluster
on a central server.
You can also install testing databases locally on the developers' machines,
but that will mean that you have to have a database server process running
on each machine.
Yours,
Laurenz Albe
Marco Mariani
> What options are there for creating and connnecting to a PostgreSQL
> database that exists only in memory, similar to the same thing in
> SQLite (and, I believe, MySQL)?
I don't think you would gain anything by keeping the DB in RAM with
postgres, because it's CPU-bound if there is little data.
If the test DB fits in RAM, this means Postgres already leverages the
memory as cache (if configured properly)
In my current application, I truncate and refill all the tables after
each of the 500 test.
I tried creating a tablespace in ramdisk, and had no performance
improvement at all.
Marco Mariani
> I am very surprised that you even consider running unit tests with
> anything else than the database you want to use in production. Doesn't
> that take away much of the benefits of unit testing?
Not if the DB schema has been evolved with "emergent design" and makes
no use of triggers, stored procedures, advanced check constraints... if
you design an app like some of the youngsters do, the only difference
between sqlite and postgres is that the latter has better support for
unicode ;-)
Ben Finney
> I am very surprised that you even consider running unit tests with
> anything else than the database you want to use in production.
> Doesn't that take away much of the benefits of unit testing?
No. The point of unit testing is to test a unit of code, not the
entire application. To do so, it's best to isolate that section of
code from its external dependencies; a "real" database certainly
counts as an external dependency.
For "acceptance testing" or any other kind of full-application testing
(i.e. not unit testing), certainly one wants to use the real database.
That's not the issue here though.
> If it is the overhead of establishing server connections that
> worries you, why don't you write your unit tests so that the
> connection is kept open?
The point is less about the overhead, and more about isolating the
code so that each unit test exercises as small a part of the code as
feasible.
--
\ "Pinky, are you pondering what I'm pondering?" "Well, I think |
`\ so, Brain, but it's a miracle that this one grew back." -- |
_o__) _Pinky and The Brain_ |
Ben Finney
Ben Finney
> Ben Finney wrote:
>
> > What options are there for creating and connnecting to a PostgreSQL
> > database that exists only in memory, similar to the same thing in
> > SQLite (and, I believe, MySQL)?
>
> I don't think you would gain anything by keeping the DB in RAM with
> postgres, because it's CPU-bound if there is little data.
I'm not interested in "performance", but rather reduction in external
dependencies for unit tests (i.e. unit tests should not rely on a
running database server).
Now that we've established that I want what I actually asked for, can
someone give suggestions on how to achieve it?
--
\ "If you can't annoy somebody there is little point in writing." |
`\ -- Kingsley Amis |
_o__) |
Ben Finney
Marco Mariani
> I'm not interested in "performance", but rather reduction in external
> dependencies for unit tests (i.e. unit tests should not rely on a
> running database server).
>
> Now that we've established that I want what I actually asked for, can
> someone give suggestions on how to achieve it?
I'm not sure I understand.
You cannot connect to postgres and use all of the postgres features,
like you asked, withouth connecting to a running database server.
Is this what you're asking?
Because.. with sqlite in RAM you _ARE_ using the whole stack of the
database, only there is no separate server process.
Ben Finney
> Ben Finney wrote:
>
> > I'm not interested in "performance", but rather reduction in external
> > dependencies for unit tests (i.e. unit tests should not rely on a
> > running database server).
>
> I'm not sure I understand.
> You cannot connect to postgres and use all of the postgres features,
> like you asked, withouth connecting to a running database server.
>
> Is this what you're asking?
Yes. I want to connect to a PostgreSQL database without the external
dependency of a cluster and already-running server.
> Because.. with sqlite in RAM you _ARE_ using the whole stack of the
> database, only there is no separate server process.
And no filesystem interaction, and no concurrency, and many other
things that come with a "dedicated to this one process" database.
All of these make it very useful for unit testing a unit of code that
would, under normal operation, connect to the concurrent database with
its on-disk cluster and all the rest.
I'm wanting to have these advantages, but have it be PostgreSQL so
that I don't have to make an in-memory SQLite database pretend to be
an in-memory PostgreSQL one.
--
\ "Pinky, are you pondering what I'm pondering?" "I think so, |
`\ Brain, but if we give peas a chance, won't the lima beans feel |
_o__) left out?" -- _Pinky and The Brain_ |
Ben Finney
Marco Mariani
> Yes. I want to connect to a PostgreSQL database without the external
> dependency of a cluster and already-running server.
>
>> Because.. with sqlite in RAM you _ARE_ using the whole stack of the
>> database, only there is no separate server process.
>
> And no filesystem interaction, and no concurrency, and many other
> things that come with a "dedicated to this one process" database.
Oh well.
How can I phrase that... the whole of Postgres is on the server.
There is nothing you can fake client-side.
With sqlite, there is no server part, so the point is moot.
> All of these make it very useful for unit testing a unit of code that
> would, under normal operation, connect to the concurrent database with
> its on-disk cluster and all the rest.
Actually, I see nothing wrong with having a running db for testing
purposes. I've read about that argument in some book, but I feel free to
disagree.
You could have mock objects that return sets of data, sure, and many
people just do that.
Ben Finney
> Actually, I see nothing wrong with having a running db for testing
> purposes.
Note that I'm specifically talking about unit testing here, which *by
definition* is supposed to be isolating the unit of code under test as
much as feasible.
For any kind of testing other than unit testing, I agree than having a
running database server is fine.
> I've read about that argument in some book, but I feel free to
> disagree.
I think you've just got different terminology.
> You could have mock objects that return sets of data, sure, and many
> people just do that.
Yes, that's what I'm doing in my unit tests at the moment.
However, mocking the feature-rich behaviour of PostgreSQL is rather
cumbersome, hence my desire to have PostgreSQL do the work of
implementing itself, without so many other dependencies during the
unit test cases.
Thanks for your feedback so far, it's useful to see what others think.
--
\ "Know what I hate most? Rhetorical questions." -- Henry N. Camp |
`\ |
_o__) |
Ben Finney
Thomas Guettler
> Howdy all,
>
> I'm using PostgreSQL as the server for several of my applications, but
> unit testing the application by setting up and SQLite in-memory
> database during the test case. This is workable, and certainly better
> than the unacceptable overhead of connecting to a PostgreSQL server
> when running the unit tests.
Somewhere I read that current postgres version can disable fsync on commit for
a session. Maybe this helps you.
Durability is not needed for a unittest database.
Thomas
--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de
Guido Neitzer
> Note that I'm specifically talking about unit testing here, which *by
> definition* is supposed to be isolating the unit of code under test as
> much as feasible.
The problem is, that you see the unit test as an isolated piece, which
it is not as long as you can't get rid of the DB dependancy completely.
If you can't you need to simulate production environment because if you
don't, your test is useless.
> > I've read about that argument in some book, but I feel free to
> > disagree.
>
> I think you've just got different terminology.
If you obviously need a context for your test, you can't test "context
free" as it seems you want to do.
[Mock objects]
> Yes, that's what I'm doing in my unit tests at the moment.
And that is the right thing to do.
> However, mocking the feature-rich behaviour of PostgreSQL is rather
> cumbersome, hence my desire to have PostgreSQL do the work of
> implementing itself, without so many other dependencies during the
> unit test cases.
You need to have a server running. No way around it.
The problem I see here: your unit test either does NOT rely on any PG
features and can therefore made independent, or it DOES rely on PG
features, than you need to test with real PG, not mock objects which
might behave differently.
For what it's worth: for a book I wrote I have set up a test server,
that checked out the recent version of my application from SVN, build
it, ran all the tests against a PostgreSQL database server that was
configured exactly like production but loading test data for each test
from a dump and a FrontBase DBMS as I wanted to be compatible with that
one too. It emailed the results every two hours to me so I was always
aware whether my code works against all the tests I have.
And in my opinion, that is the only correct way for unit testing: if the
unit relies on external data from a database, you need to test against
exactly the same environment you run in production or you will - sooner
or later - get to a situation where your tests work in development but
your application will fail miserably in production.
I try to keep dependancies as small as possible but for my database
driven applications, I need to test against "the real thing".
cug
Ben Finney
> Somewhere I read that current postgres version can disable fsync on
> commit for a session. Maybe this helps you.
Worth looking at, thank you.
> Durability is not needed for a unittest database.
Exactly.
--
\ "Technology is neither good nor bad; nor is it neutral." |
`\ —Melvin Kranzberg's First Law of Technology |
_o__) |
Ben Finney
Laurenz Albe
> Now that we've established that I want what I actually asked for, can
> someone give suggestions on how to achieve it?
Certainly, and I think I did so:
You cannot achieve it with PostgreSQL.
Yours,
Laurenz Albe
Ben Finney
Appreciated.
Now to start my fiendish plan to muster support for such a feature :-)
--
\ “Holy astringent plum-like fruit, Batman!” —Robin |
`\ |
_o__) |
Ben Finney
Laurenz Albe
>> You cannot achieve it with PostgreSQL.
>
> Appreciated.
>
> Now to start my fiendish plan to muster support for such a feature :-)
Fair.
I'd start with looking through the mailing list archives of pgsql-hackers
and pgsql-general. There have been plenty threads on this or similar
topics.
To suggest a new feature, I'd recommend that you write to the "general"
mailing list, but many people will be pissed off unless you make clear
that you have read the archives and have something new to contribute to
the discussion.
I think the chances are not too high that any such thing will be
implemented. If you show understanding and are willing to contribute work
to this, e.g. by thinking of an intelligent design for the feature you
desire, your odds increase.
Yours,
Laurenz Albe
PS: I'd still suggest that you have a central PostgreSQL server in your
company to run everybody's unit tests against. Unless your people develop
out by the seaside with no LAN access.
Ben Finney
> I'd start with looking through the mailing list archives of
> pgsql-hackers and pgsql-general. There have been plenty threads on
> this or similar topics.
>
> To suggest a new feature, I'd recommend that you write to the
> "general" mailing list, but many people will be pissed off unless
> you make clear that you have read the archives and have something
> new to contribute to the discussion.
>
> I think the chances are not too high that any such thing will be
> implemented. If you show understanding and are willing to contribute
> work to this, e.g. by thinking of an intelligent design for the
> feature you desire, your odds increase.
All good advice, taken on board. Thank you.
--
\ "I have one rule to live by: Don't make it worse." -- Hazel |
`\ Woodcock |
_o__) |
Ben Finney
Ben Finney
> PS: I'd still suggest that you have a central PostgreSQL server in
> your company to run everybody's unit tests against.
This assumes that the code is all run (or even tested) inside one
company. Since I want the unit tests to run even when no PostgreSQL
server is available, I can't do what you suggest.
--
\ "Some mornings, it's just not worth chewing through the leather |
`\ straps." -- Emo Philips |
_o__) |
Ben Finney
Peter Jones
> This assumes that the code is all run (or even tested) inside one
> company. Since I want the unit tests to run even when no PostgreSQL
> server is available, I can't do what you suggest.
I'm beating a dead horse here, but...
If your unit tests are testing code that expects specific PostgreSQL
behaviors from a database, then I guess you need to use PostgreSQL.
You can't embed PostgreSQL, so the next obvious solution is to write a
slick test system that can download, build, and run a temporary
PostgreSQL database specifically for testing. This actually wouldn't
be too difficult either.
What I don't understand is this, your code obviously needs PostgreSQL.
Why would someone running the unit tests not have access to a
database, which is a clear dependency of the application you are
testing?
--
Peter Jones [pjones at domain below]
pmade inc. - http://pmade.com
Ben Finney
> Ben Finney <bignose+h...@benfinney.id.au> writes:
> > This assumes that the code is all run (or even tested) inside one
> > company. Since I want the unit tests to run even when no
> > PostgreSQL server is available, I can't do what you suggest.
>
> I'm beating a dead horse here, but...
I'm getting valuable feedback from this thread. Don't be misled:
merely because I rebut points doesn't mean I'm not listening and
thinking about them.
> If your unit tests are testing code that expects specific PostgreSQL
> behaviors from a database, then I guess you need to use PostgreSQL.
Right.
> You can't embed PostgreSQL
I'd like, for the specific purpose of unit testing only, to get as far
from full-blown PostgreSQL, and as close to "embedded" PostgreSQL, as
possible.
> so the next obvious solution is to write a slick test system that
> can download, build, and run a temporary PostgreSQL database
> specifically for testing. This actually wouldn't be too difficult
> either.
This is what I'm doing to unit test the actual database as it will be
deployed. It's working fine, albeit slower than a code unit test.
(Side topic: Would people be interested to see how this is being done?
Should I start a new thread on this?)
It's too much overhead for the code unit tests though, hence my quest
to find a way to sacrifice some subset of {concurrency, durability,
network access, separate process, etc.} for the sake of having real,
in-memory, fast PostgreSQL dedicated to the unit test process on a
single connection.
> What I don't understand is this, your code obviously needs
> PostgreSQL. Why would someone running the unit tests not have access
> to a database, which is a clear dependency of the application you
> are testing?
Unit tests should run as fast as feasible, with as few external
dependencies as feasible, while still having each test case exercise
the specific code unit of interest to that test case. All these
properties are important for unit tests, because they encourage
running the unit tests very frequently and enable knowing exactly what
section of code causes a test case to fail.
I'm looking for leverage on both the above instances of "feasible".
This tradeoff is obviously inappropriate for the deployed application;
it's also inappropriate for any kind of test that should be testing
the entire application (e.g. feature tests, acceptance tests,
performance tests, and so on).
For unit tests, though, it's a tradeoff I'm convinced this tradeoff is
the right one. All (!) that remains is getting PostgreSQL closer to
it.
--
\ "The number of UNIX installations has grown to 10, with more |
`\ expected." -- Unix Programmer's Manual, 2nd Ed., 12-Jun-1972 |
_o__) |
Ben Finney
Laurenz Albe
>> so the next obvious solution is to write a slick test system that
>> can download, build, and run a temporary PostgreSQL database
>> specifically for testing. This actually wouldn't be too difficult
>> either.
>
> This is what I'm doing to unit test the actual database as it will be
> deployed. It's working fine, albeit slower than a code unit test.
> (Side topic: Would people be interested to see how this is being done?
> Should I start a new thread on this?)
Actually I forgot to point that out to you, there *is* such a thing
bundled with core PostgreSQL. It is the regression tests.
> It's too much overhead for the code unit tests though [...]
Hmm. It should take about 10 to 30 seconds to install, prepare and start a
PostgreSQL server.
Do your programmers run the unit test after each line of code they write?
Yours,
Laurenz Albe
Ben Finney
> Ben Finney <bignose+h...@benfinney.id.au> wrote:
> > [Setting up a PostgreSQL cluster and server is] too much overhead
> > for the code unit tests though [...]
>
> Hmm. It should take about 10 to 30 seconds to install, prepare and
> start a PostgreSQL server.
This is orders of magnitude longer than anything else the code unit
tests do. They execute hundreds or thousands of tests in a few
seconds.
> Do your programmers run the unit test after each line of code they
> write?
Not quite, but there is a testing loop that automatically runs the
unit tests every time a file changes in the project tree; the results
are available in a separate window. Whenever one makes a change and
cares about the effect on the tests, one need only look over to the
testing loop window and the unit test suite result is available within
a second or two.
It thus becomes natural to consider the impact on the test suite of
*every* change being made to the tests and the code. Not necessarily
to slavishly check the tests after changing every line of code, but to
keep it as part of a very tight development feedback loop. The barrier
to running the unit test suite is minimised to insignificance, along
with any reasonable objection to implementing unit tests at the same
time as developing the code.
To introduce a tens-of-seconds overhead to this process would
completely change the workflow; this is why currently the unit test
database that provides test data for the code unit tests is
implemented in an in-memory SQLite instance, which adds negligible
overhead to create and tear down.
I want to replace that SQLite component with PostgreSQL, without
losing the benefits that lead me to use SQLite for the unit test
database in the first place.
--
\ "What you have become is the price you paid to get what you |
`\ used to want." -- Mignon McLaughlin |
_o__) |
Ben Finney
Laurenz Albe
>> Do your programmers run the unit test after each line of code they
>> write?
>
> Not quite, but there is a testing loop that automatically runs the
> unit tests every time a file changes in the project tree; the results
> are available in a separate window. Whenever one makes a change and
> cares about the effect on the tests, one need only look over to the
> testing loop window and the unit test suite result is available within
> a second or two.
I guess it's too late for you to make such big a change in your development
process, but given the above I would do something like the following:
I hope I get it right that when you say "every time a file changes in
the project tree" you mean "every time a file is committed in the source
repository".
You could have a unit test server that has a database server installed.
Every 5 minutes it checks out the project source, initializes the
database and runs the unit tests.
The result is made availabe vie HTTP, and every developer can look at it.
Something similar to the PostgreSQL build farm.
The big downside with your setup that I see is that continuously running
unit tests in the background on the developers' machines use lots of
CPU cycles, even if you use an embedded database.
Developers need fast machines so that they can compile and run quickly.
Yours,
Laurenz Albe
Ben Finney
> I hope I get it right that when you say "every time a file changes
> in the project tree" you mean "every time a file is committed in the
> source repository".
No, I mean "every time any source file is changed in the working
tree". There's a loop watching for filesystem events on the working
tree the developer is working in, and executing the unit test suite
whenever that happens.
That way, the developer never needs to distract themselves
distinguishing between "an edit that means I should run the unit test
suite" versus "an edit that isn't worth running the unit test suite".
The unit test suite is *always* run after an edit, and the developer
can choose when to look at the results.
It makes for a much more fluid feedback cycle, with near-immediate
opt-in feedback on whether an edit has had a positive or negative
effect on the result of the unit test suite.
It also doesn't cost the developer any effort, beyond starting off the
loop, because the unit tests, by design, are fast and unobtrusive.
And it's that speed and unobtrusiveness that I'm trying to maximise
while still haing real PostgreSQL, by getting dedicated in-memory
instances of a PostgreSQL database for the unit tests.
> You could have a unit test server that has a database server
> installed. Every 5 minutes it checks out the project source,
> initializes the database and runs the unit tests.
This is also an excellent practice, and we follow it, but is more in
line for an "integration" cycle: check out the current source code
every time a commit is done, and attempt to build the entire project,
making a report available on completion for anyone to check when they
feel like.
It's an altogether different purpose from unit testing, though.
> The big downside with your setup that I see is that continuously
> running unit tests in the background on the developers' machines use
> lots of CPU cycles
The principle at work here is that CPU cycles are cheap, compared to
developer time spent chasing down errors that could be caught by early
feedback. This is true especially if the unit test cases are (as
discussed) each focussed on testing one specific area of code without
exercising the entire application stack.
--
\ “Our task must be to free ourselves from our prison by |
`\ widening our circle of compassion to embrace all humanity and |
_o__) the whole of nature in its beauty.” —Albert Einstein |
Ben Finney
Laurenz Albe
>> I hope I get it right that when you say "every time a file changes
>> in the project tree" you mean "every time a file is committed in the
>> source repository".
>
> No, I mean "every time any source file is changed in the working
> tree". There's a loop watching for filesystem events on the working
> tree the developer is working in, and executing the unit test suite
> whenever that happens.
>
> That way, the developer never needs to distract themselves
> distinguishing between "an edit that means I should run the unit test
> suite" versus "an edit that isn't worth running the unit test suite".
> The unit test suite is *always* run after an edit, and the developer
> can choose when to look at the results.
Ouch.
From my experience as a coder, you save code not only when you have an
error-free or even compiling state.
This means that the tests will be all red most of the time, and after some
time, people will stop watching them at all.
Have you talked to your developers about that?
>> You could have a unit test server that has a database server
>> installed. Every 5 minutes it checks out the project source,
>> initializes the database and runs the unit tests.
>
> This is also an excellent practice, and we follow it, but is more in
> line for an "integration" cycle: check out the current source code
> every time a commit is done, and attempt to build the entire project,
> making a report available on completion for anyone to check when they
> feel like.
>
> It's an altogether different purpose from unit testing, though.
It seems that we have a different concept of what a unit test is,
but that's ok.
Yours,
Laurenz Albe
Ben Finney
> Ben Finney <bignose+h...@benfinney.id.au> wrote:
> > That way, the developer never needs to distract themselves
> > distinguishing between "an edit that means I should run the unit
> > test suite" versus "an edit that isn't worth running the unit test
> > suite". The unit test suite is *always* run after an edit, and the
> > developer can choose when to look at the results.
>
> Ouch.
>
> From my experience as a coder, you save code not only when you have
> an error-free or even compiling state.
That's right.
> This means that the tests will be all red most of the time, and
> after some time, people will stop watching them at all.
As I said above, they look at them only when they make a change whose
effect they're interested in (e.g. adding a new test that should fail,
or making a change that should fix a failing test).
> Have you talked to your developers about that?
Yes, they are ecstatic with the rapid feedback. They never have to
decide when to run the tests, they only need to decide when to look at
the results. Much faster feedback loop, and less time dithering over
whether the tests should be run.
Naturally, the unit tests must all pass before committing the code to
a shared VCS branch, so they have an active interest in making the
tests all pass; but the rapid feedback means they can see exactly what
they're doing to progress toward that.
It's been a great hit; one of the largest gains for smallest effort in
recent years for me. I never thought a simple 'while' loop could bring
such joy to my team's faces :-)
--
\ "I got up the other day, and everything in my apartment has |
`\ been stolen and replaced with an exact replica." -- Steven |
_o__) Wright |
Ben Finney
Lew
> You need to have a server running. No way around it.
>
> The problem I see here: your unit test either does NOT rely on any PG
> features and can therefore made independent, or it DOES rely on PG
> features, than you need to test with real PG, not mock objects which
> might behave differently.
>
> For what it's worth: for a book I wrote I have set up a test server,
> that checked out the recent version of my application from SVN, build
> it, ran all the tests against a PostgreSQL database server that was
> configured exactly like production but loading test data for each test
> from a dump and a FrontBase DBMS as I wanted to be compatible with that
> one too. It emailed the results every two hours to me so I was always
> aware whether my code works against all the tests I have.
>
> And in my opinion, that is the only correct way for unit testing: if the
> unit relies on external data from a database, you need to test against
> exactly the same environment you run in production or you will - sooner
> or later - get to a situation where your tests work in development but
> your application will fail miserably in production.
>
> I try to keep dependancies as small as possible but for my database
> driven applications, I need to test against "the real thing".
You are correct, Guido, and your advice is consistent with what I've
experienced in well-managed projects. You don't eliminate the database
*engine* for unit tests - you hold it constant so as not to alter the
environment of the test. Otherwise you introduce variation by having a
different database (e.g., SQLlite) which might cause different emergent
phenomena.
What you replace is the database *instance*, one with controlled data so you
can get predictable results from your unit test. Those well-managed projects
had parallel environments, one for developers, one for testers, one for
production.
The only complaint I've heard from professionals about such environments has
been when they differ (e.g., use a different database engine) - the experts
grouse that tests are meaningless if you change too many variables all at once.
--
Lew
Lew
> Laurenz Albe <inv...@spam.to.invalid> writes:
>
>> Ben Finney <bignose+h...@benfinney.id.au> wrote:
>>> That way, the developer never needs to distract themselves
>>> distinguishing between "an edit that means I should run the unit
>>> test suite" versus "an edit that isn't worth running the unit test
>>> suite". The unit test suite is *always* run after an edit, and the
>>> developer can choose when to look at the results.
>> Ouch.
>>
>> From my experience as a coder, you save code not only when you have
>> an error-free or even compiling state.
>
> That's right.
But, oh, so wrong.
I've been on teams where the commit rule was, "Don't break the build!", on
penalty of ridicule.
The code might not run right, but it must compile before committal.
One thing you must do is permit frequent checkins. Processes that discourage
checkin result in programmers waiting too long between commits. "Don't break
the build!" is a reasonable limit - it encourages frequent checkin but not
carelessness.
Red X's are not ignored. Usually teams allocate reasonably independent
portions of a project to different workers - workers will mostly ignore red
X's in their colleagues' tests, but not in their own. Maybe at the end of the
day they'll expect fewer red from everyone, but not when they know people are
hot on that new feature and will have those bugs fixed in a few hours.
Integration tests' red X's are another matter, of course.
From experience, I recommend "Don't break the build!" as a checkin rule. It
balances the needs nicely: Let programmers commit often, but at least the
unit test build cycle can run.
--
Lew
Ben Finney
> Ben Finney wrote:
> > Laurenz Albe <inv...@spam.to.invalid> writes:
> >
> >> Ben Finney <bignose+h...@benfinney.id.au> wrote:
> >>> That way, the developer never needs to distract themselves
> >>> distinguishing between "an edit that means I should run the unit
> >>> test suite" versus "an edit that isn't worth running the unit
> >>> test suite". The unit test suite is *always* run after an edit,
> >>> and the developer can choose when to look at the results.
> >> Ouch.
> >>
> >> From my experience as a coder, you save code not only when you
> >> have an error-free or even compiling state.
> >
> > That's right.
>
> But, oh, so wrong.
Note that "save code" in the context of the above is "save a code
source file to disk", not "commit".
> I've been on teams where the commit rule was, "Don't break the
> build!", on penalty of ridicule.
That's right. However, before committing, one generally saves files
many times, often in a state that would break the code if committed.
The continuously-run, rapid-completion local unit test suite loop is
an aid for the developer to quickly and effortlessly know when a local
save *can* be committed without breaking the unit test suite.
--
\ "Pinky, are you pondering what I'm pondering?" "Wuh, I think |
`\ so, Brain, but how will we get three pink flamingos into one |
_o__) pair of Capri pants?" -- _Pinky and The Brain_ |
Ben Finney
dnoyeB
>>
>> No, I mean "every time any source file is changed in the working tree".
>> There's a loop watching for filesystem events on the working tree the
>> developer is working in, and executing the unit test suite whenever
>> that happens.
>>
>> That way, the developer never needs to distract themselves
>> distinguishing between "an edit that means I should run the unit test
>> suite" versus "an edit that isn't worth running the unit test suite".
>> The unit test suite is *always* run after an edit, and the developer
>> can choose when to look at the results.
>
> Ouch.
>
> From my experience as a coder, you save code not only when you have an
> error-free or even compiling state.
> This means that the tests will be all red most of the time, and after
> some time, people will stop watching them at all.
>
By save im sure he means commit to the repository. And whoever breaks the
unit test will certainly be fixing it again. And more than likely before
any other commits. So he will receive pressure to get it done. I can't
see any pressure to ignore the failed test.
> Have you talked to your developers about that?
>
>>> You could have a unit test server that has a database server
>>> installed. Every 5 minutes it checks out the project source,
>>> initializes the database and runs the unit tests.
>>
>> This is also an excellent practice, and we follow it, but is more in
>> line for an "integration" cycle: check out the current source code
>> every time a commit is done, and attempt to build the entire project,
>> making a report available on completion for anyone to check when they
>> feel like.
>>
>> It's an altogether different purpose from unit testing, though.
>
Can't you create a RAM disk, and point postgresql server toward that. Now
when the unit test starts you just create the ram disk, start the server,
run your default sql to setup the data, and your set. Cleanup is as
simple as stopping the server and dumping the ramdisk.
However, if speed is not an issue, I don't see why you have a problem with
just using the harddisk on a full postgresql server.
Besides, your trying to cheat anyway. You know for a proper unit test you
can't use a proper DB at all. You need to use some almost scripted
responses to SQL commands. You appear to be trying to avoid that. Once
you do, you might as well just run a full db...
CL