Re:unit tests and databased [was Re: Merging classserver extensions and other needed features into staging]
Trent Waddington
> Unless I missed something, what you did was to always
> pass the test, even when it failed ... which makes testing
> pointless.
The test should pass for an ordinary user running make test. If you
can't setup the db automatically then you should not be failing
automatically.
> Basically, if you have ODBC installed and configured, the
> test should pass. If it fails, it means that you have ODBC
> installed, but mis-configured. If it is not installed, then
> the test won't even be run -- this is the correct behaviour.
That sounds great, but you're not doing that. You're just polluting
the unit tests with a test that fails every time and this encourages
others to be sloppy with unit test acceptance.
Trent
Joel Pitt
I can accept Linas's point that if ODBC is installed and configured
for use with OpenCog, then it should run the test. However, as I
understand it, the setup of the database is specific to Linas. From
the details in the repository's opencog.conf:
SENSE_SIMILARITY_DB_NAME = "lexat"
SENSE_SIMILARITY_DB_USERNAME = "linas"
SENSE_SIMILARITY_DB_PASSWD = "asdf"
I suggest that these login details be blank by default. Then the test
should only run if these have been made non-blank.
J
Linas Vepstas
> On Mon, Mar 9, 2009 at 11:35 PM, Linas Vepstas <linasv...@gmail.com> wrote:
>> Unless I missed something, what you did was to always
>> pass the test, even when it failed ... which makes testing
>> pointless.
>
> The test should pass for an ordinary user running make test.
OK. uninstall your odbc devel environment. If you don't need
it, don't compile it, don't test it. But if you do compile it, then
you *must* test it. You can't just disable the test because you feel like it.
--linas
Trent Waddington
> OK. uninstall your odbc devel environment. If you don't need
> it, don't compile it, don't test it. But if you do compile it, then
> you *must* test it. You can't just disable the test because you feel like it.
The correct way to unit test this stuff is to stub the db connection classes.
As I doubt you'll do that, Joel's suggestion seems the most practical.
Trent
Linas Vepstas
> On Tue, Mar 10, 2009 at 9:58 AM, Linas Vepstas <linasv...@gmail.com> wrote:
>> OK. uninstall your odbc devel environment. If you don't need
>> it, don't compile it, don't test it. But if you do compile it, then
>> you *must* test it. You can't just disable the test because you feel like it.
>
> The correct way to unit test this stuff is to stub the db connection classes.
Disagree. If you are not going to use the DB, don't compile
the code. But if you are going to compile the code, then
you must run the test case.
--linas
Trent Waddington
> Stop flaming.
Stop burying your head in the sand.
> I've given you a very simple solution to this: just
> don't compile the database-specific code. If you
> don't compile it, then the test won't be run.
I want the DB specific code.. I want to run the unit test.. I just
want it to pass.
> You've already done some of the initial configuration.
> As a result of this, the database code is getting compiled,
> and the test is being run. Finish doing the configuration,
> and the test will pass.
Huh? You do the configuration.. automatically.. that's what you need
to do to write a unit test.
> However, please don't complain that the test fails, even
> though you clearly have a partially configured system,
> and aren't bothering to finish up. Either finish the
> configuration to the end, or undo/uninstall the ODBC
> pieces that you've done so far. Either will lead to the
> case where the tests pass.
Ok, you're suffering under the delusion that the unit test is supposed
to test the db library.. it isn't. That's the db library developers
interest, not ours. We want to test the code that we wrote.. the way
to do that is to stub the db library. This is unit testing 101 here.
I'm going to ask you again, please fix your unit test. If you don't,
then I'm going to ask you to please stop checking in code, cause
you're being inconsiderate and uncooperative. If you don't, then I
will, but hey, that's what you wanted all along isn't it. Jerk.
Trent
Linas Vepstas
> On Tue, Mar 10, 2009 at 10:54 AM, Linas Vepstas <linasv...@gmail.com> wrote:
>> I've given you a very simple solution to this: just
>> don't compile the database-specific code. If you
>> don't compile it, then the test won't be run.
>
> I want the DB specific code.. I want to run the unit test.. I just
> want it to pass.
In that case, you have to finish performing the configuration.
>> You've already done some of the initial configuration.
>> As a result of this, the database code is getting compiled,
>> and the test is being run. Finish doing the configuration,
>> and the test will pass.
>
> Huh? You do the configuration.. automatically..
Unfortunately, if you wish to use the database code,
you will have to perform manual configuration. This
step is not currently automated, and there is no
particularly easy way of automating it.
> I'm going to ask you again, please fix your unit test.
The test case is not broken. Your installation is broken.
You have failed to correctly install and configure your
system. If you install and configure correctly, then the
test case will pass.
--linas
--linas
Trent Waddington
> The test case is not broken. Your installation is broken.
> You have failed to correctly install and configure your
> system. If you install and configure correctly, then the
> test case will pass.
Dude, I refuse to argue with you on this. A unit test should not
connect to a local db. Period. If you don't agree then you've got no
right to be writing unit tests. Sheesh.
If you insist on connecting to a local db, make it transparent for the user.
Hide your shame.
Trent
Joel Pitt
Linas, you're stubbornly refusing to acknowledge that, for everyone
bar you, the unit test won't work. We'll have to create a database and
change the opencog.conf file to use a user other than "linas". Will
you care if one of us changes this to their username and database
information, resulting in you having to change the conf file every
time you update? (or I guess ignoring changes to the conf file). I
think my suggestion of only carrying out the tests if the conf file
doesn't have blank entries for the database login information is a
good compromise.
Trent, you're turning nasty. Despite your protestation, name calling
does not convince other people to your view.
I suggest we don't make any further follow-ups to this thread until at
least 24 hours have passed. I don't have any official position to do
this, it's just a suggestion since the current direction of this
thread isn't helping anything.
J
Linas Vepstas
> Hey guys, lets just take a step back for the moment.
>
> Linas, you're stubbornly refusing to acknowledge that, for everyone
> bar you, the unit test won't work.
I'm not trying to be stubborn. I am trying to point out that
the reason that it won't work is because everyone's system
is mis-configured.
*IF* you don't want to use the database, then UNINSTALL
the unixodbc-dev package! Then the database code
will not be built, and the test will not be run. All the other
tests will pass.
*IF* you DO want to use the database, then you MUST
configure it properly. Otherwise the database will not
be usable, and will fail to function.
> We'll have to create a database and
> change the opencog.conf file to use a user other than "linas".
Yes, all of that is a part of the config process.
>Will
> you care if one of us changes this to their username and database
??
Every user must change the config file to match their
specific setup. Different users will have different config
files. That's the whole point of having a config file in the
first place.
> good compromise.
Well, I was waiting for someone to propose the obvious
compromise position, and that would be to NOT build the
database code, even though the unixodbc-dev package
is installed. Those users who do want to have database
support would have to type some special cmake incantation
to enable the building of the database code. However, those
users would then also have to perform *all* of the steps
needed to get the database code working.
I find this conversation bizarre ... why did everyone decide
to install unixodbc-dev? Presumably because they
thought it might be fun to play with the database. But
then, half way through, everyone changed their mind,
and decided its too much work. The obvious solution
is to uninstall the devel package, and then everything
returns to normal, all tests pass, and all arguments go
away.
> least 24 hours
sorry, I finished writing above, before I saw this.
--linas
Trent Waddington
> I'm not trying to be stubborn. I am trying to point out that
> the reason that it won't work is because everyone's system
> is mis-configured.
Hehe, you're right, that's not being stubborn.. You honestly think
it's ok for a unit test to talk to a local db don't ya? Here's an
idea, why don't you go read the 4 million flaming opinions on this
stupidity so I don't have to repeat them here.
>> good compromise.
>
> Well, I was waiting for someone to propose the obvious
> compromise position,
Learn what a unit test is and then come back and apologize for your ignorance.
Learn what a stub is, then come back and implement one for the db lib
so your unit test is actually testing the code under test in an
acceptable isolated manner.
In the mean time, disable the test cause you clearly don't know what
you're doing.
Trent
David Hart
Trent is correct, this stuff is unit testing 101.
The unit test should be disabled (or run conditionally, per Joel's suggestion) until it's rewritten correctly (stubbing the db connection classes, running with no specific system configuration, etc.).
-dave
David Hart
Oops, sorry for breaking the 24-hour timeout on this thread.
No more from me for 24 hours.
-dave
Linas Vepstas
> Hi,
>
> Trent is correct, this stuff is unit testing 101.
OK, if we're going to split hairs, then no, this is not
unit testing, its "program verification testing" aka PVT.
Unit tests are things that developers write while the
develop their code; they involve scaffolding that is
in place while the unit is being developed; the
scaffolding is dismantled as the unit is completed,
and, upon full completion, both the scaffolding and
the unit test are deleted (although traces of the
scaffolding and the tests can typically be found in early
versions of the tree).
As far as I know, there is exactly one unit test that is
checked into the opencog bzr tree, and I wrote it.
I should probably delete it, as it no longer serves
a purpose.
All of the tests in the opencog tests directory are
properly called "PVT tests", and not "unit tests".
They verify that the various components of opencog
operate in harmony with one-another. Some people
mistakenly call this "system tests" or "system
verification tests" (SVT), but that term is inappropriate,
because these are not actually testing the system as a
whole (which would require populating the system with
and then running tests on the data.)
One easy way to tell the difference between unit tests
and PVT tests is that unit tests are never automated,
and PVT tests are always automated. PVT tests are
automated so that you can quickly and easily see if
they work. Unit tests are never automated, because
they depend on scaffolding which may no longer exist,
they may test functions that have since been refactored,
obsoleted, etc. Unit tests are highly variable, and change
as the code is being developed, and are discarded when
the code is complete.
> The unit test should be disabled (or run conditionally, per Joel's
> suggestion) until it's rewritten correctly (stubbing the db connection
> classes, running with no specific system configuration, etc.).
Jeez. Just remove the unixodbc-devel package, and
the PVT test will not be run.
--linas
Trent Waddington
On Mar 10, 3:11 pm, Linas Vepstas <linasveps...@gmail.com> wrote:
> Jeez. Just remove the unixodbc-devel package, and
> the PVT test will not be run.
you're still in it is and idiot too.
Trent
Joel Pitt
For those playing at home, Trent's opinion's are his own and not the
official opinion of the OpenCog project,
;P
J
Joel Pitt
respected it, but still, it's the principal...
On Tue, Mar 10, 2009 at 6:11 PM, Linas Vepstas <linasv...@gmail.com> wrote:
>
> 2009/3/9 David Hart <dh...@novamente.net>:
>> Trent is correct, this stuff is unit testing 101.
>
> OK, if we're going to split hairs, then no, this is not
<snip stuff about unit and program test semantics>
Ok, maybe this was the formal definition that people used to use.
Nowadays "unit tests" refer to a range of tests that are not removed
from the system once scaffolding is implemented fully... they are
continuously checked on every build. They use stub objects (or Mock
objects in Cxxtest terminology
http://cxxtest.sourceforge.net/guide.html#TOC52 ) to minimise
dependencies on outside systems (since you can't always explicit
control whether those outside systems give your errors, how can you
test that your code gracefully handles them?).
Sometimes it's easier however to just use existing software or objects
in the unit tests, and I don't think there's anything wrong with that,
especially when developer time is in short supply.
>> The unit test should be disabled (or run conditionally, per Joel's
>> suggestion) until it's rewritten correctly (stubbing the db connection
>> classes, running with no specific system configuration, etc.).
>
> Jeez. Just remove the unixodbc-devel package, and
> the PVT test will not be run.
This is not an option, as some people need that package for
development on other software.
I'm happy to have the persist code only compiled if cmake is
explicitly told to. Is this satisfactory for you? I can implement it
if necessary.
J
Ben Goertzel
That kind of explicitly insulting language is just not permissible on
this mailing list ... sorry...
I agree Linas is being a bit stubborn here; but he's being civil in
defending his view, and if you want to stay on the list you'll need to
be civil too.
(I recognize Linas has not always been civil in the past, but that's
beside the point; I'm talking about the current interaction.)
Thanks
Ben
Ben Goertzel, PhD
CEO, Novamente LLC and Biomind LLC
Director of Research, SIAI
b...@goertzel.org
"I don't believe in dying. It's been done. I'm working on a new exit."
-- George Burns
Ben Goertzel
I have not been following this thread carefully, but I'm alarmed by
the level of nastiness that has developed.
Let's restore civility please. There are no idiots here, and no one
deserving of insult...
According to my limited understanding, it seems Joel's suggestion
would resolve the problem adequately for all concerned.
Linas, do you agree?
Thanks
Ben
Linas Vepstas
>
> According to my limited understanding, it seems Joel's suggestion
> would resolve the problem adequately for all concerned.
>
> Linas, do you agree?
Yes.
Thanks,
--linas