[dev] Test related improvements

[Michael M Slusarz]

1) We should add a test runner that runs all *application* tests (as
opposed to framework tests)

2) horde_lz4 is currently our only PECL package. AFAICT, PHPUnit does
not support PECL style tests - we are limited to built-in PEAR tests.
Correct?

3) If #2 is correct, we should also add a test runner that runs PECL tests.

4) Maybe a question for Gunnar: Looks like we have some nice
Horde_Test "factories" that create dummy objects for testing for
things like Horde_Db and Horde_Cache. I used this in the new Imap
Client DB tests, for example:

https://github.com/horde/horde/blob/master/framework/Imap_Client/test/Horde/Imap/Client/Cache/DbTest.php#L30

Wondering if this is the correct intended usage.

5) Ran into the test paradigm yesterday where I wanted to test
multiple backends with the same data/tests. For now, I created one
test as the "master" (e.g. Horde_Imap_Client_Test_CacheTest), and then
overwrote only the parts needed to setup the backend in another test
(Horde_Imap_Client_Test_DbTest).

Obviously, the better solution would be to create a "master" test that
all active tests directly extend. The problem with this approach is
that the "master" test must extend PHPUnit_Framework_TestCase itself
and, as such, it will attempt to be run as a unit test by the phpunit
test runner.

You hackishly could work around this by having the base setUp() method
call markTestSkipped -- and have the extended class overwrite this --
but this is not a proper solution since it will cause tests to appear
as being skipped which is not the case.

Any ideas on how to do this properly? I'm sure there is a way to tell
phpunit to "filter" the name of files not to run, but this also seems
hackish.

michael

___________________________________
Michael Slusarz [slu...@horde.org]

--
dev mailing list
Frequently Asked Questions: http://wiki.horde.org/FAQ
To unsubscribe, mail: dev-uns...@lists.horde.org

[Jan Schneider]

Zitat von Michael M Slusarz <slu...@horde.org>:

> 1) We should add a test runner that runs all *application* tests (as
> opposed to framework tests)

This could be the same runner. The only reason that we didn't do this
so far is the fact that the application test are mostly in a bad shape
at this moment.

> 2) horde_lz4 is currently our only PECL package. AFAICT, PHPUnit
> does not support PECL style tests - we are limited to built-in PEAR
> tests. Correct?
>
> 3) If #2 is correct, we should also add a test runner that runs PECL tests.

Is there something like special PECL tests? I expected those were
normal PHPT test, testing the interfaces of a *built* extension. I.e.
they don't test or even include building of the extension. But I might
be wrong.

> 4) Maybe a question for Gunnar: Looks like we have some nice
> Horde_Test "factories" that create dummy objects for testing for
> things like Horde_Db and Horde_Cache. I used this in the new Imap
> Client DB tests, for example:
>
> https://github.com/horde/horde/blob/master/framework/Imap_Client/test/Horde/Imap/Client/Cache/DbTest.php#L30
>
> Wondering if this is the correct intended usage.
>
> 5) Ran into the test paradigm yesterday where I wanted to test
> multiple backends with the same data/tests. For now, I created one
> test as the "master" (e.g. Horde_Imap_Client_Test_CacheTest), and
> then overwrote only the parts needed to setup the backend in another
> test (Horde_Imap_Client_Test_DbTest).
>
> Obviously, the better solution would be to create a "master" test
> that all active tests directly extend. The problem with this
> approach is that the "master" test must extend
> PHPUnit_Framework_TestCase itself and, as such, it will attempt to
> be run as a unit test by the phpunit test runner.
>
> You hackishly could work around this by having the base setUp()
> method call markTestSkipped -- and have the extended class overwrite
> this -- but this is not a proper solution since it will cause tests
> to appear as being skipped which is not the case.
>
> Any ideas on how to do this properly? I'm sure there is a way to
> tell phpunit to "filter" the name of files not to run, but this also
> seems hackish.

We do this already in many places where we have different backends,
e.g. Horde_Group, Horde_Share, kronolith, turba. The trick is to call
the base class something like TestBase.php, i.e. not ending with "Test".
--
Jan Schneider
The Horde Project
http://www.horde.org/

[Michael M Slusarz]

Quoting Jan Schneider <j...@horde.org>:

> Zitat von Michael M Slusarz <slu...@horde.org>:
>

>> 2) horde_lz4 is currently our only PECL package. AFAICT, PHPUnit
>> does not support PECL style tests - we are limited to built-in PEAR
>> tests. Correct?
>>
>> 3) If #2 is correct, we should also add a test runner that runs PECL tests.
>
> Is there something like special PECL tests? I expected those were
> normal PHPT test, testing the interfaces of a *built* extension.
> I.e. they don't test or even include building of the extension. But
> I might be wrong.

Yeah, the difficulty in requiring a build/compile complicates things.
For now, I think the current status quo is fine. I'm not expecting
many (any?) changes in horde_lz4 in the future, and if they do occur
it is not unreasonable to run the PECL unit tests manually.

>> 5) Ran into the test paradigm yesterday where I wanted to test
>> multiple backends with the same data/tests. For now, I created one
>> test as the "master" (e.g. Horde_Imap_Client_Test_CacheTest), and
>> then overwrote only the parts needed to setup the backend in
>> another test (Horde_Imap_Client_Test_DbTest).
>>
>> Obviously, the better solution would be to create a "master" test
>> that all active tests directly extend. The problem with this
>> approach is that the "master" test must extend
>> PHPUnit_Framework_TestCase itself and, as such, it will attempt to
>> be run as a unit test by the phpunit test runner.
>>
>> You hackishly could work around this by having the base setUp()
>> method call markTestSkipped -- and have the extended class
>> overwrite this -- but this is not a proper solution since it will
>> cause tests to appear as being skipped which is not the case.
>>
>> Any ideas on how to do this properly? I'm sure there is a way to
>> tell phpunit to "filter" the name of files not to run, but this
>> also seems hackish.
>
> We do this already in many places where we have different backends,
> e.g. Horde_Group, Horde_Share, kronolith, turba. The trick is to
> call the base class something like TestBase.php, i.e. not ending
> with "Test".

Thanks for the pointer. This has been fixed up in Horde_Imap_Client.

michael

___________________________________
Michael Slusarz [slu...@horde.org]

[Jan Schneider]

Zitat von Michael M Slusarz <slu...@horde.org>:

> Quoting Jan Schneider <j...@horde.org>:
>
>> Zitat von Michael M Slusarz <slu...@horde.org>:
>>
>>> 2) horde_lz4 is currently our only PECL package. AFAICT, PHPUnit
>>> does not support PECL style tests - we are limited to built-in
>>> PEAR tests. Correct?
>>>
>>> 3) If #2 is correct, we should also add a test runner that runs PECL tests.
>>
>> Is there something like special PECL tests? I expected those were
>> normal PHPT test, testing the interfaces of a *built* extension.
>> I.e. they don't test or even include building of the extension. But
>> I might be wrong.
>
> Yeah, the difficulty in requiring a build/compile complicates
> things. For now, I think the current status quo is fine. I'm not
> expecting many (any?) changes in horde_lz4 in the future, and if
> they do occur it is not unreasonable to run the PECL unit tests
> manually.

If building isn't covered by PHPT either, why use it? You can use
PHPUnit test like with any other package.

Jan Schneider
The Horde Project
http://www.horde.org/