> Hi everybody,

> What is the best way to write tests for your plugins and are there
> special standards for that? Should a create a completely dedicated
> testing environment for each plugin I have? How can I invoke the
> testing of such tests? I tried some searching for the subject but
> didn't find a lot of stuff about it. Any help is much appreciated. I
> just moved a lot of replicative business logic to plugins and want to
> write tests for them so I can port them all to 1.2:-)

> Thanks,

> Marijn

> Hi Marijn,

> Unfortunately there is no officially proposed way of testing plugins
> yet. I will post my experiences below though in hope to help you out.

> P.S.: This reply probably is more worth a blog post or two, but I
> don't have a blog. I hope I did not write too much :-)

> Directory Structure
> ========================================
> The biggest question about how to test your plugins probably is that
> of a useful directory structure. There are two ways to organize your
> tests:

> 1. Keeping plugin tests separate from the plugin files.
> ----------------------------------------
> Example:
> /
>  plugin/
>    modules/
>    lib/
>  test/

> The benefit is that your plugin stays very small and application
> developers don't need to care about your tests. The downside though is
> that your SVN structure becomes more complicated to understand (you
> need two "root" directories - the real root, meant for plugin
> developers, and the plugin/ subfolder, meant for plugin users).

> This is the way the symfony project does it. If you look at it's SVN
> repository, it contains all project files including unit and
> functional tests. The only thing included in your project though are
> the subfolders data/ and lib/.

> 2. Keeping plugin tests inside the plugin.
> ----------------------------------------
> Example:
> /
>  lib/
>  modules/
>  test/

> The advantage of this way is a clear SVN structure - there is only one
> root directory that users need to care about. Additionally, plugins
> can be developed from within applications, since the tests are inside
> the plugin and can always be run. The drawback is that the plugin
> becomes bigger since every application using your plugin must also
> contain its tests.

> I have only ever used way 1 that I described here, but I found it to
> be overly complicated. Also I think that the advantages of the second
> way far outweigh the increased plugin size. Thus the rest of this post
> will consider a directory structure as described in way 2.

> sfTaskExtraPlugin by Kris Wallsmith supports creating such a skeleton
> directory structure. Read more about this plugin on its release page:

> http://www.symfony-project.org/blog/2008/11/08/additional-tasks-to-st...

> Running Tests
> ========================================
> Running LIME tests is easy. All you need to do is launch them with the
> command "php".

>  php path/to/my/test.php

> Doing this manually might be a tedious job though. Fortunately,
> symfony provides the tasks "test:all", "test:functional" and
> "test:unit" to launch a batch of tests at once, but these tasks do not
> include tests located inside plugins.

> Klemens Ullmann and I have created a patch for these tasks which
> changes this behaviour. You can enable specific plugins to be always
> tested when running the "test:xxx" tasks and launch custom sets of
> plugin tests. Read more about this patch on the wiki page
> http://trac.symfony-project.org/wiki/PluginTesting

> Plugin Fixtures
> ========================================
> Fixtures are test data that are required to run tests within a
> specified test bed. For functional tests, fixture applications are a
> good practice. For your model unit tests you will need a given
> preloaded database setup instead.

> Fixture Database
> ----------------------------------------
> Both your functional and model unit tests probably need access to some
> database. I recommend to name the database equally to the plugin name
> in underscore writing with the suffix "_test" (e.g.
> sf_my_plugin_test).

> A problem arises though when people want to launch your plugin tests
> from within their applications, since they probably have different
> database users with different access rights than you do. To work
> around this problem, I created a separate user for each plugin, whose
> name and password was the name of the plugin in underscore writing
> (e.g sf_my_plugin) and who had only access to the specific databases
> required for the tests.

> The obvious downside of this problem is that a customization of the
> database server is required to run the plugin tests successfully. I
> don't know how to solve this though.

> Fixture Applications
> ----------------------------------------
> Fixture applications are sample applications that are dedicated to
> running the functional tests of your plugin. Again, sfTaskExtraPlugin
> provides ways of pregenerating such a fixture application inside the
> plugin's test/fixtures/project/ directory.

> Whenever you run the functional tests of your plugin, this sample
> application will be used as test application. Thus you can and should
> customize it to the needs of your tests.

> Database Fixtures
> ----------------------------------------
> Database fixtures help you to turn your test database into a
> predefined state (schema and data). For example, when testing the
> peer/table class of the model "User" you might assume that a set of
> records already exists in the database to test custom query methods.

> One way to achieve this is to fill your database by hand by creating
> and saving the required records inside the test files. This is
> complicated and prone to errors though, since a little mistake in
> creating/saving the records might well invalidate your tests.

> A different way to do this is to automatically recreate the database
> and to load YAML fixture files before running each test. This is
> basically what the "propel/doctrine:build-all-reload" tasks do. I
> found this to be a very convenient way to organize your fixtures,
> since editing the YAML files is a fast and easy process.

> I created a plugin called sfDoctrineTestPlugin that supports exactly
> this automatic reloading of a test database based on YAML fixture
> files. This plugin is not released yet though due to a simple reason:
> It's slow. Recreating and reloading the database before every test
> case slows testing down a lot, which might prevent your from writing
> tests as much as you should. If anyone knows a possible solution for
> this problem, I'd be glad to hear about it.

> A different plugin that seems to do this (but not in a very
> sophisticated manner) is sfModelTestPlugin. I found at least the
> Doctrine part to be broken though.

> Conclusion
> ========================================
> After this lengthy post I want to conclude with my personal plugin
> testing preferences:

> 1. Keep the plugin tests inside your plugin.
> 2. Use sfTaskExtraPlugin to create your plugin directory structure
> including test directories and empty fixture applications.
> 3. Install the patch for the "test:xxx" tasks to easily run batches of
> plugin tests from within your application (I hope that this patch will
> be included in 1.3).
> 4. Use YAML fixture files to fill your databases with predefined sets
> of data. I will try to release sfDoctrineTestPlugin soon together with
> a couple of guides.

> That's it. I hope I could help you :-)

> Bernhard

> On Tue, Dec 9, 2008 at 3:17 PM, Marijn <marijn.huizendv...@gmail.com>
> wrote:

> > Hi everybody,

> > What is the best way to write tests for your plugins and are there
> > special standards for that? Should a create a completely dedicated
> > testing environment for each plugin I have? How can I invoke the
> > testing of such tests? I tried some searching for the subject but
> > didn't find a lot of stuff about it. Any help is much appreciated. I
> > just moved a lot of replicative business logic to plugins and want to
> > write tests for them so I can port them all to 1.2:-)

> > Thanks,

> > Marijn