Adding a Testsuite option to DefaultEnv of Yesod.Default.Config

[Erik de Castro Lopo]

Hi Greg and Yesodweb,

I chatted to Michael about this on #yesod and he thought it was a good
idea, but that I should solicit input from Greg.

Currently for DefaultEnv we have; Development, Testing, Staging and
Production. I would like to see the addition of one more; Testsuite.

Once used of the Testsuite environment would be for a database that
gets dropped at the start of each run of the testsuite. The developer
would do his in-development testing with the Development database
(which he/she may keep around for days or even weeks) but each time
he/she runs 'cabal test' the tests can drop the table before running
the tests to make sure that all DB tests operate on a DB in a known
state.

Obviously, I can roll my own replacement for DefaultEnv, but I thought
this is something that would be more widely useful.

Cheers,
Erik
--
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/

[Greg Weber]

I guess this is what I envisioned the testing environment would be for. What is the difference between testing and testsuite then?

[Michael Snoyman]

I thought we use them for our different environments: Development for local testing, Testing for a non-public server deployment, Staging for non-public server deployment which may be promoted to production, and Production for... well... production. Testsuite would be filling a different role.

To be honest, at FP Complete we've been putting more of the configuration in environment variables instead of YAML files, so we could almost get rid of the environment distinctions. But I think for many use cases, doing config in the YAML file will still be easier.

--
You received this message because you are subscribed to the Google Groups "Yesod Web Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to yesodweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Kirill Zaborsky]

Would share some example how you do it with evironment variables?

Seems to be a good receipt for Yesod cookbook.

Kind regards,

Kirill Zaborsky

2013/4/9 Michael Snoyman <mic...@snoyman.com>

[Michael Snoyman]

The basic idea is simple: read environment information in getApplication, process it somehow, and put the results into the foundation datatype. I'm planning on adding a number of new examples to the book with the 1.2 release; I'll include an example of this technique.

[Kirill Zaborsky]

So online book will support only 1.2 in near future?

The reason I'm asking is that Russian translation of the book is almost finished. So It's not clear to me whether we need to take into account your futher changes or there will be some 1.1 version online so people who buy Russian book won't be confused from differences between the book and online version (and haddocks).

Kind regards,

Kirill Zaborsky

[Michael Snoyman]

That was pretty insane timing... I just added an item to my personal worklist to provide both Yesod 1.1 and 1.2 docs from the website. I think the only sustainable approach forward will be to allow users to read docs on whatever version they're using.

[Greg Weber]

So do you use Testing as a sanity check before Staging then? I can see how this could be very useful if you are concerned about breaking staging. In that case I would probably call Staging something like Beta and call Testing Staging.

I think most people will start their development by not needing Testing and being willing to break Staging: it makes more sense to me to not have the Testing concept by default and instead have the proposed TestSuite (could be shortened to Test) environment that is designed for running automated tests.

[Michael Snoyman]

No, we use testing for testing :). Not everyone is a dev, and everyone else needs to be able to access pre-release versions of the site for testing and/or to understand what it's providing. Testing is basically "I want people to see this, I'll put it up." It's also a real environment, unlike development, and so can demonstrate real problems.

[Greg Weber]

In all of my limited professional experience, the pre-release production-like environment has been called staging, and the test environment has been for running automated tests. This is the convention that Rails uses. It looks like Django [1] and node.js [2] for example have a similar naming convention at least for the test environment, although I am not familiar with them, this is just from googling.

[1] https://docs.djangoproject.com/en/1.4/topics/testing/

[2] http://brianstoner.com/blog/testing-in-nodejs-with-mocha/

So while the naming conventions being used by Michael are perfectly fine, I am worried that they are going against modern convention. But perhaps there are other frameworks using different conventions that someone can point to an example of? Even still, I think using the names Testing & TestSuite are not ideal since they both being with 'Test' but their environment will be completely different.

[David Thomas]

For what little it's worth, my assumption based on the names was that Testing should be to run the automated tests against.

[Eric Rasmussen]

I wouldn't take issue with Yesod choosing to name things differently if there's a reason, but here's what I'm used to from the python world:

Testing: running test suites (unit tests, functional tests, acceptance tests, etc.)
Alpha: a real environment made available to non-developers for quality assurance and interactive testing
Stage: the pre-production environment

I think everyone agrees on the meaning of development and production, but I do think the current use of "testing" is misleading, and falls more in line with what's commonly known as alpha testing. It's worth noting that this isn't specific to python, and this terminology goes way back:

https://en.wikipedia.org/wiki/Software_release_life_cycle

I don't believe beta testing is too common for modern web applications (at least in my circles), unless it's a public beta. I am most familiar with Development -> Alpha -> Stage -> Production, having the Testing environment existing in parallel as a way to execute automated tests at each step.

[Michael Snoyman]

I don't really want to get into a bikeshedding discussion about naming; I use the environments the way I do because those are the needs I'd seen, and those are the environment names provided by default. If we want to change them, I'm not opposed (besides wanting to be careful about any unnecessary breaking changes). But we really do need to support the concept of an environment somewhere between development and staging.

[Greg Weber]

Currently there is actually documentation in that generated scaffolding that Nubis originally added about Testing being for running automated tests:

https://github.com/yesodweb/yesod/blob/yesod1.2/yesod-bin/hsfiles/mongo.hsfiles#L5292

I like alpha + staging. We should probably just make sure there is no place in the yesod that is actually hard-wired to an environment name so that users can add or rename environments as needed and Yesod needs not worry about supporting any environment. So what we are talking about here is then just what the scaffolding suggests, but it is easy enough to change the generated code.