Testing

[lpg]

Hi all,

At the risk of repeating myself: IMHO it would be immensely valuable to have a tightly integrated testing solution baked into w2p.

Can we bump up the priority of this topic in the roadmap?

https://trello.com/c/gWKAOtaC/52-application-tests-integration-https-github-com-viniciusban-web2py-test-and-https-github-com-niphlod-welcome-augmented

We're currently using testrunner.py by jonromero for unit tests. It's a great tool, but it's dated (from 3 years ago). Issues with scopes and DB fixtures require considerable learning curve, so it has kept us from making more advanced unit and functional tests. 

Am I missing another contribution that is better suited, documented, and endorsed by this fine group of developers?

That's all, 

Lucas

[Anthony]

+1

[Vinicius Assef]

+1.

I've been working in a solution. I'll make a small update to it today.

It is http://github.com/viniciusban/web2py.test

> --
> -- mail from:GoogleGroups "web2py-developers" mailing list
> make speech: web2py-d...@googlegroups.com
> unsubscribe: web2py-develop...@googlegroups.com
> details : http://groups.google.com/group/web2py-developers
> the project: http://code.google.com/p/web2py/
> official : http://www.web2py.com/
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py-develop...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Niphlod]

to my knowledge, at least one company has been using welcome_augmented and contributed a little bit back. Until someone starts using those projects nobody will care to update/refine/add features 

[Richard Vézina]

+10

To make it easier to make test in app!

Richard

[lpg]

Hi Simone, that was me I think  :)

Would love to have others.

Thanks for the +1s

[Massimo DiPierro]

I agree with you and others. I apologize I have not dedicated enough time to this the more time goes by the more I see being an important issue.

[lpg]

Here's a nice guide on testing tools for:  unit, functional, behavioral, (+ load, fuzz, layout).

http://www.slideshare.net/ericholscher/testing-slides-1387499

It's also a nice comparison to the state of django testing...

[Vinicius Assef]

Yes, it is a good resource, a little outdated and we must keep in mind
some points:
1) Don't use fixtures from external source to your test. It's has been
proven a bad practice. In web2py we have the better populate()
approach.
2) A good start point is not to accept a new pull request or issue fix
without tests.

As said in this presentation: tests are not difficult, but you need to start.

[lpg]

Hi Vinicius, 

Does it make sense for your test suite to be included in Niphlod's  welcome_augmented? That way we can aim to have it be part of Welcome?

Best,

L

[Vinicius Assef]

Actually, web2py.test focuses in integration tests, aka, testing controllers.

Welcome_augmented focus on behaviour. They're different approaches but
complementary.

Behaviour tests considers javascript and pages with dynamic
interation. Integration tests, don't.

So, I think yes, it makes sense.

[Jeremy Dillworth]

+1

Vinicius, forgive my ignorance, but what do you mean by "fixtures from external source to your test"? Maybe a quick link for my further reading?

Thanks also, Niphlod for mentioning welcome_augmented. I'm eager to have a look at that. Starting from testRunner, I've had some difficulty getting things to work right with contexts and scopes. In particular running testRunner the "python web2py.py -S app_name" I find things start to work strangely because a context is already setup with a db, request and response object. Admittedly I wasn't focusing on this, I was trying to find quick solutions while doing something else :/

It'd be nice to use multiprocess or some other means to be able to run tests that invoke web2py requests and create contexts. e.g. Django has a test client which lets you login and invoke URLs, then you can pull objects out of the context that would've been passed to the template as well as get the HTML as text. With some splinter integration this could be really nice.

[Vinicius Assef]

Jeremy,
by definition, each test case (unit or integration) should act upon an
empty application and prepare it's own desirable state.

It isn't good having a test case depending on external data because of
context. By definition, too, a test case should not depend on
execution of other test cases. It must act as if it was the only one
test case at all.

But I'm talking about unit and integration tests. Acceptance tests
(mainly made with selenium) can lay on external data, due to
performance problems.

[Jeremy Dillworth]

Thanks for clarifying terms. Those are the standards I already follow for my tests. Each test loads its data and then the db is cleared at the end of each test.

You received this message because you are subscribed to a topic in the Google Groups "web2py-developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py-developers/cIukC7_f0e4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py-develop...@googlegroups.com.

[lpg]

Vinicius, speaking of integrating your work in welcome_augmented. If you want to lead the project I can contribute some resources from my team. 

What are your thoughts on implementing your testing suite using Behave. Does it make sense, or will it be too much extra work to write each unit test?

Best, 

Lucas

[Vinicius Assef]

Sorry for the absense of an answer. I haven't read this mesage.

I don't know Bevahe.
Could you add me to your gtalk and call me for a chat? Let's talk about it. :-)

[lpg]

Thought I might bump this before the 1 year anniversary of the post ;)