Using Cucumber for a PHP app: Why, What, and How?

[Andrew Havens]

I have been impressed with what I have seen of Cucumber so far. I am
interested in using it to test some PHP applications. I started
setting up Cucumber with Cuke4PHP but have now reached a point where I
need to better understand Cucumber and some BDD philosophies before I
can go any further.

I think I understand how to use Cucumber for writing Selenium tests.
Our team has a bunch of brittle Selenium tests written in PHP that I
would like to replace. Actually, our team only has Selenium tests (not
even any unit tests). The part I don't understand that well is setting
up test data. What are some best practice approaches to creating/
deleting test data and running the application using that test data?

Another part I don't understand is how Cucumber works outside of the
context of Selenium. Also, being new to Ruby, I'm not sure I
understand what RSpec is for and how it ties in to using Cucumber.
Also...should I be thinking about using Cucumber as a wrapper for unit
tests as well?

Thanks,
Andrew

[Matt Wynne]

On 7 Feb 2011, at 21:00, Andrew Havens wrote:

> I have been impressed with what I have seen of Cucumber so far. I am
> interested in using it to test some PHP applications. I started
> setting up Cucumber with Cuke4PHP but have now reached a point where I
> need to better understand Cucumber and some BDD philosophies before I
> can go any further.
>
> I think I understand how to use Cucumber for writing Selenium tests.
> Our team has a bunch of brittle Selenium tests written in PHP that I
> would like to replace. Actually, our team only has Selenium tests (not
> even any unit tests). The part I don't understand that well is setting
> up test data. What are some best practice approaches to creating/
> deleting test data and running the application using that test data?

I don't like the term 'best practice' but I'll tell you what has worked for me, and some things I've seen other people do that have made them struggle.

Wheras with manual test scripts, you might have had one scenario build on the data from another, Cucumber is designed to reset the state of the system between each scenario. This helps your tests be less brittle, because changing one scenario doesn't affect the data that will be seen by another. Each scenario is thus responsible for setting up the data that it needs. This makes tests easier to read, because all of the state for the scenario can be seen in your Given steps.

The down-side of this is that, if you have a complex schema, you can find yourself needing to set up a lot of test data to get to 'ground zero' during each scenario. This can make your tests slow. I've generally found this is a price worth paying for reliable tests.

Also, you can end up with a lot of noise in your Given steps. For example, if you're testing an application that requires every user to be authenticated, it's pointless to specify, in every single scenario Given I have a valid account And I am logged in etc. You're better off doing that implicitly with a hook (see Cucumber's Wiki).

I don't have time to write more now, but hopefully that helps.

> Another part I don't understand is how Cucumber works outside of the
> context of Selenium. Also, being new to Ruby, I'm not sure I
> understand what RSpec is for and how it ties in to using Cucumber.
> Also...should I be thinking about using Cucumber as a wrapper for unit
> tests as well?

In Cucumber, RSpec is basically used for assertions. Ruby people who use Cucumber tend to also be RSpec users, but you don't need to use RSpec with Cucumber. You can use any assertion library you like.

I would avoid the temptation to use Cucumber for unit tests. In my experience, testing works best with a two-layer combination of end-to-end tests (e.g. Cucumber) that prove you're building the right thing, and low-level unit tests (e.g. RSpec) that prove you're building the thing right. These tests are for different audiences and so suit different tools.

> Thanks,
> Andrew

cheers,
Matt

ma...@mattwynne.net
07974 430184

[Aslak Hellesøy]

On Feb 7, 2011, at 17:00, Andrew Havens <misbe...@gmail.com> wrote:

> I have been impressed with what I have seen of Cucumber so far. I am
> interested in using it to test some PHP applications. I started
> setting up Cucumber with Cuke4PHP but have now reached a point where I
> need to better understand Cucumber and some BDD philosophies before I
> can go any further.
>

I don't have time to give philosophical advice today, just thought I'd point out

http://everzet.com/Behat/

You may want to compare it with Cuke4PHP.

Aslak

> I think I understand how to use Cucumber for writing Selenium tests.
> Our team has a bunch of brittle Selenium tests written in PHP that I
> would like to replace. Actually, our team only has Selenium tests (not
> even any unit tests). The part I don't understand that well is setting
> up test data. What are some best practice approaches to creating/
> deleting test data and running the application using that test data?
>
> Another part I don't understand is how Cucumber works outside of the
> context of Selenium. Also, being new to Ruby, I'm not sure I
> understand what RSpec is for and how it ties in to using Cucumber.
> Also...should I be thinking about using Cucumber as a wrapper for unit
> tests as well?
>
> Thanks,
> Andrew
>

> --
> You received this message because you are subscribed to the Google Groups "Cukes" group.
> To post to this group, send email to cu...@googlegroups.com.
> To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cukes?hl=en.
>

[_Kevin]

I've been doing a fair amount of testing of PHP code lately using
cucumber and I can offer the following suggestions...

For the most part use PHPUnit to do unit level tests. You *can* write
unit style tests using cuke4php, but that doesn't mean you should.
PHP doesn't really have a unit testing library that is as flexible or
expressive as RSpec (at least not one I've found), but PHPUnit is
pretty good. I have used cuke4php to do behavioral testing on very
complicated legacy classes (more like functional tests). It's biggest
value there has been to help document hidden/existing behavior of
legacy code (which helps when communicating with product owners, QA
testers, and is also useful for new developers). These sorts of tests
will be inherently slower than PHPUnit tests, so use them carefully or
be prepared for your test runs to take longer.

The other place that cuke4php is potentially useful is to allow
integration tests to talk to a PHP backend. In a rails project,
cucumber can talk directly to ActiveRecord and directly interact with
the database as needed. This normally isn't possible with a PHP
project, but can be achieved using cuke4php. When used this way, you
have the ability to mix and match step definitions in ruby and PHP as
needed. This was the primary motivation for developing cuke4php. We
have a fairly extensive set of integration tests built with cucumber +
celerity, but were frustrated by the inability to talk directly to the
backend so that we could enforce certain states during testing.

Most of my experiences have been around adding cucumber style tests to
an existing suite of integrated PHP applications. I might have made
different design choices if I were building a new application from
scratch and had more control over how things were tested from the
beginning.

-Kevin

On Feb 8, 6:16 am, Aslak Hellesøy <aslak.helle...@gmail.com> wrote:

[Andrew Havens]

Thanks for the advice! Probably my biggest question I still have is
this: Can/should Cucumber be used by itself to test a PHP web app, or
do I need to/should I use Webrat, Capybara, Watir, Celerity, Culerity,
etc? (I don't fully understand the purpose of all these other
projects, the headless browser idea, and how/if it would work to test
a PHP app). I understand how Selenium works because I've used it
before (outside of Cucumber). Any advice/clarification would be much
appreciated.

Thanks!

Andrew

[Andrew Premdas]

[On 9 February 2011 16:59, Andrew Havens <misbe...@gmail.com> wrote:

Thanks for the advice! Probably my biggest question I still have is
this: Can/should Cucumber be used by itself to test a PHP web app, or
do I need to/should I use Webrat, Capybara, Watir, Celerity, Culerity,
etc? (I don't fully understand the purpose of all these other
projects, the headless browser idea, and how/if it would work to test
a PHP app). I understand how Selenium works because I've used it
before (outside of Cucumber). Any advice/clarification would be much
appreciated.

Thanks!

Cucumber uses a tools like capybara to communicate with a web browser (headless or real). Capybara, and other similar tools, use other tools such as Selenium to do low(er) level communication with the browser. Capybara provides an easy to use DSL (domain specific language) that you can use in you Cucumber step definitions that makes driving a browser much simpler than using say Selenium directly. You can also use the same tool to drive different browsers, using different drivers in different environments. This means you can use the same step definitions to test in process a headless browser in a rails application as you use to test  out of process a PHP application using Selenium. 

Currently I use Cucumber with Capybara and the Selenium driver to test PHP applications. This stack has the following characteristics

1. Step definitions are written in Ruby (and so are elegant readable and easy to write)

2. Interactions with the browser (in Ruby) use Capybara so I can say things like (page.should have_css("#foo .bar", :text => "baz) )

3. The features can be seen running in the browser (which is useful for business folk - at least initially)

4. The features take along time to run because they are running in a browser.

There are a number of other stacks you can use to test a PHP application which will have different characteristics. I'm not convinced there is any obvious choice, though I am personally happy with the stack I use. IMO ruby and capybara are far better tools to write step definitions than PHP, but hey thats just me :)

HTH

Andrew

--

------------------------

Andrew Premdas

blog.andrew.premdas.org