BDD with Selenium(JBehave)

[Keki Rabadi]

I am a bit confused in the concepts of BDD, my question is from a user
story a scenario is developed after that, is it a UNIT TEST that is
written by a developer or is it an AUTOMATED TEST written by a TESTER,
which is made to fail and then followed by real CODING by the
developer and the same test should pass???
I hope I have put in the right way!

[raghu nandan]

Hi ,

Unit Tests should be written by tester   when the developer has provided empty methods that means with no buisiness logic,then developer should write the logic in order to pass those unit tests written by tester.
Automation units tests are something which tester will start write when the screen is developed and a basic manual test is done.
In some companies I heard that even automation scripts will also implement before the screen is ready when you have given Id of the objects on that screen.


Please correct me if I am wrong.

Regards,
Raghunandan.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To post to this group, send email to seleniu...@googlegroups.com.
To unsubscribe from this group, send email to selenium-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/selenium-users?hl=en.

--
with regards
RaghuNandan

Do you know that it takes 10 litres of water to produce 1 sheet of A4 paper?

Don't print this unless you have to :)

 

[raghu nandan]

add to below writing unit tests before coding is called as  TEST DRIVEN DEVELOMENT [TDD] initiall all test will fail then will pass after the logic will implemented by programmer.

[Thomas Sundberg]

On Thu, May 3, 2012 at 9:01 AM, raghu nandan <nand...@gmail.com> wrote:
> Hi ,
>
> Unit Tests should be written by tester   when the developer has provided
> empty methods that means with no buisiness logic,then developer should write
> the logic in order to pass those unit tests written by tester.

This would be true only if you are doing thing wrong. The unit of
functionality you test if you do it this way would probably be on a
higher level than on a class level or even method level.

Unit tests are written by developers as they drive their code forward
to a solution that has been agreed upon. Tester are more than welcome
to participate in this process, but I would never expect them to write
the actual code in the unit test. This is the way TDD works.

> Automation units tests are something which tester will start write when the
> screen is developed and a basic manual test is done.

Not necessarily true, the original poster asked about BDD and JBehave.
Using tools like JBehave, Cucumber, Robot Framework could help you to
develop the functionality without a user interface. If the system
behaves as expected, the step definitions connecting the test code
with the system could then be re-written to use a tool that will
connect through the user interface. This would result in that the same
test definitions are being used to develop the core functionality and
later, without changing the definitions, used to assert the user
interface. This will my main point at GeeCON 2012 [1]

/Thomas


[1] http://2012.geecon.org/speakers/thomas-sundberg

Thomas Sundberg
M. Sc. in Computer Science

Mobile: +46 70 767 33 15
Blog: http://thomassundberg.wordpress.com/
Twitter: @thomassundberg

Better software through faster feedback

[Keki Rabadi]

Could you guys make it simpler say just in three steps how things are
implemented in BDD I am a novice at this something liem
1- Scenario
2- Test written by a Tester / Developer and what type of test is a a
unit test or a selenium automated test
3- test is runa and it fails or made to fail
4- Developer write the code

On May 4, 3:37 pm, Thomas Sundberg <t...@kth.se> wrote:

> On Thu, May 3, 2012 at 9:01 AM, raghu nandan <nandu...@gmail.com> wrote:
> > Hi ,
>
> > Unit Tests should be written by tester   when the developer has provided
> > empty methods that means with no buisiness logic,then developer should write
> > the logic in order to pass those unit tests written by tester.
>
> This would be true only if you are doing thing wrong. The unit of
> functionality you test if you do it this way would probably be on a
> higher level than on a class level or even method level.
>
> Unit tests are written by developers as they drive their code forward
> to a solution that has been agreed upon. Tester are more than welcome
> to participate in this process, but I would never expect them to write
> the actual code in the unit test. This is the way TDD works.
>
> > Automation units tests are something which tester will start write when the
> > screen is developed and a basic manual test is done.
>

> Not necessarily true, the original poster asked aboutBDDand JBehave.

> Using tools like JBehave, Cucumber, Robot Framework could help you to
> develop the functionality without a user interface. If the system
> behaves as expected, the step definitions connecting the test code
> with the system could then be re-written to use a tool that will
> connect through the user interface. This would result in that the same
> test definitions are being used to develop the core functionality and
> later, without changing the definitions, used to assert the user
> interface. This will my main point at GeeCON 2012 [1]
>
> /Thomas
>
> [1]http://2012.geecon.org/speakers/thomas-sundberg
>
>
>
>
>
>
>
>
>
> > In some companies I heard that even automation scripts will also implement
> > before the screen is ready when you have given Id of the objects on that
> > screen.
>
> > Please correct me if I am wrong.
>
> > Regards,
> > Raghunandan.
>

> > On Thu, May 3, 2012 at 12:05 PM, Keki Rabadi <keki.rab...@gmail.com> wrote:
>
> >> I am a bit confused in the concepts ofBDD, my question is from a user

[Keki Rabadi]

What sort of documents / methods / process are given to a tester to
develop test cases ??
What exact process a Automation tester is expected to follow in an
Agile environment, for managing his testing activities and Automation
Framework???
I havent used a continuous integration system so how do we do that
using say for eg. Jenkins?? looking for technical steps or config
How do we use BDD like JBehave or integrate in our automation
environment??

[Paul Hammant]

I'm one of the developers of JBehave.

1) Business Analysts and QAs work together to agree the grammar/style
of scenarios for the application in question, and between them deliver
some examples.

2) A Developer makes the initial version of the JBehave suite. That'd
be principally 'steps' classes and 'page objects' as well as a single
bootstrap class.

3) With that as an example, BAs would define new stories in JIRA (or
Rally, or Mingle etc) and QAs would pick them up and try to implement
them having understood (2). It may be that some small rework is
needed of things done to that moment in order to capture the growing
understanding of the app. It may be that devs have to be consulted
again.

4) Repeat (3).

-ph

[David]

I can add some perspective on the RobotFramework process:

In step 2 mentioned by Paul, the developer (or it could be QA
automation engineer w/ help from developers) writes the test library
that implements common BDD functions (that can be reused. E.g. login
function where BDD will test against valid and invalid logins, etc.).

In step 3, the functions in the library then can be pieced together
with BDD syntax, incorporating the test data to go along with it. And
these BDD tests are simple enough that QA testers and BAs can write
them.

The developers (or QA automation engineer/architect) only needs to be
consulted again when there is missing functionality for writing a BDD
test that needs to be added in the test library. If the
functionalities are broken down modularly, the developer can implement
those all at once, so that from then on, the BAs and QA testers can
just start writing tests with the library, minimizing all the back and
forth collabaration that might be needed otherwise.

For more details on this for Robot Framework visit the site and read
the User Guide's BDD section on test case syntax.

[Keki Rabadi]

THANKS Paul and David very helpful..
I have some more as well, I am sorry to ask this but I do not
understand BDD much.
When you say AT writes test library, does it mean the library for his
keyword driven framwork ?
Also is it ok for an automation tester to straightaway write his test
in Selenium instead of using Pageobjects ?
Could you demonstrate as to what are the steps and prerequsities to
setup JBehave on a local machine. I use Selenium Webdriver with Java
an Eclipse as a preferred IDE.

[Paul Hammant]

You need a developer to set up JBehave and Selenium for you on your
machine, and write the first test for your site/app.

[Keki Rabadi]

I dont have an app I just finished learning selenium webdriver and
automation testing, thus the question

[Mike Riley]

Paul,

So if I understand what you are saying here, you can't run a test case directly from Rally (which we use here), but instead a QA needs to take what is captured in Rally and write something that works with the JBehave suite that a developer wrote, correct?

I am trying to determine if it is worth me looking into using JBehave before I go much further in writing my own test cases, based on this discussion here.  If there is a good place to learn more about it that you can point me to I would appreciate it.

Thanks.

Mike

>> >> To post to this group, send email to selenium-users@googlegroups.com.

>> >> To unsubscribe from this group, send email to

>> >> selenium-users+unsubscribe@googlegroups.com.

>> >> For more options, visit this group at
>> >>http://groups.google.com/group/selenium-users?hl=en.
>>
>> > --
>> > with regards
>> > RaghuNandan
>>
>> > Do you know that it takes 10 litres of water to produce 1 sheet of A4 paper?
>>
>> > Don't print this unless you have to :)
>>
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "Selenium Users" group.

>> > To post to this group, send email to selenium-users@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > selenium-users+unsubscribe@googlegroups.com.

>> > For more options, visit this group at
>> >http://groups.google.com/group/selenium-users?hl=en.
>>
>> --
>> Thomas Sundberg
>> M. Sc. in Computer Science
>>
>> Mobile: +46 70 767 33 15
>> Blog:http://thomassundberg.wordpress.com/
>> Twitter: @thomassundberg
>>
>> Better software through faster feedback
>
> --
> You received this message because you are subscribed to the Google Groups "Selenium Users" group.

> To post to this group, send email to selenium-users@googlegroups.com.
> To unsubscribe from this group, send email to selenium-users+unsubscribe@googlegroups.com.

[Paul Hammant]

There's no reason why someone could not take a story enshrined in a
page (Rally, Jira etc) and run that at intervals. It'd be a different
story loader to the default one that pulls from the file-system.

Personally, I think stories are best kept under SCM, but I get that
people want to see them in page technologies. Thus an Issue tracker
(Story Manager) that uses SCM as it's backend is one of my
holy-grails.

- Paul

>> >> >> seleniu...@googlegroups.com.

>> >> >> To unsubscribe from this group, send email to

>> >> >> selenium-user...@googlegroups.com.

>> >> >> For more options, visit this group at
>> >> >>http://groups.google.com/group/selenium-users?hl=en.
>> >>
>> >> > --
>> >> > with regards
>> >> > RaghuNandan
>> >>
>> >> > Do you know that it takes 10 litres of water to produce 1 sheet of A4
>> >> > paper?
>> >>
>> >> > Don't print this unless you have to :)
>> >>
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "Selenium Users" group.

>> >> > To post to this group, send email to seleniu...@googlegroups.com.

>> >> > To unsubscribe from this group, send email to

>> >> > selenium-user...@googlegroups.com.

>> >> > For more options, visit this group at
>> >> >http://groups.google.com/group/selenium-users?hl=en.
>> >>
>> >> --
>> >> Thomas Sundberg
>> >> M. Sc. in Computer Science
>> >>
>> >> Mobile: +46 70 767 33 15
>> >> Blog:http://thomassundberg.wordpress.com/
>> >> Twitter: @thomassundberg
>> >>
>> >> Better software through faster feedback
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Selenium Users" group.

>> > To post to this group, send email to seleniu...@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > selenium-user...@googlegroups.com.

>> > For more options, visit this group at
>> > http://groups.google.com/group/selenium-users?hl=en.
>> >
>
> --

> You received this message because you are subscribed to the Google Groups
> "Selenium Users" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/selenium-users/-/df9ihIhnZOEJ.
>
> To post to this group, send email to seleniu...@googlegroups.com.

> To unsubscribe from this group, send email to

> selenium-user...@googlegroups.com.

[Keki Rabadi]

It would be great if you could PLEASE provide some insight to this jargon.

TDD stands at a class level it is done by the developers, BDD is execution of test from the SH perspective using a fixed syntax and grammar.
My Question is: If we are following BDD using Jbehave, we are trying to satisfy the Acceptance Criteria of a story under test and development(using Given, When, Then) . So does that not mean, when we do BDD we are doing Acceptance Testing as well ?
IF I stand correct aren't the tools like Cucumber, Robot and Jbehave categorized as an Acceptance Test Frameworks?
Your Views Please:

If I put in a few basic preliminary steps the team will do something like this

1. During iteration planning, the team agrees on a set of User Stories to be delivered. A User Story describes what the system should do, in a way that emphasizes value to a user.
2. Testers ask questions that elicit examples and clarification
   
3. User stories are given Acceptance Criteria (AC). AC are business oriented tests that let us know if the implementation meets Product Owner expectations
4. Acceptance Criteria are written depending on the methodology followed eg. BDD, Acceptance testing etc.
5. Developers starts writing the code for that particular story

Now if I stop here for a moment, my question is, at this point what sort of Automation is a tester expected to do and using what tools

Your Views Please:

I am completely lost here as to How does Selenium Webdriver Automation fits in this ATTD and BDD, Yes I am aware that the BDD story file gets wired to your selenium test files and configuration files but how becasue the UI will not be up at that point in time

Your Views Please:

When the UI is not up how automation is done using Selenium Webdriver

Your Views Please:

[Mark Collin]

Easier to point you towards Liz Keogh here:

 

http://lizkeogh.com/2011/06/27/atdd-vs-bdd-and-a-potted-history-of-some-related-stuff/

 

Now that you know that ATDD and BDD are the same thing, you need to stop worrying about using a “ATDD” or “BDD” tool.  Tools can be used for many things, it all depends on how you use them.  Selenium can be used as a ATDD/BDD tool if you want it to be, it’s all about how you decide to build your test framework for its use and how you lay out your tests.  You don’t even need a separate Given, When, Then story file, you can do it all programmatically with something like http://code.google.com/p/yatspec/.

 

There is no one size fits all tool for everything; you will need to adapt the tools that you have to achieve what you want.  So if there is no UI don’t use Selenium, use an appropriate tool for the job in hand.

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To post to this group, send email to seleniu...@googlegroups.com.
To unsubscribe from this group, send email to selenium-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/selenium-users/-/Eww81pTbUhwJ.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Keki Rabadi]

Hi Mark Thanks,
Liz Keogh link says a lot about my confusion, and that also means we cannot use Selenium automation in agile except for regression testing. During an iteration we can use tools like finesse for acceptance testing right.
 

[Mark Collin]

Where I am working right now we have weekly sprints and use Selenium to automate everything on the web page we are producing, we don’t use fitnesse, or cucumber or jbehave.  So no it doesn’t mean you can’t use Selenium.

 

Automation is one of the best ways to make regression testing easy, there is nothing that says you can’t write your regression scripts as soon as you get the product in front of you.  We don’t classify a story as done until exploratory testing has been completed and working automated scripts have been written.  These scripts are then run every full build of the product (and all devs need to do a full build before checking code in).  This means that anything that breaks the tests is picked up very quickly and we are almost always in a state where we can release a new version of the site after a new code check in.

 

There is no single defined way to work in an agile environment, what we are doing probably isn’t classed as BDD/ATDD because we are not always writing the automaton scripts before the code (It all depends upon time constraints, some are written in advance some are written as the code is completed).  We do perform TDD though, none of the code is written before unit tests are produced.  The whole point of agile is that you tweak it to work for you and your team, there is no defined right or wrong way, there is a series of principles and you can decide how to implement them to ensure that you have a happy and productive team.

 

In my mind the thing that will decide when you write you Selenium test will be your experience in writing automated test scripts and your familiarity with Selenium.  The more experienced you are the more likely you will be able to write something useful in advance.

To view this discussion on the web visit https://groups.google.com/d/msg/selenium-users/-/LPUiDOL8l3sJ.

[Keki Rabadi]

Thanks Mark for helping, I can use selenium webdriver along with Java to quite an extent and is familiar with keyword and hybrid framework concepts and what basis they are build. My confusion is how can we use Selenium when the UI is not up and this will be the case in the initial iterations, so does that mean we can use webdriver at a class level and use the same screen when the ui is plugged in. How do team implementing Webdriver go on building up their automation every iteration can you give me some idea briefly, hope not bothering you. thanks in advance

[David]

This is the way I see it for doing web UI automation before the UI is available for TDD, or ATDD.

Figure out the UI navigation flow you need to test against. Assuming you have good user stories, flow chart, mockups, that will really help. If the business unit / designers & software developers haven't collaborated enough to know what the UI will look like, you're gonna have a problem.

Knowing the UI flow, you'll know at a general level, what page to go to, what elements to click, type in, etc. actions to perform. The only thing missing is the actual element locators, page URLs, etc.

So you simply design your page generically to account for the pages to be developed and the elements as well, stubbing in dummy values for elements and pages, etc. And replace them with actual values when they are implemented. This does mean that your code may not run initially because the site UI isn't up. Same approach to TDD unit testing, where tests keep failing until the code is implemented. UI tests will keep failing until UI implemented, with you updating tests as you go along (to fill in locators, page URLs, navigation flow changes, etc.).

But if you have no idea what the flow is, or in general what buttons to click or fields to type into (e.g. no vague mockup), you can't do much and your business users and designers haven't worked hard enough. You at least need mockups and flow charts to start planning the automated UI tests.

The only caveats to initial planning at least in terms of WebDriver is whether the page will have iframes, frames, AJAX, etc. Sometimes, those things are not all thought out until the developer actually starts the work because the build the UI using some common libraries that may generate some of those things. So you won't know if you'll have to switch to an iframe to do some action OR execute some javascript code to handle tricky actions, etc. until the UI is there. Then there's also whether the element locators can be easily defined or how it's defined (XPath, CSS, IDs, etc.) unless you negotiate with developers and designers ahead of time. Because those sometimes are dynamically generated by the 3rd party UI libraries developers use, they may not have too much control with element naming conventions. So you won't know whether locators will be easily defined/found until the UI is there.

[Keki Rabadi]

Hi David thanks for the explanation,
I am concerned about the webdriver part that you have explained, does that mean we write our individual test scripts using webdriver, as per the user stories without id /locators etc. and wait for the UI to be plugged in and then run the same? and slowly start building the regression suite / a framework or whatever as the iteration progresses ?.

I keep on hearing people say continuous automation in agile, but its still not very clear to me as to how they do in a given practical situation iteration by iteration and with what tools.
thanks again help appreciated.

[David]

As Mark & others pointed out earlier, it will vary by organization how you can and choose to do your automation with agile processes. There's no clear standard practice for doing so.

Clearly the most ideal situation is that from or with the user stories, we have all the info including flow diagrams, page URLs, page layouts, and element locators, etc. all defined and just need to be implemented in code by the developers and the corresponding CSS and graphics done by the graphic designers, etc. In such a situation, you have all you need to code up your test, you just can't execute it successfully until the UI is done. And if the UI is coded to spec, and your test is coded to spec, assuming no cross browser quirks, the test would pass when all is done.

But that is very rarely the case, so you do the best you can to try to follow that ideal model. In my earlier comment, what I suggest is that you write up your code substituting the actual ID/locators with a dummy value like:

someBtn = driver.findElement(By.id('dummy1'));

someBtn.click();

then when the locator is available, you ran replace dummy1 with actual value. Granted you'd have to change By.id as well if the locator is not defined by an ID.

As for the rest of your questions, the answer is yes.

You can also start running your completed tests even when the UI isn't done or only partially ready, just expect to see many test failures in that case, same with TDD unit testing. You can use this technique to show management or just for QA to gauge how far development is progressing. Maybe it could also be used to pester developers to hurry up and finish the UI because your tests are still breaking. But in reality the whole team will likely just end up omit running the failed tests (considered as "noise") and finish the UI whenever they can get around to it.

[Mark Collin]

One possible option would be creating page objects.

 

You can then write your test using the page object and retrospectively add the @FindBy annotations to locate the WebElement’s that you are going to use when the code is available.

 

This of course assumes that you are going to know the basic structure of the page in advance and you will probably need to tweak them when the real code is available, but it can give you a basic structure to start from.  In my mind BDD implies that you work very closely with the developers (that will most likely include pairing with them at some point), it’s very hard to try and write tests for fetures that are being developed if you are not part of the development process.

 

As David has said, it varies from team to team and depends on your interpretation of agile.  One person’s BDD is another person’s WAgile…

To view this discussion on the web visit https://groups.google.com/d/msg/selenium-users/-/qcAwQK8h_jEJ.

[Keki Rabadi]

Thanks David, Thanks Mark, with your views and inputs, picture is much less fuzzier then what it was. David, if I understood what you said, we are not making use of any acceptance tools like Jbehave or cucumber but instead we are writing our webdriver scripts in accordance with the story to be executed in parallel with code being written by the developers. But at he same time there is no stopping us from using frameworks like Jbehave/cucumber to wire our steps and stories.
thanks again...

[David Luu]

Keki,

Your understanding is correct. I want to point out that as others probably mentioned already, agile process does not dictate acceptance/TDD/BDD tools. Even TDD/BDD/acceptance testing does not dictate use of JBehave or Cucumber. Some people use them because it helps them implement the automated tests following ATDD/BDD processes or syntax. But you could implement your own framework instead to do same or choose not to use BDD, etc. which can be a lot of work in itself to use, at least initially.

Earlier I was trying to present the simplest example excluding use of BDD and other tools and going with pure Selenium (even w/o page objects). But you can choose to incorporate other tools as needed.

If you do wish to look into BDD, or ATDD, I personally recommend Robot Framework, which also has Selenium support via a test library.

To view this discussion on the web visit https://groups.google.com/d/msg/selenium-users/-/3WmSLTKFK6wJ.

[Keki Rabadi]

David Thanks again n again, and in fact I like your idea of implement just the webdriver without any BDD or page objects because it makes as much sense as any other tools if deployed can. I was just going in circles with these tools and technologies but now I am clear of how to approach as an AT in AE... t   h  a   n   k   s