Re: [Cucumber] [Ruby] How to execute a step after all scenarios have executed?

[Andrew Premdas]

On 7 June 2012 22:12, dking <dan.n...@gmail.com> wrote:

I would like to keep a user logged into the browser session for as long as needed for a group of tests.

Feature
    Scenario1
          Given I login as "user1"   <--- my login step checks to see if I'm logged in, if not then login as user1
          When I do something
          Then I see "blah"            <--- no logout occurs

    Scenario2
          Given I login as "user2"   <--- my login step then checks to see if I'm logged in as user2, if not then logout user() and login as "user2"
          When I do something
          Then I see "blah"            <--- no logout occurs

    Scenario3
          Given I login as "user2"   <--- my login step sees I'm logged in as user2, so does nothing...continues to next step
          When I do something
          Then I see "blah"            <--- no logout occurs

    Scenario4
          Given I login as "user1"   <--- my login step then checks to see if I'm logged in as user1, if not then logout user() and login as "user1"
          When I do something
          Then I see "blah"            <--- no logout occurs

Once the final scenario has executed, I must logout the final user before the browser closes, so that my user doesn't get locked out of the system.

I attempted to use the at_exit method, but that occurs after the browser closes.

Is there a way to do one final step (logout the last user), before the browser closes?

First of all you can't assume that the scenarios will be run in any particular order.

Secondly state should not persist between scenarios, so each scenario should be having a new session and starting from being logged out.

If a new session is created for each scenario, then I can't see how a user could be locked out. Are doing something unusual which needs further explanation, or perhaps misunderstanding how isolated each scenario is.

HTH

All best

Andrew

 

-- There are two rules:
 
1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en

--

------------------------

Andrew Premdas

blog.andrew.premdas.org

[dking]

On Jun 7, 7:39 pm, Andrew Premdas <aprem...@gmail.com> wrote:

Understood

> Secondly state should not persist between scenarios, so each scenario
> should be having a new session and starting from being logged out.

I guess I don't understand the absolute importance of your second
rule. I feel that there are situations in which the right thing to do
could be to keep a user logged in (not logging out and logging in each
scenario). Our scenarios have no data dependancies, and in "real
life" users can do more than one thing at a time without logging out
and logging back in.

>
> If a new session is created for each scenario, then I can't see how a user
> could be locked out. Are doing something unusual which needs further
> explanation, or perhaps misunderstanding how isolated each scenario is.

I'm a couple months new to cucumber/ruby/selenium so I don't feel like
there is much out of the usual :). I am attempting to speed up our
tests (approximately 3%), by extracting unnecessary steps (logging out
and back in as the same user I was in the previous scenario) based on
my environments conditions.

The session is currently being locked because the very last scenario
to execute does not log the user out. So, the next time that we run
the next suit of scenarios, the last user of the previous suite run
can not log in. We have thought about trying to disable the session
lock, but I don't want to alter the environoment in order to get
passing tests.

It seems as though there should be something that I can modify in
selenium/webrat/whatever, that will allow me to do something just
before the browser closes after the last scenario. If there are any
suggestions/directions I would love to hear them.

[Andrew Premdas]

The reason for this is that if your tests don't have a consistent starting point then you get failures that are intermittent and caused by where you are starting from rather than what you are testing. This is precisely what is causing you problems now.

>
> If a new session is created for each scenario, then I can't see how a user
> could be locked out. Are doing something unusual which needs further
> explanation, or perhaps misunderstanding how isolated each scenario is.

I'm a couple months new to cucumber/ruby/selenium so I don't feel like
there is much out of the usual :).  I am attempting to speed up our
tests (approximately 3%), by extracting unnecessary steps (logging out
and back in as the same user I was in the previous scenario) based on
my environments conditions.

 You are making a mistake by trying to increase speed by preserving sessions so I'd strongly recommend you stop doing that. The most effective way to get set of scenarios to run faster is to run them in parallel and throw more hardware at them. This can give you vast improvements in speed whilst preserving the integrity of your features.  

The session is currently being locked because the very last scenario
to execute does not log the user out.  So, the next time that we run
the next suit of scenarios, the last user of the previous suite run
can not log in.  We have thought about trying to disable the session
lock, but I don't want to alter the environoment in order to get
passing tests.
 

It seems as though there should be something that I can modify in
selenium/webrat/whatever, that will allow me to do something just
before the browser closes after the last scenario.  If there are any
suggestions/directions I would love to hear them.

There isn't something here because its not needed if you follow testing fundamentals one of which is that state should not persist between tests. Think about what happens when you have a couple of thousand scenarios. With your current approach every single scenario will have to check your starting position, as it can either be logged out, or logged in as a user. This gets even worse if logging in redirects some users to one page and other users to another (e.g. admin's to an admin page).

So please change your approach

All best

Andrew

[Rex Hoffman]

Suggestion.  Use background steps in this scenario.   Use a managed pool (using something like commons-pool) of webdriver instances.

Your background steps could be built something like:

Background:

Given application is accessible 

Given I am logged in as default user

I always use the first given to grab instances of webdriver and ensure it can connect to an application (usually wired through spring, grabbing an instance of firefox from a pool)

I navigate to the home page in that step.

The given I am logged in step would also have another  step associated with it (Given I am not logged in).

The point is you can fully declare your expected state at run time.  Obviously if you don't declare either of the logged in/not logged in then your tests may misbehave.  Which I believe is fine, as your tests basically state that they don't care if you are logged in or not.

But you need to choose to do this or not.  You background "Given application is accessible " could instead always log ensure you are logged out.

Regardless, as long as you are fully explicit about your state and what the steps mean then you should be fine.

Rex

> > cukes+unsubscribe@googlegroups.com. For more options, visit this group at

> >https://groups.google.com/d/forum/cukes?hl=en
>
> --
> ------------------------
> Andrew Premdas
> blog.andrew.premdas.org

-- There are two rules:

1) Please prefix the subject with [Ruby], [JVM] or [JS]. This allows people to filter messages.
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style

You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en

[dking]

Rex,

That is exactly the approach that I have attempted to take.  The only issue that I have ran into is logging out the final user in my test suite (By the way, is there a term in the cuke world for the aggregate of all features? test suite?)

>There isn't something here because its not needed if you follow testing fundamentals one of which is that state should not persist between tests. Think about what happens when you >have a couple of thousand scenarios. With your current approach every single scenario will have to check your starting position, as it can either be logged out, or logged in as a user. >This gets even worse if logging in redirects some users to one page and other users to another (e.g. admin's to an admin page).

>

>So please change your approach

>All best

>Andrew

Andrew,
I do understand the importance of state, and I deliberately write my scenarios to be state and data agnostic.   Thank you for your time/help.  I am very excited to be apart of a great/active cucumber group, and hope to communicate more with you.

On Friday, June 8, 2012 11:51:17 AM UTC-5, Rex Hoffman wrote:

Suggestion.  Use background steps in this scenario.   Use a managed pool (using something like commons-pool) of webdriver instances.

Your background steps could be built something like:

Background:

Given application is accessible 

Given I am logged in as default user

I always use the first given to grab instances of webdriver and ensure it can connect to an application (usually wired through spring, grabbing an instance of firefox from a pool)

I navigate to the home page in that step.

The given I am logged in step would also have another  step associated with it (Given I am not logged in).

The point is you can fully declare your expected state at run time.  Obviously if you don't declare either of the logged in/not logged in then your tests may misbehave.  Which I believe is fine, as your tests basically state that they don't care if you are logged in or not.

But you need to choose to do this or not.  You background "Given application is accessible " could instead always log ensure you are logged out.

Regardless, as long as you are fully explicit about your state and what the steps mean then you should be fine.

Rex

On Friday, June 8, 2012 9:11:42 AM UTC-7, apremdas wrote:

On 8 June 2012 16:30, dking wrote:

On Jun 7, 7:39 pm, Andrew Premdas wrote:

[Rex Hoffman]

On Friday, June 8, 2012 10:41:02 AM UTC-7, dking wrote:

Rex,

That is exactly the approach that I have attempted to take.  The only issue that I have ran into is logging out the final user in my test suite (By the way, is there a term in the cuke world for the aggregate of all features? test suite?)

 

The point is that you can't trust the browser your using to be in a pool that only belongs to this set of features (unless you choose to create a pool like this).  You will need conditional logic in your "Given I am not logged in" to ensure you are logged out, and it will need to be in all your feature files background that you don't want to share the same user.  Think non-deterministic run of all the features/scenarios in them.  Some scenarios want to be logged in, some want to ensure they start logged out.  Your step just needs to ensure they end up in the state they expect to be in, logging in or out as appropriate....  don't do this in the after or before hooks, ensure the state in the background.

 

As for what to call the name... cucumber invocation would be the most honest name currently, as a objectfactory tends to be used across the entire invocation.

[Andrew Premdas]

On 9 June 2012 00:45, Rex Hoffman <r...@e-hoffman.org> wrote:

On Friday, June 8, 2012 10:41:02 AM UTC-7, dking wrote:

Rex,

That is exactly the approach that I have attempted to take.  The only issue that I have ran into is logging out the final user in my test suite (By the way, is there a term in the cuke world for the aggregate of all features? test suite?)

 

The point is that you can't trust the browser your using to be in a pool that only belongs to this set of features (unless you choose to create a pool like this).  You will need conditional logic in your "Given I am not logged in" to ensure you are logged out, and it will need to be in all your feature files background that you don't want to share the same user.  Think non-deterministic run of all the features/scenarios in them.  Some scenarios want to be logged in, some want to ensure they start logged out.  Your step just needs to ensure they end up in the state they expect to be in, logging in or out as appropriate....  don't do this in the after or before hooks, ensure the state in the background.

 

As for what to call the name... cucumber invocation would be the most honest name currently, as a objectfactory tends to be used across the entire invocation.

Putting a step in background is exactly the same as putting a step at the beginning of a scenario. Backgrounding is about avoiding duplication in features, not about avoiding duplication in execution which is what the OP was trying to do. Instead of using a pool of browsers your initial step can just log out before logging in. If the OP did this then there wouldn't be any problem. You could implement the step def as something like

   logout if logged_in?

   login_as(user)

Any savings in execution gained by using a pool of browsers or any optimisation involving persisting sessions across scenarios would be minuscule in comparison to running scenarios in parallel.

All best

Andrew

  

-- Rules:
 
Rules:

 
1) Please prefix the subject with [Ruby], [JVM] or [JS].

2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style

3) If you have a question, don't reply to an existing message. Start a new topic instead.
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en

[dking]

Andrew, I agree with you 100% that running tests in parallel would be optimal, unfortunately that is not an option in my environment.

I have taken it upon myself to shave off all the unnecessary time spent in our testing environment, including the unneeded login and logout (not sure how removing this step could possibly leak state? ...unless your test required you to know when you logged in/out?  Do users typically logout and login between each task they perform in a system...?  Sorry, I'm a little frustrated with some of these responses)  Any time saved not having to watch the disk spin is a win in my book and every developer that I work with.  Multiply that across a program = incredible $ and frustration savings.

>There isn't something here because its not needed if you follow testing fundamentals one of which is that state should not persist between tests. Think about what happens when you >have a couple of thousand scenarios. With your current approach every single scenario will have to check your starting position, as it can either be logged out, or logged in as a user. >This gets even worse if logging in redirects some users to one page and other users to another (e.g. admin's to an admin page).

>

>So please change your approach

>All best

>Andrew

Andrew,
I do understand the importance of state, and I deliberately write my scenarios to be state and data agnostic.   Thank you for your time/help.  I am very excited to be apart of a great/active cucumber group, and hope to communicate more with you.

 

My question is... did you provide me this response to "protect" me from my own ignorance, or can this really not be done? 

My original question was "Is there a way to do one final step (logout the last user), before the browser closes?"

Just wondering anyone has thought about this anymore?

Thanks

[Oscar Rieken]

why not put an after hook that clears out the cookies or something? 

or at_exit

-- Rules --

[Andrew Premdas]

On 3 July 2012 21:30, dking <dan.n...@gmail.com> wrote:

Andrew, I agree with you 100% that running tests in parallel would be optimal, unfortunately that is not an option in my environment.

Why? What is special about your environment.

The problem with shaving of time in running acceptance tests by the sort of optimisations you are considering is that doing so often makes the tests less acceptable, and in particular more prone to subtle bugs, which take ages to fix. This can undermine the whole testing strategy for relatively small gains in execution speed.

Its not like your the first person to go down this road, and you won't be the last, but in my experience what you are doing is counter-productive. 

Whilst unit test suites can be optimized to be lightning fast, acceptance tests have to go through the full stack and the UI and are inherently slower. As your projects size increases you just won't be able to run a whole suite quickly enough no matter how many optimisations you make. For example on a ecommerce product our features took 20 minutes to run. Using your approach might have reduced that to 15 minutes. Going parallel on a very small scale reduced it to < 3 minutes. The bigger the suite gets the larger the problem. Going parallel is the only solution that scales. Joseph Wilk use to demonstrate this by running a 4 hour suite of features on his laptop in less than a minute using 100 amazon ec2 instances.

Anyhow I've said my bit, and you are free to do whatever you want :) Good luck

Andrew

ps please don't top post 

-- Rules --

 
1) Please prefix the subject with [Ruby], [JVM] or [JS].
2) Please use interleaved answers http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
3) If you have a question, don't reply to an existing message. Start a new topic instead.
 
You received this message because you are subscribed to the Google Groups Cukes group. To post to this group, send email to cu...@googlegroups.com. To unsubscribe from this group, send email to cukes+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/cukes?hl=en