Why does FindState() force FindAllCss() with a predicate to not wait

[Kevin Berridge]

My team and I have been banging our heads against a sporadically failing Coypu test and we finally figured out what the cause was today, but I don't understand why Coypu behaves this way.  Could someone help clarify this behavior, and maybe point me to a different way to implement this?

The scenario is a classic search, but it's complicated by a Full Text indexing delay:

1. We insert some test data.
2. We perform a search.  
3. We do a FindAllCss with a predicate to expect the number of rows to be returned.  It's quite likely that the Full Text catalog will not be up to date yet, so this is likely to not return the full results.
4. If the FindAllCss fails, we want to re-try from Step #2

This was implemented in Coypu with a State that contains a FindAllCss with a predicate as follows:

var searchState = new State(() =>
{
  try
  {
    browser.FillIn("searchText").With(query);
    browser.FindAllCss("#results tr", rows => rows.Count() == expectedNumRows);
    return true;
  }
  catch { return false; }
});


browser.FindState(searchState);

We expected the FindAllCss call to retry like it always does, waiting for the search to complete.  And if the full text index hadn't been updated yet, then we expected the entire state to retry.

However, it was discovered that the FindAllCss call does not retry when called within a State!  Some spelunking in Coypu's source reveals that StateFinder sets:

timingStrategy.ZeroTimeout = true;

Thus when timingStrategy.Synchronise is executed with the FindAllCssWithPredicateQuery the timeout is overriden to zero, causing FindAllCss to operate as if no predicate had been specified at all.

In practice, this means it doesn't give the search time to complete, so FindState fails more often than not.

Once we understood this, the work around was simply to not use FindState and implement the retry manually.

But this was a huge surprise to us!  We never expected FindAllCss would behave differently inside a state than outside a state.  But this is clearly by design.

So my questions are:

1. Can anyone explain why FindState behaves this way?
2. Does anyone have a suggestion of how else we might implement a test like this (without writing the retry manually)?

Thanks in advance!

[Adrian Longley]

On Wed, Feb 11, 2015 at 12:23 AM, Kevin Berridge <kevin.w....@gmail.com> wrote:

My team and I have been banging our heads against a sporadically failing Coypu test and we finally figured out what the cause was today, but I don't understand why Coypu behaves this way.  Could someone help clarify this behavior, and maybe point me to a different way to implement this?

The scenario is a classic search, but it's complicated by a Full Text indexing delay:

1. We insert some test data.
2. We perform a search.  
3. We do a FindAllCss with a predicate to expect the number of rows to be returned.  It's quite likely that the Full Text catalog will not be up to date yet, so this is likely to not return the full results.
4. If the FindAllCss fails, we want to re-try from Step #2

This was implemented in Coypu with a State that contains a FindAllCss with a predicate as follows:

var searchState = new State(() =>
{
  try
  {
    browser.FillIn("searchText").With(query);
    browser.FindAllCss("#results tr", rows => rows.Count() == expectedNumRows);
    return true;
  }
  catch { return false; }
});


browser.FindState(searchState);

We expected the FindAllCss call to retry like it always does, waiting for the search to complete.  And if the full text index hadn't been updated yet, then we expected the entire state to retry. 

The purpose of FindState is to find the first of a list of possible states in the page, not something you'd use very often, but when you have some nondeterministic situation where one of various states may exist it can be useful. See https://github.com/featurist/coypu#finding-states-nondeterministic-testing

It uses a zero timeout inside each state query so that they can all be re-evaluated continuously until the first one becomes true. That is the only purpose of FindState.

If you only supply one state then you are not going to get any particularly useful behaviour from it whatever you do. It's not designed for retrying actions like you are, but for comparing the state of the page.

I think I'll validate that at least 2 states are supplied in future.

If you simply want to put together an action followed by a test and retry the lot till they succeed you want BrowserSession.RetryUntilTimeout. You'll also want to set two different timeout settings. One for the inner FindAllCss query, e.g. 1 second to get results back, and a longer one for the outer retry, e.g. 10 secs for your index to populate. It would look something like this (untested):

var waitForEachSearch = new Options{Timeout = TimeSpan.FromSeconds(1)};

var waitForIndexToPopulate = new Options{Timeout = TimeSpan.FromSeconds(10)};

browser.RetryUntilTimeout(() =>

  {

    browser.FillIn("searchText").With(query);

    browser.FindAllCss("#results tr", rows => rows.Count() == expectedNumRows, waitForEachSearch);

  },

  waitForIndexToPopulate

);

I notice RetryUntilTimeout is not described on the readme so I'll add an example like the above too.

Hope that all helps

However, it was discovered that the FindAllCss call does not retry when called within a State!  Some spelunking in Coypu's source reveals that StateFinder sets:

timingStrategy.ZeroTimeout = true;

Thus when timingStrategy.Synchronise is executed with the FindAllCssWithPredicateQuery the timeout is overriden to zero, causing FindAllCss to operate as if no predicate had been specified at all.

In practice, this means it doesn't give the search time to complete, so FindState fails more often than not.

Once we understood this, the work around was simply to not use FindState and implement the retry manually.

But this was a huge surprise to us!  We never expected FindAllCss would behave differently inside a state than outside a state.  But this is clearly by design.

So my questions are:

1. Can anyone explain why FindState behaves this way?
2. Does anyone have a suggestion of how else we might implement a test like this (without writing the retry manually)?

Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "Coypu" group.
To unsubscribe from this group and stop receiving emails from it, send an email to coypu+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.