Major issue - Parallel execution of tests in Spock along with Geb

[Dan]

Hello everyone, 

As the summary describes there is a big issue while trying to execute parallel execution along with Spock and Geb. 

As you know it Spock team introduced recently this feature: https://spockframework.org/spock/docs/2.0-M4/parallel_execution.html with nice execution modes. The problem is that when you want to use that feature along with GEB it will not work properly. This is my setup: 
Spock Config file contains: runner { parallel { enabled true } }
Geb Config file contains:

cacheDriverPerThread = true 
System.setProperty("webdriver.chrome.driver", "path to chromedriver.exe")
driver = {new ChromeDriver()}

Test class is a child of a custom test manager class that extends GebReportingSpec
Note: In the custom test manager class I used the go baseUrl to open a specific page.
CustomPage (e.g LoginPage) class is a child of a class manager that extends Page from Geb 

Now everything works as expected while I try to execute a test or multiple tests from a class in the traditional way (with the parallel { enabled false }) however when I enable the parallel option from the config file the spock tests starts in parallel BUT the GEB does not initiate the new browser instances, all the tests are executed simultaneously on the same instance one and only leaving the new instances empty (having in the url: data;) or with the base url  mentioned in the GebConfig! 
If you have cacheDriverPerThread set on false you will have the same behavior but without opening new browser instances.

The big question is: Was anyone able to execute parallel tests using Spock and GEB without having this issue? 

I tried on several devices (different configurations) with several parameters and configs, in the end I had the same behavior, multiple browser instances were opened but only one of them received the tests and the execution of tests was at the same time for all of them. (one test tring to press on the Login button, another trying to write something, another trying to delete something) - all this actions at the same time. 

I've verified all the subjects related to this from the stackoverflow and the most important containing similar subject was this: https://stackoverflow.com/questions/41897862/running-spock-tests-in-parallel

I've even wrote an email to Leonard who does some work for Spock but he suggested me to write the issue on stackoverflow and maybe erdi from GEB will have a look. 

Well I don't want to give up using this combination and I really want to have my tests running in parallel not via gradle but via spock along with GEB :) 

Note: If I use gradle task to execute them in parallel it will work BUT the execution is performed per class not per feature, so you will have class tests running in parallel (tests from a class will run one after another) not tests from a class running in parallel. And you must have the feature of parallel set on false in the SpockConfig file otherwise you will run test class in parallel and also tests from a class in parallel but with the issue that I've already mentioned.

The desired behavior is to be able to execute class tests in parallel AND tests from a class in parallel but the initialization should be performed for all new browser instances. 

Please have a look and please let me know if this setup works for you guys :)

Thanks :) 

[Marcin Erdmann]

Hi Daniel,

I'm not surprised that running features in parallel from a single spec extending GebSpecification does not work. Browser class is not thread safe and Browser instance management in GebSpec/GebTestManager is not designed to work when used from multiple threads. With Spock's defaultExecutionMode set to CONCURRENT you are essentially using a single instance of Browser from multiple threads. This does not work by design and I do not have plans to change it - the amount of problems that you would have to solve far outweighs the potential benefits from supporting what you are asking for. I suggest that you stick to defaultSpecificationExecutionMode = CONCURRENT and defaultExecutionMode = SAME_THREAD in Spock's config which together with cacheDriverPerThread = true in Geb's config should work without problems.

Cheers,
Marcin

--
You received this message because you are subscribed to the Google Groups "Geb User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/b807d7fa-3b64-45b7-bf3a-1830d2162752n%40googlegroups.com.

[Dan]

Hi  Erdmann,

Thanks for your reply, and thanks for your clarification, however having this limitation is not helpful for GEB and you know why, is not the same thing to have a suite of tests that are running in 1h versus the same suite of tests running in 10 minutes. 
This is a limitation only when using Spock and GEB. I've tried a setup using JUnit and GEB and all the instances are successfully instantiated and everything is working as expected. Since Spock is based on JUnit I don't know why a workaround would not be possible or a logic for thread safe execution is not in plan. As I mentioned the potential is high because of the time execution. Anyway I'll not be only me that will be forced to decide if I continue to use GEB due to this limitation or not. I liked GEB because of its features but in this context most probably I'll need to drop it off unfortunately. 

Thanks, 

Daniel

[Marcin Erdmann]

Daniel,

I was convinced that running Geb specs in parallel at spec level was supported. I was about to insist that you parallelise your execution at spec level and not feature level because getting the latter working was more trouble than it's worth and parallelisation at spec level should already provide a significant performance improvement as long a you don't have massive specs with many feature methods in them. While I was looking at what it would take to get parallelisation at feature level working I've realised that parallelisation at spec level cannot work with the current implementation due to some assumptions I made and I believe that parallelisation should at minimum work at spec level so that will need fixing. While I'm working on that I might even look at changing GebTestManager to have thread safe implementations of all methods apart from beforeTestClass() and afterTestClass() which should never be called from multiple threads even when using Spock's parallel execution and also return a different Browser instance per thread getBrowser() is called from which would enable parallelism at feature level.

Thanks for bringing this to my attention and raising the issue, I will reopen it.

Cheers,

Marcin

To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/84cf4b16-bea9-426e-bdc0-143efae706f6n%40googlegroups.com.

[Dan]

Hi  Erdmann,

This is awesome, now I really reconsider using the initial setup and I'll wait patiently for your new implementation to be released. Is great to see that you don't treat things superficially. :) 

Best regards, 

Daniel