Is there something like 'Categories' within Geb?

[Lee]

I'm new to Geb and would like to be able utilize something like Categories in JUnit to annotate tests, and then be able to execute only tests with certain categories/annotations.

I've tried annotating some tests and then executing a Gradle task, as noted at https://github.com/junit-team/junit4/wiki/Categories, but have not been successful.

Is there another way to do this in Geb?  Or, does anyone have any suggestions on how I could use JUnit Categories.

Example Test I tried:

package GebishOrgTest

import geb.junit4.GebReportingTest
import org.junit.Test
import org.junit.runner.RunWith
import org.junit.experimental.categories.Category
import org.junit.runners.JUnit4

interface FastTest { /* category marker */ }

@RunWith(JUnit4)
class GebishOrgTest extends GebReportingTest {

    @Category (FastTest)
    @Test
    void canGetToTheCurrentBookOfGeb() {
        to GebishOrgHomePage

        manualsMenu.open()

        //first link is for the current manual
        assert manualsMenu.links[0].text().startsWith("current")

        manualsMenu.links[0].click()

        at TheBookOfGebPage
    }

}

Example Gradle Task:

task useCategory(type: Test) {

useJUnit {

        includeCategories 'GebishOrgTest.FastTest'

    }

}

[Stefan Hildebrandt]

Hi,

it's not a geb question. You are use plain junit in your example. It's a gradle configuration.

Along to  https://docs.gradle.org/3.3/userguide/java_plugin.html#test_grouping or your Example (https://github.com/junit-team/junit4/wiki/Categories) you are missing the

excludeCategories

to disable some tests.

Best Regards

Stefan

---

--
You received this message because you are subscribed to the Google Groups "Geb User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+u...@googlegroups.com.
To post to this group, send email to geb-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/beef3ac1-e1ea-4a7b-a104-0c0dc19e2f54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lee]

Thanks Stefan.

We may have lots of Categories though.  It would be preferable to just 'include' what I'd like to run and then just ignore everything else without having to implicitly define the 'exclude' line (especially if we have tests that aren't annotated yet).

Alternatively, I was hoping that there may just be something built into Geb to handle this.  I've previously used C#, Selenium and MSBuild where we could pass in 'categories' as parameters via the command line and only those tests would be executed.  Everything else was ignored.  I got spoiled to that and was hoping there was someway to do something similar with Geb and Gradle.  ;-)

[Marcin Erdmann]

Lee,

Stefan is definitely right that this question is not really a Geb question as Geb is test and build framework agnostic (although I'm sure a lot people will agree that using it with Spock is the idiomatic way) and running particular tests is a concern for the test and build framework in my opinion.

On the other hand, I don't think Stefan is right when saying that you need both includes and excludes for categories on a gradle test task as I use JUnit categories extensively with Spock  and Gradle myself and only ever specify either includes or excludes.

Your setup looks right to me, but can you please try to remove @RunWith(JUnit4) annotation from the class (by default it is not necessary to run JUnit4 tests with Gradle) and move your category annotation onto the class (I know that it should also work on methods but I know for sure that categories work when they are located on classes so I'm trying to remove as many variables as possible).

I don't know if it has anything to do with you seeing unexpected results, but can you please change the package name to be lowercase? By convention, package names start with a lower case in jvm languages.

Finally, what exactly do you mean when you say that you have not been successful? Do you see no tests executed, all tests executed or some error when running your custom useCategory test task?

Cheers,

Marcin

To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+unsubscribe@googlegroups.com.

To post to this group, send email to geb-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/30f2643b-189d-41c1-b2f3-4cb1f0bb6cb7%40googlegroups.com.

[Lee]

Marcin,

Thanks for your response.  I think I'm getting it now.

My end game here was to use Categories in JUnit with the hope that it would be similar to using 'tags' in Cucumber.  But, once you referred to Spock, I think it clicked what you and Stefan meant that this isn't a Geb question.  It seems that my understanding was incorrect that it's a testing framework versus a tool to work with webdriver.    

The code examples I posted before were based on the Geb GitHub example https://github.com/geb/geb-example-gradle with modifications to use Categories.

I've moved on to using code from my company and making some of the modifications you suggested.  Expected tests are executed when I have a Gradle task that points to tests with valid categories I've defined.  THANK YOU.

But, if I run a Gradle task with a Category that is not yet defined in a test or an interface (for example someone misspelled a Category in the Gradle task), it executes all tests and fails them all versus just skipping them as they don't have that Category  (see examples below).

Do you have any suggestions for this?

Example:

I've got two categories defined via Interfaces:  Login and UploadFile

• I have a test with a Category of 'Login' and I have a Gradle task that runs it.
  
• It executes just that one test as expected.
• I have NO tests with the Category of 'UploadFile' , but I have it defined in an interface and have a Gradle task that runs it.
  
• It executes no tests as expected.
• I have a Gradle task that tries to execute tests with a Category of 'Database' (which hasn't been defined in any interface yet), and I have NO tests with that Category.
  
• It executes ALL tests in my package and fails them all with a message like:

Can't load category class [com.package.testclass.Database].

org.gradle.api.InvalidUserDataException: Can't load category class [com.package.testclass.Database].

[Marcin Erdmann]

Hi Lee,

Sorry for a delayed response.

To be honest all three bullet points that you listed in your email describe the exact behaviour I'd expect. Specifically the failure when you specify a Gradle test task that uses a non-existing category class in its definition which IMHO is exactly what I'd want as it's most probably an error on my my side like for example a typo, which you mentioned yourself. I think it's better for the tool to fail in such situation because I don't see a reason why I would use a non-existing category in a test task definition on purpose. What is your rationale for expecting different behaviour in such situation?

Cheers,

Marcin

To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+unsubscribe@googlegroups.com.

To post to this group, send email to geb-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/8d109779-af24-44f7-bf84-9151b75f453c%40googlegroups.com.

[Lee]

Hi Marcin,

I totally agree that it should fail if a non-existing category is used and it should output an error message.  I was wondering if is should execute and fail ALL tests though.  I'd think just failing the test run with one error message like "No tests with a Category of '<catagory name used here>' were found. Test run Failed." would suffice.    But, thanks for letting me know that is the expected behavior.

I have another related question if you don't mind.

In order use JUnit Categories with Gradle, one would have to put the then entire package reference with Category in the 'includeCategories' statement.

Example:

test {
  useJUnit {
    includeCategories 'org.gradle.junit.CategoryA'
  }
}

Is there a way enter a reference to a parent package with a Category (possibly using a wildcard) so that it will look for all children with that Category under the parent and execute all relevant tests?

Example:

test {
  useJUnit {
    includeCategories 'org.gradle.*.CategoryA'
  }
}




I can see where we may have tests with the same Category spread across packages, and just want to execute all those tests regardless of where they reside.




I appreciate all of your help,




Lee

[Marcin Erdmann]

Lee,

Your last question gets us deeper and deeper into Gradle's JUnit Category support. I hope that you won't mind me saying that but I believe that Gradle Forum at https://discuss.gradle.org/ is a better channel for this kind of question.

Good luck,

Marcin

To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+unsubscribe@googlegroups.com.

To post to this group, send email to geb-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/33f4029e-dc28-4dc5-ae25-36cceb27e09e%40googlegroups.com.

[Lee Leibke]

Marcin,

Sounds good to me.

Thanks again for your help,

Lee

--
You received this message because you are subscribed to a topic in the Google Groups "Geb User Mailing List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/geb-user/dRqWFUlD5O0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to geb-user+unsubscribe@googlegroups.com.

To post to this group, send email to geb-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/CA%2B52dQQu7UsMLPH%3D%3D1d5BupojZbZugJ7Co4J0nTOj%2B1OCGhfWA%40mail.gmail.com.