A use case for QuarkusComponentTest?

[Martin Kouba]

Hi everyone,

recently I came across an issue [1] with a long discussion about
@QuarkusTest being a little bit slow in some cases. Especially when you
run a single test from the IDE and your app has a lot of extensions and
possibly some dev services.

So here comes a working POC called "QuarkusComponentTest".

What is it for?
- The goal is to test business logic in one or more components/CDI beans
without the need to wire and mock all dependencies manually.

How is it implemented?
- It's a regular JUnit5 extension; a static field of a test class
annotated with @RegisterExtension.

What does it do?
- Starts the CDI container (ArC)
- Registers a dedicated SmallRyeConfig
- Unsatisfied dependencies are mocked automatically; you can inject them
in the test and configure the mocks right away
- There is also a builder-like API to provide custom mock beans
- The CDI request context is activated and terminated per test method

What's missing?
- Advanced SmallRye Config integration
- QuarkusComponentTests are currently skipped during continuous testing;
we would need some class loading magic to make it work
- Further integration with extensions, e.g. an extension could register
a customized mock automatically?

Examples? Code?
- There is a quickstarts branch with examples:
https://github.com/mkouba/quarkus-quickstarts/blob/component-test/config-quickstart/src/test/java/org/acme/config/MyComponentV1Test.java
- The POC branch is available here:
https://github.com/mkouba/quarkus/tree/component-test

The question is whether this prototype is worth further investigation
and possibly integrating into the Quarkus core testing tools?

--
Martin Kouba
Principal Software Engineer
Red Hat, Czech Republic

[1]
https://github.com/quarkusio/quarkus/issues/14927

[Ladislav Thon]

Hi,

great work there! I've always thought (and mentioned multiple times on Zulip and likely elsewhere) that Quarkus needs a proper way of writing actual unit tests [1], and this looks like exactly that. I'd probably put it directly into `quarkus-junit5` instead of a separate artifact, but that's a trifle. The integration into continuous testing would be very interesting, but that can wait IMHO (and not sure if that could be made fast enough in case of hundreds of unit tests -- that's a great upside to the current testing infrastructure, if profiles are used sparingly).

LT

[1] because "being executed by Surefire" doesn't make it a unit test -- `@QuarkusTest` is more of an integration test with extensive white-box capabilities

út 23. 5. 2023 v 15:04 odesílatel Martin Kouba <mko...@redhat.com> napsal:

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/b19e52ec-fefe-f415-1530-0aa80cb21837%40redhat.com.

[Martin Kouba]

On 23. 05. 23 15:52, Ladislav Thon wrote:
> Hi,
>
> great work there! I've always thought (and mentioned multiple times on
> Zulip and likely elsewhere) that Quarkus needs a proper way of writing
> actual unit tests [1], and this looks like exactly that.

> I'd probably
> put it directly into `quarkus-junit5` instead of a separate artifact,
> but that's a trifle.

The thing is that quarkus-junit5 does not depend on ArC and Mockito. It
might make more sense to put it into quarkus-junit5-mockito instead.
We'll see if we find a home for this little one ;-)

> The integration into continuous testing would be
> very interesting, but that can wait IMHO (and not sure if that could be
> made fast enough in case of hundreds of unit tests -- that's a great
> upside to the current testing infrastructure, if profiles are used
> sparingly).
>
> LT
>
> [1] because "being executed by Surefire" doesn't make it a unit test --
> `@QuarkusTest` is more of an integration test with extensive white-box
> capabilities
>
> út 23. 5. 2023 v 15:04 odesílatel Martin Kouba <mko...@redhat.com

> <mailto:mko...@redhat.com>> napsal:

> https://github.com/mkouba/quarkus-quickstarts/blob/component-test/config-quickstart/src/test/java/org/acme/config/MyComponentV1Test.java <https://github.com/mkouba/quarkus-quickstarts/blob/component-test/config-quickstart/src/test/java/org/acme/config/MyComponentV1Test.java>

> - The POC branch is available here:
> https://github.com/mkouba/quarkus/tree/component-test
> <https://github.com/mkouba/quarkus/tree/component-test>
>
> The question is whether this prototype is worth further investigation
> and possibly integrating into the Quarkus core testing tools?
>
> --
> Martin Kouba
> Principal Software Engineer
> Red Hat, Czech Republic
>
> [1]
> https://github.com/quarkusio/quarkus/issues/14927
> <https://github.com/quarkusio/quarkus/issues/14927>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to quarkus-dev...@googlegroups.com

> <mailto:quarkus-dev%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/b19e52ec-fefe-f415-1530-0aa80cb21837%40redhat.com <https://groups.google.com/d/msgid/quarkus-dev/b19e52ec-fefe-f415-1530-0aa80cb21837%40redhat.com>.

[Stuart Douglas]

This sounds like an approach that will be more familiar to Spring users.

How does it compare to @QuarkusTest performance wise? I assume a single one will be faster, but a large amount might be considerably slower?

Stuart

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/b19e52ec-fefe-f415-1530-0aa80cb21837%40redhat.com.

[Georgios Andrianakis]

Very interesting indeed!

I have the same question as Stuart about performance over multiple tests.

Furthermore, as you already mentioned, it probably does need to be based on  quarkus-junit5-mockito (or some new common module)

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CAD%2BL2cyjxyUA%3DLd4wQT8uq3GdWeeq%2B7r6mqxvV26%2BYiUOr4Hew%40mail.gmail.com.

[Melloware]

I love this idea!   +1

[Martin Kouba]

On 24. 05. 23 1:18, Stuart Douglas wrote:
> This sounds like an approach that will be more familiar to Spring users.
>
> How does it compare to @QuarkusTest performance wise? I assume a single
> one will be faster, but a large amount might be considerably slower?

The first run/single test usually takes ~ 1 second, the "Time elapsed"
reported by Surefire on my machine. A great deal of the time is consumed
by Mockito initialization. For multiple/subsequent tests it's usually
100-300 ms...

> https://github.com/mkouba/quarkus-quickstarts/blob/component-test/config-quickstart/src/test/java/org/acme/config/MyComponentV1Test.java <https://github.com/mkouba/quarkus-quickstarts/blob/component-test/config-quickstart/src/test/java/org/acme/config/MyComponentV1Test.java>

> - The POC branch is available here:
> https://github.com/mkouba/quarkus/tree/component-test
> <https://github.com/mkouba/quarkus/tree/component-test>
>
> The question is whether this prototype is worth further investigation
> and possibly integrating into the Quarkus core testing tools?
>
> --
> Martin Kouba
> Principal Software Engineer
> Red Hat, Czech Republic
>
> [1]
> https://github.com/quarkusio/quarkus/issues/14927
> <https://github.com/quarkusio/quarkus/issues/14927>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Quarkus Development mailing list" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to quarkus-dev...@googlegroups.com

> <mailto:quarkus-dev%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/quarkus-dev/b19e52ec-fefe-f415-1530-0aa80cb21837%40redhat.com <https://groups.google.com/d/msgid/quarkus-dev/b19e52ec-fefe-f415-1530-0aa80cb21837%40redhat.com>.

[Martin Kouba]

On 24. 05. 23 15:57, Martin Kouba wrote:
> On 24. 05. 23 1:18, Stuart Douglas wrote:
>> This sounds like an approach that will be more familiar to Spring users.
>>
>> How does it compare to @QuarkusTest performance wise? I assume a
>> single one will be faster, but a large amount might be considerably
>> slower?
>
> The first run/single test usually takes ~ 1 second, the "Time elapsed"
> reported by Surefire on my machine. A great deal of the time is consumed
> by Mockito initialization. For multiple/subsequent tests it's usually
> 100-300 ms...

After a small SmallRye Config optimization [1] we're down to 50-120 ms
for the subsequent tests which is probably acceptable. Of course, it
depends on the number of the tested components and complexity of the
test methods.

[1]
https://github.com/mkouba/quarkus/commit/047b414319320179102b3ea5b4f677f0d7f68c85

[Martin Kouba]

I've sent a DRAFT pull request:
https://github.com/quarkusio/quarkus/pull/33607

Feel free to comment...