pit and integration tests for microservices - possible?

[Paulina Bujacz]

Hi,
I've introduced mutation testing (pit framework used) in my team. It works perfectly for unit tests.
We are mostly focused on writing microservices (REST API, JAX-RS) with a little logic inside, but we have a lot of integration tests (verification of headers, queryPrams, backing services cooperation, etc.). Is it possible to use pit framework to get mutation coverage in that case ?
Maybe it is worth to add that integration tests are run by failsafe maven plugin and jetty is started by cargo maven plugin.
How the pit runs mutated bytecode ? (does jetty need to be restarted every mutation is introduced?)

Best regards,
Paulina

[Henry Coles]

On 29 July 2014 10:36, Paulina Bujacz <bujacz....@gmail.com> wrote:

 

I've introduced mutation testing (pit framework used) in my team. It works perfectly for unit tests.
We are mostly focused on writing microservices (REST API, JAX-RS) with a little logic inside, but we have a lot of integration tests (verification of headers, queryPrams, backing services cooperation, etc.). Is it possible to use pit framework to get mutation coverage in that case ?

Pit is designed and optimised for running unit tests. It currently offers no support for integration tests. It inserts mutants into child jvm it controls via a java agent - it cannot insert mutants into another jvm such as one started by cargo.

There are a few issues with integration tests

1) Mutant insertion. The naive approach here would be to require a container restart for each mutant - this would be incredibly slow. It could however be fixed with a custom java agent.

2) Test selection. Automating this would be a little tricky - coverage data detecting within the container is likely to be non-deterministic. You could just run all the tests, but again this would be very slow.

3) Integration tests are by their nature slow.

It would be possible to adapt pit to provide some support, but I'm not convinced that mutation testing with integration tests would be practical on anything beyond toy projects.

The custom agent to allow mutants to be inserted into arbitrator jvm might be worth writing, but I don't think it would be worth integrating into pitest as a whole.

Other forms of fault seeding might suit your needs better. I've never tried it, but I believe byteman should allow you to hand craft faults.

http://byteman.jboss.org/

Henry

[Paulina Bujacz]

Thanks for response.

Cheer,
Paulina