Standard baseline/regression test projects?

[Julian Gamble]

Hi Everyone, 

I'm a user of checkstyle with sonar with my team. I made a contribution to the codebase back when it was on sourceforge. 

Thanks for all the work you've done on checkstyle - it's great to see such an active codebase. 

Suppose someone were looking at making some major changes to the codebase- and wanted some big hairy codebases to run before and after the changes to the checkstyle codebase. 

Is there a standard set of 'baseline test projects'? Something that is likely to test out a majority of the checkstyle checks?

(The JDK codebase, the eclipse codebase, etc). 

Cheers

Julian

[Roman Ivanov]

Hi Julian,

If you pay attention to our CIs (yml files in root of repo) you will see that we

already did some steps to prevent regressions.

There are two types of completely automated testing:

1) tests for Exceptions

2) tests for actual change in violations.

> 1) test for Exceptions

look at  https://github.com/checkstyle/contribution/tree/master/checkstyle-tester

there are property files that describe list of project that do testing.

One example: https://github.com/checkstyle/contribution/blob/master/checkstyle-tester/projects-to-test-on.properties

> 2) tests for actual change in violations

we use certain project that already enforced checkstyle on their codebase , and do each PR validation on them. Look at "NoErrorTest -XXX" 

https://github.com/checkstyle/checkstyle/blob/master/wercker.yml#L31

For now it is only 3 project.

=================

if addition to automated testing we ask each contributor to provide as diff-report of violation of Check he update (partly automated process). That is not automated completely .... but one day we will make it.

========================

There are still a lot rooms for improvements in this area, but this is what we have now, if smb want to help us in automatons  - be are welcome (I will show places to improve).

thanks,

Roman Ivanov

--
You received this message because you are subscribed to the Google Groups "checkstyle" group.
To unsubscribe from this group and stop receiving emails from it, send an email to checkstyle+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julian Gamble]

This is brilliant. Thankyou.

https://github.com/checkstyle/contribution/blob/master/checkstyle-tester/projects-to-test-on.properties

JG

To unsubscribe from this group and stop receiving emails from it, send an email to checkstyle+...@googlegroups.com.