Unit Test Results and Per-Test Coverage
Per-Test Coverage
Clover has the ability to integrate Unit Test results and per-test coverage information into Coverage reports. This unique feature gives you a powerful insight into how well tested your covered code actually is, by showing you exactly which statements each of your tests cover.To take advantage of this feature all you need to do is:
- Allow Clover to instrument your test classes.
You must ensure that all your test classes are included for Clover instrumentation. To enable Clover instrumentation of your source code, use the<clover-setup>task.
This allows Clover to specially mark test methods to enable per-test coverage reporting. The Clover instrumenter automatically detects test methods for Junit 3.x, Junit 4.x and TestNG. If you are using another testing framework, see "Advanced Test Case Detection" below.
Advanced Test Case Detection
If you are using a testing framework which does not use Junit or TestNG conventions for determining test methods, the nested <testsources/> element of <clover-setup> and <clover-instr> will allow you to specify the convention used by your framework.
The Instinct framework for example uses the following:
<clover-setup>
<testsources dir="tests">
<testclass>
<testmethod annotation="Specification"/>
<testmethod name="^should.*"/>
<testmethod name="^must.*"/>
</testclass>
</testsources>
</clover-setup>
You can optionally group <testclass/> definitions in <or/> or <and/> elements. Each <testmethod/> element is automatically grouped by an OR.
<clover-setup>
<testsources dir="tests">
<or>
<testclass name=".*Context">
<testmethod annotation="Specification"/>
</testclass>
<testclass name=".*Test">
<testmethod name="test.*"/>
</testclass>
</or>
</testsources>
</clover-setup>
Differences between Groovy and Java with respect to <clover-setup> tasks.
- In Java, the names of annotations and classes that you use to define your test code could be either fully-qualified or not fully-qualified, and depend on how you declared these annotations and classes in your actual Java source code.
- In Groovy, fully qualified annotation and class names are required in your test code, regardless of how you have declared these annotations and classes in your actual Groovy source code.
Advanced Test Result Configuration
Clover does its best to record your test results in the Clover database. In some instances however, Clover can not always do so. Although unit tests using @Test(expected=Exception.class) annotations will be marked as passed, more novel JUnit constructs such as rules may not be recognized and Clover will flag those tests as failed when in fact they passed. To integrate these test results into your Clover reports follow these steps:
Enable XML reports from your Unit Test execution.
In an Ant build, if using the<junit>task, you need to add an XML result formatter:<property name="testreport.dir" value="build/test-reports"> <junit ...> ... <formatter type="xml"/> <batchtest todir="${testreport.dir}"> ... </batchtest> </junit>Such results can be similarly produced by the Ant TestNG tasks.
- Supply the
<testresults>element at report time.
If the<testresults>element is specified,<clover-report>and similar tasks will use these results instead of those collected by Clover. Clover's test result collection may also be switched off via thedontRecordTestResultsattribute on<clover-setup>or<clover-instr>.
e.g.
<testresults dir="test-results" include="TEST-*.xml"/>
Limitations
Clover's per-test coverage collection does not support parallel test execution.