This is the documentation for Clover 3.3. View this page for the

Unknown macro: {spacejump}

of Clover, or visit the latest Clover documentation.

Introduction

Clover can be used to generate code coverage statistics from practically any kind of test - unit, integration, functional, regression ... - both automatic and manual. The only thing that has to be done is to instrument source code and run it with proper options.

The most frequent Clover usage is to run unit test with code coverage - typically the maven-surefire-plugin is used for this purpose - and thus Clover-for-Maven was designed to cooperate with Surefire plugin "out of the box".

In this short tutorial you will learn how to configure Clover with the Maven Failsafe Plugin, which is used for integration tests.

Comparison of maven-surefire-plugin and maven-failsafe-plugin

 

 maven-surefire-pluginmaven-failsafe-plugin
Main purposeunit testsintegration tests
Bound to build phasetest

pre-integration-test

integration-test

post-integration-test

verify

Build fails in phasetestverify
Default wildcard pattern

**/Test*.java

**/*Test.java

**/*TestCase.java

**/IT*.java

**/*IT.java

**/*ITCase.java

Default output directory${basedir}/target/surefire-reports${basedir}/target/failsafe-reports

 

Setting up Clover with maven-failsafe-plugin (only)

In order to have code coverage statistics from integration tests and excluding unit tests, you have to do the following:

  1. Disable Surefire plugin, e.g. by setting <skip>true</skip> option.
  2. Enable Failsafe plugin in your build
    1. Failsafe plugin requires a test framework provider, e.g. JUnit or TestNG - declare it.
  3. Tell Clover to use target/failsafe-reports as report directory - use the <reportDescriptor> for this.
  4. Tell Clover to use test case wildcard pattern for both plugins - use the <reportDescriptor> for this.
  5. Instrument sources, execute tests and generate reports
    1. we recommend calling clover goals from command line (as typically projects are multi-module and we have to call clover2:aggregate)
    2. we recommend calling "verify" target instead of "integration-test" (because when you call "integration-test", the Failsafe plugin will not perform post-integration-test cleanup)

 

Content of pom.xml

<dependencies>
    <!-- Test framework which will be used by Failsafe plugin. Version number is mandatory -->
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.7</version>
        <scope>test</scope>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>com.atlassian.maven.plugins</groupId>
            <artifactId>maven-clover2-plugin</artifactId>
            <configuration>
                <!-- Use custom report descriptor -->
                <reportDescriptor>clover-report.xml</reportDescriptor>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <!-- Disable unit tests -->
                <skip>true</skip>
            </configuration>
        </plugin>
    </plugins>
</build>

 

Content of clover-report.xml

<project name="Clover Report" default="current">
    <!-- Initialize Clover -->
    <clover-setup initString="${cloverdb}"/>
    <target name="historical">
        <!-- Empty as we're not interested in historical reports right now -->
    </target>
    <target name="current">
        <clover-report>
            <current outfile="${output}" title="${title}">
                <format type="html"/>
                <!-- Declare naming convention in order to have test classes listed on the "Tests" tab in HTML report -->
                <testsources dir="src/test">
                    <!-- Use Maven-Failsafe-Plugin naming convention -->
                    <include name="**/IT*.java"/>
                    <include name="**/*IT.java"/>
                    <include name="**/*ITCase.java"/>
                    <!-- Use Maven-Surefire-Pugin naming convention. 
                    NOTE: Although we don't run unit tests, we still want to have them on "Tests" tab instead of "Classes" -->
                    <include name="**/Test*.java"/>
                    <include name="**/*Test.java"/>
                    <include name="**/*TestCase.java"/>
                </testsources>
                <!-- Tell Clover to get test results from failsafe. They will be listed on "Results" tab -->
                <testresults dir="target/failsafe-reports" includes="TEST-*.xml"/>
            </current>
        </clover-report>
    </target>
</project>

 

Maven command

mvn clean clover2:setup verify clover2:aggregate clover2:clover

 

 

Setting up Clover with maven-surefire-plugin and maven-failsafe-plugin (combined report)

In order to have combined coverage statistics from unit and integration tests, you have to do the following:

  1. Set <reportsDirectory> option for both Surefire and Failsafe plugin pointing to the same location.
  2. Enable both Surefire and Failsafe plugin in your build.
    1. Failsafe plugin requires a test framework provider, e.g. JUnit or TestNG - declare it.
  3. Tell Clover to use location from point 1 as report directory - use the <reportDescriptor> for this.
  4. Tell Clover to use test case wildcard pattern for both plugins - use the <reportDescriptor> for this.
  5. Instrument sources, execute tests and generate reports
    1. we recommend calling clover goals from command line (as typically projects are multi-module and we have to call clover2:aggregate)
    2. we recommend calling "verify" target instead of "integration-test" (because when you call "integration-test", the Failsafe plugin will not perform post-integration-test cleanup)

 

Content of pom.xml

<properties>
    <!-- A common location in which a surefire report from 'test' and failsafe report from
    'integration-test' phase will be stored. See also the clover-report.xml file which refers
    to this location -->
    <surefire.and.failsafe.report.dir>target/test-report</surefire.and.failsafe.report.dir>
</properties>
<dependencies>
    <!-- Test framework which will be used by Failsafe plugin. Version number is mandatory -->
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.7</version>
        <scope>test</scope>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>com.atlassian.maven.plugins</groupId>
            <artifactId>maven-clover2-plugin</artifactId>
            <configuration>
                <!-- Use custom report descriptor -->
                <reportDescriptor>clover-report.xml</reportDescriptor>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <reportsDirectory>${surefire.and.failsafe.report.dir}</reportsDirectory>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <reportsDirectory>${surefire.and.failsafe.report.dir}</reportsDirectory>
            </configuration>
        </plugin>
    </plugins>
</build>

 

Content of clover-report.xml

<project name="Clover Report" default="current">
    <!-- Initialize Clover -->
    <clover-setup initString="${cloverdb}"/>
    <target name="historical">
        <!-- Empty as we're not interested in historical reports right now -->
    </target>
    <target name="current">
        <clover-report>
            <current outfile="${output}" title="${title}">
                <format type="html"/>
                <!-- Declare naming convention in order to have test classes listed on the "Test" tab in HTML report -->
                <testsources dir="src/test">
                    <!-- Use Maven-Failsafe-Plugin naming convention -->
                    <include name="**/IT*.java"/>
                    <include name="**/*IT.java"/>
                    <include name="**/*ITCase.java"/>
                    <!-- Use Maven-Surefire-Pugin naming convention -->
                    <include name="**/Test*.java"/>
                    <include name="**/*Test.java"/>
                    <include name="**/*TestCase.java"/>
                </testsources>
                <!-- Tell Clover to get test results directory as defined in pom.xml. They will be listed on "Results" tab -->
                <testresults dir="target/test-report" includes="TEST-*.xml"/>
            </current>
        </clover-report>
    </target>
</project>

 

Maven command

mvn clean clover2:setup verify clover2:aggregate clover2:clover

 

Test optimization

Test Optimization feature is available for Surefire plugin. You you have to use:

(warning) Test Optimization feature for maven-failsafe-plugin is not available yet.

Sample project

A sample project shows usage of Surefire and Failsafe plugins together.

Checkout code from Bitbucket: https://bitbucket.org/atlassian/maven-clover2-plugin 

Go to: src/it/surefire-and-failsafe-plugins

Run mvn command with goals as specified in goals.txt file in this project. Use Maven 2.x or higher and Java5 or higher.

References

See also:

 

  • No labels





Except where otherwise noted, content in this space is licensed under a Creative Commons Attribution 2.5 Australia License.