Part 4 - Test Optimization Tutorial

This section of the tutorial walks through the process of setting up Clover Test Optimization, which efficiently runs only the tests for code which has changed since the last build.

On this page:

This tutorial assumes you have completed the other Clover Tutorial steps and have fully set up Ant in those steps to build and test the MoneyBag Java project, with Clover testing the JUnit code coverage. We will make use of the Ant tasks set up in build.xml from the previous tutorial chapters, here. Knowing that, read on.

The process described here will change your build file to always run in Test Optimization mode.

Adding Test Optimization Tasks to build.xml

1. Adding Paths to Resources

Open your build.xml file.
You should already have this line included:

<taskdef resource="cloverlib.xml" classpath="${clover.jar}"/>

from the earlier Clover-for-Ant tutorial steps.

Edit around this line to add in one additional line of code, as shown below:

<taskdef resource="cloverlib.xml" classpath="${clover.jar}"/>
<clover-env/>

2. Choosing a Location for the Snapshot File

Test Optimization uses the concept of a 'snapshot' file. This is a file that records information about the previous build, as a point of comparison. This is what allows Clover to run Optimized tests, by comparing the data in the snapshot file with the current build that you are intending to launch.

For the purposes of this tutorial, leave the snapshot file in its default location, (next to the clover database clover.db ) here:

<PROJECT_DIR>/.clover/coverage.db.snapshot

3. Adding a new Ant Target to Generate the Optimized Test 'Snapshot'

Now' we'll and a new Ant target to generate the test snapshot:

<target name="clover.snapshot" depends="with.clover">
    <clover-snapshot file="${clover.snapshot.file}"/>
</target>

4. Editing the JUnit Task to Add the 'clover-optimized-testset' Element

In your build.xml file, edit the test.run target ,specifically the junit portion and its sub-element, batchtest.

Having just completed the earlier steps in the Clover-for-Ant tutorial, your code inside batchtest will look like so:

<junit fork="yes" printsummary="true">
    <classpath refid="testbuild.classpath"/>
    <formatter type="xml"/>
    <batchtest fork="yes" todir="${test.result}">
        <fileset dir="${test.src}" includes="**/*Test.java"/>
    </batchtest>
</junit>

To add Test Optimization to the build: add a new element, clover-optimized-testset, move the fileset element inside the new clover-optimized-testset element.

Edit your new batchtest code block until it is the same as the following:

<junit fork="yes" printsummary="true">
    <classpath refid="testbuild.classpath"/>
    <formatter type="xml"/>
    <batchtest fork="yes" todir="${test.result}">
        <clover-optimized-testset fullrunevery="10"
                                  enabled="true"
                                  ordering="failfast"
                                  minimize="true"										
		                          snapshotfile="${clover.snapshot.file}">
            <fileset dir="${test.src}" includes="**/*Test.java"/>
        </clover-optimized-testset>
    </batchtest>
</junit>

Your Test Optimization configuration is now complete.

Demonstrating that Test Optimization is Working

Finally, we will build our project, creating the essential Clover 'snapshot' file. Next, we will edit one of the Java files in the 'Money' project, commenting out one of the unit tests. When we run the Test Optimized build a second time, Clover will compare the snapshot file against the new coverage database and then run targeted tests which incorporate only those files which have changed, (which in this case will be only one, the Java file that we edited). This saves valuable time, which is the key advantage of Test Optimization.

5. Running the Test Optimized Build

Run your build with the following command:

ant with.clover clean test.run clover.snapshot

Adding the clover.snapshot target here will create the additional snapshot database, which is used as a point of comparison for the Test Optimization.
This is essential for enabling Test Optimization of future builds.

Clover will output this text to the console, showing that it has created the all-important snapshot file:

test.run:
    [mkdir] Created dir: /tutorial/build/testresult
    [junit] Running com.cenqua.samples.money.MoneyBagTest
    [junit] Tests run: 26, Failures: 0, Errors: 0, Time elapsed: 0.141 sec
    [junit] Running com.cenqua.samples.money.MoneyTest
    [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.079 sec

...

clover.snapshot:
[clover-snapshot] Clover Version ..., built on ...
[clover-snapshot] Loaded from: /tutorial/lib/clover.jar
[clover-snapshot] Clover: Commercial License registered to Atlassian.
[clover-snapshot] Snapshot file not found, creating new file at 
/tutorial/.clover/clover.db.snapshot

6. Running an 'Empty' Optimized Build

If we re-run the same Ant build, Clover will detect that none of the source files have changed. Because we are running a Test Optimized build, Clover won't build or test anything, the result is zero tests.

With a Test-Optimized build, Clover will output the following if no files have changed since the last build:

test.run:
    [junit] Clover estimates saving around 3 seconds on this optimized test run.
    [junit] Clover is including 0 test classes in this run (total test classes: 2)

This is desired behaviour, especially in a continuous integration environment where builds are automated and run regularly.

7. Editing a Java File in the Project

To show how Test Optimization works, we'll change one of the files in the project. When we run the Test Optimized build, Clover will detect that this file has changed and build it exclusively (rather than rebuilding everything).

Edit the file MoneyBag.java from the tutorial project. For the purposes of this demonstration, add a System.out.println() in the add method on line #27:

public IMoney add(IMoney m) {
    System.out.println("Adding: " + m);
    return m.addMoneyBag(this);
}

Now save the file.

8. Rebuilding the Project with Test Optimization

Now having changed a file in the project, we will run the same Ant tasks again.

ant with.clover test.run clover.snapshot

Clover will detect that the source file has changed, rebuilding and only running the tests for that file specifically. We can see this illustrated in the console output:

test.run:
    [junit] Clover estimates saving around 3 seconds on this optimized test run.
    [junit] Clover is including 1 test class in this run (total test classes: 2)
    [junit] Running com.cenqua.samples.money.MoneyBagTest
    [junit] Tests run: 26, Failures: 0, Errors: 0, Time elapsed: 0.146 sec

This Clover output shows that only one of two test classes was included. Note that all the test methods in this one test class were run, since Clover currently optimizes to the class level only. Clover also estimates the time saved in this particular build and test run. In this case the saving is only seconds, but in more complex projects it could well be multiple minutes or hours.

That concludes the Clover-for-Ant Test Optimization tutorial. For more information on integrating Test Optimization, see the related links below.

Page tree