Run Integration Tests Separately Within Your Maven Build

There are several ways to configure Maven to run designated tests separately in a Java project. Usually, people want to distinguish between unit tests and other types of automated tests during a build. Unit tests are fast because you are mocking all the external services that the particular code under test is relying upon. They’re also typically smaller than functional tests, since they are (supposed to be ;) testing a unit of code rather than an entire feature.

However, functional tests are also critical to the success of your project. You or your managers are probably interested in seeing automated end-to-end usage of your application running constantly without errors, but how is this possible without annoying the developers as they wait for all the tests to finish?

The Maven Failsafe plugin is most helpful in separating unit from functional tests. By default, it focuses on tests whose filename follows the specific pattern:

**/IT*.java

**/*IT.java

**/*ITCase.java

Of course, you can add (or even exclude) files of particular naming patterns by modifying your POM file as described in the documentation.

The Circle of Life(cycles): It Builds Us All

A very simple way to get started with Failsafe is simply to add the following to your POM file:

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-failsafe-plugin</artifactId>

<id>integration-test</id>

<goals>

<goal>integration-test</goal>

</goals>

</execution>

<id>verify</id>

<goals>

<goal>verify</goal>

</goals>

</execution>

</executions>

</plugin>

This tells Maven to add Failsafe’s integration-test goal to the integration-test stage of your build when you run it, then the same for the verify goal. This means that to run your functional tests, all you need to do is run Maven with a build goal of “integration-test” or later in the lifecycle, including the popular “mvn install”. To skip your functional tests, simply pick a build goal prior to “integration-test”, such as “mvn package”.

Of course, this leaves you with the disadvantage that you won’t be able to deploy the application to any environments until you wait on all your tests to finish, and it probably won’t deploy the application until all your tests are passing. If you want to use “mvn install” to deploy your application to your test environment without waiting on the functional tests to complete, consider using Maven profiles.

Separation Via Profiles

In Maven, you can construct different profiles in order to specify different ways you want a build to work, such as utilizing different plugins such as running Surefire (for unit tests) versus Failsafe (for functional tests). Here is an example of what you would put in your POM to run Failsafe when the Maven profile with-functional-tests is specified:

<id>with-functional-tests</id>

<build>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-failsafe-plugin</artifactId>

<id>integration-test</id>

<goals>

<goal>integration-test</goal>

</goals>

</execution>

<id>verify</id>

<goals>

<goal>verify</goal>

</goals>

</execution>

</executions>

</plugin>

</plugins>

</build>

</profile>

</profiles>

Notice that everything within <plugin></plugin> is exactly the same as in the first example; the only difference is the Failsafe plugin only runs when this profile is specified (on the command line with -Pwith-functional-tests). It also gives you the benefit of limiting which environments the integration & regression tests actually run on, since developers won’t want to run every single functional test just to make sure the build succeeds before they can push changes to the code repository, and they won’t run unless they specify this profile explicitly (unless you put this under the “default” profile, and then they’ll just hate you :-P).

Annotation As a Solution

Yet another approach suggests creating an empty interface simply for marking purposes and then using that interface as a @Category to distinguish between your test types.

You might define a file such as IntegrationTest.java:

package com.test.annotation.type.IntegrationTest;

public interface IntegrationTest {}

And then use it in a real test as such:

import org.junit.experimental.categories.Category;

@Category(IntegrationTest.class)

public class RealTest {

// etc...

}

You then need to set up the POM so that the Surefire plugin (for unit tests) explicitly ignores your IntegrationTest type:

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-surefire-plugin</artifactId>

<groupId>org.apache.maven.surefire</groupId>

<artifactId>surefire-junit47</artifactId>

</dependency>

</dependencies>

<include>**/*.class</include>

</includes>

<excludedGroups>com.test.annotation.type.IntegrationTest</excludedGroups>

</configuration>

</plugin>

Also note the choice of surefire-junit47 for the artifact ID of Surefire, since this particular version correctly detects categories assigned with @Category.

Finally, you need to set up the POM so that the Failsafe plugin will actually run your IntegrationTest type (and only that type) during the integration-test build stage:

<artifactId>maven-failsafe-plugin</artifactId>

<groupId>org.apache.maven.surefire</groupId>

<artifactId>surefire-junit47</artifactId>

</dependency>

</dependencies>

<groups>com.test.annotation.type.IntegrationTest</groups>

</configuration>

<goals>

<goal>integration-test</goal>

</goals>

<include>**/*.class</include>

</includes>

</configuration>

</execution>

</executions>

</plugin>

The downside to this approach is that you have to write that IntegrationTest.java interface file into each module you plan to use integration tests in. If you have a multi-module Maven project, it violates the principle of DRY. Plus, it seems to involve more XML (or at least more complex XML) than the previous methods, and dependencies on surefire-junit47 and org.junit.experimental.categories.Category that you wouldn’t otherwise need.

Article Sources:

Adding a @Category annotation to describe test affiliation: http://www.javaworld.com/article/2074569/core-java/unit-and-integration-tests-with-maven-and-junit-categories.html

Simple use of the plugin and adding profiles: http://stackoverflow.com/questions/19594556/exclude-maven-module-with-intergration-tests-from-test-phase

Search This Blog

GOSHtastic - Game shows, Options, Software, & Hardware!