Running Kotlin Tests With Maven

Published: January 3, 2019

After we have finished this webinar, we can create a Maven project that can compile and run both unit and integration tests which use Kotlin and JUnit 5.

Get the source code from Github .

The Requirements of Our Maven Build

The requirements of our Maven Build are:

All code (application and tests) must use Kotlin.
Unit and integration tests must have separate source and resource directories.
It must be possible to run only unit or integration tests.
It must be possible to run both unit and integration tests.
Only unit tests are run by default.

Let’s start by taking a quick look at the system under test.

Introduction to System Under Test

During this webinar we will write both unit and integration tests for a MessageService class. This class has one method called getMessage() which returns the message: ‘Hello World!’.

The source code of the MessageService class looks as follows:

1class MessageService {
2
3    fun getMessage(): String {
4        return "Hello World!"
5    }
6}

Next, we will find out how we can create a simple Kotlin project that can run only unit tests.

Running Unit Tests With Maven

When we want to create a Kotlin project that can run only unit tests, we have to follow these steps:

First, we have to specify three properties:

The kotlin.version property specifies the used Kotlin version.
The kotlin.compiler.incremental property enables incremental compilation.
The junit-jupiter.version property specifies the used JUnit 5 version.

After we have specified these properties, the properties section of our POM file looks as follows:

1<properties>
2	<kotlin.version>1.3.11</kotlin.version>
3	<kotlin.compiler.incremental>true</kotlin.compiler.incremental>
4	<junit-jupiter.version>5.3.2</junit-jupiter.version>
5</properties>

Second, we have to specify the required dependencies. Because we want to write tests for a Kotlin application by using Kotlin and JUnit 5, we have to specify the following dependencies:

The Kotlin standard library (kotlin-stdlib).
AssertJ Core (not mandatory).
The JUnit 5 dependencies (JUnit Jupiter API and JUnit Jupiter engine).

After we have specified the required dependencies, the dependencies section of our POM file looks as follows:

 1<dependencies>
 2    <dependency>
 3        <groupId>org.jetbrains.kotlin</groupId>
 4        <artifactId>kotlin-stdlib</artifactId>
 5        <version>${kotlin.version}</version>
 6    </dependency>
 7
 8    <!-- Testing Dependencies -->
 9    <dependency>
10        <groupId>org.assertj</groupId>
11        <artifactId>assertj-core</artifactId>
12        <version>3.11.1</version>
13        <scope>test</scope>
14    </dependency>
15    <dependency>
16        <groupId>org.junit.jupiter</groupId>
17        <artifactId>junit-jupiter-api</artifactId>
18        <version>${junit-jupiter.version}</version>
19        <scope>test</scope>
20    </dependency>
21    <dependency>
22        <groupId>org.junit.jupiter</groupId>
23        <artifactId>junit-jupiter-engine</artifactId>
24        <version>${junit-jupiter.version}</version>
25        <scope>test</scope>
26    </dependency>
27</dependencies>

Third, we have to configure the source and test directories of our Maven build. We will use the following directories:

The src/main/kotlin directory contains the source code of our application.
The src/test/kotlin directory contains the source code of our unit tests.

After we have configured these directories, the build section of our POM file looks as follows:

1<build>
2    <sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
3    <testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
4</build>

Fourth, we have to ensure that Maven compiles our Kotlin code. We can do this by using the Kotlin Maven plugin. We can use this plugin by adding the following snippet to the plugins section of our POM file:

 1<plugin>
 2    <artifactId>kotlin-maven-plugin</artifactId>
 3    <groupId>org.jetbrains.kotlin</groupId>
 4    <version>${kotlin.version}</version>
 5
 6    <executions>
 7        <execution>
 8            <id>compile</id>
 9            <goals>
10                <goal>compile</goal>
11            </goals>
12        </execution>
13
14        <execution>
15            <id>test-compile</id>
16            <goals>
17                <goal>test-compile</goal>
18            </goals>
19        </execution>
20    </executions>
21</plugin>

Fifth, we have to ensure that Maven runs our unit tests are run by using the Maven Surefire plugin version 2.22.1. We can do this by adding the following snippet to the plugins section of our POM file:

1<plugin>
2    <groupId>org.apache.maven.plugins</groupId>
3    <artifactId>maven-surefire-plugin</artifactId>
4    <version>2.22.1</version>
5</plugin>

We have now configured our Maven build. However, before we can run our unit tests, we have to write one test class and put this class to the src/test/kotlin directory. The source code of our test class looks as follows:

 1import org.assertj.core.api.Assertions.assertThat
 2import org.junit.jupiter.api.BeforeEach
 3import org.junit.jupiter.api.DisplayName
 4import org.junit.jupiter.api.Test
 5
 6class MessageServiceTest {
 7
 8    private lateinit var service: MessageService
 9
10    @BeforeEach
11    fun configureSystemUnderTest() {
12        service = MessageService()
13    }
14
15    @Test
16    @DisplayName("Should return the correct message")
17    fun shouldReturnCorrectMessage() {
18        val message = service.getMessage()
19        assertThat(message).isEqualTo("Hello World!")
20    }
21}

We can now run our tests by using the command: mvn clean test. When we run this command, we should see that the Maven Surefire plugin runs our unit test:

 1[INFO] -------------------------------------------------------
 2[INFO]  T E S T S
 3[INFO] -------------------------------------------------------
 4[INFO] Running com.testwithspring.master.kotlin.MessageServiceTest
 5[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 s -
 6in com.testwithspring.master.kotlin.MessageServiceTest
 7[INFO]
 8[INFO] Results:
 9[INFO]
10[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Additional Reading:

So, we have just created a Maven project that can compile and run our unit tests. However, our build doesn’t have any support for integration tests. Let’s move on and find out how we can support both unit and integration tests.

Running Unit and Integration Tests With Maven

Because we want to run both unit and integration tests with Maven, we have to make some changes to our Maven build. Let’s start by configuring the source and resource directories of our Maven build.

Configuring the Source and Resource Directories

We can configure the source and resource directories of our unit and integration tests by following these steps:

First, we have add the resource directory of our integration tests to our Maven build. We can do this by using the Build Helper Maven plugin . When we want to use this plugin, we have to follow these steps:

Add the Build Helper Maven plugin to our Maven build.
Create an execution that adds the src/integration-test/resources directory to our Maven build.

After we have configured the Build Helper Maven plugin, its configuration looks as follows:

 1<plugin>
 2    <groupId>org.codehaus.mojo</groupId>
 3    <artifactId>build-helper-maven-plugin</artifactId>
 4    <version>3.0.0</version>
 5    <executions>
 6        <execution>
 7            <id>add-integration-test-resources</id>
 8            <phase>generate-test-resources</phase>
 9            <goals>
10                <goal>add-test-resource</goal>
11            </goals>
12            <configuration>
13                <resources>
14                    <resource>
15                        <filtering>true</filtering>
16                        <directory>src/integration-test/resources</directory>
17                    </resource>
18                </resources>
19            </configuration>
20        </execution>
21    </executions>
22</plugin>

If our integration tests don’t have any resources (such as DbUnit data sets), we can naturally skip this step.

Additional Reading:

Build Helper Maven Plugin: Usage

Second, because our unit and integration tests have separate source directories, we have to ensure that Maven can compile the source code of our unit and integration tests. We can do this by following these steps:

Remove the testSourceDirectory element from the build section of our POM file.
Ensure that the Kotlin Maven plugin can compile the source code found from the src/test/kotlin and src/integration-test/kotlin directories.

After we have made these changes to our Maven build, the configuration of the Kotlin Maven plugin looks as follows:

 1<plugin>
 2    <artifactId>kotlin-maven-plugin</artifactId>
 3    <groupId>org.jetbrains.kotlin</groupId>
 4    <version>${kotlin.version}</version>
 5    <executions>
 6        <execution>
 7            <id>compile</id>
 8            <goals>
 9                <goal>compile</goal>
10            </goals>
11        </execution>
12        <execution>
13            <id>test-compile</id>
14            <goals>
15                <goal>test-compile</goal>
16            </goals>
17            <configuration>
18                <sourceDirs>
19                    <sourceDir>${project.basedir}/src/test/kotlin</sourceDir>
20                    <sourceDir>${project.basedir}/src/integration-test/kotlin</sourceDir>
21                </sourceDirs>
22            </configuration>
23        </execution>
24    </executions>
25</plugin>

After we have configured the source and resource directories of our unit and integration tests, we have to ensure that we can run our integration tests Maven.

Running Integration Tests With Maven

We can run our integration tests with Maven by using the Maven Failsafe plugin (version 2.22.1). We can add this plugin to our Maven build by adding the following snippet to the plugins section of our POM file:

1<plugin>
2    <groupId>org.apache.maven.plugins</groupId>
3    <artifactId>maven-failsafe-plugin</artifactId>
4    <version>2.22.1</version>
5</plugin>

After we have added the Maven Failsafe plugin to our Maven build, we have to configure it by following these steps:

Ensure that the Maven Failsafe plugin runs all test classes whose name ends with the suffix: Test. We can do this by changing the wildcard pattern that is used to select the invoked test classes.
Ensure that the Maven Failsafe plugin runs both integration-test and verify goals. The integration-test goal runs our integration tests and the verify goal checks the results of our integration tests and fails the build if our integration tests failed.

After we have configured the Maven Failsafe plugin, its configuration looks as follows:

 1<plugin>
 2    <groupId>org.apache.maven.plugins</groupId>
 3    <artifactId>maven-failsafe-plugin</artifactId>
 4    <version>2.22.1</version>
 5    <configuration>
 6        <includes>
 7            <include>**/*Test.java</include>
 8        </includes>
 9    </configuration>
10    <executions>
11        <execution>
12            <id>integration-tests</id>
13            <goals>
14                <goal>integration-test</goal>
15                <goal>verify</goal>
16            </goals>
17        </execution>
18    </executions>
19</plugin>

We don’t use the default include pattern because I don’t like its naming conventions. However, If you want to use it, the Maven Failsafe plugin includes all test classes whose name starts with the prefix: IT or ends with the prefixes: IT or ITCase.

Additional Reading:

Maven Failsafe Plugin: Inclusions and Exclusions of Tests

After we have configured the Maven Failsafe plugin, we have to create the Maven profiles which allow us to run either unit tests, integration tests, or all tests. Let’s move on and find out how we create these Maven profiles.

Creating the Required Maven Profiles

We can create the required Maven profiles by following these steps:

First, we have to specify the default values of these two properties:

The value of the skip.integration.tests property specifies if our build should skip our integration tests. Because only unit tests are run by default, the default value of this property must be true.
The value of the skip.unit.tests property defines if our build should skip our unit tests. Again, because only unit tests are run by default, the default value of this property must be false.

After we have specified these default values, the properties section of our POM file looks as follows:

1<properties>
2    <kotlin.version>1.3.11</kotlin.version>
3    <kotlin.compiler.incremental>true</kotlin.compiler.incremental>
4    <junit-jupiter.version>5.3.2</junit-jupiter.version>
5
6    <skip.integration.tests>true</skip.integration.tests>
7    <skip.unit.tests>false</skip.unit.tests>
8</properties>

Second, we have to create these three Maven profiles:

The dev profile is the default profile of our Maven build, and it runs only unit tests.
The integration-test profile runs only integration tests.
The all-tests profile runs both unit and integration tests.

After we have created these Maven profiles, the profiles section of our POM file looks as follows:

 1<profiles>
 2    <profile>
 3        <id>dev</id>
 4        <activation>
 5            <activeByDefault>true</activeByDefault>
 6        </activation>
 7    </profile>
 8    <profile>
 9        <id>integration-test</id>
10        <properties>
11            <skip.integration.tests>false</skip.integration.tests>
12            <skip.unit.tests>true</skip.unit.tests>
13        </properties>
14    </profile>
15    <profile>
16        <id>all-tests</id>
17        <properties>
18            <skip.integration.tests>false</skip.integration.tests>
19            <skip.unit.tests>false</skip.unit.tests>
20        </properties>
21    </profile>
22</profiles>

Third, we have to ensure that the Maven Surefire plugin doesn’t run unit tests if the value of the skip.unit.tests property is true. After we have made this change to the configuration of the Maven Surefire plugin, its configuration looks as follows:

1<plugin>
2    <groupId>org.apache.maven.plugins</groupId>
3    <artifactId>maven-surefire-plugin</artifactId>
4    <version>2.22.1</version>
5    <configuration>
6        <skipTests>${skip.unit.tests}</skipTests>
7    </configuration>
8</plugin>

Fourth, we have to ensure that the Maven Failsafe plugin doesn’t run integration tests if the value of the skip.integration.tests property is true. After we have made this change to the configuration of the Maven Failsafe plugin, its configuration looks as follows:

 1<plugin>
 2    <groupId>org.apache.maven.plugins</groupId>
 3    <artifactId>maven-failsafe-plugin</artifactId>
 4    <version>2.22.1</version>
 5    <configuration>
 6        <includes>
 7            <include>**/*Test.java</include>
 8        </includes>
 9    </configuration>
10    <executions>
11        <execution>
12            <id>integration-tests</id>
13            <goals>
14                <goal>integration-test</goal>
15                <goal>verify</goal>
16            </goals>
17            <configuration>
18                <skipTests>${skip.integration.tests}</skipTests>
19            </configuration>
20        </execution>
21    </executions>
22</plugin>

We have now created the required Maven profiles. Next, we will find out how we can filter the invoked tests by using JUnit 5 tags.

Filtering the Invoked Tests With JUnit 5 Tags

Because the Maven Surefire and Failsafe plugins use the same include pattern, we have to filter the invoked tests by using JUnit 5 tags. We can do this by following these steps:

First, we have to ensure that the Maven Surefire plugin runs only tests which have the tag: unitTest. After we have made the required changes to the configuration of the Maven Surefire plugin, it looks as follows:

1<plugin>
2    <groupId>org.apache.maven.plugins</groupId>
3    <artifactId>maven-surefire-plugin</artifactId>
4    <version>2.22.1</version>
5    <configuration>
6        <groups>unitTest</groups>
7        <skipTests>${skip.unit.tests}</skipTests>
8    </configuration>
9</plugin>

Second, we have to ensure that the Maven Failsafe plugin runs only tests which have the tag: integrationTest. After we have made the required changes to the configuration of the Maven Failsafe plugin, it looks as follows:

 1<plugin>
 2    <groupId>org.apache.maven.plugins</groupId>
 3    <artifactId>maven-failsafe-plugin</artifactId>
 4    <version>2.22.1</version>
 5    <configuration>
 6        <groups>integrationTest</groups>
 7        <includes>
 8            <include>**/*Test.java</include>
 9        </includes>
10    </configuration>
11    <executions>
12        <execution>
13            <id>integration-tests</id>
14            <goals>
15                <goal>integration-test</goal>
16                <goal>verify</goal>
17            </goals>
18            <configuration>
19                <skipTests>${skip.integration.tests}</skipTests>
20            </configuration>
21        </execution>
22    </executions>
23</plugin>

Third, we have to ensure that our unit test has the tag: unitTest. After we have added this tag to our unit test, the source code of our test class looks as follows:

 1import org.assertj.core.api.Assertions.assertThat
 2import org.junit.jupiter.api.BeforeEach
 3import org.junit.jupiter.api.DisplayName
 4import org.junit.jupiter.api.Tag
 5import org.junit.jupiter.api.Test
 6
 7@Tag("unitTest")
 8class MessageServiceTest {
 9
10    private lateinit var service: MessageService
11
12    @BeforeEach
13    fun configureSystemUnderTest() {
14        service = MessageService()
15    }
16
17    @Test
18    @DisplayName("Should return the correct message")
19    fun shouldReturnCorrectMessage() {
20        val message = service.getMessage()
21        assertThat(message).isEqualTo("Hello World!")
22    }
23}

Fourth, we have to write an integration test class which ensures that the system under test returns the correct message. After we have written this class, its source code looks as follows:

 1import org.assertj.core.api.Assertions
 2import org.junit.jupiter.api.BeforeEach
 3import org.junit.jupiter.api.DisplayName
 4import org.junit.jupiter.api.Tag
 5import org.junit.jupiter.api.Test
 6
 7@Tag("integrationTest")
 8class GetMessageTest {
 9
10    private lateinit var service: MessageService
11
12    @BeforeEach
13    fun configureSystemUnderTest() {
14        service = MessageService()
15    }
16
17    @Test
18    @DisplayName("Should return the correct message")
19    fun shouldReturnCorrectMessage() {
20        val message = service.getMessage()
21        Assertions.assertThat(message).isEqualTo("Hello World!")
22    }
23}

Additional Reading:

We have now added integration test support to our Maven build. Let’s move on and find out how we can run our unit and integration tests.

Running Our Tests

If we want to run only unit tests, we have to run the command: mvn clean test -P dev. When we run this command at command prompt, we see that Maven runs only our unit tests:

 1[INFO] --- maven-surefire-plugin:2.22.1:test (default-test)
 2@ running-kotlin-tests-with-maven ---
 3[INFO]
 4[INFO] -------------------------------------------------------
 5[INFO]  T E S T S
 6[INFO] -------------------------------------------------------
 7[INFO] Running com.testwithspring.master.kotlin.MessageServiceTest
 8[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 s -
 9in com.testwithspring.master.kotlin.MessageServiceTest
10[INFO]
11[INFO] Results:
12[INFO]
13[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

If we want to run only integration tests, we have to run the command: mvn clean verify -P integration-test. When we run this command at command prompt, we see that Maven skips our unit tests and runs only our integration tests:

 1[INFO] --- maven-surefire-plugin:2.22.1:test (default-test)
 2@ running-kotlin-tests-with-maven ---
 3[INFO] Tests are skipped.
 4[INFO]
 5[INFO] --- maven-failsafe-plugin:2.22.1:integration-test (integration-tests)
 6@ running-kotlin-tests-with-maven ---
 7[INFO]
 8[INFO] -------------------------------------------------------
 9[INFO]  T E S T S
10[INFO] -------------------------------------------------------
11[INFO] Running com.testwithspring.master.kotlin.GetMessageTest
12[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.043 s -
13in com.testwithspring.master.kotlin.GetMessageTest
14[INFO]
15[INFO] Results:
16[INFO]
17[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
18[INFO]
19[INFO]
20[INFO] --- maven-failsafe-plugin:2.22.1:verify (integration-tests)
21@ running-kotlin-tests-with-maven ---

If we want to run both unit and integration tests, we have to run the command: mvn clean verify -P all-tests. When we run this command at command prompt, we see that Maven runs our unit and integration tests:

 1[INFO] --- maven-surefire-plugin:2.22.1:test (default-test)
 2@ running-kotlin-tests-with-maven ---
 3[INFO]
 4[INFO] -------------------------------------------------------
 5[INFO]  T E S T S
 6[INFO] -------------------------------------------------------
 7[INFO] Running com.testwithspring.master.kotlin.MessageServiceTest
 8[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.056 s -
 9in com.testwithspring.master.kotlin.MessageServiceTest
10[INFO]
11[INFO] Results:
12[INFO]
13[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
14[INFO]
15[INFO] --- maven-failsafe-plugin:2.22.1:integration-test (integration-tests)
16@ running-kotlin-tests-with-maven ---
17[INFO]
18[INFO] -------------------------------------------------------
19[INFO]  T E S T S
20[INFO] -------------------------------------------------------
21[INFO] Running com.testwithspring.master.kotlin.GetMessageTest
22[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.058 s -
23in com.testwithspring.master.kotlin.GetMessageTest
24[INFO]
25[INFO] Results:
26[INFO]
27[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
28[INFO]
29[INFO]
30[INFO] --- maven-failsafe-plugin:2.22.1:verify (integration-tests)
31@ running-kotlin-tests-with-maven ---

We can now create a Maven build that can run unit and integration tests which use Kotlin and JUnit 5. Let’s summarize what we learned from this webinar.

Summary

This webinar has taught us four things:

We have to declare the Kotlin standard library (kotlin-stdlib) dependency in our POM file.
We have to compile our test code by using the Kotlin Maven plugin.
If our integration tests have resource files (such as DbUnit data sets), we have to add the resource directory to our Maven build by using the Build Helper Maven plugin.
We can filter the invoked tests by using Maven profiles and JUnit 5 tags.

Get the source code from Github .