Creating Soft Assertions With AssertJ

After we have finished this lesson, we

• Know what a soft assertion is.
• Understand why and when we should use soft assertions.
• Can create soft assertions with AssertJ.

Sometimes One Assertion Is Not Good Enough

It’s a somewhat common (and equally controversial) opinion that a test case should have only one assertion. Although I think that we should follow this rule when we are writing data-centric tests (i.e. tests that verify that a property has the expected value), I think that it is not a good fit for test methods which verify that the state of an object is correct. The problem is that typically verifying the state of an object requires multiple assertions.

However, adding these assertions into one test method is not problem-free either. For example, if we have to verify that the created Task object is an open task, we have to write the following test method:

 1import org.junit.Test;
 2
 3import static org.assertj.core.api.Assertions.assertThat;
 4
 5public class TaskTest {
 6
 7    private static final Long ID = 1L;
 8    private static final Long ASSIGNEE_ID = 99L;
 9    private static final Long CREATOR_ID = 44L;
10    private static final String TITLE = "Write an example project";
11    private static final String DESCRIPTION = "description";
12
13    @Test
14    public void build_shouldCreateAnOpenTask() {
15        Task task = Task.getBuilder()
16                .withId(ID)
17                .withAssignee(ASSIGNEE_ID)
18                .withCreator(CREATOR_ID)
19                .withTitle(TITLE)
20                .withDescription(DESCRIPTION)
21                .build();
22
23        assertThat(task.getStatus())
24                .overridingErrorMessage(
25						"The status of an open task must be: %s but was: %s",
26                        TaskStatus.OPEN,
27                        task.getStatus()
28                )
29                .isEqualTo(TaskStatus.OPEN);
30
31        assertThat(task.getResolution())
32                .overridingErrorMessage(
33						"An open task cannot have a resolution but had: %s",
34                        task.getResolution()
35                )
36                .isNull();
37    }
38}

This test method has three problems:

• It adds business logic into our test suite. This is a bad idea. If we make changes to this logic, we have to make the same changes to all tests which which are testing the changed logic. In other words, this is a maintenance nightmare.
• It is not as easy to read as tests which have only one assertion. I don’t think that our test is horrible, but I think that each assertion makes it harder to read. This is just a simple example, but typically real-life applications have complex states that require “too many” assertions. If we would add these assertions into our test methods, our tests could look quite puzzling for people who see them at the first time.
• It increases the length of our feedback loop. If the first assertion fails, the second assertion is not invoked. This means that after we fix our code, the second assertion might fail when we run our tests. This is really frustrating and will slow us down.

As we can see, our test method has serious problems, and this lesson helps us to solve the last problem. Let’s find out how we can keep our feedback loop as fast as possible.

Soft Assertions to the Rescue

Soft assertions are assertions that follow these three rules:

• A single assertion error doesn’t cause instant test failure.
• All assertions are invoked and possible assertion errors are saved for later use.
• If assertion errors are found, the test fails and all assertion errors are shown.

We can create soft assertions with AssertJ by making the following changes into our test class:

First, we have to remove the regular assertions found from our test method. The source code of our test new test method looks as follows:

 1import org.junit.Test;
 2
 3public class TaskTest {
 4
 5    private static final Long ID = 1L;
 6    private static final Long ASSIGNEE_ID = 99L;
 7    private static final Long CREATOR_ID = 44L;
 8    private static final String TITLE = "Write an example project";
 9    private static final String DESCRIPTION = "description";
10
11    @Test
12    public void build_shouldCreateAnOpenTaskSoft() {
13        Task task = Task.getBuilder()
14                .withId(ID)
15                .withAssignee(ASSIGNEE_ID)
16                .withCreator(CREATOR_ID)
17                .withTitle(TITLE)
18                .withDescription(DESCRIPTION)
19                .build();
20    }
21}

Second, we have to create a new SoftAssertions object. This object allows us to write soft assertions with AssertJ. After we have created this object, the source code of our test class looks as follows:

 1import org.assertj.core.api.SoftAssertions;
 2import org.junit.Test;
 3
 4import static org.assertj.core.api.Assertions.assertThat;
 5
 6public class TaskTest {
 7
 8    private static final Long ID = 1L;
 9    private static final Long ASSIGNEE_ID = 99L;
10    private static final Long CREATOR_ID = 44L;
11    private static final String TITLE = "Write an example project";
12    private static final String DESCRIPTION = "description";
13
14    @Test
15    public void build_shouldCreateAnOpenTaskSoft() {
16        Task task = Task.getBuilder()
17                .withId(ID)
18                .withAssignee(ASSIGNEE_ID)
19                .withCreator(CREATOR_ID)
20                .withTitle(TITLE)
21                .withDescription(DESCRIPTION)
22                .build();
23
24        SoftAssertions assertions = new SoftAssertions();
25    }
26}

Third, we have to write our assertions by using the API that is provided by the SoftAssertions class. Because the SoftAssertions object provides us proxies of the AssertJ assertion objects, we can use a similar API that was described in the previous lesson .

After we have written assertions, which ensure that the status of the Task object is TaskStatus.OPEN and its resolution is null, the source code of our test class looks as follows:

 1import org.assertj.core.api.SoftAssertions;
 2import org.junit.Test;
 3
 4import static org.assertj.core.api.Assertions.assertThat;
 5
 6public class TaskTest {
 7
 8    private static final Long ID = 1L;
 9    private static final Long ASSIGNEE_ID = 99L;
10    private static final Long CREATOR_ID = 44L;
11    private static final String TITLE = "Write an example project";
12    private static final String DESCRIPTION = "description";
13
14    @Test
15    public void build_shouldCreateAnOpenTaskSoft() {
16        Task task = Task.getBuilder()
17                .withId(ID)
18                .withAssignee(ASSIGNEE_ID)
19                .withCreator(CREATOR_ID)
20                .withTitle(TITLE)
21                .withDescription(DESCRIPTION)
22                .build();
23
24        SoftAssertions assertions = new SoftAssertions();
25
26        assertions.assertThat(task.getStatus())
27                .overridingErrorMessage(
28						"The status of an open task must be: %s but was: %s",
29                        TaskStatus.OPEN,
30                        task.getStatus()
31                )
32                .isEqualTo(TaskStatus.OPEN);
33
34        assertions.assertThat(task.getResolution())
35                .overridingErrorMessage(
36						"An open task cannot have a resolution but had: %s",
37                        task.getResolution()
38                )
39                .isNull();
40    }
41}

Fourth, we have to run our assertions by invoking the assertAll() method of the SoftAssertions class. The source code of our finished test class looks as follows:

 1import org.assertj.core.api.SoftAssertions;
 2import org.junit.Test;
 3
 4import static org.assertj.core.api.Assertions.assertThat;
 5
 6public class TaskTest {
 7
 8    private static final Long ID = 1L;
 9    private static final Long ASSIGNEE_ID = 99L;
10    private static final Long CREATOR_ID = 44L;
11    private static final String TITLE = "Write an example project";
12    private static final String DESCRIPTION = "description";
13
14    @Test
15    public void build_shouldCreateAnOpenTaskSoft() {
16        Task task = Task.getBuilder()
17                .withId(ID)
18                .withAssignee(ASSIGNEE_ID)
19                .withCreator(CREATOR_ID)
20                .withTitle(TITLE)
21                .withDescription(DESCRIPTION)
22                .build();
23
24        SoftAssertions assertions = new SoftAssertions();
25
26        assertions.assertThat(task.getStatus())
27                .overridingErrorMessage(
28						"The status of an open task must be: %s but was: %s",
29                        TaskStatus.OPEN,
30                        task.getStatus()
31                )
32                .isEqualTo(TaskStatus.OPEN);
33
34        assertions.assertThat(task.getResolution())
35                .overridingErrorMessage(
36						"An open task cannot have a resolution but had: %s",
37                        task.getResolution()
38                )
39                .isNull();
40
41        assertions.assertAll();
42    }
43}

When we we invoke the assertAll() method, the SoftAssertions object invokes the assertion proxies, gathers all assertion errors, and shows these assertion errors to us. If both of our assertions fail, we see an error message that looks like this:

1org.assertj.core.api.SoftAssertionError:
2The following 2 assertions failed:
31) The status of an open task must be: OPEN but was: CLOSED
42) An open task cannot have a resolution but had: DONE

In other words, soft assertions help us to keep our feedback loop as fast as possible because we can see all assertion errors without rerunning our test case.

Let’s move on and find out when we should use soft assertions.

When Should We Use Soft Assertions?

This lesson has demonstrated that soft assertions help us to keep our feedback loop as fast as possible because we can get all the required information without rerunning our test cases. This is definitely a good thing, and it will help us to save a lot of time in the long run.

Also, I think that it makes sense to use soft assertions when we are writing tests, which verify that the state of the asserted object is correct, because soft assertions help us to treat the state as a unique property of the asserted object. I argue that this is a better option than treating the state as a group of individual properties.

That being said, when we are writing tests, we should always follow these two rules:

• We should never add business logic into our test methods because it makes our tests painful to maintain.
• We should ensure that our choices make our tests easy to read and write.

Unfortunately I must admit that our finished test method doesn’t follow these rules because of two reasons:

• The person who writes it must know that the status of an open task must be TaskStatus.OPEN and its resolution must be null. This makes our tests hard to write. Also, if we decide to change this state, we have to make the same changes to every test method that tests it. This is a maintenance nightmare.
• When we decided to use the SoftAssertions class, we added technical implementation details into our test method. I argue that our tests should hide these details from the people who read them because these details can shift their attention from the essence of our test to the implementation details. In other words, we made our test harder to read.

To summarize, even though I think that soft assertions are a useful tool, I also think that we shouldn’t use them in our test classes because they don’t help us to write tests that are easy to read, write, and maintain.

Let’s move on and summarize what we learned from this lesson.

Summary

This lesson has taught us three things:

• Soft assertions help us to keep our feedback loop as fast as possible.
• We can create soft assertions with AssertJ by using the SoftAssertions class.
• We shouldn’t use soft assertions in our test classes because soft assertions don’t make our tests easier read, write, and maintain.

Get the source code from Github