DbUnit Best Practices: Inserting Test Data Into Our Database

After we have finished this lesson, we

• Can identify the characteristics of a good DbUnit data set.
• Can avoid making the most common mistakes that are often found from DbUnit data sets.
• Know how we can create DbUnit data sets which are easy to read and maintain.

Our Goals

When we create DbUnit data sets, we have to fulfill these two goals:

• Our data sets should be easy to read. If a test case fails, the odds are that we have to read the data sets that are used to initialize our database into a known state before the failed integration test is run. If our data sets are not easy to read, it is a lot harder to figure out the reason why our integration test failed.
• Our data sets should be easy to maintain. When we make changes to our application, we have to make some changes to our integration tests and DbUnit data sets. If our data sets are not easy to maintain, making these changes slows us down, and this might eventually convince us to stop maintaining our integration tests.

Unfortunately, it’s not easy to obey these rules. Next we will take a look at three mistakes that are often found from DbUnit data sets.

The Three Most Common Mistakes Found From DbUnit Data Sets

There are three common mistakes that stop us from creating data sets that are easy to read and maintain. These mistakes are:

• We create our data sets by exporting them from the real database. The problem is that these data sets are so big that quite often no one bothers to clean them up because it would be an enormous task. It should be no surprise that these data sets can be extremely hard to read and even harder to maintain.
• We create only one data set. If we add all data into the same data set, its size will grow over time. When enough time has passed, we end up with a huge data set that has the same problems as the data sets which are exported from the real database.
• We create test specific data sets. It is natural to think that we can solve the problems of large data sets by creating a test specific data set which contains all data that is required to run one test case. The problem of this approach is that we end up creating similar data sets that contain slightly different test data. Even though these data sets are typically easy to read, they are really painful to maintain because if we make changes to our database, we have to make these changes to every data set that inserts data into the changed database tables.

It should be clear that we shouldn’t use any of these three practices because they prevent us from creating data sets that are easy to read and maintain. Let’s find out how we can create DbUnit data sets which don’t suck.

Creating DbUnit Data Sets Which Don’t Suck

We can create clean DbUnit data sets by following these steps:

1. Split our database into logical sections.
2. Create the required DbUnit data sets. One data set should initialize all tables that belong to the same logical section.

This is of course easier said than done. One of the first problems that we face when we try to follow this approach is that it can be hard to identify the logical sections. That is why I will demonstrate the usage of this approach by using an example, but first I want to introduce a software development pattern that has helped me to identify the logical sections of my databases.

This pattern is called aggregate, and Martin Fowler defines this pattern as follows:

Aggregate is a pattern in Domain-Driven Design. A DDD aggregate is a cluster of domain objects that can be treated as a single unit.

I know that this definition might not be very useful. Unfortunately this is a quite broad topic and it is impossible to cover it on this course. If you are not familiar with this pattern, you should take a look at the following blog posts / articles:

Let’s find out how we can split the database of our task tracker into logical sections.

Example: Splitting the Database of Our Task Tracker

Let’s assume that all integration tests use the same data set for initializing the database into a known state. This data set looks as follows:

 1<dataset>
 2	<persons id="1"
 3	         email_address="john.doe@gmail.com"
 4			 name="John Doe"
 5			 password="password"
 6			 user_role="ROLE_USER"/>
 7	<persons id="2"
 8	         email_address="anna.assignee@gmail.com"
 9			 name="Anna Assignee"
10			 password="password"
11			 user_role="ROLE_USER"/>
12
13    <tasks id="1"
14		   assignee_id="2"
15           creator_id="1"
16           description="bar"
17           title="foo"
18           status="OPEN"/>
19    <tags id="1"
20          name="Easy"/>
21    <tasks_tags task_id="1"
22                tag_id="1"/>
23</dataset>

When we take a closer look at our data set, we can see that it has two logical sections:

1. The information that is inserted into the persons table. These two rows contain the information of the persons who use our task tracker.
2. The information that is inserted into the tasks, tags, and tasks_tags tables. These rows contain the tasks of our task tracker and the tags of these tasks.

This means that we should split this data set by following these steps:

First, we have to create a data set that initializes the persons table. This data set looks as follows:

 1<dataset>
 2	<persons id="1"
 3	         email_address="john.doe@gmail.com"
 4			 name="John Doe"
 5			 password="password"
 6			 user_role="ROLE_USER"/>
 7	<persons id="2"
 8	         email_address="anna.assignee@gmail.com"
 9			 name="Anna Assignee"
10			 password="password"
11			 user_role="ROLE_USER"/>
12</dataset>

Second, we have to create a data set that initializes the tasks, tags, and tasks_tags tables. This data set looks as follows:

 1<dataset>
 2    <tasks id="1"
 3		   assignee_id="2"
 4           creator_id="1"
 5           description="bar"
 6           title="foo"
 7           status="OPEN"/>
 8    <tags id="1"
 9          name="Easy"/>
10    <tasks_tags task_id="1"
11                tag_id="1"/>
12</dataset>

We have now created two small data sets, but this doesn’t help us to achieve our goals if we don’t use these data sets in the right way. Let’s find out how we can do it.

Using Our New DbUnit Data Sets

After we have split our database into logical sections and created the data sets that contain our test data, we must use these data sets by following these rules:

• We must initialize our database by using these data sets. If we need to insert test data into tables that belong to different logical sections, we must initialize our database by using the data sets which contain the information that is required by the invoked integration test.
• We must not create new data sets unless we have added new logical sections into our database. If we need to add new information into existing logical sections, we should add new rows into our existing data set files.
• We are not allowed to add new information into our existing data sets unless it’s absolutely necessary. We should keep our data sets as small as possible because this makes them easier to read and maintain. Also, this rule forces us to think about our test cases before we write them. This helps us to avoid writing unnecessary tests.

These rules might seem strict, and I know that it is hard to follow them. However, if we follow these rules, we can enjoy from these two benefits:

• Our data sets are as small as possible and this makes them as easy to read as possible. This means that we don’t have to spend endless hours debugging our integration tests just because we cannot read the data set file that initializes the used database.
• If we make changes to our database, we have to make changes only to the data sets which insert information into the changed database tables. This is a lot faster than making the same changes to a huge data set that initializes our entire database or updating multiple test specific data sets.

Let’s summarize what we learned from this lesson.

Summary

This lesson has taught us five things:

• A good DbUnit data set is easy to read and maintain.
• We shouldn’t export our test data from the real database.
• We shouldn’t use test specific data sets or data sets that initialize the entire database.
• When we create our DbUnit data sets, we should split our database into logical sections and remember that one data set must initialize all tables that belong to the same logical section.
• We must not create a new data set unless we have added a new logical section into our database.