Saturday, October 12, 2013

Integration testing approaches: Should we use in-memory database?

Integration testing is a form of testing that verifies that components of our application properly work together & with external resources(database, disk-drive etc).

Let's say we have application that uses database in some way. How can we cover components that use database by Integration tests?

The most important principle of testing is Isolation:

Tests should be isolated in the data creating and quering from other tests.

So, each good Integration test that uses database should consists of the following steps:
  • Cleaning database 
  • Inserting testing data 
  • Calling component being tested 
  • Checking result / database state 

There are 3 popular ways to write Integration Tests:
  1. Use the same development database for Integration tests 
  2. Generate empty database before each test 
  3. Use some in-memory database 
1. The first choice is to use the same development database for testing. This choice is the worst one, because we can't populate development database by testing data without breaking Isolation principle.

So, it's read-only mode testing. We can test some read-only requests to database, but even then, we are unable to make assertions well, because this data is fragile(we can make changes in our development database at any time and tests will be broken).

2. The second choice is to generate empty database with testing data before each test.
It's good practice to have testing environment that resembles real production environment as close as possible. So, in this case we use the same database-engine for testing as production's one, which makes possible to catch some database-specific problems early.

Disadvantage of this approach is that Integration tests can be pretty slow.

3. The third choice is a to use in-memory database.
If for some reasons we can't use the second choice then we can use in-memory database for testing. It's the compromise choice, because we can ensure Isolation of our tests + high speed of execution, but we sacrifice the closeness to the real environment.

Sqlite is a great example of such database.

For example, if we use NHibernate it's very easy to inject Sqlite database(instead of our real one) in our tests. This database will be based on the same NHibernate mappings and will use the same database-logic.

If you are interested in this approach, you can find example project of using NHibernate + Sqlite for testing on GitHub.