November 15, 2011

Spring pitfalls: transactional tests considered harmful

One of the Spring killer-features is an in-container integration testing. While EJB lacked this functionality for many years (Java EE 6 finally addresses this, however I haven't, ekhem, tested it), Spring from the very beginning allowed you to test the full stack, starting from web tier, through services all the way down to the database.

Database is the problematic part. First you need to use in-memory self-contained database like H2 to decouple your tests from an external database. Spring helps with this to a great degree, especially now with profiles and embedded database support. The second problem is more subtle. While typical Spring application is almost completely stateless (for better or worse), database is inherently stateful. This complicates integration testing since the very first principle of writing tests is that they should be independent on each other and repeatable. If one test writes something to the database, another test may fail; also the same test may fail on subsequent call due to database changes.

Obviously Spring handles this problem as well with a very neat trick: prior to running every test Spring starts a new transaction. The whole test (including its setup and tear down) runs within the same transaction which is... rolled back at the end. This means all the changes made during the test are visible in the database just like if they were persisted. However rollback after every test wipes out all the changes and the next test works on a clean and fresh database. Brilliant!

Unfortunately this is not yet another article about Spring integration testing advantages. I think I have written hundreds if not thousands of such tests and I truly appreciate the transparent support Spring framework gives. But I also came across numerous quirks and inconsistencies introduces by this comfortable feature. To make matters worse, very often so-called transactional tests are actually hiding errors convincing the developer that the software works, while it fails after deployment! Here is a non-exhaustive but eye-opening collection of issues:


@Test
public void shouldThrowLazyInitializationExceptionWhenFetchingLazyOneToManyRelationship() throws Exception {
  //given
  final Book someBook = findAnyExistingBook();

  //when
  try {
    someBook.reviews().size();
    fail();
  } catch(LazyInitializationException e) {
    //then
  }
}