Analysis of developing and testing software that depends on a database

Preface

The goals of the paper: 1. compare what options to write tests we have to what we desire, 2. assess alternatives DbUnit integration into bootique-cayenne-test and being on the line with JUnit rules.

To meet the goals, let's answer the questions: what kind of variants to write test cases DBUnit (a JUnit extension) suggests?; how we can use JUnit5 new features to organize lifecycle of the tests?; are there some known xUnit patterns to improve our test-framework?; would be possible to provide Json, CSV data import feature into the framework?

Introduction into xUnit

xUnit is a collective name for several unit testing frameworks (JUnit for Java, RUnit for R, etc.). All xUnit frameworks share basic component architecture:

• Test Runner - an application that instantiates a Test Suite Object and executes all the Testcase Objects it contains.
• Test Case - a procedure, whether manually executed or automated, that can be used to verify that the system under test (SUT) is behaving as expected. Often called a test case.
• Test Fixture - all the things we need to have in place to run a test and expect a particular outcome (i.e., the test context). Some variants of xUnit keep the concept of the test context separate from the Testcase Class that creates it; JUnit and its direct ports fall into this camp.
• Test Suit - a way to name a collection of tests that we want to run together.
• Test Run - a test or test suite can be run many times, each time yielding a different test result. Some commercial test automation tools record the results of each test run for prosperity.
• Assertions - a statement that something should be true. In xUnit-style Test Automation Frameworks, an assertion takes the form of an Assertion Method that fails when the actual outcome passed to it does not match the expected outcome.

Testing with Database

Applications with databases present some special challenges when writing tests.

Issues with Databases

• Persistent Fixture - the data in a database may potentially persist long after we run our test.
• Shared Fixture - the problem arises when the tests being run from two or more Test Runners interact by accessing the same fixture objects in the shared database instance.
• General Fixture - tests rely on databases as a large General Fixture that many tests use for different purposes.

Techniques for testing with databases

• Database Sandbox - provide a separate test database for each developer or tester, it helps to avoid Test Run Wars. Implementation: Dedicated Database Sandbox, DB Schema per Test Runner, Database Partitioning Scheme.
• Stored Procedure Test. Implementation: In-Database Stored Procedure Test, Remoted Stored Procedure Test.
• Table Truncation Teardown - truncate the tables modified during the test to tear down the fixture. Implementation: Lazy Teardow.
• Transaction Rollback Teardown - roll back the uncommitted test transaction as part of the teardown. Implementation: Object Transaction Rollback Teardown, Database Transaction Rollback Teardown.

Variants of xUnit

spring-test-dbunit comprises Junit, DbUnit

It uses one of xUnit patterns Four-Phase Test answering the question: How do we structure our test logic to make what we are testing obvious?

Each test has four distinct phases that are executed in sequence: fixture setup, exercise SUT, result verification, and fixture teardown. Why? The test reader must be able to quickly determine what behavior the test is verifying.

• Fixture setup establishes the SUT’s state prior to the test, which is an important input to the test.
• The exercise SUT phase is where we actually run the software we are testing.
• The result verification phase of the test is where we specify the expected outcome.
• The final phase, fixture teardown, is all about housekeeping.

//spring-test-dbunit test skeleton

@Test
@DatabaseSetup(value = "insert.xml")
@DatabaseSetup(connection="customerDataSource", value="insert-custs.xml")
@DatabaseTearDown(type = DatabaseOperation.DELETE_ALL, value = "clear.dataset.xml")
@ExpectedDatabase("expectedData.xml")
public void testInsert() throws Exception {
	// Inserts "insert.xml" into dataSource and "insert-custs.xml" into customerDataSource
	// ...
}

Bootique-Cayenne-Test

Use cases

//TODO:...