Test Driven Development with Unit tests and Integration Tests

The golden rule of Test Driven Development is never write new functionality without a failing test.

Yes. You need to start each feature writing a failing acceptance test, which guides your application design and defines application behaviour. While this test is failing, the feature is not implemented by your application. Then you should write unit tests for separate units that make up the feature.

This outer loop verifies that all the classes involved in the feature are working together as expected (integration tests). Inner loop tests verify that each class works as expected on its own (unit tests).

We should have many unit tests for all the components, but few integration tests to verify how they bind together.

Quality code at the cost of development speed

Integration tests are generally much more expensive because of the resources you need to set up, like real database. It slows down development and build time too.

The Shutl team works on the eBay Shipping Platform for Label Printing and Tracking. We always encourage and keep focus on engineering excellence with ongoing projects. While working on a project, we had integration tests to verify system flows. But these integration tests were using the staging database, hence slowing down the development and eventually increasing the build time.

So we had a spike using an in-memory database (HSQL) to run these integration tests. And our solution worked well.

HSQLDB is a portable RDBMS implemented in pure Java. It can be embedded within your application. It is very a small database that supports almost all features of the standard database system.  http://hsqldb.org/

Steps for HSQL DB configuration with Spring Framework:

Add maven dependency in pom.xml

<dependency>
   <groupId>org.hsqldb</groupId>
   <artifactId>hsqldb</artifactId>
   <version>${hsqldb.version}</version>
</dependency>

Configuration with xml file

You can configure JdbcTemplate in src/test/resources/spring/test-spring-context.xml file with these configurations:


<context:component-scan base-package="com.hsqlDemo.dao" />

<!-- dataSource configuration -->

<beans:bean id="datasource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<beans:property name="driverClassName" value="org.hsqldb.jdbcDriver" />
<beans:property name="url" value="jdbc:hsqldb:mem:test;" />
<beans:property name="username" value="sa" />
<beans:property name="password" value="" />

</beans:bean>

<jdbc:embedded-database id="test" type="HSQL">
<jdbc:script location="classpath: dbl/create-db.sql" />
<jdbc:script location="classpath: db/insert-data.sql" />
</jdbc:embedded-database>

<!-- JdbcTemplate Configuration -->

<beans:bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<beans:property name="dataSource" ref="datasource" />
</beans:bean>

Here “test” is HSQL database created in memory.
create-db.sql is a sql file having schema definitions for all required database tables.
insert-data.sql is an optional sql file with insert queries for pre-defined data, if any.

Configuration with java class

import javax.sql.DataSource;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabase;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType;

@Configuration
public class DataSourceConfig {
	@Bean 
	public DataSource dataSource(){
		EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
		EmbeddedDatabase db = builder.setType(EmbeddedDatabaseType.HSQL)
				// Spring will assign default database name as “testdb” if we skip this setter
				.setName("demoTestDb")
				.addScript("db/sql/create-db.sql")
				.addScript("db/sql/insert-data.sql")
				.build();
		
		return db;
	}
}

@ComponentScan(basePackages = "com.hsqlDemo.dao")
@Configuration
public class IntegrationTestConfig {
	
	@Autowired 
	DataSource dataSource;
	
	@Bean 
	public JdbcTemplate jdbcTemplate() {
		return new JdbcTemplate(dataSource);
	}
}

Now you can autowire jdbcTemplate in any DAO to connect to database.

Team achievement in Test Performance

I feel happy to share that HSQL DB for integration tests reduced the build time significantly. Also local integration tests time reduced effectively from 6 minutes 50seconds to 0 minutes 20 seconds, speeding up the development time for the developer.

Time taken by all integration tests before HSQL

 

 

 

 

Time taken by all integration tests using HSQL

 

 

 

 

So with this new feature in our project, adding and debugging integration tests was a quick and painless process. With this post I would like to spread the word for HSQL support in Spring Framework projects to make integration tests faster and help people increase their productivity.