Using Testcontainers for Better Integration Tests - NashTech Insights (2024)

• Application Engineering
• August 7, 2024

Table of Contents

Table of Contents

It’s no secret that software applications rarely operate in isolation. They need to communicate to several external systems like databases, messaging systems, cache providers, and many 3rd party services. And it’s up to you to ensure everything functions correctly. That why testing is one of the important parts in software development.

There are numerous types of software testing techniques that you can use to ensure changes to your code work as expected. Not all testing is equal though, and we explore how some testing practices differ. But Unit tests and Integration tests are common and required for all of software applications.

Unit tests are very low level and close to the source of an application. They consist in testing individual methods and functions of the classes, components, or modules used by your software. Unit tests are generally quite cheap to automate and can run very quickly by a continuous integration server.

Integration tests verify that different modules or services used by your application work well together. For example, it can be testing the interaction with the database or making sure that microservices work together as expected. These types of tests are more expensive to run as they require multiple parts of the application to be up and running.

However, for some reason, testing will be hindered by the dependencies or real services. For example, when testing database access logic, a popularapproach is to use in-memory databases like H2. However, this will make the test results unreliable because the In-Memory Database is completely different from the Production Database. Fortunately, we have an alternative: testcontainers.

What is Testcontainers?

Testcontainers is a library that provides easy and lightweight APIs for bootstrapping local development and test dependencies with real services wrapped in Docker containers. Using Testcontainers, you can write tests that depend on the same services you use in production without mocks or in-memory services.

An Example

Let’s assume we like to write the HTTP resource/employees. This resource retrieves the employee data from a database, requests tax information from a remote service and executes some payroll calculation logic. Our class composition might look like this:

Integration Tests

Some popular approaches are to use in-memory databases or mock-based for integration test. But it is not good approach.Test against thereal database (via testcontainers) instead of an in-memory-databaseto be even closer to production.

• We write a single integration test which tests all four classes together. So we only have oneEmployeeControllerIntegrationTestthat tests theEmployeeControllerwhich is wired together with the realEmployeeDAO,TaxServiceClientand thePayrollCalculator.
• We start the production database in a docker container and configure the wiredEmployeeDAOwith the container’s address. The libraryTestcontainersprovides an awesome Java API for managing container directly in the test code.
• The responses of the remote tax service are the only thing left which have to be mocked.

What’s benefit?

• Accurate, meaningful and production-close tests.
• Easy setup and execution.
• On-demand isolated infrastructure provisioning
• Advanced networking capabilities
• Supports a wide range of databases, message brokers, and other services, and you can also use custom Docker images.
• Can be easily integrated into CI/CD pipelines.
• Supports parallel execution of tests by providing isolated containers for each test.

Drawbacks

• Execution speed.Starting and stopping Docker containers can be time-consuming, especially if the containers are large or if there are many of them.
• Resource Consumption.Running Docker containers consumes system resources such as CPU, memory, and disk space.
• Version Compatibility.Ensuring compatibility between different versions of Docker, Testcontainers, and the services being containerized can be challenging.

Let take a look at source code

@TestInstance(TestInstance.Lifecycle.PER_CLASS)public class EmployeeControllerIntegrationTest { private MockWebServer taxService; private JdbcTemplate template; private MockMvc client; @BeforeAll public void setup() throws IOException { // EmployeeDAO PostgreSQLContainer db = new PostgreSQLContainer("postgres:11.2-alpine"); db.start(); DataSource dataSource = DataSourceBuilder.create() .driverClassName("org.postgresql.Driver") .username(db.getUsername()) .password(db.getPassword()) .url(db.getJdbcUrl()) .build(); this.template = new JdbcTemplate(dataSource); SchemaCreator.createSchema(template); EmployeeDAO dao = new EmployeeDAO(template); // TaxServiceClient this.taxService = new MockWebServer(); taxService.start(); TaxServiceClient client = new TaxServiceClient(taxService.url("").toString()); // PayrollCalculator PayrollCalculator calculator = new PayrollCalculator(); // EmployeeController EmployeeController controller = new EmployeeController(dao, client, calculator); this.client = MockMvcBuilders.standaloneSetup(controller).build(); } @Test public void databaseDataIsCorrectlyReturned() throws Exception { insertIntoDatabase( new EmployeeEntity().setId("2").setName("John"), new EmployeeEntity().setId("3").setName("Mary") ); taxService.enqueue(new MockResponse() .setResponseCode(200) .setBody(toJson(new TaxServiceResponseDTO(Locale.ENGLISH, 0.19))) ); String responseJson = client.perform(get("/employees")) .andExpect(status().is(200)) .andReturn().getResponse().getContentAsString(); assertThat(toDTOs(responseJson)).containsOnly( new EmployeeDTO().setId("2").setName("John"), new EmployeeDTO().setId("3").setName("Mary") ); }}

Apart from this example we have a project called YAS which is implementing many good examples related to Testcontainers that you can refer to.You can refer to YAS here

CI integration with TestContainers

To use Testcontainers in your CI/CD environment, you only require Docker installed. A local installation of Docker is not mandatory; you can also use a remote Docker installation. We can take the YAS example to demo CI integration.

name: inventory service cion: push: branches: ["main"] paths: - "inventory/**" - ".github/workflows/actions/action.yaml" - ".github/workflows/inventory-ci.yaml" - "pom.xml" pull_request: branches: ["main"] paths: - "inventory/**" - ".github/workflows/actions/action.yaml" - ".github/workflows/inventory-ci.yaml" - "pom.xml" workflow_dispatch:jobs: Build: runs-on: ubuntu-latest env: FROM_ORIGINAL_REPOSITORY: ${{ github.event.pull_request.head.repo.full_name == github.repository || github.ref == 'refs/heads/main' }} steps: - uses: actions/checkout@v4 with: fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis - uses: ./.github/workflows/actions - name: Run Maven Build Command run: mvn clean install -DskipTests -f inventory - name: Run Maven Unit Test run: mvn test -f inventory - name: Run Maven Integration Test run: mvn test -Pintegration-test -f inventory - name: Unit Test Results uses: dorny/test-reporter@v1 if: ${{ env.FROM_ORIGINAL_REPOSITORY == 'true' && (success() || failure()) }} with: name: Inventory-Service-Unit-Test-Results path: "inventory/**/surefire-reports/*.xml" reporter: java-junit - name: OWASP Dependency Check uses: dependency-check/Dependency-Check_Action@main env: JAVA_HOME: /opt/jdk with: project: 'yas' path: '.' format: 'HTML' - name: Upload OWASP Dependency Check results uses: actions/upload-artifact@master with: name: OWASP Dependency Check Report path: ${{github.workspace}}/reports - name: Analyze with sonar cloud if: ${{ env.FROM_ORIGINAL_REPOSITORY == 'true' }} env: SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} run: mvn org.sonarsource.scanner.maven:sonar-maven-plugin:sonar -f inventory - name: Log in to the Container registry if: ${{ github.ref == 'refs/heads/main' }} uses: docker/login-action@v3 with: registry: ghcr.io username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Build and push Docker images if: ${{ github.ref == 'refs/heads/main' }} uses: docker/build-push-action@v6 with: context: ./inventory push: true tags: ghcr.io/nashtech-garage/yas-inventory:latest

Here is result:

Conclustion

Testcontainers offers a robust and flexible solution for integration testing in Java applications by leveraging Docker containers. It provides a realistic testing environment, ensuring tests interact with actual instances of dependencies like databases and message brokers. This results in more accurate and reliable tests compared to using mocks or in-memory substitutes. Key benefits include ease of use, isolation of test environments, automatic cleanup, and seamless integration with CI/CD pipelines. While there are some challenges, such as dependency on Docker and potential performance overhead, the advantages of Testcontainers—such as improved test accuracy, environment parity, and support from a strong community—make it a valuable tool for ensuring high-quality software. By addressing integration issues early and maintaining consistent test environments, Testcontainers helps developers build more reliable and maintainable applications.

Suggested Article