Unit Testing

Verifying Functional Correctness

Randy J. Fortier
randy.fortier@uoit.ca
@randy_fortier

Outline

  • Unit Testing
  • Unit Testing Frameworks
  • jUnit
    • A Simple Example
    • Assertions
    • Scaffolding Methods
    • Running jUnit from the command line
    • Running jUnit from Gradle
  • Test-Driven Development
    • Code Coverage

Unit Testing

What is unit testing?

What is Unit Testing?

  • Testing each modularity unit in isolation
    • Does the actual result match the expected result?
    • Test each unique possibility
    • e.g. For sorting a list of numbers
      • A randomized list
      • An empty list
      • An already sorted list
      • A reverse sorted list

Other Kinds of Testing

  • Unit testing
    • Tests that the class/method works as expected
  • Black box (functional) testing
    • Tests that the software works as expected
  • Acceptance testing
    • Ensures that customer requirements are fulfilled
  • Regression testing
    • Ensures that previous functionality that worked continues to work after a change
  • Performance testing
    • Verify that the software works at the expected speed
  • Load testing
    • Test the behaviour of the software under heavy loads

Unit Testing Frameworks

Automating Unit Tests

Who Needs a Framework?

  • Disclaimer: A framework is not strictly necessary
  • Unit testing frameworks are actually quite simple
    • You could easily write your own

Unit Testing Frameworks

  • Java:
    • jUnit
    • TestNG
    • JTest
    • Mockito (a object mocker)
  • Other platforms:
    • Catch, Boost.Test, Google Test (C++)
    • PyTest, Nose, PyUnit (Python)
    • Mocha, Jasmine (JavaScript)
    • NUnit, XUnit.NET, CSUnit (.NET)
  • Note: Many unit test frameworks follow the xUnit pattern
    • Thus, learning one framework translates well to others

jUnit

Java's Most Popular Unit Testing Framework

jUnit

  • JUnit is free and open-source
  • Tests are written as methods in a Java class
    • Annotations are used to denote methods and classes as tests and test suites
      • For those unfamiliar, annotations are similar to meta-data for classes and methods

A Simple jUnit Test Class

  • An assertion verifies that some condition is true
    • Any assertions that fail are recorded as failed tests
    • A given test may have multiple assertions

import static org.junit.jupiter.api.Assertions.*;

import org.junit.jupiter.api.*;

public class BasicTest {
	@Test
	public void simpleTest() {
		assertEquals(1, 1, "1 should be equal to 1");
	}
}
	

jUnit Assertions

  • Assertions available in jUnit:
    • assertEquals - fails if two provided values are not equal
      • One version, for floating points, checks if the two values are close enough (by an arbitrary delta)
    • assertSame - fails if two provided object references do not reference the same object
    • assertFalse - fails if the provided value is not false
    • assertTrue - fails if the provided value is not true
    • assertNull - fails if the provided value is not null
    • assertNotNull - fails if the provided value is null
    • assertTimeout - fails if the lambda does not execute within the specified timeout duration
      • assertTimeoutPreemtively - fails and stops the lambda if the lambda does not execute within the specified timeout duration
    • fail - always fails
      • Usually used in combination with a conditional to perform a manul assertion

jUnit Annotations

  • Here are the basic annotations available in jUnit:
    • @Test - specifies that a method is to be executed as a test
    • @DisplayName - specifies the human-recognizable name of the test
    • @Tag - used to attach one or more keywords to a test
      • Tags can be used to filter tests
      • e.g. 'fast' versus 'slow'
      • e.g. 'performance'
    • @Disabled - used when you want to keep test code, but not execute it (right now)

jUnit Fixtures

  • A fixture is a data item that is used for testing
  • The process works like this:
    • A set of known data values (fixtures) are set up
      • These values are the data on which the test will execute
    • The test is run, and succeeds, fails, or generates an error
    • A number of other tests can also be run here (using the same fixture)
    • The fixtures are cleaned up (torn down)

Fixture Setup

  • jUnit annotations related to setting up/tearing down fixtures:
    • @BeforeEach - a method that shoud be called before each test
    • @BeforeAll - a method that should be called before all tests
    • @AfterEach - a method that should be called after each test
    • @AfterAll - a method that should be called after all tests

Fixture Setup

  • These annotations make it easy to setup one or more shared fixtures:

@BeforeAll
public static void setUp() {
	// setup any shared fixture(s)
}

@AfterAll
public static void tearDown() {
	// free up any resources from the shared fixture(s)
}
	

Fixture Setup

  • These annotations also make it easy to setup one or more fixtures for one-time use (per test):

@BeforeEach
public static void setUp() {
	// setup any one-time use fixture(s)
}

@AfterEach
public static void tearDown() {
	// free up any resources from the one-time use fixture(s)
}
	

TestInfo

  • Any test method can optionally take a TestInfo argument
  • This object contains information about the test, such as:
    • getDisplayName() - returns the display name of this test
    • getTags() - returns a set of tag keywords for this test
    • getTestClass() - returns the Class of this test
    • getTestMethod() - returns the Method of this test

Running jUnit Tests from the Command Line

  • jUnit provides a separate command-line tool

$ java -cp "lib/*" org.junit.platform.console.ConsoleLauncher
       -cp "build/classes/test:build/classes/main" --scan-class-path
	

Running jUnit Tests from Gradle

  • jUnit provides a Gradle plug-in
  • To enable the plug-in:

buildscript {
  repositories {
    mavenCentral()
  }
  dependencies {
    classpath 'org.junit.platform:junit-platform-gradle-plugin:1.0.0-M3'
  }
}

apply plugin: 'java'
apply plugin: 'org.junit.platform.gradle.plugin'
	

Running jUnit Tests from Gradle

  • To configure jUnit:

junitPlatform {
  filters {
    engines {
      include 'junit-jupiter'
    }
    tags {
      exclude 'slow'
    }
  }
}

ext.junitJupiterVersion  = '5.0.0-M3'
dependencies {
  testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")
  testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")
}
	

Test-Driven Development

TDD

Test-Driven Development

  • With TDD, tests are usually created before the code is written
  • Here is a typical scenario:
    1. Decide on a small behaviour to implement
    2. Write a test to ensure that behaviour is present
    3. Run the test to ensure it fails
    4. Write the code to make the test pass
    5. Refactor the code to be simple and fit with the overall design
    6. Run the test to verify that it passes
  • This new test becomes part of a test suite, run before each release

Test-Driven Development

  • A slightly different scenario happens when a bug is discovered:
    1. Learn about a bug from a customer or QA team member
    2. Write a test to ensure that the bug is fixed
    3. Run the test to ensure it fails
    4. Re-write the code to make the test pass
    5. Refactor the code to be simple and fit with the overall design, if necessary
    6. Run the test to verify that it passes
  • This new test will be added to the list of tests that will be run before each release
    • This prevents the re-introduction of bugs
    • This is called regression testing

Test-Driven Development

  • The basic cycle, illustrated:

Too Many Tests

  • In a large project, you end up with many tests
    • Some of these tests (or all of them, combined), may take a long time to execute
    • Executing all tests every build is generally not practical
  • Most unit testing frameworks will make it easy to enable/disable tests
  • In jUnit:
    • You can explicitly disable tests (or a whole class of tests) with @Disable
    • You can @Tag tests into arbitrary categories, and disable all tests with that tag
  • Most projects will have:
    • Small test suites, concentrated on the code being modified
    • More comprehensive test suites for testing release candidates

Test Coverage

  • Testing a project will only find bugs if you are testing all code
    • Testing 100% of code (100% code coverage) is unrealistic for most projects
    • Certain parts of the program, however, may require 100% code coverage
      • e.g. The Linux Kernel
  • There are many tools that can test your code coverage
    • For Java
      • Jacoco
        • Has a Gradle plug-in
        • Integrates into IntelliJ IDEA and Eclipse
      • JCov
      • Clover

Wrap-Up

  • In this section we learned about:
    • Unit Testing
    • jUnit
    • Test-Driven Development