Parameterized tests with Kotest

We make use of parameterized test when we want to check system under test against various inputs while using same or similar test logic.

Real life example for parameterized test may be unit conversion – meters to kilometers converter for the sake of UI display. In some place in an application (either web or mobile) we want to display distance value, but in more user friendly way.

Test arguments:

Let’s write basic implementation of that converter:

class DistanceConverter{    fun toKilometer(meters: Meter): Kilometer{        return meters.value.div(1000.0).let(::Kilometer)    }}inline class Meter(val value: Long)inline class Kilometer(val value: Double)

Note – implementation is not complete for purpose (it does not round up or down values), so we could see how failing tests are reported

Method 1: data.forAll()

First option, suggested in Kotest is data driven testing. In io.kotest.data package we can find forAll() method which accepts Table consisting of table headers and rows.

• table row will accept up to 22 parameters (API limitation)
• test params are available in test lambda directly
• additionally, we can provide table header
• by default, table header would be inferred by reflection from param names:

import io.kotest.core.spec.style.StringSpecimport io.kotest.data.headersimport io.kotest.data.rowimport io.kotest.data.tableimport io.kotest.matchers.shouldBeclass DistanceConverterTest : StringSpec({    val converter = DistanceConverter()    "it should convert meters to kilometers (data.forAll)"{        io.kotest.data.forAll(            table(                headers("meters", "expected kilometers"),                row(Meter(2000L), Kilometer(2.0)),                row(Meter(2100L), Kilometer(2.1)),                row(Meter(3999L), Kilometer(3.99)),                row(Meter(333L), Kilometer(0.33))            )        ) { meters: Meter, kilometers: Kilometer ->            converter.toKilometer(meters) shouldBe kilometers        }    }})

Method 2: inspectors.forAll()

Collections<>.forAll will perform one test on collection elements and will gather the result into assertion error. Here we will use listOf<Pair<Meter, Kilometer>> as test input:

import io.kotest.core.spec.style.StringSpecimport io.kotest.inspectors.forAllimport io.kotest.matchers.shouldBeclass DistanceConverterTest : StringSpec({    val converter = DistanceConverter()    "it should convert meters to kilometers (inspectors.forAll)"{        listOf(            Meter(2000L) to Kilometer(2.0),            Meter(2100L) to Kilometer(2.1),            Meter(3999L) to Kilometer(3.99),            Meter(333L) to Kilometer(0.33)        ).forAll { (meters, expectedKilometers) ->            converter.toKilometer(meters) shouldBe expectedKilometers        }    }})

• by default, 10 passed elements and 10 failed elements would be displayed in test log
• can be run on any Collection<T>
• test params should be unwrapped (use it or destructing declaration)

Method 3: Generating test cases with FreeSpec

If for some reason built-in inspector or data methods are not sufficient for you, you may try to design your own parameterized test using standard collections and FreeSpec:

import io.kotest.core.spec.style.FreeSpecimport io.kotest.matchers.shouldBeclass DistanceConverterTest : FreeSpec({    val converter = DistanceConverter()    "convert meters to kilometers" - {        listOf(            Meter(2000L) to Kilometer(2.0),            Meter(2100L) to Kilometer(2.1),            Meter(3999L) to Kilometer(3.99),            Meter(333L) to Kilometer(0.33)        ).forEach { (meters: Meter, expectedKilometers: Kilometer) ->            "it should convert $meters to $expectedKilometers"{                converter.toKilometer(meters) shouldBe expectedKilometers            }        }    }})

With FreeSpec it is possible to generate custom parameterized test cases – with the following notation one can create test group:

import io.kotest.core.spec.style.FreeSpecclass DistanceConverterTest : FreeSpec({    "convert meters to kilometers" - {        ...    }})

And then inside put test cases:

import io.kotest.core.spec.style.FreeSpecclass DistanceConverterTest : FreeSpec({    "convert meters to kilometers" - {        listOf(...).forEach { (a,b) ->            "it should convert $a to $b"{                // assertion            }        }    }})

Few thoughts  

• inspectors.forAll() may be run on any collection
• data.forAll() will accept vararg of row as test params
• keep type safety while working with similar types in Kotlin – use typealiase or inline class to increase readability
• design your parameterized test to report assertion errors in readable way – you may use FreeSpec to provide custom name for test
• make use of built-in methods – sometimes it may be convenient to write Table<> for your test parameters
• spend some time to discover inspectors and matchers APIs – methods that you need may already be there!

See also

Parametrized tests with Spek

Using Spek to generate test cases

Kotlin TestingJarosław Michalik

kotest/kotest

Powerful, elegant and flexible test framework for Kotlin – kotest/kotest

GitHubkotest