In Junit based testing framework there are few possible test outcomes: passed, failed and crashed. In this short note we will dive into that mechanisms.
Test will pass, if no exception is thrown. Empty test should always pass:
class TestFlowShowcase : StringSpec({ "it should pass"{ // nothing here }})Or test with assertion that passes:
class TestFlowShowcase : StringSpec({ "1 == 1"{ (1 == 1) shouldBe true }})Test will fail, if AssertionException (or descendant) would be thrown:
class TestFlowShowcase : StringSpec({ "it should fail"{ throw AssertionError("fail") }})Test will crash, if other exception would be thrown:
class TestFlowShowcase : StringSpec({ "it should crash"{ throw Exception("it crashed") }})
How to treat different test outcomes?
| Passed | No error was thrown |
| Failed | AssertionError was thrown in test |
| Crashed | Other exception was thrown |
„Green” test means only that no error was thrown during test execution – it may also mean that no assertion was performed.
Usually test failed means that something is wrong with system under test – it doesn’t meet expectations defined in assertions.
Test crash indicate that something is wrong with test itself – it does not handle exception thrown in system under test, or its mocks are misconfigured.