Why my date formatters test fails in CI and passes locally?

The issue

For new Android project I decided to go with Github Actions as CI environment – we use Github across organization and some backend projects are already running in Github workflows, creating new project with is a good idea.

I’ve prepared configuration using following tutorial:

Slightly modified it, added some steps:

name: Buildon: [pull_request, push]jobs:  build:    runs-on: ubuntu-latest    steps:      - name: Checkout the code        uses: actions/checkout@v2      - name: Build the b2c app        run: ./gradlew testDebugUnitTest      - name: Publish Unit Test Results        uses: EnricoMi/publish-unit-test-result-action@v1.9        if: always()        with:          files: app/build/test-results/**/*.xml      - name: Detekt check        run: ./gradlew detekt

Checkout code, run unit tests, publish test results, run static code analysis.

I wanted to unit test some component – responsible for displaying few fields of date time. I made use of Joda-Time formatting to achieve desired behavior:

data class CalendarDisplayable(    val dateTime: DateTime) {    val dayOfMonth: String        get() = dateTime.toString("d")    val month: String        get() = dateTime.toString("MMM")    val dayAndHour: String        get() = dateTime.toString("EEE, HH:mm")

The app is designed for Polish market – so I wanted to assert against Polish string literals in tests:

"it should format day properly"{        val givenDateTime = DateTime("2021-02-16T11:00:14.012+01:00")        val displayable = CalendarDisplayable(            dateTime = givenDateTime        )        assertSoftly {            displayable.dayOfMonth shouldBe "16"            displayable.month shouldBe "lut"            displayable.dayAndHour shouldBe "Wt, 11:00"        }    }

I knew that I have to somehow set proper timezone offsets and default locale, so I added the following to the test configuration:

    override fun beforeTest(testCase: TestCase) {        Locale.setDefault(Locale.forLanguageTag("pl-PL"))        DateTimeZone.setDefault(DateTimeZone.forOffsetHours(1))        super.beforeTest(testCase)    }

Well, it passed. Checked for false-positives by setting different offset on DateTimeZone and by setting en-US locale – I’m good to go. Nothing to worry, I’m now ready to proceed with other stuff.

Adding few more commits and pushing to remote git repo. Test is failing. But why? I already checked time zone offsets and locale. Let’s take a look at test report on CI:

org.opentest4j.AssertionFailedError: expected: "Wt, 11:00" but was: "wt., 11:00"

Again, what the hell is going on?

I double-checked on my local machine that tests is passing by running it via Android Studio and by invoking gradle test from command line. It’s passing. Well. I’m in trouble now.

The debugging process

I started to discuss the issue with colleagues. Does it matter that I have OSX? Is locale pl-PL a universal notation for Polish locale?

Let’s point out major differences between my local setup and remote machine:

on Github Action there is ubuntu-latest defined
my local environment is OSX
locally I have JDK8
and on remote? Who knows? Let’s check.

I’ve added simple step to my workflow:

      - name: Print Java version        run: javac -version

And observed what this step will print.

Well. That should be my answer. Locally I have java 8, on remote machine there is java 11.

So it turns out that Java 11 is default on ubuntu-latest image!

The solution

Should I upgrade locally to JDK 11, or downgrade Java version on CI? I don’t have too much confidence about building Android project with 11, so decided to downgrade CI to use Java 8.

I’ve added the following step to my Github Action:

      - name: Set up JDK 1.8        uses: actions/setup-java@v1        with:          java-version: 1.8      - name: Print Java version        run: javac -version

Pushed the code, checked the whole pipeline… And it’s passing. What a relief.

Key takeaways

The issue is somehow fixed, but:

Which formatting is valid? „wt.” or „Wt” (shortcut od Wtorek – Tuesday)? One thing is how designer designed it, second how we write shortcuts in Polish and third, how other platforms (web and iOS) will handle it.

This change (Wt -> wt.) is actually good change since it is now compliant with Polish language rules.

Is using Joda-Time still worth it, when there is improved time package in Java8 and kotlinx-datetime library? Maybe it’s time to leave library that I’m used to and introduce modern replacement?
Is setting Locale.setDefault and DateTimeZone.setDefault in unit test reasonable? Or maybe I should introduce additional layer of abstraction and provide test doubles?

What changed in Java?

So it turns out that core-libs in Java 9 has changed. Now it uses CLDR standard by default. Which changed how the day of the week is displayed in Polish.

Testing
Locale-sensitive services such as date, time, and number formatting may behave differently for locales not supported in JDK 8. Existing tests and applications will need to be modified.

Basically many people encountered similar issues when upgrading Java version. In Android world I am happy for now using good old Java 8.

What can change in Android?

While I solved problem in unit tests, this does not mean I have full confidence how this formatting will behave on device, on different Android API versions.

That kind of verification is good candidate for instrumented test. But thats topic for other article, which I may write in the future.

How I didn’t encounter that issue sooner?

Java 9 with that change is here since 2017. I never had to use Java higher than 8 to build Android apps. And backend applications? Well, on backend I always kept date time encoded in ISO8601 and let frontend client decide how it should be displayed.

Ready to take your software skills to the next level? Schedule consultation with me.