👋 Hello, a colleague of mine (all cred to @Albin) just discovered a pretty bad bug with Konsist in our CI. I've also been able to reproduce the issue on the sample projects. We basically had failing tests that was running with successful outcome. Steps to reproduce: 1. Run the project

konsist-starter-android-gradle-kotlin-junit-4

(Other probably work too but this was the most similar to our setup) 2. Run the konsist tests:

./gradlew konsist_test:test

, see successful result 3. Modify

app/src/main/java/com/sample/MainActivity.kt

and change class name to

MainActivit

4. Run the konsist tests again:

./gradlew konsist_test:test

, see successful result 👀 My full output:

rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) ./gradlew konsist_test:test

BUILD SUCCESSFUL in 4s
26 actionable tasks: 6 executed, 20 up-to-date
rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) vim app/src/main/java/com/sample/MainActivity.kt
rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) ✗ ./gradlew konsist_test:test

BUILD SUCCESSFUL in 447ms
26 actionable tasks: 26 up-to-date
rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) ✗

It seems like it does not care to refresh any cache, I'm not an expert on the build system. But creating an explicit dependency on the project will trigger a rebuild of

:app

module, and then the project will be recompile and tests fail correctly. In

konsist_test/build.gradle.kts

add dependency:

implementation(project(":app"))

I’ve seen the same thing, it’s not really a bug I’d say, it’s just that if you have konsist tests in a separate module, and you didn’t change anything in that module then Gradle will consider the “unit tests” up to date, as it doesn’t know they are in fact testing other modules. You can add

--rerun

to the Gradle command to make it run anyway.

Ohh now I see

Yeah just use

./gradlew konsist_test:test --rerun

to prevent Gradle thinking it’s up to date.

does not work for me

What’s the result?

Konsist Tess are passing

Do you know if they are being run, or skipped by Gradle?

I got it

./gradlew konsist_test:test --rerun-tasks

is a way to go

Our CI is currently setup to use the cache to improve some speed, this is why we found this issue. So it is cache related.

I wonder if there is any flag that can be added to gradle config to always treat tests in this module as invalid 🤔