:wave: Hello, a colleague of mine (all cred to <@U...

# konsist

[David]

👋 Hello, a colleague of mine (all cred to @Albin) just discovered a pretty bad bug with Konsist in our CI. I've also been able to reproduce the issue on the sample projects. We basically had failing tests that was running with successful outcome.

Steps to reproduce:
1. Run the project konsist-starter-android-gradle-kotlin-junit-4 (Other probably work too but this was the most similar to our setup)
2. Run the konsist tests: ./gradlew konsist_test:test , see successful result
3. Modify app/src/main/java/com/sample/MainActivity.kt and change class name to MainActivit
4. Run the konsist tests again: ./gradlew konsist_test:test , see successful result 👀
My full output:

rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) ./gradlew konsist_test:test

BUILD SUCCESSFUL in 4s
26 actionable tasks: 6 executed, 20 up-to-date
rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) vim app/src/main/java/com/sample/MainActivity.kt
rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) ✗ ./gradlew konsist_test:test

BUILD SUCCESSFUL in 447ms
26 actionable tasks: 26 up-to-date
rawa@Davids-MacBook-Pro ➜  konsist-starter-android-gradle-kotlin-junit-4 git:(main) ✗

It seems like it does not care to refresh any cache, I'm not an expert on the build system. But creating an explicit dependency on the project will trigger a rebuild of :app module, and then the project will be recompile and tests fail correctly.
In konsist_test/build.gradle.kts add dependency:

implementation(project(":app"))

[David]

@igor.wojda ☝️ Hope you don't mind me pinging you.

[Marcin Wisniowski]

I’ve seen the same thing, it’s not really a bug I’d say, it’s just that if you have konsist tests in a separate module, and you didn’t change anything in that module then Gradle will consider the “unit tests” up to date, as it doesn’t know they are in fact testing other modules.

You can add --rerun to the Gradle command to make it run anyway.

[igor.wojda]

Ohh now I see

[Marcin Wisniowski]

Yeah just use ./gradlew konsist_test:test --rerun to prevent Gradle thinking it’s up to date.

[igor.wojda]

does not work for me

[Marcin Wisniowski]

What’s the result?

[igor.wojda]

Konsist Tess are passing

[Marcin Wisniowski]

Do you know if they are being run, or skipped by Gradle?

[igor.wojda]

I got it
./gradlew konsist_test:test --rerun-tasks is a way to go

[igor.wojda]

I will update docs

[igor.wojda]

thx for input @Marcin Wisniowski (I would not have figure this one out quickly without you)

[igor.wojda]

BTW I am still curios why this fails on CI for you @David - I thin kCI should use most up to date version ideally without any caches, otherwise you will see problem like this (I guess they may happen locally, but CI should be bullet proof, so I always prefer to do a proper clean/fresh run)

[David]

Our CI is currently setup to use the cache to improve some speed, this is why we found this issue. So it is cache related.

[igor.wojda]

I wonder if there is any flag that can be added to gradle config to always treat tests in this module as invalid 🤔

[igor.wojda]

FYI: I have started a discussion on Gradle Slack
https://gradle-community.slack.com/archives/CAHSN3LDN/p1697195314443339