Sorry if this was asked before but searching didn t yield an kotlinlang #gradle

Sorry if this was asked before, but searching didn...

Tom Yuval

08/22/2022, 2:04 PM

Sorry if this was asked before, but searching didn’t yield anything meaningful: When some of my tests fail and I try to detect that using Gradle (e.g.

./gradlew allTests

./gradlew check

), this doesn’t work when I’ve already run the tests before. What happens is that the first time I run

gradlew

it indeed fails because of the failing tests, but then if I run it a second time it doesn’t run the tests again as the tasks are already “up-to-date” – but it also doesn’t fail (as if the tasks being up-to-date is a good enough reason to pass, even though the tasks’ up-to-date state is that they’re failing). Is there some way to tell Gradle to fail when the “up-to-date” tasks it doesn’t want to run again failed the last time? Clarification: I’m not asking how to make Gradle rerun the tests – if nothing’s changed since the last run then I’m happy with Gradle considering everything up-to-date and not wasting time re-executing tasks. I just want it to fail (= return with a non-zero exit code) in that state, like it did upon the first run of the failing tests, and not “pretend” that everything’s OK.

Sam

08/22/2022, 2:07 PM

I don’t think the behaviour you’re describing should be the default. If a test task fails it’s not supposed to be cached. Have you customised the test task at all, e.g. by settings something like

ignoreFailure = true

Vampire

08/22/2022, 2:23 PM

If he would have set that, then it would also have failed the first time. But what Sam said. If a task failed last time you executed it and you run the task again, it will run again. I'm not even aware of any setting to change this. Can you please provide an MCVE that demonstrates this behavior?

Tom Yuval

08/22/2022, 2:26 PM

Thanks, @Sam and @Vampire. I have not done any customisation. I’ve just reproduced the behaviour with a totally fresh project (on Kotlin 1.7.10, with Gradle 7.4.2, using IntelliJ’s wizard to create a multi-platform library with the default configuration taken as is). I’ve got a single test that calls

fail

, and that’s it. Running

./gradlew check

fails because of the failing test, but rerunning it exits with no error.

Vampire

08/22/2022, 2:26 PM

Again, can you provide that MCVE?

Tom Yuval

08/22/2022, 2:29 PM

`build.gradle.kts`:

Copy code

plugins {
    kotlin("multiplatform") version "1.7.10"
}

group = "com.example"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
}

kotlin {
    jvm {
        compilations.all {
            kotlinOptions.jvmTarget = "1.8"
        }
        withJava()
        testRuns["test"].executionTask.configure {
            useJUnitPlatform()
        }
    }
    js(IR) {
        browser {
            commonWebpackConfig {
                cssSupport.enabled = true
            }
        }
    }
    val hostOs = System.getProperty("os.name")
    val isMingwX64 = hostOs.startsWith("Windows")
    val nativeTarget = when {
        hostOs == "Mac OS X" -> macosX64("native")
        hostOs == "Linux" -> linuxX64("native")
        isMingwX64 -> mingwX64("native")
        else -> throw GradleException("Host OS is not supported in Kotlin/Native.")
    }


    sourceSets {
        val commonMain by getting
        val commonTest by getting {
            dependencies {
                implementation(kotlin("test"))
            }
        }
        val jvmMain by getting
        val jvmTest by getting
        val jsMain by getting
        val jsTest by getting
        val nativeMain by getting
        val nativeTest by getting
    }
}

`src/commonTest/kotlin/T.kt`:

Copy code

import kotlin.test.Test
import kotlin.test.fail

class T {
    @Test fun t() { fail("Supposed to fail") }
}

Tom Yuval

08/22/2022, 2:33 PM

Output of first invocation of `./gradlew check`:

Copy code

> ./gradlew check

> Configure project :
Kotlin Multiplatform Projects are an Alpha feature. See: <https://kotlinlang.org/docs/reference/evolution/components-stability.html>. To hide this message, add 'kotlin.mpp.stability.nowarn=true' to the Gradle properties.

The property 'kotlin.mpp.enableGranularSourceSetsMetadata=true' has no effect in this and future Kotlin versions, as Hierarchical Structures support is now enabled by default. It is safe to remove the property.

The property 'kotlin.native.enableDependencyPropagation=false' has no effect in this and future Kotlin versions, as Kotlin/Native dependency commonization is now enabled by default. It is safe to remove the property.


> Task :jvmTest

T[jvm] > t()[jvm] FAILED
    org.opentest4j.AssertionFailedError at T.kt:8

1 test completed, 1 failed
There were failing tests

> Task :jsBrowserTest

T.t FAILED
    AssertionError at /var/folders/5r/jzhxz6rd7fz236lzbs9sm7m00000gn/T/_karma_webpack_670497/commons.js:17012

1 test completed, 1 failed
There were failing tests

> Task :nativeTest

T.t FAILED
    kotlin.AssertionError at /opt/buildAgent/work/67fbc2b507315583/kotlin/kotlin-native/runtime/src/main/kotlin/kotlin/Throwable.kt:24

1 test completed, 1 failed
There were failing tests

> Task :allTests FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':allTests'.
> There were failing tests. See the report at: file:///path/to/project/build/reports/tests/allTests/index.html

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at <https://help.gradle.org>

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See <https://docs.gradle.org/7.4.2/userguide/command_line_interface.html#sec:command_line_warnings>

BUILD FAILED in 10s
22 actionable tasks: 12 executed, 10 up-to-date

Tom Yuval

08/22/2022, 2:34 PM

Output of second invocation of

./gradlew check

(with nothing else done in between):

Copy code

> ./gradlew check

> Configure project :
Kotlin Multiplatform Projects are an Alpha feature. See: <https://kotlinlang.org/docs/reference/evolution/components-stability.html>. To hide this message, add 'kotlin.mpp.stability.nowarn=true' to the Gradle properties.

The property 'kotlin.mpp.enableGranularSourceSetsMetadata=true' has no effect in this and future Kotlin versions, as Hierarchical Structures support is now enabled by default. It is safe to remove the property.

The property 'kotlin.native.enableDependencyPropagation=false' has no effect in this and future Kotlin versions, as Kotlin/Native dependency commonization is now enabled by default. It is safe to remove the property.


Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See <https://docs.gradle.org/7.4.2/userguide/command_line_interface.html#sec:command_line_warnings>

BUILD SUCCESSFUL in 1s
22 actionable tasks: 2 executed, 20 up-to-date

Tom Yuval

08/22/2022, 2:37 PM

(@Vampire, is that enough? I’m not very experienced in the

kotlinlang

Slack and don’t know if there’s a better way to supply MCVEs.)

Vampire

08/22/2022, 2:53 PM

That's not an MCVE, it is far from being *C*omplete. Either build a ZIP with your project that you attach here if possible or through some service like swisstransfer.ch, or push it to a new GitHub project.

Tom Yuval

08/22/2022, 4:04 PM

Thanks, @Vampire. Please take a look here: https://github.com/tomyuval/gradle-not-re-running-failed-tests.

Vampire

08/22/2022, 11:12 PM

I've had a look. If there is a problem, you probably have to bring this up with JetBrains and their Kotlin plugin. But actually, it works fine here. I run

check

three times and every time the tests were executed.

Vampire

08/22/2022, 11:13 PM

Maybe the up-to-date data is corrupted somehow, I see this happening from time to time. Try deleting all folders in

<GRADLE_USER_HOME>

and the

.gradle

in your project and test again.

Tom Yuval

08/23/2022, 11:38 AM

Thanks, @Vampire. I tried cloning the repository into another directory (which is akin to deleting

.gradle

, as it is Git-ignored) and the same problem repeated. Deleting

<GRADLE_USER_HOME>

might help, but I’m not sure what that is – do you mean

~/.gradle

Vampire

08/23/2022, 12:05 PM

That is the default Gradle user home, yes

Tom Yuval

08/23/2022, 1:33 PM

I deleted that too – didn’t help!

Vampire

08/23/2022, 1:34 PM

Well, I cannot say more than that it works like expected here.

Tom Yuval

08/23/2022, 1:36 PM

Thanks for trying, @Vampire! I guess there’s some other difference in system/configuration – but I’ve got no idea what that might be.

If there is a problem, you probably have to bring this up with JetBrains and their Kotlin plugin.

Would love to do that! Anyone from JetBrains and their Kotlin plugin around here?

Vampire

08/23/2022, 1:39 PM

Maybe. Otherwise youtrack.jetbrains.com

Tom Yuval

08/23/2022, 1:43 PM

Yeah, will open an issue there (I’m a bit reluctant to do that when my reproduction instructions don’t work on others’ machines, but I may have no other choice, and maybe someone there will know what other parts of the system I should look at).

Tom Yuval

08/26/2022, 1:55 PM

Hey @Vampire, sorry to bother you again with this, but a quick question: when you tried it and got it to fail twice, was the failure actually due to the failed tests? Asking because I’ve asked a couple of colleagues to try on their machines and at first they said it failed twice, but then we noticed the failure was due to other problems (missing Java/JDK or unsupported platform), but once they solved those problems and got it to actually fail because of the tests it didn’t fail anymore on subsequent runs.

Vampire

08/26/2022, 3:33 PM

Pretty sure. It was the "Supposed to fail" assertion.

Vampire

08/26/2022, 3:35 PM

Hm, strange. Actually now when I try it, I see the behavior you describe. 😕

Tom Yuval

08/26/2022, 3:55 PM

Nice! 😉 So I’ll file an issue 👍 But where should I file an issue? Is it a Kotlin thing, not a Gradle thing?

Vampire

08/26/2022, 3:55 PM

No, idea what changed though. But then we are back now at the place where I blame the plugin's configuration of the tasks. The individual test tasks are up-to-date as no inputs or outputs changed and the aggregating

allTests

task is rerun as it failed last time, but now does not fail as it obviously only fails if the individual test tasks had failures.

Vampire

08/26/2022, 3:56 PM

It is a thing of how the Kotlin plugin configures Gradle most probably. Gradle behaves exactly like expected, rerunning the task that failed in the previous run which now runs successful.

Tom Yuval

08/26/2022, 4:01 PM

OK, thanks! 👍

Vampire

08/26/2022, 4:03 PM

Argh, I tested from within IntelliJ. I thought it only makes task always rerun when using "Run test", but not when running an arbitrary Gradle task. But it seems even when running an arbitrary Gradle task IntelliJ injects the init script that makes tests always rerun and thus hide your issue. So it is important to not run it from inside IntelliJ to reproduce.

Tom Yuval

08/26/2022, 5:40 PM

I just opened https://youtrack.jetbrains.com/issue/KT-53738. I see there’s also https://youtrack.jetbrains.com/issue/KT-32150 (over three years old!) which might be the same issue.

14 Views

Open in Slack

Previous Next