I'm having issues with the latest versions of Kote...

# kotest

[Niko]

I'm having issues with the latest versions of Kotest (5.0.2) with IDEA (2021.3) and plugin (1.1.49-IC-213-EAP-SNAPSHOT), and Kotlin 1.6.0:
Some time ago, everything was fine and dandy, but then something changed (it was either IDEA update, me changing the project, or upgrading Gradle or Kotlin version changed), and first I started having problems with running individual tests (like Ross wrote earlier).
Then, again something changed, and now I'm able to run the tests via IDEA's Run <test class>: <test> gutter icon of individual tests launching the Kotest runner panel, but the Run <test class> now simply launches a Gradle build, no tests run.
IDEA's Gradle > project > verification > test runs no tests, nor does running ./gradlew test .

Currently, I have a root project with multiple subprojects (modules), and I'm applying some conventions (like applying Kotest) to all the subprojects using a Gradle plugin. I'm wondering, if this setup has somehow screwed up the discovery

(more in thread)

[Niko]

Also,
• I've removed .gradle and build dirs, no effect
• Running ./gradlew dependencies lists all Kotest deps correctly
This is the part in the plugin, that applies Kotest:

private fun initTesting(project: Project) = project.run {
    <http://logger.info|logger.info>("Setting up testing for the project (kotest ${Versions.kotestVersion})")

    tasks.withType<Test>() {
        useJUnitPlatform()
    }
    // TODO: combine with above?
    val test by tasks.getting(Test::class) {
        // bootstrap Kotest system properties (-Dx=y) to enable targeting test tags
        // via CLI: <https://kotest.io/docs/framework/tags.html>
        systemProperties = System.getProperties()
            .map { it.key.toString() to it.value }.toMap()
        // TODO: once the build script moves forward from 1.4.31, migrate to
        //  .associate {}
    }

    dependencies {
        add("testImplementation", Dependencies.kotestRunner)
        add("testImplementation", Dependencies.kotestAssertionsCore)
    }
}

[sam]

When you run a class does intellij appear to run but just no tests appear?

[Niko]

Correct. Class (StringSpec) > Run '<class>' Ctrl+Shift+F10 > IDEA shows the Run panel with which seems to be a Gradle run config, and not a Kotest one (I'm guessing the Gradle runner should delegate the test execution somehow to Kotest..?)

[Niko]

But when running individual tests via gutter icon, all's as expected:

[sam]

can you clean the build, run the class (so via gradle), and check the build folder after, see if a test report is generated

[Niko]

Also the HTML file for the class seems to contain both the tests run and passed

[sam]

so they're being run

[sam]

you're just not seeing the output in intelli

[Niko]

Looks like it

[sam]

I have since something similar

[sam]

it's like sometimes intellij doesn't open the "tests" window

[Niko]

I don't think it's only IDEA though, since running gradle clean test in terminal will also run the tests in silence, and only report build failure if I make a test fail:

> Task :test

dev.niko.sc.LoadFactorsTest > all load factors detected FAILED
    java.lang.AssertionError at LoadFactorsTest.kt:23
        Caused by: java.lang.AssertionError at LoadFactorsTest.kt:23

19 tests completed, 1 failed

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///.../sc/build/reports/tests/test/index.html

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at <https://help.gradle.org>

BUILD FAILED in 11s
26 actionable tasks: 5 executed, 21 up-to-date

I remember previously seeing a terminal output of the Kotest runner itself..?

[sam]

not unless you configure gradle to output tests, and if your only line is useJUnitPlatform then it won't

[sam]

maybe that's what you need

[sam]

I have this

tasks.named<Test>("test") {
   useJUnitPlatform()
   testLogging {
      showExceptions = true
      showStandardStreams = true
      exceptionFormat = TestExceptionFormat.FULL
   }
}

[sam]

And here is the run configuration for a test after I have ran it at the class level, what does yours look like ?

[Niko]

[Niko]

setting the testLogging {} , now the assertion errors are printed, but no other changes (it's a start).
Previously I remember running the class for tests would create a Kotest run config, and bring up the Kotest panel, and not the Gradle run config one

[sam]

That's because intellij changed the default in 2021.3

[sam]

You can change it back here:

[Niko]

Right... So it would be IDEA "issue" after all...

[sam]

well I guess them changing the default possibly, but it should work with gradle anyway, and does for me

[sam]

The underlying issue seems to be that under certain circumstances, intellij doesn't realise the task you are running is a "Test" task and doesn't switch the run window to be a tests output window

[Niko]

It does..? Damn. Well, changed it to run tests with IDEA, now I'm getting the Kotest panel atleast for some proper test cases

[sam]

So I guess you can just do it that way

[sam]

the kotest plugin is better anyway, as you'll be able to navigate to tests properly

[sam]

gradle doesn't do it properly because its implementation of junit5 is broken

[Niko]

☝️ true that

[Niko]

Yeah, there seems to be some issues with the Gradle Kotlin build scripts (I'm having some red squiggly with applying some of the project configs as IDEA cannot handle the SAM interfaces of the build script builders and mis-identifies this/it with the scopes...) so no wonder if under some circumstances the process fails with IDEA as well

[sam]

yeah possibly

[Niko]

But that fixes the main issue of not getting proper reporting ✅ thanks Sam.
I'll check back in a year or so, whether the tests runner option could be re-changed as Gradle...