Hi, I'm looking into adopting Kotest, but it seems...

# kotest

[CLOVIS]

Hi, I'm looking into adopting Kotest, but it seems the multiplatform example doesn't work. When cloning the project and running ./gradlew check , only the JVM tests are executed. I was expecting to see the JS tests and the MacOS tests as well. Full trace in thread.

• only the JVM tests are printed to the terminal
• only the JVM tests are present in the Gradle-generated test report (the other platforms are generated but are empty)

[CLOVIS]

./gradlew check
Downloading <https://services.gradle.org/distributions/gradle-8.3-all.zip>
...................10%...................20%...................30%...................40%...................50%...................60%...................70%...................80%...................90%...................100%

Welcome to Gradle 8.3!

Here are the highlights of this release:
 - Faster Java compilation
 - Reduced memory usage
 - Support for running on Java 20

For more details see <https://docs.gradle.org/8.3/release-notes.html>

Starting a Gradle Daemon (subsequent builds will be faster)

> Configure project :

Please wait while Kotlin/Native compiler 1.9.10 is being installed.
Download <https://download.jetbrains.com/kotlin/native/builds/releases/1.9.10/linux-x86_64/kotlin-native-prebuilt-linux-x86_64-1.9.10.tar.gz> (192.68 MB)
Download kotlin-native-prebuilt-linux-x86_64-1.9.10.tar.gz finished, took 33 s 500 ms
Unpack Kotlin/Native compiler to /home/ivan/.konan/kotlin-native-prebuilt-linux-x86_64-1.9.10
Unpack Kotlin/Native compiler to /home/ivan/.konan/kotlin-native-prebuilt-linux-x86_64-1.9.10 finished, took 2 s 168 ms

> Task :kotlinNpmInstall
warning Ignored scripts due to flag.

> Task :compileKotlinLinuxX64
v: Using Kotlin home directory dist/kotlinc
w: file:///tmp/kotest-examples-multiplatform/src/linuxX64Main/kotlin/io/kotest/examples/multiplatform/generateUUID.kt:9:16 'getTimeNanos(): Long' is deprecated. Use measureTime() or TimeSource.Monotonic.markNow() instead.

> Task :compileTestKotlinLinuxX64
v: Using Kotlin home directory dist/kotlinc

> Task :jvmTest

io.kotest.examples.mpp.UUIDJvmTest[jvm] > uuids should be in be type 4 format[jvm] PASSED

io.kotest.examples.mpp.data.DataDrivenTest[jvm] > PythagTriple(a=3, b=4, c=5)[jvm] PASSED

io.kotest.examples.mpp.data.DataDrivenTest[jvm] > PythagTriple(a=6, b=8, c=10)[jvm] PASSED

io.kotest.examples.mpp.uuid.UUIDTestCommon[jvm] > uuids should be somewhat unique![jvm] PASSED

> Task :compileKotlinMingwX64
v: Using Kotlin home directory dist/kotlinc
Downloading native dependencies (LLVM, sysroot etc). This is a one-time action performed only on the first run of the compiler.

(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (0/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (3881842/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (9970698/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (17579590/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (24684556/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (32171282/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (39583752/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (46389762/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (53335862/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (60275340/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (66050874/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (73119016/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (80700606/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (88094728/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (95440058/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (102850760/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (110293092/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (117389074/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (124228130/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (130864170/132131559). 
(KonanProperies) Downloading dependency: <https://download.jetbrains.com/kotlin/native/msys2-mingw-w64-x86_64-2.tar.gz> (132131559/132131559). Done.
Extracting dependency: /home/ivan/.konan/cache/msys2-mingw-w64-x86_64-2.tar.gz into /home/ivan/.konan/dependencies
w: file:///tmp/kotest-examples-multiplatform/src/mingwX64Main/kotlin/io/kotest/examples/multiplatform/generateUUID.kt:9:16 'getTimeNanos(): Long' is deprecated. Use measureTime() or TimeSource.Monotonic.markNow() instead.

> Task :compileTestKotlinMingwX64
v: Using Kotlin home directory dist/kotlinc

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to <https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings> in the Gradle documentation.

BUILD SUCCESSFUL in 2m 52s
27 actionable tasks: 27 executed

[Oliver.O]

Could you try ./gradlew allTests?

[CLOVIS]

allTests is a dependency of check

[Oliver.O]

allTests is what I am using with multiplatform tests in my projects. And I'm seeing JVM and Js/Browser test results.

[CLOVIS]

Can you try it on that repository?

[CLOVIS]

Maybe your config is different.

[Oliver.O]

Trying...

[Oliver.O]

Tried the above project, commenting out the native stuff in the build script. And yup, you're right, there is something missing: build.gradle.kts needs alias(libs.plugins.kotest.multiplatform) in its plugins block, so that the entire plugins block looks like this:

plugins {
   java
   id("java-library")
   alias(libs.plugins.kotlin.multiplatform)
   alias(libs.plugins.kotest.multiplatform)
}

Then run ./gradlew cleanAllTests allTests.

[Oliver.O]

https://github.com/kotest/kotest-examples-multiplatform/pull/45

[sam]

I've merged the PR

[sam]

@Oliver.O I can give you merge rights on the repos if you want

[Oliver.O]

If that can be helpful in simple cases like this, yes, why not? For serious stuff, I'd always favor a review, of course.

[sam]

yeah for simple stuff just merge it in

[CLOVIS]

Thanks for the quick update, but I'm afraid it doesn't fix it…

./gradlew --no-build-cache check 

> Task :kotlinNpmInstall
warning Ignored scripts due to flag.

> Task :compileKotlinLinuxX64
v: Using Kotlin home directory dist/kotlinc
v: Installing Kotest SpecIrGenerationExtension
v: UUID.kt contains 0 spec(s): 
v: generateUUID.kt contains 0 spec(s): 
v: Detected 0 configs:
v: Detected 0 JS specs:
w: file:///tmp/kotest-examples-multiplatform/src/linuxX64Main/kotlin/io/kotest/examples/multiplatform/generateUUID.kt:9:16 'getTimeNanos(): Long' is deprecated. Use measureTime() or TimeSource.Monotonic.markNow() instead.

> Task :compileTestKotlinLinuxX64
v: Using Kotlin home directory dist/kotlinc
v: Installing Kotest SpecIrGenerationExtension
v: DataDrivenTest.kt contains 1 spec(s): io.kotest.examples.mpp.data.DataDrivenTest
v: UUIDTestCommon.kt contains 1 spec(s): io.kotest.examples.mpp.uuid.UUIDTestCommon
v: UUIDNativeTest.kt contains 1 spec(s): io.kotest.examples.mpp.UUIDNativeTest
v: Detected 0 configs:
v: Detected 3 JS specs:
v: io.kotest.examples.mpp.data.DataDrivenTest
v: io.kotest.examples.mpp.uuid.UUIDTestCommon
v: io.kotest.examples.mpp.UUIDNativeTest

> Task :jvmTest

io.kotest.examples.mpp.UUIDJvmTest[jvm] > uuids should be in be type 4 format[jvm] PASSED

io.kotest.examples.mpp.data.DataDrivenTest[jvm] > PythagTriple(a=3, b=4, c=5)[jvm] PASSED

io.kotest.examples.mpp.data.DataDrivenTest[jvm] > PythagTriple(a=6, b=8, c=10)[jvm] PASSED

io.kotest.examples.mpp.uuid.UUIDTestCommon[jvm] > uuids should be somewhat unique![jvm] PASSED

> Task :compileKotlinMingwX64
v: Using Kotlin home directory dist/kotlinc
v: Installing Kotest SpecIrGenerationExtension
v: UUID.kt contains 0 spec(s): 
v: generateUUID.kt contains 0 spec(s): 
v: Detected 0 configs:
v: Detected 0 JS specs:
w: file:///tmp/kotest-examples-multiplatform/src/mingwX64Main/kotlin/io/kotest/examples/multiplatform/generateUUID.kt:9:16 'getTimeNanos(): Long' is deprecated. Use measureTime() or TimeSource.Monotonic.markNow() instead.

> Task :compileTestKotlinMingwX64
v: Using Kotlin home directory dist/kotlinc
v: Installing Kotest SpecIrGenerationExtension
v: DataDrivenTest.kt contains 1 spec(s): io.kotest.examples.mpp.data.DataDrivenTest
v: UUIDTestCommon.kt contains 1 spec(s): io.kotest.examples.mpp.uuid.UUIDTestCommon
v: Detected 0 configs:
v: Detected 2 JS specs:
v: io.kotest.examples.mpp.data.DataDrivenTest
v: io.kotest.examples.mpp.uuid.UUIDTestCommon

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to <https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings> in the Gradle documentation.

BUILD SUCCESSFUL in 1m 1s
28 actionable tasks: 28 executed

[CLOVIS]

It seems it fixes the Windows tests (I'm running Linux), but I still don't see JS tests 😕

[CLOVIS]

Ah, no, they do appear in the HTML test report this time, so I guess they did run?

[Oliver.O]

Could you try exactly ./gradlew cleanAllTests allTests? Gradle can be hard to convince that it should actually run tests. Sometimes it does without nudging. Sometimes cleanAllTests is required. Sometimes you'll even need --rerun and on a bad day even --rerun-tasks.

https://stackoverflow.com/questions/29427020/how-to-run-gradle-test-when-all-tests-are-up-to-date

[CLOVIS]

^ my previous run was ./gradlew clean; ./gradlew --no-build-cache check

[CLOVIS]

I get the same output with your command

[Oliver.O]

That should be sufficient. Do you use IntelliJ IDEA? Did you consider installing the Kotest plugin? I'm asking because that is my standard setup, and I might be seeing test indicators in the Run window that won't appear with other setups.

[CLOVIS]

I was running this on the CLI directly without IDE to ensure my setup wasn't impacting the result.

[Oliver.O]

Just tried that, too with ./gradlew clean jsBrowserTest (on Linux). Runs successfully, but without useful test output. I am not particularly familiar with how the Js test runners do their reporting, but it seems that these interact differently with Gradle. It's probably much easier to use the IDE, since that seems to be the most common use case (even there you'll notice that the test reporting is different between JVM and Js).

[CLOVIS]

IDEA, without the Kotest plugin, does report test execution for JVM, JS and LinuxX64 (which seems to be the expected result)

[Oliver.O]

Sounds good. The Kotest plugin will additionally give you gutter icons to start individual tests, and a test explorer.

[sam]

the gutter icons don't work for multiplatform yet, only for JVM.

[sam]

That will change once compiler plugins go stable, and we can build on top of them in a reliable way (hopefully kotlin 2.1)

[rocketraman]

Yep, I have the opposite problem: the CLI/gradle runs my tests in commonTest, but IntelliJ doesn't!

Error: Could not find or load main class io.kotest.engine.launcher.MainKt
Caused by: java.lang.ClassNotFoundException: io.kotest.engine.launcher.MainKt

As @sam noted, this is the lack of multiplatform support. 😞

[CLOVIS]

Summary of what I found:
• the Kotest gradle plugin is important
• the JVM test task should be configured with JUnit5
• the JVM test source set should have a dependency on kotest-runner-junit5

[CLOVIS]

doing all of that, it does work