Testing on JS platforms: 1. The Kotlin test infra...

# kotest-contributors

[Oliver.O]

Testing on JS platforms:

1. The Kotlin test infra (Compiler, kotlin-test, KGP) uses the Mocha runner for jsBrowserTest, jsNodeTest, wasmJsBrowserTest, but not wasmJsNodeTest.
2. For wasmJsNodeTest with Kotest, I'm using Kotest's own runner via DefaultSpecExecutorDelegate(defaultCoroutineDispatcherFactory, context), in conjunction with Kotest's TeamCity reporting, which works well.
3. I also have a solution for https://github.com/kotest/kotest/issues/3329. To fix test reporting for jsNodeTest, we'd have to use a transformation function of the KGP-provided kotlin-test-js-runner. While possible, this would tie us to an internal, undocumented Kotlin API.
4. Then there are nested tests, which are not currently supported on JS.
Taking this into account: Do we actually need to use an external test runner? Can't we just make Kotest use its own runners on all platforms, including JS, and drop interfacing with Mocha altogether?

[Emil Kantis]

I think the long term plan is to use our own gradle plugin, compiler plugin and runner. I think that would solve this for us as well?

[Oliver.O]

Gradle and compiler plugin are required now anyway to use the Kotest framework on non-JVM platforms. I’m just wondering whether there might be some compelling reason to use an external test runner.

[sam]

you still need something to execute the compiled javascript code - that's where mocha comes in right ?

[Oliver.O]

Yes, but used as is, it is too opinionated to accommodate the Kotest test structure (no async in test containers). The major upside from what I know is that via KGP, it comes pre-configured and integrated with Karma. I'm not aware if folks are using any Mocha-specific plugins in conjunction with Kotlin tests.

On JS/Node, even JetBrains have stopped using Mocha for wasm targets. They are still using it on JS/browser, probably due to the existing Karma-integration. But Karma is deprecated anyway. And as the Kotest Gradle plugin must be used on non-JVM targets, we could provide our own test run targets and use whatever Karma replacement we'd like and from there just use existing Kotest runners with full structured coroutine support as on the JVM.

Maybe right now we could just package all Kotest tests into one overarching Mocha test on JS, and just use our own runner and test reporting inside?

[sam]

The reason why there's no async inside parent tests is because the underlying javascript runners don't support it. They only support promises at the leaf level.

[sam]

As for mocha vs karma vs whatever, I don't think it matters what we use, so happy to switch. They all mostly have the same structure with the describe/it syntax that we map to.

[sam]

Maybe right now we could just package all Kotest tests into one overarching Mocha test on JS, and just use our own runner and test reporting inside?

This is an interesting idea that might mean we could support async at all levels. We could have a top level describe test just called "kotest" and then have all our tests inside a single "it" which we then break down and report on. We'd have to play around with the way the test frameworks output results, because right now its mocha/karma that's doing the report generation and if we did a single test it would output as a single test.

[sam]

I'd love to completely ditch the gradle test plugin and use our own. The gradle test plugin is simply not powerful enough for what we want to do, and doesn't even fully support the junit platform spec. There's open tickets on this going back 6 years. If we used our own plugin completely, then we could support IDE integration for all platforms and use gradle to run tests instead of executing java directly from intellij.

[Oliver.O]

So let's see. Maybe we could do "top-level only Mocha" right away and then migrate to something more sensible afterwards. I have some experiments which suggest that this might work, but I'd like to wait for the wasm target PR getting reviewed and merged first, as it has already moved partially along that path (for wasmJsNodeTest). OK to proceed that way?

[sam]

Yep.