https://kotlinlang.org logo
Join Slack
Powered by
I want to update Kaml to support Kotlin/JS <https:...
# kotest
a
I want to update Kaml to support Kotlin/JS https://github.com/charleskorn/kaml/pull/507, but Kaml uses nested Kotest tests, which Kotest doesn't support. Aside from refactoring all of the tests so they're not nested, is there anything I can do to workaround this? I've tried implementing a custom DescribeSpec, but I still get dinged with 
IllegalStateException: Nested tests are not supported
I've updated this branch https://github.com/kotest/kotest/pull/2957 with the latest changes from the master branch, and deployed it locally, and it works for me! Can this be merged into Kotest?
I've created a new PR https://github.com/kotest/kotest/pull/3913 - could someone please trigger the CI tests? :)
👍 1
👍🏻 1
e
I branched your branch and tried to verify and see if I could add tests for this somehow.. https://github.com/kotest/kotest/compare/kantis/js-nested_tests There's an old test project in kotest-tests which does some JS testing which I figured could be extended with some nested test to ensure it works, but it looks weird when I try to run them
a
ah great, thanks, I wasn't sure where to start with adding tests. I'll take a look...
I see what you mean, it's not showing the nested tests. 
a context
should have two children, one success and one failure
trying to run with Kotest IJ plugin gets an unhappy error
Copy code
Error: Could not find or load main class io.kotest.engine.launcher.MainKt
Caused by: java.lang.ClassNotFoundException: io.kotest.engine.launcher.MainKt
e
I havn't gotten around to testing the same thing by publishing through mavenLocal.. is it the same?
a
I'm 10 minutes into publishing MavenLocal now :)
does Kotest have any funding? It'd really help getting a remote Gradle Build Cache instance set up
e
No funding afaik, but I know gradle sometimes support open source projects
I've been considering asking them to help us out but haven't had time to look into it
a
That'd be cool, and I'd be happy to help out with setting up the build config
🙏 1
it doesn't need to be a special Gradle Build Cache instance either, there are plugins for using AWS, GCP, etc
or a home server
e
Could the GH actions cache be used? 😄
a
I think not, but possibly GitHub Pages could be used!
or maybe storing the cache as workflow artifacts - but that'd be a lot of custom work
e
Probably better to ask if Gradle is interested in sponsoring 🙂
a
Right! I've made some progress
🫡 1
I've no idea where "context" comes from - it's certainly not from my tests!
but at least it's listed!
e
Very nice!
a
I think the issue with the name is that at the point of test execution, it's registering all of the parent describes(), and so the top level NestedTests name is being de-duplicated in a strange way.
I think that's a simple fix
I've been wanting nested JS tests for years so I'm pretty pleased!
😄 1
e
Thx for putting in the work!
🤟 1
a
hmmm maybe it's not so simple. Coroutines are hard! I thought that using suspendCoroutine might help, because that's supposed to be able to bridge non-suspending code (like the 
describe()
and 
it()
js test functions), but it doesn't seem to make any difference.
I also tried to properly call describe() and it(), so that they would be called in the correct places and the test should properly appear nested, but that didn't seem to work at all. The nested tests/contexts don't seem to be rendered. Perhaps I'm misunderstanding how SpecExecutorDelegate should be working, and my code is calling it() when it should be calling describe(), but it's hard to tell
sometimes it seems like a 
describe("outer") { describe("inner") {} }
results in 'inner' replacing 'outer'
Ahh okay, this is why https://jestjs.io/docs/setup-teardown#order-of-execution
Jest executes all describe handlers in a test file before it executes any of the actual tests.
So Kotest shouldn't launch the tests until all the describes() are finished registering the test cases...
Workaround: concatenate the context & test names with an arrow
o
Additional things to consider: • The upcoming 
wasmJs
target also uses Kotlin's JS test infra, which comes with a subtle difference: With Wasm, the Mocha framework (
describe
, 
it
) is used on 
JS/browser
(via Karma), but not on `Node.js`: https://kotlinlang.slack.com/archives/CDFP59223/p1702832777305729?thread_ts=1702761028.987939&amp;cid=CDFP59223 • There may be scheduling/timeout problems with coroutines running as part of test containers (
describe
); Mocha allows Promises for leaf tests only.
e
Can’t we make our own multiplatform runner instead? 🤓
o
Yes, I've also been thinking about that. The runner is easy. I'm not so sure about the interop implications (Gradle plugin, Karma invocation, reporting, message formats understood by the IDE, ...).
a
thanks for checking @Oliver.O, it's useful to know there are differences in behaviour.
The upcoming wasmJs target also uses Kotlin's JS test infra
Could we just have separate externals for wasmJs and JS?
o
Yes, actually we need to due to different signatures, with Wasm relying on strict typing. So that's what I have already in place in https://github.com/kotest/kotest/pull/3805: • src/jsMain/kotlin/io/kotest/engine/jasmineTestAPI.js.kt • src/wasmJsMain/kotlin/io/kotest/engine/jasmineTestAPI.wasmJs.kt Obviously, I'd also love to see nested tests being supported on JS if it can be made to work correctly. It would be cool if you could familiarize yourself with PR #3805 and base your changes on what's already there. My contacts with JS testing suggest that it's somewhat tricky to get everything right the way test infra layers work now. OTOH, it seems possible to cooperate with the Kotlin/JS folks in order to get a more uniform and easy to use test API which would then serve kotlin-test and Kotest.
s
To my understanding, this can't work because the underlying JS test frameworks don't support promises in the parent tests, but Kotest defines all test scopes as suspendable. So we can't get a promise in order to get a coroutine scope for the parent test scopes. That's why I made it just say you can't do it. Some alternatives - don't use mocha at all? We're just executing a JS program, why not just have it run the test launcher directly. We would need to write a node program to launch it though. Or try to hack it so that all kotest tests are in a single mocha parent scope.
a
It would be cool if you could familiarize yourself with PR #3805 and base your changes on what's already there.
Sure thing, I see it's ready to merge so I'll take a look
thanks for checking @sam, I'm pretty sure that https://github.com/kotest/kotest/pull/3913 works, albeit in a hacky way. As far as I can tell it's not a problem that the `describe()`s aren't async, so long as the actual test execution can be launched GlobalScope, and when its's finished complete the 
done
promise.
I was thinking more about it though, maybe the JS tests should be based on the InstancePerLeafSpecRunner. So Kotest will gather up all of the declared tests and the actions in each parent 
describe()
, and run each leaf independently inside a JS 
it()
. But yes, your idea of executing the test launcher directly sounds better. I'm already over the limits of my test-engine and JS knowledge though so I wouldn't know where to start with that :)
s
If you launch GlobalScope inside a describe block, there is no done handle, and mocha won't know to wait for it.
o
Some alternatives - don't use mocha at all? We're just executing a JS program, why not just have it run the test launcher directly. We would need to write a node program to launch it though.
...and a Karma framework plugin.
or rather consider alternatives like Jest and Web Test Runner.
e
Why is Karma needed?
o
It is currently used to run tests in headless browsers.
a
Just asking questions so I can keep up: • We could use Web Test Runner for browser tests? (The custom framework setup looks pretty simple) • And then something else (Jest?) for NodeJS and WasmJS tests?
o
Web Test Runner looks reasonable to me (though I'm no expert in overall JS ecosystem offerings). And beyond that we might actually not need anything else. Kotest does already run tests so what value would a test runner like Jest add? Less moving parts -> less complexity to maintain, more runtime efficiency.
a
so what value would a test runner like Jest add?
No idea :) I thought you were proposing to use it
the JS ecosystem is mostly a mystery to me, all I want is to have my Kotlin/JS tests run, because at the moment they don't. But I'm happy writing Kotlin externals for JS libs, I'm just not sure what to write.
s
We could duplicate something like mocha pretty easily, because as @Emil Kantis points out, all the "test features" kotest already has. All we need is something that executes a js file and takes the output and pipes it back to intellij. The bit I'm not sure on is how easy it is to get intellij to pick it up.
I know that intellij hardcodes support for gradle test mode in some places.
The native runner is even more simple. It's just builds an executable that invokes the tests.
o
From a pure Kotlin point of view, I'm always asking myself "is this piece of JS really necessary"? Some innocent looking packages come with lots of dependencies to track and manage. And even writing a test server like Karma or Web Test Runner does not seem such a big deal in Kotlin. Then again, things often look simpler at the start and then drown us in interop issues. 🤔
a
The bit I'm not sure on is how easy it is to get intellij to pick it up.
I know for JVM tests IntelliJ injects some Gradle config that adds a test listener to the Gradle test tasks. The test listener reacts to the events, and emits some XML over stdout that IntelliJ picks up. So if KMP&Kotest tests trigger the same Gradle test events, then it should just work. Otherwise, Kotest could detect if IntelliJ is currently active (via 
idea.active
system property) and emit the XML itself.
If that helps
s
Yeah we do that already (it's called TeamCity format actually).
today i learned 1
The problem is getting intellij to automatically show the "test window" which it does by knowing that "gradle test" is special
a
Ahh right, you mean like how the test tasks are green highlighted?
s
when you run gradle test, the "run" window shows the test pane.
I ran gradle check, and the view flips:
4 Views
Open in Slack
PreviousNext
Powered by