I finally stitched together a clean-and-simple, st...

# kotest-contributors

[Bernd Prünster]

I finally stitched together a clean-and-simple, stand-alone KMP project with manual wiring of the provided JUnit XML reporter here. As you can see, it doesn't nearly produce enough test reports either on macos or on linux. there should be two report files per platform, but if you check the "List reports" output of both jobs, you'll notice two rather lonely files created by the linux test run and eight somewhat less lonely files created by the macos runner. Things don't really get better when adding the junit runner, except that jvm test reports start appearing. so my question is: how to run tests on all targets?

[Bernd Prünster]

manual wiring of ksp is now running. but I don't expect any change

[sam]

Would you say this is the last blocker to 6.0?

[sam]

Does the standard kotlintest stuff make an xml report even for JS?

[Bernd Prünster]

Well, Auto-wiring of the XML reporter should be in 6.0 final, unless you want some backlash 😉.
Now that you mention it: I never even tried kotlin.test because it is so limited and the last serious JS targets we needed in production were Kotest 4 with a kotkon/js alpha or so. Anyway, I'll duplicate the project with kotlin.test so we can do a side-by-side comparison tomorrow.

[sam]

oh I wasn't suggesting that we ship without XML lol

[sam]

just that once I get XML working, is there anything ELSE that would stop 6.0

[sam]

I feel like this is the last blocker? Anything else could be 6.0.1 or 6.1

[Bernd Prünster]

kotest does not seem to run tests for all targets. interestingly: kotlin.test also does not include all targets in the allTests task, but at least it runs them, when invoked directly.

I use the exact same project, branched off to wire kotlin.test and voilá: All the test reports are there:
• macos cant run linux tests, which is fine
• linux cant run apple tests, (which is also fine)
compare with kotest:
• macos run
• linux run
I've not invoked jvmTest

[sam]

no JS xml

[sam]

which makes sense as no way to write to the file I guess

[sam]

I see wasmJS

[Bernd Prünster]

I still don't fully understand what's going on (or rather, waht isn't)

[sam]

this will add native XML support https://github.com/kotest/kotest/pull/4983

[sam]

so after this is merged, we'll have JVM + Native

[sam]

with zero setup required by you

[sam]

the plugin configures it all

[sam]

(the reason target was "todo" is because I might just get the name from the configuration then it's fully dynamic, tbd if I can do that)

[Bernd Prünster]

I saw that, but if you look at the cli logs from the runner, you don't see enough tests being executed. Most invocation are a noop when using Kotest

[sam]

ok I'll take a look at that next, I haven't gone back to my KMP test project for a few days, probably something broke

[sam]

was focusing on getting the final few bits in

[Bernd Prünster]

Feel free to clone my project and try locally. Apple targets seem to wok perfectly fine, but js and wasm don't

[sam]

do you have to do something in github actions to make files appear like this?

[sam]

which project should I clone?

[Bernd Prünster]

https://github.com/JesusMcCloud/kotestM8Test

[sam]

thanks, cloned

[sam]

I have this too: https://github.com/kotest/kotest-examples/tree/master/kotest-multiplatform

[sam]

which is meant to be a source of truth for people to see how it works

[sam]

would be good to get that equivalent to your project

[sam]

you should have write access to that if you want to make pushes

[Bernd Prünster]

The results from the actions you see is just the dorny test reporter v2 in summary mode and a glob pattern to catch all test files

[Bernd Prünster]

Before pushing to the sample, I want to know what's going on. That is why I pushed it to my personal namespace, as I consider it an experiment

[sam]

those example projects are experiments too atm

[sam]

I consider them WIP for 6.0 until 6.0 is released, I'm using them as my test bed

[sam]

I'll work against yours for now, I just can't push to it

[Bernd Prünster]

i just tried again localy on ma mac: i only get the apple targets and jvm result and I only get invocations for apple targets and jvm. in summary: only jvm tests are run, but not js (as wasmWasi is not working for M8, I cannot say anything about wasi). kotlin.test provides XML reports for all targets (and so did Kotest 5.9)

[Bernd Prünster]

and now i need to go. I'll probably be back later, but no guarantees

[Bernd Prünster]

i invited you as collaborator, in case you want to push to my repo

[Bernd Prünster]

just realized that the xml report has to go into the system temp dir first, and must only then be copied over by gradle, since some targets allow only writing to temp dirs. anyways, I updated my test project:
• Linux summary
• macOS summary
Still, no changes to before except that I now also execute jvm tests and they show up as expected. The raw logs are probably more interesting, though, as they make it easy to comprehend what is and isn't executed.