FYI: I concocted a multiplatform JUnit XML test re...

# kotest-contributors

[Bernd Prünster]

FYI: I concocted a multiplatform JUnit XML test reporter here: https://github.com/a-sit-plus/signum/commit/76ee8cb5a0f97972fbd1e0b4d920d263e378ba29 since project config only works on JVM, I had to manually hook it into every test in the indispensable module. Not pretty but you get the gist of it. feel free to shamelessly copy integrate it into Kotest 😉

[Bernd Prünster]

and some minor patch ups: https://github.com/a-sit-plus/signum/commit/05c7be6057e9686c0db920d10d2e7354a220d014

[Bernd Prünster]

Not working for all targets though, but better than nothing. Then again, this may be unrelated

[Bernd Prünster]

also: this state is intentionally producing outputs for the JVM as well, which means you have junit output and the custom output (as seen here).
At the same time, there is some flakiness and ios test reports don't show up at all (see the error here) even though locally i see the reports

[Bernd Prünster]

I have something for you guys as I've now finished tinkering with our conventions plugin. JVM otuputs are off by default (though i still needed to subclass FreeSpec because project config is not working for anything but JVM)l

Lo and behold: automatic generation of junit test reports for all targets (that are actually run by the kotest runner; so no JS). I don't feel confident making this into a PR, but this is now properly working as it should, so for anyone familiar with the kotest source, integrating it is essentially a finger exercise.
There is one caveat though: the temp path resultion does not work as expected, because of sandboxing, so the reports are there, but not collected from the gradle plugin because they reside at a different directory. with some KSP magic you could fix that

[Bernd Prünster]

(i just pushed a new commit to that branch with a dodgy workaround for the path things)

[Bernd Prünster]

here are sample reports: https://github.com/a-sit-plus/signum/runs/45853327290
https://github.com/a-sit-plus/signum/runs/45853143471

[sam]

this is great, will copy this into kotest asap

[Bernd Prünster]

awesome! i discovered a flaw in the auto-wiring of kotest and in the copy task of this thingy here: it tries to wire targets that may not be buildable and the copy task fails hard if no test report is presen (i.e. for the js targets as they are not running). i just pushed a supposed fix, but i need to run the pipeline on the github linux runners (i am no mac here). I'll prepare a PR with the target exclusion once verified

[Bernd Prünster]

here's the missing runCatching for test copying in the gradle task and here's the temp path resolution for apple targets.

[Bernd Prünster]

also: i get cinterops errors on non-apple hosts that I do not understand. full disclosure: I am not even sure it is related, as i did some rebasing

[Bernd Prünster]

linux reports get written relative to the project dir apparently and not to a temp dir

[Bernd Prünster]

grrr. i need a linux machine to investigate further

[sam]

Isn't the project dir where junit writes them too?

[Bernd Prünster]

the native test runner i concocted is running as a process native to the target it is testing, which is why i rely on kotlinx.io to get the temp dir. this directory may differ from target to target, I only see the path from the test runner / report writer as it is used there. should be a relative path to the project/module directory if it does not start with a forward slash. I tried working around it, but still no dice.

[Bernd Prünster]

(i.e. the native report collector is just hooked into the ksp-generated test runner code as an extension)

[sam]

Gotcha

[sam]

The gradle plugin could pass the build dir as a launcher param. Then the extension would be able to use that. Similar to how junit does it.

[sam]

I can look at adding that.

[Bernd Prünster]

that would be perfect!

[sam]

I think having junit xml.output for all targets is needed before 6.0 final

[Bernd Prünster]

i would still follow the approach to write to the system-wide temp dir though, because that is a directory that every test runner is allowed to wrote to, regardless of target

[sam]

Ok

[Bernd Prünster]

how would the gradle plugin pass that parameter to native runners? part of ksp-generated code? bceause System.getenv() is not a thing on non-JVM targets

[Bernd Prünster]

(as for varying temp dirs dependeing on the target, see here for the simulator workaround)

[Bernd Prünster]

but when it runs on a macos host, it is marvellous: https://github.com/a-sit-plus/signum/runs/45882236483

[sam]

Native supports env

[sam]

And for js we pass it into the invoked function

[Bernd Prünster]

if you are onto it anyways then theres no need for me fiddlign with it because everythign i put together will be the foundation for a natively supported feature anyway

[sam]

Yeah I'm gonna take your reporter code and make a kotest pr. Probably tonight