How to use Kotlin/JS nested suites using Prepared?...

# opensavvy

[CLOVIS]

How to use Kotlin/JS nested suites using Prepared?

Original question: https://kotlinlang.slack.com/archives/CT0G9SD7Z/p1736762977403649?thread_ts=1736762977.403649&cid=CT0G9SD7Z

[CLOVIS]

First, since the user is already using Kotest, I'm assuming they'll want to use the Kotest runner, with the boilerplate

class SchemaTest : PreparedSpec({
    // ...
})

that I will thus leave out for the other snippets

[CLOVIS]

The original snippet was:

class SchemaTest : FunSpec({
     beforeSpec {
         validate = Ajv(Options(strict = true)).compile(
             JSON.parse(readFile(schemaFile, utf8))
         )
     }
 
     withData(readdir(dataDir).asIterable()) {
         readFile(path.join(dataDir, it), utf8) should beValid()
     }
 })
 
 lateinit var validate: ValidateFunction
 
 fun beValid(): Matcher<String> {
     return Matcher { data ->
         MatcherResult(
             validate(YAML.parse(data)),
             { "..." },
             { "..." },
         )
     }
 }

Where readdir is the "problematic" call that is not working as not in a suspending context.

[CLOVIS]

I'm having a bit of difficulty to understand this, so:
• we have a beforeSpec action that reads a schema file into an Ajv object. I don't know what that is, so I'll assume it's part of the SUT.
• then, we call readdir which returns an iterable. I'll assume it's listing the files that are part of the directory somehow.
• for each of these, we want to execute a test that reads a file and uses a Kotest-style matcher

[CLOVIS]

First, since we're using the Kotest compatibility, there's no need to touch the beValid function in any way. We can make it a top-level function and not have to touch anything else.

[CLOVIS]

So a direct translation would be something like:

val validator by prepared {
    Ajv(Options(strict = true)).compile(...)
}

test("Validate all") {
    val files = readdir(dataDir).asIterable()
    for (file in files) {
        readFile(...) should beValid()
    }
}

That works, but it uses a single test, so there's not much info if one of them fails. We'd prefer having a test per case.

[CLOVIS]

wth did slack do with that snippet, why are all the lines duplicated??

[CLOVIS]

So we want a test for each file, but we need to suspend to know which files exist. That's a problem because we can't suspend directly in suites (we want test discovery to be deterministic and immediate for purposes of IDE integration in the future)

[CLOVIS]

The initial snippet uses a trick taking advantage of the fact that Kotest resolves test containers lazily, so beforeSpec is initialized before Kotest knows what the data is going to be. Honestly I'm surprised this even works, since tests are registered in a push-fashion internally. The same trick won't work with :runner-kotest because all tests are registered to Kotest before anything else happens.

[CLOVIS]

In this situation I think the best solution would be the upcoming emulated test parametrization feature, but it's not implemented yet, and won't work with Kotest anyway (though it will probably work with KTI).

[CLOVIS]

@Vampire I assume there really is no way to get the list of files without suspension? I'm curious what the overall use-case is, it's the first time I find a parametrization/nested issue in Kotest Kotlin/JS that Prepared doesn't fix (Prepared started as a prototype to rethink Kotest suspension to work better on Kotlin/JS)

[CLOVIS]

There are extremely fragile things that could be done, like taking advantage that Kotest executes stuff sequentially, so we could declare the readdir in a first class and access its result in a second test class that executes later, but I can't say i like that

[Vampire]

You can ignore the beforeSpec that is not relevant for the topic.
It just initializes a variable once (Ajv which is a JSON schema validator) that is then used in the matcher.

The essence is, how to produce multiple test leafs from the result of a suspending call.
I would prefer something that uses kotest as it is going to be a contribution to a project using kotest already, but if it does not work with kotest but with something else that would also be ok for me.

I assume there really is no way to get the list of files without suspension?

Only if you tell me how to list a directory without suspending.
Besides that I can of course do it in Gradle and give the list of files to the test task, that is my fallback plan.
But I'd prefer something inline.

I'm curious what the overall use-case is

I intend to test a JSON schema against a directory of valid JSON files and a direcotry of invalid JSON files to make sure the schema is doing what it is intended to do also in the future.
And I want to do that using Ajv as that is also going to be used later in production and known to work as expected.
So I just throw in all valid files in one directory and all invalid files into another, without the need to add a test case for each manually as any manual action is prone to be forgotten or did wrongly, especially if it could have been automated.

[Vampire]

Oh, I'm stupid.
Thanks for rubber-ducking.
I think I can just list the two directories in beforeSpec then I can just use normal withData, as the result of that call will not change so it does not have to be right before the test but in beforeSpec is fine.
🙈

[CLOVIS]

Yeah that sounds like it would work in pure Kotest

[CLOVIS]

Now I'm confused, that's the first example where Kotest can do something and Prepared can't 😅 I'll study this and figure out something

[CLOVIS]

Have a great day!

[Vampire]

Hm, no, that doesn't work. beforeSpec is done after the argument for withData is necessary 😕

[Vampire]

So back to square one. 😞
Any other ideas with using prepared?

[Vampire]

Oh my god, I am an idiot, thanks again for rubber-ducking.
I just have to use readdirSync instead of readdir as that is not suspending but blocking.
The general question is still interesting, but my use-case is solved with that.
🙈