Hey guys, I have a somewhat arbitrary question/req...

# kotest

[Niko]

Hey guys, I have a somewhat arbitrary question/request about test runners, config(enabledIfOrReason) and TestCase.isFocused() :
I'd like to be able to, e.g. via IntelliJ IDEA or command line, target a certain test, that is by default disabled.

If I add enabledOrReasonIf = ... { ... Enabled.disabled("will only run if targeted") the test will be skipped by the runner, even if I'd specifically targeted it via the --spec and --testpath.
Naturally, I could have

object EnabledOnlyIfTargeted : EnabledOrReasonIf {
    override fun invoke(test: TestCase): Enabled =
        if (test.isFocused()) {
            Enabled.enabled
        } else Enabled.disabled(
            """Focus (add "f:" prefix) test to run it"""
        )
}

but I'm not a fan of `f:`/ ! prefixes as I wouldn't like to touch the sources just to run a test. My use case is somewhat non-tests with e.g. printing graphs or other data for otherwise tested complex data manipulators to aid in narrowing down issues. Obviously I could use kotlin scripts, but for convenience kotest test cases would be better.

How should I approach this, as clearly if the EnabledOrReasonIf.invoke() gets called, the test runner has already chosen the test to be run, but I'd like the test to be skipped if it was not particularly targeted?

[Rob Elliot]

I wonder if you can do something with tags? You can tag tests and then use a TagExtension and override tags(): TagExpression to decide which tags to include / exclude. If you can set an environment variable to be different in the case where you want it to be run and don't want it to be run you can then evaluate that there and include / exclude tagged tests as appropriate.

[Rob Elliot]

I do something like this here:
https://github.com/Mahoney-playground/goos/blob/main/app-src/ui/swing/test/UI.kt
to ensure that running the build on a Mac doesn't pop up windows all over the place.

[Niko]

Yes, tags would (and I'm already using them for this) somewhat work, though would not fully cover the case, where I'd like to directly reference a single test case I wish was run. With tags, I'd have to create tag per test case

[Niko]

Currently, I have

object EnabledOnlyIfTargeted : EnabledOrReasonIf {
    override fun invoke(test: TestCase): Enabled =
        if (
            test.isFocused() ||
            // null if run via IDEA plugin
            syspropOrEnv("kotest.filter.tests") == test.descriptor.id.value ||
            // TODO: read in testpath from command args when using IDEA plugin (process is forked and ProcessHandler will have no args)
            sysprop("testpath") == test.descriptor.id.value
        ) {
            Enabled.enabled
        } else Enabled.disabled(
            """Disabled as the test isn't explicitly targeted. Focus (add "f:" prefix) or target via --testpath/-Dkotest.filter.tests"""
        )
}

which seems to be working. Some TODOs as running via IDEA isn't working but directly with gradle test -Dkotest.filter.tests at least now works

[Rob Elliot]

That's not what I'm suggesting - I'm suggesting a tag something like object OnlyRunWhenTargeted : Tag()

[Niko]

Ah, right

[Rob Elliot]

Not sure it works though - I don't think you know the test name in a TagExpression's tags() method

[Niko]

Yeah, not sure either, if I had any better visibility into the args as they are partly lost when the runner forks... Might still take a look