I am trying to use `withData` and `DescribeSpec` but I can’t get the tests to be named as I expect. ...

# kotest

[Michael Strasser]

I am trying to use withData and DescribeSpec but I can’t get the tests to be named as I expect.

Given a function fun messageMap(id: String): Map<String, Message> that returns 3 different messages, if I use:

describe("adds messages") {
    val id = randomId()
    it("sets ID from message") {
        withData(messageMap(id)) { message ->
            Builder.newInstance().addMessage(message).getId() shouldBe id
        }
    }
}

The test runs once with title “sets ID from message”. But if I use:

describe("adds messages") {
    val id = randomId()
    withData(messageMap(id)) { message ->
        it("sets ID from message") {
            Builder.newInstance().addMessage(message).getId() shouldBe id
        }
    }
}

The test runs 3 times with titles:

sets ID from message
(1) sets ID from message
(2) sets ID from message

I expected from the docs that the string keys for the map would be used as test names.

I am using Kotest 4.6.3.

[BorzdeG]

describe("adds messages") {
    it("sets ID from message") {
        val id = randomId()
        withData(messageMap(id)) { message ->
            Builder.newInstance().addMessage(message).getId() shouldBe id
        }
    }
}

[sam]

Note that in 5.0 you can only nest withData inside parent tests (so not it). Otherwise the output doesn't work properly on gradle.

[BorzdeG]

hm. its works me on 5.0.0.RC

[sam]

it does ??

[sam]

lol

[sam]

ahh it's probably binding to the outer scope

[sam]

I'll have another attempt this weekend to see if we can relax that restriction before 5.0.0 final, but it's gradle not respecting the junit spec properly causing issues.

[Michael Strasser]

It does work as expected when I use the right combination of functions: inside describe but without specifying it. For example:

describe("adds messages") {
    describe("sets ID from message") {
        withData(messageMap(randomId())) { message ->
            Builder.newInstance().addMessage(message).getId() shouldBe id
        }
    }
}

[Michael Strasser]

Run by IntelliJ but not run by Gradle. Sigh

[sam]

What happens with Gradle ?

[Michael Strasser]

Gradle does not run the tests at all. Tests specified using it run, but those generated by withData do not. By contrast, the Kotest plugin runs all of them.

In one test file, Gradle and Kotest both report 9 tests (the number of `it`s) but Kotest runs the 92 extra withData tests, while Gradle does not (as far as I can tell).

I ran a small experiment with that file. I caused a test to fail by modifying a value returned by the messageMap() function.
• Kotest reported details of the failing test, as expected.
• Gradle reported failure of the parent describe block but with no details.
I hope this helps. (Sorry, I can’t share actual client code with you.)

[sam]

Ok I can take a guess as to the issue. The tests are not nesting properly in 4.6.x with describe is my guess.

[sam]

There's a bug in gradle where it doesn't adhere to the JUnit spec properly. 5.0 has a workaround

[Michael Strasser]

I switched to Kotest 5.0.0.RC. Both Gradle and Kotest runners now report 101 tests for the file.

When I change messageMap() to break tests, both Gradle and Kotest correctly report which tests failed.

[sam]

ok cool

[Michael Strasser]

Thanks for your help!

[sam]

You're welcome

[sam]

Just don't nest tests inside it as that will be disallowed later

[sam]

leave them in the describe blocks

[Michael Strasser]

Yep, I understand now.