Is there a way to mix JUnit5 and Kotest tags, e.g. have all the tests marked as “integration” execut...

# kotest

[Mykola Gurov]

Is there a way to mix JUnit5 and Kotest tags, e.g. have all the tests marked as “integration” executed with one (gradle) run - be it junit or kotest?

My context: I’d like to introduce kotest to an existing project with loads of JUnit5 tests categorized by tags (e.g. integration, e2e, `contract`; no tag == unit test) and executed in separate jobs on CI (gradle).

[Mykola Gurov]

Ideally, I’d like to be able to use the same tags we’re using for JUnit, e.g. add

import org.junit.jupiter.api.Tag

@Tag("integration")
class KotestIT(): FuncSpec({
    ...
})

but that doesn’t seem to be working out of the box ( kotest 5.2.1 ), and the following command doesn’t pick this test:

./gradlew test \
      -DincludeTestTags='integration & !e2e & !contract' \
      --tests my.example.KotestIT

[Mykola Gurov]

when I switch to the kotest tag, the test is picked up OK:

val Integration = NamedTag("integration")

@Tag("integration")
class KotestIT(): FuncSpec({
    tags(Integration)
})

./gradlew test \
      -Dkotest.tags='integration & !e2e & !contract' \
      --tests my.example.KotestIT

✅
But…

[Mykola Gurov]

The problem is that this test doesn’t seem to be picked up when combining junit and kotest tags in the gradle command:

./gradlew test \
      -DincludeTestTags='integration & !e2e & !contract' \
      -Dkotest.tags='integration & !e2e & !contract' \
      --tests my.example.KotestIT

Looks like kotest-runner-junit5 doesn’t propagate the @Tag("integration"), and the test is skipped:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> No tests found for given includes: [my.example.KotestIT](--tests filter)

[Mykola Gurov]

Question: do I understand correctly, running both kotest and junit tests of the same type using tags won’t be feasible ATM?

[Mykola Gurov]

If so, is there a way to separate kotest tests from the junit w/o tagging all JUnit tests explicitly with their tag?

[sam]

Kotest tags just use the string inside the tag, so you should eb able to have a Kotest specific tag with the same name as the JUnit one, and mix and match.

[Mykola Gurov]

I do have the same string integration in my examples - both for JUnit and Kotest. Are you saying that you’d expect that to work? I got a feeling, the kotlin test is filtered out by the junit runner before the kotest tags can get into the play.

[sam]

Yes I'd expect that to work unless junit is filtering them out as you say. JUnit might do that if it considers no tags on the kotest tests because it doesn't understand what a kotest tag is. Perhaps you annotate kotest specs with both tags ?

[Mykola Gurov]

I do both.

val Integration = NamedTag("integration")

@Tag("integration")
class KotestIT(): FuncSpec({
    tags(Integration)
})

[sam]

is the @Tag a junit one ?

[sam]

I would add them both at the class level to test

[Mykola Gurov]

yes

[sam]

I would also setup gradle to not fail on no tests

[Mykola Gurov]

import org.junit.jupiter.api.Tag

[sam]

tasks.withType<Test> {
   useJUnitPlatform()
   filter {
      isFailOnNoMatchingTests = false
   }

[Mykola Gurov]

Is there other way to add the kotlin tag “at the class level” than the one I do ?

[sam]

@Tags is a class level annotation

[Mykola Gurov]

The JUnit tag is already on the class level AFAIU

[Mykola Gurov]

the

filter {
      isFailOnNoMatchingTests = false
   }

doesn’t help much - the build isn’t failing, but this time with 0 tests executed.

> Task :test

SUCCESS: Executed 0 tests in 7.4s

[sam]

can you make a standalone repo with two tests, one junit, one kotest that exhibit this behavior

[sam]

and I'll get it to work

[Mykola Gurov]

yes, sure - thanks!

[Mykola Gurov]

https://github.com/mgurov/tags-kotest-junit is the repo

[Mykola Gurov]

the evidence of the fact the Kotest tests aren’t tagged for JUnit is that the following test is picked with includeTags = setOf("none()") , and not with includeTags = setOf("integration")

import io.kotest.core.annotation.Tags

@Tags("integration")
@org.junit.jupiter.api.Tag("integration")
class KotestIT : StringSpec({

    tags(NamedTag("integration"))
    ...
})