Testbleeding with @TestFactory: Hi, I am trying ...

# mockk

[huehnerlady]

Testbleeding with @TestFactory:

Hi,

I am trying to minimize duplications in tests, so I want to Use JUnit5's TestFactory.
It works fine for everything, unless it concerns mokk mocks.

Example

import io.mockk.every
import io.mockk.mockk
import io.mockk.verify
import org.junit.jupiter.api.DynamicTest
import org.junit.jupiter.api.TestFactory

class Test {

  private val testService: TestService = mockk(relaxed = true)


  private val testStrings = listOf("1", "2")

  class TestService {
    fun test(string: String): String {
      return string
    }
  }

  @TestFactory
  fun tests() = testStrings
      .map { input: String ->
        DynamicTest.dynamicTest("should call service once for '$input'") {
          every {
            testService.test(any())
          } returns "foo"

          testService.test(input)

          verify(exactly = 1) { testService.test(any()) }
        }
      }
}

As you can see in the test, I am calling the mock just once. still, the testoutput is as shown on the screenshot. The second tests always fails.

What am I doing wrong here? Can anybody help me? 🙂

I am using mokk 1.10.0, kotlin 1.3.72 and junit 5.6.2

[thanksforallthefish]

if I am not mistaken, test factories creates dynamic tests, which means junit only starts a context once. this means your mock is not reset, thus keeping previous counters.

I don’t remember anymore if @AfterEach can help you

fun @AfterEach() cleanUp { clearAllMocks() }

otherwise you can just add clearAllMocks() after your verify

EDIT: forgot fun name 🙂

[huehnerlady]

@thanksforallthefish many thanks, I did not know that. This is really annoying, as I just want to prevent test duplication 😞
I though I found some nice workaround in junit5 for the behaviour of spock’s where. But I guess I have to look further

[christophsturm]

you can try something like #minutest or #kotest

[thanksforallthefish]

I am currently using kotest btw and I find it great. however, this particular would still be there. kotest otoh offers several testing style, you might find one that approximate spock’s where (though I just know spock by name, not sure what where even is)

[christophsturm]

with minutest and kotest you can just create your tests in a loop like in the test factory, and you get before each and after each for free

[huehnerlady]

super cool, I will try kotest for that now, thanks guys 🙂

[huehnerlady]

@christophsturm is this the way you would do that? At least this is what I will try out now 🙂

seems to do the trick (with FreeSpec) so far

[christophsturm]

I’m a minutest user myself and would probably do it with a loop. with kotest you can probably also do a data driven test, which looks even better. https://github.com/kotest/kotest/blob/master/doc/data_driven_testing.md

[sam]

Here is your example converted to Kotest @huehnerlady

You can just loop around like @christophsturm mentioned. Create the tests and fixtures inside the loop.

I've used FunSpec as that's what I prefer, but you can use FreeSpec or anything else.

class Test : FunSpec() {

   init {
      val testStrings = listOf("1", "2")
      testStrings.forEach { input ->
         test("should call service once for '$input'") {
            
            val testService: TestService = mockk(relaxed = true)
            
            every {
               testService.test(any())
            } returns "foo"
            
            testService.test(input)
            
            verify(exactly = 1) { testService.test(any()) }
         }
      }
   }

   class TestService {
      fun test(string: String): String {
         return string
      }
   }
}

[sam]

And if your TestService has multiple inputs per test, you can consider data driven tests which group each set of inputs together. Something like:

class Test : FunSpec() {
   init {
      test("should call service once") {
         forAll(
            row("1", true, 5.4),
            row("2", false, 14.2)
         ) { a, b, c ->
            val testService: TestService = mockk(relaxed = true)
            every {
               testService.test(any())
            } returns "foo"
            testService.test(a, b, c)
            verify(exactly = 1) { testService.test(any()) }
         }
      }
   }

   class TestService {
      fun test(string: String, boolean: Boolean, double: Double): String {
         return string
      }
   }
}

[huehnerlady]

@sam many thanks. this looks awesome. I actually started testig kotest last week and I love it. I am going to migrate all tests now, thanks for your help 🙂

[huehnerlady]

You used the FunSpeck. There are so many different specs, so I was wondering, why are you using FunSpec? 🙂 I had a look and so far FreeSpec looks very appealing to me, so just wanted to make sure that there isn’t anything important I miss as a difference 🙂

[sam]

It's just preference really.

[huehnerlady]

Ah ok, thanks 🙂