Hey folks, I've got an intermittently failing mock...

# mockk

[Davis Mohar]

Hey folks, I've got an intermittently failing mockk issue that I'm curious if anybody else has ever dealt with. We have a metrics interface with a timed() function that I'm trying to mock:

interface Metrics {
    suspend fun <T> timed(name: String, f: suspend () -> T): T
}

So you would think you could mock that with:

val mockMetrics: Metrics = mockk()
val funSlot = slot<suspend () -> Either<Nothing, Transfer>>()
coEvery { mockMetrics.timed(any(), capture(funSlot)) } coAnswers { funSlot.captured() }
mockMetrics.timed("metricName") { foo() }

That is only sometimes correct, which is the worst kind of correct. Depending on how I run the tests (kotest tests running on gradle), the test containing this mock will pass sometimes. When it fails, we get a

io.mockk.MockKException: no answer found for: Metrics(#3).timed(mmp.process.transfer.latency, continuation {}, continuation {})

which implies that timed takes three arguments. I believe that's expected behavior, based on how suspend functions compile behind the scenes. But that also implies that our installed answer on our mocked timed call complied differently -- but only sometimes. Has anyone dealt with behavior like this before? My current workaround is just writing a custom little MockMetrics class instead of using mockk for it.

[Alder]

Hi Davis,

depending on how I run the tests

are there particular run strategies that always fail vs others that succeed?

[Davis Mohar]

The test passes in the following scenarios:
• clicking the 'run test' icon in IntelliJ on the individual test or the class
• right-clicking the test directory and hitting 'run tests in <service>'
• Running the individual test using the 'kotest' IntelliJ plugin (see picture: instrumented processor returns the result)
And the test fails in the following scenarios:
• running the entire module using the 'kotest' IntelliJ plugin (see picture: modules: xxx.core.test ) -- this is the super weird one to me
• running ./gradlew test
It smells like a test runner issue to me, but the fact that running the tests differently using the kotest plugin produces different results is baffling me

[Davis Mohar]

It looks to me like Issue #288 is related/similar. Doesn't explain the inconsistent behavior, but looks like a known issue is in the same ballpark.

[Alder]

I have my … work laptop up and running this morning so I’m going to try and reproduce. I feel like I’ve seen similar to this in the past. Are you able to mock appropriately without using the slot?

[Davis Mohar]

The slot does not appear to affect it. I see the same behavior when I hardcode the return value

[Davis Mohar]

Interesting- this appears to be even more inconsistent than I had initially described. The test now sometimes passes (~ 70% of the time) when I ./gradlew test or run the entire module with the kotest plugin