I’m using `runTest` to perform a test of some Grap...

# coroutines

[Sam Pengilly]

I’m using runTest to perform a test of some GraphQL client logic (mocking the underlying Apollo code and testing my own behaviours on top). This test has been fine for months, but for some reason it’s now acting flaky in the CI environment (and seemingly only in the CI environment), triggering the “UncompletedCoroutinesError: After waiting for 10s, the test coroutine is not completing” error. I can’t see anything in the body of the test that would be causing a hung coroutine. Could I get a sanity check on it from someone?

@Test
fun `should return parsed data upon network success`() = runTest {
    // Mocks the `call.execute()` suspending function to return an immediate result

    val call = successApolloCall(data = TestQueryData, errors = null)
    val mockApolloClient = mockk<ApolloClient> { every { query(TestQuery) } returns call }

    // executeQuery just wraps the Apollo query/execute calls to add retry behaviours and catching any exceptions using Arrow's Either
    // The retry policy uses Arrow's Scheduler which just uses delay() under the hood and should get skipped in test execution?

    val result = MyWrappingClient(mockApolloClient).executeQuery(TestQuery) { data, errors ->
        // Part of my client API that takes GraphQL response/error data and parses it
        (data == TestQueryData && errors == null).right()
    }

    result shouldBeRight true
    coVerify(exactly = 1) { call.execute() }
}

[Sam Pengilly]

For additional context, peer tests which are all in a similar format pass in well under a second (with the exception of retry test). The test listed above is sometimes taking 16 seconds according to JUnit output

[Sam Pengilly]

seems like it could relate to test bootstrapping as described here? https://github.com/Kotlin/kotlinx.coroutines/issues/3270#issuecomment-1562335370

[Sam Pengilly]

Yep, looks like it’s MockK/ByteBuddy being slow to bootstrap on the first test, by moving mock setup code outside of the body of runTest {} it resolves the issue. Confirmed on a more performant machine by reducing the runTest timeout to 2s

[Rohan Maity]

How were you able to identify its mockk/byteBuddy installation setup and not particularly runTest

[Sam Pengilly]

An educated guess from the combination of the observed behaviours and issue threads above. Confirmed by running tests with mock creation inside and outside the runTest block. Seems to be that:
• runTest now has a 10s timeout, thought to be enough for most out-of-the-box cases
• Anything inside the block body lambda of runTest contributes to the time spent (though coroutine delays would get skipped)
• ByteBuddy has a known issue where the first test in a suite takes a significant amount of time to bootstrap (can be seen in my test timings in the report)

[Sam Pengilly]

runTest is just doing what it does, throwing an exception due to a timeout after 10s without knowing what caused the timeout really. Could there be some additional smarts there? Maybe? But I’d say that it’s not necessarily the responsibility of runTest to determine what operations should or should not contribute to the timeout inside its lambda body