Hey folks :wave: I’m upgrading a project to ktor 1.5.3 and it breaks some tests when run on JVM (iO...

# ktor

[Nikolay Kasyanov]

Hey folks 👋

I’m upgrading a project to ktor 1.5.3 and it breaks some tests when run on JVM (iOS tests are fine).

In these tests, a class is tested that provides a reactive interface to its underlying HttpClient instance,
by using Reaktive helpers to “convert” coroutines into Observable (or other Reaktive types).

To do actual asserts, Reaktive helpers like TestObserver are used:

fun `it does whatever a spider cat does`() {
    // ktorMockServerManager is just a way to configure mock engine handlers      
    ktorMockServerManager.enqueueRequest(
        ...
    )

    sut.doApiStuff(...)
        .test()
        .assertComplete()
}

The tests are completely synchronous, but as soon as I update ktor to 1.5.1 (1.5.0 is fine, though), they start failing because by the time assertComplete is called, the underlying coroutine hasn’t yet finished, so the tests fail. If I add a sleep call like this:

fun `it does whatever a spider cat does`() {

    ktorMockServerManager.enqueueRequest(
        ...
    )

    val observer = sut.doApiStuff(...)
        .test()

    sleep(1000)

    observer.assertComplete()
}

I wonder what change between 1.4.3 and 1.5.1 could lead to this and what’s the way around it?

[Nikolay Kasyanov]

looks like it this might be it, the only “real” change to MockEngine since 1.4.3, added in 1.5.1: https://github.com/ktorio/ktor/commit/a6d103f6c3f13c1a136a313112a47a6af7927137#diff-3d93b6d65cea408ebd78460acc5fea1e9fb27799795984595067d5e421f939d1

[Nikolay Kasyanov]

I’d appreciate any clues here cc @e5l 🙏

[Nikolay Kasyanov]

This actually checks out: on native, Dispatchers.clientDispatcher yields Unconfined dispatcher, but on JVM it’s ClosableBlockingDispatcher

[Nikolay Kasyanov]

thus tests don’t on iOS but fail on JVM

[e5l]

I'm not sure that I can guess what coroutine is stale. It would be great to have more details. Could you file an issue with coroutines dump and reproducer?

[Nikolay Kasyanov]

I see, I’ll try get some more details, thanks!

[Nikolay Kasyanov]

Hey @e5l, I’ll look into it but not sure about the timeline.

Here’s what I can share now, by the time previously Reaktive-completable-under-test was already completed, that’s what I see in coroutine debugger. The ClosableBlockingDispatcher was added to MockEngine in ktor 1.5.1 if I remember correctly. Previously, the same coroutine would have executed immediately on the Unconfined dispatcher.

[e5l]

Yep, that’s expected. The reason it was the same with Unconfined, but we didn’t await for running jobs

[e5l]

The main question is what job is running and how to stop it

[Nikolay Kasyanov]

if I expand call-context:2, here’s what I see:

[Nikolay Kasyanov]

"call-context:2": RUNNING on thread "ktor-client-dispatcher-worker-1": RUNNING
 findLoadedClass:1281, ClassLoader (java.lang)
 loadClassOrNull:593, BuiltinClassLoader (jdk.internal.loader)
 loadClass:579, BuiltinClassLoader (jdk.internal.loader)
 loadClass:178, ClassLoaders$AppClassLoader (jdk.internal.loader)
 loadClass:522, ClassLoader (java.lang)
 ByteReadChannel:45, ByteChannelCtorKt (<http://io.ktor.utils.io|io.ktor.utils.io>)
 respond:75, MockUtilsKt (io.ktor.client.engine.mock)
 respond$default:73, MockUtilsKt (io.ktor.client.engine.mock)
 invokeSuspend:37, MockHttpClientFactory$get$1$2$1 (com.careem.captain.common.networking.mockserver)
 execute:61, MockEngine (io.ktor.client.engine.mock)
 invokeSuspend:86, HttpClientEngine$executeWithinCallContext$2 (io.ktor.client.engine)
 execute:61, MockEngine (io.ktor.client.engine.mock)
 invokeSuspend:86, HttpClientEngine$executeWithinCallContext$2 (io.ktor.client.engine)
 resumeWith:33, BaseContinuationImpl (kotlin.coroutines.jvm.internal)
 run:106, DispatchedTask (kotlinx.coroutines)
 runSafely:571, CoroutineScheduler (kotlinx.coroutines.scheduling)
 executeTask:750, CoroutineScheduler$Worker (kotlinx.coroutines.scheduling)
 runWorker:678, CoroutineScheduler$Worker (kotlinx.coroutines.scheduling)
 run:665, CoroutineScheduler$Worker (kotlinx.coroutines.scheduling)

[e5l]

could you share the code of test as well?

[Nikolay Kasyanov]

let me try…

[e5l]

It looks like some coroutine from the test scope is not finished

[e5l]

You actually can setup CoroutineTimeout for test to get test timeout for all hanging coroutines

[e5l]

It will also print you all coroutines stack traces

[Nikolay Kasyanov]

good to know, thanks! I’ll try to create a reproducer, should be easier than I initially assumed

[Nikolay Kasyanov]

sending the reproducer to you as a direct message

[e5l]

You also can file a private issue in YT, and share it only with Ktor team

[louiscad]

Or make the issue public, but attach the reproducer privately 🙂

[Nikolay Kasyanov]

sounds good

[Nikolay Kasyanov]

https://youtrack.jetbrains.com/issue/KTOR-2629