What is actually the difference between `StandardT...

# coroutines

[Lukasz Kalnik]

What is actually the difference between StandardTestDispatcher and UnconfinedTestDispatcher in a case where you always inject a dispatcher in the system under test?
Let's say we have a presenter, like in my post above, which always uses one dispatcher for all coroutine calls (e.g. Dispatchers.Default). For tests it gets injected a test dispatcher, so during tests every coroutine launched inside the presenter uses this injected dispatcher.
Is there a difference which dispatcher is used then?

[Lukasz Kalnik]

Is UnconfinedTestDispatcher meant as kind of a patch for bad architecture, where the dispatchers are hardcoded inside withContext calls in the code under test?

[Lukasz Kalnik]

I don't understand this explanation:

Using this TestDispatcher can greatly simplify writing tests where it's not important which thread is used when and in which order the queued coroutines are executed.

Don't the tests (in a case where you are always injecting your dispatchers) anyway always run on one thread? Isn't it kind of the point of tests to run everything sequentially on one thread?

[Lukasz Kalnik]

Also what does "event loop" mean here?

the tasks that it executes are not confined to any particular thread and form an event loop

[Joffrey]

The unconfined dispatcher just means that top-level launch/async will be immediately executed (up to the first suspension point) instead of being "queued" for later. I don't like relying on this in tests though.

[Lukasz Kalnik]

So a StandardTestDispatcher doesn't execute the coroutines? It just waits until you call runCurrent()?

[Lukasz Kalnik]

Also this (from the Dispatchers.Unconfined documentation, which has apparently the same thread switching behavior as the UnconfinedTestDispatcher):

A coroutine dispatcher that is not confined to any specific thread. It executes the initial continuation of a coroutine in the current call-frame and lets the coroutine resume in whatever thread that is used by the corresponding suspending function, without mandating any specific threading policy.

What does "lets the coroutine resume in whatever thread that is used by the corresponding suspending function" mean? Does it mean that, if there are dispatcher switches in between, e.g. using withContext, then the coroutine is resumed on the last switched thread?

[Joffrey]

So a StandardTestDispatcher doesn't execute the coroutines? It just waits until you call runCurrent()?

It does execute them, but at a moment that is more in line with what classic dispatchers do (when the top level actually suspends for instance). runCurrent() would be one way to force their immediate execution (at least until the next delay() of course), advanceUntilIdle is another way to even go through delays and reach a point when the coroutine cannot progress further.

If you write:

scope.launch {
    println("Inside coroutine")
}
println("Top level")

With a standard dispatcher, Top level will be printed first and then Inside coroutine, while with an unconfined dispatcher the body of the launch will immediately start before the top level carries on.

This can help in tests, but I personally believe it's kinda wrong to rely on this. launch is supposed to express async behaviour, so if you want to express clearly in the test that the launched coroutines should have reached a suspension point that they can't get past, I find it better to use advanceUntilIdle and then assert stuff, instead of relying on unconfined to sort of hope that coroutines will have started and reached a point sufficiently far to be acceptable

[Joffrey]

Isn't it kind of the point of tests to run everything sequentially on one thread?

It depends on the test. A lot of tests should just not care at all whether there is one or more threads, and still consistently produce the same reproducible result

[Joffrey]

Don't the tests (in a case where you are always injecting your dispatchers) anyway always run on one thread?

I believe they should, at least runTest by default runs on a single-threaded dispatcher AFAIK.

[Lukasz Kalnik]

Thank you for the explanation, that was really helpful. Can you recommend some good lecture on this topic, because the official documentation is not always so clear?

[Joffrey]

I don't think I can recommend anything unfortunately. I usually refer to the doc, and when imprecise I check the code or do little experiments to check what happens

[Joffrey]

I've seen many not-so-great resources on the internet in general, so I'm usually wary about those

[Lukasz Kalnik]

I just found this one, looks quite reliable:
https://doordash.engineering/2021/11/09/the-beginners-guide-to-kotlin-coroutine-internals/

[Dmitry Khalanskiy [JB]]

The mention of the threads in UnconfinedTestDispatcher is for these cases:

fun x() = runTest(UnconfinedTestDispatcher()) {
        println(Thread.currentThread())
        withContext(Dispatchers.Default) {
            println(Thread.currentThread())
        }
        println(Thread.currentThread())
    }

This produces

Thread[Test worker @coroutine#1,5,main]
Thread[DefaultDispatcher-worker-1,5,main]
Thread[DefaultDispatcher-worker-1,5,main]

This weird behavior is due to how the unconfined dispatcher works (and so the unconfined test dispatcher, too). In essence, if some dispatcher needs to resume a coroutine whose dispatcher is unconfined, it will not go through a dispatch and will instead execute the code directly until the next suspension point.

[Dmitry Khalanskiy [JB]]

This behavior can be useful in cases where you do want to observe some coroutine resumptions immediately, but it is also fairly surprising, and sometimes, when you need some code to execute on specific threads, it may even be wrong.

[Lukasz Kalnik]

Thank you for the explanation.
But the example under test looks for me like a bad architecture - withContext in the test has a hardcoded dispatcher.
Shouldn't we strive to have all dispatchers injected, and then they can easily be replaced for testing?

[Dmitry Khalanskiy [JB]]

This depends on what you're trying to test.

For example, if you're trying to check something like "during the execution of Z, the method X will only be called from thread *Y*"—it's common to require, for example, the UI events to happen on the Main thread—then a sensible approach would be to mock Y with a new dispatcher backed by its own thread, mock X to check that it's running on the mocked Y, and call Z. Using the unconfined dispatcher will break the test in strange ways.

Of course, one has to stray far from the beaten path to use coroutines in a manner where such things are a concern, but it's possible, and for some advanced use cases, it's needed.

[Lukasz Kalnik]

Yes, but in your example there would be also a dispatcher injection to replace Y during the test, or am I wrong?

[Lukasz Kalnik]

So essentially tests should be able to run on StandardTestDispatcher?

[Dmitry Khalanskiy [JB]]

Replacing Y with a TestDispatcher would not do the trick if X doesn't know about coroutines at all. Then, the only option is to check the value of Thread.currentThread(), so using separate threads is a necessity.

[Dmitry Khalanskiy [JB]]

Which dispatcher to use also depends on what you're trying to test. Consider this code:

@Test
    fun z1() = runTest {
        val stateFlow = MutableStateFlow(0)
        val results = mutableListOf<Int>()
        val job = launch(UnconfinedTestDispatcher(testScheduler)) {
            stateFlow.collect {
                results.add(it)
            }
        }
        stateFlow.value = 1
        stateFlow.value = 2
        stateFlow.value = 3
        assertEquals(listOf(0, 1, 2, 3), results)
        job.cancel()
    }

    @Test
    fun z2() = runTest {
        val stateFlow = MutableStateFlow(0)
        val results = mutableListOf<Int>()
        val job = launch(start = CoroutineStart.UNDISPATCHED) {
            stateFlow.collect {
                results.add(it)
            }
        }
        assertTrue(results.contains(0))
        job.cancel()
    }

z1 tests that the collect block will correctly process every value that the StateFlow may be in (in this case, will put them in a list). The intention of this test is to check that there isn't a value on which the collection procedure does anything strange. So, by selecting UnconfinedTestDispatcher, we choose the interleaving that ensures that every emitted value reaches the collect block.

z2, on the other hand, tests that launching the collect block in an undispatched manner will at least lead to the initial value being processed before a single suspension. Here, we use StandardTestDispatcher, because with UnconfinedTestDispatcher, the test would obviously pass since every value gets processed immediately.

The intentions of z1 and z2 are different: z1 ensures the correct behavior in the case of the most eager execution, whereas z2 ensures the correct behavior in the case where everything will only happen when it absolutely has to.

[Lukasz Kalnik]

That clarifies things for me, thank you.

[Lukasz Kalnik]

Just one more question: so using StandardTestDispatcher in the first test would cause the collect block to not be executed on the subsequent assignments to stateFlow.value? Why is that? Is it because StateFlow also internally launches a coroutine somehow which is not being dispatched?

[Dmitry Khalanskiy [JB]]

collect would suspend, and the .value assignment places it in the queue for resumption (instead of, as for the unconfined dispatcher, just executing the next portion of the resumed code), but since the current thread is busy with the test body—notice that the test body never suspends—there is no chance for the queue to be emptied.

Options include adding yield() after .value assignment (giving the thread a chance to run other things), or advanceUntilIdle() or runCurrent() to force the enqueued tasks to be run.

[Lukasz Kalnik]

Ah, so the reason is that executing the test body just never allows the suspended coroutines to resume!

[Lukasz Kalnik]

Now I understand...

[Lukasz Kalnik]

So you could write the first test just as well as:

@Test
    fun z1() = runTest {
        val stateFlow = MutableStateFlow(0)
        val results = mutableListOf<Int>()
        val job = launch {
            stateFlow.collect {
                results.add(it)
            }
        }
        stateFlow.value = 1
        stateFlow.value = 2
        stateFlow.value = 3
        advanceUntilIdle()
        assertEquals(listOf(0, 1, 2, 3), results)
        job.cancel()
    }

[Dmitry Khalanskiy [JB]]

Nope, the successive assignments would be conflated into a single one.

[Lukasz Kalnik]

Ah right, so then advanceUntilIdle() would have to be triggered after every .value assignment.