I'm trying to grasp the concepts behind `yield()` but I'm lost. I'd expect this code to print `[1, 2...

# coroutines

[Nino]

I'm trying to grasp the concepts behind yield() but I'm lost. I'd expect this code to print [1, 2, 3] \n [1, 2, 3, 4] \n End but it doesn't... Why ?

import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*

fun main() = runBlocking {
    val mutableStateFlow = MutableStateFlow(emptyList<Int>())

    val collectJob = launch {
        combine(
            flowOf(listOf(1, 2, 3)),
            mutableStateFlow
        ) { list1, list2 ->
            list1 + list2
        }.collect { list ->
            println(list)
        }
    }

    yield()
    mutableStateFlow.value = listOf(4)
    yield()
    collectJob.cancel()
    
    println("End")
}

Playground : https://pl.kotl.in/EpSQVK30I

PS : with delay(100) instead of yield(), it works. With lower values, I get random results.

[Nick Allen]

Just for some context, there's more that two coroutines here. combine has to collect from those two Flows concurrently which means it's launching coroutines inside and those coroutines are also contending for the thread.

yield gives up the thread, which allows other coroutines to run. There's no rule about which other coroutines actually get to run, nor any rule that all other coroutines will be scheduled to make progress before the one that called yield resumes.

It appears here that the first call to yield gives up the thread allowing the explicitly launched coroutine to run until collect suspends waiting on values to be emitted from the result of combine. The the top level coroutine resumes and the second yield gives up the thread allowing the internal implementation of combine to receive values from it's arguments and put those into a Channel. Then the Job is cancelled before your explicit collect call is able to read anything from the Channel.

To figure out what was happening when I updated collectJob and played with adding/removing yield calls:

val collectJob = launch {
        println("inner job")
        combine(
            flowOf(listOf(1, 2, 3)).onEach { println("first flow") },
            mutableStateFlow.onEach { println("second flow") }
        ) { list1, list2 ->
            list1 + list2
        }.collect { list ->
            println(list)
        }
    }

[julian]

Nice!
I put a printlln in the lambda argument to combine and it never ran, which suggests that the emissions from the two flows haven't even been put on the channel yet at the time the job is cancelled, since it's the result of the lambda that gets put on the channel. I peeked into the implementation of combine and saw that, as @Nick Allen says, the emissions are transformed after they're pulled from the channel, not before they're put on the channel, as I'd thought.

[julian]

@Nick Allen’s excellent explanation inspired me to try putting another yield after the second one. Now the result is as expected:

[1, 2, 3, 4]
End

Purely speculation, but it seems like the added (third) yield is the one that actually allows collect to become unsuspended. Whereas the second yield un-suspends machinery with combine.

[julian]

I put some breakpoints in here. I found that the second yield allows these coroutines that collect the two flows to run. And the third yield allows the outer coroutine within which these are launched to proceed past the suspension point here, where the channel is being read. This reading was suspended because, at the time it was executed, the two flow collection coroutines hadn't run yet, so the channel was empty, with nothing to read.

[Nino]

Thanks a lot for the explaination, so basically yield() is useless with Flows because depending of the flow used (or their implementation which might change with future versions), it might or might not work the same ?

[julian]

I think that's mostly right. It's not completely useless, per se. Use it as much as you want in code that you own to drive its behavior. But be cautious about using it to drive behavior of code that you don't - for the reason you stated.

As we see from the implementation of combine, the kotlinx.coroutines library uses yield to good effect, allowing combined flows to take turns emitting, i.e. to try be fair.

[Nick Allen]

yield is a false solution in general for affecting the order in which code runs. This isn't unique to Flow. Any code relying on yield is at the mercy of the implementation details of the CoroutineDispatcher being used. Methods like Job.join, Deferred.wait, Channel.receive allow you to explicitly and robustly affect the ordering of coroutine code.

[julian]

@Nick Allen Can you expand on this a bit:

Any code relying on yield is at the mercy of the implementation details of the CoroutineDispatcher being used.

[Nick Allen]

yield basically just hands control of the thread back to the CoroutineDispatcher, at which point it's up to the CoroutineDispatcher to choose what to run next. There's no guarantee that the CoroutineDispatcher won't just immediately schedule the coroutine that yielded sometimes.