I'm trying to execute a coroutine in a specific Co...

# coroutines

[jeggy]

I'm trying to execute a coroutine in a specific Context, but if I have understood it correctly, when using the withContext function, it waits for that coroutine to finish before continuing. But I want to launch multiple coroutines into another context and then later on, I will call join on that coroutine when I know I'm finished. How would I achieve something like this? I wrote a bare minimum example of what I'm trying to achieve here: https://pl.kotl.in/Ns6Hwa_8u

[octylFractal]

is launch(coroutineContext) not good enough?

[Casey Brooks]

withContext is usually used to change the Dispatcher, so the coroutines execute on a different thread. The lambda of that function is packed and tossed in a queue on the specified dispatcher, and the parent coroutine will continue executing in its original context.

Since you’re using runBlocking as the parent context, the join() will wait for the doLaunch calls to complete. But since runBlocking is single-threaded, it cannot exit the parent context and process the new jobs added to the queue from the withContext call. It’s deadlocked.

[Casey Brooks]

So, the problem here is with using runBlocking, not withContext

[jeggy]

Alright that wasn't the best example.
The problem I'm having is that I'm doing recursive coroutines and somehow some of them are not waited for.
I will write a new example with bit more complicated code.

[Casey Brooks]

Like a fork-join approach? You’d need to launch the sub-tasks on a different, async dispatcher. It is based on both the parent and child contexts you’re launching in. If you launch child-jobs from Dispatchers.Main onto <http://Dispatchers.IO|Dispatchers.IO>, you’ll be just fine, and you can safely join the child jobs to the parent in the Main context. But launching from Main onto Main will cause problems, again, because it is single-threaded, and the queue cannot complete the parent task to start executing the child-tasks

[jeggy]

kotlinx.serialization isn't supported on the playground, but this is exactly what I'm trying to achieve: https://pl.kotl.in/7p7j_A47S

[octylFractal]

why are you doing withContext before async? that doesn't really affect anything, async isn't a suspend function so withContext doesn't affect it

[jeggy]

If I change it to only async(this.coroutineContext) instead of withContext(this.coroutineContext){async(this.coroutineContext){block(..)}} all data get's lost

[octylFractal]

what do you expect this playground code to do with loadAllAtOnce(it).await()? you never complete any of the deferreds, so of course that suspends forever?

[octylFractal]

oh, nvm, in index = 2

[octylFractal]

this is a little confusing 🙂

[Casey Brooks]

The logic here is a bit off. The DeferredJsonMap is a map of objects, but you’re initializing it with deferred objects. That means at any given node in that tree, it’s going to synchronously wait for all objects in that node to be ready before continuing to populate the rest. But you’re awaiting a result in the middle that won’t be completed until later. Or, if you run the jobs async, the tree will be fully constructed before they get executed, and so cannot be added to the tree when they finish.

What you need to do is first construct a tree of purely deferred objects. Once all the entire deferred tree has been constructed, then go through the entire thing and join to wait for the results of all of them to be finished.

I’ve updated your example to populate the tree with deferreds first, and then complete. I think this is doing what you’re wanting. I’ve also added some logging traces which help illustrate how it’s executing https://pl.kotl.in/-BRc6p9wD

[octylFractal]

the core problem here is that withContext provides a CoroutineScope as this as well, so withContext { async {} } will suspend until the async block completes, rather than launching on the DeferredJsonMap scope

[octylFractal]

that's probably also why removing withContext changes things up

[Casey Brooks]

My previous link wasn’t entirely correct. I’ve added more logging and updated the example. It’s now clear exactly when things are set up and when they are executed, and the result should be as you expect https://pl.kotl.in/ap0xJOw0y

[jeggy]

@Casey Brooks Thanks a lot! 🤩 I will try this out and report back 👍

[jeggy]

If you could help me with one more thing that would be perfect!
I need to work with arrays also. Thinking something like this:

"someArray" toDeferredValue listOf(25, 50, 100).asDeferredArray { num ->
    delay(100)
    "id" toValue JsonPrimitive(num)
}

[Casey Brooks]

It’s mostly the same, but create a DeferredJsonArray class which holds its data in lists instead of maps, and set it up similar to the Map one. And then add a toDeferredArray method to the map class which will create the builder for it. Here’s an updated example (it could probably be cleaned up/abstracted a bit, but it looks like it works just fine) https://pl.kotl.in/FUE07Z7Lu

[jeggy]

Thanks for everything! We should make this code to a library 😉