This is a common pattern but I m still unsure how to go about it I have a prepared list of items I want to process with parallelization is this the preferred way ```suspend fun processAll items List lt Item gt = coroutineScope val channel = Channel lt Item gt Channel UNLIMITED items forEach channel trySendBlocking it getOrThrow repeat 4 launch for item in channel process ``` Bonus point is there a facility to automatically pick an optimal level of parallelism instead of in this example hardcoded 4

This is a common pattern but I'm still unsure how to go about it: I have a prepared list of items I want to process with parallelization, is this the preferred way:

suspend fun processAll(items: List<Item>) = coroutineScope {
  val channel = Channel<Item>(Channel.UNLIMITED)
  items.forEach { channel.trySendBlocking(it).getOrThrow() }

   repeat(4) {
        launch {
            for (item in channel) // process
        }
    }
}

Bonus point - is there a facility to automatically pick an optimal level of parallelism, instead of, in this example, hardcoded 4?

Sam

03/02/2022, 10:18 AM

If the list is already prepared, and of known size, the channel might be overkill. You could just use something like

items.chunked(...)

to split the list into parts, and launch a separate coroutine for each chunk.

That said, you might be thinking about this as if you were dealing with threads rather than coroutines

Remember that coroutines are lightweight

The simplest solution might be to launch a coroutine for every item in the list.

That way, you'll get that maximum level of parallelism based on the dispatcher you're using.

A typical pattern would be

suspend fun processAll(items: List<Item>) = coroutineScope {
  items.map { async { process(item) } }.awaitAll()
}

mcpiroman

03/02/2022, 10:22 AM

The with

chuncked

thing is that each item can take different time to process, so near the end the level of parallelism will decrease

But yes, I seem to forgot the simples approach to just launch all at the same time, thanks

3 Views

#coroutines Join Slack