https://kotlinlang.org logo
Title
m

mcpiroman

03/02/2022, 10:14 AM
This is a common pattern but I'm still unsure how to go about it: I have a prepared list of items I want to process with parallelization, is this the preferred way:
suspend fun processAll(items: List<Item>) = coroutineScope {
  val channel = Channel<Item>(Channel.UNLIMITED)
  items.forEach { channel.trySendBlocking(it).getOrThrow() }

   repeat(4) {
        launch {
            for (item in channel) // process
        }
    }
}
Bonus point - is there a facility to automatically pick an optimal level of parallelism, instead of, in this example, hardcoded 4?
s

Sam

03/02/2022, 10:18 AM
If the list is already prepared, and of known size, the channel might be overkill. You could just use something like
items.chunked(...)
to split the list into parts, and launch a separate coroutine for each chunk.
That said, you might be thinking about this as if you were dealing with threads rather than coroutines
Remember that coroutines are lightweight
The simplest solution might be to launch a coroutine for every item in the list.
That way, you'll get that maximum level of parallelism based on the dispatcher you're using.
A typical pattern would be
suspend fun processAll(items: List<Item>) = coroutineScope {
  items.map { async { process(item) } }.awaitAll()
}
m

mcpiroman

03/02/2022, 10:22 AM
The with
chuncked
thing is that each item can take different time to process, so near the end the level of parallelism will decrease
But yes, I seem to forgot the simples approach to just launch all at the same time, thanks