Hi, is is possible to uses coroutines to perform b...

# coroutines

[Ayfri]

Hi, is is possible to uses coroutines to perform big operations like comparing every pixels colors of one image to multiple to find the closest image from all the other ones in a parallel way ? (Sort of multi-threading)

[Zach Klippenstein (he/him) [MOD]]

Coroutines can execute in parallel on multiple threads, so anything you can do with threads you can do with coroutines.

[Ayfri]

Nice, do you have any example ?

[Joffrey]

You can simply scope the whole computation in coroutineScope { .. } and use multiple launch inside that do the computation you need. If you call all of this on a multithreaded dispatcher like Dispatchers.Default, those coroutines will run in parallel

[Ayfri]

But I guess that I have to split my computation so it can work within parallel coroutines so it's faster than in one thread ? I was asking if there is a way to do this "automatically"

[Joffrey]

Ah! Ok I see now. Well no there is no real magic here, it will depend on your computation and its parallelizability

[Joffrey]

In your case you could at least very easily launch the comparison of the image with each other image in a separate coroutine. Then, if you only have very few images to compare (less than the number of threads), you could also make a single comparison parallel if the image itself is big enough for it to be useful.

[Ayfri]

well I have thousands of images (all 16x16 so not very big but a lot), so I guess I can try to use multiple coroutines and batches of images

[Joffrey]

Technically thousands of coroutines are pretty OK and you might not even have to implement batching yourself. The coroutines will be dispatched on a smaller number of threads automatically. But either way is totally doable.

[Ayfri]

oh okay, I didn't know this much coroutines wasn't finally that big, thanks for helping me !

[Joffrey]

It does have a little bit of overhead, of course, but way less than threads. And it's also easy to implement batching (using chunked() for instance, and passing a chunk to a coroutine). So you can for sure do that.

Yet another option is to use a Channel and spawn the number of coroutines you want to process that channel.

[Ayfri]

What are channels ?

[Joffrey]

It's a communication primitive for coroutines, which abstract away all synchronization etc. It's basically a buffer to which you can send elements (send is suspending, and it suspends when the buffer is full), and other coroutines can receive elements from the channel.

If many coroutines iterate on the same channel, the elements will be sent to them in a fan-out fashion, meaning that each item will only be sent to one of the consuming coroutines (so it's what you want).

[Joffrey]

So in your case this means you could have one coroutine send all the images to compare to a channel, and N coroutines reading elements from this channel and performing the distance operation, and maybe sending the "measured images" to a second channel so the initial sender can combine the results.

[Joffrey]

The initial approach I had in mind was the following.

Assuming you have this function:

fun distance(img1: Image, img2: Image): Long = TODO()

You could write this:

// returns null if the list is empty, you could also throw an exception instead
suspend fun findClosestImageOrNull(src: Image, candidates: List<Image>): Image? {

    // you can also define this helper class outside the function if needed elsewhere
    data class MeasuredCandidate(val img: Image, val distance: Long)

    val measuredCandidates = coroutineScope {
        candidates
            .map { candidate ->
                async { MeasuredCandidate(candidate, distance(src, candidate)) }
            }
            .awaitAll()
    }
    return measuredCandidates.minByOrNull { it.distance }?.img
}

This defines the computation in a concurrent way. If you want to run this in parallel, simply run findClosestImageOrNull in a multi-threaded dispatcher such as `Dispatchers.Default`:

suspend fun main() {
    withContext(Dispatchers.Default) {
        findClosestImageOrNull(...)
    }
}

You could also replace coroutineScope with withContext(Dispatchers.Default) right inside findClosestImageOrNull , but this prevents the caller from choosing another dispatcher (e.g. in tests).

That said, because the distance function is not suspending, the coroutines will be run sequentially if you run this function on a single-threaded dispatcher, so choosing a good dispatcher inside the function might avoid mistakes on the calling side. Up to you!

[Ayfri]

Wow thanks for these big explaination, I understand much more coroutines now, I'll try this and see if it is faster than before ^^