https://kotlinlang.org logo
Title
a

Ayfri

12/06/2021, 6:19 AM
Hi, is is possible to uses coroutines to perform big operations like comparing every pixels colors of one image to multiple to find the closest image from all the other ones in a parallel way ? (Sort of multi-threading)
:yes: 2
z

Zach Klippenstein (he/him) [MOD]

12/06/2021, 6:49 AM
Coroutines can execute in parallel on multiple threads, so anything you can do with threads you can do with coroutines.
a

Ayfri

12/06/2021, 12:36 PM
Nice, do you have any example ?
j

Joffrey

12/06/2021, 12:46 PM
You can simply scope the whole computation in
coroutineScope { .. }
and use multiple
launch
inside that do the computation you need. If you call all of this on a multithreaded dispatcher like
Dispatchers.Default
, those coroutines will run in parallel
a

Ayfri

12/06/2021, 12:54 PM
But I guess that I have to split my computation so it can work within parallel coroutines so it's faster than in one thread ? I was asking if there is a way to do this "automatically"
j

Joffrey

12/06/2021, 12:55 PM
Ah! Ok I see now. Well no there is no real magic here, it will depend on your computation and its parallelizability
In your case you could at least very easily launch the comparison of the image with each other image in a separate coroutine. Then, if you only have very few images to compare (less than the number of threads), you could also make a single comparison parallel if the image itself is big enough for it to be useful.
a

Ayfri

12/06/2021, 1:19 PM
well I have thousands of images (all 16x16 so not very big but a lot), so I guess I can try to use multiple coroutines and batches of images
j

Joffrey

12/06/2021, 1:20 PM
Technically thousands of coroutines are pretty OK and you might not even have to implement batching yourself. The coroutines will be dispatched on a smaller number of threads automatically. But either way is totally doable.
a

Ayfri

12/06/2021, 1:22 PM
oh okay, I didn't know this much coroutines wasn't finally that big, thanks for helping me !
j

Joffrey

12/06/2021, 1:25 PM
It does have a little bit of overhead, of course, but way less than threads. And it's also easy to implement batching (using
chunked()
for instance, and passing a chunk to a coroutine). So you can for sure do that. Yet another option is to use a
Channel
and spawn the number of coroutines you want to process that channel.
a

Ayfri

12/06/2021, 1:27 PM
What are channels ?
j

Joffrey

12/06/2021, 1:29 PM
It's a communication primitive for coroutines, which abstract away all synchronization etc. It's basically a buffer to which you can
send
elements (
send
is suspending, and it suspends when the buffer is full), and other coroutines can receive elements from the channel. If many coroutines iterate on the same channel, the elements will be sent to them in a fan-out fashion, meaning that each item will only be sent to one of the consuming coroutines (so it's what you want).
So in your case this means you could have one coroutine send all the images to compare to a channel, and N coroutines reading elements from this channel and performing the distance operation, and maybe sending the "measured images" to a second channel so the initial sender can combine the results.
The initial approach I had in mind was the following. Assuming you have this function:
fun distance(img1: Image, img2: Image): Long = TODO()
You could write this:
// returns null if the list is empty, you could also throw an exception instead
suspend fun findClosestImageOrNull(src: Image, candidates: List<Image>): Image? {

    // you can also define this helper class outside the function if needed elsewhere
    data class MeasuredCandidate(val img: Image, val distance: Long)

    val measuredCandidates = coroutineScope {
        candidates
            .map { candidate ->
                async { MeasuredCandidate(candidate, distance(src, candidate)) }
            }
            .awaitAll()
    }
    return measuredCandidates.minByOrNull { it.distance }?.img
}
This defines the computation in a concurrent way. If you want to run this in parallel, simply run
findClosestImageOrNull
in a multi-threaded dispatcher such as `Dispatchers.Default`:
suspend fun main() {
    withContext(Dispatchers.Default) {
        findClosestImageOrNull(...)
    }
}
You could also replace
coroutineScope
with
withContext(Dispatchers.Default)
right inside
findClosestImageOrNull
, but this prevents the caller from choosing another dispatcher (e.g. in tests). That said, because the
distance
function is not suspending, the coroutines will be run sequentially if you run this function on a single-threaded dispatcher, so choosing a good dispatcher inside the function might avoid mistakes on the calling side. Up to you!
a

Ayfri

12/06/2021, 10:42 PM
Wow thanks for these big explaination, I understand much more coroutines now, I'll try this and see if it is faster than before ^^
👍 1