I have a data structure that represents tasks to e...

# coroutines

[CLOVIS]

I have a data structure that represents tasks to execute.

data class Task(
    val groupId: Uuid,
    val payload: …
)

I receive new tasks asynchronously. I want to execute tasks in parallel as fast as possible (= so, a channel with fan-out to many workers) but: two tasks with the same groupId may not execute in parallel. Also, tasks with a given groupId must execute sequentially in the same order as they were created.

If there were only a dozen groups, I would create a channel each, but here there are thousands of groups (if not more). It's actually very rare that two tasks currently in flight have the same group (it will probably only happen once or twice every minute), so creating that many channels just for this sounds expensive.

What's a simple way to represent this?

[PHondogo]

What if create concurrent map where key is group id and value is the list of pending tasks for each group.
For registration use map::compute and check if there were no previous list, then launch the task and return empty list from compute. Else add task to list and return it from compute.
For lunched task executor: after finish executing task call map::compute where check previous value - if list is empty, return null from compute. Else remove first element from list, return resulting list from compute and process task in the same coroutine. Do it while list remains not empty.

[CLOVIS]

That sounds doable. Do we have multiplatform non-locking concurrent maps?

[Michael Krussel]

ktor has one, but the documentation says "Please do not use it.". I'm not aware of any others. I'm not aware of any others. This is something I really wish would get ported from the JVM.

[rnett]

https://youtrack.jetbrains.com/issue/KT-78661/Provide-common-concurrent-collections-in-the-stdlib-or-kotlinx-library 😁 I asked about it on the atomics in stdlib keep and they mentioned it was possibly something they would look into after that work is done