Reposting because I never got an answer: What's the recommended way to do a worker pool like structure? eg computing values concurrently (like by using the IO dispatcher) but only allowing a certain number to run at once

a Channel with a buffer?

I can't figure out how to make a channel work. Let's take a naive example and assume I have to launch 100 tasks:

val ch = Channel<Int>(5)
repeat(100) {
   launch {
      delay(Random.nextLong(1000))
      ch.send(it)
   }
}

It actually launches 100 coroutines immediately. It should launch 5 and start launching new ones as the channel's buffer is emptied. But the launching of the task doesn't occupy the channel's buffer, only a task finishing does.

Have you looked at the coroutines guide? https://github.com/Kotlin/kotlinx.coroutines/blob/master/docs/channels.md#fan-out

I believe he was talking about using a buffered channel on input to your coroutine

Fan-Out is how I'd do it

does sound like something that would be in the guide...

Yeah I have @withoutclass. A fan out has the same issue and I don't think it's what I'm hoping to accomplish

What issue?

Hm. I'll try it again

Check the example in the guide again that I sent, particularly:

fun main() = runBlocking<Unit> {
//sampleStart
    val producer = produceNumbers()
    repeat(5) { launchProcessor(it, producer) }
    delay(950)
    producer.cancel() // cancel producer coroutine and thus kill them all
//sampleEnd
}

Right it's just strange to think of my loop of 100 actions as a producer of the values 0-99 considering it's completely sync

They aren't the same, you have work to populate your

channel

, the

repeat(100)

. This is completely separate from your "worker` coroutines, of which you want 5

The real world example is making API requests. 5 requests should be made at once and as soon as a single one finishes, another should be launched (so there should always be 5 running). The blocking work is making the request itself

That doesn't really change anything here, you'll still want to fan-out if you're driving those 5 workers off of values in a channel

And what values would be coming through the channel in that example, the requests?

No, I assumed that your real example was somehow related to the example you've posted here

I guess that would work too, it just feels weird to rely on the thread limit instead

Here's a sample implementation https://gist.github.com/Tolriq/dcbdc62c0e29d3b321034e990c3c85ce

Thanks @Tolriq I'll take a look

Executor with limited amount of threads will not help in case of coroutines if you use asyncronous http client