Hi. I have problems limiting the thread pool in co...

# coroutines

[Jakub Gwóźdź]

Hi. I have problems limiting the thread pool in coroutines: I have my simple helper function:

fun <T, R> runConcurrently(nThreads: Int, params: Collection<T>, op: suspend (T) -> R): List<R> {
    return runBlocking {
        Executors.newFixedThreadPool(nThreads).asCoroutineDispatcher().use { pool ->
            params.map { p -> async(pool) { op(p) } }
        }.awaitAll()
    }
}

and it works well when I DON'T use any yielding call inside my op(), e.g:

runConcurrently(3, (1..8).toList()) { param ->
    counter.increase()
    val sleepTime = 1000L / param
    println("Started $param in thread ${Thread.currentThread().name}, counter is $counter, sleepTime is $sleepTime")

    Thread.sleep(sleepTime)
    println("Stopping $param in thread ${Thread.currentThread().name}")
    counter.decrease()
}

But as soon as I call any suspending function inside, e.g. change Thread.sleep() to delay(), it switches rest of execution to DefaultExecutor, and my pool is not limiting anymore. What am I doing wrong? And what is current best way to limit thread pool? I know newFixedThreadPool is deprecated, but there is no alternatives afaik.

[Jakub Gwóźdź]

without suspending calls:

Started 1 in thread pool-2-thread-1 @coroutine#2, counter is Counter(current=1, max=1), sleepTime is 1000
Started 2 in thread pool-2-thread-2 @coroutine#3, counter is Counter(current=2, max=2), sleepTime is 500
Started 3 in thread pool-2-thread-3 @coroutine#4, counter is Counter(current=3, max=3), sleepTime is 333
Stopping 3 in thread pool-2-thread-3 @coroutine#4, counter is Counter(current=3, max=3)
Started 4 in thread pool-2-thread-3 @coroutine#5, counter is Counter(current=3, max=3), sleepTime is 250
Stopping 2 in thread pool-2-thread-2 @coroutine#3, counter is Counter(current=3, max=3)
Started 5 in thread pool-2-thread-2 @coroutine#6, counter is Counter(current=3, max=3), sleepTime is 200
Stopping 4 in thread pool-2-thread-3 @coroutine#5, counter is Counter(current=3, max=3)
Started 6 in thread pool-2-thread-3 @coroutine#7, counter is Counter(current=3, max=3), sleepTime is 166
Stopping 5 in thread pool-2-thread-2 @coroutine#6, counter is Counter(current=3, max=3)
Started 7 in thread pool-2-thread-2 @coroutine#8, counter is Counter(current=3, max=3), sleepTime is 142
Stopping 6 in thread pool-2-thread-3 @coroutine#7, counter is Counter(current=3, max=3)
Started 8 in thread pool-2-thread-3 @coroutine#9, counter is Counter(current=3, max=3), sleepTime is 125
Stopping 7 in thread pool-2-thread-2 @coroutine#8, counter is Counter(current=3, max=3)
Stopping 8 in thread pool-2-thread-3 @coroutine#9, counter is Counter(current=2, max=3)
Stopping 1 in thread pool-2-thread-1 @coroutine#2, counter is Counter(current=1, max=3)

[Jakub Gwóźdź]

with suspending calls (e.g. delay):

Started 1 in thread pool-2-thread-1 @coroutine#2, counter is Counter(current=1, max=1), sleepTime is 1000
Started 2 in thread pool-2-thread-2 @coroutine#3, counter is Counter(current=2, max=2), sleepTime is 500
Started 3 in thread pool-2-thread-3 @coroutine#4, counter is Counter(current=3, max=3), sleepTime is 333
Started 6 in thread pool-2-thread-1 @coroutine#7, counter is Counter(current=6, max=6), sleepTime is 166
Started 5 in thread pool-2-thread-3 @coroutine#6, counter is Counter(current=5, max=5), sleepTime is 200
Started 4 in thread pool-2-thread-2 @coroutine#5, counter is Counter(current=5, max=5), sleepTime is 250
Started 7 in thread pool-2-thread-1 @coroutine#8, counter is Counter(current=7, max=7), sleepTime is 142
Started 8 in thread pool-2-thread-3 @coroutine#9, counter is Counter(current=8, max=8), sleepTime is 125
Stopping 8 in thread kotlinx.coroutines.DefaultExecutor @coroutine#9, counter is Counter(current=8, max=8)
Stopping 7 in thread kotlinx.coroutines.DefaultExecutor @coroutine#8, counter is Counter(current=7, max=8)
Stopping 6 in thread kotlinx.coroutines.DefaultExecutor @coroutine#7, counter is Counter(current=6, max=8)
Stopping 5 in thread kotlinx.coroutines.DefaultExecutor @coroutine#6, counter is Counter(current=5, max=8)
Stopping 4 in thread kotlinx.coroutines.DefaultExecutor @coroutine#5, counter is Counter(current=4, max=8)
Stopping 3 in thread kotlinx.coroutines.DefaultExecutor @coroutine#4, counter is Counter(current=3, max=8)
Stopping 2 in thread kotlinx.coroutines.DefaultExecutor @coroutine#3, counter is Counter(current=2, max=8)
Stopping 1 in thread kotlinx.coroutines.DefaultExecutor @coroutine#2, counter is Counter(current=1, max=8)

[Jakub Gwóźdź]

Ok, nvm, I used Semaphore instead.