What’s the best or proper way to use coroutines with api and database calls? The api is using ktor client (it can suspend) and the db is using jdbc (blocking) with sqldelight+hikari. Say I need 1000 api calls and then 1000 db inserts, right now I just make 1000 coroutines for the api and then 1000 for the db as the api calls return (all using Dispatchers IO). Right now main handles all the coroutines. Should they instead be in the api/db classes themselves or even in both spots? This is just a script, but would the approach change for a web app in ktor? Simplified code below:

fun main() {
    runBlocking { 
        launch {
            values.forEach {
                CoroutineScope(coroutineContext + <http://Dispatchers.IO|Dispatchers.IO>).launch {
                    val data = api.getData(it)
                    channel.send(data)
                }
            }
            for (i in 1..values.size) {
                val data = channel.receive()
                CoroutineScope(coroutineContext + <http://Dispatchers.IO|Dispatchers.IO>).launch {
                    db.insert(data)
                }
            }
        }.join()
    }
}

I would use a single coroutine for the database and a channel to send the items to insert

Wouldn’t that make the database calls execute sequentially?

Yes

I see, so if they were reads then it would make sense.

Write locks are exclusive so even with 1000 coroutines and 1000 threads executing perfectly in parallel it will be sequential

wait. maybe dumb question. but is that basically true of all SQL* databases? I never knew that it'd be sequential without anyway to write it. How do backend DBs keep up with tons of writes? 🤯

Depends on the DB, but generally you write to a log file which is append only

I should probably read up on how databases work at a fundamental level, but i was reading a topic similar to this which is funny that this came up in kotlinlang slack. as always thanks for teaching.

They're a great source of learning because performance and correctness are both critical

You can replace

CoroutineScope(coroutineContext + <http://Dispatchers.IO|Dispatchers.IO>).launch

by just

launch(<http://Dispatchers.IO|Dispatchers.IO>)

FYI.

I remember messing around with that. If I don’t have

coroutineContext

then the main thread exits immediately. I think it’s something with parent child relationship.

You'd be using the context of the nearest launch, so the behavior should be exactly the same in the snippet you initially showed.

Oh got it, my bad. If this was instead a generic

suspend fun

instead of main do I have to create a new coroutine scope?

Yes, with

coroutineScope { }