Hi guys. Curious as to how you handle async in KTo...

# ktor

[Anders Sveen]

Hi guys. Curious as to how you handle async in KTor server. We have withContext(Dispatchers.IO) {...} on the slowest calls but not everything. Do you do async on everything? I suspect some requests on our end are "slow" because of queuing requests. Is there some way to log/analyse queue length? I can't find anything that lets me tap into that as a metric...

[mp]

You need Dispatchers.IO when doing blocking i/o, not just because something is slow

[mp]

Code that is slow, but in a coroutine friendly way, won't block anything else (e.g. a CompletableFuture that you await())

[Anders Sveen]

Hi. The context is that it is usually DB or third party services that are slow so I guess Dispatchers.IO is right most of the time? I do worry that I might be saturating that one as well though... Do you know a way to monitor the dispatchers for queuing and times? I am using New Relic so any way to expose metrics there is my preferred way, but at this point I guess I'll take anything. 😉

[gildor]

DB or third party services that are slow so I guess Dispatchers.IO is right most of the time

It depends on how you work with DB and third party service. Dispatcher.IO doesn’t do anything faster, it just optimized to be blocked

If you use asyncronous driver for DB and third party service accessed on HTTP using asyncronous http client (one is available in ktor - ktor-client), then you don’t need any dispatchers

[gildor]

async access to DB is a bit more tricky, not all drivers support it, so if you use blocking one, it’s better to wrap all access to db to some dispatcher, myabe IO, maybe use own dispatcher

[gildor]

some way to log/analyse queue length? I

What kind queue?
There is debug agent for cororutines
https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-debug#using-as-jvm-agent

But before use it, it better to understand just what is going on in general. If your code is not blocking itself, no need to wrap to any dispatcher, it will not help

[Anders Sveen]

Thanks for the detailed input and pointers. Queue might be the wrong name when it comes to coroutines, but I guess wait time (that is not IO bound) for all of the coroutines I put on the dispatcher. I have looked into the async db drivers and async http. We are using standard postgres jdbc and okhttp so might switch in the future, but this is an established codebase so we have to do it gradually. 🙂

[gildor]

okhttp is blocking, but it also has own dispatcher for async operations, so if you use await() for okhttp requests you should be fine
https://github.com/square/okhttp/blob/master/okhttp/src/main/kotlin/okhttp3/Dispatcher.kt

[gildor]

so in general, if you use blocking http requests and database requests, then yes, would be better to wrap them to own dispatchers, but not sure that just IO is good enough strategy especially under load, IO also has own limits for threads

[Anders Sveen]

Yep, I know. IO says 64 OR the number of cores. So I could of course do separated dispatchers, but what I am kind of missing is the information about when/where I should do it. So how loaded is IO right now. I have an ok-ish grasp of this now, but find it hard to validate that the changes I do have the effect I think it should have. 🙂 But I'll definitely look into the debug agent and see what I can find there. 🙂

[gildor]

So how loaded is IO right now

You can just get standard JVM CPU profiling and see how many threads are busy

[gildor]

I recommend this talk, it shows some techniques of analyzing and optimising performance on JVM (and uses server app as example)

https://www.youtube.com/watch?v=Zw_z7pjis7k▾

(also part 2)

[Anders Sveen]

I love Yourkit, but it adds overhead so I'd rather have some kind of sampling/metrics that can be plugged in over time. Thanks for the input, I'll have a look at stuff. 🙂

[gildor]

But it probably negligible if you only monitor threads, not do actual profiling

[Anders Sveen]

Ah, that might be a good idea. Thanks. 🙂

[gildor]

also any profiling is better on some level, even if it’s not perfect, at least it worth to try if you have no idea where is bottleneck now and how your actions change performance