Hi guys Curious as to how you handle async in KTor server We

Question

Hi guys Curious as to how you handle async in KTor server We have withContext <http Dispatchers IO|Dispatchers IO> on the slowest calls but not everything Do you do async on everything I suspect some requests on our end are slow because of queuing requests Is there some way to log analyse queue length I can t find anything that lets me tap into that as a metric

mp · Answer

You need <http Dispatchers IO|Dispatchers IO> when doing blocking i o not just because something is slow

mp · Answer

Code that is slow but in a coroutine friendly way won t block anything else e g a CompletableFuture that you await

Anders Sveen · Answer

Hi The context is that it is usually DB or third party services that are slow so I guess <http Dispatchers IO|Dispatchers IO> is right most of the time I do worry that I might be saturating that one as well though Do you know a way to monitor the dispatchers for queuing and times I am using New Relic so any way to expose metrics there is my preferred way but at this point I guess I ll take anything wink

gildor · Answer

gt DB or third party services that are slow so I guess is right most of the time It depends on how you work with DB and third party service doesn t do anything faster it just optimized to be blocked If you use asyncronous driver for DB and third party service accessed on HTTP using asyncronous http client one is available in ktor ktor client then you don t need any dispatchers

gildor · Answer

async access to DB is a bit more tricky not all drivers support it so if you use blocking one it s better to wrap all access to db to some dispatcher myabe IO maybe use own dispatcher

gildor · Answer

gt some way to log analyse queue length I What kind queue There is debug agent for cororutines <https github com Kotlin kotlinx coroutines tree master kotlinx coroutines debug using as jvm agent> But before use it it better to understand just what is going on in general If your code is not blocking itself no need to wrap to any dispatcher it will not help

Anders Sveen · Answer

Thanks for the detailed input and pointers Queue might be the wrong name when it comes to coroutines but I guess wait time that is not IO bound for all of the coroutines I put on the dispatcher I have looked into the async db drivers and async http We are using standard postgres jdbc and okhttp so might switch in the future but this is an established codebase so we have to do it gradually slightly smiling face

gildor · Answer

okhttp is blocking but it also has own dispatcher for async operations so if you use await for okhttp requests you should be fine <https github com square okhttp blob master okhttp src main kotlin okhttp3 Dispatcher kt>

gildor · Answer

so in general if you use blocking http requests and database requests then yes would be better to wrap them to own dispatchers but not sure that just IO is good enough strategy especially under load IO also has own limits for threads

Anders Sveen · Answer

Yep I know IO says 64 OR the number of cores So I could of course do separated dispatchers but what I am kind of missing is the information about when where I should do it So how loaded is IO right now I have an ok ish grasp of this now but find it hard to validate that the changes I do have the effect I think it should have slightly smiling face But I ll definitely look into the debug agent and see what I can find there slightly smiling face

gildor · Answer

gt So how loaded is IO right now You can just get standard JVM CPU profiling and see how many threads are busy

gildor · Answer

I recommend this talk it shows some techniques of analyzing and optimising performance on JVM and uses server app as example <https www youtube com watch v=Zw z7pjis7k> also part 2

Anders Sveen · Answer

I love Yourkit but it adds overhead so I d rather have some kind of sampling metrics that can be plugged in over time Thanks for the input I ll have a look at stuff slightly smiling face

gildor · Answer

But it probably negligible if you only monitor threads not do actual profiling

Anders Sveen · Answer

Ah that might be a good idea Thanks slightly smiling face

gildor · Answer

also any profiling is better on some level even if it s not perfect at least it worth to try if you have no idea where is bottleneck now and how your actions change performance