https://kotlinlang.org logo
Join Slack
Powered by
Hello again with a different question Today. :disa...
# ktor
h
Hello again with a different question Today. 😞 We have an application that it is currently running in production and it usually fires ~7k requests every minute to a server. The service is using Ktor client 
2.3.6
withe the 
CIO
engine. We noticed that after ~5 days we start to see degradation in performance, such as latency increasing etc. We decided to extract the 
jmap
and we noticed that the number of thread ids in the 
dispatchers
are quite high, see screenshot for example. We are thinking that the issue might be related to the performance of the connections, since despite the 
HttpRequestTimeouts
we see a lot of 
ConnectionTimeoutExceptions
. We were thinking that maybe by having a Connection Pool would help, but I could not find any information in the documentation about the best practice to handle these type of improvements... I did however, found in the channel that 
pipelining=true
option should be able to help, but I don't have much context about what it does or know if this could improve somehow the performance of the application. Do you have any recommendation for such scenarios like that to improve the performance of the client?
Copy code
engine {
            pipelining = true
            endpoint {
                maxConnectionsPerRoute = 500
            }
        }
👀 1
Also, when it comes to 
maxConnectionPerRoute
am I right to say that for the example below, this is going to be accountable for 2 different routes, which means the client could fire up 500 requests to 
$1
and 
$2
? Or Ktor Client will go up to 
/users
? 
<https://domain.com/api/v1/users/$1>
<https://domain.com/api/v1/users/$2>
a
so what's the problem? is it high number of threads or high number of active TCP/IP connections?
h
Based on the JVM information, we don't have a lot of active threads. I got really suspicious by the thread ids though... it seems that they are not being reused as they should apparently.
In regards to the TCP connections, the service doesn't receive a lot. It actually sends a lot of requests to a different server. I did not find any issue related to it. 🤔
e
Hey @Helio, could you please log an issue with a sample?
h
Hello @e5l, I would be more than happy to raise an issue. But first, prior assuming this is an issue with the client itself, based on what I said above, do you reckon is there any optimisation we could probably be looking at, such as 
pipelining
(which I'm not entirely confident on how this is used for since I could not find a lot in documentation)... maybe increasing the 
timeAlive
of the connections, etc? At the moment, I'm still trying to prove / investigate what could be happening with the service.
e
Could you try setting 
kotlinx.coroutines.io.parallelism
to values higher than 64?
h
I could absolutely try to set that. I saw that in the thread, but I was quite confused if that was related to the CIO engine. Does this flag should be set as JVM flags?
I will raise a PR by increasing the parallelism. But would you be able to give the TL;DR on the why this might work? I got very confused with information of the thread. Also, do you guys recommend any practice for Connection Pooling? Since it would be inviable every minute create thousands of connection to send requests to the server? the 
pipelining
option is really a thing? or not? Btw, thanks for trying to assist @e5l
By the way @e5l, we've deployed the change Today by increasing parallelism to 128. We will be monitoring closely for next days and I will let you know how it goes. 😉 If you could just give some insights on the questions I asked above it would be great, please. Because I could not find anything about that in any docs. 😞 (That's the main reason I'm asking about those properties / best practices).
e
Hey, I'm not sure if it's the same problem with pooling. Investigating at the moment
h
No worries at all! I really appreciate your help here.
Hello Leonid, just double-checking if you found anything?
e
Sorry for the delay, I’m working on it
👍🏽 1
managed all details, working on the fix
👀 2
h
Interesting.. What exactly did you find? Super keen on understanding it more if possible, please. Is this what you were working on? https://github.com/ktorio/ktor/compare/main...e5l/cio-connection-close BTW, thanks once again for looking into it
🚫 1
Is there any chance you could share a little bit more details? We are super curious to understand what the issue is. 😃
e
At the moment we pool connections only in case of enabled pipelining. I'm going to move connection handling in the factory and enable pooling
h
Got it, Thanks for explaining! So in my case, probably it might worth setting pipelining to true in the meanwhile? OOC, what's the purpose of pipilining? Is there any other benefit than the connection pool?
e
Pipelining can schedule multiple requests without awaiting responses to arrive. It increases throughput, but also have downsides. You can check more info here: https://en.wikipedia.org/wiki/HTTP_pipelining
👀 1
h
Interesting... so, based on what you're saying we dont have a lot of options despite waiting for the fix, or consider setting pipilining to true correct? Do you know if Ktor has any documentation around the connection pooling? I'd love to know more about it... if it doesn't have, do you know if it is possible for ourselves to configure it in the client? Or any tip for best practices around the CIO? Amazing working btw Leonid!
Hello again Leonid! Quick question, do you have any update on this? I've left our application running for sometime ~10days now, and even with 
pipelining=true
, we are still seeing a high number of threads.
It is worth mentioning that we haven't upgraded to 2.3.7 yet though.
Hello @e5l, Happy new year. I just wanted to do a quick follow-up on this issue... Do you guys have any update or anywhere else I could track the issue you've found? Even after enabling the 
pipelining=true
we are still seeing the latency increase from the HTTP Client.
35 Views
Open in Slack
PreviousNext
Powered by