And yes, I already tried doing this:

// Maximum number of socket connections.
				this.setMaxConnTotal(100_000)

				// Maximum number of requests for a specific endpoint route.
				this.setMaxConnPerRoute(100_000)

...but it didn't fix the issue 😞

There is a

http.engine.config.threadsCount

which by default is set to 4, but I'm not sure that this is the variable that should be changed.

It isn't the coroutine dispatcher itself (created in my application with a cached thread pool) that is running out of threads because other stuff that relies on coroutines work fine

May take a while because well... this issue is very rare (only happens in one of my instances and happens sometimes after restarts during peak hours, rarely happens after the bootup) I think this happens because the instance that has the issue is the main instance and it receives http callbacks from other services (like YouTube videos notifications) and relays the request to my other five instances. Because Google/YT keeps sending notifications if the instance is down, after the instance restarts it receives a lot of notifications and relays them to the other instances... causing that issue I described above where

@e5l updating to ktor 1.3.0 didn't fix the issue 😞

I decided to see if increasing the

threadsCount

would fix the issue

val http = HttpClient(Apache) {
        this.expectSuccess = false

        engine {
            this.socketTimeout = 25_000
            this.connectTimeout = 25_000
            this.connectionRequestTimeout = 25_000

            customizeClient {
                // Maximum number of socket connections.
                this.setMaxConnTotal(100_000)

                // Maximum number of requests for a specific endpoint route.
                this.setMaxConnPerRoute(100_000)
            }
        }
    }.apply {
        this.engineConfig.threadsCount = 128
    }

...and it looks like it fixed the issue, or well, I wasn't able to reproduce it now. Nevermind, the issue still persists. I'm trying to get a thread dump with

jstack

when the issue happens to see if the threads are all stuck when this happens (or not).

I will try investing the issue a bit more, it seems very strange and to be honest... I just can't find any clues pointing to why it breaks sometimes. (I thought that maybe the webserver is getting stuck but not the client... but accessing the same endpoint with the browser or any other JVM instance works fine) I could also try changing client engines just to see if the issue goes away.

@e5l after a lot of investigation I found out that it wasn't ktor's issue, it was due to two unrelated issues: My webserver is sometimes getting stuck/taking waaaay too long while compiling Kotlin Scripts (I need to investigate that later) so a lot of requets were getting stuck; The reason it was "queueing requests" and "the timeout only starts after the request was executed" was because I was using

GlobalScope

without a custom dispatcher, so sometimes the default GlobalScope coroutine dispatcher was full and coroutines start being queued. (and that explains the "queueing requests"-like behavior) 😭 Anyway, thanks for helping 🙂