I'm launching a million coroutines on <Dispatchers.IO> that do an HTTP post and then doing a joinAll...

# coroutines

[Luis Munoz]

I'm launching a million coroutines on Dispatchers.IO that do an HTTP post and then doing a joinAll but the loop gets stuck (posts stop being sent out). If I put a delay of 2 ms between each launch then it seems to work. Would it better to create my own thread pool? Or anyone have a guess of what the problem could be? Simplified Code:

var scope = CoroutineScope(<http://Dispatchers.IO|Dispatchers.IO> + CoroutineName("myLoop"))
 job = scope.launch {
          while (this.isActive) {
                (0 until appCfg.count).map {
                    delay(delayBetweenDeviceStarts)
                    val sn = appCfg.startSn + it
                    launch{
                      //  DO SOME HTTP OPERATIONS
                    }
                }.joinAll()
          
            } 
}

[TwoClocks]

I'm not quite sure I understand what's happening. A few questions : Why launch twice?

[TwoClocks]

Also, what do you mean by "the loop gets stuck"?

[TwoClocks]

what are you hitting w/ your HTTP requests? You sure it has a backlong deep enough to handle 2 million requests? You are you arn't eating a IoException someplace?

[Luis Munoz]

The first launch creates a while loop that runs "forever", the loop continuously launches 1million HTTP post requests and waits, then does it again,.

Loop gets stuck means post requests stop going out. Don't know why, I see no errors. I have a try catch

launch{
 try {
//  DO SOME HTTP OPERATIONS
} catch(ex: Exception) { 

 }

Good point about the IOException, I probably should be catching Throwable not Exception just to make sure

[Luis Munoz]

the servers are known to be able to handle 5 million requests so for sure it isn't the server

[TwoClocks]

You never know, if the accept queue isn't configured big enough, a server can stop accepting connections if to many sockets try to connect at once.

[TwoClocks]

I bet your getting a IoException in your HTTP logic and you've eaten it.

[Lukas Lechner]

Maybe it has to do with the networking library that you use to post your requests? It’s probably using a ThreadPool under the hood and at some point it might exhaust the thread pool?