Just to understand better what I'm doing: what's t...

# ktor

[Edoardo Luppi]

Just to understand better what I'm doing: what's the effective role of the dispatcher in SelectorManager(dispatcher = <http://Dispatchers.IO|Dispatchers.IO>)?
In my KMP library, I'm wrapping Ktor in a way similar to:

override suspend fun connect(address: NetAddress): ServerProperties {
  selectorManager = SelectorManager(dispatcher)

  val tcp = aSocket(selectorManager).tcp()
  socket = tcp.connect(address.host, address.port)
  ...

I suppose a SelectorManager should be used to create multiple socket connections, so having one SelectorManager instance per socket instance isn't strictly correct.

[Oleg Yukhnevich]

AFAIR, SelectorManager on JVM and Native behaves differently regarding dispatcher:
• on Native, dispatcher is ignored and separate Worker is created for handling of non-blocking sockets selection
• on JVM, it uses dispatcher and fully blocks (in a blocking loop) one thread from dispatcher (that's why you need IO and not Default)
And yes, one selector should be enough to handle multiple sockets (client/server)
You can check on how ktor CIO engine (client or server) is implemented to get more insights

[Edoardo Luppi]

Thanks @Oleg Yukhnevich. To clarify further, what I offer in this library is a multiplatform implementation of a TCP protocol.
My confusion here spawns from the fact I'm not sure which thread pool / dispatcher should be passed to the SelectorManager.

When I instantiate the JVM socket client abstraction, e.g. MyJVMSocketClient(), I instantiate it using a custom coroutine scope, and a custom dispatcher that limits to a single thread. That is, each MyJVMSocketClient instance should use one thread at most.

public actual fun Dispatchers.forSocket(): CoroutineDispatcher =
  IO.limitedParallelism(1)

However, currently, inside MyJVMSocketClient.connect I'm doing

override suspend fun connect(address: NetAddress): ServerProperties {
  selectorManager = SelectorManager(dispatcher = <http://Dispatchers.IO|Dispatchers.IO>)

Which is obviously wrong from a logical perspective, because I'm not even re-using the forSocket() dispatcher.

[Edoardo Luppi]

Probably what I should use is

selectorManager = SelectorManager(currentCoroutineContext())

This would ensure the context is set outside of the socket client abstraction

[Oleg Yukhnevich]

be careful with multiple SelectorManager instances, as it could suffer performance. As I said, on JVM it will block one thread - and using it for one socket will be very not efficient

I'm not sure which thread pool / dispatcher should be passed to the SelectorManager.

I would say, it's better to pass <http://Dispatchers.IO|Dispatchers.IO> like in your example, it should work fine, though, It still would be better to reuse it per sockets
Why do you need separate dispatcher for socket with io.limitedParallelism(1)?

[Edoardo Luppi]

> Why do you need separate dispatcher for socket with io.limitedParallelism(1)?
That's actually a good question, in the sense that I don't recall why it was done like that a long time ago.
It was probably done because of the misunderstanding between concurrency and parallelism, or probably because the idea was to constrain access to resources for each active socket connection

[Edoardo Luppi]

constrain access to resources

In the sense we did not want to let coroutines use/spawn N threads freely, we wanted to use the bare minimum.

[Oleg Yukhnevich]

okay, I don't know about all your requirements, so I would just write some suggestion, and then you will be able to decide 🙂
1. use single SelectorManager per all your client sockets, it should be closed to free resources after all client sockets are closed - or you will have a deadlock
2. use SelectorManager(IO) + handle sockets on IO until it becomes a problem
3. If for some reason you will see, that there are some problems with performance, I would suggest to create a separate dispatcher or via IO.limited(N) (though, I haven't tested it, but most likely it should work) or via newFixedThreadPoolContext (don't forget to close it) , where N > 1 (as 1 thread will be blocked) to reduce amount of threads used (and so memory and thread-switching)
4. Don't create a lot of SelectorManager instances, as it will cause a lot of platform threads to be just blocked, and so reduce performance (may be Virtual Threads will work fine here, but I haven't tested it)
5. it's fine to use different dispatcher for SelectorManager and sockets handling, but it will cause additional thread-switching, which could or couldn't affect your use case 🙂
6. Before replacing IO with something else - try to understand the problem better and if possible do some benchmarking/stress testing
If you have an open-source repository somewhere, it could help to understand better what do you want to achieve 🙂
Still, ktor-network is really not so easy to use, so don't be shy to ask questions here, I will try to help with all my knowledge received during development/research for rsocket-kotlin, or folks from ktor could also help

[Edoardo Luppi]

Thank you! First I'll go back to the "design" stage and see if I can improve how we pass the coroutine context / dispatcher around, or if we can just use IO without thinking too much about it.

[Edoardo Luppi]

Just to verify I understood the concept of `SelectorManager`: it's basically the central piece that will coordinate socket connections.
It's what switches between one or the other while one is suspended

[Oleg Yukhnevich]

You can read about NIO Selector (JDK documentation is a good start) - SelectorManager is like a coroutines abstraction over it (on JVM, and similar concept on Native)
So, yes, basically it's a coordinator which decides when some of the application sockets should read/write to platform socket

[Edoardo Luppi]

I was just looking at the code and noticed the similarity in naming, I'm going to read that, thanks!

[Edoardo Luppi]

I suppose being a wrapper on the JDK selector is also why it allocates one thread

[Oleg Yukhnevich]

Yeah
as per documentation of select (https://docs.oracle.com/javase/8/docs/api/java/nio/channels/Selector.html#select--)

This method performs a blocking selection operation. It returns only after at least one channel is selected

So, if sockets are idle it just blocks thread awaiting for read/write

[Edoardo Luppi]

Finally starting to see what's going on. This really shows what a good piece of KDoc could do to prevent this headache

[Oleg Yukhnevich]

treat ktor-network as mostly internal ktor module to support CIO engine 🙂
At least, this is how I see it right now 🙃
I still don't fully understand how to use it efficiently and frequently looking at CIO (client/server) engines implementation to understand what's going on

[Edoardo Luppi]

Another thing one could do is revert to use the blocking java.net.Socket in the wrapper, instead of using Ktor

[Edoardo Luppi]

Probably it really depends on the kind of abstraction one is building

[Edoardo Luppi]

I'm now using a single selector manager with <http://Dispatchers.IO|Dispatchers.IO>. The only problem I had to workaround was closing it.
The way the library is structured prevents me from allowing consumers to explicitly close/dispose resources, so I had to register a shutdown hook.
I'm not sure which kind of resources the NIO selector allocates, and what would happen if I didn't use an hook, but just to be safe it looks like a decent workaround.

[Oleg Yukhnevich]

Yeah, it should do the trick I believe:)

[Edoardo Luppi]

I'm still left with a doubt regarding limitedParallelism
If I understand correctly the documentation, if for each socket connection I call limitedParallelism(1), and I have 1000 connections, the underlying pool will be expanded to up to 1000 threads?
If my understanding is correct, my usage of limitedParallelism is dangerous

[Oleg Yukhnevich]

Yeah, up to if all of the sockets will want to do read/write at the same moment.
I don't thinks that this will happen in reality
Still, it could be that just <http://Dispatchers.IO|Dispatchers.IO> for all sockets will be fine (without limitedParallelism), it's capped to 64 threads

[Edoardo Luppi]

It's very unlikely it will happen, you're right. But I like being defensive when it comes to allocated resources.
So I'm left with two choices:
1. Use <http://Dispatchers.IO|Dispatchers.IO> - will allocate 1 up to 64 threads on demand, capped to 64 or number of cores
2. Use a custom ExecutorService - however this will require manual shutdown. Added complexity, worth it? Maybe not.
Since the executor is an implementation detail, I'd have to find a way to shut it down without the consumer knowing about it.

[Edoardo Luppi]

At this point I'm also curious to understand how the IO dispatcher handles shutdown