https://kotlinlang.org logo
#ktor
Title
# ktor
e

Edoardo Luppi

03/26/2024, 1:34 PM
Just to understand better what I'm doing: what's the effective role of the
dispatcher
in
SelectorManager(dispatcher = <http://Dispatchers.IO|Dispatchers.IO>)
? In my KMP library, I'm wrapping Ktor in a way similar to:
Copy code
override suspend fun connect(address: NetAddress): ServerProperties {
  selectorManager = SelectorManager(dispatcher)

  val tcp = aSocket(selectorManager).tcp()
  socket = tcp.connect(address.host, address.port)
  ...
I suppose a
SelectorManager
should be used to create multiple socket connections, so having one
SelectorManager
instance per socket instance isn't strictly correct.
o

Oleg Yukhnevich

03/26/2024, 1:41 PM
AFAIR,
SelectorManager
on JVM and Native behaves differently regarding dispatcher: • on Native, dispatcher is ignored and separate
Worker
is created for handling of non-blocking sockets selection • on JVM, it uses dispatcher and fully blocks (in a blocking loop) one thread from dispatcher (that's why you need IO and not Default) And yes, one selector should be enough to handle multiple sockets (client/server) You can check on how
ktor
CIO engine (client or server) is implemented to get more insights
👍 1
e

Edoardo Luppi

03/26/2024, 1:49 PM
Thanks @Oleg Yukhnevich. To clarify further, what I offer in this library is a multiplatform implementation of a TCP protocol. My confusion here spawns from the fact I'm not sure which thread pool / dispatcher should be passed to the
SelectorManager
. When I instantiate the JVM socket client abstraction, e.g.
MyJVMSocketClient()
, I instantiate it using a custom coroutine scope, and a custom dispatcher that limits to a single thread. That is, each
MyJVMSocketClient
instance should use one thread at most.
Copy code
public actual fun Dispatchers.forSocket(): CoroutineDispatcher =
  IO.limitedParallelism(1)
However, currently, inside
MyJVMSocketClient.connect
I'm doing
Copy code
override suspend fun connect(address: NetAddress): ServerProperties {
  selectorManager = SelectorManager(dispatcher = <http://Dispatchers.IO|Dispatchers.IO>)
Which is obviously wrong from a logical perspective, because I'm not even re-using the
forSocket()
dispatcher.
Probably what I should use is
Copy code
selectorManager = SelectorManager(currentCoroutineContext())
This would ensure the context is set outside of the socket client abstraction
o

Oleg Yukhnevich

03/26/2024, 2:42 PM
be careful with multiple
SelectorManager
instances, as it could suffer performance. As I said, on JVM it will block one thread - and using it for one socket will be very not efficient
I'm not sure which thread pool / dispatcher should be passed to the
SelectorManager
.
I would say, it's better to pass
<http://Dispatchers.IO|Dispatchers.IO>
like in your example, it should work fine, though, It still would be better to reuse it per sockets Why do you need separate dispatcher for socket with
io.limitedParallelism(1)
?
e

Edoardo Luppi

03/26/2024, 3:02 PM
> Why do you need separate dispatcher for socket with
io.limitedParallelism(1)
? That's actually a good question, in the sense that I don't recall why it was done like that a long time ago. It was probably done because of the misunderstanding between concurrency and parallelism, or probably because the idea was to constrain access to resources for each active socket connection
constrain access to resources
In the sense we did not want to let coroutines use/spawn N threads freely, we wanted to use the bare minimum.
o

Oleg Yukhnevich

03/26/2024, 4:24 PM
okay, I don't know about all your requirements, so I would just write some suggestion, and then you will be able to decide 🙂 1. use single
SelectorManager
per all your client sockets, it should be closed to free resources
after
all client sockets are closed - or you will have a deadlock 2. use
SelectorManager(IO)
+ handle sockets on IO until it becomes a problem 3. If for some reason you will see, that there are some problems with performance, I would suggest to create a separate dispatcher or via
IO.limited(N)
(though, I haven't tested it, but most likely it should work) or via
newFixedThreadPoolContext
(don't forget to close it) , where N > 1 (as 1 thread will be blocked) to reduce amount of threads used (and so memory and thread-switching) 4. Don't create a lot of
SelectorManager
instances, as it will cause a lot of platform threads to be just blocked, and so reduce performance (may be Virtual Threads will work fine here, but I haven't tested it) 5. it's fine to use different dispatcher for
SelectorManager
and sockets handling, but it will cause additional thread-switching, which could or couldn't affect your use case 🙂 6. Before replacing
IO
with something else - try to understand the problem better and if possible do some benchmarking/stress testing If you have an open-source repository somewhere, it could help to understand better what do you want to achieve 🙂 Still,
ktor-network
is really not so easy to use, so don't be shy to ask questions here, I will try to help with all my knowledge received during development/research for rsocket-kotlin, or folks from
ktor
could also help
e

Edoardo Luppi

03/26/2024, 4:32 PM
Thank you! First I'll go back to the "design" stage and see if I can improve how we pass the coroutine context / dispatcher around, or if we can just use IO without thinking too much about it.
Just to verify I understood the concept of `SelectorManager`: it's basically the central piece that will coordinate socket connections. It's what switches between one or the other while one is suspended
o

Oleg Yukhnevich

03/26/2024, 4:43 PM
You can read about NIO Selector (JDK documentation is a good start) -
SelectorManager
is like a coroutines abstraction over it (on JVM, and similar concept on Native) So, yes, basically it's a coordinator which decides when some of the application sockets should read/write to platform socket
e

Edoardo Luppi

03/26/2024, 4:43 PM
I was just looking at the code and noticed the similarity in naming, I'm going to read that, thanks!
👍 1
I suppose being a wrapper on the JDK selector is also why it allocates one thread
o

Oleg Yukhnevich

03/26/2024, 4:46 PM
Yeah as per documentation of
select
(https://docs.oracle.com/javase/8/docs/api/java/nio/channels/Selector.html#select--)
This method performs a blocking selection operation. It returns only after at least one channel is selected
So, if sockets are idle it just blocks thread awaiting for read/write
e

Edoardo Luppi

03/26/2024, 4:47 PM
Finally starting to see what's going on. This really shows what a good piece of KDoc could do to prevent this headache
o

Oleg Yukhnevich

03/26/2024, 4:49 PM
treat
ktor-network
as mostly
internal
ktor module to support
CIO
engine 🙂 At least, this is how I see it right now 🙃 I still don't fully understand how to use it efficiently and frequently looking at
CIO
(client/server) engines implementation to understand what's going on
✔️ 1
e

Edoardo Luppi

03/26/2024, 4:51 PM
Another thing one could do is revert to use the blocking
java.net.Socket
in the wrapper, instead of using Ktor
Probably it really depends on the kind of abstraction one is building
I'm now using a single selector manager with
<http://Dispatchers.IO|Dispatchers.IO>
. The only problem I had to workaround was closing it. The way the library is structured prevents me from allowing consumers to explicitly close/dispose resources, so I had to register a shutdown hook. I'm not sure which kind of resources the NIO selector allocates, and what would happen if I didn't use an hook, but just to be safe it looks like a decent workaround.
o

Oleg Yukhnevich

03/26/2024, 7:16 PM
Yeah, it should do the trick I believe:)
✔️ 1
e

Edoardo Luppi

03/27/2024, 3:34 PM
I'm still left with a doubt regarding
limitedParallelism
If I understand correctly the documentation, if for each socket connection I call
limitedParallelism(1)
, and I have 1000 connections, the underlying pool will be expanded to up to 1000 threads? If my understanding is correct, my usage of
limitedParallelism
is dangerous
o

Oleg Yukhnevich

03/27/2024, 4:16 PM
Yeah, up to if all of the sockets will want to do read/write at the same moment. I don't thinks that this will happen in reality Still, it could be that just
<http://Dispatchers.IO|Dispatchers.IO>
for all sockets will be fine (without
limitedParallelism
), it's capped to 64 threads
e

Edoardo Luppi

03/27/2024, 4:29 PM
It's very unlikely it will happen, you're right. But I like being defensive when it comes to allocated resources. So I'm left with two choices: 1. Use
<http://Dispatchers.IO|Dispatchers.IO>
- will allocate 1 up to 64 threads on demand, capped to 64 or number of cores 2. Use a custom
ExecutorService
- however this will require manual shutdown. Added complexity, worth it? Maybe not. Since the executor is an implementation detail, I'd have to find a way to shut it down without the consumer knowing about it.
At this point I'm also curious to understand how the IO dispatcher handles shutdown
7 Views