Hello dear people! I am trying out ktor 3.4.2 (lat...

# kotlinx-rpc

[Joost Klitsie]

Hello dear people! I am trying out ktor 3.4.2 (latest) and krpc 0.10.2 (latest if we don't could the 0.11 grpc builds 🙂 ) with kotlin 2.3.21-RC.
It seems like flows aren't cancelled very well -> my server hangs and I have reboot, and my client seems to hang as well
• at least in my wasmJS web app, the client.awaitClose() is endless and never closes. I see in the logs that connection was interrupted but this doesn't make it into the app
• The JVM client also seem to be hanging -> this could also be because my server is hanging
I have a flow to observe data (so it could take a while before the first element is sent out, it will only push changes), and my understanding is that if my connection fails, these flows will be cancelled automatically. Am I living under the right assumption?
The main thing I am trying is my client to reconnect after a connection breakage -> I had it running in the past but now even if I restart my client, somehow a socket connection is made with the server but nothing really happens afterwards. I can see some old connections aren't cleaned up as well and I had on the server side a flow.invokeOnCompletion which is only called if I gently cancel the flow from client side, but on connection drop with the client it seems endless.

[Alexander Sysoev]

Hi! Yes, this a right assumption
The issue might be a Ktor's because they sometime don't cancel websocket coroutine scope properly in some engines. I can suggest you to try a different engine for a experiment

[Joost Klitsie]

I am using Netty -> any suggestion here?

[Joost Klitsie]

Do you have a favourite? 🙂

[Alexander Sysoev]

CIO might work afair

[Alexander Sysoev]

Also, is it a dangling server or a client? Client engine might the problem too

[Joost Klitsie]

so definitely things are dangling on the server (because if a client-restart doesn't help, then it has to be the server)

[Joost Klitsie]

and on web assembly side I think the connection break/cancellation isn't propagated at all. Funny thing -> in previous ktor versions there was an illegalstateException thrown because of a bug which I could intercept (like listening to errors window.onerror and then closing connections). But now that bug is fixed so I can no longer catch it 😄

[Joost Klitsie]

I will then also replace the client engine with CIO see what happens

[Joost Klitsie]

okay CIO client is not possible on JS so will not bother with that -> for now the whole thing seems flaky. I am trying to tweak my koin injection as well, maybe I did something wrong there. If I find anything conclusive I will write, but for now I will check the exact steps I need to reproduce this

[Joost Klitsie]

Okay I made many changes and I no longer know what is left or right 😄 The thing I did notice was if during server startup I tried to make calls, things had a chance to go south. I was restarting the client/server a bunch of times and making sure that while the server was restarting the client would retry to connect. Sometimes none of the rpc functions I called worked, I now changed that I block server connection during initialization phase.

Also, I had an issue where I did have a proper connection and could run one RPC function, but not another. Here it did look like perhaps something went wrong with Koin -> I noticed that some code (in a use case) simply didn't run. With which I mean, I saw UseCase#1 trigger UseCase#2, but then it did absolutely nothing (like override fun run() = println("log me!") didn't print anything but was hanging indefinitely...). If I change the use case to be a singleton it runs -> no idea what is wrong here, maybe koin gets confused when it should supply this use case fast multiple times, I did see this particular use case being instantiated multiple times (per design) concurrently.

Other than that, I changed that my client will first connect with a ping() before making any other calls. In this way, connection is made first before more concurrent requests are made, as I saw in ktor that my route { rpc("..") {} } was called multiple times but only 1 rpc service actually being instantiated, I guess that isn't bad but still I made the change.

My current hypothesis is that perhaps the culprit can be koin -> maybe if multiple rpc services are instantiated and a dependency is quickly constructed multiple times, accross multiple threads (didn't check this but I imagine ktor to handle traffic multithreaded by default), it may cause some form of deadlock. Due to this deadlock, connections aren't handled properly once a service is hit (like what I saw with flows not being cancelled).

[Joost Klitsie]

nothing conclusive -> but it runs now 😄 if I have more time on a sunday I will try to setup a project with an over-eager client, and a server with koin that does some similar dependency injection

[Joost Klitsie]

okay so because I opened the topic and there should be no shame in making mistakes (but boy oh boy) ->

Apparently, I was referencing enums from other enums. During initialization, obviously if I reference enum A from enum B and vice versa, there is a circular dependency. You will not be warned about this and I only noticed it when fields of an enum was null which really shouldn't be null 😄 so perhaps it was an endless loop that I caused which made my app non-responsive. Also, it didn't crash and also the bug didn't always happen. Long story short -> when I fixed this, my server problems went away.

To close off the topic with "how I fixed wasmJS": I use now some javascript hook to intercept WebSocket errors and with those I close the rpc websocket session, I set these when my application launches. If I don't do this, my web application will not close sockets if it loses connection:

/**
 * Ugly-ass function to detect socket interruptions
 */
@OptIn(ExperimentalWasmJsInterop::class)
fun applyJsPatch(@Suppress("unused") callback: () -> Unit) {
    js("""
        (function() {
            const OriginalWS = window.WebSocket;
            window.WebSocket = function(url, protocols) {
                const socket = new OriginalWS(url, protocols);
                socket.addEventListener('error', () => {
                    callback(); // Calls the Kotlin lambda passed in
                });
                return socket;
            };
            window.WebSocket.prototype = OriginalWS.prototype;
        })();
    """)
}

// I call it connection closer, do whatever you want with it :D
fun setupInterruptionDetection(connectionCloser: ConnectionCloser) { 
    val onInterrupted = {
        println("Interruption detected via JS hook!")
        connectionCloser.close()
    }

    applyJsPatch(onInterrupted)
}

[Joost Klitsie]

happy weekend

[Alexander Sysoev]

Glad you figured it out! Happy weekend you too!