On the ktor server side, is there something specia...

# ktor

[Carter]

On the ktor server side, is there something special that needs to be done to stream responses?

I have a client that’s sending a JSON request, then expecting a bunch of JSON objects to be emitted as a newline separated stream.

The server is doing this

call.respondTextWriter(status = HttpStatusCode.Created) {
    answerFlow.collect { answer ->
        write(jsonParser.encodeToString(answer))
        write("\n")
        flush()
    }
}

And the client is doing this

httpClient.prepareRequest {
    method = <http://HttpMethod.Post|HttpMethod.Post>

    contentType(ContentType.Application.Json)
    setBody(questionBuilder)
}.execute { response ->
    if (!response.status.isSuccess()) {
        // handle error
    } else {
        val channel: ByteReadChannel = response.body()
        while (!channel.isClosedForRead) {
            val line = channel.readUTF8Line()
            if (line?.isNotBlank() == true) {
                println("Receiving: $line")
                val responseObj = jsonParser.decodeFromString<Obj>(line)
                emit(responseObj)
            }
        }
    }
}

I’m pretty confident the client-side code is correct, because I tested it against a different web service known to return similarly streamed responses.

As best I can tell, it seems like the server is buffering the response, ignoring the flush calls, and returning all the data once the Writer is closed. Is there something I’m missing here?

I also tried respondOutputStream with similar results.

[Aleksei Tirman [JB]]

Can you try using the ApplicationCall.respondBytesWriter method that isn’t blocking?

call.respondBytesWriter(status = HttpStatusCode.Created) {
    answerFlow.collect { answer ->
        writeStringUtf8(jsonParser.encodeToString(answer))
        writeStringUtf8("\n")
        flush()
    }
}

[Carter]

The behavior is the same—it appears to write everything first, then the client starts to receive it.

[Carter]

I just tried another variation to eliminate the flow collection

call.respondBytesWriter {
                repeat(10) {
                    println("writing")
                    writeStringUtf8("\n")
                    flush()
                    delay(500)
                }

And I can see the logs show writing completes before the client receives anything

[Aleksei Tirman [JB]]

Unfortunately, I cannot reproduce your problem with the following code:

embeddedServer(Netty, port = 3333) {
    routing {
        get("/stream") {
            call.respondBytesWriter {
                repeat(10) {
                    writeStringUtf8("Hello you\n")
                    flush()
                    delay(1000)
                }
            }
        }
    }
}.start(wait = true)

Both curl and Ktor’s HTTP client with the CIO engine receive a line as soon as it is sent.

[Carter]

Thanks for the help – let me see what else I can uncover and I will follow up.

[Carter]

I have some progress. If I use Curl on the command line, I am able to see that the server streams the response. So the server is working mostly correctly, it seems.

With Java client/server, there seems to be some buffering somewhere because the client seems to be emitting in a bursty fashion. Right now I’m using Netty on the server and the Java http client.

My primary use case though is Ktor client running under Javascript in the browser. I’m still working to see if I can get a minimal reproduction case for that.

[Zoltan Demant]

There was a new feature added in 2.3.0 which let's you 'serialize' a flow, seems like it would do the trick for you! I don't have a link to it but you can find it mentioned in the changelog!

[Carter]

From what I can tell, serializing a flow buffers the response from the server instead of streaming it (at least when testing with kotlinx serialization).

[Carter]

Aleksei, I can confirm that the issue is with Kotlin JS in the browser. The exact same client code under the JVM will receive the data as a stream, while in the browser under Kotlin JS it will wait until the entire response is received.

Do you have any suggestions to try, or is this a bug in Ktor under Javascript?

[Aleksei Tirman [JB]]

That seems to be a bug. Can you please file an issue?

[Carter]

I filed this with a sample application that reproduces the issue in the browser https://youtrack.jetbrains.com/issue/KTOR-5867/Ktor-Client-Unable-to-Stream-Responses-in-Javascript

[Carter]

Aleksei, I see that fixes for the streaming bug were implemented in Ktor 2.3.1. Thank you for looking into this issue so soon.

I’m still seeing some issues with streaming responses in Javascript clients. I reproduced again with my sample project attached to the YouTrack issue (after bumping ktor to 2.3.1) Is there something else special that needs to be done in order to see streaming happen in JS clients?

[Aleksei Tirman [JB]]

Unfortunately, I cannot reproduce the problem with Ktor 2.3.1. Please try to install the CORS plugin on the server:

install(CORS) {
    anyHost()
}

[Carter]

I do not have the CORS plugin installed, so let me try that.

[Carter]

I identified the root cause. Enabling server response compression causes the client to fail to stream the response. If I disable compression on the server, I am able to stream the response in Javascript Ktor client 2.3.1.

Is this compression limitation also a bug in the Ktor Javascript client?

[Aleksei Tirman [JB]]

I've created an issue https://youtrack.jetbrains.com/issue/KTOR-5977.

[Carter]

Thanks for filing the issue Aleksei.

[Carter]

I think there’s one additional, independent bug besides compression.

I’m trying to narrow that one down now and will report back once I have more specifics. What I’m seeing is that once compression is disabled, I can get streaming to work locally on my machine but it fails when my server is deployed to Google Cloud Run.

The differences that I know of so far between localhost and deployed versions are that deployed is definitely using SSL and might also be using http2.

[Carter]

I think the final issue I mentioned above is Google Cloud Run. Google runs a HTTP/2 proxy which terminates the SSL connection, then connects over plaintext privately to the container running the ktor application over HTTP/1.1. I think this is why the ability to stream is lost. If I connect with curl (bypassing ktor JS client), I can confirm that streaming does not work.

Google states that Google Cloud Run supports streaming over HTTP and GRPC. This makes me wonder if ktor supported http2 without SSL (h2c), would it then enable streaming on Google Cloud Run?