Hi, i have a use case to stream the response (basically JFR text event logs) liked chunked transfer ...

# ktor

[suresh]

Hi, i have a use case to stream the response (basically JFR text event logs) liked chunked transfer encoding and using the following call.respondTextWriter to do that. Seems like things are not working as expected. 🧵

[suresh]

The sample code is something like

get("/jfr") {
    call.respondTextWriter {
      // start jfr stream and collect events and when event available calls
       appendLine(event.toString())
      // Might go on for few minutes
    }
}

[suresh]

If the collection is under 30 seconds, i will get the response only after collecting all events. I even tried setting chunked transfer encoding, which throws an error as engine is not allowed to that header.

[suresh]

Seems like 30 sec is the default Netty engine write timeout and write does happens continuously and i assumed engine would have handled that write timeout for respondTextWriter calls.

So for anything more than 30 seconds i am getting

2023-12-20 19:27:10.800 PST ERROR localhost [eventLoopGroupProxy-4-2] Application - Internal Server Error
io.netty.handler.timeout.WriteTimeoutException: null
Wrapped by: kotlinx.coroutines.JobCancellationException: Parent job is Cancelling
	Suppressed: [CIRCULAR REFERENCE: kotlinx.coroutines.JobCancellationException: Parent job is Cancelling]
	Wrapped by: java.io.IOException: kotlinx.coroutines.JobCancellationException: Parent job is Cancelling; job=CompletableDeferredImpl{Cancelled}@1b0beb07
		at io.ktor.utils.io.jvm.javaio.OutputAdapter.close(Blocking.kt:134)
		at java.base/sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:396)
		at java.base/sun.nio.cs.StreamEncoder.lockedClose(StreamEncoder.java:231)

[suresh]

So the question, whats the proper way to do text streaming (chunked transfer encoding) on ktor server?

[Rustam Siniukov]

Have you tried calling flush() ?

[suresh]

@Rustam Siniukov yes, flush is also not working

[Aleksei Tirman [JB]]

What client do you use for the testing?

[suresh]

@Aleksei Tirman [JB] Browser client (chrome)

[Aleksei Tirman [JB]]

Chrome might buffer the response. Can you try using curl with the --no-buffer option?

[suresh]

No, same thing. Repose is not getting streamed. Here is the response header

❯ curl -v --no-buffer <http://0.0.0.0:8080/jfr>
*   Trying 0.0.0.0:8080...
* Connected to 0.0.0.0 (127.0.0.1) port 8080 (#0)
> GET /jfr HTTP/1.1
> Host: 0.0.0.0:8080
> User-Agent: curl/8.1.2
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Engine: App-0.10.0.26+b2ee034
< Date: Wed, 03 Jan 2024 19:05:43 GMT
< Server: Ktor/0.10.0.26+b2ee034
< Content-Type: text/plain; charset=UTF-8
< transfer-encoding: chunked
<