Hello everyone We use ktor 2.0.0 with Netty engine We process large tsv, tsv transmitted over http W...

# ktor

[Kirillov Mikhail]

Hello everyone
We use ktor 2.0.0 with Netty engine
We process large tsv, tsv transmitted over http
We represent http request body as Flow<ByteBuffer>
How we read from channel

fun ApplicationCall.receiveFlow(): Flow<ByteBuffer> = flow<ByteBuffer> {

    val channel = receiveChannel()
    try {
        while (true) {
            val buffer = ByteBuffer.allocate(4096)
            if (channel.readAvailable(buffer) == -1) {
                break
            }
            buffer.flip()
            emit(buffer)
        }
    } catch (e: Exception) {
        log.error("URL: '${request.uri}' error reading body", e)
        throw e
    }
}

If server slowly process http request - we see increase direct byte buffers (non-heap), this buffers create netty for buffering request (buffers grows to 2 gb)
I expected back pressure in ktor, but buffers are growing...
I communicated with netty users - they recommend disable autoread at channel and use read() manually
Does ktor support back pressure? If so, how enable it?

[Kirillov Mikhail]

@Rustam Siniukov

[Kirillov Mikhail]

@Aleksei Tirman [JB]

[Kirillov Mikhail]

@e5l

[Kirillov Mikhail]

https://youtrack.jetbrains.com/issue/KTOR-4397/Increasing-direct-byte-buffers-if-server-slowly-process-request

[bitkid]

i was fighting with large files and ktor for quite a while (before ktor 2) .. and my solution in the end was to split up my project and swap the netty engine for the jetty engine for the servers which handles the files. ugly .. but working