I want to use `ktor` http client with `nodejs` and...

# coroutines

[Alexander]

I want to use ktor http client with nodejs and found out that Js engine only works within browser environment. So I took code of js engine and try to adopt it to work with http module of nodejs. But it uses streams to read data (an example of how to read data: https://nodejs.org/api/http.html#http_http_request_url_options_callback). And I can't figure out how to do it better. Now I pass all data via channel:

class ResponseReader(response: IncommingMessage) {
    private val chunksChannel = Channel<ByteArray>(UNLIMITED)

    init {
        response.on("data") { chunk: Uint8Array ->
            chunksChannel.offer(chunk.asByteArray())
        }
        response.on("end") { ->
            chunksChannel.close()
        }
    }

    suspend fun readChunk(): ByteArray? {
        return try {
            chunksChannel.receive()
        } catch (exc: ClosedReceiveChannelException) {
            null
        }
    }
}

With this solution I cannot cancel reading data. Also as I understand it misses a backpressure propagation thus if data arrives faster then we can process it may lead to a huge memory consumption.

[Alexander]

https://github.com/anti-social/ktor-client-node-engine/blob/master/src/main/kotlin/io/ktor/client/engine/js/node/engine.kt

[gildor]

But if you need back pressure, why do you use unlimited channel and offer instead of limited buffer and send?

[Alexander]

I cannot use suspend functions inside non-suspend lambdas.

[Alexander]

And I can't wrap it with suspendCancellableCoroutine because callback passed to response.on("data", callback) is called multiple times.

[gildor]

Yeah, I see, this is problem of JS, when you just cannot block the thread

[gildor]

So you need some node.js API that supports backpressure

[gildor]

Are you sure that back pressure is supported at all for http api?

[gildor]

I'm not really familiar with node, but looks that you can do that with stream API pipe

[gildor]

Anyway, looks that it would be not so straight forward to implement, but probably possible

[Alexander]

Frankly speaking I don't need backpressure at all. I just want to practice with kotlin multiplatform implementing library for elasticsearch. Thus I could read all data from response into some buffer. But I want to do it right way.