https://kotlinlang.org logo
#coroutines
Title
# coroutines
a

Alexander

11/13/2018, 10:35 AM
I want to use
ktor
http client with
nodejs
and found out that
Js
engine only works within browser environment. So I took code of js engine and try to adopt it to work with
http
module of nodejs. But it uses streams to read data (an example of how to read data: https://nodejs.org/api/http.html#http_http_request_url_options_callback). And I can't figure out how to do it better. Now I pass all data via channel:
Copy code
class ResponseReader(response: IncommingMessage) {
    private val chunksChannel = Channel<ByteArray>(UNLIMITED)

    init {
        response.on("data") { chunk: Uint8Array ->
            chunksChannel.offer(chunk.asByteArray())
        }
        response.on("end") { ->
            chunksChannel.close()
        }
    }

    suspend fun readChunk(): ByteArray? {
        return try {
            chunksChannel.receive()
        } catch (exc: ClosedReceiveChannelException) {
            null
        }
    }
}
With this solution I cannot cancel reading data. Also as I understand it misses a backpressure propagation thus if data arrives faster then we can process it may lead to a huge memory consumption.
g

gildor

11/13/2018, 10:45 AM
But if you need back pressure, why do you use unlimited channel and offer instead of limited buffer and send?
a

Alexander

11/13/2018, 10:49 AM
I cannot use suspend functions inside non-suspend lambdas.
And I can't wrap it with
suspendCancellableCoroutine
because callback passed to
response.on("data", callback)
is called multiple times.
g

gildor

11/13/2018, 12:11 PM
Yeah, I see, this is problem of JS, when you just cannot block the thread
So you need some node.js API that supports backpressure
Are you sure that back pressure is supported at all for http api?
I'm not really familiar with node, but looks that you can do that with stream API pipe
Anyway, looks that it would be not so straight forward to implement, but probably possible
a

Alexander

11/13/2018, 1:15 PM
Frankly speaking I don't need backpressure at all. I just want to practice with kotlin multiplatform implementing library for elasticsearch. Thus I could read all data from response into some buffer. But I want to do it right way.
57 Views