Hello, trying to download 200Mb files with Ktor an...

# ktor

[Vlad]

Hello, trying to download 200Mb files with Ktor and okhttp engine for android.
The docs for streaming download:

val client = HttpClient(CIO)
val file = File.createTempFile("files", "index")

runBlocking {
    client.prepareGet("<https://ktor.io/>").execute { httpResponse ->
        val channel: ByteReadChannel = httpResponse.body()
        while (!channel.isClosedForRead) {
            val packet = channel.readRemaining(DEFAULT_BUFFER_SIZE.toLong())
            while (!packet.isEmpty) {
                val bytes = packet.readBytes()
                file.appendBytes(bytes)
                println("Received ${file.length()} bytes from ${httpResponse.contentLength()}")
            }
        }
        println("A file saved to ${file.path}")
    }
}

What would be reasonable value for the DEFAULT_BUFFER_SIZE , there are no such constant available.
I tried to load with 4096 limit but faced different issues:
• Extremely slow (I don't write to disc, just read)
• 90% throws okhttp3.internal.http2.StreamResetException: stream was reset: CANCEL
• Sometimes says that read was successful but read bytes size is somewhere between 56-58 Mb when the file is 102Mb and the duration is around 330s. And we end up with corrupted file with success state.
• I thought debug build messes with me and tried release. No luck.
I try download from our server (we don't have any timeouts) and from the opensource https://link.testfile.org/PDF200MB
Not much in the google for the topic. Feels like the thing is simple and everything is working for everyone.

I tried to play with .readRemaining limit with 4096 * 10 and noticed that speed also multiplied and I often can download 200Mb.
Any guidance please?

UPD:
After I connected download with sink into files I noted that everything of that is not happening anymore.
I was wondering why and start researching back.
For the test purposes after every packet.readBytes() I was adding them into totalBytes

totalBytes += bytes

And that what was causing the delay (and hopefully all other issues).
Adding into the array without initial size was slowing it like 30 times 🤦‍♂️

[Aleksei Tirman [JB]]

Can you try writing the response body to a file using the ByteReadChannel.copyTo method?

val client = HttpClient(CIO)
val file = File.createTempFile("files", "index")
client.prepareGet("<https://ktor.io/>").execute { httpResponse ->
    val channel: ByteReadChannel = httpResponse.body()
    channel.copyTo(file.writeChannel())
}

[Vlad]

copyTo accepts ByteWriteChannel. So if you have that sure. I don't

[Aleksei Tirman [JB]]

The File.writeChannel method creates a ByteWriteChannel for the file.

[Vlad]

We don't have File in KMP 🙂