Hello, trying to download 200Mb files with Ktor and okhttp engine for android.
The docs for streaming download:
val client = HttpClient(CIO)
val file = File.createTempFile("files", "index")
runBlocking {
client.prepareGet("<https://ktor.io/>").execute { httpResponse ->
val channel: ByteReadChannel = httpResponse.body()
while (!channel.isClosedForRead) {
val packet = channel.readRemaining(DEFAULT_BUFFER_SIZE.toLong())
while (!packet.isEmpty) {
val bytes = packet.readBytes()
file.appendBytes(bytes)
println("Received ${file.length()} bytes from ${httpResponse.contentLength()}")
}
}
println("A file saved to ${file.path}")
}
}
What would be reasonable value for the
DEFAULT_BUFFER_SIZE
, there are no such constant available.
I tried to load with 4096 limit but faced different issues:
•
Extremely slow (I don't write to disc, just read)
• 90% throws
okhttp3.internal.http2.StreamResetException: stream was reset: CANCEL
• Sometimes says that read was successful but read bytes size is somewhere between 56-58 Mb when the file is 102Mb and the duration is around 330s. And we end up with corrupted file with success state.
• I thought debug build messes with me and tried release. No luck.
I try download from our server (we don't have any timeouts) and from the opensource
https://link.testfile.org/PDF200MB
Not much in the google for the topic. Feels like the thing is simple and everything is working for everyone.
I tried to play with .readRemaining limit with 4096 * 10 and noticed that speed also multiplied and I often can download 200Mb.
Any guidance please?
UPD:
After I connected download with sink into files I noted that everything of that is not happening anymore.
I was wondering why and start researching back.
For the test purposes after every packet.readBytes() I was adding them into totalBytes
totalBytes += bytes
And that what was causing the delay (and hopefully all other issues).
Adding into the array without initial size was slowing it like 30 times 🤦♂️