hello all, i’m trying to download a 512 mb file with ktor, but after 250mb the app throws OOM. I’m j...

# ktor

[Andre Servidoni]

hello all, i’m trying to download a 512 mb file with ktor, but after 250mb the app throws OOM. I’m justing showing what I receive

val bytes = this
val byteBufferSize = 1024 * 100
val byteBuffer = ByteArray(byteBufferSize)

do {
    val currentRead = bytes.readAvailable(byteBuffer, 0, byteBufferSize)
    if (currentRead > 0) {
        println("received: $currentRead")
    }

} while (currentRead > 0)

[Aleksei Tirman [JB]]

I cannot reproduce OOM with a following code:

val client = HttpClient(CIO)
client.get<HttpStatement>("<https://mirror.linux-ia64.org/ubuntu-releases/20.04.2.0/ubuntu-20.04.2.0-desktop-amd64.iso>").execute { response ->
    val channel: ByteReadChannel = response.receive()
    val byteBufferSize = 1024 * 100
    val byteBuffer = ByteArray(byteBufferSize)
    do {
        val currentRead = channel.readAvailable(byteBuffer, 0, byteBufferSize)
        if (currentRead > 0) {
            println("received: $currentRead")
        }
    } while (currentRead > 0)
}

Could you please share a complete code snippet?

[Andre Servidoni]

did you tried on a mobile device? I’m using on android

[Aleksei Tirman [JB]]

On a real device or an emulator?

[Andre Servidoni]

real device.

This is my client setup

private val httpClient = HttpClient(engineConfig) {
    expectSuccess = false

    install(JsonFeature) {
        val json = kotlinx.serialization.json.Json {
            ignoreUnknownKeys = true
            encodeDefaults = true
        }
        serializer = KotlinxSerializer(json)
    }
    install(Logging) {
        logger = Logger.DEFAULT
        level = LogLevel.ALL
    }
    defaultRequest {
        url {
            protocol = URLProtocol.HTTPS
            host = cloudHost
        }
    }
}

[Aleksei Tirman [JB]]

What engine do you use?

[Andre Servidoni]

CIO, since this part is on the shared code module

[Aleksei Tirman [JB]]

I cannot reproduce it on a real device too. According to profiler data, GC doesn't let that happen.

[Andre Servidoni]

😞
I run again and get the same issue, Failed to allocate and OOM

[Andre Servidoni]

I don’t know what I’m doing wrong, my call is

suspend fun getFile(fileId: String, objectPath: String): HttpStatement =
    httpClient.get("$START_PATH/$userId/$fileId/$objectPath") {
            onDownload { received, contentLength ->
                println("Received $received bytes from $contentLength")
            }
        }

[Aleksei Tirman [JB]]

Could you please share a sample project?

[Andre Servidoni]

I can prepare one

[Andre Servidoni]

another question

do {
        val currentRead = channel.readAvailable(byteBuffer, 0, byteBufferSize)
        if (currentRead > 0) {
            println("received: $currentRead")
        }
    } while (currentRead > 0)

I’m not using the offset, maybe I need to use it?

[Aleksei Tirman [JB]]

This works fine with zero offset.

[Aleksei Tirman [JB]]

I've checked on https://ars.els-cdn.com/content/image/1-s2.0-S0960982203005347-mmc6.txt

[Andre Servidoni]

I tried with the offset like (offset += currentRead) and it just stop after 4-5MB downloaded

[Andre Servidoni]

if I left it 0, it will download the same range infinitelly. I save what I was downloading in a file and after oom it was 10GB but the file is 512mb

[Aleksei Tirman [JB]]

The following code works as expected:

val client = HttpClient(CIO)
val output = File("output.txt")

client.get<HttpStatement>("<https://ars.els-cdn.com/content/image/1-s2.0-S0960982203005347-mmc6.txt>").execute { response ->
    val channel: ByteReadChannel = response.receive()
    val byteBufferSize = 1024 * 5
    val byteBuffer = ByteArray(byteBufferSize)
    do {
        val currentRead = channel.readAvailable(byteBuffer, 0, byteBufferSize)
        output.appendBytes(byteBuffer)
        if (currentRead > 0) {
            println("received: $currentRead")
        }
    } while (currentRead > 0)
}