Hello :blush: I'm using Ktor client 2.2.2 on a KMM...

# ktor

[Jon Bailey]

Hello 😊 I'm using Ktor client 2.2.2 on a KMM project, on iOS specifically, and I'm downloading a large file (say 1GB) using streaming to read the data from the ByteReadChannel into a buffer and writing the buffer to file, and the reading the next chunk into the same buffer and writing to file and so on. So I'd expect the memory usage to be about the size of the buffer.
However when I run the app the memory usage is much higher than the size of the file, to 2GB and then falls to 1GB and stays there.

My code is like this and if I comment out the NSData/writing to file bit the memory usage is the same, its specifically the readRemaining line that is causing the high memory usage (I've also tried using readAvailable, not entirely sure what the difference is?)

client.prepareRequest(request).execute {
    val byteBufferSize = 1024 * 100
    val fileHandle = NSFileHandle.fileHandleForWritingAtPath(file)!!
    val channel = it.bodyAsChannel()

    while (!channel.isClosedForRead) {
        val packet = channel.readRemaining(byteBufferSize.toLong())
        while (!packet.isEmpty) {
            val bytes = packet.readBytes()
            bytes.usePinned {
                val data = NSData.dataWithBytesNoCopy(it.addressOf(0), it.get().size.toULong(), freeWhenDone = false)
                fileHandle.writeData(data)
            }
        }
    }
    fileHandle.closeFile()
}

Is there something I'm missing to write a large file directly from the network to disk without high memory usage?

[ribesg]

But did you try to run it with less memory available? If there's available memory there's no problem using it

[Jon Bailey]

Yep fair, though that it's double the filesize and that it stays at a high usage is not a good sign. Also there's no suggestion this will happen based on the code and the ktor docs.

[Jon Bailey]

Also if I implement the download using URLSession completely in Swift then the memory usage tops out at 80MB