Is there a recommended way of writing a large resp...

# ktor

[Vincent Barthelemy]

Is there a recommended way of writing a large response to a file?
I'm having issue with writeChannel and bodyAsChannel().copyAndClose : destination file does not exist yet when copyAndClose returns.

[Vincent Barthelemy]

Example:

val client = HttpClient(CIO)
repeat(10) {
    val dest = File("tmp_$it.html").apply { delete() }
    println("Iteration $it")
    val response = client.get("<https://example.com/>")
    val nbByteCopied = response.bodyAsChannel().copyAndClose(dest.writeChannel())
    println("Copied $nbByteCopied bytes - Dest file exists? ${dest.exists()} with length ${dest.length()}")
}

Iteration 0
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 1
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 2
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 3
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 4
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 5
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 6
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 7
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 8
Copied 1256 bytes - Dest file exists? false with length 0
Iteration 9
Copied 1256 bytes - Dest file exists? false with length 0

[Vincent Barthelemy]

Found https://youtrack.jetbrains.com/issue/KTOR-3003 that might be related, but I'm using ktor 3.0.1 here.
Also copyAndClose is marked with @OptIn(InternalAPI::class) so it might be a better / recommended way to copy a response to a file?

[Vincent Barthelemy]

Am I doing a mistake assuming/thinking that when response.bodyAsChannel().copyAndClose(dest.writeChannel()) returns the underlying file should exist and contain all the copied bytes?

[Aleksei Tirman [JB]]

Also copyAndClose is marked with @OptIn(InternalAPI::class) so it might be a better / recommended way to copy a response to a file?

That means this method uses an internal Ktor API for its implementation. For the client, the copyAndClose method isn't internal.

Am I doing a mistake assuming/thinking that when response.bodyAsChannel().copyAndClose(dest.writeChannel()) returns the underlying file should exist and contain all the copied bytes?

Your assumption is correct. This is a bug, so I've filed an issue.

[Vincent Barthelemy]

Thanks a lot for the clarification on the internal API and the issue 🙏

[Vincent Barthelemy]

(For now, I will use this workaround)

val nbByteCopied = response.bodyAsChannel().copyAndClose(destFile.writeChannel())
withTimeout(1000L) {
    while (isActive && destFile.length() != nbByteCopied) delay(50L)
}

It might be naive but it works in my use case.