I'm also exploring using a ByteReadChannel instead...

# ktor

[Christopher Mederos]

I'm also exploring using a ByteReadChannel instead. I've used the example from the docs that copies from the received channel to a new write channel

val file = File("server/uploads/image.jpg")
call.receiveChannel().copyAndClose(file.writeChannel())

However, I would like to write my own version that short circuits after exceeding a maximum file upload size... any good ideas on how to handle that? I've been experimenting with consumeEachBufferRange, but can't figure out how to actually pipe the bytes elsewhere correctly

channel.consumeEachBufferRange { buffer, last -> // write to new file?? }

Thanks for the help!

[Aleksei Tirman [JB]]

You can store the total bytes written to the file channel and use that value to determine when to stop visiting the buffer. Here is an example:

val channel = ByteChannel()

launch {
    repeat(20) {
        channel.writeStringUtf8("a".repeat(5555))
        delay(150)
    }
}

val fileChannel = File("output.txt").writeChannel()

val maxSize = 50 * 1024
var total = 0
channel.consumeEachBufferRange { buffer, last ->
    total += buffer.capacity()
    println(total)
    fileChannel.writeFully(buffer)
    total <= maxSize
}

[Christopher Mederos]

@Aleksei Tirman [JB] thats' extremely helpful, thanks for sorting me out!

[Christopher Mederos]

I follow up question I have...
Is there a graceful way for the server to end the connection after exceeding the size limit / some other validation? I've tried both readChannel.cancel() and call.respond(someHttpErrorCode) . However, both cause a ConnectionResetException: Connection reset exception in the client.

Is it better just to handle these exceptions on the client side anyhow? In practice, It's an unlikely exception anyhow. I have control over the clients and can enforce file size validations before attempting the upload.

[Aleksei Tirman [JB]]

Is there a graceful way for the server to end the connection after exceeding the size limit / some other validation?

Can you explain your problem in more detail?

[Christopher Mederos]

thanks again for the help - just thought I'd share what I ultimately ended up with for implementing a passthrough stream on ktor for uploading a file to s3

val readChannel = call.receiveChannel()
var totalRead = 0
val maxRead = 10 * 1024 * 1024 // 10 MB

val byteFlow = flow<ByteArray> {
                readChannel.consumeEachBufferRange { buffer, last ->
                    totalRead += buffer.capacity()
                    emit(buffer.moveToByteArray())

                    totalRead <= maxRead
                }
            }

val s3request = PutObjectRequest {
                bucket = "my-bucket"
                key = "my-object-key"
                metadata = metadataVal
                body = byteArrayFlow.toByteStream(this@withContext)
                checksumAlgorithm = ChecksumAlgorithm.Sha256 // necessary since default checksum requires knowing the total content length 
            }

s3client.putObject(request)