I recently tried upgrading to ktor 3.0.0-beta-2, b...

# ktor

[Todd]

I recently tried upgrading to ktor 3.0.0-beta-2, but I'm running into a issue with multipart uploads. Previously for FileItems I would do steamProvider() and use the inputStream to consume the file. However, I modifed this code to use provider() since streamProvider() is deprecated. First, I'm not sure how I can just consume this as kotlinx.io.Source without reading the whole thing into memory. Regardless, I tried provider().readRemaining() but I got the error in the thread 🧵

[Todd]

java.io.EOFException: Not enough data available, required 106496 bytes but only 0 available
	at io.ktor.utils.io.ByteReadChannelOperationsKt.readPacket(ByteReadChannelOperations.kt:320)
	at io.ktor.http.cio.MultipartKt$parseMultipart$1.invokeSuspend(Multipart.kt:267)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:231)
	at kotlinx.coroutines.DispatchedTaskKt.resumeUnconfined(DispatchedTask.kt:187)
	at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:159)
	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:466)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:500)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl$default(CancellableContinuationImpl.kt:489)
	at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:364)
	at io.ktor.utils.io.ByteChannel.resumeSlot(ByteChannel.kt:166)
	at io.ktor.utils.io.ByteChannel.moveFlushToReadBuffer(ByteChannel.kt:79)
	at io.ktor.utils.io.ByteChannel.awaitContent(ByteChannel.kt:68)
	at io.ktor.utils.io.ByteChannel$awaitContent$1.invokeSuspend(ByteChannel.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)

[Todd]

Is this a known issue? Or am I doing something wrong? Happy to file a youtrack for this

[Aleksei Tirman [JB]]

Can you share your server code and the request payload to reproduce the problem?

[Todd]

No, but I'll work on a reproducer when I get a chance

[Todd]

Hey @Aleksei Tirman [JB] I have a reproducer: https://github.com/dronda-t/ktor-playground/blob/main/src/main/kotlin/org/dronda/plugins/Routing.kt

It seems to be fine with small sizes such as 10 bytes but larger payload sizes such as the 1 megabyte payload in the test fails. I confirmed that similar code works in ktor 2.3.12.

I remember reading the code for 2.x.x and there was some condition where larger upload sizes would be saved to a file whereas smaller upload sizes would just be put into a bytearray. I'm curious if that is the issue here. If I get a chance I can try to hunt down the problem, but hopefully this helps!

[Aleksei Tirman [JB]]

Thank you for the reproducer. I can also reproduce it using Ktor 3.0.0-beta-2 with the following test:

@Test
fun test() = testApplication {
    routing {
        post {
            call.receiveMultipart().forEachPart { part ->
                when (part) {
                    is PartData.FileItem -> {
                        val bytes = part.provider().readRemaining().readByteArray()
                        assertEquals(1 * 1024 * 1024, bytes.size)
                    }

                    else -> {}
                }
            }
        }
    }

    client.post {
        setBody(MultiPartFormDataContent(formData {
            append("file", ByteArray(1 * 1024 * 1024) { 1 }, Headers.build {
                append(HttpHeaders.ContentType, "image/png")
                append(HttpHeaders.ContentDisposition, "filename=\"ktor_logo.png\"")
            })
        }))
    }
}

However, I cannot reproduce this problem with Ktor built from the main branch, so the upcoming RC version must contain the fix.

[Todd]

That's great to hear! Thanks for taking a look