I'm having an odd issue with writing tests for mul...

# ktor

[mp]

I'm having an odd issue with writing tests for multipart upload endpoints. (It seems to work ok outside of tests.) When I upload data in a file part, the byte contents seem to get mangled. https://bitbucket.org/marshallpierce/ktor-multipart-upload-test-repro/src/master/src/test/kotlin/org/mpierce/ktor/multipart/MultipartTest.kt (small, runnable project) exhibits the issue for me.

[mp]

@Deactivated User any thoughts on this failure?

[Deactivated User]

Sorry, missed this. Let me check

[Deactivated User]

Verified.

val bytes = byteArrayOf(1, 2, 3) // Works
        //val bytes = byteArrayOf(0xFF.toByte(), 2, 3) // Fails

[Deactivated User]

Looks like something related to encoding. Can you file an issue? I will look to it later

[mp]

Sure. I thought it might be. It seems like maybe bytes with the high bit set might be related

[Deactivated User]

https://github.com/ktorio/ktor/blob/3d7b08889756ec2ecadcd76d9fee6a01d060bc29/ktor-server/ktor-server-test-host/src/io/ktor/server/testing/TestApplicationRequest.kt

The problem is here, it seems that the body is being generated as a String

And probably it should be generated as a ByteArray/Stream instead

[Deactivated User]

The only suitable charset that maps byte<->unicode char is ISO-8859-1:

fun main(args: Array<String>) {
    val availableCharsets = Charset.availableCharsets()
    val keySet = availableCharsets.keys
    for (charset in keySet) {
        val suitableForBytes = (0 until 256).all {
            try {
                "${it.toChar()}".toByteArray(Charset.forName(charset)).toList() == byteArrayOf(it.toByte()).toList()
            } catch (e: Throwable) {
                false
            }
        }
        if (suitableForBytes) {
            println("$charset: $suitableForBytes")
        }
    }
}

But decoding in ISO, but later encoding as UTF would produce wrong results

[mp]

Yep, that sounds plausible.

[mp]

it would definitely explain the high-bit sensitivity

[mp]

append(it.streamProvider().reader(charset).readText()) no good will come of that for random bytes.

[Deactivated User]

yep

[Deactivated User]

https://github.com/ktorio/ktor/pull/486

[mp]

Unbeatable turnaround 🙂