Hi! I'm running a server using CIO engine and I'm getting HTTP 403 when trying to call a PATCH endpo...

# ktor

[Piotr Krzemiński]

Hi! I'm running a server using CIO engine and I'm getting HTTP 403 when trying to call a PATCH endpoint that uploads a photo. It doesn't even enter the handler logic, just returns 403 without any details. I stumbled upon URLConnection works fine but ktor client returns 403 forbidden during a get request : KTOR-309 but it's about CIO in client engine. It works locally and I see HTTP/1.1 is used, but it doesn't work when I deploy it to AWS where HTTP/2 is used - it may be relevant, but since there's no error message, it's hard to tell.

Any idea why PATCH endpoint causes such issues? My next step would be to try configure HTTP/2 as https://ktor.io/docs/advanced-http2.html, but since it works with the rest of requests, I'm not sure if I need it 🤷

[Gerard de Leeuw]

To my knowledge the CIO engine doesn't support HTTP/2. Only the Netty and Jetty engines do.
It's also mentioned here https://ktor.io/docs/http-client-engines.html#cio:

CIO is a fully asynchronous coroutine-based engine that can be used for both JVM and Android platforms. It supports only HTTP/1.x for now. To use it, follow the steps below:

[Aleksei Tirman [JB]]

@Piotr Krzemiński I cannot reproduce your problem by sending an image in a request body to a server with the following configuration:

val file = File("files", "output.jpg")

embeddedServer(CIO, port = 7070) {
    routing {
        patch("/") {
            call.receiveStream().copyTo(file.outputStream())
        }
    }
}.start(wait = true)

Please share a code snippet to reproduce your problem.

[Piotr Krzemiński]

Thanks, tomorrow I'll try to narrow down the problem, if it's really related to PATCH, HTTP/2 and CIO or something else. Will get back to you!

[Piotr Krzemiński]

I found an interesting behavior that doesn't even entail PATCH. I tried with two similar GET requests, the only difference is that one has Origin request header, and the other one doesn't. I got HTTP 403 and HTTP 200 respectively. This behavior would be good to explain on its own, and I'm still working to make my specific PATCH work. Removing Origin header from PATCH request doesn't work, the browser seems to add it automatically.

[Piotr Krzemiński]

image.png

[Piotr Krzemiński]

three new interesting findings:
• when I disable HTTP/2 in my browser and test it with a very simple PATCH endpoint, it still doesn't work
• when I test it with some non-browser HTTP client (HTTP scratch files in IntelliJ), it works fine - uses HTTP/1.1
• switching from CIO to Jetty doesn't seem to change a thing

[Piotr Krzemiński]

I decided to cut a bug: PATCH requests return HTTP 403 when called over AWS ELB : KTOR-3014 . Regardless of the root cause, the lack of any detailed error message makes it a show-stopper for me.

[Aleksei Tirman [JB]]

Could you please try adding the following line to the CORS plugin configuration?

method(HttpMethod.Patch)

[Piotr Krzemiński]

unfortunately no change in behavior 😞

[Piotr Krzemiński]

is there any more verbose ktor mode which can show where this 403 is returned from?

[Aleksei Tirman [JB]]

You can try to use the following code to get an exceptions's stack trace if status is 403:

sendPipeline.intercept(ApplicationSendPipeline.Before) { x ->
    if (x is HttpStatusCode && x == HttpStatusCode.Forbidden) {
        throw RuntimeException("Show stacktrace")
    }
}

[Piotr Krzemiński]

got some valuable data, see https://youtrack.jetbrains.com/issue/KTOR-3014#focus=Comments-27-5111639.0-0 - I'll troubleshoot it further after lunch

[Piotr Krzemiński]

thanks Aleksei for your lightning-fast support!