Hi folks, we're working on ensuring some standard ...

# ktor

[Jacob Schwartz]

Hi folks, we're working on ensuring some standard headers are maintained through downstream requests across our platform. Ideally, I'd like to make this transparent to our developers. My initial thought was to use the coroutine context, server and client plugins to pass the set of known headers - however, the CIO engine for the HttpClient uses GlobalScope.launch when launching the request coroutine and it seems the parent context is lost.

Is anyone aware of any project/plugin that does this? Or of a way to get the parent job's context (the parent job does seem to be maintained). Or if I'm barking up the completely wrong tree?

[Aleksei Tirman [JB]]

So do you want to ensure that some set of headers are always sent for every request?

[Jacob Schwartz]

Right, certain headers the server receives on ingress, we want automatically forwarded on every child egress request

[Aleksei Tirman [JB]]

So you have some kind of Ktor server/client setup?

[Jacob Schwartz]

Ah, yes, I should have specified that more clearly.

We recieve requests in the ktor server, during the handling of that parent request, we send several child requests to downstream services and those downstream requests need to maintain a subset of the headers received on ingress

[Jacob Schwartz]

Eg. a correlation ID

[Aleksei Tirman [JB]]

I think the simplest solution is to call a method, which copies the headers, inside the HttpRequestBuilder 's block for each request. Here is an example:

fun main() {
    val client = HttpClient(CIO)

    embeddedServer(Netty, port = 3333) {
        routing {
            get("/a") {
                client.get("<https://example.com>") {
                    addStandardHeaders(call.request)
                }
            }

            get("/b") {
                <http://client.post|client.post>("<https://example.com>") {
                    addStandardHeaders(call.request)
                }
            }
        }
    }.start(wait = true)
}

fun HttpRequestBuilder.addStandardHeaders(request: ApplicationRequest) {
    if (request.headers.contains("custom")) {
        headers.append("custom", request.headers["custom"]!!)
    }
}

[Aleksei Tirman [JB]]

If the HttpClient is abstracted away, then you can try to simplify the calling side a little more.

[Jacob Schwartz]

Right, but this means every call to httpClient.get() will need to directly invoke addStandardHeaders and the callers will need to have access to the call.

I'm looking for a way to handle this more transparently by using something like the BaseApplicationPlugin and the client hooks.

[Jacob Schwartz]

There are instrumentation libraries for some frameworks like for OpenTelemetry, which handle similar things to what I'm looking for, where the header x-b3-traceid is forwarded from ingress to egress requests without the caller needing to manually pass the fields around. I'm not sure there's something like that available for ktor

[Jacob Schwartz]

Circling back on this - is there no way to determine and retrieve information on the parent ingress request into a ktor server to pass along in the ktor clients via some coroutine context or otherwise without directly passing the original call around?

[CLOVIS]

At least on the server, you can create a plugin to intercept data from the call and add it to the coroutine context. I haven't tried client-side, but I expect it shouldn't be much different?

[Jacob Schwartz]

Yes, this what I was alluding to earlier. Adding to the coroutine context is relatively simple, but pulling that data is difficult because the context is easily lost.

As it turns out, the coroutine context is maintained except in cases where there are calls into suspend inline functions, where for some reason the coroutine not maintained and a new Job is used. Turns out we also had some folks misunderstand that routes are already processed as suspend functions so they were adding runBlocking to some calls.

I'm investigating this further as a potential solution, but it is very fragile.

[CLOVIS]

I recommend going through your entire codebase and removing all runBlocking calls… especially if you're using ktor, there are very few legitimate reasons to have one, and they will cause this kind of problems

[CLOVIS]

As it turns out, the coroutine context is maintained except in cases where there are calls into suspend inline functions, where for some reason the coroutine not maintained and a new Job is used.

This seems very strange to me. There is probably something else going on than just suspend inline .

[Jacob Schwartz]

I recommend going through your entire codebase and removing all runBlocking

Yes, that's the plan. We had some folks unfamiliar with Ktor (and really Kotlin) working on these services for a while

[Jacob Schwartz]

This seems very strange to me. There is probably something else going on than just suspend inline .

It was the only thing I changed for that function to see it start maintaining the context

[CLOVIS]

Can you share the function in question?

[Jacob Schwartz]

Sure thing. The method didn't actually need reification so we were able to remove the inline