Hey, I observed something interesting and would li...

# http4k

[DanielZ]

Hey, I observed something interesting and would like to have a clarification about the desired behaviour.

class ExampleShould {
    @ParameterizedTest(name = "{index} - {1}")
    @MethodSource("clients")
    fun `clarify how to handle umlaute in header fields`(httpClient: HttpHandler, name: String) {
        val customTrace = "mööp"
        myService()
            .asServer(SunHttp(0))
            .start()
            .use { main ->
                val client = ClientFilters.SetBaseUriFrom(Uri.of("<http://localhost>:${main.port()}")).then(httpClient)
                val response = shouldNotThrowAny {
                    client(
                        Request(GET, "/")
                            .with(Header.MY_CUSTOM_TRACING of customTrace)
                    )
                }
                response shouldHaveStatus I_M_A_TEAPOT
                response.shouldHaveHeader(Header.MY_CUSTOM_TRACING, be(customTrace))
            }
    }

    companion object {
        @JvmStatic
        fun clients(): List<Arguments> = listOf(
            arguments(OkHttp(), "OkHttp"),
            arguments(JavaHttpClient(), "JavaHttpClient"),
            arguments(Java8HttpClient(), "Java8HttpClient"),
            arguments(ApacheClient(), "ApacheClient"),
            arguments(HelidonClient(), "HelidonClient"),
        )
    }
}

val Header.MY_CUSTOM_TRACING: BiDiLens<HttpMessage, String>
    get() = Header.required("My-Custom-Tracing")

fun myService(): HttpHandler = { Response(I_M_A_TEAPOT).with(Header.MY_CUSTOM_TRACING of Header.MY_CUSTOM_TRACING(it)) }

Result is from returning passed in mööp in custom tracing field to java.lang.AssertionError: Header "My-Custom-Tracing": expected:<"mööp"> but was:<"mööp">
and even throwing an exception.

Shouldn’t all clients behave similar? Which should be the desired one? (Spring keeps the original mööp so I need either good arguments against such a value or at least a good strategy in http4k)
(That custom tracing field is used in structured logging.)

I’m aware that I can and should handle the exception thrown by OkHttp in a Filter . Is the only other option to switch to ApacheClient ?

[s4nchez]

If I’m not mistaken, header fields should be ASCII encoded (eg using base64). Spring or some specific clients support extra characters but there’s no guarantees servers will understand those either.

We can look into making it consistent but most likely we’d still be limited by what each client and server implementation supports.

[James Richardson]

Probably need to be careful here.... spring when testing will often not serialise the stuff on the wire, and so can hide encoding problems. To test what it actually does I'd encourage to check across two jvms.
You cannot use utf8 in headers. They have to be plain ascii. https://www.rfc-editor.org/rfc/rfc7230#section-3.2.6

[James Richardson]

So the header value of mööp is not allowed. If you are sending such a value, it would be advisable to encode it, but then you'd have to define that encoding scheme. If it's client supplied aka untrusted, then it should be validated before sending, really, so should be rejected before reaching the http layer.

[DanielZ]

Thanks for your answers, what I observed in our production system is that that custom tracing id was passed through by a spring service to one of our services (using http4k, with helidon as server and OkHttp as client). Receiving such a header is fine and we log it accordingly, calling OkHttp with such a request results in an IllegalArgumentException . In order to understand that challenge better I was writing the test above.

When I understand it correctly I could either disallow such header values or to encode it using base64 (which would create next challenges with other upstream services 😄 )
Most likely I would like to prevent such header values, sounds like it is the cleanest solution.

[James Richardson]

Yeah, so depending on how you want to handle this, http4k can help you!

Let's say we define some validation rules for the tracing parameter. Then the header value could be a types4k type with validation rules, and you could use a lens to get the header value. If it didn't validate you'd immediately get a lensfailure.

[DanielZ]

oh, nice - didn’t consider using types4k, since we are not using it so far. I was writing a simple filter to check all header fields to be in ASCII and returning otherwise an error response. Will experiment with types4k 😄

[James Richardson]

Doh. Sorry values4k. https://github.com/fork-handles/forkhandles/tree/trunk/values4k

[James Richardson]

you can define a thing like this:

class TraceID(v: String) : StringValue(v) {
    companion object : StringValueFactory<TraceID>(::TraceID, validation = "^trace-[a-z]+$".regex)
}
}

So that's defined a value type that can only be trace-xxxxxx

Now you can use a lens to get it, first define a lens

val traceID = Header.value(TraceID).required("my-trace-header")

Then in a handler, get the value:

val requestTraceID = traceID(request)

This way value you know will be strongly typed, and also validated!

This is written in slack on mobile, so might be a couple of typos!

[James Richardson]

If the header is optional, the value of requestTraceID will be of type TraceID?

[James Richardson]

But it will be validated if given