Hi there, I'm using ktor with the apache async http client, and I'm sending some POST requests to a ...

# ktor

[Antero Duarte]

Hi there, I'm using ktor with the apache async http client, and I'm sending some POST requests to a server that kept failing because they said the body was empty.

After enabling debug logging, I can see the http client is sending the request in this format:

DEBUG org.apache.http.wire - http-outgoing-0 >> "POST /the/endpoint HTTP/1.1[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Type: x-www-form-urlencoded[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "Accept: */*[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Length: 201[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "Host: localhost:8888[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "User-Agent: Apache-HttpAsyncClient/4.1.3 (Java/1.8.0_172)[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 >> "url_encoded=form_params&which_are=fine"

Everything seems fine, but for some reason, the client is prepending a line before the form body, with just \r\n, which for some reason is tripping the server on the other end.

I don't control the server I'm sending the request to, I don't know if an empty line before the body conforms to the HTTP Spec, but my question is, is there any way to avoid it?

For completeness sake, the way I'm creating the request is as follows:

ApacheHttpClient.request<HttpResponse> {
                        method = <http://HttpMethod.Post|HttpMethod.Post>
                        url { takeFrom("<http://some.url/the/endpoint>") }
                        headers.append("Content-Type", ContentType.Application.FormUrlEncoded.contentSubtype)
                        this.body = body.formUrlEncode()
                    }

body is a List<Pair<String,String>>
and .formUrlEncode is the extension method from ktor HttpUrlEncode.kt

[e5l]

Could you share something about the server API and request format?

[Antero Duarte]

The request is a refresh token request sent to an Oauth server

[Antero Duarte]

Ideally I wouldn't implement this manually, but I couldn't figure out how to get ktor's Oauth to do it for me

[e5l]

And what is the status code of the request?

[Antero Duarte]

400

[Antero Duarte]

https://www.ory.sh/docs/api/hydra/#operations-oAuth2-oauthToken

[Antero Duarte]

That's the server

[Antero Duarte]

the full response:

DEBUG org.apache.http.wire - http-outgoing-0 << "HTTP/1.1 400 Bad Request[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Type: application/json;charset=UTF-8[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 << "Vary: Origin[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 << "Date: Fri, 10 Aug 2018 13:53:42 GMT[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Length: 259[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 << "[\r][\n]"
DEBUG org.apache.http.wire - http-outgoing-0 << "{"error":"invalid_request","error_description":"The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed","error_hint":"The POST body can not be empty.","status_code":400}"

[Antero Duarte]

Is the buggy eyes emoji for the formatting? sorry I know it looks crap, but I don't know how to present it in a better way 😂

[e5l]

Could you check ContentType.Application.FormUrlEncoded.contentSubtype

[e5l]

Change to
ContentType.Application.FormUrlEncoded.toString()

[Antero Duarte]

The thing is, I try other clients, like postman and it works

[e5l]

?

[Antero Duarte]

I can try that, sure, but I put it there to try to fix it, if you look at the request debug, the content type is correct

[Antero Duarte]

But give me a second, I'll try it

[e5l]

It should be application/x-www-form-urlencoded, but here is x-www-form-urlencoded

[Antero Duarte]

you're right, sorry, I misread it

[e5l]

It's obvious an API ambiguity. Noted and probably would be fixed soon.

[Antero Duarte]

I'll try it and let you know

[e5l]

Thanks 🙂

[Antero Duarte]

I can't believe that was it... 🤦

[Antero Duarte]

Thanks for the help and fresh pair of eyes 😂

[e5l]

No problem 🙂

[cy]

What about this extension to avoid it?

fun StringValuesBuilder.append(name: String, value: HeaderValueWithParameters) {
    append(name, value.toString())
}

[cy]

So headers.append(Header, ContentType.Something) will be resolved to it