So a bug I've found is that `ApacheClient` will ju...

# http4k

[Quy D X Nguyen]

So a bug I've found is that ApacheClient will just stop querying large files (3Mb) and not close the connection. This leads to thread starvation and eventually an unresponsive app.

val client = ApacheClient(responseBodyMode = BodyMode.Stream, client = HttpClients.custom()
    .setDefaultRequestConfig(RequestConfig.custom()
        .setCookieSpec(CookieSpecs.IGNORE_COOKIES)
        .setConnectTimeout(3000)
        .setSocketTimeout(3000)
        .build())
    .setMaxConnTotal(10)
    .build())

I'm just piping the resulting stream directly into a response, and when I augmented the ReadCalls with some printlns, they just suddenly stop after a while - and then my app freezes.

Edit: It can also happen on smaller files. What's important is that the connection doesn't appear to be closed.

[dave]

hey. could you isolate this into a test case so.we can take a.oser look?

[Quy D X Nguyen]

It appears that sometimes http4k isn't closing the streams that are passed to it in a Repsone. Any ideas what could cause this? Reproducers have not managed to reproduce the bug, were still working on it.

[dave]

in a proxy scenario the only places which even have anything to do with streams (other than passing the reference) are the edges - ie the http client and the ServerConfig layer.

[dave]

which server are you using? if it's Netty then we would recommend switching it out for undertow, as there have been problems with the benchmarks that we have never managed to reproduce or actually used that backend in prod ourselves.

[dave]

there is an open issue for that problem which is possibly related to what you're seeing: https://github.com/http4k/http4k/issues/141

[s4nchez]

@Quy D X Nguyen if you don’t consume the response body from the server it may keep the client thread indefinitely, which would explain your issue. I’d look for any conditions (exception or business logic) that would cause you to not read the response body, even for a subset of your requests.

[Quy D X Nguyen]

It turns out that the default max connections for apacheclient is 2, and long lasting downloads along with a large number of incoming connections causes Netty to deadlock.

[Quy D X Nguyen]

It appears that under load with a blocking handler, the Netty server queues a lot of incoming connections, and then just stops responding.

This happens even when streaming, which makes sense given the synchronous nature of http4k.