I've been having a really rough time getting a Net...
# http4k
a
I've been having a really rough time getting a Netty server to handle meaningful load. I think I've found the issue, and it's because the
Http4kChannelHandler
blocks the Netty thread for the entire duration of the response. I think by default, Netty only opens a very small number of threads, based on the CPU count. And so if you're blocking on those threads, you're making it impossible to handle concurrent requests. What I think SHOULD be done is to inject an executor into the handler, and offload the request to it. Once that returns, you get the netty executor and use it to submit the response to the outgoing channel.
d
interesting. Personally I never use Netty (I use Undertow or JettyLoom by default), and the benchmarks seem to be ok on it (IIRC), but there is every possibility that there's a bug 🙂 . Feel free to PR in a fix if you find something! 🙂
a
I wonder if the benchmarks aren't stressing concurrency? For me, the issue is I've got long running requests, and so I get thread starvation. I normally use Jetty or Undertow too, but this server is already using Netty for some special TCP sauce, so I figured I'd use it for the control plane and websockets too.
d
Tech empower does 256 concurrent requests AFAIK - but none of those are long running so it's entirely possible that there is a problem there
a
Yeah, I'm almost certain the issue has to do with the request run time. If the request returns immediately, then it's essentially not blocking the netty thread.
This is all that's needed to fix it. I'm choosing a cached thread pool executor by default, but using a virtual thread pool for my server. I'll let you know how it goes.
👍 1
So, while the executor will be ideal for network-bound applications, it will just add overhead to cpu-bound or no-op apps. I also know you don't really want the default server configs to be configurable. I think most apps are going to be network-bound, so the executor is probably the correct choice; which will make Netty more viable for general use cases.
d
👍
a
I've found that Netty's websocket handler has always been susceptible to a race condition where it will drop messages before the
wsConsumer
has finished being invoked. This is made worse when offloading the
wsConsumer
invocation to an executor. So I'm going to add a buffer that's used for this interim period.
I introduced two race conditions that allowed for out-of-order websocket frame processing: 1. Offloading to an executor practically guarantees parallel frame processing 2. Once the
WsConsumer
returns, draining the buffer would run in parallel to new frames being received I have solutions for all of these, but all of this just goes to show why Netty is not a great out-of-the-box backend for higher level protocols.