Hey guys :wave: I'm evaluating server side technol...

# server

[João Rodrigues]

Hey guys 👋 I'm evaluating server side technologies for a new backend, we're leaning towards ktor. One hot topic is that we're probably going to have to return large responses, in json format. What is the usual procedure to do so? Does ktor have a standard way to do this? Also, if you had success with this with any other server side frameworks, let me know! Thanks

[Arjan van Wieringen]

Hi, what is a large response? Is it 1MB, 10MB, 10000MB? ;-)

[Big Chungus]

Ktor with json serializer and compression plugin should do the trick

[Big Chungus]

Otherwise consider protobuf

[João Rodrigues]

@Arjan van Wieringen it could reach a 1000MB of size. it's still early to tell, but I'd say it is a realistic number

[João Rodrigues]

@Big Chungus Thank you! Will look into that

[Arjan van Wieringen]

Ah, that is some serious data. I need to check our codebase, we use Ktor there for some services, and I think we send about 200MB if we are unlucky.

[Arjan van Wieringen]

You return from server to client? E.g. a web browser, or it is for service to service communication?

[João Rodrigues]

Service to service, but we can't change the client code unfortunately

[christophsturm]

I think there is no support for streaming json replies in ktor. so if you use jackson or kotlin.serialization the reply will be built in memory and then sent. (which is of course not a problem even if its 1000mb big, as long as you don’t have many concurrent requests)

[christophsturm]

but you can always generate the json yourself, by just writing to the response. as long as you have good test coverage that should not be a problem

[christophsturm]

i mean writing json is much easier as parsing it, and if you use a real parser in your tests thats totally ok

[João Rodrigues]

@Arjan van Wieringen did you have any issues with 200MB response sizes? That is actually a good ballpark number, we could consider other ways of providing larger responses

[Jilles van Gurp]

The key thing to do here is not buffer responses in memory and serialize straight to the output stream of the response. This can be a bit tricky. I recently implemented this for spring boot with webflux and co-routines. The use case was exporting arbitrarily large ndjson from our data store. The business logic produces a flow of items that pages through the data. The flow then maps each item to a json object in String form.

A lot of frameworks instead serialize the entire response in memory by default and then write it. This kind of buffering has a memory and performance penalty as you won't write a single byte until you have the entire response ready. The larger the response, the more memory you need.

If you are interested, here's how you would do this with Spring. You'd want something similar in ktor. I've tested this with multi GB responses. You start receiving data immediately instead of running out of memory while the server tries to build a huge response.

ok().body { serverResponse, _ ->
            val factory = serverResponse.bufferFactory()
            serverResponse.writeWith(
                flow
                    .map {
                        "$it\n"
                    }
                    .asFlux()
                    .map { s -> factory.wrap(s.toByteArray()) })
        }.block() ?: throw IllegalStateException("block never returns null")

[João Rodrigues]

@Jilles van Gurp that's very insightful, will definitely give it try. Thanks!

[Jilles van Gurp]

No problem. I'd be interested in learning how to do this in ktor as well.

[João Rodrigues]

@christophsturm for some reason your messages didn't load, sorry for the late reply. The concurrent requests should not be a problem at this stage, we're probably not going to handle more than 5 requests per second. And yes, I'll thoroughly test this as it is the most pressing topic, with kotlin serialization

[Arjan van Wieringen]

@João Rodrigues We don't have issues with 200mb response sizes. But they are send to a web client, so don't know if that is an issue. In the backend we serialize to XML, so we have the same 'in-memory'- serialization issue. However, for us it is not really a problem. We dont have 1000's of simultaneous requests.

We just use ApplicationCall.respondBytes and that works ok for now.

@Jilles van Gurp is right of course. But our usecase doesn't warrant the more tricky solution. A streaming solution would be better, which is pretty trivial with JSON I'd say.

[João Rodrigues]

@Arjan van Wieringen thanks a ton for your feedback, really appreciated. You kinda eased my concerns. I will test this through and probably report my findings

[Arjan van Wieringen]

No problem. Let me know how it goes.

[corneil]

This is the kind of responses you want to send via a message queue.
If it is a list of objects consider using paging, then you can limit the page sizes and not blow up your server.

[asad.awadia]

it could reach a 1000MB ? No mater what library you use for your backend - this is an odd thing to do - sending 1GB responses as json??

[Big Chungus]

Yeah, this definitely looks like a resource that should be paginated at least

[Jilles van Gurp]

Streaming responses is perfectly valid. Old school as well. I was doing this almost 20 years ago with the servlet API. Buffering responses in memory is technically a bit silly/lazy from a performance point of view. Twenty years ago, not doing that was a rookie mistake because it really sucked from a performance/throughput point of view. These days we get away with some inefficiencies.

Basically, there's no need to ever have the entire response in memory on the server if you have a database that supports paging/scrolling and a server that can process that as it comes in and put it on the wire.

The core use case is stuff like CSV import/export, json exports, etc. A common requirement. Perfect use case for flows. Most decent serialization frameworks support serializing directly to a stream as well. So there's no real technical need for keeping things in memory.

Spring webflux makes this quite hard but not impossible. I never quite figured out how to deal with sending big files and stream processing those without buffering in memory. That's OK because our gateway in any case blocks large uploads. But downloading a few GB of json is not that hard to do in a sane way.

IMHO ktor should be able to do this. And if not, that would be something worth addressing.