Hello, what are the differences (pros/cons) of a ktor backend on Kotlin Native vs GraalVM Native Image ? (Assume Ktor 2.0) Thanka

Ktor native doesn't require any extra configs? Con is that you're locked out of jvm ecosystem.

Thanks @Big Chungus, would be curious to find a comparison of start up time and performance of native vs GraalVM native image. Also do you know if kotlin native is stable/production ready (eg: Coroutine support not only single thread etc)? Thanks again

Presumably Kotlin Native would also have these advantages over GraalVM: • Lower memory usage • Smaller binary size • Quicker startup time • No need to fine tune the GC • High level of platform integration/interop • Easier platform integration/interop • Access to C/platform ecosystem (eg on Linux that means access to POSIX and Linux Kernel Userspace libraries like V4L for example - https://en.wikipedia.org/wiki/Video4Linux ) • No Virtual Machine performance overhead/baggage • Low reliance on meta programming (eg don't need to worry about Annotitus - Annotation overuse, which is prevalent in the JVM ecosystem)

Thanks for the great addition @napperley, I will be creating a kotlin multiplatform library that can be deployed as a library or as a ktor web service. The ktor web service will receive images, perform some network calls, some business logic, some preprocessing/postprocessing (with heavy use of Coroutines!) And return back the images. I still have to check if all my dependencies are available outside of the jvm world but first I'm curious about the Coroutines support of kotlin native.

I got a little bit curious about this question so the decision was made to run some benchmarks and see how K/N, JVM, and GraalVM perform. Environment: Premium AMD DigitalOcean droplet (1 GB RAM / 1 AMD CPU) OS: Ubuntu 21.10 x64 JRE: openjdk-16-jre-headless Benchmark results: JVM Peak throughput: 3400~ RPS Peak memory usage: 62~ MB Executable size: 9.36 MB K/N Peak throughput: 125~ RPS Peak memory usage: 233~ MB Executable size: 4.14 MB GraalVM Doesn't compile. See P.S. I couldn't find a reliable way to test memory consumption, so it might be a little bit biased. Conclusion I see no use case for ktor on K\N besides FaaS.

Wow thanks a lot @nordiauwu! It's sad that your GraalVM didn't compile, maybe try using the agent to generate the config file. But yeah K/N not looking good. https://www.graalvm.org/reference-manual/native-image/Agent/ Thanks for sharing.

Yep, I did use the reflection.json and build script from their sample, but no success =(

Still way too early to involve a Kotlin Native program in benchmarks. The Kotlin Native version of Ktor lacks maturity and hasn't been heavily optimised for memory usage. Kotlin Native's memory model (the new one) isn't mature yet. Also Kotlin Native doesn't have any dedicated benchmark tools yet. Presumably the tools are still being worked on by the Kotlin team. Which Kotlin Native memory model was used in the benchmark? Would expect the Kotlin Native program to have lower memory usage than the JVM one, however Kotlin Native and Ktor needs to mature first before that occurs.

@nordiauwu can you pls share code? BTW, K/N server should work much better, after coroutines 1.6 with new MM support will be integrated to ktor, to support multi-thread dispatcher inside it, and also removed overhead of using old MM inside ktor (atomics just to support mutation, custom lists implementation, that supports freezing and so on)

Apparently I used the old MM, gotta struggle a bit more with GraalVM and redo the benchmark. Thanks for the feedback @napperley

also, current K/N GC with new MM isn't production ready, so we should also wait for new one 🙂

@Oleg Yukhnevich https://pastebin.com/2bnGGLvE

btw Our native tests are running about 2 times faster than

jvm

. I guess the startup time and workload without warm up should be better

@e5l thanks to your help we're not done yet! I was able to solve the GraalVM issue by upgrading to the latest EE release. I'll post new results shortly.

btw, we would be happy to accept your contributions here: https://github.com/ktorio/ktor-benchmarks

K\N with the new MM looks really promising, I'd even already use it in production to save up on memory somewhere 🙂 GraalVM is also super awesome with its instant warm-up, while for JVM ten

ab

runs isn't enough to get fully warmed up. JVM Peak throughput: 2700~ RPS Peak memory usage: 59~ MB Executable size: 10 MB K\N (new MM) Peak throughput: 1400~ RPS Peak memory usage: 16~ MB Executable size: 5 MB GraalVM Peak throughput: 5000~ RPS Peak memory usage: 89~ MB Executable size: 38 MB I also took a look at locust.io as @Alexandre Brown suggested, it's clearly a great tool for simulating real user behavior but a bit overkill for our "Hello World" application, so I ended up running

ab -n 1000 -c 10 -k

10 times for each platform. The source code is the same for all platforms - https://pastebin.com/VvZ9UnzY

@e5l sure, but setting up automated benchmarks for all 3 platforms we discussed will be pretty tough, especially for GraalVM, which requires a good amount of RAM to compile (1 GB wasn't enough when I tested). Maybe I'll get back to it later and see what we can do.

@nordiauwu Thanks a lot for your efforts! I'd expect GraalVM memory footprint to be considerably lower than JVM under GraalVM Native (no JVM). Was your tests on GraalVM done with GraalVM native or Hotspot? I'd argue the "question" is more whether K/N outperforms GraalVM Native as both of these options are no-jvm options.

@Alexandre Brown, I'm not sure what you mean by Hotspot, but I was able to run the executable produced by native-image without java installed so I guess we can call it native.

@nordiauwu if you use GraalVM without native image then it uses Java Hotspot VM (which is a little different than JRE). If you run with native image then it does not use any VM

Thanks for the clarification, I'm pretty new to GraalVM so... :)

Ok no worries , did you use "native-image" to build your graal binary ?

Yes

Did you use the "--static" parameter as well?

Here is what I used: https://github.com/ktorio/ktor-samples/blob/main/graalvm/build.sh The only thing I changed is the path to Main class.

Because If you don't use --no-fallback parameter then it might still use Hotspot VM even if you used native-image command

There's pretty much nothing to show, really.

If you try to add

--static

this will create a static native image which does not need anything to run (as opposed to a mostly static image). This could potentially reduce the size of it. I'd be curious to see the benchmark results with a

--static

image. This should reduce the size of the executable.

Thanks, I'll try the

--static

in the near future and let you know

Thanks again for the benchmarks, really nice of you @nordiauwu

You're welcome 🙂

Throughput with the new Kotlin Native MM looks much better than the old one. As expected Kotlin Native comes out of top in the benchmark with Peak Memory Usage, and Executable Size, but is the worst with Throughput (comes as no surprise). From the benchmark Kotlin Native would perform much better with Serverless than with Microservices. No surprises there.

@Alexandre Brown, it increased the executable size by 200 kb

@e5l what io multiplexing is being used under the hood for ktor native? On suitable linux kernels, is it by any chance io_uring?

Right now it's a simple

select

it is compatible with allmost every platform

@nordiauwu could you, please, provide more details about used options. Did I understand right that you use

binaryOptions["freezing"] = "disabled"

to build and run with new K/N memory MM? Or did you build ktor locally? What ktor version did you use?

@Elena Lepilkina build.gradle.kts gradle.properties

@nordiauwu I was confused by the results because I was comparing executable size. Yes JVM is smaller than graalvm but if you take into account docker image size then it's more realistic as the JVM will need a JVM and be ~120MB compared to the unchanged 30MB of graalVM static image. Maybe something to keep in mind as this is more real world scenario/comparison

Thank you @nordiauwu, but why do disable EscapeAnalysis? it seems unneeded I didn't see any failures with EA when reproduced. Was it made because of default memory limit?

@Elena Lepilkina I was getting some weird error asking to disable it earlier, maybe it’s fixed in new MM or something

It's gradle default memory limitation, the problem is described in the bottom, just add memory with adding to

gradle.properties

org.gradle.jvmargs=-Xmx2048m

I’m not sure if the different multiplexing will give much performance boost at this level of load, but we can accept PR with different a implementation

@e5l can you clarify? if io isn’t the bottleneck in ktor native, what do you think is limiting throughput?

It depends on the workload, right now IO taking about 5%

Thanks for clarifying!

A bit more information for people interested in this benchmark, the next release will have some performance improvements for new MM. Current measurements on early version of 1.6.20 show that this sample should be faster in

~15%

with same ktor version and disablede freezing. Also 1.6.20 will include one more flag that you can try

-Xgc=cms

that enables concurrent mark and sweep strategy that should also speed up this sample a bit (

3-5%

). Please, pay attention that results may differ from platform to platform so numbers of improvements can differ.

Does the ~15% refer to throughput?

Does the ~15% refer to throughput?

Yes

What is the default GC used by the K/N's new MM?

Now stop-the-world mark and sweep . But later stop-the-world mark + concurrent sweep that now is under flag should become default