In the <antlr-kotlin> repository we've set up benc...

# kotlin-native

[Edoardo Luppi]

In the antlr-kotlin repository we've set up benchmarks to compare performance across Kotlin target platforms.
The results are shown in the attached screenshot.

Native (mingwX64) seems to be much slower than all the others.
Not sure if K/N devs are interested into looking at where the performance bottleneck is, but in case there is a PR to checkout.
https://github.com/Strumenta/antlr-kotlin/pull/169

[Oleg Yukhnevich]

At my side on Mac M2 I see that native behave similar to JS (2 graphs because inserts has a lot different values)
regarding mingw target - AFAIK it works slower then the other native targets, at least in my old tests with rsocket (looks like still true)
Still, it's not good, I would expect it to be similar to WasmJs...

[Edoardo Luppi]

Damn your Native results look much better than mine.
I think it's a combination of platform (Mac vs Win) and laptop - mine uses a pretty old i7-7820HQ.

I was expecting better results in general, that's why I decided to push the code, might be useful to someone!

[Edoardo Luppi]

Did you change the Gradle script / IO code to run on ARM64?

[Oleg Yukhnevich]

Yeah, I've adapted some things to be able to run and enable optimizations

[Oleg Yukhnevich]

here is a patch if needed 🙂
BTW, you can setup GitHub actions to run benchmarks on all runners (windows, linux, macos Arm64 and macos X64)
but you will be able to compare results only when running inside one OS(runner), as they are different and could be different even per CI run
If needed, I could help to setup benchmark runs on CI to track changes
Also, you could try to experiment with different native memory allocators (or other flags, may be there will be difference)

macos_arm64_support.patch

[Edoardo Luppi]

Thanks!
I did think about running benchmarks on CI, but since the org is not mine, I didn't want to use too much free CI time 😆

[Edoardo Luppi]

Btw, looks like JS - WASM results are consistent, more than x2 faster, so if you need that high performance worker, delegating to a WebAssembly module might do the trick instead of trying to fiddle with JS

[Oleg Yukhnevich]

Yeah, jvm - wasm - js relations looks consistent ( JVM faster than Wasm in 2, Wasm faster than Js in 2) 🙂

[Stefan Oltmann]

That are roughly the numbers/relations I would have expected, too, based on what I read so far.

But I must say that I was surprised about the low native performance the first time I did something with it. I had this "native must be like C, so super fast" in mind at first.

[Edoardo Luppi]

Yeah I get what you mean. Being on par with JS would be good enough for me!
I was expecting a bit more given it's compiled using LLVM, so you can get a lot of optimizations for free, although there is still the custom runtime part to consider.

[Stefan Oltmann]

Exactly. I even expected it to be more performant than JVM.
But a Kotlin/Native EXE is not the same as compiled with GCC or even GO.

[Edoardo Luppi]

I even expected it to be more performant than JVM

Start up times are, but that's useful only for short lived programs, like CLI tools

[Stefan Oltmann]

Makes me worship how optimized the JVM already is.

I endured long years all this "Java is slow" talk, but that's not true. 😄