Hey everyone. :wave: I joined here in the hope to ...

# compiler

[Daniel Kuschny]

Hey everyone. 👋
I joined here in the hope to reach some of the Kotlin compiler experts. I'm following up on my post here.
Long story short: is there a way I can analyze why the Kotlin compiler takes so long to compile the same codebase which compiles in part of the the time on other compilers like C#? It seems quite long that my codebase compiles in 30s for TypeScript (👍), 1:15min for C# (👍) but 4:48min for Kotlin (👎). I just upgraded to Kotlin 1.5 in the hope to get a speed up but it's basically the same speed as with 1.4.x. Example compilation here.

[Iaroslav Postovalov]

1. Consider using Kotlin 1.5.0.
2. There are two compilations of your program: the JVM compilation and the ART one

[Daniel Kuschny]

1. That's what I have done already. The shared example compilation link from this message is one with 1.5.0 integrated.
2. The android compilation is not active, it's commented in the build.gradle.kts (https://github.com/CoderLine/alphaTab/blob/feature/update-dependencies/src.kotlin/alphaTab/build.gradle.kts) Also according to the executed tasks I cannot see a ART compilation. Be sure to check the link from my Slack post, not the outdated ones from the Forum post 😉

[Daniel Kuschny]

Here a screenshot from the compilation log if something's wrong with the link 😁.

[Fleshgrinder]

Kotlin compilation only takes a minute ++ in your log, the time it gives at the end is for the entire Gradle stuff.

[Daniel Kuschny]

@Fleshgrinder Thanks for the hint. I wasn't looking maybe well enough after the upgrade. I was able to improve the build by a few mins but Kotlin compilation still seems to be a big player.
Here two example runs that indicate still a 3 minute pure compilation time where C# needs 1-1.5min.

https://scans.gradle.com/s/ectwxd2v5hjze/timeline
https://github.com/CoderLine/alphaTab/runs/2540060766?check_suite_focus=true

https://scans.gradle.com/s/ddqjnlkdkd7dm/timeline
https://github.com/CoderLine/alphaTab/runs/2540283320?check_suite_focus=true

Hence I would still be interested into finding out where these long times potentially come from.

[Fleshgrinder]

Kotlin compilation is generally known to be slow and the team is working on improving it. I cannot provide you more than that. Maybe someone else...

[Daniel Kuschny]

Thanks anyhow and greetings from Vorarlberg 😉. I hope maybe some compiler expert sees my message and has some hints how I might be able to profile my project compilation and improve the situation.

[edrd]

@Daniel Kuschny you may want to check the post by Andrey Breslav on this subject, if you didn't already: https://blog.jetbrains.com/kotlin/2020/09/the-dark-secrets-of-fast-compilation-for-kotlin/

[Daniel Kuschny]

@edrd Thanks for the hint and I had a look at the post. Unfortunately the post mostly talks about how to improve compilation for local development (point 3 noted in the post) while only mentioning that things like the compiler itself play a role (point 1 and 2). My goal is to have fast compilation on my CI system where it will always be a clean/dry build (and I would prefer to keep it clean). I understand that the compiler is quite smart and I fear that some of my generated code hits the cost of these smart checks. But to find out which ones, I need input form experts how I can find this out. 😖

[edrd]

Are you using Gradle? If so, you can add compilation cache to your CI (it's a common practice) and declare your generated code as inputs to the build tasks. This way Gradle knows which parts of the cache should be invalidated, while keeping the rest. (@melix am I correct?).
I've done this in a project and it worked great. The build times were pretty much always at around 2 min, and we were using Kotlin 1.3, which didn't have the optimizations of newer versions.

[Fleshgrinder]

👍 exactly what @edrd wrote. The problem in CI with Gradle is that it needs to pull the dependencies and compile things from scratch, on top of the JVM cold start. Usually you want to cache $GRADLE_HOME/caches/ and $GRADLE_HOME/wrapper/ in a location close to your CI (network proximity) and use https://docs.gradle.org/current/userguide/build_cache.html (again, close network proximity). Various CI systems have things available to set this up for you with a varying degree of sophistication (CircleCI, GitHub Actions).

[Daniel Kuschny]

Yes I'm using Gradle and I gave it a try today extending my builds with caches. It brought no significant improvement. The Kotlin build still takes ~2times longer than my other targets.
Of my 5m27s overall compilation, 3m:25s are in the compileKotlinJvm and 60s in compileKotlinMetadata. I would prefer to improve on the 4m25s of the Kotlin compilation instead of improving the 1m that is coming from the other parts of the toolchain. https://github.com/CoderLine/alphaTab/runs/2575136396?check_suite_focus=true#step:7:65
https://scans.gradle.com/s/bek73v7f5y6hy

[Fleshgrinder]

The build scan shows that no caches were used. You have to enable build caching. 😉

[Fleshgrinder]

https://scans.gradle.com/s/bek73v7f5y6hy#switches here

[Daniel Kuschny]

Thanks everyone for the hints so far. I was temporarily able to improve the build situation. But it seems I messed up completely again. I did some rework of my codebase for better runtime performance and now the kotlin toolchain goes completely wild. Android Studio completely freezes with 100% CPU load, memory spikes up to >6GB and my github CI system just has a crash on the gradle process (out of memory I guess).

I still think there is something in my codebase that triggers some weird compiler behavior but I am somehow lacking the tools to dig deeper. Did anybody of you guys perform successfully some cpu and memory profiling of the compiler in the past and could give some hints on the tools?

[edrd]

@Daniel Kuschny I suggest you start a new thread so more people can see your question